Hey Edge Community, i build a simple python app, which only imports numpy and pandas and runs a empty while loop. The app runs fine on my local docker and on my virtual edge device. But if i deploy it on a physical edge like a 427 ( ied-os-1.8.0-6-amd64 ) or 227 (ied-os-1.9.0-27-amd64 ), it crashes. I get the error: OpenBLAS blas_thread_init: pthread_create failed for thread 1 of 4: Operation not permitted OpenBLAS blas_thread_init: RLIMIT_NPROC -1 current, -1 max OpenBLAS blas_thread_init: pthread_create failed for thread 2 of 4: Operation not permitted OpenBLAS blas_thread_init: RLIMIT_NPROC -1 current, -1 max OpenBLAS blas_thread_init: pthread_create failed for thread 3 of 4: Operation not permitted OpenBLAS blas_thread_init: RLIMIT_NPROC -1 current, -1 max Traceback (most recent call last): File \"/usr/local/lib/python3.8/runpy.py\", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File \"/usr/local/lib/python3.8/runpy.py\", line 87, in _run_code exec(code, run_globals) File \"/app/app.py\", line 6, in \u003cmodule\u003e import pandas File \"/usr/local/lib/python3.8/site-packages/pandas/__init__.py\", line 16, in \u003cmodule\u003e raise ImportError ImportError: Unable to import required dependencies: numpy: IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE! Importing the numpy C-extensions failed. This error can happen for many reasons, often due to issues with your setup or how NumPy was installed. We have compiled some common reasons and troubleshooting tips at: https://numpy.org/devdocs/user/troubleshooting-importerror.html Please note and check the following: * The Python version is: Python3.8 from \"/usr/local/bin/python\ * The NumPy version is: \"1.22.4\ and make sure that they are the versions you expect. Please carefully study the documentation linked above for further help Original error was: PyCapsule_Import could not import module \"datetime\ The error is raised from the pandas init.py, when calling __import__(“numpy”). Does anyone has an idea, why the numpy packages can not be found on a physical edge? (I install pandas and numpy with pip when building the docker). I printed __import__(numpy) and got the same error, but printing __import__(“pytz”) and __import__(“dateutil”) gave me the right path: '/usr/local/lib/python3.8/site-packages/dateutil/__init__.py' Let me know, if you need any more information about my setup. Br, Pascal
I fixed the problem – i used python3.8-slim as image for my docker. For whatever reason, it is not working anymore on the edges. Using python3.8-slim-buster works, incase anyone facing the same problem.
The smoking gun is in here:
OpenBLAS blas_thread_init: pthread_create failed for thread 1 of 4: Operation not permitted [emphasis mine]
This is probably caused by the seccomp profile on the physical IEDs. This profile behaves incorrectly when presented with newer clone(2) syscall parameters that are getting used in recent base images. This seccomp profile should report an unsupported syscall/parameter when in fact it currently reports an access denied. This coerces newer libc implementations to believe that there is no reason to attempt to fall back to an older syscall/parameter combination and immediately give up.
What IED OS version(s) are used on the physical devices? Did you update to the most recent IED OS version? There is no way for users to update the IED’s seccomp profile by themselves, except by updating the IED OS in toto.
I was able to reproduce exactly the same behavior. To be honest I am not sure why the slim image does not work and the buster does. Maybe its because the buster uses debian 10 as a base image.
I will forward this to R&D and lets see the feedback. Thank you for your findings!
Thanks for the clarification. I hope we dont run in the same issue with the slim-buster image too.
We tested it on the edge OS 1.8 and 1.9. Our Siemens contact tested it with the newest 1.12 and got the same behavior.