psutil fails to compile against MUSL C library

What's MUSL?

On Aug 18, 2016 12:37 PM, "Yegor Yefremov" [email protected] wrote:

Following error occurs, when compiling psutil against MUSL. Any idea?

/home/user/Documents/versioned/buildroot/bsp/host/usr/bin/x86_64-linux-gcc
-fno-strict-aliasing -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE
-D_FILE_OFFSET_BITS=64 -Os -DNDEBUG -g -fwrapv -O3 -Wall
-Wstrict-prototypes -fPIC -DPSUTIL_VERSION=431 -I/home/user/Documents/
versioned/buildroot/bsp/host/usr/x86_64-buildroot-linux-
musl/sysroot/usr/include/python2.7 -c psutil/_psutil_linux.c -o
build/temp.linux-x86_64-2.7/psutil/_psutil_linux.o
In file included from /home/user/Documents/versioned/buildroot/bsp/host/
usr/x86_64-buildroot-linux-musl/sysroot/usr/include/linux/kernel.h:4:0,
from /home/user/Documents/versioned/buildroot/bsp/host/
usr/x86_64-buildroot-linux-musl/sysroot/usr/include/linux/ethtool.h:16,
from psutil/_psutil_linux.c:35:
/home/user/Documents/versioned/buildroot/bsp/host/
usr/x86_64-buildroot-linux-musl/sysroot/usr/include/linux/sysinfo.h:7:8:
error: redefinition of ‘struct sysinfo’
struct sysinfo {
^
In file included from psutil/_psutil_linux.c:21:0:
/home/user/Documents/versioned/buildroot/bsp/host/
usr/x86_64-buildroot-linux-musl/sysroot/usr/include/sys/sysinfo.h:10:8:
note: originally defined here
struct sysinfo {
^
psutil/_psutil_linux.c: In function ‘psutil_linux_sysinfo’:
psutil/_psutil_linux.c:244:17: warning: passing argument 1 of ‘sysinfo’
from incompatible pointer type
if (sysinfo(&info) != 0)
^
In file included from psutil/_psutil_linux.c:21:0:
/home/user/Documents/versioned/buildroot/bsp/host/
usr/x86_64-buildroot-linux-musl/sysroot/usr/include/sys/sysinfo.h:26:5:
note: expected ‘struct sysinfo *’ but argument is of type ‘struct sysinfo *’
int sysinfo (struct sysinfo *);
^
psutil/_psutil_linux.c: In function ‘init_psutil_linux’:
psutil/_psutil_linux.c:644:15: warning: unused variable ‘v’
[-Wunused-variable]
PyObject *v;
^
error: command '/home/user/Documents/versioned/buildroot/bsp/host/usr/bin/x86_64-linux-gcc'
failed with exit status 1

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/giampaolo/psutil/issues/872, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAplLFXcm6IbMvfDbxQf8-erdd6d3GToks5qhDXngaJpZM4JnW6d
.

@giampaolo MUSL is a new C library, that tries to strictly follow the standards (POSIX etc.). These rules often lead to compilation issues, when non-stadard headers will be included etc. MUSL is for example quasi standard C library for OpenWrt/LEDE projects.

In the case of psutil, there is glibc leftover issue with struct sysinfo. It is defined in linux/sysinfo.h and in sys/sysinfo.h. OpenWrt project provides following patch for the kernel header, but this is not applicable for psutil of course. If I remove #include from psutil/_psutil_linux.c I can compile the package, but I also get following warning:

psutil/_psutil_linux.c: In function ‘psutil_linux_sysinfo’:
psutil/_psutil_linux.c:244:5: warning: implicit declaration of function ‘sysinfo’ [-Wimplicit-function-declaration]
if (sysinfo(&info) != 0)

So I don't know, if it is a proper solution.

Here's my output on Alpine Linux, using MUSL

# pip install psutil
Collecting psutil
  Using cached psutil-4.4.1.tar.gz
Installing collected packages: psutil
  Running setup.py install for psutil ... error
    Complete output from command /usr/bin/python2 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-s1iYoK/psutil/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-d3266X-record/install-record.txt --single-version-externally-managed --compile:
    /tmp/tmpp7WhDB.c:1:27: fatal error: linux/ethtool.h: No such file or directory
     #include <linux/ethtool.h>
                               ^
    compilation terminated.
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-2.7
    creating build/lib.linux-x86_64-2.7/psutil
    copying psutil/_psposix.py -> build/lib.linux-x86_64-2.7/psutil
    copying psutil/_psosx.py -> build/lib.linux-x86_64-2.7/psutil
    copying psutil/__init__.py -> build/lib.linux-x86_64-2.7/psutil
    copying psutil/_pssunos.py -> build/lib.linux-x86_64-2.7/psutil
    copying psutil/_pswindows.py -> build/lib.linux-x86_64-2.7/psutil
    copying psutil/_psbsd.py -> build/lib.linux-x86_64-2.7/psutil
    copying psutil/_pslinux.py -> build/lib.linux-x86_64-2.7/psutil
    copying psutil/_compat.py -> build/lib.linux-x86_64-2.7/psutil
    copying psutil/_common.py -> build/lib.linux-x86_64-2.7/psutil
    creating build/lib.linux-x86_64-2.7/psutil/tests
    copying psutil/tests/test_memory_leaks.py -> build/lib.linux-x86_64-2.7/psutil/tests
    copying psutil/tests/test_bsd.py -> build/lib.linux-x86_64-2.7/psutil/tests
    copying psutil/tests/test_windows.py -> build/lib.linux-x86_64-2.7/psutil/tests
    copying psutil/tests/__init__.py -> build/lib.linux-x86_64-2.7/psutil/tests
    copying psutil/tests/test_misc.py -> build/lib.linux-x86_64-2.7/psutil/tests
    copying psutil/tests/runner.py -> build/lib.linux-x86_64-2.7/psutil/tests
    copying psutil/tests/test_linux.py -> build/lib.linux-x86_64-2.7/psutil/tests
    copying psutil/tests/test_system.py -> build/lib.linux-x86_64-2.7/psutil/tests
    copying psutil/tests/test_osx.py -> build/lib.linux-x86_64-2.7/psutil/tests
    copying psutil/tests/test_posix.py -> build/lib.linux-x86_64-2.7/psutil/tests
    copying psutil/tests/test_sunos.py -> build/lib.linux-x86_64-2.7/psutil/tests
    copying psutil/tests/test_process.py -> build/lib.linux-x86_64-2.7/psutil/tests
    running build_ext
    building 'psutil._psutil_linux' extension
    creating build/temp.linux-x86_64-2.7
    creating build/temp.linux-x86_64-2.7/psutil
    gcc -fno-strict-aliasing -Os -fomit-frame-pointer -g -DNDEBUG -Os -fomit-frame-pointer -g -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=441 -DPSUTIL_LINUX=1 -DPSUTIL_ETHTOOL_MISSING_TYPES=1 -I/usr/include/python2.7 -c psutil/_psutil_linux.c -o build/temp.linux-x86_64-2.7/psutil/_psutil_linux.o
    psutil/_psutil_linux.c:19:27: fatal error: linux/version.h: No such file or directory
     #include <linux/version.h>
                               ^
    compilation terminated.
    error: command 'gcc' failed with exit status 1

    ----------------------------------------
Command "/usr/bin/python2 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-s1iYoK/psutil/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-d3266X-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-s1iYoK/psutil/

bump getting the same issue with Alpine linux:3.4.

Please provide instructions on how to build/compile psutil by using MUSL on Linux (I'm on Ubuntu).

I have a solution to the problem for Alpine linux. I can't be more specific with the MUSL portion outside of that.
apk add linux-headers
pip install psutil

i'd already entered apk add --upgrade py-pip and psuti, so they may be needed as well.

The easiest way to reproduce the issue would be to use Buildroot:

1. git clone git://git.buildroot.net/buildroot
2. cd buildroot
3. git checkout d8a19c5e4969a5d6485d3704cc72310989109150
4. vi local.mk (it should have following content: PYTHON_PSUTIL_OVERRIDE_SRCDIR=/home/user/Documents/versioned/psutil, just provide the path to your psutil clone)
5. create .config with the content shown below
6. invoke make

BR2_x86_i686=y
BR2_TOOLCHAIN_BUILDROOT_MUSL=y
BR2_PACKAGE_PYTHON3=y
BR2_PACKAGE_PYTHON_PSUTIL=y

How about this?
http://askubuntu.com/a/391037
It appears you may need linux-libc-dev and / or linux-headers packages installed.
Those instructions are for ubuntu, not sure what's the corresponding package name on alpine linux.

That's what I mention above. Installing linux-headers in alpine fixed it for me.

Oh sorry, I missed that. Closing this out then.

@giampaolo have you tried to build psutil in Buildroot following the steps I've provided? The issue with musl is still there.

@DamionWaltermeyer and @ConorIA had unrelated issue, because kernel headers were missing.