Gunicorn: Gunicorn 20.0.0: find_library('c') doesn't work in Alpine Linux

Right. We should probably test if python=< 3.7 to use socket.from_fd in https://github.com/benoitc/gunicorn/blob/438371ee90b9676336a44c7abaeb30ee7fc57a5c/gunicorn/socketfromfd.py#L65

First, thanks for a great piece of software!
Second, I can confirm this issue; it has occurred in every installation of 20.0.0 on Alpine.

Right. We should probably test if python=< 3.7 to use socket.from_fd in

https://github.com/benoitc/gunicorn/blob/438371ee90b9676336a44c7abaeb30ee7fc57a5c/gunicorn/socketfromfd.py#L65

Can confirm that the issue isn't related to a specific Python version, have it happening in 3.6,3.7,3.8 so long as it's in Alpine.

It's a side effect of using an operating system without all the basic libs in a container unfortunately. Maybe you could use an image using a debian? or an alpine image coming with a libc?

This api is there is pre-3.7 to replace a lack of API for it in Python. Detecting if the api is available (for python >= 3.7) would at least solve the issue for these versions. Before that we need to investigate what could be done.

It's a side effect of using an operating system without all the basic libs in a container unfortunately. Maybe you could use an image using a debian? or an alpine image coming with a libc?

This api is there is pre-3.7 to replace a lack of API for it in Python. Detecting if the api is available (for python >= 3.7) would at least solve the issue for these versions. Before that we need to investigate what could be done.

Alpine uses musl libc in place of glibc as part of it being based on busybox, of course using other base images would be a workaround but it will limit gunicorn uses as both alpine & other minimal OS that are growing in popularity (which are often based on it\busybox\etc) all have musl instead of glibc so the affect is likely not limited to alpine (just the most popular one affected).

Sounds to me that something like detecting if musl is installed instead of glibc & if so use whatever solution was in place in 19.9.x is a potential fix that isn't python version specific as we know 19.9.x works well.

found a related issue in docker-library/python#111

maybe this code can be reused in a way: https://github.com/python/cpython/pull/10453/commits/e3f67780aab24401a50af64e688d38c24ee41ad0

There is one possible workaround for that, as twisted users encountered the same:
https://stackoverflow.com/questions/48234723/twisted-server-monkey-patch-file
That piece of code from gunicorn/socketfromfd.py could be rewritten as following:

_libc_name = find_library('c')
if _libc_name is None:
    _libc_name = 'libc.so.6'
libc = ctypes.CDLL(_libc_name, use_errno=True)

Just one of the possible solutions,but probably not the best one.

maybe this code can be reused in a way: python/cpython@e3f6778

I think that's this issue: https://bugs.python.org/issue21622

Let's find a workaround and make a patch release.

@tilgovi the patch I linked above does the trick (and is linked in the bug ticket you added). I will work on a PR that use it.

@aljinovic can you try fda61b5479646f68faef30f3cb034ce16f3c61b6 . It should fix your issue. Let me know :)

I tried it and it doesn't work. I found a bug in your implementation, here's a proposal of the fix:

for suffix in ['so', '*.so.*']:
    for f in glob('{0}.{1}'.format(prefix, suffix)):

hrm odd, can you paste me the error you had?

Your code: find_library("c") returns None.
After my change it returns: libc.musl-x86_64.so.1

Also, I had to manually set LD_LIBRARY_PATH first, so maybe this could help:

paths = ['/lib', '/usr/local/lib', '/usr/lib']
if 'LD_LIBRARY_PATH' in os.environ:
    paths = os.environ['LD_LIBRARY_PATH'].split(':') + paths

Taken from:
https://github.com/alpinelinux/aports/blob/202f4bea916b0cf974b38ced96ab8fca0b192e3f/main/python2/musl-find_library.patch

ok fixing that, not sure about why the glob is nlot working the first way though. I need to test

@aljinovic should be OK now, let me know

Still doesn't work unless I change from:

for suffix in ['so', 'so.*']:

to:

for suffix in ['so', '*.so.*']:

The actual libc name is: libc.musl-x86_64.so.1

sorry I missed the additional star.
https://github.com/benoitc/gunicorn/commit/e150ca4ff87d52b1c2b3d5376fc71789b65a3fb4
should fix it now.

On Thu, Nov 21, 2019 at 11:47 AM Ante Aljinovic notifications@github.com
wrote:

Still doesn't work unless I change from:

for suffix in ['so', 'so.*']:

to:

for suffix in ['so', '.so.']:

The actual libc name is: libc.musl-x86_64.so.1

—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/benoitc/gunicorn/issues/2160?email_source=notifications&email_token=AAADRISHZX5Q265IJQ5OCKLQUZRMJA5CNFSM4JLJUUP2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEZZJPY#issuecomment-557028543,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAADRIXWUCJLU6ZCD2R2Z6DQUZRMJANCNFSM4JLJUUPQ
.

Great, that worked, thank you!

Nice work everyone!

can you all test #2208 as i changed the implementation to simply use the python standard library. Any feedback is welcome :)