Gunicorn: Custom Worker Causing Issues

Created on 11 Jul 2017 · 4Comments · Source: benoitc/gunicorn

Hello,

I am trying to set a custom worker to retrieve client cert information from the server and then get it into my Flask app like so:

gunicorn_options.py:

import multiprocessing
import os
from mypackage.client_auth_worker import CustomSyncWorker

def config_dir():
    return os.path.dirname(os.path.realpath(__file__))

bind = "0.0.0.0:8000"
workers = multiprocessing.cpu_count() * 2 + 1
worker_class = CustomSyncWorker
timeout = 30
ca_certs = os.path.join(config_dir(), 'ca.cert.pem')
certfile = os.path.join(config_dir(), 'server.cert.pem')
keyfile = os.path.join(config_dir(), 'server.key.pem')
cert_reqs = 2
do_handshake_on_connect = True

custom_auth_worker.py:

from gunicorn.workers.sync import SyncWorker

class CustomSyncWorker(SyncWorker):
    def handle_request(self, listener, req, client, addr):
        subject = dict(client.getpeercert().get('subject')[0])
        headers = dict(req.headers)
        headers['X-USER'] = subject.get('commonName')
        print(headers)
        req.headers = list(headers.items())
        super(CustomSyncWorker, self).handle_request(listener, req, client, addr)

I keep getting the following exception:

File "/Users/me/.virtualenvs/mypackage/lib/python3.5/site-packages/gunicorn/arbiter.py", line 162, in start
    self.log.info("Using worker: %s", self.cfg.worker_class_str)
  File "/Users/me/.virtualenvs/mypackage/lib/python3.5/site-packages/gunicorn/config.py", line 68, in __getattr__
    raise AttributeError("No configuration setting for: %s" % name)
AttributeError: No configuration setting for: worker_class_str

I tried multiple things to fix this, for example, I tried setting a worker_class_str in the gunicorn_options.py file, that did not resolve the issue. I then started digging in the code of the actual package and I found that when I modified the gunicorn/arbiter.py file to comment out the line: self.log.info("Using worker: %s", self.cfg.worker_class_str) everything works perfectly fine, the CustomSyncWorker is loaded just fine and does its job as it is supposed to setting the X-USER header and everything.

Is this a bug? Or am I just doing something drastically wrong? I figure I should not be taking it upon myself to modify the code locally just to force things to work...

Feedback Requested

Source

Abraxos

Most helpful comment

I think you need to pass a string to worker_class. So changing

worker_class = CustomSyncWorker

worker_class = 'mypackage.client_auth_worker.CustomSyncWorker'

would solve your problem.

berkerpeksag on 11 Jul 2017

👍2

All 4 comments

I think you need to pass a string to worker_class. So changing

worker_class = CustomSyncWorker

worker_class = 'mypackage.client_auth_worker.CustomSyncWorker'

would solve your problem.

berkerpeksag on 11 Jul 2017

👍2

And note that this is already documented at http://docs.gunicorn.org/en/stable/settings.html#worker-class

Optionally, you can provide your own worker by giving Gunicorn a Python path to a subclass of gunicorn.workers.base.Worker. This alternative syntax will load the gevent class: gunicorn.workers.ggevent.GeventWorker.

Of course, we could always improve the documentation if you have a better suggestion.

Thank you for the report!

berkerpeksag on 11 Jul 2017

I believe I had tried that, and it did not work, however, I am not certain and I may have written the string differently (like in a "mypackage.client_auth_worker:CustomSyncWorker" format). I will have to check when I get home later today.

Aside from that I would recommend putting a direct example into the documentation or at least somewhere else (wherever you think it's appropriate) simply because I had seen that documentation and it wasn't clear that that's what I should've done in the case of a python configuration file. I ended up doing a bunch of trial-and-error which resulted in me getting a working version that I posted after commenting out the printout.

Thanks for getting back to me so quickly!

Abraxos on 11 Jul 2017

OK, confirmed, that worked. Thank you for the help!

Abraxos on 13 Jul 2017

👍1

Was this page helpful?

0 / 5 - 0 ratings