Gunicorn: Add Windows support

wip. planned for R18.

Some update on that:

• SOCKET support is quite done, a branch will be posted online next week

TODO:

handle service start/stop/restart to use gunicorn as a full windows service. It is replacing somehow the use of signals under unix.

the support will finally happen in R20. i will create the new PR to track the development of this feature next week.

Things to fix are:

• adapt Worker notifications to windows
• port the signaling to windows: reload and reexec
• make sure sockets can be inherited among processes

What is the progress on adding windows support for gunicorn?. Or rather How far are you from R20? I am having the same problem as #587

Slow progress. I want to release R19 with the threaded work now. I will push a version on monday. the windows support will arrive right after.

Is this windows supported version available somewhere ?

Windows support will be in R20, so I guess it will be a while till guincorn will have windows support... I created a vm with Ubuntu, and this works for me.....

Just curious as to any progress in this. As a first pass it doesn't even need to be fast; a big advantage will be just being able to develop with Gunicorn on Windows and push to a Linux server (e.g. Heroku) without changing config.

@robertlagrant it's on my todo. I need to find a windows license first. R20 will be released in october,

How about the VM images at modern.ie ? Those are free and legal
On Sep 22, 2014 3:30 PM, "Benoit Chesneau" [email protected] wrote:

@robertlagrant https://github.com/robertlagrant it's on my todo until
the end of the year. I need to find a windows license first.

—
Reply to this email directly or view it on GitHub
https://github.com/benoitc/gunicorn/issues/524#issuecomment-56372470.

I used modern.ie images to test a web app in IE8 recently. Easy to get going and worked great.

@benoitc If needed, I believe the PSF can provide you with a windows license.

@benoitc is there a WIP branch for this? I'm not a Windows user, but I can take a look at this.

@berkerpeksag no not yet :/ we should really start one asap.

like @cyberdelia said we could ask a license of windows at the PSF that the gunicorn project could use for such development. Thoughts?

It's a good idea. https://www.python.org/psf/grants/ has all information about grant proposals.

Any updates on this?

I am also looking for updates to this.

@berkerpeksag let's have a talk asap on IRC, just ping me sometimes before since i'm away from any instant thing these days :)

@benoitc any update on this one?

@lealhugui as far as I know there has been no progress since. Anyway I came with a plan:

1. we should put gunicorn inside a CI system that handle windows to knows where it fails
2. start the port over the new IPC library (sometimes next month) to start the work on windows

I can expect a release sometimes in march. Possibly sooner but no promises :) Work can of course go faster if someone that actually knows window better than me (hint I didn't use windows since 1995...) can help!

@benoitc after mid january, i can help out (basically testing, or anything that you need and i could help).

@lealhugui thanks! I will provide updates ASAP

Hope things are going well. Are we still good with this release in March ?

@tannaalok unlikely. Hopefully, @benoitc can publish the IPC work and then maybe Windows support will be possible. It will take some time for the IPC work to land, then the rest of the Windows work to be discovered, triaged, worked on. It's very hard to say.

@tilgovi the IPC work will land in a branch sometimes next week finally :)

Now about the windows version, here is the plan I have in head. The first thing to do would be hooking Gunicorn to a windows CI to start to note issues. There are then 2 scenarios, running gunicorn in cygwin or running gunicorn in plain windows. The two can trigger different results and need to be tested. As far as I know the things to make gunicorn portable are the following:

• signals handling: signals work differently under windows and we need to find a way to expose the following features: http://docs.gunicorn.org/en/stable/signals.html
• worker spawning: there maybe some difference on windows. The IPC work will remove the usage of temporary file to check if the workers are alive replacing it by a pipe. This part should be made compatible
• worker supports: i am not sure yet how behave the gevent, eventlet and tornado workers on windows. this is a good call to split them outside gunicorn anyway. So we can upgrade them apart. The gthread worker can work transparently if we go to python 3. which will be the case.

Did I miss anything? Any help is welcome, I am not a windows daily user myself. So anyone who is experienced on windows that wants to take the lead on that part to make the windows platform a first citizen in gunicorn is welcome!

Python 3.5 adds an useful feature to handle signals on Windows: signal.set_wakeup_fd() now accepts a socket handle on Windows.
https://docs.python.org/dev/library/signal.html#signal.set_wakeup_fd

like @cyberdelia said we could ask a license of windows at the PSF

Microsoft gives free MSDN licenses for Python core developers. For example, as a core developer, I have access to Visual Studio 2015.

@haypo thanks. Actually that could also be used to wake up the arbiter on unix too... Very useful.

If a core dev could help then that will be useful :) Any idea on how to ask for one when it's an external project?

You also have the WSL (Windows for Linux) to target for Windows capability

Is there any way a person with limited linux and gunicorn knowledge can help port this to windows, or are the problems too non-trivial?

Why is this issue closed? I can't find any info on working on windows

This issue is still open.

@DominikDitoIvosevic it's still open. I will provide some kind a roadmap sometimes this week.

I stumbled into some compatibility reading today. I was looking at my work on #1310. I handled FD_CLOEXEC differently than systemd C libraries do. That lead me to PEP 446, thinking about fork behavior, and remembering that there is no fork on Windows.

To get things started, here are a few things we need on the roadmap:

• Set up CI
• Replace os.fork, probably with multiprocessing.Process
• Replace WorkerTemp with multiprocessing.Pipe
• Add compatibility versions of FD inheritance helpers
• (find other, likely more minor, things and fix them)

Stuck with Windows at work this would really help me. Highly anticipating this feature!

I recognize this may not be an answer, but I find waitress really good for
Windows.

2016-08-08 22:39 GMT+03:00 hirolau [email protected]:

Stuck with Windows at work this would really help me. Highly anticipating
this feature!

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/benoitc/gunicorn/issues/524#issuecomment-238351654,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AALnfBAXBrmPlU1JPx6RXI0vFQPLmBlyks5qd4XhgaJpZM4AnyCw
.

Yours faithfully,
Gleb

@chester89 this is surely not really appropriate there....

@hirolau i just got an MSDN developer license last month so i can test it easily on windows. Hopefully @berkerpeksag will be able to help for tests. I will draft a plan so we make sure to have it available sometimes next month :)

@benoitc - shameless bump!

I'm usually using windows so gunicorn work on windows will be very appreciated. Thanks !
Do you have any plans when it will be ready ?

any news on this? we are forced to use Windows in production. If work has stalled could you share the draft plan? cheers!

@cmorgan The same situation here. Does anyone has updates on this issue?

@benoitc any news on this?

hello everyone
Any news on this?

I'm sure any updates will be added to the issue. There's no need to ask for updates.

Hi all,

Sorry to not have answered before. I've been quite busy these days. So it seems there are a lot of people waiting for this feature it's about time to make it. I I will draft a plan by sunday about it and make to find a way to make it happen :)

I believe splitting this up into smaller more actionable tasks might be the best way forwards.

As such I've filed #1467 for the "set up CI" part with PR #1468 for adding an AppVeyor config. An example test run can be seen here:
https://ci.appveyor.com/project/edmorley/gunicorn/build/windows-ci.18

In addition to the bigger changes mentioned in previous comments (signals, worker IPC, handling lack of fork etc), the following need to be resolved before the tests will even run at all:

• ImportError: No module named fcntl (fcntl.fcntl() is used by close_on_exec() and set_non_blocking())
• ImportError: No module named grp (grp.getgrnam() is used by validate_group())
• ImportError: No module named pwd (pwd.getpwnam() is used by validate_user())
• ImportError: No module named resource (resource.getrlimit() is used by get_maxfd(), though that method appears unused - I've opened #1469 to remove it)
• AttributeError: 'module' object has no attribute 'AF_UNIX' (is used by the FAMILY class attribute on UnixSocket)
• AttributeError: 'module' object has no attribute 'geteuid' (os.geteuid() is used by the User setting's default value)
• AttributeError: 'module' object has no attribute 'getegid' (os.getegid() is used by the Group setting's default value)

@edmorley thanks, will look at it asap :)

When can I respect this function？ This issue began about four year ago。

Hi,
We are looking to have this functionality as well, do we have any idea when this might start to happen?

Hi there, when is this going to be handled? :)

If anyone wants to help, tackling any of the issues that @edmorley has found would be useful PRs.

Be aware that some things might change more dramatically to make this possible.

There is no timeline for this right now. If you want to help, IRC might be the best place to chat about it, or open smaller issues for individual work items.

Please subscribe to this issue if you want to be notified about progress. Right now there is no timeline.

FWIW, Gunicorn works in Cygwin on Windows. I know that's not hugely helpful but there it is...

@Zorlin I am not able to get it working with Cygwin on Windows. I tried doing the example from the official documentation

$ pip install gunicorn
$ cat myapp.py
  def app(environ, start_response):
    data = b"Hello, World!\n"
    start_response("200 OK", [
        ("Content-Type", "text/plain"),
        ("Content-Length", str(len(data)))
    ])
    return iter([data])
$ gunicorn -w 4 myapp:app

I keep getting this error in Cygwin

<project-name>\flask\lib\site-packages\gunicorn\util.py", line 9, in <module>
    import fcntl
ModuleNotFoundError: No module named 'fcntl'

this also run in the subsystem on windows 10 so maybe it's not needed anymore?

To support windows natively we need the following:

• [ ] go to python3 to simplify the code
• [ ] use CreateProcessW to spawn a process instead of os.fork (maybe reusing the _winapi abstraction used in multiprocessing?
• [ ] sockets i believe there is already some code ported for asyncio and available in python 3 that can be use for gthreads
• [ ] permissions. we should support instead the way windows allows the launch of the services to daemonize. also privileges should be handled differently

(maybe reusing the _winapi abstraction used in multiprocessing?

BTW, multiprocessing uses CreateProcess in popen_spawn_win32.py and it's not part of Python's backwards compatibility process since it's an internal module.

Hello everyone! How is the progress coming along for this issue? Do we have any target dates in sight for the completion? Thanks!

@bartmika no. There will not be a target date. We have lots of work to do as Benoitc outlined. We welcome help. As I said above, please subscribe to this issue for updates. We will update when there is anything to share.

Thank you @tilgovi,

I would like to discuss potentially putting a bounty on closing this issue. May a developer from Gunicorn please contact me at bart [ at ] simalam.com

Thank you!

@bartmika sorry was a busy week. I will contact you tomorrow once I have been in contact with @tilgovi & @berkerpeksag . Good idea anyway.

@benoitc just wanted to check the status on Gunicorn for Windows. I see that there is not a release date yet but wanted to see if there is anything beta that I can test on Windows.

@jmaulsby work have not really been started yet. I'm slowly back on it. Plan for me is to make the next major release happen sometimes in December including some support of windows.

@bartmika i have been side-tracked these days. I will try to setup a plan an make it public later this week.

So is there now a support for Windows?. It's December

Looking forward to it! It's January.

@benoitc Any updates?

Hi,

Just checking in again, is there anything new?

Thanks
Mike

Hi

Can i use Apache Airflow on windows now?

Thanks
Sudhakar

Hi,

I am trying to the following command but I get this error. Can you help on this?

(flaskms) C:\Users\Socrates>gunicorn --bind 0.0.0.0:8000 hello-world:app
Traceback (most recent call last):
  File "c:\programdata\anaconda3\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\programdata\anaconda3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\ProgramData\Anaconda3\Scripts\gunicorn.exe\__main__.py", line 5, in <module>
  File "c:\programdata\anaconda3\lib\site-packages\gunicorn\app\wsgiapp.py", line 10, in <module>
    from gunicorn.app.base import Application
  File "c:\programdata\anaconda3\lib\site-packages\gunicorn\app\base.py", line 12, in <module>
    from gunicorn import util
  File "c:\programdata\anaconda3\lib\site-packages\gunicorn\util.py", line 9, in <module>
    import fcntl
ModuleNotFoundError: No module named 'fcntl'

As a hack, I created fcntl.py file with below content, compiled it as fcntl.cpython-36.pyc file, placed it under __pycache__ folder, restarted the terminal, and ran the command gunicorn --bind 0.0.0.0:8000 hello-world:app, but still no luck.

fcntl.py content:

FD_CLOEXEC = 1
F_DUPFD = 0
F_GETFD = 1
F_SETFD = 2
F_GETFL = 3
F_SETFL = 4
F_GETOWN = 5
F_SETOWN = 6
F_GETLK = 7
F_SETLK = 8
F_SETLKW = 9
F_RGETLK = 10
F_RSETLK = 11
F_CNVT = 12
F_RSETLKW = 13
F_RDLCK = 1
F_WRLCK = 2
F_UNLCK = 3
F_UNLKSYS = 4

def fcntl(fd, op, arg=0):
    return 0

def ioctl(fd, op, arg=0, mutable_flag=True):
    if mutable_flag:
        return 0
    else:
        return ""

def flock(fd, op):
    return

def lockf(fd, operation, length=0, start=0, whence=0):
    return

Got this from https://stackoverflow.com/questions/11087682/does-gunicorn-run-on-windows

Would be helpful is anyone can suggest the best Gunicorn alternative for Windows for the time being?

Update: I'm using waitress now, very good alternative :)

It has been 5 years and any update on this one? Thanks.

@hsluoyz the last thing that happened publicly was on 16 March of this year, I believe.

Hi,
No luck yet with getting the fork to work in windows?

Hi,

I would be interested in working on this ticket.
What is the current state of the port?

What is the current state of the port?

PR #1468 was merged a year ago that added testing on AppVeyor on Windows, but the AppVeyor account needs activating before it will run.

After that, someone needs to iterate through the failures that occur in the Windows test runs.

@edmorley I will take care about appveyor today.

@Sweenpet nothing much yet . The first thing to do is to look at the test results and create from tickets from it.

Also we need to decide about the scope:

• [ ] does it ru non WSL ? What is missing for iy?
• [ ] cygwin support?
• [ ] native support : which kind of supervision we want. I guess we want it running as a service.

I propose to discuss that over IRC sometimes tomorrow if you want to?

I installed appveyor on the repository:

https://ci.appveyor.com/project/benoitc/gunicorn

@benoitc , I have the application (examples/flaskapp.py) and all the tests running on WSL, for Ubuntu 18.04 and Python 3.6.5. Presumably it should work for all the other version specified in the tox file. The fact it runs on WSL would solve most Windows users problems i'd imagine unless they are not running on Windows 10. Is Cygwin still widely used?

Happy to have a discussion tomorrow about native support on Windows, what time suits?

Will create those tickets tomorrow too, seems to be mainly around native Linux modules and sockets.

FWIW, I use cygwin on windows

If Python does not run on Windows properly (e.g. psycopg2, weasyprint and what not), why the fuck is Python still claims that it is cross platform? So far, it does not even run properly on Windows 10.

@jtan4nih that attitude is uncalled for.

Python is cross platform, gunicorn is not.

sorry I have been horribly side tracked. I would say if anyone want to run natively gunicorn on windows I would be happy to accept any patch and and answer to any questions.

see cs01/gdbgui#18, maybe help

It has been 7 years, and any luck to make it happen?

i may have some cycle yo spend on it during march. I will need to find a
way to execute windows somewhere. you can already run it in the terminal
anyway.

also anyone that actually use windows can help on that topic.

Try waitress.
gunicorn's design is to take advantage of Unix and Unix-like kernels to form a web server.

looking at those youtube tutorials on how to deploy django on heroku using gunicorn really makes me wonder how is it possible for them to do that, meanwhile I'm still stuck with gunicorn problem lol

Try waitress.
gunicorn's design is to take advantage of Unix and Unix-like kernels to form a web server.

Waitress does not support proper multiprocessing.

Is there a rough idea how it could be done on Windows to support proper multiprocessing?
Do we know what the major obstacles are so far for this getting implemented?

i may have some cycle yo spend on it during march. I will need to find a way to execute windows somewhere. you can already run it in the terminal anyway.

Could the Win10 dev VM's be a solution for you to run Windows and check gunicorn compatibility?

Perhaps we can use pywin32 for windows instead of fnctl on windows, it seems they have file locking capabilities .