Bootcamp: Help with setting up production.

Hi,
Im not using caddy. Im using gunicorn for all my django projects.

Did you installed all requirements?
channels>=2.1.1 # https://github.com/django/channels
channels-redis>=2.2.1 # https://github.com/django/channels_redis

You can check via console

$ python -c "import channels"
$ echo $?

0 means, exist
1 means, module does not exist in system

channels is installed. I encounter another issue now. Here are my docker file and .env. I am running into issue with the worker service. Not sure what it does. Really appreciate your help, your project is very interesting and can help many people in my university if this come into production.

env.txt

docker-compose.txt

Here is the error log.

worker_1    | /usr/local/lib/python3.6/site-packages/daphne/server.py:13: UserWarning: Something has already installed a non-asyncio Twisted reactor. Attempting to uninstall it; you can fix this warning by importing daphne.server early in your codebase or finding the package that imports Twisted and importing it later on.
worker_1    |   UserWarning,
django_1    | DEBUG 2019-10-17 21:28:37,751 base 1 140450066271560 Configuring Raven for host: https://sentry.io
worker_1    | DEBUG 2019-10-17 21:28:35,663 base 1 140638836505928 Configuring Raven for host: https://sentry.io
worker_1    | Running worker for channels ['channels', 'notifications']
worker_1    | DEBUG 2019-10-17 21:28:36,297 base 1 140638836505928 Sending message of length 5534 to https://sentry.io/api/1782977/store/
worker_1    | Traceback (most recent call last):
worker_1    |   File "manage.py", line 30, in <module>
worker_1    |     execute_from_command_line(sys.argv)
worker_1    |   File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
worker_1    |     utility.execute()
worker_1    |   File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 375, in execute
worker_1    |     self.fetch_command(subcommand).run_from_argv(self.argv)
worker_1    |   File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 323, in run_from_argv
worker_1    |     self.execute(*args, **cmd_options)
worker_1    |   File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 364, in execute
worker_1    |     output = self.handle(*args, **options)
worker_1    |   File "/usr/local/lib/python3.6/site-packages/channels/management/commands/runworker.py", line 46, in handle
worker_1    |     worker.run()
worker_1    |   File "/usr/local/lib/python3.6/site-packages/asgiref/server.py", line 60, in run
worker_1    |     event_loop.run_until_complete(self.handle())
worker_1    |   File "/usr/local/lib/python3.6/asyncio/base_events.py", line 484, in run_until_complete
worker_1    |     return future.result()
worker_1    |   File "/usr/local/lib/python3.6/site-packages/channels/worker.py", line 30, in handle
worker_1    |     [listener.result() for listener in listeners]
worker_1    |   File "/usr/local/lib/python3.6/site-packages/channels/worker.py", line 30, in <listcomp>
worker_1    |     [listener.result() for listener in listeners]
worker_1    |   File "/usr/local/lib/python3.6/site-packages/channels/worker.py", line 37, in listener
worker_1    |     message = await self.channel_layer.receive(channel)
worker_1    |   File "/usr/local/lib/python3.6/site-packages/channels_redis/core.py", line 461, in receive
worker_1    |     return (await self.receive_single(channel))[1]

change in your env
REDIS_URL=redis://127.0.0.1:6379

Can you show me how to configure worker service? And why do we need it? New to this project so not sure where to look for that information

uhh, its a lot. This is not done in some minutes...

First of all, please check if you installed all requirements on your server environment

pip install -U -r requirements/production.txt

Second, your project is working on local?

Configure a project is not possible without all files. And Im not the best to configure it. I can check my configuration and share, but this is NOT tested and approved for bootcamp. May you need additional settings.

I am spinning up the service using docker compose. The local deployment work fine but production.yml is a totally different thing. I involve redis, daphne, postgres. If you have a docker-compose file handy somewhere, please share. Thanks

Get Outlook for iOShttps://aka.ms/o0ukef

From: samazaphikel notifications@github.com
Sent: Thursday, October 17, 2019 5:03:30 PM
To: vitorfs/bootcamp bootcamp@noreply.github.com
Cc: Tri Minh Pham phamminhtris@outlook.com; Author author@noreply.github.com
Subject: Re: [vitorfs/bootcamp] Help with setting up production. (#193)

uhh, its a lot. This is not done in some minutes...

First of all, please check if you installed all requirements on your server environment

pip install -U -r requirements/production.txt

Second, your project is working on local?

Configure a project is not possible without all files. And Im not the best to configure it. I can check my configuration and share, but this is NOT tested and approved for bootcamp. May you need additional settings.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/vitorfs/bootcamp/issues/193?email_source=notifications&email_token=ADEK4AB5JEUP5LQ4APTJ6ITQPDOLFA5CNFSM4JB4YOCKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBRVI2A#issuecomment-543380584, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADEK4ADBCG24WLWIT3IM6OLQPDOLFANCNFSM4JB4YOCA.

In my case, IM using python 3.7.1 on Ubuntu server machine

Optional set-up Python version
sudo update alternatives --install /usr/bin/python/ python /usr/bin/python3.7.1

With this command, python 3.7.1 is my first priority

First, install enviroment on the machine
sudo -H pip3 install virtualenv

cd /home/pages to change to the correct folder

mkdir bootcamp to create the projectfolder

cd bootcamp/

create env
virtualenv bootcampenv
Activate your enviroment
source bootcampenv/bin/activate

installl gunicorn
pip3 install gunicorn

go back to pages directory
cd ..
and create a folder for the bootcamp project

django-admin startproject bootcampprojectdir

cd bootcampprojectdir

And install the requirements
pip install -U -r requirements/production.txt

If you use different dabases or something, start migration and collectsstatic

Update firewall to allow port 8000
sudo ufw allow 8000

create gunicorn wsgi
gunicorn --bind 0.0.0.0:8000 settings.wsgi

create a system socket
sudo nano /etc/systemd/system/gunicorn.socket

[Unit]
Description=gunicorn.sock

[Socket]
ListenStream=/run/gunicorn.sock

[Install]
WantedBy=sockets.target

create a service
sudo nano /etc/systemd/system/gunicorn.service

[Unit]
Description=gunicorn daemon
Requires=gunicorn.socket
After=network.target

[Service]
User=pages
Group=www-data
WorkingDirectory=/home/pages/bootcampprojectdir/bootcampenv/bin/gunicorn
--access-logfile -
--workers 3
--bind unix:run/gunicorn.sock
settings.wsgi.application

[Install]
WantedBy=multi-user.target

enable everything
sudo systemctl start gunicorn.socket
sudo systemctl enable gunicorn.socket

check it
file /run/gunicorn.sock
When you have no error, then is fine

Check status
sudo systemctl status gunicorn.socket

check service
curl --unix-socket /run/gunicorn.sock localhost

When you dont see htnl code, something with the gunicorn.service

make sure the service is running
sudo systemctl status gunicorn

When is all fine, next step NGinx pass to gunicorn
install nginx
sudo apt install nginx

create a site in nginx
create a service
sudo nano /etc/nginx/sites-available/bootcampproject

server {
listen 80;
server_name 99.99.99.99

  location = /favicon.ico { access_log off; log_not_found off; }
  location /static/ {
        root /home/pages/bootcampprojectdir;
  }

location / {
include proxy_params;
proxy_pass http://unix:/run/gunicorn.sock;
}
}

save and copy this file to sites-enabled
sudo ln -s /etc/nginx/sites-available/bootcampproject /etc/nginx/sites/enabled

and check nginx setup

sudo nginx -t

In case of no errors, restart nginx
sudo systemctl restart nginx

add nginx firewall ports, but first delete the port 8000 from testing

sudo ufw delete allow 8000

add nginx
sudo ufw allow 'Nginx Full'

Hi @phamminhtris I'm glad you are using the project, and than you find it useful. There are two main ways to deploy the project to a production environment. The first one would be using Docker, but it has its caveats, and obviously it requires a fair amount of familiarity with Docker and docker-compose. I'm far from an expert and that is why I only use it for development, but perhaps you can find something useful here.

The other way is to implement the project as it is, which is "easier" (or at least you can find more resources and guides to deploy it), just remember to load your .env file with the required variables, you can use the example version included to understand how to declare them.

Is it enough to just use gunicorn as the only server? I saw some of the feature required an ASGI application server like daphne to serve web socket traffic. Please correct me if I am wrong. Thank you for you kind help. Really appreciate that.

Nginx is where requests from the internet arrive first. It can handle them very quickly, and is usually configured to only let those requests through, which really need to arrive at your web application.

Gunicorn translates requests which it gets from Nginx into a format which your web application can handle, and makes sure that your code is executed when needed.

Each can do something, which the other can’t.
Nginx is a web server and reverse proxy. It’s highly optimized for all the things a web server needs to do.

Take care of domain name routing (decides where requests should go, or if an error response is in order)
Serve static files
Handle lots of requests coming in at once
Handle slow clients
Forwards requests which need to be dynamic to Gunicorn
Terminate SSL (https happens here)
Save computing resources (CPU and memory) compared to your Python code
And a lot more, if you configure it to do so (load balancing, caching, …)

Gunicorn
Once Nginx decides, that a particular request should be passed on to Gunicorn (due to the rules you configured it with).

Gunicorn is highly optimized and has a lot of convenient features.

Running a pool of worker processes/threads (executing your code!)
Translates requests coming in from Nginx to be WSGI compatible
Translate the WSGI responses of your app into proper http responses
Actually calls your Python code when a request comes in
Gunicorn can talk to many different web servers
Gunicorn is just one of many valid WSGI servers. Your app doesn’t care which one you use, and Nginx doesn’t care either.

I understand the above. However, this project is quite special in term of protocol. It serve notification via websocket and to my understanding we will need more than a regular gunicorn and nginx to drive the app? When I run the local docker compose, I notice that /notifications does not seems to work. I know that I am asking a lot but really appreciate your time and effort to help me @samazaphikel. Will try the local configuration (no docker-compose, docker) just bare ubuntu and see if that works. Thanks y'all

Very helpful guide. Closing.