Celery: ์ตœ๋Œ€ ํด๋ผ์ด์–ธํŠธ ์ˆ˜์— ๋„๋‹ฌํ–ˆ์Šต๋‹ˆ๋‹ค.

์— ๋งŒ๋“  2015๋…„ 09์›” 17์ผ  ยท  3์ฝ”๋ฉ˜ํŠธ  ยท  ์ถœ์ฒ˜: celery/celery

์…€๋Ÿฌ๋ฆฌ ๋ฐ redis์™€ ํ•จ๊ป˜ django๋ฅผ ์‚ฌ์šฉํ•˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค.
์ด ์˜ค๋ฅ˜๊ฐ€ ๊ณ„์† ๋ฐœ์ƒํ•ฉ๋‹ˆ๋‹ค . redis.exception:ResponseError ์ตœ๋Œ€ ํด๋ผ์ด์–ธํŠธ ์ˆ˜์— ๋„๋‹ฌํ–ˆ์Šต๋‹ˆ๋‹ค.
์ €๋Š” heroku๋ฅผ ์‚ฌ์šฉํ•˜๊ณ  ์žˆ์œผ๋ฉฐ redis ๋ฐฑ์—”๋“œ์˜ ์ตœ๋Œ€ ์—ฐ๊ฒฐ ์ˆ˜๋Š” 400์ž…๋‹ˆ๋‹ค. ๊ธฐ๋ณธ ์•ฑ์— ๋Œ€ํ•ด 20๊ฐœ์˜ dyno๋ฅผ ์‹คํ–‰ํ•˜๊ณ  ์žˆ๊ณ  celery์— ๋Œ€ํ•ด 5๊ฐœ์˜ dyno๋ฅผ ์‹คํ–‰ํ•˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค. ์ตœ๋Œ€ ์—ฐ๊ฒฐ ์ˆ˜๋Š” ์–ด๋–ป๊ฒŒ ์„ค์ •ํ•ฉ๋‹ˆ๊นŒ? ๋‹ค์Œ๊ณผ ๊ฐ™์ด celery.py์— ๋„ฃ์–ด ๋ณด์•˜์Šต๋‹ˆ๋‹ค.

from __future__ import absolute_import

import os

from celery import Celery

# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'imdowntags.settings')

from django.conf import settings

app = Celery('imdowntags',
             broker=os.environ['REDIS_URL'],
             backend=os.environ['REDIS_URL'],
             include=['imdowntags.tasks'])

# Using a string here means the worker will not have to
# pickle the object when using Windows.
app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)
app.conf.update( CELERY_REDIS_MAX_CONNECTIONS = 20,)

๋˜ํ•œ ๋‚ด settings.py ์•ˆ์— CELERY_REDIS_MAX_CONNECTIONS = 20 ๋ฅผ ๋„ฃ์–ด ๋ณด์•˜์Šต๋‹ˆ๋‹ค.

๊ฐ€์žฅ ์œ ์šฉํ•œ ๋Œ“๊ธ€

์•ˆ๋…• @๋ฌป๋‹ค
Celery 4.0 ์ถœ์‹œ ์ดํ›„ ๋ฌธ์ œ๊ฐ€ ํ•ด๊ฒฐ๋˜์—ˆ์Šต๋‹ˆ๊นŒ?
celery-3.1.23 ๋ฐ redis-2.6์—์„œ ๋™์ผํ•œ ๋ฌธ์ œ๊ฐ€ ์žˆ์Šต๋‹ˆ๋‹ค.
redis ๊ธฐ๋ณธ maxclients๋Š” 10000์ด๊ณ  celery๊ฐ€ 3๊ฐœ์›” ์‹คํ–‰๋œ ํ›„ redis ํด๋ผ์ด์–ธํŠธ๋Š” 10000์œผ๋กœ ์ฆ๊ฐ€ํ•ฉ๋‹ˆ๋‹ค.
๋‚˜๋Š” redis ์„œ๋ฒ„์˜ ์—ฐ๊ฒฐ์ด ๊ฐ€๋“ ์ฐผ๋‹ค๋Š” ๊ฒƒ์„ ์•Œ์•˜์ง€ ๋งŒ celery๋Š” ์•ฝ๊ฐ„์˜ ์—ฐ๊ฒฐ ( netstat ์— ์˜ํ•ด) ๋งŒ ์žˆ์œผ๋ฏ€๋กœ redis ์„œ๋ฒ„๋ฅผ ๋‹ค์‹œ ์‹œ์ž‘ํ•˜๋ฉด ์ž‘๋™ํ•ฉ๋‹ˆ๋‹ค.
ํ•˜์ง€๋งŒ ์„ค์ •์ด๋‚˜ ๋‹ค๋ฅธ ๋ฐฉ๋ฒ•์œผ๋กœ ๋ฌธ์ œ๋ฅผ ์–ด๋–ป๊ฒŒ ํ•ด๊ฒฐํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๊นŒ?

๋ชจ๋“  3 ๋Œ“๊ธ€

์ด ์ž‘์—…์„ ์™„๋ฃŒํ•  ๋ฆฌ์†Œ์Šค๊ฐ€ ์—†์œผ๋ฏ€๋กœ ์ด ์ž‘์—…์„ ์ข…๋ฃŒํ•ฉ๋‹ˆ๋‹ค.

๋งˆ์Šคํ„ฐ์—์„œ ์ˆ˜์ •๋  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. 4.0 ๋ฆด๋ฆฌ์Šค ์ดํ›„์— ๋‹ค์‹œ ๋‚˜์˜ค๋Š”์ง€ ๋ด…์‹œ๋‹ค.

์–ด๋–ค ์‚ฌ๋žŒ๋“ค์€ BROKER_POOL_LIMIT=0 ๊ฐ€ ๋น„์Šทํ•œ ๋ฌธ์ œ๋ฅผ ํ•ด๊ฒฐํ•œ๋‹ค๊ณ  ๋งํ•ฉ๋‹ˆ๋‹ค. http://stackoverflow.com/questions/12013220/celery-creating-a-new-connection-for-each-task

@ask ์™€ ๊ด€๋ จ์ด ์žˆ์Šต๋‹ˆ๊นŒ?

์•ˆ๋…• @๋ฌป๋‹ค
Celery 4.0 ์ถœ์‹œ ์ดํ›„ ๋ฌธ์ œ๊ฐ€ ํ•ด๊ฒฐ๋˜์—ˆ์Šต๋‹ˆ๊นŒ?
celery-3.1.23 ๋ฐ redis-2.6์—์„œ ๋™์ผํ•œ ๋ฌธ์ œ๊ฐ€ ์žˆ์Šต๋‹ˆ๋‹ค.
redis ๊ธฐ๋ณธ maxclients๋Š” 10000์ด๊ณ  celery๊ฐ€ 3๊ฐœ์›” ์‹คํ–‰๋œ ํ›„ redis ํด๋ผ์ด์–ธํŠธ๋Š” 10000์œผ๋กœ ์ฆ๊ฐ€ํ•ฉ๋‹ˆ๋‹ค.
๋‚˜๋Š” redis ์„œ๋ฒ„์˜ ์—ฐ๊ฒฐ์ด ๊ฐ€๋“ ์ฐผ๋‹ค๋Š” ๊ฒƒ์„ ์•Œ์•˜์ง€ ๋งŒ celery๋Š” ์•ฝ๊ฐ„์˜ ์—ฐ๊ฒฐ ( netstat ์— ์˜ํ•ด) ๋งŒ ์žˆ์œผ๋ฏ€๋กœ redis ์„œ๋ฒ„๋ฅผ ๋‹ค์‹œ ์‹œ์ž‘ํ•˜๋ฉด ์ž‘๋™ํ•ฉ๋‹ˆ๋‹ค.
ํ•˜์ง€๋งŒ ์„ค์ •์ด๋‚˜ ๋‹ค๋ฅธ ๋ฐฉ๋ฒ•์œผ๋กœ ๋ฌธ์ œ๋ฅผ ์–ด๋–ป๊ฒŒ ํ•ด๊ฒฐํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๊นŒ?

์ด ํŽ˜์ด์ง€๊ฐ€ ๋„์›€์ด ๋˜์—ˆ๋‚˜์š”?
0 / 5 - 0 ๋“ฑ๊ธ‰