Celery: ModuleNotFoundError : fonctionne via Terminal, pas par Django

Créé le 20 juil. 2018  ·  3Commentaires  ·  Source: celery/celery

J'utilise du céleri pour la première fois depuis quelques semaines déjà, donc je suis assez frais pour la mise en file d'attente des tâches. Le problème auquel je suis actuellement confronté est lié à l'utilisation du céleri via Django et à une erreur ModuleNotFoundError .

J'utilise Django 1.11 et celery 4.2.0 . Mes fichiers sont les suivants :

céleri_tasks.py

from celery import Celery
from celery import task
import os

app = Celery('myapp')                         
#app.config_from_object('myapp.celeryconfig', namespace='CELERY')  # WORKS WHEN CALLED THROUGH VIEW/DJANGO: Tell Celery instance to use celeryconfig module
app.config_from_object('celeryconfig', namespace='CELERY')  # WORKS WHEN CALLED THROUGH TERMINAL

# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'myapp.celeryconfig')

suf = lambda n: "%d%s" % (n, {1: "st", 2: "nd", 3: "rd"}.get(n if n < 20 else n % 10, "th"))
@app.task(name='celery_tasks.fav_doctor')
def fav_doctor():
    """Reads doctor.txt file and prints out fav doctor, then  addsa new
    number to the file"""

    with open('doctor.txt', 'r+') as f:
        for line in f:
            nums = line.rstrip().split()

            print ('The {} doctor is my favorite'.format(suf(int(nums[0]))))

            for num in nums[1:]:
                print ('Wait! The {} doctor is my favorite'.format(suf(int(num))))

            last_num = int(nums[-1])
            new_last_num = last_num + 1

            f.write(str(new_last_num) + ' ')


@app.task(name='celery_tasks.reverse')
def reverse(string):
    return string[::-1]

@app.task(name='celery_tasks.add')
def add(x, y):
    return x+y


if __name__ == '__main__':
    app.start()

celeryconfig.py

from __future__ import absolute_import, unicode_literals
from datetime import timedelta

## List of modules to import when celery starts.
CELERY_IMPORTS = ['celery_tasks',]

## Message Broker (RabbitMQ) settings.
BROKER_URL = 'amqp://'
BROKER_PORT = 5672

## Result store settings.
CELERY_RESULT_BACKEND = 'rpc://' 

## Worker settings
#CELERYD_CONCURRENCY = 1
#CELERYD_TASK_TIME_LIMIT = 20
#CELERYD_LOG_FILE = 'celeryd.log'
#CELERYD_LOG_LEVEL = 'INFO'

## Misc
#CELERY_IGNORE_RESULT = False
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_ACCEPT_CONTENT=['json']
CELERY_TIMEZONE = 'Europe/Berlin'
CELERY_ENABLE_UTC = True

CELERYBEAT_SCHEDULE = {
    'doctor-every-10-seconds': {
        'task': 'celery_tasks.fav_doctor',
        'schedule': timedelta(seconds=3),
    },
}

__init__.py

from .celery_tasks import app as celery_app # Ensures app is always imported when Django starts so that shared_task will use this app.

__all__ = ['celery_app']

--> mon application-- __init__.py-- céleri_tasks.py-- celeryconfig.py--> vues-- admin_scripts.py

Après être allé dans le répertoire où se trouvent les fichiers de céleri et être passé à l'environnement virtuel, puis tapez :

celery -A celery_tasks worker -l info

mais UNIQUEMENT avec cette configuration dans mon fichier celery_tasks.py :

#app.config_from_object('myapp.celeryconfig', namespace='CELERY')  # WORKS WHEN CALLED THROUGH VIEW/DJANGO: Tell Celery instance to use celeryconfig module
app.config_from_object('celeryconfig', namespace='CELERY')  # WORKS WHEN CALLED THROUGH TERMINAL

seulement après avoir reconfiguré les lignes de code ci-dessus, puis-je utiliser le céleri via Django, donc :

app.config_from_object('myapp.celeryconfig', namespace='CELERY')  # WORKS WHEN CALLED THROUGH VIEW/DJANGO: Tell Celery instance to use celeryconfig module
#app.config_from_object('celeryconfig', namespace='CELERY')  # WORKS WHEN CALLED THROUGH TERMINAL

Si je ne fais pas cette étape, le message d'erreur et configure comme ça sans changer à mi-chemin, c'est-à-dire,

app.config_from_object('myapp.celeryconfig', namespace='CELERY') # FONCTIONNE LORSQUE L'APPEL VIA VIEW/DJANGO: Dites à l'instance Celery d'utiliser le module celeryconfig

app.config_from_object('celeryconfig', namespace='CELERY') # FONCTIONNE LORSQUE L'APPEL VIA LE TERMINAL

Puis je reçois cette erreur :

[20/Jul/2018 07:03:06] ERROR [django.request:135] Internal Server Error: /myapp/adminscripts/
Traceback (most recent call last):
  File "/Users/my_mbp/anaconda3/lib/python3.6/site-packages/celery/local.py", line 317, in _get_current_object
    return object.__getattribute__(self, '__thing')
AttributeError: 'PromiseProxy' object has no attribute '__thing'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/my_mbp/anaconda3/lib/python3.6/site-packages/kombu/utils/objects.py", line 42, in __get__
    return obj.__dict__[self.__name__]
KeyError: 'data'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/my_mbp/anaconda3/lib/python3.6/site-packages/celery/loaders/base.py", line 144, in _smart_import
    return imp(path)
  File "/Users/my_mbp/anaconda3/lib/python3.6/site-packages/celery/loaders/base.py", line 98, in import_from_cwd
    package=package,
  File "/Users/my_mbp/anaconda3/lib/python3.6/site-packages/celery/utils/imports.py", line 104, in import_from_cwd
    return imp(module, package=package)
  File "/Users/my_mbp/anaconda3/lib/python3.6/site-packages/celery/loaders/base.py", line 92, in import_module
    return importlib.import_module(module, package=package)
  File "/Users/my_mbp/anaconda3/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 978, in _gcd_import
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load
  File "<frozen importlib._bootstrap>", line 948, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'celeryconfig'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/my_mbp/anaconda3/lib/python3.6/site-packages/django/core/handlers/exception.py", line 41, in inner
    response = get_response(request)
  File "/Users/my_mbp/anaconda3/lib/python3.6/site-packages/django/core/handlers/base.py", line 187, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/Users/my_mbp/anaconda3/lib/python3.6/site-packages/django/core/handlers/base.py", line 185, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/Users/my_mbp/anaconda3/lib/python3.6/site-packages/django/contrib/auth/decorators.py", line 23, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/Users/my_mbp/Software/myapp/myappsite/myapp/views/admin_scripts.py", line 56, in admin_script_dashboard
    async_result = add.delay(24, 54) # delay() invokes the Celery, responsible for running this task asynchronously
  File "/Users/my_mbp/anaconda3/lib/python3.6/site-packages/celery/local.py", line 146, in __getattr__
    return getattr(self._get_current_object(), name)
  File "/Users/my_mbp/anaconda3/lib/python3.6/site-packages/celery/local.py", line 319, in _get_current_object
    return self.__evaluate__()
  File "/Users/my_mbp/anaconda3/lib/python3.6/site-packages/celery/local.py", line 349, in __evaluate__
    thing = Proxy._get_current_object(self)
  File "/Users/my_mbp/anaconda3/lib/python3.6/site-packages/celery/local.py", line 109, in _get_current_object
    return loc(*self.__args, **self.__kwargs)
  File "/Users/my_mbp/anaconda3/lib/python3.6/site-packages/celery/app/base.py", line 454, in _task_from_fun
    task.bind(self)  # connects task to this app
  File "/Users/my_mbp/anaconda3/lib/python3.6/site-packages/celery/app/task.py", line 319, in bind
    setattr(cls, attr_name, conf[config_name])
  File "/Users/my_mbp/anaconda3/lib/python3.6/site-packages/celery/utils/collections.py", line 429, in __getitem__
    return getitem(k)
  File "/Users/my_mbp/anaconda3/lib/python3.6/site-packages/celery/utils/collections.py", line 278, in __getitem__
    return mapping[_key]
  File "/Users/my_mbp/anaconda3/lib/python3.6/collections/__init__.py", line 989, in __getitem__
    if key in self.data:
  File "/Users/my_mbp/anaconda3/lib/python3.6/site-packages/kombu/utils/objects.py", line 44, in __get__
    value = obj.__dict__[self.__name__] = self.__get(obj)
  File "/Users/my_mbp/anaconda3/lib/python3.6/site-packages/celery/app/base.py", line 141, in data
    return self.callback()
  File "/Users/my_mbp/anaconda3/lib/python3.6/site-packages/celery/app/base.py", line 924, in _finalize_pending_conf
    conf = self._conf = self._load_config()
  File "/Users/my_mbp/anaconda3/lib/python3.6/site-packages/celery/app/base.py", line 934, in _load_config
    self.loader.config_from_object(self._config_source)
  File "/Users/my_mbp/anaconda3/lib/python3.6/site-packages/celery/loaders/base.py", line 126, in config_from_object
    obj = self._smart_import(obj, imp=self.import_from_cwd)
  File "/Users/my_mbp/anaconda3/lib/python3.6/site-packages/celery/loaders/base.py", line 147, in _smart_import
    return symbol_by_name(path, imp=imp)
  File "/Users/my_mbp/anaconda3/lib/python3.6/site-packages/kombu/utils/imports.py", line 56, in symbol_by_name
    module = imp(module_name, package=package, **kwargs)
  File "/Users/my_mbp/anaconda3/lib/python3.6/site-packages/celery/loaders/base.py", line 98, in import_from_cwd
    package=package,
  File "/Users/my_mbp/anaconda3/lib/python3.6/site-packages/celery/utils/imports.py", line 104, in import_from_cwd
    return imp(module, package=package)
  File "/Users/my_mbp/anaconda3/lib/python3.6/site-packages/celery/loaders/base.py", line 92, in import_module
    return importlib.import_module(module, package=package)
  File "/Users/my_mbp/anaconda3/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 978, in _gcd_import
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load
  File "<frozen importlib._bootstrap>", line 948, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'celeryconfig'

Je ne vois pas exactement où je me trompe. Comme brève explication de mon objectif initial : j'aimerais imprimer dans une nouvelle fenêtre les informations récupérées à partir de diverses tâches, donc comme une barre de progression mais au lieu d'un pourcentage, la sortie est comme ce que vous verriez dans Terminal.
Merci.

Tous les 3 commentaires

c'est une erreur dans votre code pas dans le céleri.

N'hésitez pas à me dire où se trouve l'erreur dans mon code ?

@pymatffm L' avez-vous résolu ? J'ai le même problème

Cette page vous a été utile?
0 / 5 - 0 notes