Celery: Puluhan ribu tugas backend_cleanup dijalankan setiap malam sejak pukul 22.42. Semua fungsi async berhenti bekerja

Saya baru saja menemukan bahwa masalahnya mungkin tentang tugas bawaan, celery.backend_cleanup. Saya baru saja menjalankan aplikasi di lingkungan pengembangan lokal. Saat itu 10:42, tugas backend_cleanup yang luar biasa dimulai dan dijalankan dengan sukses. Namun, setelah beberapa detik, terjadi kesalahan. Inilah kesalahannya.

[2017-10-23 22:51:27,677: CRITICAL/MainProcess] Unrecoverable error: Exception('Request Empty body  HTTP 599  Server aborted the SSL handshake (None)',)
Traceback (most recent call last):
  File "/Users/user/code/accusize-021/lib/python3.4/site-packages/celery/worker/worker.py", line 203, in start
    self.blueprint.start(self)
  File "/Users/user/code/accusize-021/lib/python3.4/site-packages/celery/bootsteps.py", line 119, in start
    step.start(parent)
  File "/Users/user/code/accusize-021/lib/python3.4/site-packages/celery/bootsteps.py", line 370, in start
    return self.obj.start()
  File "/Users/user/code/accusize-021/lib/python3.4/site-packages/celery/worker/consumer/consumer.py", line 320, in start
    blueprint.start(self)
  File "/Users/user/code/accusize-021/lib/python3.4/site-packages/celery/bootsteps.py", line 119, in start
    step.start(parent)
  File "/Users/user/code/accusize-021/lib/python3.4/site-packages/celery/worker/consumer/consumer.py", line 596, in start
    c.loop(*c.loop_args())
  File "/Users/user/code/accusize-021/lib/python3.4/site-packages/celery/worker/loops.py", line 88, in asynloop
    next(loop)
  File "/Users/user/code/accusize-021/lib/python3.4/site-packages/kombu/async/hub.py", line 354, in create_loop
    cb(*cbargs)
  File "/Users/user/code/accusize-021/lib/python3.4/site-packages/kombu/async/http/curl.py", line 111, in on_readable
    return self._on_event(fd, _pycurl.CSELECT_IN)
  File "/Users/user/code/accusize-021/lib/python3.4/site-packages/kombu/async/http/curl.py", line 124, in _on_event
    self._process_pending_requests()
  File "/Users/user/code/accusize-021/lib/python3.4/site-packages/kombu/async/http/curl.py", line 132, in _process_pending_requests
    self._process(curl, errno, reason)
  File "/Users/user/code/accusize-021/lib/python3.4/site-packages/kombu/async/http/curl.py", line 178, in _process
    buffer=buffer, effective_url=effective_url, error=error,
  File "/Users/user/code/accusize-021/lib/python3.4/site-packages/vine/promises.py", line 150, in __call__
    svpending(*ca, **ck)
  File "/Users/user/code/accusize-021/lib/python3.4/site-packages/vine/promises.py", line 143, in __call__
    return self.throw()
  File "/Users/user/code/accusize-021/lib/python3.4/site-packages/vine/promises.py", line 140, in __call__
    retval = fun(*final_args, **final_kwargs)
  File "/Users/user/code/accusize-021/lib/python3.4/site-packages/vine/funtools.py", line 100, in _transback
    return callback(ret)
  File "/Users/user/code/accusize-021/lib/python3.4/site-packages/vine/promises.py", line 143, in __call__
    return self.throw()
  File "/Users/user/code/accusize-021/lib/python3.4/site-packages/vine/promises.py", line 140, in __call__
    retval = fun(*final_args, **final_kwargs)
  File "/Users/user/code/accusize-021/lib/python3.4/site-packages/vine/funtools.py", line 98, in _transback
    callback.throw()
  File "/Users/user/code/accusize-021/lib/python3.4/site-packages/vine/funtools.py", line 96, in _transback
    ret = filter_(*args + (ret,), **kwargs)
  File "/Users/user/code/accusize-021/lib/python3.4/site-packages/kombu/async/aws/connection.py", line 253, in _on_status_ready
    raise self._for_status(response, response.read())
Exception: Request Empty body  HTTP 599  Server aborted the SSL handshake (None)

Sementara itu, antrian menghilang dari AWS SQS. Kemudian beberapa tugas backend_cleanup terus dijalankan. Dan akhirnya, sebuah peringatan muncul.

[23-10-2017 22:51:28,967: PERINGATAN/Proses Utama] Memulihkan 10 pesan yang tidak diakui

Dan akhirnya, koneksi HTTPS ke antrian SQS berhenti. Aplikasi tidak dapat terhubung kembali ke SQS kecuali saya menerapkan ulang aplikasi ke AWS Elastic Beanstalk.

Adakah yang bisa membantu saya dengan masalah ini? Setiap tanggapan akan sangat dihargai.

Saya pikir saya memecahkan masalah. Masalah tidak pernah terjadi setelah saya mengubah CELERY_RESULT_BACKEND menjadi "redis" dari "Django-db", dan mengubah CELERY_TIMEZONE menjadi "UTC". Meskipun Hasil Tugas di Django Admin berhenti menambahkan hasil tugas apa pun, masalah saya sebagian besar telah diselesaikan.

Anda cukup menimpa jadwal dan menghindari penggunaan crontab:
'celery.backend_cleanup': { 'task': 'celery.backend_cleanup', 'schedule': 86400, # every 24 hours instead of crontab('0', '4', '*') which is used in celery beat 'options': {'expires': 12 * 3600}} ,