Skip to content

celery

celery logo

Installing Celery

Success

pip install celery==5.2.7

Using Django with Celery and RabbitMQ

Celery is a distributed task queue that can process vast amounts of messages. We will use Celery to define asynchronous tasks as Python functions within our Django applications. We will run Celery workers that will listen to the message broker to get new messages to process asynchronous tasks.

Using Celery, not only can you create asynchronous tasks easily and let them be executed by workers as soon as possible, but you can also schedule them to run at a specific time. You can find the Celery documentation at https://docs.celeryq.dev/en/stable/index.html.

Celery communicates via messages and requires a message broker to mediate between clients and workers. There are several options for a message broker for Celery, including key/value stores such as Redis, or an actual message broker such as RabbitMQ.

RabbitMQ is the most widely deployed message broker. It supports multiple messaging protocols, such as the Advanced Message Queuing Protocol (AMQP), and it is the recommended message worker for Celery. RabbitMQ is lightweight, easy to deploy, and can be configured for scalability and high availability.

django-celery

Monitoring Celery with Flower

Besides the RabbitMQ management UI, you can use other tools to monitor the asynchronous tasks that are executed with Celery. Flower is a useful web-based tool for monitoring Celery. Install Flower using the following command:

Success

pip install flower==1.1.0

celery -A myshop flower

Celery Defination

Celery is an open-source task queue system written in Python and available as a Python package

Celery offers this functionality:
  • Offload tasks asynchronously
  • Schedule tasks

Because we focus on reactive microservices, we’ll only address offloading tasks with Celery.

Celery runs on top of message queue brokers like RabbitMQ and Redis. Celery has become a standard for Django microservices because Celery:

  • Abstracts the technical details of RabbitMQ and Redis and lets us concentrate on offloading tasks from Django.
  • Integrates well with Django.

Reference