This document describes the current stable version of Celery (5.1). For development docs, go here.
Internals: The worker¶
The worker consists of 4 main components: the consumer, the scheduler, the mediator and the task pool. All these components runs in parallel working with two data structures: the ready queue and the ETA schedule.
Receives messages from the broker using Kombu.
When a message is received it’s converted into a
Tasks with an ETA, or rate-limit are entered into the timer, messages that can be immediately processed are sent to the execution pool.
ETA and rate-limit when used together will result in the rate limit being observed with the task being scheduled after the ETA.
The timer schedules internal functions, like cleanup and internal monitoring, but also it schedules ETA tasks and rate limited tasks. If the scheduled tasks ETA has passed it is moved to the execution pool.