Message Queues and Async ProcessingLesson 5.3
How BullMQ and Redis-backed job queues work
BullMQ architecture, job states, worker concurrency, job priority, scheduled jobs, job events, Redis Streams, failed job handling
How BullMQ and Redis-backed job queues work
BullMQ Basics
BullMQ is a Node.js job queue built on Redis Streams. Jobs move through states: waiting, active, completed, or failed. Workers claim jobs atomically — no two workers process the same job simultaneously.
import { Queue, Worker } from 'bullmq';
// Producer
const emailQueue = new Queue('emails', { connection: { host: 'redis' } });
await emailQueue.add('send-welcome', { userId: 42 }, {
attempts: 3,
backoff: { type: 'exponential', delay: 2000 }
});
// Consumer
const worker = new Worker('emails', async (job) => {
await sendEmail(job.data.userId);
}, { connection: { host: 'redis' }, concurrency: 5 });Concurrency and Priority
Set concurrency per worker to process multiple jobs in parallel. Add priority (lower number = higher priority) so critical jobs jump the queue. Set attempts and backoff for automatic retry with exponential delay on failure.
Scheduled and Delayed Jobs
// Run after 10 minutes
await queue.add('cleanup', data, { delay: 600000 });
// Repeating job via cron
await queue.add('daily-report', data, {
repeat: { cron: '0 8 * * *' }
});BullMQ replaces many cron setups with a more reliable, observable system — job history, failure tracking, and retry logic are built in.
