The Problem with Synchronous Processing

When a client sends a request triggering a slow operation — resize an image, send emails to 10,000 users, generate a PDF — a synchronous API holds the connection open until completion. HTTP connections time out, users think the app is broken, and one slow operation blocks server resources for everyone.

The Async Pattern

Accept the request, enqueue a job, return 202 Accepted immediately with a job ID. A separate worker process picks up the job and executes it independently of the HTTP request lifecycle:

// Producer: API handler
const job = await queue.add('generate-report', { userId, filters });
res.status(202).json({ jobId: job.id, statusUrl: `/jobs/${job.id}` });

// Consumer: Worker process
queue.process('generate-report', async (job) => {
  const report = await generateReport(job.data);
  await storeReport(report);
});

What Queues Provide

Durability: jobs survive server restarts if persisted. Back-pressure: if workers fall behind, the queue absorbs the load rather than crashing the service. Retry logic: failed jobs are retried with configurable backoff. Decoupling: producers and consumers are independent — scale workers separately without changing any code.