Designing a notification system that scales to millions
notification system architecture, fan-out on write vs read, push vs pull delivery, user preference service, notification templates, delivery receipts, rate limiting notifications
Designing a notification system that scales to millions
Core Architecture
Notification systems decouple event production from delivery. An event fires such as order placed or password reset. The notification service looks up user preferences and enqueues delivery jobs for each active channel. Workers process each channel independently and can be scaled separately.
Fan-Out Strategies
Fan-out on write: when an event occurs, immediately create a notification record for every recipient. Fast reads but writes multiply with follower count — a celebrity posting triggers 10M database writes.
Fan-out on read: store the event once and compute recipients at read time. Efficient writes but expensive reads for users with many notification sources.
Hybrid: fan-out on write for normal users, fan-out on read for celebrity or high-follower accounts.
Rate Limiting Notifications
# No more than 5 emails per user per hour
const key = `notif:email:${userId}:${hour}`;
const count = await redis.incr(key);
await redis.expire(key, 3600);
if (count > 5) return; // drop or queue for next hourCheck user preferences (opted-out channels, quiet hours, digest mode) before enqueuing — not inside the worker. This prevents pointless job execution for users who will never receive the notification.
