Design an In-Memory Message Queue
Build a thread-safe, in-memory message queue that supports multiple topics, publishers, subscribers, and concurrent message delivery using the publish-subscribe model.
Design and implement an in-memory message queue that follows the Publisher-Subscriber (Pub/Sub) model.
The system should allow publishers to send messages to topics, while multiple subscribers can listen to one or more topics. Whenever a message is published, every subscriber registered for that topic should receive it.
The system must also support concurrent publishing and consumption, allowing multiple producers and consumers to operate independently and in parallel.
Core Idea
The system can be thought of as:
┌──────────────┐
│ Producer │
└──────┬───────┘
│ publish
▼
┌─────────────────┐
│ Topic │
└────────┬────────┘
│
┌──────────┼──────────┐
▼ ▼ ▼
Subscriber A Subscriber B Subscriber C
A subscriber registered with a topic should receive every message published to that topic.
This one tests whether you can model pub/sub fan-out cleanly — topic isolation, subscriber registry, and ordered broadcast — while keeping delivery thread-safe and non-blocking when multiple producers and consumers run in parallel.
Objective
The primary objective is to design and implement a thread-safe, in-memory message queue that supports multiple independent topics, multiple publishers per topic, subscribers listening to many topics, and concurrent broadcast delivery — with extensibility for unsubscribe, ordering, and failure isolation.
Functional Requirements
Requirements are split into three tiers so you know what to prioritize under time pressure: build first, build if time allows, and guard concurrency correctly.
Part 1 — Basic Requirements
Core functionality you must build and get working first.
1. Topics
2. Publishers
3. Subscribers
4. Message Delivery
For example:
Topic: orders → Subscribers: C1, C2, C3 → Message: "Order #101 created"
→ C1 received Order #101 created
C2 received Order #101 created
C3 received Order #101 created
5. Multiple Topic Subscriptions
A subscriber should be allowed to listen to multiple topics and receive messages from all of them.
C1 → orders, payments, notifications — receives from all three.
6. In-Memory Storage
The entire queue operates in memory. No database, files, or external storage — focus on topic management, subscription handling, and concurrent delivery.
Part 2 — Bonus Features
Extensibility checks — can your design add unsubscribe and delivery guarantees without touching the core broadcast path.
Part 3 — Concurrency & Delivery Guarantees
1. Multiple Producers and Consumers
Producer 1 ──┐
├──► Topic A
Producer 2 ──┘
Consumer 1 ──┐
Consumer 2 ──┼──► Topic A
Consumer 3 ──┘
2. Concurrent Publishing
3. Concurrent Consumption
Subscribers should process messages independently and in parallel — a slow consumer must not block others.
4. Thread Safety
Thread 1 → adding a subscriber, Thread 2 → publishing, Thread 3 → removing a subscriber — all at once. Protect with concurrent collections, locks, or copy-on-write snapshots.
5. Delivery Isolation
6. Message Output
When a subscriber receives a message, print:
<consumer_id> received <message>
For example:
consumer1 received Payment Successful
consumer2 received Payment Successful
Example Usage: Full Walkthrough
Here's how a sample session runs end-to-end.
1. Create topics
topicA
topicB
2. Create producers
producerA
producerB
3. Create consumers
consumerA, consumerB, consumerC, consumerD, consumerE
4. Subscriptions
topicA: consumerA, consumerB, consumerC, consumerD, consumerE
topicB: consumerA, consumerC, consumerE
5. Publishing messages
producerA → topicA → "Order Created"
producerA → topicA → "Order Confirmed"
producerB → topicA → "Order Shipped"
producerA → topicB → "Payment Initiated"
producerB → topicB → "Payment Completed"
6. Expected behavior
Three messages on topicA are received by all five subscribers:
consumerA received Order Created
consumerB received Order Created
consumerC received Order Created
consumerD received Order Created
consumerE received Order Created
Two messages on topicB are received only by:
consumerA, consumerC, consumerE
→ consumerA received Payment Initiated
consumerC received Payment Initiated
consumerE received Payment Initiated
Exact output ordering may vary because consumers are allowed to execute concurrently — but per-topic order per subscriber must be preserved.
The queue correctly fans out per-topic — five subscribers get all three topicA messages, only the three subscribed to topicB get its two messages, and concurrent publishers/consumers make progress without racing on the registry.