Pensieve

Recent Notes

See 67 more →

Message Queues vs. Publish-Subscribe Systems in System Design

2 min read

  • Message Queues deliver messages to one or more consumers in a point-to-point model
  • Publish-Subscribe systems allow messages to be broadcast to multiple subscribers

Message Queues

Message queues are a communication mechanism in system design where messages are stored in a queue and delivered to one or more consumers in a point-to-point model. Producers send messages to the queue, and consumers retrieve them, ensuring that the messages are processed in a FIFO (First In, First Out)* order.

Message queues decouple producers and consumers, allowing them to operate independently, thereby improving system scalability and reliability. They are commonly used in distributed systems to handle asynchronous communication, load balancing, and task distribution.

Key Features

Pros:

  • Messages are guaranteed to be delivered even if the receiver is temporarily unavailable
  • Maintains the order of the messages, ensuring they are processed sequentially
  • Distributes tasks evenly across many consumers, improving performance

Cons:

  • A message can only be processed by one receiver, which is not suitable for broadcasting
  • Can be more difficult to scale when many consumers need to process the same data.

Publish-Subscribe (Pub-Sub) Systems

Publish-Subscribe (Pub-Sub) systems are a messaging pattern where producers, called publishers, send messages to a topic or channel, and consumers, known as subscribers, receive messages based on their subscription to that topic

Unlike message queues, in Pub-Sub systems, messages are broadcast to multiple subscribers simultaneously, enabling many-to-many communication.

Pros:

  • One message can be sent to multiple consumers at once, making it ideal for systems with multiple subscribers.
  • Publishers and subscribers don’t need to know about each other, improving modularity.
  • Easy to add or remove subscribers without affecting the system.

Cons:

  • If a subscriber is offline, it may miss messages, depending on the system configuration.
  • Messages can be duplicated, leading to higher processing requirements for subscribers.
  • Messages may not always arrive in the order they were sent.

Ephemeral pub/sub (non-durable)

  • If no subscriber is listening (or it’s disconnected), the message can be lost.
  • Great for real-time updates where “latest state” matters more than “never miss anything.”
  • Examples: classic Redis pub/sub, some WebSocket event buses, non-durable NATS (core).

Durable pub/sub (persistent)

  • Messages are stored, and subscribers can:
    • ack to mark processed
    • catch up if they were offline
    • sometimes replay from earlier points
  • Examples:
    • Kafka / Pulsar / Redpanda (log-based topics with retention + replay)
    • AWS SNS with durable subscriptions (e.g., SNS → SQS makes it durable per subscriber)
    • NATS JetStream (durable streams + consumers)

Linked Map of Contexts

Graph View