Message Queue Patterns: Point-to-Point vs Publish-Subscribe vs Request-Reply

Message queues enable asynchronous communication between distributed systems, decoupling services and improving resilience. Selecting the correct pattern determines system coupling, scalability, and delivery guarantees.

Point-to-Point (P2P)

The Point-to-Point pattern uses a Queue to establish a one-to-one relationship between a producer and a consumer.

Mechanics

1. Producers send messages to a specific named queue.
2. Messages are stored in the queue until consumed.
3. Consumers compete for messages; each message is delivered to only one consumer.
4. Once a message is acknowledged, it is removed from the queue.

Technical Characteristics

• Load Balancing: Multiple consumers can listen to the same queue to distribute load.
• Temporal Decoupling: The producer does not need to know if the consumer is online.
• Guarantee: "At-least-once" delivery. Exactly-once processing requires application-level idempotency or specific broker configurations (e.g., Kafka transactions).
• Ordering: The queue maintains FIFO order for delivery. While a single consumer processes messages sequentially, multiple consumers process in parallel, resulting in out-of-order completion.

Use Cases

• Task execution queues (e.g., image processing, email sending).
• Work distribution where strict ordering is required per processing thread.

Publish-Subscribe (Pub/Sub)

The Publish-Subscribe pattern uses a Topic to establish a one-to-many relationship between a publisher and subscribers.

Mechanics

1. Publishers send messages to a specific topic.
2. The messaging broker creates copies of the message.
3. Subscribers receive a copy of every message sent to the topic.
4. Subscribers use filtering (subscription criteria) to receive only relevant subsets of messages.

Technical Characteristics

• Fan-out: Efficiently delivers data to multiple consumers simultaneously.
• Spatial Decoupling: Publishers have no knowledge of subscriber existence or count.
• Ephemeral vs Durable: Non-durable subscriptions only receive messages sent while actively subscribed. Durable subscriptions ensure offline subscribers receive missed messages upon reconnection.
• Filtering: Implemented via broker-side routing (e.g., RabbitMQ topic exchanges matching routing keys) or consumer-side filtering (discarding irrelevant messages after receipt).

Use Cases

• Event notification systems (e.g., user updates, stock price feeds).
• Data synchronization across multiple disparate services.

Request-Reply

The Request-Reply pattern simulates synchronous request-response behavior over an asynchronous messaging infrastructure.

Mechanics

1. The Client sends a request message to a designated queue.
2. The message includes two critical properties: Reply-To (a temporary queue name) and CorrelationId (a unique identifier).
3. The Server consumes the request, processes it, and sends the response to the queue specified in Reply-To.
4. The Client listens on the temporary queue and matches the incoming CorrelationId to the original request.

Technical Characteristics

• Temporal Coupling: The client blocks or waits asynchronously for the response.
• Stateful: Requires the client to maintain pending request state.
• Bidirectional: Requires two distinct channels (request queue and reply queue).

Use Cases

• Remote Procedure Calls (RPC) where messaging middleware is required.
• Querying stateful services in an asynchronous architecture.

Message Lifecycle & Reliability

Persistence & Durability

Queue durability ensures the queue definition survives broker restarts. Message persistence ensures individual messages are written to disk rather than stored only in memory. Both are required to prevent data loss during a restart.

Time-to-Live (TTL)

TTL defines the expiration time for a message. If a message is not consumed within the specified TTL, it is removed from the queue. TTL can be applied per-message or configured as a policy for an entire queue to prevent stale data accumulation.

Dead Letter Queues (DLQ)

A DLQ is a secondary queue where messages are routed if they cannot be processed successfully. Common triggers include exceeding retry limits, message format validation failures, or processing exceptions. DLQs allow operators to inspect and reprocess failed messages without blocking the main processing pipeline.

Code Example

The following example demonstrates the Request-Reply pattern using a generic AMQP-style JavaScript implementation. It includes both the Server and Client implementations, queue assertion, error handling, and connection cleanup.

const amqp = require('amqplib');
const crypto = require('crypto');

// --- Server Implementation ---
async function rpcServer() {
  try {
    const connection = await amqp.connect('amqp://localhost');
    const channel = await connection.createChannel();
    const requestQueue = 'rpc_queue';

    // Assert queue to ensure it exists and is durable
    await channel.assertQueue(requestQueue, { durable: true });
    channel.prefetch(1);

    console.log(' [x] Awaiting RPC requests');

    channel.consume(requestQueue, (msg) => {
      const n = parseInt(msg.content.toString());
      console.log(' [.] Received request:', n);

      const response = n * 2; // Example processing logic

      channel.sendToQueue(msg.properties.replyTo, 
        Buffer.from(response.toString()), 
        { correlationId: msg.properties.correlationId }
      );
      channel.ack(msg);
    });
  } catch (error) {
    console.error('Server Error:', error);
  }
}

// --- Client Implementation ---
async function rpcClient() {
  const connection = await amqp.connect('amqp://localhost');
  const channel = await connection.createChannel();
  
  try {
    const requestQueue = 'rpc_queue';
    
    // Assert request queue to prevent errors if it doesn't exist
    await channel.assertQueue(requestQueue, { durable: true });
    
    const replyQueue = await channel.assertQueue('', { exclusive: true }).then(q => q.queue);
    const correlationId = crypto.randomUUID();

    return new Promise((resolve, reject) => {
      // Listen for the response
      channel.consume(replyQueue, (msg) => {
        if (msg.properties.correlationId === correlationId) {
          console.log(' [.] Got response:', msg.content.toString());
          channel.ack(msg);
          resolve(msg.content.toString());
        }
      }, { noAck: false });

      // Send the request
      channel.sendToQueue(requestQueue, 
        Buffer.from('10'), 
        { correlationId: correlationId, replyTo: replyQueue }
      );

      // Timeout to prevent resource leaks if no response is received
      setTimeout(() => {
        reject(new Error('Request timed out'));
      }, 5000);
    });
  } catch (error) {
    console.error('Client Error:', error);
    throw error;
  } finally {
    await connection.close();
  }
}

This code sets up a temporary exclusive queue for the reply, generates a unique Correlation ID, asserts the request queue exists, and implements error handling and connection cleanup.

Getting Started

1. Define your coupling needs: Use P2P for work distribution (one consumer), Pub/Sub for broadcasting (many consumers), and Request-Reply for querying.
2. Select your broker: Choose a broker that supports your required patterns (e.g., RabbitMQ excels at routing, Kafka excels at Pub/Sub streams).
3. Configure Reliability: Enable persistence for critical data, set TTLs to manage queue bloat, and configure Dead Letter Queues (DLQ) to capture and analyze failed messages.
4. Monitor depth: Track queue depth to prevent backpressure buildup that can crash consumers.