What is Pub/Sub? The Publish/Subscribe model explained

Pub/Sub (or Publish/Subscribe) is an architectural design pattern used in distributed systems for asynchronous communication between different components or services. Although Publish/Subscribe is based on earlier design patterns like message queuing and event brokers, it is more flexible and scalable. The key to this is the fact that Pub/Sub enables the movement of messages between different components of the system without the components being aware of each other’s identity (they are decoupled). 

The Pub/Sub pattern evolved out of the necessity to expand the scale of information systems. In the pre-Internet era, and even during the early days of the Internet, the systems were mostly scaled statically. However, with the expansion of the Internet and web-based applications, fueled by the massive adoption of mobile and IoT devices, systems needed to scale dynamically. 

The decoupled nature of the Pub/Sub pattern makes it a good candidate for governing the architecture of dynamically scalable systems. Pub/Sub makes it possible to manage scale without overloading the program logic of system components.

Pub/Sub provides a framework for exchanging messages between publishers (components that create and send messages) and subscribers (components that receive and consume messages). Note that publishers don’t send messages to specific subscribers in a point-to-point manner. Instead, an intermediary is used - a Pub/Sub message broker, which groups messages into entities called channels (or topics).

The pub/sub pattern

Learn more about how Pub/Sub works and its architecture

The Pub/Sub pattern brings many benefits to the table, including but not limited to:

• Loose coupling between components, making your system more modular and flexible.

• High scalability (in theory, Pub/Sub allows any number of  publishers to communicate with any number of subscribers).  

• Language-agnostic and protocol-agnostic, which makes it straightforward and fast to integrate Pub/Sub into your tech stack. 

• Asynchronous, event-driven communication that’s ideal for realtime, low-latency apps. 

Learn more about the benefits of Pub/Sub

Pub/Sub’s loose coupling, asynchronous nature, and inherent scalability make it an excellent solution for distributed systems with a high and fluctuating number of publishers and subscribers. You can use Pub/Sub for many different purposes, such as:

• Sending event notifications.

• Distributed caching.

• Distributed logging.

• Working with multiple data sources.

• Broadcasting updates (one-to-many messaging). 

• Building responsive, low-latency end-user experiences like live chat and multiplayer collaboration functionality. 

Learn more about Pub/Sub use cases

Every design pattern has limitations and trade-offs. There are scenarios where Pub/Sub is the wrong choice:

• Overkill for simpler systems. Pub/Sub is an overkill for simple systems which are unlikely to scale up. If you’re operating systems where elastic scaling is not required and where static scaling will suffice, think twice before using Pub/Sub. Similarly, if your system only needs to communicate with a few other components or services, using Pub/Sub might be excessive.

• Not suitable for media streaming. Audio and video streaming have nuanced requirements for smooth rendering on the user’s end. Synchronous point-to-point communication between the two endpoints is the best solution for media streaming. Pub/Sub is not suitable for carrying VoIP or video telephony traffic over the Internet. 

• Inappropriate for periodic/background tasks. Pub/Sub is an asynchronous method of sending information. Therefore, it is not suitable for systems that run as periodic background tasks, like cron jobs which are triggered over a particular time period.

• Non-event-driven systems. Pub/Sub is typically used in event-driven systems, where components react to changes in the system state. If your system is not event-driven, but instead follows a more traditional request/response or batch processing model, Pub/Sub might not be the best choice.

• Synchronous communication: Pub/Sub is designed for asynchronous communication, where the publisher doesn't wait for a response from the subscriber. If you need a synchronous request-response communication pattern, you should consider using something else instead of Pub/Sub.

The Pub/Sub model is highly scalable by design. In theory, Pub/Sub allows any number of producers to send events to any number of subscribers. In practice, the actual number of publishers and subscribers that a Pub/Sub system can support depends on a variety of factors, including the hardware and software resources of the system, the message size and frequency, and the network bandwidth. Robust Pub/Sub systems like Kafka and Ably can reliably handle up to millions of concurrent publishers and subscribers, and billions of messages.

However, if you wish for your Pub/Sub system to be dependable at scale, there are other factors to consider in addition to the number of publishers and subscribers it can handle:

• How do you maintain consistently low latencies at scale?

• How do you increase bandwidth performance?

• How will you preserve data integrity (guaranteed message ordering and delivery)?

• How do you ensure your system is highly available?

• What can you do to make the system reliable and fault tolerant?

If you choose the Pub/Sub route, it's essential that you can rely on the tooling you choose. For most teams, that means using a managed pub/sub platform rather than building a solution in-house.

In our own research, we've found that in-house realtime infrastructure projects have a strong tendency to run over budget and over time. And even after all that, they often cost more overall.

If you decide to use a managed Pub/Sub service though, you need to make sure it is fit for purpose. This means evaluating it based on scalability, reliability, performance (including latency), guarantees, ease of development, cost of ownership, and its ability to deliver functionality beyond Pub/Sub.

Thankfully, we have done a review of the five best Pub/Sub solutions to help you to compare their offerings.

Ably is a realtime experience platform. Our APIs and SDKs power realtime functionality for over 350 million end-users each month, and remove the need for developers to maintain and scale messy infrastructure - whether adding collaborative features with our Spaces product, or using Pub/Sub channels to sync edits with the backend or broadcast live updates to clients. 

Key Ably features and capabilities include:

• Pub/Sub messaging over serverless WebSockets, with rich features such as message delta compression, automatic reconnections with continuity, user presence, message history, and message interactions.

• Multi-protocol capabilities; in addition to WebSockets, we support protocols like SSE, MQTT, and HTTP/REST. 

• Managed integrations with tools like Kafka, AWS SQS, AWS Kinesis, and RabbitMQ. 

• A globally-distributed network of datacenters and edge acceleration points-of-presence. 

• Guaranteed message ordering and delivery. 

• Global fault tolerance and a 99.999% uptime SLA.

• < 65ms round-trip latency (P99).

• Dynamic elasticity, so we can quickly scale to handle any demand (billions of messages sent to millions of Pub/Sub channels and client connections).