In this post, we take a look at Pub/Sub examples that will help us understand how companies and organizations across all types of industries are leveraging this architectural design pattern to solve common engineering challenges and improve their architecture.
What Is Pub/Sub?
Pub/Sub (or Publish/Subscribe) is an architectural design pattern used in distributed systems for asynchronous communication between different components or services. Although Publish/Subscribe is based on earlier design patterns like message queuing and event brokers, it is more flexible and scalable. The key to this is the fact that Pub/Sub enables the movement of messages between different components of the system without the components being aware of each other’s identity (they are decoupled).
Learn more about:
When to use Pub/Sub
Pub/Sub’s loose coupling makes it a powerful solution for many common engineering challenges. Its scalability makes it a great solution for distributed architectures with a high number of clients. Let’s take a look at a few examples of how you can use Pub/Sub messaging to address said problems and improve your architecture.
Event notifications
Sending events to large numbers of recipients simultaneously is a common technical challenge. You need the system to rapidly deliver events to your clients, but without negative side effects that materialize when the number of destinations increases massively. Depending on the design, clients might also not always be available. When they do come online after the fact, you need them to receive the messages they missed.
Pub/Sub’s loose coupling means that publishers can send events without worrying about which clients are online. They don't even need to know when new clients connect to the system. Furthermore, assuming a robust implementation of a broker which can persist messages and has exactly-once delivery (or at-least-once delivery at a minimum), publishers can simply “fire and forget,” knowing that the message broker will deliver the messages when the client needs them.
Split uses Ably’s Pub/Sub capabilities to distribute feature flag events to tens of millions of client applications. Delivering feature flag changes within seconds is critical to Split’s customers' needs since breaking changes need to be reverted as quickly as possible. Or Split might need to coordinate a new feature with a marketing announcement. Either way, events need to be delivered on time, and reach all clients, regardless of whether said clients were online at the time or end up connecting later.
Split made this possible by shifting their system from a polling model to a responsive, event-driven architecture. Client applications receive events in realtime if they're online. If they're not, Ably updates their application state as soon as they reconnect to the network.
Distributed caching
Caches are an important tool for improving system performance by having the latest message available to the subscribers stored locally/nearby and updating this copy only when changes occur. Depending on your application, you may need to distribute them across multiple systems, in order to, for example:
- Set up parallel systems to overcome memory or CPU constraints.
- Distribute caches across multiple locations to reduce network traffic and minimize load on central servers.
- Keep the state of multiple caches synchronized.
With Pub/Sub, you can seed caches asynchronously. As processes create data and events, the messaging system delivers events to caches. As such, rather than waiting for a nearby client to make a related request, the data is already in the cache. When new information supersedes data in the cache, a new event replaces it. This design is far more efficient in updating state than relying on polling semantics. Using an event stream to populate caches prevents the case where your clients are polling for information that isn’t up to date.
Seats.io is a seating reservation system. They draw their floor plan, assign seats to it, and then manage the live seat-booking events. As clients buy tickets, Ably’s Pub/Sub system updates the floor plan everywhere in realtime and keeps the distributed cache system in sync. Applications thus cache the seating chart and can update it in realtime. Clients never end up requesting a seat only to find out someone had bought it while they were still in the browsing/shopping phase.
Distributed logging
Another critical tool for distributed systems is application logs. Logs are an inexpensive tool for reconstructing events, triggering alerts as events occur, and tracking critical transactions.
With the Pub/Sub pattern, you can send logs to many subscribed destinations simultaneously. You have the option of creating explicit logging channels or having message recipients log events in more than one destination. For example, one endpoint can subscribe to logs and send them on to a centralized system like Splunk or Elasticsearch. At the same time, other endpoints can send logs to screens or save them to a file for localized troubleshooting. Yet another use case is subscribing to transactional logs for compliance purposes.
Pub/Sub is well suited for this and can thus add extra flexibility to one of your most important monitoring resources in a number of ways without writing new code or building expensive redundant systems.
Multiple data sources
Loose coupling between message senders and message receivers makes Pub/Sub systems naturally elastic. You can add new processes at any time, and you can remove or replace them as needed. Message senders and receivers can be located anywhere that a connection to the message broker is possible. Additionally, any system can be a sender, a receiver, or both.
Experity uses Ably’s Pub/Sub capabilities to collect data from multiple sources and process it with Kafka. Combining Kafka’s streaming capabilities with a last-mile delivery Pub/Sub system that matches its realtime guarantees displays data from diverse sources on a single dashboard for their users. Since Experity’s business logic isn’t coupled to the messaging system, devices can leave and join the network without additional configuration or network changes.
Use cases that require burstability and scale
A well-designed Pub/Sub system has the ability to send ordered messages out to large numbers of subscribers suddenly and simultaneously as they come and go, connecting and reconnecting unpredictably, and keeping their state in sync. In terms of design, this is best handled by outsourcing the task to a designated broker component that is designed to handle busy, haphazard network behavior (and which ideally also handles data integrity). Offloading the responsibility of massive fanout bursts messages to a broker gives Pub/Sub systems a distinct advantage in terms of scale.
Wooclap is an educational tool that allows students to participate in the classroom via a web-based app. Wooclap’s large numbers of simultaneous clients often encounter issues such as unreliable Wi-Fi or unpredictable cellular networks. The robust design of Ably’s Pub/Sub system recovers their connection when they rejoin the network, and is able to handle the often sudden, large numbers. With the added benefit of data integrity, this allows the participants to rejoin the classroom experiences seamlessly.
Seamless and responsive end-user experiences
Pub/Sub is a good choice for delivering realtime experiences to end-users. By using the Publish/Subscribe pattern, communication can happen asynchronously, meaning that the sender doesn't have to wait for the receiver to respond before continuing its work. Furthermore, by design, Pub/Sub is equipped to handle a volatile and constantly changing number of concurrent users. With Pub/Sub as the backbone of your messaging system, you can:
- Power live chat experiences.
- Broadcast realtime event data, such as live scores and traffic updates.
- Send realtime alerts and notifications.
- Facilitate multiplayer collaboration on shared projects and whiteboards.
- Keep your backend and frontend in realtime sync.
- Add live location tracking capabilities to your apps.
Situations where Pub/Sub is a bad choice
While Pub/Sub is a useful communication pattern in many scenarios, there are situations where it might not be the best choice. Here are a few examples:
- Overkill for simpler systems and small-scale applications that only need to communicate with a few other components or services.
- Not suitable for audio and video streaming.
- Inappropriate for periodic or background tasks.
- Not ideal for non-event-driven systems and use cases where you need synchronous communication.
Learn more about scenarios where you shouldn’t use Pub/Sub
Deliver realtime experiences at scale with Ably’s Pub/Sub messaging
Ably is a realtime PaaS. Our APIs and SDKs help developers ship realtime experiences like chat and multiplayer collaboration without having to worry about maintaining and scaling messy infrastructure.
Key Ably features and capabilities:
- Pub/Sub messaging over serverless WebSockets, with rich features such as message delta compression, automatic reconnections with continuity, user presence, message history, and message interactions.
- Multi-protocol capabilities; in addition to WebSockets, we support protocols like SSE, MQTT, and HTTP/REST.
- Managed integrations with tools like Kafka, AWS SQS, AWS Kinesis, and RabbitMQ.
- A globally-distributed network of datacenters and edge acceleration points-of-presence.
- Guaranteed message ordering and delivery.
- Global fault tolerance and a 99.999% uptime SLA.
- < 65ms round-trip latency (P99).
- Dynamic elasticity, so we can quickly scale to handle any demand (billions of messages sent to millions of Pub/Sub channels and client connections).
Explore our documentation to find out more and get started with a free Ably account.
