In usage and use case, webhooks and WebSub are similar… but different. The article below sheds some more light on when and how to use each.
What are Webhooks?
From GitHub to Mailchimp, webhooks are everywhere. They are a user-defined Callback-URL, which needs to be registered at the service provider’s platform. This URL will be called by the service provider using a POST request to send information about an event to the vendor.
The vendor will either send the 200OK response (or a response with standards specified by the provider), to acknowledge the response was received. If the return response does not match the provider’s pre-defined rules then the provider will keep sending the information on the same registered callback-URL. This behavior can be customized and configured by the service provider.
Information on updates will be stored by the vendor to keep their databases in sync with the service provider's information.
Webhooks are generally used when there is one subscriber (company or a domain) that needs to be notified of events related to their business/domain playing out on different providers, who both expose webhooks.
What is WebSub?
WebSub meanwhile is a network of Publishers, Subscribers, and Hubs that make it possible to share content over the internet to whoever has opted to receive this data. The Publisher is the content producer, the Subscriber the content consumer and the Hub the content distributor.
WebSub has the advantage of inbuilt security mechanism for sharing, accepting and distributing content more securely between publishers and subscribers. This inbuilt security is known as 'verification of intent'. This security handshake happens at any stage of the process, often at the time of subscribing and unsubscribing.
Similarly, the subscriber subscribes the topic URL to the Hub, as advertised by the publisher. After this, whenever the publisher publishes content or updates existing content to the Hub, the Hub automatically pushes these contents to all the Subscriber(s) who are tuned in to this topic. So, instead of a server sending periodic polling requests, each Subscriber will receive updates - ie have the content pushed to them - in real time. The Subscriber can unsubscribe from this topic anytime - they just need to send the request to the Hub.
WebSub comes into its own where the same data from Publisher (eg an article from Medium author) is floated to millions of Subscribers using a Hub. So, the data is not really personal anymore. Anyone can hook to a Topic and get the data. There is minimal human input in the transfer of data. Automated fanning out of information is where WebSub shines.
Webhooks and WebSub in action - which is good when?
The paragraphs below give an overview of webhooks and WebSub in different contexts to give an idea of their optimal function.
Webhooks
Where webhooks do well
Realtime updates
Webhooks function in near realtime to transfer information to vendors on events that have occurred provider-side, ensuring the two are in sync. This realtime approach is known as event-driven notifications for Webhooks and is useful both for providing immediately actionable, up-to-date information and also for keeping two remote systems in sync.
Loose-Coupling
A webhook loosely couples two applications to establish a communication channel between them. These two systems or applications can exchange information on particular events, acknowledging the respective event without having to ‘know’ the other party’s server architecture, functionality or configuration. The communication happens asynchronously and is event-driven. This saves time and hassle when integrating systems.
Downtime Intolerance (delivery guarantees)
With webhooks, the event information will never get lost, even if the vendor party is experiencing downtime. This is because the Provider will keep sending event data periodically to the callback-URL until the vendor responds back with the 200OK response with the Provider’s predefined response body. Usually, the Provider follow-ups the retry several times if they don’t get a success response from the vendor and before they mark this data as an abandoned. This functionality differs provider by provider, so it is worth reading their individual strategies.
Easy integration and consumption
WebHooks is easy to integrate into any application or system. It’s just a matter of registering the callback-URL. All other control will be in the consumer’s hands once they consume the data from the webhook call.
Custom Security
Webhooks don’t have widely accepted security standards nor any in-built security system. But if we can build our own security layer on top of the webhook then we can keep ownership of the security controls and enhance the security according to the existing system architecture’s security standard. Here security of each webhook connection would be governed by webhooks’ provider and consumer. Both will follow a predefined set of security standards before or while communicating via webhook.
Where webhooks fall short
Debugging
Webhook callback-URL is called from the remote application with a data payload. As it is executed from an external environment, it is not easy to debug the request on the client-side. To overcome this problem, a provider should expose the sandbox environment to developers. This allows for testing and debugging using trial and error.
DDoS Attack
As long as the webhook publicly exposes the callback-URL, there’s a risk it will be misused by attackers. The attacker can send malicious event data to the webhook and generate high traffic to bring down a vendor’s servers. To avoid this kind of situation, strong security is required. Such attacks can attack both provider and vendor side, thus anti-DDoS measures should be put in place by both the parties.
Missed Notifications
At the time of downtime, the vendor could miss notifications from the provider if the provider is not retrying (or limited retry) the WebHook to deliver the information to the respected vendor. Here webhooks are absolutely controlled by the third-party service provider. No control at the vendor’s hand.
Deduplication
The webhook payload should be consumed properly. The return response must be the 200OK and follow the provider’s pre-defined standards. Otherwise, the provider will keep calling the webhook multiple times and the database will get clogged up with duplicate entries vendor side. As discussed above, the debugging process is difficult for webhooks and this use case must be taken care as a top priority.
WebSub
Where WebSub does well
Realtime updates
A consumer receives the topic contents and updates in realtime, avoiding the need for periodic polling. A Publisher posts content to the Hub, which further optimizes distribution to subscribers.
Security
WebSub comes with an in-built security feature and follows the community adopted security standards at all stages of the data transfer process. At the time of subscription, unsubscription and publish content, the Hub checks the locally computed HMAC keys to ensure the security of the connection.
Loose coupling
In WebSub pattern, the Subscriber doesn’t know anything about the architecture, configurations or structure of the Hub’s or Publisher’s server in order to get contents. This allows for an enhanced level of security abstractions between all the WebSub network entities.
Where WebSub falls short
Debugging & Integration
Just as with webhooks, debugging can be complex in WebSub as it’s functions in a loosely-coupled architecture and the content format is not predictive. Integration also requires several steps - finding the topics, subscription request, and security check request, which is a lengthy process.
Hub downtime
The Publisher and Subscriber are both dependent on the Hub. The publisher sends the content to Hub and shares with the Subscriber. If the Hub goes down then the whole WebSub pattern cycle is interrupted.
Which is the better choice (and when)?
The sections above both give a comparative overview of Webhooks and WebSub and their strengths and weaknesses. Which one is better? Both the standards are built to deliver the content or updates almost in realtime to the consumer. But still, the perfect answer is it depends - on the use case. The paragraphs below will help you decide.
Webhooks
Webhooks is a good choice for the event-driven notification system. When the consumer needs to be notified on every event occurrence with their account at the providers' platform, the webhook implementation is ideal for achieving this. WebHooks help a consumer keep in sync with the provider’s status as the provider sends data on each event back to the consumer via callback-URL.
The Webhook is a better choice when SLA of the delivery is considerable. Webhooks (by their nature) may have a stricter SLA requirements, while WebSub may have a loose constraints on timely delivery.
Examples for when webhooks win out are payment gateways, email providers, SMS & calling service provider, analytics platforms and so on. When any payment happens at the payment gateway, the vendor system has to know about it so they can confirm the order. Here payment gateway will return the payment information using webhooks to inform the vendor about it. Stripe, SendGrid, Twilio, Slack are good real-life examples for WebHooks.
On every charge or refund, Stripe uses webhooks to report these results to the vendors/consumer synchronously to keep them updated with stripe’s events.
SendGrid provides webhooks to get notified on every event occurrence for email delivery and its engagement. SendGrid’s webhook will return the success delivery event data. The various events are delivery, opened, click on a link, spam.
Twilio uses webhooks to send event data back to vendor application on Voice Call and SMS events like call received, call hangup, SMS delivered, read, etc.
CRMs
HubSpot, Freshdesk, SalesForce and other CRMs provide the webhooks to return the event information like account creation, permission assign, ticket rise/deletion/response and other CRM centric events.
Microservices
A fully functional system can architect with multiple individual microservices, each microservice is responsible to perform a specific job. Like email service, Auth service, notification service. Here webhooks can be a communication bridge between all the microservices. They can share the event information for the communication and controlling purpose and keep the system synchronized across all the servers.
WebSub
Websub is a good choice for use-cases such as broadcasting public information/data from a single publisher (or small number of publishers) to millions of subscribers via hub. In a broader way, many subscribers subscribe to their interested topics and get notified when any publisher publish those topics to hub. In such a case the provider and the consumer don’t need to know each other. They communicate solely relevant information, and contact is dealt with by the intermediary Hubs.
For platforms who frequently generate or update new content, by whom we mean news aggregators, podcast networks, publishers, streaming services, and so on, WebSub can be considered the perfect solution. It will distribute content changes in real time, as updates are tailor-made for the audience.
WebSub is a better choice when SLA of the delivery is not a concern. WebSub (by their nature) may have a looser SLA requirements, vs the Webhook, which may have a strict constraints on timely delivery.
Real-life examples of WebSub implementations in action include:
Publishers: Medium, Wordpress, Blogger, CNN, etc
Subscribers are Flipboard, Feedly, NewsBlur etc.
Hubs are pubsubhubbub.appspot.com and pubsubhubbub.superfeedr.com
Publishers, Hubs, and Subscribers
All these blogs and article publishers publish their content to some community hosted Hubs like pubsubhubbub.appspot.com and pubsubhubbub.superfeedr.com which are maintained and operated by Google and Superfeedr.
And subscriber examples are Flipboard, Feedly, NewsBlur which subscribes to the Hubs to get the latest updates from the Publishers.
The table below provides a quick comparison based on use case:
Our view of WebHooks and WebSub
At Ably, we provide realtime messaging infrastructure as a service and certainly think WebHooks and WebSub are both great ways to keep up with the event-driven world.
WebHooks are increasingly being used by our customers to perform various actions as a realtime reaction to events. WebHooks also allow users to implement well understood scaling mechanisms to deal with the incoming event notifications, generally making way for an easier integration and usage overall.
At Ably, we offer a 100% reliability or message delivery guarantees to our users, in other words, no message ever get lost - there are ways to store or queue the messages if the receiver of that data is temporarily unavailable. However, with our WebHooks offering, there’s currently no way with HTTP (other than retries for a given time) to queue undelivered messages, meaning, if the endpoint is unavailable for a particular duration, the message will essentially be lost. We are working to make our WebHook events more durable to avoid this in the future.
WebSub is not something we already offer to our users but it’s in the making. We decided to build a WebSub integration as it not only looks like a promising specification but it also overcomes some of the limitations of WebHooks. WebSub would be a nice addition to our diverse set of integrations that we offer as part of our Reactor service, to be in-sync with the ongoing industry standards, so we are really looking forward to it.
Resources
Topic Pages
Engineering Blog
WebSub – A deep dive
Ably Realtime provides cloud infrastructure and APIs to help developers simplify complex realtime engineering. We make it easy to power and scale realtime features in apps, or distribute data streams to third-party developers as realtime APIs. Create a free Ably account to get started.
