Latency is a technical term that, even for the people who use it every day, tends to feel abstract. When a metric calls for measurement by the millisecond, it’s easy to forget just how valuable each millisecond is.
But if your application promises to deliver realtime updates – especially in the sports, media, and entertainment industries – low latency has to become a high priority and the experience your users have with latency needs to be vivid.
Think of the sports fan in a taxi on the way to the airport – a business meeting across the country demanding their presence and keeping them away from their television. If their favorite baseball team is in the playoffs, you can imagine how anxious they might be as they look at their phone and wait for updates.
Here, latency is pivotal. A near-instantaneous update after a player runs across home base is a delight and the user can text their friends, participating in the excitement even from afar. But a delayed update is frustrating. As text messages roll in and the user refreshes and refreshes without an update, even a small delay can feel like an eternity.
In this context, latency is make-or-break, and maintaining low latency is a constant priority. But even though engineers in every company care about latency and most have had experience improving performance, few have had experience thinking about latency at this scale and with these expectations.
In a previous article, we covered why realtime updates matter to the sports, media, and entertainment industries and explained, at a high level, the core elements that ensure an effective realtime update infrastructure. Here, we’re going to go deep into latency and explore why it matters and how to address it.
What do we mean by low latency and what are the parameters for realtime updates?
Latency refers to the duration of time it takes for data to travel from the backend of an application (connected to a data center or public cloud) to the device of an end user.
Developers have always made latency reduction a priority – faster delivery of data is always good – but in most contexts, moments of high latency aren’t too costly. If it takes a few extra seconds for a user to load Instagram, for example, users will most likely just be annoyed. But if a news app promises realtime updates and a user is left with a spinning wheel, the feature is effectively broken.
This almost feels unfair at first glance, but in the context of realtime updates, the value of a “live” update can collapse based on latency alone. A fresh update makes users feel connected to an event, team, or celebrity but a delayed update – even if the information remains relevant and accurate – doesn’t provide the same value.
That’s why it’s worthwhile – necessary, even – for companies providing realtime updates to measure latency by the millisecond. Updates are never actually live; there will always be some kind of gap between when a message is created and when it hits a user’s device. Realtime updates rely on the way human perception experiences certain levels of speed as effectively instant.
For latencies to feel “live” or instantaneous, latencies need to be in the low hundreds of milliseconds. For companies offering realtime updates, a latency gap as small as 100ms can make the difference between a delighted user and a frustrated user.
Two reasons why low latency is essential for realtime sports, media, and entertainment apps
Latency levels of 100ms or less are hard to achieve at any one point in time but treating latency as a static goal misses the point and misses the true challenge. For apps with realtime updates, latency has to be low and remain low, such that users can consistently expect it to be low.
To see this better, it’s worth turning to the two major reasons that show why low latency is so important: The brand promise of realtime and the user expectations of scalability.
1.The brand promise of realtime
Coca-Cola CEO Muhtar Kent once said that “A brand is a promise and a good brand is a promise kept”. This framing is useful but a little abstract because one promise might entail a different level of commitment than another.
A task management app, for example, might build its brand on the promise of making users feel “empowered” or “productive.” But those qualities are subjective and the company will have some space for defining what that really means.
When you build a sports, media, or entertainment app with a realtime updates feature, however, the promise is inherently stricter. The other parts of the app might have some flexibility – whether an interface is “intuitive,” for example, is again subjective – but the realtime promise is close to black-and-white.
Research has shown that the feeling of instantaneousness, despite sounding fuzzy, has clear metrics – even though these metrics are mostly unconscious for end-users. To deliver an experience that fulfills the promise of realtime, latency needs to be in the low hundreds of milliseconds.
But as companies aim for that number, they need to realize there’s very little wiggle room. If latency gets worse, the experience doesn’t just feel worse. The experience can be compromised and the promise can feel broken.
2. Expectations of scalability
Despite the strict expectations outlined above, latency would still be more easily solvable if it were a single problem. What makes latency significantly more difficult is the parallel expectation users have for scalability.
When a product promises a live experience, users expect to get that live experience at any time and from any location. Nothing should damage the latency levels that make the experience feel instantaneous.
Even if an event is disproportionately popular and draws in a huge fluctuation of users – the World Series, for example – users will still expect a “live” experience.
From the inside, companies and developers know how hard it can be to fulfill these expectations in different circumstances. Delivering realtime updates gets much harder if users are connected to flakey networks on their phones, for example, but users will expect a realtime experience nonetheless.
How sports, media, and entertainment apps can guarantee low latency
Throughout this article, we’ve focused on the benefits and the challenges of ensuring low latency in realtime contexts. The main decision companies then face is whether to take on the work of building their own realtime update systems or shopping for and purchasing a third-party system.
With either path, companies need to consider a few factors:
- How will you ensure regional and global coverage? Distributed core data centers and edge acceleration points-op-presence (POPs) are often necessary to ensure low latency levels for global and round trips.
- How will you handle message routing? To deliver messages with the fewest amount of network hops, your system will need intelligent routing.
- How will you deal with fluctuating message payloads? The size of a message and the number of messages will often fluctuate. If a new score during a football game, for example, feeds into an update that includes other data about the ongoing game, your system will need to be able to handle that payload.
- How will you ensure efficient encoding? Latency can increase if you have inefficient encoding processes because these processes can slow down the system’s ability to translate data into a transmittable format and back again.
Building a realtime updates system that can maintain low enough latency over time and at scale requires a lot of time and skill. Even companies that can afford the investment or have experienced engineers assigned to the work often prefer to buy a solution because outsourcing this investment enables them to focus that work on other features.
The Ably team is deeply familiar with every challenge and every edge case. We believe in the value of realtime updates but we also know – from experience – that building a system to support realtime updates can be more resource-intensive than most companies can predict or handle. That’s why we’ve done it for you.
This is the second in a series of four blog posts about the three fundamentals for building realtime updates in sports, media, and entertainment apps. In other posts, we look at why data integrity and elasticity are so important when you're trying to deliver realtime updates to end users at scale.
