Hardly anyone is stricter about data integrity than people who closely follow the news – even if they don’t use the term.
If a user is monitoring a live feed of updates about a legal case or a bill in progress, they will switch feeds without hesitation if updates are repeated or out of order. In these contexts, which occur all over the media, sports, and entertainment industries, users are looking for a live experience and even relatively minor data integrity issues spoil the value a realtime feature is supposed to offer.
In a previous article, we explained the three fundamentals that make for a successful realtime update system: low latency, data integrity, and elasticity. Here, we’re going to focus on data integrity and explain why it’s such an important part of an effective realtime experience.
What do we mean by data integrity?
Data integrity refers to the overall accuracy, completeness, and consistency of data. For realtime update systems to maintain data integrity, they need to be able to preserve data as it travels from the backend to the frontend to the end-user’s device.
Data integrity is important in many other contexts but it’s never as essential as it is for features offering realtime updates.
For example, if Tik Tok notifies a user that a few users have either commented on or dueted their video, the user likely won’t notice if the notifications are out of order.
Even in contexts where users likely will notice – if, for example, Netflix reminds users a few times that a new movie is out – the user might be annoyed but the experience will remain functional.
But for realtime systems, data integrity is essential and users can become more than frustrated even for relatively smaller data integrity issues.
Why is data integrity essential for sports, media, and entertainment apps with realtime updates features?
If a Eurovision fan is relying on a realtime update feed, out-of-order data could mean missing the performance they were looking forward to.
If a football fan is relying on realtime draft updates, incorrect data could confuse them and make their fantasy football plans hard to figure out.
If a blogger is updating their readers on an ongoing event, incomplete data can make it impossible for them to help their readers understand what’s going on.
Despite the casual nature of some of these activities, the standards are strict and users might churn based on a single data integrity flaw.
Scalability and reliability – both of which have to rise to similarly high expectations – make data integrity even harder to ensure. A truly seamless realtime update feature needs to work on any device and via any network connection, anywhere in the world.
Again, this isn’t the result of unfair expectations but the result of the nature of a realtime experience. An Instagram user can be flexible if they’re using a flakey network but if a user is expecting realtime updates on an ongoing event, the experience fundamentally fails if the realtime updates system can’t maintain data integrity.
How sports, media, and entertainment apps can guarantee data integrity for realtime updates
Sports, media, and entertainment companies building apps with realtime update features need to carefully evaluate the investment necessary for building a realtime updates system. That evaluation is a necessary first step before deciding whether to build or buy.
In terms of data integrity, a realtime system will only succeed given a few factors:
- Realtime updates systems need to be able to ensure that all subscribers receive messages in the same order that they were originally published, no matter where they are in the world or the quality of their connection.
- Messages can’t be lost and messages can’t be delivered multiple times. All messages need to be delivered precisely once.
- If users disconnect and reconnect, they still need to receive the correct messages in the correct order.
Data integrity needs to be near perfect despite a wide variety of complicating factors. But for realtime updates, that’s just one challenge among many. These further challenges, including scalability and reliability, compound. If you’re copying data across different data centers to ensure greater resiliency, for example, there’s a higher likelihood of redundant messages.
At Ably, we’ve seen all these challenges. We’ve built a realtime infrastructure that incorporates everything we’ve seen so that our customers benefit from this experience without having to do the work alone.
To learn more, read about how our data broadcast solution supports realtime updates and find out how companies like BlueJeans and Metra use Ably.
This is the third in a series of four blog posts about the three fundamentals for building realtime updates in sports, media, and entertainment apps. In other posts, we look at why low latency and elasticity are so important when you're trying to deliver realtime updates to end users at scale.
