Myth-busting: Jamstack can't handle dynamic content

Introduction

Jamstack has brought forward a great way to rethink the infrastructure of modern-day websites. It shows us just how much we can abstract away in the process of serving websites and, as a result, gain tremendous benefits to User and Developer Experience.

However, much confusion exists around what kind of websites can actually fall under this classification. The whole premise of Jamstack apps is based on the fact that these sites can be served directly from a CDN (Content Delivery Network), without needing an origin server. You might ask: “So these are static sites then? That means only pre-rendered content with nothing dynamic?” Well, that’s untrue and is one of the biggest myths around Jamstack.

In this article, we’ll understand everything about Jamstack sites with dynamic content and specifically look at a realtime synced streaming application we built (dubbed as a live watch party app) to show off the wonderful benefits of Jamstack and the APIs around us, enabling us to elevate its possibilities. This app allows you and your friends to watch a video on your respective machines, synchronously, while chatting alongside - much like the Netflix watch party.

What is the Jamstack?

Before we take a stab at explaining anything here, we highly recommend watching this video where Phil Hawksworth of Netlify takes us through a beautiful explanation of the Jamstack and why it is great.

We’ve copied over one of his slides directly from the talk:

The JAM in Jamstack stands for JavaScript, APIs, and Markup - pretty much everything we’ve been using already in most of our web apps.

So, what’s different?

It’s the way these apps are architected and served to users across the globe.

As you see in the slide from Phil’s talk - for a traditional website that is dynamically served by a web server, the journey involves a few steps at the least. Your web browser goes to the CDN to get any static assets, then to the load balancer placed in front of the web servers capable of serving that particular site. The load balancer resolves which of the available web servers is best equipped to serve the page. The selected web server then serves the page back to the user following the same path. In some cases, the web server might request some data from the database before serving back the page to the client.

In contrast to having all these components between the user and the page they want to see, Jamstack proposes serving statically generated websites directly from the CDN, doing away with the need for an origin server to serve the site. This can be a little bit confusing. To clarify - this doesn’t mean that we can’t have a server at all, we could have one for the app logic, but this server won’t be responsible for serving our HTML page back to the user. In fact, the best option here would be to make use of the myriad of serverless platform options available out there, to avoid managing any infrastructure in the backend.

Although the initial site that’s loaded from the CDN is static, containing pre-rendered assets and data, we can immediately enhance the experience and functionality by retrieving dynamic data via API calls to our own server or any third-party endpoints.

This results in many benefits, most obvious of which are improved performance and better user and developer experience.

A bit more on static sites with dynamic/ realtime data

There is a huge assumption that static sites mean static data. The static assets served by Jamstack projects can contain JavaScript files; after all the “j” in Jamstack represents JavaScript. Just as JavaScript brought dynamic data to websites in the 90s, it can still do the same today. We can use the static JavaScript files in our Jamstack projects to interact with out webpages and provide dynamic experiences for our end users - hook that up with a pub/sub or real-time infrastructure service like Ably, and we have dynamic data on the Jamstack very easily.

Let’s understand with an example

For this project, we've been working closely with Daniel Phiri and the Strapi team. It all started a couple of weeks ago when we started to build a realtime Jamstack app in public for the dev community to follow along:

So excited about this project @malgamves and I are working on! A live watch party app to host dedicated sync stream parties with your friends. We'll update this thread as we make progress #BuildingInPublic pic.twitter.com/5LnnVNzcSR

— Srushtika Neelakantam (@Srushtika) April 16, 2021

The watch party app allows a host to select a video from the library and watch it synchronously with their friends anywhere on the globe.

To give you a good idea, the host would follow these steps:

1. Enter their username, create a private watch party room, and share an invite link with friends.

2.  Select a video from the library to watch along with friends.

3.  Watch the synchronized video with friends, share live comments, and see who’s currently online.

The host gets to control the video playback -  if they play, the video starts playing for everyone else, same for pause, seek, and so on. If the host leaves, that’s the end of the party.

Tech stack for our watch party app

To build out this application, we’ve leveraged four pieces of technology - Nuxt.js, Strapi, Ably and Netlify. Let’s get into what each does for us in this project.

What is Nuxt.js?

Nuxt.js is a web framework based on Vue.js - besides being one of the most convenient ways to build out Vue applications, it gives us the option to build server-side rendered or static websites. For our project, we’ve gone with the static option and hosted these files on Netlify.

What is Strapi?

Strapi is an open source Node.js-based headless CMS. Since it's headless, we can use Strapi to deliver content to almost any digital device via its API. We’ve used Strapi as a video content manager. We’ve managed the videos available in our Nuxt.js app with Strapi, as well as leveraged its extra customisation options to build out a service to handle token requests with Ably.

What is Ably?

Ably is a realtime messaging infrastructure as a service solution. It allows you to enable publish/subscribe-based messaging in your application with just a few lines of code. Ably provides low-latency messaging, high reliability, and is able to work globally on any platform or device. It completely abstracts away the complex problem of scaling realtime communication across multiple regions, from the developer space.

What is Netlify?

Netlify is a cloud hosting company that offers hosting and serverless backend services for web applications and static sites. It is particularly known for bringing the Jamstack approach of building websites to the attention of developers around the world.

App architecture - bringing all these technologies together

Jamstack sort of forces you to have a simplified architecture and infrastructure for your web app. For the watch party, the static site itself (that is just the initial page that allows hosts to create private watch party rooms) is hosted on Netlify’s CDN.

1. Both the admin version and non admin version of the site can be retrieved directly from the CDN (based on URL routing).
2. Ably’s Pub/Sub platform requires you to authenticate before you can use the service. There are two options for this - either embed the API key directly into the front-end web app (which would be a bad idea because anyone can steal it), or use Token authentication by requesting an auth server to help the front-end clients to authenticate securely. We’ll use Strapi as our auth server (in addition to its beautiful CMS capabilities which we’ll touch upon soon).
3. After we’ve received an Ably Token Request back from Strapi, we can send it to Ably to securely authenticate with the service and initialize the SDK. This sets up a persistent realtime connection with Ably, allowing any new updates to be pushed directly to our app and vice versa. We’ll use this to synchronize the video stream, as well as to share comments and live online status of participants.
4. After the host has authenticated with Ably (and transparently with Strapi via dummy user credentials), they’ll be able to share an invite link with any participants they’d like to invite to their private watch party.
5. Next, the host will be able to request the video library from the Strapi CMS. This will show them a grid of various videos to choose from. After they’ve chosen a video, the unique reference code for that video will be immediately published to all the participant apps via Ably. These non-admin participants can then (behind the scenes) request the particular video resource directly from the Strapi CMS.
6. On this final screen, everyone will be able to add live comments and it is up to the host to play the video, pause it, seek it to a certain timestamp etc - all of which would be synchronized with the rest of the viewers.

Code snippets and explanations

Let’s understand the main components of the app.

Creating a video collection and adding content to Strapi

After getting started with your Strapi app, a browser tab will open and take you to the Admin Panel. Create a user and log in. Once that's done, we can start building out the content schema to store our videos. Once you’re in, navigate to Content-Types Builder under Plugins in the left-hand menu.

• Click the "+ Create new collection type" link.
• Name it videos and click Continue.
• Add a Text field (short text) and name it title.
• Click the "+ Add another field" button.
• Add another Text field (long text) and name it description.
• Click the "+ Add another field" button.
• Add a Media field and name it video.
• Click the "+ Add another field" button.
• Add another Media field and name it thumbnail.
• Click the Save button and wait for Strapi to restart.

Everything should look like this once done:

These fields will store the video details for your pages. Now we can go on and add content to them by clicking Videos on the left menu. Adding content should look something like this:

Hosting Strapi

Strapi is a self-hosted headless CMS. With that in mind you have an array of deployment options, most of which are laid out nicely in Strapi’s Deployment Documentation.

For our project, we deployed our Strapi App to Heroku following the guide provided. We’d recommend this approach if you want a free hosting provider that lets you provision a Postgres database for your Strapi app with minimal effort.

Retrieving info from Strapi into our app

Before we can make requests to our Strapi server, we need to make sure that we have the right permissions set up to get them. To do this, we go to your User Permissions > Roles > Public Role > click the find and find one checkbox under videos, as well as the auth checkbox under ably-auth as shown below.

For the project we’re using the Strapi GraphQL API, so we’ll have to install it with yarn strapi install graphql - you have the option to use the REST API and the Strapi Nuxt module too.

Once the GraphQL plugin is installed, we can go to `http://localhost:1337/graphql` to access our GraphQL Playground and play around with different GraphQL operations.

Authenticating Ably via Strapi

Ably’s realtime messaging service expects client devices to be authenticated before they can start using the service. This can be done in two ways - either by using the API key directly in the front-end app (Basic Authentication strategy) or via tokens issued by an auth server (Token Authentication strategy).

As you might have guessed, embedding the API key wouldn’t be a wise choice because it can be easily misused. To implement Token Auth strategy, we’ll need to have a backend service use a direct API Key securely on our behalf and generate a valid token request via Ably. The frontend client can then use this token request to authenticate with Ably without leaking the API Key details. You can learn more about it in a short YouTube video.

For our app, since we are already using Strapi as a CMS for our data, we’ll also make use of it as a backend service generating our token request.

We took advantage of Strapi custom controllers and built out the logic to generate a token request for Ably.  This is shown below:

'use strict';

const Ably = require('ably/promises');
const ABLY_API_KEY = process.env.ABLY_API_KEY;

const realtime = Ably.Realtime({
    key: ABLY_API_KEY,
    echoMessages: false
});


module.exports = {
    async auth(ctx) {
        const clientId =  'id-' + Math.random().toString(36).substr(2, 16)
        const tokenParams = { clientId };
        try {
            const ablyThing = await realtime.auth.createTokenRequest(tokenParams);
            console.log(ablyThing)
            return ablyThing
        }
        catch (err) {
            return ctx.badRequest("Daas not good!!")
        }
    }
}; 

Pub/Sub messaging with Ably

Ably’s core offering is a scalable realtime messaging that follows the Publish/Subscribe pattern.

Go to the dashboard, where we can divide realtime data into different categories, based on what the data represents and which participants are concerned with that data. These different categories of data can be published on different ‘channels’ within the app. Here’s an example:

const ably = new Ably.Realtime(<auth endpoint or api key>);
const channel = ably.channels.get(‘jamstack-news’);

// Publish a message to the jamstack-news channel
channel.publish('greeting', 'hello');

// Subscribe to messages on jamstack-news channel
channel.subscribe('greeting', function(message) {
  alert(message.data);
});

In the watch party app, we’ve used the following channels:

1. mainParty: used mainly to share presence data (this is explained below in this article).
2. video: used to share updates related to the video player, including play, pause and seek events, along with the current timestamp.
3. comments: used to share live comments between participants of the specific watch party.

Unique channels

Given that we use the same app to allow different groups of people to spin up their own breakout room, we also need to think about a way to separate out the realtime messages for each of those rooms. To do this, we assign a unique random code to each watch party room and use that to uniquely identify channels in the same Ably app. Given that different channels can have different participants and the data from one channel doesn’t go into another, unless explicitly published, this should be a good way for us to separate out concerns.

Another option is to use channel namespaces. These are useful when we want to apply certain features or restrictions to a set of channels as a whole. As we won’t be needing that for this app, we’ve just gone with the channel names to be watch-party-<random-room-code>, video-<random-room-code> and comments-<random-room-code>.

Utilizing Nuxt’s central store for efficient fan out of messages to required components

We’ve made use of the VueX store, which comes built into Nuxt. You can find this in store/index.js. This file works as a central store for most of the data in our static site. A typical VueX store contains four objects (possibly more depending on your specific app) - state, getters, mutations and actions.

State: This is a single object containing the application level state which represents the single source of truth and allows different components to be in sync with each other.

Getters: Getters are methods that allow us to compute derived states to be used anywhere in the app.

Mutations: Mutations are methods that change the value of a certain state object. Mutations should always be synchronous - this is to ensure that we have a good view of the state changes. If you need to update the state based on an asynchronous operation, you’d use actions described next.

Actions: You’d use actions to perform asynchronous operations and call a mutation when ready to change the state as a result of that async operation.

This central store is especially useful for the watch party app, because we have various channels, the async data from which is being used in different components. And because VueJS is reactive, our components can watch for changes on any of the variables and react to them immediately with UI updates.

The key things to notice in the store for our project are listed below:

1. The currentVideoStatus state object:

 currentVideoStatus: {
        isVideoChosen: false,
        didStartPlayingVideo: false,
        chosenVideoRef: null,
        currentTime: null,
        isPlaying: false,
        isPaused: false
      },

This is a single source of information about the video being played. For the host, this is always in sync with their video player. We publish this object whenever a new non host participant joins. This is also the object published when an existing participant clicks on the ‘force sync with admin’ button.

2.  The instantiateAbly() method:

In this method, we instantiate Ably using Token authentication. As described previously, Token authentication is facilitated by a Strapi endpoint. So, in the init method, we pass in the url of that endpoint as a value to the authUrl object. We receive a client id when the connection is successful, which we then save in a local state object.

const ablyInstance = new Ably.Realtime({
  authUrl: this.$config.API_URL + "/auth-ably"
});

3.  The attachToAblyChannels() method:

In this method, we attach to the three channels. Note that we add the unique room code to these channel names to make sure they are uniquely identified for this watch party room, across the app.

    attachToAblyChannels(vueContext, isAdmin) {
       //mainPartyChannel
       const mainParty = this.state.ablyRealtimeInstance.channels.get(
         this.state.channelNames.mainParty +
           "-" +
           this.state.watchPartyRoomCode
       );
	// similarly for the video and comments channels

4.  The subscribeToChannels() method:

In this method, we subscribe to the channels we previously attached to. When a new update is published on that channel, the respective callback method will be triggered. We simply update the state variables to contain the latest message that has arrived.

state.channelInstances.comments.subscribe(msg => {
  state.channelMessages.commentsChMsg = msg;
});

5.  The publishCurrentVideoStatus() method:

This method enables the admin to publish the currentVideoStatus object we described previously.

state.channelInstances.video.publish(
  updateEvent,
  this.state.currentVideoStatus
);

6.  The requestInitialVideoStatus() method:

This method is used by non admin participants to request the latest video status. This is invoked once at the beginning when they’ve just joined, then again whenever they click on the `force sync` button.

requestInitialVideoStatus({ state }) {
       state.channelInstances.video.publish(
         "general-status-request",
         "request"
       );
       },

7.  The publishMyCommentToAbly() method:

This method publishes the user's comments. This will be displayed in the list of comments next to the video player.

publishMyCommentToAbly({ state }, commentMsg) {
       state.channelInstances.comments.publish("comment", {
         username: state.username,
         content: commentMsg
       });
     },

The utility methods are self explanatory but the rest of the methods are described in the next section.

Presence

Presence is an Ably feature that you can use to subscribe to realtime changes to a device or client’s online status (aka their connection status). Presence allows us to see who is currently online in the watch party room. This information is displayed in a tab next to the live comments. A live counter of the number of people online is also displayed above the video player for a quick look.

Here’s some explanation of the presence related methods in the store:

1. The getExistingAblyPresenceSet() method

Apart from a live subscription to ongoing presence updates, we also need a list of people who were already there when a user joins. In this method, we perform an API request to get the existing presence set.

this.state.channelInstances.mainParty.presence.get((err, members) => {....});

2.  The subscribeToAblyPresence() method:

In this method we set up a subscription to presence on the main party channel and invoke various methods to handle new people joining or existing people leaving.

this.state.channelInstances.mainParty.presence.subscribe("enter", msg => {....});
       this.state.channelInstances.mainParty.presence.subscribe("leave", msg => {....));

3.  The handleNewMemberEntered() and handleExistingMemberLeft() methods:

In these methods we update our local array with the latest presence set information and also update our local counters reflecting the aggregate number of people present in the watch party at any given time.

4.  The enterClientInAblyPresenceSet() method:

In this method, we make the current client enter the presence set on the main party channel. This will publish an update to everyone else who is subscribed to the presence set and also include this user in the global presence set.

History and Rewind

Given that Ably is a pub/sub messaging service at its core, almost all of the messaging is transient. While Ably doesn’t store messages in the long term, it does provide storage options up to a certain extent. For example, you saw in the previous sections we were able to retrieve the presence set via an API call. Similar to that, even for regular messages on regular channels, Ably offers two ways to retrieve previously published messages:

• History - an explicit API call to request previously published messages.
• Rewind - allows you to get some previously published messages at the time of attaching and subscribing to a channel for realtime messages.

We can use rewind on the comments channel so that all the participants are able to see the comments published even before they join the watch party. With rewind, we can either specify a time period or number to indicate how many previously published messages we’d like to retrieve.

Hosting static sites with Netlify

As mentioned above when we introduced the service, we’ll deploy our watch-party app to Netlify!

To start out, create a Netlify account and make sure your project source code is hosted on GitHub. Click “Create new site from Git” and connect your GitHub to Netlify. Select your repo and fill in the details. Under Basic Build Settings, your build command should be yarn generate, and your publish directory should be dist. Select Advanced Settings and define your environment variables, add API_URL to key, and replace <Strapi App URL> with the URL of your deployed Strapi app.  

It’s worth noting that should you have both your Strapi app and watch-party apps in a monorepo configuration (both apps in the same Git repository) like our repository, then you need to add a base directory as shown below. These settings are available in Site Settings under Build & Deploy.

Should you have any trouble, you can reference the Nuxt documentation on deploying to Netlify.

Add-on options via Webhooks to further enhance our app

In terms of your product-specific custom architecture, you may want to add other components (such as a database), maybe trigger a cloud function to perform some computation, or even stream messages to a third-party service. Ably provides easy ways to integrate with external APIs and services via webhooks, serverless functions, message queues, or event streaming. You can also use incoming webhooks to trigger a message on an Ably channel from an external service. (Think of a scenario where you allow participants to answer your quiz via SMS messages!)

Conclusion

We’ve built a realtime Jamstack app and busted the myth. Jamstack CAN handle dynamic content.  Jamstack is a great concept and works well if applied correctly.

I hope this article has given you a good view into realtime Jamstack apps, and got you quickly up and running with Strapi and Ably. It has been great to collaborate with Daniel on this project! We'll also be doing a webinar together: Realtime data on the Jamstack with Ably and Strapi, where we'll talk about the watch-party app and do a live Q&A.

You can check out the watch party yourself at: https://jamstack-watch-party.ably.dev/. Have feedback or want to exchange ideas? You can always find me on Twitter: @Srushtika. Happy to any questions too, my DMs are open!