Ably v1.0 client libraries released

Today we officially announce the release of our v1.0 Realtime and REST client libraries. We now recommend that all customers consider upgrading to take advantage of the improved API, new functionality and lots of refinements in the underlying protocols.

Please see the upgrade notes at the bottom of this post if you want a quick summary of what you need to consider when upgrading to v1.0.

What’s new in v1.0?

Version 1.0 is largely an incremental improvement across the board on v0.8, our previous stable production-ready release, and does bring some breaking changes. The improvements are grouped into four key areas: improved channel and connection state management, inline realtime authorization, access to bleeding edge features and various small fixes and optimizations.

Better connection and channel state management
Prior to v1.0, if it was not possible to provide continuity of service on a channel (typically as a result of a connection being disconnected for a long period of time and thus old messages becoming unrecoverable), the client libraries would indicate this loss by forcibly detaching the affected channel. According to our documentation, it was then up to developers to decide how to recover from that situation by, for example, reattaching and then recovering prior messages with our history API.

The overwhelming feedback we had in regards to this v0.8 design was that that behaviour was not intuitive and developers assumed our client libraries would handle reattachment of channels automatically for them. Users indicated that if they were interested in loss of continuity events, then they would like to be able to subscribe to these events but would prefer channels within client libraries, by default, do their best to provide continuity.

As such, the v1.0 client libraries have changed as follows:

• New suspended state — channels now additionally have a suspended state. A channel enters this state when its connection has become suspended (i.e. disconnected for more than two minutes and thus the connection state and continuity of messaging lost). Channels in a suspended state will automatically be reattached by the client library when the connection becomes connected again in the future. See all channel states.
• New ChannelStateChange object — this object is now emitted for each channel state change or event and contains the current state, previous state, emitted event and a boolean resumed attribute. Developers wishing to know if continuity on a channel has been lost should subscribe to channel state changes and inspect the value of the resumed attribute on ATTACHED or UPDATE events. An ATTACHED event for a newly attached channel will always have the resumed attribute set to false as this is a new channel attachment. If however you get an ATTACHED event with resumed set to false for a non-newly-attached channel (e.g. a channel that had become SUSPENDED due to lack of internet connection), this indicates that the channel has been automatically re-attached, but without continuity from the last time it was attached. (i.e. some messages may have been lost). In the case of an UPDATE event being emitted with resumed set to false, this indicates that the Ably service has been unable to provide continuity for reasons such as a message backlog building up that the client cannot keep up with.
• A ChannelEvent is now emitted instead of a ChannelState for channel state changes. A ChannelEvent is simply a superset of the ChannelState enum with the addition of UDPATE which is used to indicate that the channel’s internal state has been updated yet the channel remains ATTACHED.
• A ConnectionEvent is now emitted instead of a ConnectionState for connection state changes. Much like the ChannelEvent object, a ConnectionEvent object is simply a superset of the ConnectionState enum with the addition of UPDATE which is used to indicate that the connection’s internal state has been updated yet the connection remains CONNECTED.
• Presence re-enter — if a channel moves to the SUSPENDED state due to the connection being disconnected for too long, then the last known presence state for the members entered by current client are stored locally. When the connection is later resumed, and the channel is automatically reattached, the client will also ensure that the local presence state for the client is synchronized with the Ably service. So if for example a client entered presence on the channel, the connection was then lost and was unable to resumed within 2 minutes, but later resumed, the client library would automatically reattach the channel and then re-enter the local client on the channels it was present on prior to the disconnection. Please note that the presence state is synchronized as opposed to re-submitted meaning that a enter or update event will only emitted if the Ably service no longer has that member present. It is possible in spite of being disconnected that a member will remain present for at least 15 seconds and thus synchronization is necessary to avoid unnecessary emission of presence events.

Authorization improvements

• Inline Authorization — in v0.8 of our client libraries, when a realtime connection was established to the Ably service using token authentication, the connection would remain open until the token it was using expired. Once expired, the connection was briefly disconnected, the client library would obtain a new token, and would resume the connection to Ably. This ensured that there was no loss of continuity (i.e. the connection was resumed), however it did result in a brief period of up to a few seconds where messages were being queued client-side and server-side whilst the connection was being reestablished.
  In v1.0, authorization is now performed inline ensuring a connection can remain connected indefinitely whilst using tokens that expire. The Ably service now notifies a realtime client when a token is nearing expiry. Clients that receive this notification will immediately obtain a new token and send this new token to Ably using the established connection. Assuming the new token is compatible and valid, the connection remains open and the inline token renewal process will start again once that token nears expiry.
• authorize always gets a token — Previously in v0.8 calling authorize would only reauthorize if a token did not exist or was expired or the force option was set to true. In v1.0, calling authorize will always result in a new authorization process and obtain a new token. This change was made to simplify the API and remove any ambiguity about what authorize does.
• authorize arguments are not merged — when calling authorize with a TokenParams and/or AuthOptions argument, unless the arguments are empty (null values in most languages), the attributes of those objects will no longer be merged with existing defaults but will instead replace the defaults completely with the provided TokenParams or AuthOptions respectively. For example, in Javascript, calling authorize({ ttl: 60000 }) will set the default TTL to 60s. A subsequent call to authorize({ capability: {“*”:[“*”]} }) results in a request for a token with the specified capability, yet the TTL becomes unspecified and the Ably service default will be used.
• Retries for failure to authorize — in v0.8 of the client libraries, if a realtime connection needed to reauthorize and failed to do so, the connection would move to the FAILED state which is a terminal state. In v1.0, the connection will move to the DISCONNECTED or SUSPENDED state ensuring the connection will automatically reattempt to authorize and connect in the future.

Access to bleeding edge features
Ably has a number of undocumented features available in our REST API, including for example, an API to enumerate active channels in an app. When we make these features available to select customers or more broadly in beta, we have to date required customers to access the raw Ably REST API. The problem with this approach is that our customers would then have to re-engineer all the functionality we provide in our client libraries such as JSON and MsgPack (binary) encoding and decoding, token and key based authentication, host fallback capabilities to work around network issues etc.

v1.0 brings a new request method to both the Realtime and REST client libraries that provides a convenient wrapper around the Ably REST API that includes automatic encoding and decoding, pagination, authentication etc.

Smart push notification ready
Messages published to the Ably service now support an extras field. This field is currently reserved for our smart push notifications, however in future this field will allow additional metadata to be included in messages that can assist with routing, filtering, grouping etc.

Other smaller changes

• Implicit attach failures — when relying an implicit attach for a channel such as subscribing to a channel that is not yet attached, the behaviour in v0.8 was that if the attach failed, the subscribe listener too would not be subscribed. We believe this is the wrong behaviour as subscribing to a channel is an operation on the client itself and is not dependent on the channel state. For example, you can explicitly attach a channel, subscribe to messages, detach the channel and reattach it, and the subscribe listeners will continue to work as the registering subscribe listeners is an operation that effects the client library state. Therefore, from v1.0 onwards, an implicit attach triggered by a request to subscribe will have no bearing on whether the subscribe listener is registered or not.
• Enterprise host or dedicated cluster configuration — we have introduced a number of additional options for customers with dedicated clusters or endpoints that allows more flexibility when configuring fallback hosts.
• Improved heartbeats — we improved how heartbeats are sent and received in our Ably protocol. This aims to reduce the overhead in heartbeats both for clients and servers as well as improve how quickly clients and servers can detect connections that are no longer active yet are still reported by TCP/IP as alive.
• Message and PresenceMessage fromEncoded* — customers that process messages from our Reactor WebHooks, Queues or Firehose, will receive these messages in an encoded and/or encrypted format when the payload requires this. The Ably client libraries automatically look after encoding and decoding when messages are published or received, however customers who process messages via the Reactor will likely be receiving messages directly via HTTP WebHooks or via AMQP for example. As a result, messages will be received in their raw format.
  The Ably client libraries provide methods to decode a single message using fromEncoded or arrays of messages using fromEncodedArray for both Message and PresenceMessage objects. If you are using the Ably Reactor, we recommend you look at the section “Decoding..” within the Presence Webhook and Message Webhook documentation for further information and examples using these methods.
• TokenDetails and TokenRequest fromJson constructor — in order to minimize bugs related to serialization of TokenDetails or TokenRequest objects by servers, and then subsequent deserialization by clients as part of the auth process (typically when using the authCallback ClientOptions attribute), we now provide a fromJson constructor for the TokenDetails and TokenRequest objects that accepts a JSON string or JSON-like object and creates an object from those attributes. We recommend that customers use this method when explicitly constructing these objects from JSON passed from their servers.
• New stats fields — stats API requests now include the new fields failed and refused for MessageCount objects, and MessageTraffic includes sharedQueue and externalQueue attributes for Reactor Queues and Reactor Firehose stats respectively.

Ably Javascript and Node specific changes

• Loading the Javascript library from the Ably CDN without a major version number is now deprecated. This is to ensure that Ably can moving forwards introduce breaking changes without breaking customers apps. For example, if a customer had referenced https://cdn.ably.io/lib/ably.min.js in the past, they would have got v0.8.*. When we released v1.0 with some breaking changes, that customer would have automatically been upgraded and as a result their app may have been broken. Instead, customers must now lock into major versions or major minor versions with CDN URLs such as https://cdn.ably.io/lib/ably.min-1.js or https://cdn.ably.io/lib/ably.min-1.0.js. This will ensure that customers lock into all v1 or v1.0 releases respectively.
• The Node.js library now defaults to using the binary protocol. However, the browser library still uses the JSON text protocol by default.
• As a convenience, the EventEmitter used to emit events for message listeners and state changes now supports arrays everywhere, so you can do e.g. channel.presence.subscribe([‘enter’, ‘update’], …) or channel.on([‘attached’, ‘update’], …) etc.
• In node, the client library assumes WebSockets is the most reliable transport and attempts to connect with that transport immediately as opposed to starting with comet and upgrading to WebSocket (as it does in a browser). To get the old upgrading behaviour back, specify transports: ['comet', 'web_socket'] in the ClientOptions.
• The Javascript library now supports ReactNative, NativeScript and additionally includes a TypeScript definition.

Background to v0.8 numbering

Our previous stable production version 0.8 was often misunderstood to be not production-ready because of the prefixed zero. This was in fact not the case and 0.8 is stable and will continue to be supported moving forwards. However, as v1.0 brings a small number of breaking changes, according to the semver versioning scheme we follow, this requires a bump to the major version number.

Diving deeper into the changes

For those developers who are interesting in looking under the hood, you will see that upgrading our client libraries from v0.8 to v1.0 was no small undertaking. Our client library specification for v0.8 has 591 requirements, each of which needs to have at least one set of tests to confirm adherence to the specification. The client library v1.0 specification has added almost one hundred additional requirements bringing the total to 683 requirements.

Caveats

• The .NET and Go libraries are still on v0.8. We are actively working to upgrade the .NET library to v1.0, but at present, there are no plans to upgrade the Go library unless there is more interest from customers.

Upgrade to v1.0 cheatsheet

Please bear in mind the following potentially breaking changes when upgrading to v1.0. Note that the detail behind each of these points is covered above in this article.

• State changes that used to emit ChannelState and ConnectionState now emit ChannelEvent and ConnectionEvent. In most languages, this has no impact as they are compatible.
• Channels have a new SUSPENDED state
• Channels will automatically reattach when they become SUSPENDED or DETACHED, and presence state will automatically be synchronized once reattached. You no longer have to manager your channel or presence state manually.
• Explicit calls to authorize will trigger authentication immediately and any arguments passed in will no longer be merged but will instead replace existing defaults.
• Loading the Javascript library from https://cdn.ably.io/lib/ably.min.js is deprecated. You must now lock into a major or minor version.