For some time now, I’ve been trying to combine Fable and WebRTC in my daily work. In this article I am going to describe the process of implementing a signaling mechanism for WebRTC.
What is WebRTC?
But first, what is WebRTC? WebRTC is a browser technology that allows exchanging data among N peers via either the UDP or TCP protocol. In this exchange peers talk to each other without a server in the middle (except when Relay connection is used). WebRTC is a native API available in all modern browsers. One of the things that are not ready by default is the signaling process. That's what this article is about.
WebRTC signaling
Before we go into details of the implementation, let’s get some basic concepts around signaling straight. To establish a connection between two peers they need to exchange messages with SDP (Session Description Protocol) information. The number of those messages depends on:
- the number of STUN/TURN server
- type of connection (P2P/Relay)
SDP contains a lot of information needed to start a connection. Messages to be exchanged are called Offer, Answer, and Candidate. We will use a Trickle-Ice mechanism. It means that each Offer, Answer, and Candidate will be sent separately in an asynchronous (async) manner. A Candidate has information about how to connect to a peer which creates them (more about them here). There is also a simpler way of establishing a connection where Answer and Offer would contain every needed Candidate. But I want to focus on an async version — we are using it in our product because of better performance.
Implementation
Switching messages as described above are named "signaling". When I was looking for a out-of-the-box solution for signaling I came across Ably. I decided to give it a try for my base application, in which I want to send a file from one peer to another.
When I was reading the Ably documentation, I realized that the only functions that I will need would be publish and subscribe. These are used in a message channel context, so I created an Ably interface in F# which is a port of JS interfaces from their official lib. To get started, you will need an Ably account.
module AblyMapping =
type IMessage =
abstract name: string
abstract data: string
type IChannel =
abstract publish: string * string -> unit
abstract subscribe: string * (IMessage -> unit) -> unit
type IChannels =
abstract ``get``: string -> IChannel
type IRealTime =
abstract channels: IChannels
type IAbly =
abstract Realtime: (string) -> IRealTime
let Ably: IAbly = importAll "Ably"
Changes in packages.json.
{
…
"ably": "^1.2.4"
}
(The code above is self-explanatory when we look at the JS example from the Ably site.)
var ably = new Ably.Realtime('apikey');
var channel = ably.channels.get('gut-tub');
// Publish a message to the gut-tub channel
channel.publish('greeting', 'hello');
var ably = new Ably.Realtime('apikey');
var channel = ably.channels.get('gut-tub');
// Subscribe to messages on channel
channel.subscribe('greeting', function(message) {
alert(message.data);
});
Initialization of Ably can be found in the init method.
let init (): Model * Cmd<Msg> =
let channel =
AblyMapping.Ably.Realtime “api-key”
|> fun x -> x.channels.get “channel name"
let model =
{
Role = None
Channel = channel
ConnectionState = WebRTC.ConnectionState.NotConnected
WaitingCandidates = []
}
model, Cmd.none
At this point, we could start sending and receiving messages via Ably, but I want to start by first installing the WebRTC-adapter library. This library gives me confidence that what I want to achieve with the native API would work the same way in every browser. Since there are some slight differences between implementations, I add the following line to the packages.json with the webrtc-adapter library.
{
…
"webrtc-adapter": "^7.7.0"
}
Usage in F# code.
module WebRTC =
let private adapter: obj = importAll "webrtc-adapter"
The above ensures the interfaces are compatible across all supported browsers. Now we can switch to the code responsible for handling WebRTC. The code snippet is big because we need to make a distinction between the sender and the receiver of a file. This is because we don’t have an application on the server-side. I started with the creation of a PeerConnection object with a communication server address.
let create () =
let conf =
[| RTCIceServer.Create( [| "turn:location:443" |], "user", "pass", RTCIceCredentialType.Password ) |]
|> RTCConfiguration.Create
|> fun x ->
x.iceTransportPolicy <- Some RTCIceTransportPolicy.Relay
x
let pc = RTCPeerConnection.Create(conf)
{ Peer = pc; Channel = None }
I test everything locally, so I force using a Relay connection. Otherwise, communication would be done via local network. That would result in skipping the signaling phase (host candidates would be used).
Note: Using public STUN/TURN servers is not recommended. We are not sure what the configuration looks like or who owns them.
We create a LifecycleModel object which contains a PeerConnection and RTCDataChannel inside, and then create an instance of it as a mutable field instead of part of the Elmish model. This is the simplest approach: the default Thoth serializer couldn’t handle serializing PeerConnection; also passing the whole big object wouldn’t be something that we want to do.
type LifecycleModel = {
Peer: RTCPeerConnection
Channel: RTCDataChannel option
}
...
let mutable peer: LifecycleModel = Unchecked.defaultof<_>
...
peer <- WebRTC.create ()
After the creation of the PeerConnection object, we see that in the UI we have a possibility to connect as sender or receiver.
This differentiates the different ways that WebRTC works, and shows a different value for a Role field in a model. When we look at how the sender logic is implemented we start with the creation of RTCDataChannel, which would then be used as a transfer channel between users. The creation of a channel is located in the createChannel function.
let createChannel pc name =
let channel = pc.Peer.createDataChannel name
{ pc with Channel = Some channel}
Moving on to the configuration of messages sent and received via Ably, when we are a sender we want to listen to Answer and Candidate(from the receiver) messages. You can see how to achieve this in the following code snippet.
let subs =
[
"answer", fun (msg: AblyMapping.IMessage) -> dispatch (Msg.WebRTC (WebRTC.Msg.Answer msg.data))
"receiver-candidate", fun (msg: AblyMapping.IMessage) -> dispatch (Msg.WebRTC (WebRTC.Msg.Candidate msg.data))
] |> Map.ofList
The combination with the Ably channel is visible in the init method from the Signaling module.
let init (channel: IChannel) (subscribers: Map<string, IMessage -> unit>) =
subscribers
|> Map.iter (fun subscriberKey subscriberFunc ->
channel.subscribe (subscriberKey, subscriberFunc)
)
channel
We thus subscribe to all messages with the given keys answer and receiver-candidate. When they occur we propagate Elmish messages that would be handled in the update method.
In the receiver scenario the difference is that we don’t create RTCDataChannel (this is why it is marked as option in LifecycleModel). We gather it when the connection would be in a connecting phase. If it were created on its own we would receive an invalid state error. This is why we only subscribe to messages Offer and Candidate (from Sender).
When a subscription to Ably messages is ready we send an Elmish message Signaling with a ready channel which is updated in the application model. Meanwhile the WebRTC configuration is updated with callbacks to functions that need to be handled in a connection/data transfer process.
Assignment of those callbacks and how they are handled is visible in a subscribe function. This should do the following.
- Initialize
RTCDataChannelthrough which data (file) would be transferred.
let private initReceiverChannel (pc: RTCPeerConnection) msgHandler dispatch =
let mutable updatedChannel: RTCDataChannel = Unchecked.defaultof<_>
let callback (ev: RTCDataChannelEvent) =
let receiveChannel = subscribeChannel ev.channel msgHandler dispatch
internalLog (sprintf "updating channel: %A" ev.channel.id)
peer <- { peer with Channel = Some receiveChannel }
pc.ondatachannel <- callback
updatedChannel
let updatedChannel =
if role = Role.Sender then
match channel with
| Some c ->
internalLog (sprintf "initialize channel for: %A" role)
subscribeChannel c msgHandler dispatch
| None -> failwith "Channel is not initilized for sender"
else
internalLog (sprintf "initialize channel2 for: %A" role)
initReceiverChannel pc msgHandler dispatch
- Handle connection state (only for diagnostic purposes).
pc.oniceconnectionstatechange <-
fun _ ->
internalLog (sprintf "Connection state changed to: %A" pc.iceConnectionState)
- Handle exchange of candidates.
pc.onicecandidate <-
fun e ->
match e.candidate with
| None -> internalLog "Trickle ICE Completed"
| Some cand ->
cand.toJSON()
|> Base64.``to``
|> Signaling.WebRTCCandidate.Candidate
|>
if role = Role.Sender then
Signaling.Notification.SenderCandidate
else Signaling.Notification.ReceiverCandidate
|> onSignal
Now the configuration of WebRTC is ready on the sender and receiver side.
We can initiate a connection by clicking the Connect button, after which the init method from the WebRTC module is called.
let init connection onSignal =
let pc, channel = connection.Peer, connection.Channel
pc.createOffer().``then``(fun desc ->
pc.setLocalDescription (desc) |> Promise.start
if isNull desc.sdp then
internalLog "Local description is empty"
else
desc
|> Base64.``to``
|> Signaling.WebRTCOffer.Offer
|> Signaling.Notification.Offer
|> onSignal)
|> Promise.catch (sprintf "On negotation needed return error: %A" >> internalLog)
|> Promise.start
{ Peer = pc
Channel = channel }
We send Offer via the Ably channel and set it as LocalDescription via setLocalDescription method on the PeerConnection object. Right now the application flow is not discernible from the code at first glance. The other side of communication should receive Offer via the Ably channel which would then be propagated as an Offer Elmish message and handled in the update method.
let setOffer (lifecycle: LifecycleModel) onSignal remote =
try
let desc = rtcSessionDescription remote
internalLog (sprintf "setting offer: %A" desc)
lifecycle.Peer.setRemoteDescription desc
|> Promise.catch (sprintf "Failed to set remote description: %A" >> internalLog)
|> Promise.start
lifecycle.Peer.createAnswer().``then``(fun desc ->
lifecycle.Peer.setLocalDescription (desc) |> Promise.start
if isNull desc.sdp then
internalLog "Local description is empty"
else
internalLog (sprintf "sending answer: %A" desc)
desc
|> Base64.``to``
|> Signaling.WebRTCAnswer.Answer
|> Signaling.Notification.Answer
|> onSignal)
|> Promise.catch (sprintf "On negotation needed errored with: %A" >> internalLog)
|> Promise.start
with e ->
internalLog (sprintf "Error occured while adding remote description: %A" e)
This should set Offer from a sender as a RemoteDescription on a receiver side and, in case of success, it should generate Answer. It then gets sent to the sender via Ably. The important thing to mention here is that Candidates are generated after the creation of Offer/Answer. They shouldn’t be set on the PeerConnection object before setting Local and Remote descriptions. Because of this, the Elmish model has a buffer for Candidates that could be received before setting Remote/Local descriptions.
Buffer handling:
if model.WaitingCandidates.Length > 0 then
model.WaitingCandidates
|> List.iter (WebRTC.setCandidate peer )
{ model with
ConnectionState = WebRTC.ConnectionState.Connecting
WaitingCandidates = [] }, Cmd.none
else
{ model with ConnectionState = WebRTC.ConnectionState.Connecting }, Cmd.none
Handling a message that contains Candidates looks like this.
if model.ConnectionState <> WebRTC.ConnectionState.NotConnected then
WebRTC.setCandidate peer candidate
model, Cmd.none
else
{ model with WaitingCandidates = candidate::model.WaitingCandidates }, Cmd.none
ConnectionState is set when Offer or Answer are received or when the DataChannel is open.
Handling Answer.
| WebRTC (WebRTC.Msg.Answer answer) ->
WebRTC.setAnswer peer answer
{ model with ConnectionState = WebRTC.ConnectionState.Connecting }, Cmd.none
On to the setAnswer implementation.
let setAnswer (lifecycle: LifecycleModel) remote =
try
let desc = rtcSessionDescription remote
internalLog (sprintf "setting answer: %A" desc)
lifecycle.Peer.setRemoteDescription desc
|> Promise.catch (sprintf "Failed to set remote description: %A" >> internalLog)
|> Promise.start
with e ->
internalLog (sprintf "Error occured while adding remote description: %A" e)
Here we only set RemoteDescription on a PeerConnection object.
Right now, if everything works, we should have a WebRTC connection ready, with DataChannel open between our peers. We could move on to the code which is responsible for sending files.
In UI, there should be a textBox that reacts on a file drop.
This is handled as follows.
Field.div [
Field.IsGrouped ] [
Control.p [ Control.IsExpanded ] [
div [
Class "card border-primary"
Draggable true
Style [ Height "100px" ]
OnDragOver ( fun e ->
e.preventDefault()
)
OnDrop ( fun e ->
e.preventDefault()
if e.dataTransfer.files.length > 0 then
let file = e.dataTransfer.files.[0]
{ Name = file.name; Data = file.slice () }
|> SendFile
|> dispatch
)
] []
]
]
SendFile message handling.
| SendFile file ->
MessageTransfer.send peer (ChannelMessage.File file.Data)
model, Cmd.none
Method send in MessageTransfer.
let send (lifecycle: LifecycleModel) (msg: ChannelMessage) =
match msg, lifecycle.Channel with
| File blob, Some channel ->
blob
|> U4.Case2
|> channel.send
| _, _ -> log "MessageTransfer" (sprintf "Unable to process: %A channel is: %A" msg lifecycle.Channel)
On the receiver side onmessage handling on DataChannel object looks as follows.
channel.onmessage <-
fun e ->
internalLog (sprintf "received msg in channel: %A" e.data)
let blob = e.data :?> Blob
msgHandler blob
For simplicity, we assume that only a JavaScript blob would be sent via DataChannel. We also pass to there a msgHandler which is implemented in the following way.
let download (data: Blob) =
let url = Browser.Url.URL.createObjectURL data
let mutable anchor = document.createElement "a"
anchor.hidden <- true
anchor.setAttribute ("href", url)
anchor.setAttribute ("download", "image.png")
document.body.appendChild anchor |> ignore
anchor.click ()
document.body.removeChild anchor |> ignore
Its responsibility is to receive a FileBlob and immediately download it. We assume that the file would be always a png with the name image.
Conclusion
To sum up, thanks to Ably I was able to implement signaling in a simple way. In my daily work I am always seeking to achieve signalling in the most performant way, and when making a decision, I now have to seriously consider how Ably does it. During some initial tests to compare it with standard HTTP request/response mechanisms, it looks promising.
Another interesting thing is that a user can see how the messages/connections work and flow right on the Ably dashboard, which also includes information about used quota. That's a nice bonus.
Because there are no built-in signaling solutions, Ably is for sure a good out-of-the-box option to consider for scenarios such as what I outlined above. In my next articles, I hope to compare it with other ways to do signaling.
I hope you enjoy it. Thanks for reading!
Click here for the code repository.
WebRTC and .NET resources
- What is WebRTC?
- Solving the WebRTC signaling challenge
- WebRTC tutorial series
- Developing a realtime full stack app with .NET, Angular, and MongoDB
- Building dependable realtime apps with WebSockets and .NET
- What is pub/sub and how to apply it in C# .NET to build a chat app
Guest blog by Michał Niegrzybowski and originally published at his blog. Michał currently works for Datto as a senior software engineer, he is a Software Craftsman, big fan of functional programming, TDD, DDD, and new fancy libraries/frameworks. You can connect with him on Twitter or LinkedIn.
