Scripting Your First Multipeer Effect

Scripting Your First Multipeer Effect

The MultipeerModule API provides the ability for an effect running in a video call to communicate with other instances of the effect in the same call.

Without multipeer functionality, AR effects in a video call are completely independent of one another. By using the capability, effects become more group oriented, better enabling shared experiences.

In this article, you'll learn about the important parts required to build a multipeer effect, and how to write your first MultipeerModule script.

To do this, you'll create an effect where each of the call participants are segmented against the same background. When any one of the participants taps on the screen, everyone’s background will be changed.

Download the example project here (Spark AR Studio v125 or above required). While the project requires some additional patches, their implementation won't be covered here.

Message sender and receiver logic

Communication between the separate instances of a multipeer enabled effect is achieved via the broadcasting and receiving of JSON formatted messages. Through these messages the instances are able to communicate state changes to one another and unify the experience.

When a call participant opens a multipeer effect, it's automatically applied to all other participants on the call; each participant will be running their own instance of the same effect.

This means that the effect needs to contain logic for both sending and receiving messages.

For this example project we'll be following the logic below, which we'll implement in a script at a later step. For every tap interaction, the participant that taps on the screen is classed as the message sender, while the other participants are classed as the receivers.

Message sender

  1. Determine the next background that needs to be displayed. The background selection should wrap around the array, for example: background 1 > background 2 > background 3 > background 1.
  2. Change the background of the participant that tapped on the screen.
  3. Broadcast a message to all other instances of the effect on a given message channel, notifying them that the background needs to be changed.

Messages are broadcast to all peers except the instance that the message was broadcast from - an effect can’t broadcast a message to itself. Step 2 updates the sender's local state and is an important part of ensuring a unified experience across the various instances.

Message receiver

  1. Listen out for messages sent on a given message channel.
  2. Read the contents of the JSON message received to determine the new background value.
  3. Change the receiver's background to the new value.

Now that we have an idea about the logic required, let’s jump into the code.

The multipeer script

First, we need to import the MultipeerModule into our script so that we can make use of the capability. We can write this at the top of the script file, next to other module import statements.

const Multipeer = require('Multipeer');

We need to create a message channel where messages can be sent to and retrieved from. Multipeer effects support the use of multiple message channels but we'll only be using one in the example.

In the getMessageChannel() method call we pass in SyncBGTopic as the parameter, which represents the name or topic of the message channel. This same channel name will be used to send and retrieve messages in our code.

const syncBGChannel = Multipeer.getMessageChannel('SyncBGTopic');

Next we'll add the code to send a message on the message channel we created in the previous step.

// Send message          
// 1. Determine the new background state
if (backgroundIndex >= totalBackgroundCount) {
  backgroundIndex = 0;

// 2. Set the new background state for the participant that tapped
// on the screen (by informing the Patch Editor)
// This updates the message sender's local state
Patches.inputs.setScalar("msg_background", backgroundIndex);

// 3. Broadcast a message to all other participants about the new
// background state
syncBGChannel.sendMessage({'background': backgroundIndex}, true).catch(err => {

Step 2 sends the backgroundIndex value to the Patch Editor:

As with the logic we laid out in the first section of the article, step 3 is where the message is actually sent to the message channel.

To break it down, the snippet of code in step 3 is saying: on the syncBGChannel, send a JSON message that contains the new background index, and use the Real-Time Channel messaging backend.

The Real-Time Channel (RTC) parameter, which we set to true, determines whether the message should be sent using the RTC backend. This channel has high bandwidth for message streams but doesn't guarantee synchronized message order across all participants.

If false had been passed, the Sync Channel would be used, which synchronizes message order across participants but has a message rate limit of 75 messages every 30 seconds.

The .catch(... is there to let you know if anything went wrong when sending the message.

Lastly we can add the code for the receiver logic, which changes a participant's background when a message is received.

// Receive message
// 1. Subscribe to messages sent on the 'syncBGChannel' topic
syncBGChannel.onMessage.subscribe((msg) => {
  // 2. Read the 'background' attribute of the 'msg' JSON data object received
  backgroundIndex = msg.background;
  // 3. Change the message receiver's background to the index specified, 
  // via the Patch Editor
  Patches.inputs.setScalar("msg_background", backgroundIndex);

To receive messages on a channel, we subscribe to the EventSource returned by the channel's onMessage property.

The msg parameter that’s passed into the event callback is the JSON object we created in our send message code snippet. Any data added to the JSON object in the send message code, can be accessed in the receive message code.

Within the project, the changing of the participants' backgrounds is handled by the Patch Editor, using the backgroundIndex variable sent to it from the script:

A patch graph containing three patches.

While the example project contains additional patches and capabilities, the above code snippets are all that are required to create your first multipeer script.

As you go on to build new multipeer effects, it's important to test regularly to ensure everything works as intended. The Spark AR Extension for Visual Studio Code provides a built-in tool that allows you to debug multipeer effects.

Was this article helpful?