Articles
Video Calling and Group Effects
Creating a Group Effect with the Multipeer API

Creating a Group Effect with the Multipeer API

With the multipeer capability you can create group effects that can communicate with other instances of the same effect running in a group video call.

In a script this is implemented using the MultipeerModule API, which provides the ability to create message channels that effects can send JSON formatted messages to.

The effects can also listen out for messages that other instances have sent to a given message channel, enabling data to be passed between them to create coordinated group experiences.

In this tutorial, you'll learn how to implement multipeer functionality using the API, by creating an effect where video call participants can change the other participants’ backgrounds by tapping the screen.

The unfinished project

Download the unfinished project to follow along.

In the example project, you’ll find a number of assets already set up, including a canvas group, materials and imported textures.

Additionally, in the Patch Editor you’ll find a patch graph which sends a tap event to a script and another which handles the changing of the user’s background.

The script.js asset already contains some code which is used to track the current background image and to subscribe to the event sent from the Patch Editor.

// Load in the required modules
const Patches = require('Patches');
const Diagnostics = require('Diagnostics');
 
(async function () { // Enable async/await in JS [part 1]
 
 // Initialize background tracking variables
 const totalBackgroundCount = 3;
 var backgroundIndex = 0;
 
 // Get the tap event from the Patch Editor
 const screenTapPulse = await Patches.outputs.getPulse('screenTapPulse');
 
 // Subscribe to the screen tap event
 screenTapPulse.subscribe(() => {
 
 });
 
})(); // Enable async/await in JS [part 2]
      

Message sender and receiver logic

When a call participant opens a group effect it's automatically applied to all other participants on the call, though each participant will be running their own instance.

To communicate between the various instances of the effect, we’ll implement logic that allows them to both send messages to, and receive them from, the other instances.

This way, each instance is notified of any state changes and can be updated accordingly.

For the example project we'll be following the logic below, which describes what will happen every time a screen tap occurs on any of the running instances.

For each screen tap interaction, the participant that taps on the screen is classed as the message sender, while the other participants are classed as the message receivers.


Message sender

  1. Determine the next background that needs to be displayed. The background selection should wrap around the array, for example: background 1 > background 2 > background 3 > background 1.
  2. Change the background of the participant that tapped on the screen.
  3. Broadcast a message to all other instances of the effect on a given message channel, notifying them that the background needs to be changed.


Messages are broadcast to all peers except the instance that the message was broadcast from - an effect can’t broadcast a message to itself. Step 2 updates the sender's local state and is an important part of ensuring a unified experience across the various instances.

Message receiver

  1. Listen out for messages sent to a given message channel.
  2. Read the contents of the message received to determine the new background value.
  3. Change the receiver's background to the new value.

Now that we have an idea about the logic required, we can begin to implement the functionality in a script.

Script setup

Open the script.js file in a text editor.

On line 4, import the MultipeerModule into the script with the following line of code so that you can make use of the multipeer API.

const Multipeer = require('Multipeer');
        

Creating a message channel

We need to create a message channel where messages can be sent to and retrieved from.

The multipeer API supports the use of multiple message channels but we'll only be creating one in this example.

On line 14, add the following lines of code:

// Create a message channel
const backgroundIndexChannel = Multipeer.getMessageChannel('backgroundIndexTopic');
        

The getMessageChannel() method takes a single argument, the name of the message channel to create. With the line above we’ve created the BackgroundIndexTopic message channel.

We’ll need the channel name to send and retrieve messages later on in our code.



If the method is called with a name that already exists, the existing channel is retrieved and no new channel is created.

Updating the background image index

Before we send a message to the message channel, we need to update the variable keeping track of the active background image and send its value to the Patch Editor.

We’ll do this within the tap event callback function that was already set up in the script.

Starting on line 20 copy and paste the code snippet below.

// Increment the background index to show the next background graphic
backgroundIndex++;
 
// If the index value is equal to the total number of background images,
// reset the value so that the displayed graphic loops.
if (backgroundIndex >= totalBackgroundCount) {
  backgroundIndex = 0;
}
 
// Send the new background index value to the Patch Editor, so that it can be
// used to update the background graphic displayed
Patches.inputs.setScalar('msg_background', backgroundIndex);
        

Sending messages to a message channel

To send a message to a message channel you can call the sendMessage() method exposed by MessageChannel objects.

Paste the code snippet below immediately after the previous snippet, on line 33. Make sure to add it before the callback function’s closing curly bracket (the ‘}’).

// Broadcast the new background index value to other participants on the
// previously created backgroundIndexChannel message channel
backgroundIndexChannel.sendMessage({'background': backgroundIndex }, true).catch(err => {
    
  // If there was an error sending the message, log it to the console
  Diagnostics.log(err);
});
        

The method’s first argument is a JSON formatted object containing a name and a value to send to the channel. In our case, we’re sending the backgroundIndex variable to the channel backgroundIndexChannel under the name background.

The second argument, true, refers to whether the message should be sent using the Real-Time Channel messaging backend, which has a high bandwidth for message streams but doesn’t guarantee a synchronized message order across participants.

If false had been passed, the Sync Channel backend would have been used instead, which synchronizes message order across participants but has a message rate limit of 75 messages every 30 seconds.

The catch() method and subsequent console log is there to notify us if anything goes wrong when sending the message.

Listening out for messages sent to a message channel

Now that we’ve implemented the message sending logic, we also need to make sure that we’re able to listen out for messages broadcast to the message channel from other participants.

To do so, we can subscribe to the EventSource returned by the onMessage property exposed by MessageChannel objects.

The event’s callback function contains a single argument, the JSON object broadcast to the message channel.

Within the callback function we can access the data by the name it was assigned in the sendMessage method call. In our example, it’s background.

In the callback function we’ll also need to send the new background value to the Patch Editor.

Copy and paste the following code starting on line 44:

// Listen out for messages sent to the syncBGChannel
backgroundIndexChannel.onMessage.subscribe((msg) => {

  // Retrieve the 'background' attribute from the JSON object received
  backgroundIndex = msg.background;
          
  // Send the value to the Patch Editor
  Patches.inputs.setScalar('msg_background', msg.background);
});
        

With that, our script is complete.

The finished script

// Load in the required modules
const Patches = require('Patches');
const Diagnostics = require('Diagnostics');
const Multipeer = require('Multipeer');
 
(async function () { // Enable async/await in JS [part 1]
 
  // Initialize background tracking variables
  const totalBackgroundCount = 3;
  var backgroundIndex = 0;
 
  // Get the tap event from the Patch Editor
  const screenTapPulse = await Patches.outputs.getPulse('screenTapPulse');
 
  // Create a message channel
  const backgroundIndexChannel = Multipeer.getMessageChannel('BackgroundIndexTopic');
 
  // Subscribe to the screen tap event
  screenTapPulse.subscribe(() => {
 
    // Increment the background index to show the next background image
    backgroundIndex++;
 
    // If the index value is equal to the total number of background images,
    // reset the value so that the displayed image loops.
    if (backgroundIndex >= totalBackgroundCount) {
      backgroundIndex = 0;
    }
 
    // Send the new background index value to the Patch Editor, so that it can be
    // used to update the background image displayed. This only updates the
    // background for the participant that tapped on the screen
    Patches.inputs.setScalar('msg_background', backgroundIndex);
 
    // Broadcast the new background index value to other participants on the
    // previously created backgroundIndexChannel message channel
    backgroundIndexChannel.sendMessage({'background': backgroundIndex }, true).catch(err => {
 
      // If there was an error sending the message, log it to the console
      Diagnostics.log(err);
    });
 
  });
 
  // Listen out for messages sent to the backgroundIndexChannel
  backgroundIndexChannel.onMessage.subscribe((msg) => {
 
    // Retrieve the 'background' attribute from the JSON object received
    backgroundIndex = msg.background;
 
    // Send the value to the Patch Editor
    Patches.inputs.setScalar('msg_background', msg.background);
  });
 
})(); // Enable async/await in JS [part 2]
		        

The changing of each participants' backgrounds is handled by the Patch Editor, using the backgroundIndex variable sent to it from the script:

FAQ

What's the best way to test my group effect?

When building group effects, it's important to test regularly to ensure everything works as intended. The Meta Spark Extension for Visual Studio Code provides a built-in tool that allows you to debug effects that use the multipeer capability.

  1. Open Meta Spark Studio.
  2. Enter the email address or phone number associated with your Facebook account.
  3. Enter the password for your Facebook account and click Log In.

You can also click the Login with Facebook button to log in with your Facebook account.

How do I use the multipeer capability in a block?

You can use the MultipeerModule API or patches in a block as you would in the main part of your project. The only difference is the scoping:

  • A message sent in a block instance can only be received by the same instance on the other side. A message sent from a block can’t be received by the main effect.
  • If the block is instantiated dyamically, make sure it's instantiated with an unique multipeerId:
Example:
        Blocks.instantiate('mp_block', {name: "block1"}, {multipeerId: 'block1'})