Articles
Video Calling and Group Effects
Synchronizing Data Across Participants with the State API

Synchronizing Data Across Participants with the State API

Complex group effects often require each participant to keep track of a global value, such as the turn order in a turn-based experience.

In the Creating Turn-Based Experiences with the Participants API example project this was implemented using the message channel functionality provided by the MultipeerModule API.

By sending messages back and forth between participants on these channels, we ensured that each participant maintained a synchronized turn order and background image index (to determine which background image to display).

Four participants are seen in Spark AR Player.

In this article, rather than adding functionality to the project, we'll use the State API to simplify this process.

We'll use the API to define global variables which automatically synchronize their values across the various running instances of the effect, helping to maintain a global state.

The unfinished project

Download the unfinished project to follow along.

In the example project, you’ll find a number of assets already set up, including a canvas group, materials and imported textures.

In the Patch Editor you’ll find a patch graph which sends tap and tap and hold events to a script and a second graph which handles the changing of the user’s background and the visibility of the turn indicator.

The script.js file contains a stripped down version of the Creating Turn-Based Experiences with the Participants API code, with the various message channel functionalities removed.

Adding the State API script package

In order to make use of the State API we first need to import the package from the Asset Library.

In the Assets panel click +, highlight Script and select Search AR Library.

In the pop-up window, find and select the spark-state package, then click Import Free.

You’ll see the new script package appear in the Assets panel:

We’ll also need to add the Scripting Writeable Signal Source capability from the Capabilities tab within the project’s Properties.

Before we can use the State API we need to import the package in our script.

Open the script.js file and paste the line of code below onto line 6.

const State = require('spark-state');
        

Replacing the background message channel

The previous version of the project used the backgroundIndexChannel message channel to notify participants that the background should be changed based on another participant’s screen tap.

With the State API, what we’ll do instead is define a new global variable backgroundIndex that will automatically update its value across all effect instances whenever it’s changed.

Add the following code to line 12:

// Define a new global scalar signal for the background index
const backgroundIndex = await State.createGlobalScalarSignal(0, 'backgroundIndex');
        

Now, instead of listening out for messages sent to a message channel, we can simply monitor the value of the global signal and send it to the Patch Editor.

However, one thing to keep in mind when converting functionality previously implemented via message channels to the synchronized State API, is that you may need to change where you make certain method calls.

For example, code that we may have previously included within a message channel’s onMessage subscription callback won’t necessarily run exactly the same when added to a global signal’s monitor subscription callback.

This is because the code within the signal's monitor callback doesn’t run every time that the set function is called, but rather only when the value of the signal actually changes.

On line 91 paste the code below:

// Monitor our global background signal
backgroundIndex.monitor({fireOnInitialValue: true}).subscribe((event) => {
  
  // Send the value to the Patch Editor
  Patches.inputs.setScalar('msg_background', backgroundIndex.pinLastValue());
});
        

In the previous version of the project we sent a message to the background index message channel within the screenTapPulse’s callback function, but with a global variable we can just set its value directly with the set function. We set fireOnInitialValue to true, because we want to get the latest value upon joining the effect.

Paste the following on line 80:

// If it's currently my turn
if (activeParticipants[turnIndex.pinLastValue()].id === self.id) {
  // Increment the background index to show the next background image
  let currentBackgroundIndex = (backgroundIndex.pinLastValue() + 1) % totalBackgroundCount;
 
  // Set the global variable to the new value
  backgroundIndex.set(currentBackgroundIndex);
}
        

We don’t need to add any additional checks as the value is guaranteed to be identical across all of the participants in the effect.

Replacing the turn index message channel

We’ll use the same method as in the previous step to ensure that the participants have a synchronized turn value.

First, we’ll create a new global variable by adding the following code to line 15.

// Define a new global scalar signal for the turn index
const turnIndex = await State.createGlobalScalarSignal(0, 'turnIndex');
        

Then, we’ll monitor the value of turnIndex. Paste the code below on line 107.

// Whenever the turn index changes, update the local turn index
turnIndex.monitor({fireOnInitialValue: true}).subscribe((event) => {
 
  // Check whether this participant needs to show the turn indicator graphic
  setTurnIndicatorVisibility();
});
        

Lastly, we can set the value of the global variable, this time from within the screenTapHoldPulse subscription’s callback function.

Add the following code to line 97:

// If it's currently my turn
if (activeParticipants[turnIndex.pinLastValue()].id === self.id) {
 
  // Increment the turn index to pass the turn over to the next participant
  let currentTurnIndex = (turnIndex.pinLastValue() + 1) % activeParticipants.length;
 
  // Set the global variable to the new value
  turnIndex.set(currentTurnIndex);
}
        

Updating the participant handler

Now that the message channels have been replaced, all we need to do is to make some updates to the onUserEnterOrLeave function.

Since the GlobalCounterSignal variables we defined are wrappers for ScalarSignal objects, whenever we want to access their most recent value we need to explicitly call pinLastValue.

This means we need to replace line 158 with:

let currentTurnParticipant = activeParticipants[turnIndex.pinLastValue()];
        

Later on in the function we perform a check to see whether the turn index value needs to be updated or not.

Just before this check, we’ll create a reference to the global signal’s most recent value by pasting the following code on line 192.

// Create a reference to the most recent turn index value
let currentTurnIndex = turnIndex.pinLastValue();
        

We’ll use this new variable to create an updated version of the previously mentioned check.

Replace lines 195 to 206 with the code below:

// Check if the participant whose turn it was is still in the effect
if (activeParticipants.includes(currentTurnParticipant)) {
 
  // If they are, change the turnIndex value to that participant's new index value
  currentTurnIndex = activeParticipants.indexOf(currentTurnParticipant);
 
} else {
 
  // If they're not in the effect and they were the last participant in the turn order,
  // wrap the turnIndex value back to the first participant in the turn order
  if(currentTurnIndex >= activeParticipants.length) {
    currentTurnIndex = 0;
  }
}
        

Lastly, to avoid any potential conflicts with multiple participants attempting to set the value of our global turnIndex variable, we’ll make it so that only the participant whose turn it currently is can set the value.

We can do this by adding the if statement below to line 210.

// Ensure only the participant whose turn it is sets the value of the global signal
if(activeParticipants[currentTurnIndex].id === self.id) {
 
  // Update the global turn index value
  turnIndex.set(currentTurnIndex);
}
        

The final thing we need to do in the project is to update a line within the setTurnIndicatorVisibility function due to the previously mentioned requirement for accessing a signal’s value.

Replace line 141 with the line of code below:

let isMyTurn = activeParticipants[turnIndex.pinLastValue()].id === self.id;
        

While the State API is useful when you need to synchronize data between participants, the global variables shouldn't be updated more than a few times each second, per participant.

With that, our improved group effect is complete. Take a look at the completed script below.


The finished script

// Load in the required modules
const Patches = require('Patches');
const Diagnostics = require('Diagnostics');
const Multipeer = require('Multipeer');
const Participants = require('Participants');
const State = require('spark-state');

(async function () { // Enable async/await in JS [part 1]

  // Initialize background count constant
  const totalBackgroundCount = 3;

  // Define a new global scalar signal for the background index
  const backgroundIndex = await State.createGlobalScalarSignal(0, 'backgroundIndex');

  // Define a new global scalar signal for the turn index
  const turnIndex = await State.createGlobalScalarSignal(0, 'turnIndex');

  // Get the tap event from the Patch Editor
  const screenTapPulse = await Patches.outputs.getPulse('screenTapPulse');

  // Get the tap and hold event from the Patch Editor
  const screenTapHoldPulse = await Patches.outputs.getPulse('screenTapHoldPulse');

  // Get the other call participants
  const participants = await Participants.getAllOtherParticipants();

  // Get the current participant, 'self'
  const self = await Participants.self;

  // Push 'self' to the array, since the previous method only fetched
  // other participants
  participants.push(self);

  // Get other participants active in the effect
  const activeParticipants = await Participants.getOtherParticipantsInSameEffect();

  // Push 'self' to the array, since the previous method only fetched
  // other participants
  activeParticipants.push(self);

  // Get each participant in the participant list
  participants.forEach(function(participant) {

    // Monitor each participant's isActiveInSameEffect status
    // The use of subscribeWithSnapshot here allows us to capture the participant who
    // triggered the event (ie enters or leaves the call) inside of the callback
    participant.isActiveInSameEffect.monitor().subscribeWithSnapshot({
      userIndex: participants.indexOf(participant),
    }, function(event, snapshot) {

      // Pass the participant and their active status to the custom function
      onUserEnterOrLeave(snapshot.userIndex, event.newValue);
    });
  });

  // Monitor when a new participant joins
  Participants.onOtherParticipantAdded().subscribe(function(participant) {

    // Add them to the main participant list
    participants.push(participant);

    // Monitor their isActiveInSameEffect status
    participant.isActiveInSameEffect.monitor({fireOnInitialValue: true}).subscribeWithSnapshot({
      userIndex: participants.indexOf(participant),
    }, function(event, snapshot) {

      // Pass the participant and their isActiveInSameEffect status to the custom function
      onUserEnterOrLeave(snapshot.userIndex, event.newValue);
    });
  });

  // Do an initial sort of the active participants when the effect starts
  sortActiveParticipantList();

  // Do an initial check of whether this participant should display the
  // turn indicator
  setTurnIndicatorVisibility();

  // Subscribe to the screen tap event
  screenTapPulse.subscribe(() => {

    // If it's currently my turn
    if (activeParticipants[turnIndex.pinLastValue()].id === self.id) {
      // Increment the background index to show the next background image
      let currentBackgroundIndex = (backgroundIndex.pinLastValue() + 1) % totalBackgroundCount;

      // Set the global variable to the new value
      backgroundIndex.set(currentBackgroundIndex);
    }

  });

  // Subscribe to the tap and hold event
  screenTapHoldPulse.subscribe(function() {

    // If it's currently my turn
    if (activeParticipants[turnIndex.pinLastValue()].id === self.id) {

      // Increment the turn index to pass the turn over to the next participant
      let currentTurnIndex = (turnIndex.pinLastValue() + 1) % activeParticipants.length;

      // Set the global variable to the new value
      turnIndex.set(currentTurnIndex);
    }

  });

  // Monitor our global background signal
  backgroundIndex.monitor({fireOnInitialValue: true}).subscribe((event) => {

    // Send the value to the Patch Editor
    Patches.inputs.setScalar('msg_background', backgroundIndex.pinLastValue());
  });

  // Whenever the turn index changes, update the local turn index
  turnIndex.monitor({fireOnInitialValue: true}).subscribe((event) => {

    // Check whether this participant needs to show the turn indicator graphic
    setTurnIndicatorVisibility();
  });

  // Sorts the active participant list by participant ID
  // This ensures all participants maintain an identical turn order
  function sortActiveParticipantList(isActive) {

    activeParticipants.sort(function(a, b){
      if (a.id < b.id) {
        return -1;

      } else if (a.id > b.id){
        return 1;
      }
    });
  }

  // Sets the visibility of the turn indicator graphic
  function setTurnIndicatorVisibility() {
    // Check whether this participant's ID matches the ID of the current
    // participant in the turn order and store the result
    let isMyTurn = activeParticipants[turnIndex.pinLastValue()].id === self.id;

    // Send the previous value to the Patch Editor. If the IDs match,
    // the patch graph will display the turn indicator, otherwise the
    // graphic will be hidden
    Patches.inputs.setBoolean('showTurnPanel', isMyTurn);
  }

  // Sorts the active participant list and restarts the turn sequence
  // when there's a change in the participant list.
  // If a user joined, isActive will be true. Otherwise it will be false
  function onUserEnterOrLeave(userIndex, isActive) {

    // Get the participant that triggered the change in the participant list
    let participant = participants[userIndex];

    // Store a reference to the participant before any changes to the list are made
    let currentTurnParticipant = activeParticipants[turnIndex.pinLastValue()];

    // Check if the participant exists in the activeParticipants list
    let activeParticipantCheck = activeParticipants.find(activeParticipant => {
      return activeParticipant.id === participant.id
    });

    if (isActive) {

      // If the participant is found in the active participants list
      if (activeParticipantCheck === undefined) {

        // Add the participant to the active participants list
        activeParticipants.push(participant);

        Diagnostics.log("User joined the effect");
      }
    } else {

      // If the participant is not found in the active participants list
      if (activeParticipantCheck !== undefined) {

        // Update the active participants list with the new participant
        let activeIndex = activeParticipants.indexOf(activeParticipantCheck);

        activeParticipants.splice(activeIndex, 1);

        Diagnostics.log("User left the effect");
      }
    }

    // Sort the active participant list again
    sortActiveParticipantList();

    // Create a reference to the most recent turn index value
    let currentTurnIndex = turnIndex.pinLastValue();

    // Check if the participant whose turn it was is still in the effect
    if (activeParticipants.includes(currentTurnParticipant)) {

      // If they are, change the turnIndex value to that participant's new index value
      currentTurnIndex = activeParticipants.indexOf(currentTurnParticipant);

    } else {

      // If they're not in the effect and they were the last participant in the turn order,
      // wrap the turnIndex value back to the first participant in the turn order
      if(currentTurnIndex >= activeParticipants.length) {
        currentTurnIndex = 0;
      }
    }
     // We only want one participant to send the message to everyone
     if(activeParticipants[currentTurnIndex].id === self.id) {
       turnIndex.set(currentTurnIndex);
     }


    // Check which participant should display the turn graphic
    setTurnIndicatorVisibility();
  }

})(); // Enable async/await in JS [part 2]
						

Creator days workshops

In this advanced group effect workshop, an expert creator demonstrates how to script with the Multipeer API, State API and the Participant API, plus how to record a group effect.

Rewatch the session to learn:

  • What an ambient video calling experience is.
  • How to use the multipeer capability to synchronize data across participants (State API).
  • How to tell a story without making your participants interact with the scene.
  • Writing a basic SparkSL shader.
  • Setting up SparkSL shader parameters that can pull in values from the Patch Editor.
  • How to send patch graph values through patch bridging to a SparkSL shader params with the Option Picker.