Learn Spark AR Studio

Everything you need to know to create an interactive augmented reality experience.

Clear
Back

Making Objects Apppear in Response to Face Gestures

In Spark AR Studio, you can create effects that respond to all kinds of facial movements and gestures. For example a happy face or an open mouth. You can use either the Patch Editor, or scripting.

Learn about:

You could use other types of interaction patches to control visibility, like screen interactions

Making an object appear in response to facial movement

In this example, we'll make an object appear when a happy face is detected.

Add an object to your scene

To try this out you can use your own 3D object, choose one from the AR Library, or use the plane object included in Spark AR Studio.

To add an object:

  1. Click Add Object.
  2. Choose an object from the list, or import your own.

Creating a patch to represent the visibility of an object

To do this:

  1. Select the object in the Scene tab.
  2. Click the arrow next to Visible in the Inspector panel.

This will open the Patch Editor, and create a patch:

Creating patches that detect facial movement

To detect the presence and movement of a face in your effect, you'll need to add a face tracker to your scene:

  1. Click Add Object.
  2. Select Face Tracker.

Next create a patch to represent the face tracker, by dragging it into the Patch Editor.

Any patches that detect specific facial movements or gestures will need to be connected to this face tracker patch.

To create a patch that detects a specific gesture or interaction, right-click in the Patch Editor. You'll see a menu listing all kinds of patches - they're named to reflect what they do.

Under Interaction, you'll see lots of patches that detect facial movement and gestures - from Mouth Open, to Left Eye Closed, to Kissing Face.



To follow along with this guide, select a Smile patch from the menu.



Connect:

  1. The Face output in the facetracker patch to the Face input port in the Smile patch.
  2. The output of the Smile patch to the patch that represents the Visible property of your object.

Like many interaction patches, The Smile patch outputs a boolean signal. Boolean signals are either on or off. So if the person uses the effect smiles, the signal will be switched on, and the object will be visible in the effect.

Making different objects appear in response to different gestures

If you want to show different objects in response to different gestures, all you need to do is:

  1. Create patches to represent the Visible property of each object.
  2. Create interaction patches for the gestures you want to detect.

Connect the interaction patches to the output of the face tracker, and the output of each interaction patch the input of the patch representing the object you want to show in the scene.

Making different objects appear in response to the same gesture

You can use the Counter patch to count through a series of options each time the same interaction is detected.

Combining the Counter patch with the Equals Exactly patch will tell Spark AR Studio to show a certain option, depending on the number of interactions counted.

Right-click in the Patch Editor, and select:

  1. A Counter patch.
  2. A Pulse patch.
  3. Three Equals Exactly patches.

Connect the patches, so your graph looks like this:



You'll need to make a couple of edits to the values in the patches.

Make sure the Maximum Count in the Counter patch matches the number of options you want to count through. In this example, it should be set to 3.

Edit the value in the Equals Exactly patches. Each should show a number between 1 and 3, depending on which order you want them to show in the scene.

Join the Spark AR Creator's community

Find inspiration, see examples, get support, and share your work with a network of creators.

Join Community

Frequently asked questions

Have a specific question? Maybe it's been answered.


Read FAQs
Was this document helpful?