Learn Spark AR Studio

Everything you need to know to create an interactive augmented reality experience.

Clear
Back

There may be differences between your version of Spark AR Studio and this tutorial because the product is currently in beta and we update it regularly.

Tutorial: Creating Effects that Respond to Facial Movement

In this guide, you'll learn how to use patches to build an effect that responds to facial movements.

We’re going to use the Patch Editor to:

  1. Detect facial movements.
  2. Trigger animation, in response to the movement.

As a result, two 3D objects that look like eyes will spin when the the person using the effect raises their eyebrows, like this:



Download the sample content to follow along.

Opening the Patch Editor

To get started, open the unfinished version in the sample content folder.

In the unfinished project you should already see a selection of 3D objects in the Scene Panel. In the Viewport and Simulator, you can see that these objects look like big bug eyes:



We’ve also used a blend shape to distort the shape of the face, and added a retouching effect. You can find out more about face distortion and retouching in this tutorial.

To open the Patch Editor:

  1. Click the workspace icon in the toolbar.
  2. Select Show/Hide Patch Editor.

The Patch Editor will open at the bottom of the screen:



Detecting facial movements

In this project we want the 3D objects to animate when the user’s eyebrows change position.

To detect this change, you'll start with a patch representing the face tracker. To create the patch:

  1. Select the face tracker in the Scene Panel.
  2. Drag it into the Patch Editor.

Three patches will be created:



We’ll use the 3rd patch - facetracker1. We’ll connect patches to the Face output port in this patch, to take data on the movements of the face.

To detect raised eyebrows, we'll use an interaction patch called Eyebrows Raised.

To create any interaction patch, click Add Patch at the bottom of the Patch Editor. You'll see a menu:



To find the Eyebrows Raised patch:

  1. Select Interaction.
  2. Select Eyebrows Raised.
  3. Click Insert Patch.

Patches that detect facial movements must be connected to a face tracker patch to work properly. So, connect the Face port in the facetracker0 patch to the Face port in the Eyebrows Raised patch.

You'll see a connection between the two patches:

Triggering the animation

The next step is add the patches that will trigger the animation when raised eyebrows are detected. You can use the Pulse patch to do this. To add it to your project:

  1. Click Add Patch.
  2. Select the Pulse patch from the menu.
  3. Click Insert Patch.

Connect the unnamed port on the Eyebrows Raised patch to the On/Off port on the Pulse patch.

Here's how your graph will look:



The Pulse patch will send a signal to any patches connected to it when the eyebrows are raised. In this project we'll connect it to an animation patch - to trigger an animation.

Adding animation patches

The patches we’ll use to add animation in this effect are the Animation patch and the Transition patch. To add these to your project:

  1. Click Add Patch.
  2. Insert an Animation patch, and 2 Transition patches.

We need 2 Transition patches, one to specify the movement of each of the two 3D objects.

Connect the Turned On port in the Pulse patch to the Play port in the Animation patch. This tells the effect to start the animation when the input is on, in this case when the eyebrows are raised.

Once you've done that, connect the Turned On port in the Pulse patch to the Reset port in the Animation patch. This will reset the animation every time the eyebrows are raised:

Making the eyes fotate

Now that you've set up the trigger, you can start to build the animations that will rotate the 3D objects.

You'll need to do this separately - once for each object over each eye.

Let’s start with the right eye.

First, you' need to create a patch representing the rotation properties of 4 meshes that make up this part of the eye. They're each listed under a different object in the Scene Panel. Expand the objects and any null objects underneath them to find the mesh.

When you find the mesh, select them. You can select all the mesh at the same time by holding down command on your keyboard.

Expand:

  1. eyes_3.dae - and select right_eyeAO.
  2. eyes_3.dae0 - and select right_eyeBO.
  3. eyes_3.dae1 - and select right_eyeCO.
  4. eyes_3.dae2 - and select right_eyeDO.

Your Scene Panel should look like this:



In the Inspector, click the arrow next to Rotation, to add the patches to your project.



Four patches will be added:



Now connect:

  1. The Progress output of the Animation patch to the Progress input in the Transition patch.
  2. The Value output of the Transition patch to the 3D Rotation input in each of the patches you just created.

Here’s how this part of the graph will look:



Before this part of the graph is complete, we need to edit the values in the Transition patch, to set how the animation will move. Change:

  1. The Z value next to End to 180 - to rotate the objects 180 degrees.
  2. Curve to Quadratic In-Out - the set the pacing of the animation. You could experiment with different options here.

Here’s how the Transition patch should look:



We’re basically going to do the same thing for the left eye.

Again, select the meshes in the Scene Panel:

Expand:

  1. eyes_3.dae - and select left_eyeAO.
  2. eyes_3.dae0 - and select left_eyeBO.
  3. eyes_3.dae1 - and select left_eyeCO.
  4. eyes_3.dae2 - and select left_eyeDO.

And again, click the arrow next to Rotation in the Inspector.

Now connect:

  1. The Progress output of the Animation patch to the Progress input in the second Transition patch.
  2. The Value output of the Transition patch to the 3D Rotation input in each of the patches you just created.

In the Transition patch, change:

  1. The Z value next to End to 180.
  2. The Curve to Quadratic In-Out.

And there we go, your effect should be ready!

Join the Spark AR Creator's community

Find inspiration, see examples, get support, and share your work with a network of creators.

Join Community

Frequently asked questions

Have a specific question? Maybe it's been answered.


Read FAQs
Was this document helpful?