Creating a Face Tracking Effect
Hello! In this guide, you'll learn how to make an effect that responds to someone's face. You'll use a face tracker, face mesh, material and texture.
Download the sample content to follow along with this tutorial. It includes the custom texture you'll need to make this effect. Everything else can be created in Spark AR Studio.
AR effects made with Spark AR Studio can be used in the Facebook and Instagram cameras. To create a frame for a Facebook profile picture, use Frame Studio.
Setting up
To get started, open the unfinished effect in the sample content folder.
The space in the middle of the screen is called the Viewport. This is where you can see your effect as you're building it.
The Simulator will show how your effect will look on a device.
We've already imported the texture you'll need for this effect. It's called Moustache and listed in the Assets panel:
You'll combine this texture with a material to create the effect of a moustache drawn across a person's face.
Spark AR Studio supports PNG and JPG texture files. Files can be a maximum of 1014x1024 in size. Learn more about using textures.
The face tracker
The first thing you need to do is add a face tracker to your project. The face tracker detects the position and orientation of the face of the person using your effect. It's the foundation of any effect that responds to someone's face.
To add any object to you scene, just click the Add Object button. You'll see a list of all the objects available in Spark AR Studio. For this tutorial:
- Select the Face Tracker from the list.
- Click Insert .
When the face tracker is selected in the Scene panel, you'll see axes moving with the face in the Viewport.
Adding a face mesh
You've already added a face tracker, so now you can insert a face mesh. The face mesh is a 3D model that responds to facial expressions tracked by the face tracker. You'll apply materials to the face mesh to change how it looks - creating the mask effect.
You'll add the face mesh as a 'child' of the face tracker. When an object is the child of another object it will take on any changes made to the object it's a child of - the parent object. It can also respond to the parent object. The face mesh will respond to the movement detected by the face tracker.
To add the face mesh as a child of the face tracker:
- Right-click on the face tracker in the Scene panel.
- Select Add.
- Choose the Face Mesh from the list.
You’ll see the face mesh object in the Viewport and Simulator:
You can tell if an object is a child of another by clicking the arrow to the left of the parent object in the Scene panel. This will show or hide any of its children.
Creating a material
To create a material for the face mesh, make sure it's selected in the Scene panel.
On the right of the interface is the Inspector. This is where you can make changes to the object or asset you've selected in the Scene panel - including creating materials.
In the Inspector, click + next to Material to create a material for the face mesh.
You’ll see the material covering the face mesh in the Viewport and Simulator:
The material will be listed in the Assets panel as material0:
Materials can also be created in the Assets panel. To do this, click Add Asset and select Material. To add the material to an object in the scene, go to the Inspector and click the dropdown next to Materials.
You can rename objects and assets in Spark AR Studio. It's a good idea to do this, to help stay organized as you build your project. All you need to do is right-click on the object and select rename. For this tutorial:
- Right-click on material0.
- Select Rename.
- rename the material mask_material.
Setting the Shader Type
Go back to the Inspector to edit the material.
First let's take a look at this option at the top - Shader Type:
You can use this option to choose from several different built-in shaders in Spark AR Studio to define the appearance of materials in your effects.
The mask that we're going to make will show the user's face behind it, so we're going to choose the Face Paint shader. It's the best option for mask effects where a texture is painted on top of the face because it preserves the luminance of the face, but removes the color. This means your own texture can be multiplied over it.
To change the shader:
- Click the dropdown menu next to Shader Type.
- Select Face Paint.
Now you can see the face through the material:
Applying the texture
Next let's apply the texture. Next to Texture:
- Select Choose File....
- Select moustache.
Alright, we've finished the mask! Now let's take a look at how to add the option to use an effect with a friend - by adding a second face tracker.
Adding a second face
To try this out, you can duplicate the face tracker you've just made. This will copy its children, too. Simply:
- Select the face tracker in the Scene panel.
- Hold down control and c on your keyboard, then control and v.
Another face tracker and face mesh will be listed in the Scene panel.
You'll need to make one edit in the Inspector. Make sure the second face tracker, faceTracker1, is selected in the Scene panel. Over in the Inspector you'll see a property called Tracked face. Use this to assign a particular face to a particular face tracker.
It's set to Face 1 by default. Change it to Face 2:
This will make it responds to the second face that appears in the effect. And there we go, your effect can be used by two people. You could add a different texture to the material applied to the second face mesh, to add more variety to your effect.
Congratulations! You've finished an effect with face tracking!
