Clear
Back

There may be differences between your version of Spark AR Studio and this tutorial because the product is currently in beta and we update it regularly.

Tutorial: Creating a Face Tracking Effect

Something Went Wrong
We're having trouble playing this video.

Hello! In this tutorial you'll learn how to make an effect that responds to someone's face. You'll use a face tracker, face mesh, material and texture.

We'll talk about:

Download the sample content to follow along with this tutorial. It includes the custom texture you'll need to make this effect. Everything else can be created in Spark AR Studio.

AR effects made with Spark AR Studio can be used in the Facebook and Instagram cameras. To create a frame for a Facebook profile picture, use Frame Studio.

Setting up

To get started, open the unfinished effect in the sample content folder.

The space in the middle of the screen is called the Viewport. This is where you can see your effect as you're building it.

The Simulator on the right of the interface will show how your effect will look on a device:



We've already imported the texture you'll need for this effect. It's called Moustache and listed in the Assets panel:



You'll combine this texture with a material to create the effect of a moustache drawn across a person's face.

Spark AR Studio supports PNG and JPG texture files. Files can be a maximum of 2014x1024 in size. Learn more about using textures.

The face tracker

The first thing you need to do is add a face tracker to your project. The face tracker detects the position and orientation of the face of the person using your effect. It's the foundation of any effect that responds to someone's face.

To add any object to you scene, just click the Add Object button at the bottom of the Scene panel. You'll see a list of all the objects available in Spark AR Studio. For this tutorial:

  1. Select Face Tracker from the list.
  2. Click Insert:


When the face tracker is selected in the Scene panel, you'll see axes moving with the face in the Viewport.

Adding a face mesh

You've already added a face tracker, so now you can insert a face mesh. The face mesh is a 3D model that responds to facial expressions tracked by the face tracker. You'll apply materials to the face mesh to change how it looks - creating the mask effect.

You'll add the face mesh as a 'child' of the face tracker. When an object is the child of another object it will take on any changes made to the object it's a child of - the parent object. It can also respond to the parent object. In this case the face mesh will respond to the movement detected by the face tracker. You can also drag and drop objects on to one another to create child-parent relationships.

To add the face mesh as a child of the face tracker:

  1. Right-click on the face tracker in the Scene panel.
  2. Select Add.
  3. Choose the Face Mesh from the list.

You’ll see the face mesh object in the Viewport and Simulator:



You can tell if an object is a child of another by clicking the arrow to the left of the parent object in the Scene panel. This will show or hide any of its children.

Creating a material

To create a material for the face mesh, make sure the face mesh is selected in the Scene panel.

On the right of the interface is the Inspector. This is where you can make changes to the object or asset you've selected in the Scene panel - including creating materials.

In the Inspector, click + next to Material. This will create a material for the face mesh.

You’ll see the material covering the face mesh in the Viewport and Simulator:

The material will be listed in the Assets panel as material0:

Materials can also be created in the Assets panel. To do this, click Add Asset and select Create New Material. To add the material to an object in the scene, go to the Inspector and click the dropdown next to Materials.

You can rename objects and assets in Spark AR Studio. It's a good idea to do this, to help stay organized as you build your project. All you need to do is right-click on the object and select rename. For this tutorial:

  1. Right-click on material0.
  2. Select Rename.
  3. rename the material mask_material.

Setting the Shader Type

Now go back over to the Inspector to edit the material.

First let's take a look at this option at the top of the list - Shader Type:



You can use this option to choose from several different built-in shaders in Spark AR Studio to define the appearance of materials in your effects.

The mask that we're going to make will show the user's face behind it, so we're going to choose the Face Paint shader. It's the best option for mask effects where a texture is painted on top of the face because it preserves the luminance of the face, but removes the color. This means your own texture can be multiplied over it.

To change the shader:

  1. Click the dropdown menu next to Shader Type.
  2. Select Face Paint.

Now you can see the face through the material:

Applying the texture

Next let's apply the texture. Next to Texture:

  1. Select Choose File....
  2. Select moustache.


Alright, we've finished the mask!

Creating your own textures for face effects

The face reference assets are a collection of assets to help you make face effects. Import faceFeminine.jpg or faceMasculine.jpg into the software you use to create textures. Use them as a guide for the position of facial features, and create your texture on a new layer.

Now let's take a look at how to add the option to use an effect with a friend - by adding a second face tracker.

Adding a second face

To try this out, you can duplicate the face tracker you've just made. This will copy its children, too. Simply:

  1. Select the face tracker in the Scene panel.
  2. Hold down control and c on your keyboard, then control and v.

Another face tracker and face mesh will be listed in the Scene panel.

You'll need to make one edit in the Inspector. Make sure the second face tracker, faceTracker1, is selected in the Scene panel. Over in the Inspector you'll see a property called Tracked face. Use this to assign a particular face to a particular face tracker.

It's set to Face 1 by default. Change it to Face 2:

This will make it responds to the second face that appears in the effect.

And there we go, your effect can be used by two people. You could add a different texture to the material applied to the second face mesh, to add more variety to your effect.

Congratulations! You've finished an effect with face tracking!

Learn more

In this tutorial, you've learned to create a simple mask effect. Next, learn about:

Join the Spark AR Creator's community

Find inspiration, see examples, get support, and share your work with a network of creators.

Join Community

Join the Spark AR Creator's community

Find inspiration, see examples, get support, and share your work with a network of creators.

Join Community

Frequently asked questions

Have a specific question? Maybe it's been answered.


Read FAQs
Was this document helpful?