In this tutorial, you’ll learn to create a face tracking effect using a face tracker, face mesh, materials and a custom texture. Once you’ve created a simple effect with the provided textures, you’ll learn to create your own custom face mask textures using the face mask template.
Download the sample content to follow along with this tutorial. After downloading, you’ll find a finished and unfinished version of the effect. The unfinished version contains the texture we’ll be using to make this effect, but is otherwise a completely blank project.
Once you’ve downloaded the sample content and opened the unfinished effect, Spark AR Studio will look like this:
If you look at the Assets panel, you’ll see that we've already imported the emoji texture you'll need. We’ll combine this texture with a material to make it look like there are stickers on a person’s face. Find out more about working with textures and materials.
The first thing you need to do is add a face tracker to your project. The face tracker detects the position and orientation of the face of the person using your effect. It's the basis of any effect that responds to someone's face.
To add an object to your scene, click the Add Object button at the bottom of the Scene panel. You'll see a list of all the objects available in Spark AR Studio. For this tutorial:
If you click the face tracker that was added to your Scene panel, you’ll see arrows moving in the Viewport:
The face mesh is a 3D model that responds to facial expressions detected by the face tracker. You'll apply materials to the face mesh to create the mask effect.
To get started, add the face mesh as a 'child' of the face tracker. When an object is the child of another object it will be affected by changes made to the object it's a child of. In this effect, the face mesh (child) will move with the face tracker (parent).
To add the face mesh as a child of the face tracker:
You’ll see the face mesh object in the Viewport and Simulator:
On the right of the interface is the Inspector. This is where you can make changes to any objects or asset you've selected in the Scene or Assets panel:
In the above image, you can see that our face mesh is selected in the Scene panel. The inspector is highlighted in pink.
With the face mesh still selected, click + next to Material in the inspector. This will create a new material and attach it to your face mesh. You’ll see the checkerboard pattern on the face mesh has changed to a default, standard material:
Materials can also be created in the Assets panel. To do this, click Add Asset and select Material. To add the new material to an object in the scene, go to the Inspector and click the dropdown next to Materials.
Renaming objects and assets as they are created will help you to stay organized as you build your project. Simply right-click on the object or asset and select Rename. For this tutorial:
Shader types determine how your material will look and how it interacts with lighting in your scene.
The face sticker effect that we're going to make will look best if we are using the Flat shader type. This is the best option for face sticker effects, as flat materials display color/texture values absolutely and don't respond to lighting. You might also use this shader type if there's lighting and shadow present in your texture file already, or if you're creating a material for a 2D object
To change the shader type:
You’ll see that the shadows on your face mesh have almost disappeared, giving it a flat look:
Finally, we’ll apply the face_sticker texture to our face mesh and adjust the effect so that it can be used by multiple people at the same time.
To apply the texture to the face mesh, click on the material in the Assets panel and then, in the Inspector:
The texture will change, and you’ll see the finished effect for the first time:
To make your effect look more realistic, you may want to adjust the opacity of the material being used. Changing the opacity of mask_material to around 80% will dull the stickers just enough to give us the realistic sticker look we want.
At the moment, our effect only works for one user at a time. As a finishing touch, let’s add a second face tracker. This makes it so that if two faces are detected by the camera, both will have the mask applied to their face.
To add the second tracker, simply duplicate the one you’ve already made:
You’ll see a second face tracker and face mesh has been added to your Scene panel. To finalise our second face tracker, we need to make a couple of small adjustments:
In the Scene panel, click the second face tracker (faceTracker1). In the Inspector you'll see a property called Tracked Face. It's set to Face 1 by default. Change it to Face 2.
Next, do the same for the second face mesh. Click the second face mesh (facemesh0) under faceTracker1. In the Inspector you'll see a property called Tracked Face. Change it to Face 2.
Finally, there may be times when you want to improve the accuracy of the face tracker. That is particularly useful for delicate or subtle effects, like makeup try-ons.
To change the face tracking accuracy, we'll edit the project's settings:
Congratulations, you've now finished creating a face tracking effect that can be used by two people!
Ready to use the face mask template to create your own custom face masks? To get started, download the face mask template and open it in your chosen raster editing software – we will be using Adobe® Photoshop® software, but free editors (GIMP, Photopea, etc.) are available.
Once the template is open, it will look like this:
There are many different layers and assets available in this file, all designed to help you accurately create custom assets to use with the face tracker. Explore layer groups and turn the layers on and off to get an idea of what is available. Alternatively, a detailed guide can be found here. If you need help with layer basics, visit Adobe’s guide on the subject.
As we want to create a custom mask texture, the first thing we need to do is hide the Blue Tiger Texture layer that has been included as an example. Click the eye icon next to the layer’s name. You’ll be left with just the face feminine reference asset visible:
If you hide the face feminine layer, you’ll be able to use the face masculine reference asset instead:
If you’d like to draw a mask by hand, create a new layer and ensure that it is above all the other layers in this file’s layer hierarchy. In the image below, you can see that the new layer is called Layer 1:
Once that’s set up, you can to draw custom illustrations inside that layer and have them appear on top of the face reference asset. Feel free to add as many new layers as you like – just ensure that they appear above the reference assets in your layer hierarchy.
Alternatively, you may wish to import owned or copyright free assets from your computer and use them as part of your custom mask. The easiest way to do this is to drag the assets from Finder (Mac) or File Explorer (Windows), directly into your Photoshop project. They will appear as new layers in the layer hierarchy:
Items imported in this way will appear in a new layer above the layer you previously had selected. If your new layer isn’t above the face reference assets in the layer hierarchy, simply drag it to the top.
Imported and drawn assets can be rotated, resized and moved around using the Free Transform tool. This can be found at Edit > Free Transform, or by pressing Cmd/Ctrl+T.
For more information on creating with Photoshop, you can visit Adobe’s Photoshop User Guide.
When you are done customizing your mask, the first thing you need to do is hide all the layers apart from those you’d like to be visible in your effect. To do this, click the eye icon next to each layer:
Now that we have hidden all the reference assets, our custom mask is ready to be exported as a transparent PNG file. To do this:
Now you have a transparent PNG texture file that matches up with the Spark AR Studio face mesh perfectly. This texture can now be added to your Spark AR Studio Assets panel (click Add Assets) and used in your face tracking effect:
Adobe and Photoshop are either registered trademarks or trademarks of Adobe in the United States and/or other countries. Adobe product screenshot(s) reprinted with permission from Adobe.