Tutorial: Face Zoom

In this guide we'll show you how to create the Face Zoom effect!

The effect uses the Patch Editor to make a rectangle bigger in response to different interactions, creating the zoom effect:

Download the sample content to follow this tutorial. It includes the finished project - you can use it for reference.

In this guide, we'll cover:

1. Creating the camera texture.
2. Adding the camera texture to a rectangle in the scene.
3. Using the Patch editor to make the rectangle bigger, when someone opens their mouth or raises their eyebrows.
4. Using the Screen Tap patch to switch between the two effects.

Creating the camera texture

To get started open Spark AR Studio and create a new project.

This effect uses the video taken by the camera as a texture in the scene. We'll use the Patch Editor to make it bigger, creating a zoom effect.

So start by creating a texture from the camera video:

1. Select the Camera in the Scene panel.
2. In the Inspector, click + next to Texture Extraction.

It'll be listed as cameraTexture0 in the Assets panel.

Adding the camera texture

To add the texture to the scene, you'll need to apply it to a material and apply the material to an object. In this effect the object is a rectangle, which will create a flat surface.

Create a rectangle and material

To add a rectangle:

1. Click Add Object.
2. Select Rectangle.

It'll automatically be added as a child of a canvas, so it scales with the screen of the device.

Change the size of the rectangle so it fills the whole screen. You can do this in the Inspector:

1. Click each box next to Size.
2. Select Fill Width, and Fill Height.

To create a material for the rectangle, click + next to Materials in the Inspector:

The material will be listed in the Assets panel as material0.

You'll need to make a couple of changes to the material's properties in the Inspector:

1. Change the Shader Type to Flat. This is a performant shader that can be used for materials that don't need to reflect or omit light - like 2D surfaces.
2. Instead of applying a texture under Diffuse, click the arrow next to Texture to create a patch and open the Patch Editor.

We'll use the Patch Editor to apply the camera texture to the material.

Your project should look like this:

Applying the camera texture

Start by creating a patch to represent the camera texture. Click on cameraTexture0 in the Assets Panel, and drag it into the Patch Editor - you'll see an orange patch.

Next, right-click in the Patch Editor, and select a Texture Transform patch from the menu.

The Texture Transform patch is a visual shader. It can be used to apply lots of different transformations to textures in your scene.

To connect these patches:

1. Connect the RGBA output in camera texture patch to the Texture input in the Texture Transform patch.
2. Connect the Texture output in Texture Transform patch to the Diffuse Texture port in the material patch.

Adding the Mouth Open patch

In the finished effect, the rectangle gets bigger in response to an open mouth, creating a zoom effect.

Create the patches

Building this logic starts with a face tracker, to detect the someone's facial movements:

1. Click Add Object and add a face tracker to the scene - it'll be listed as facetracker0 in the Scene Panel.
2. Drag the face tracker into the Patch Editor.

A series of patches will be created. We're going to use the final one in this effect, faceTracker0.

Next create a patch that will allow us to detect how open someone's mouth is - the Mouth Open patch. To do this:

1. Right-click in the Patch Editor.
2. Select the Mouth Open patch from the menu.

The Mouth Open patch has two output ports:

For this effect, you'll use the Mouth Openness port. This outputs a signal that indicates how open the mouth is.

To increase the size of texture in response how open the mouth is, you can use the Add patch. Select this patch from the menu in the Patch Editor.

The data type of the Add patch is set to Number by default - you can check this by selecting the patch. You'll see Number written in the blue box underneath the patch.

Click this blue box to change the data type. Select Vector 2 from the dropdown menu, because we're changing the scale of a 2D object, not a 3D object:

Finally, select a 2D Transform Pack from the menu. This patch can be used to transform 2D objects.

We'll use the Scale input in this patch to tell the texture to scale up or down.

Connect the patches

Start by connecting the Face port in the face tracker to the Face port in the Mouth Open patch.

Then connect:

1. The Mouth Openness port to either of the X inputs in the Add patch.
2. The Add patch to the Scale port in the 2D Transform Pack.

To apply the transformation to the texture, connect the 2D Transform output in the 2D Transform Pack to the Transform input in the Texture Transform patch we made earlier.

The texture is rescaling now, but it doesn't look quite right.

To fix this, we need to adjust some values in the patches.

Start with the Add patch. Change the X and Y values in the Add patch to 1. The scale is set to 0 by default, so this will tell the texture to double in size depending on how open the mouth is.

Next adjust the Pivot values in the 2D Transform Pack to 0.5. 0.5 is the center of the frame, so this sets the texture to scale from the center.

Here's how your effect should look now:

Responding to raised eyebrows

The logic in this part of the effect is slightly different from the logic we used with the Mouth Open patch. That's because the texture rescales each time the eyebrows are raised, rather than in response to how much they're raised.

Counting how many times the eyebrows are raised

Start by creating the patches that'll do this. In the Patch Editor, select:

1. An Eyebrows Raised patch.
2. A Pulse, Counter and Option Picker patch.

Change the Type of the Option Picker to Vector 2 - again because we're rescaling 2D objects. Then, connect:

1. The Face port in the face tracker patch to the Face port in Eyebrows Raised.
2. The Eyebrows Raised output to the Pulse patch.
3. The Turned On port in the Pulse patch to the Increase port in the Counter patch.

As a result of connecting these patches, when the face tracker detects eyebrows are raised the Pulse patch will pass this message to the Counter patch.

Next:

1. In the Counter patch, set the Maximum Count to 3 to detect three raised eyebrows and then start counting again.
2. Connect Counter to the Option input in Option Picker.

Rescaling the Texture

The Option Picker is where we'll add information to set how much the texture should be rescaled.

Because we've used the Increase port in the Counter, each option will be triggered in ascending order each time the eyebrows are raised. In the Option Picker patch:

1. For the first option, set X to 1 and Y to 1 - to double the texture.
2. For the second option, set X to 1.5 and Y to 1.5.
3. For the third option, set X to 2 and Y to 2.

Switching between effects

To set up the logic to switch between effects, select the Screen Tap, Switch and If Then Else patches from the menu in the Patch Editor.

Screen Tap will detect when someone has tapped the screen of their device. Connect:

1. Screen Tap to the Flip port of the Switch patch - to create the trigger to switch between effects.
2. Switch to If Then Else.

To finish the effect:

1. Disconnect the Add patch from the 2D Transform patch.
2. Connect the output of the Option Picker to the Then input of the If Then Else patch.
3. Connect Add to the Else port of the If Then Else patch.
4. Connect the If Then Else port to the Scale input of the 2D Transform patch.

This section of the graph should look like this:

Learn more

• Explore the different interaction patches available in Spark AR Studio.
• Make objects appear in response to face gestures.