As a result of the coronavirus (COVID-19) outbreak, we have a reduced capacity to review effects and delays in publishing are expected. Learn more.
There may be differences between your version of Spark AR Studio and this tutorial because the product is currently in beta and we update it regularly.
In this tutorial you'll learn how to create a zoom effect in response to different face movements, using interaction and logic patches in the Patch Editor:
This effect also includes audio, adding an echo to the user's voice that gets stronger with the opening of the mouth.
Download the sample content to follow this tutorial. It includes the finished project - you can use it for reference.
Start by opening Spark AR Studio and create a new project.
This effect uses the video taken by the camera as a texture in the scene. This texture is called the Camera Texture. To add it to your project:
It'll be listed as cameraTexture0 in the Assets panel.
To add the texture to the scene, you'll need to apply it to a material and apply the material to an object. In this effect the object is a rectangle, which will create a flat surface.
To add a rectangle:
It'll automatically be added as a child of a canvas, so it scales with the screen of the device.
Change the size of the rectangle so it fills the whole screen. You can do this in the Inspector:
To create a material for the rectangle, click + next to Materials in the Inspector:
The material will be listed in the Assets panel as material0.
You'll need to make a couple of changes to the material's properties in the Inspector:
We'll use the Patch Editor to apply the camera texture to the material.
Your project should look like this:
Start by creating a patch to represent the camera texture. Click on cameraTexture0 in the Assets Panel, and drag it into the Patch Editor - you'll see an orange patch.
Next, right-click in the Patch Editor, and select a Texture Transform patch from the menu.
The Texture Transform patch is a visual shader. It can be used to apply lots of different transformations to textures in your scene.
To connect these patches:
In the finished effect, the rectangle gets bigger in response to an open mouth, creating a zoom effect.
Building this logic starts with a face tracker, to detect the someone's facial movements:
A series of patches will be created. We're going to use the final one in this effect, faceTracker0.
Next create a patch that will allow us to detect how open someone's mouth is - the Mouth Open patch. To do this:
The Mouth Open patch has two output ports:
For this effect, you'll use the Mouth Openness port. This outputs a signal that indicates how open the mouth is.
To increase the size of texture in response how open the mouth is, you can use the Add patch. Select this patch from the menu in the Patch Editor.
The data type of the Add patch is set to Number by default - you can check this by selecting the patch. You'll see Number written in the blue box underneath the patch.
Click this blue box to change the data type. Select Vector 2 from the dropdown menu, because we're changing the scale of a 2D object, not a 3D object:
Finally, select a 2D Transform Pack from the menu. This patch can be used to transform 2D objects.
We'll use the Scale input in this patch to tell the texture to scale up or down.
Start by connecting the Face port in the face tracker to the Face port in the Mouth Open patch.
To apply the transformation to the texture, connect the 2D Transform output in the 2D Transform Pack to the Transform input in the Texture Transform patch we made earlier.
The texture is rescaling now, but it doesn't look quite right.
To fix this, we need to adjust some values in the patches.
Start with the Add patch. Change the X and Y values in the Add patch to 1. The scale is set to 0 by default, so this will tell the texture to double in size depending on how open the mouth is.
Next adjust the Pivot values in the 2D Transform Pack to 0.5. 0.5 is the center of the frame, so this sets the texture to scale from the center.
Here's how your effect should look now:
The logic in this part of the effect is slightly different from the logic we used with the Mouth Open patch. That's because the texture rescales each time the eyebrows are raised, rather than in response to how much they're raised.
Start by creating the patches that'll do this. In the Patch Editor, select:
Change the Type of the Option Picker to Vector 2 - again because we're rescaling 2D objects. Then, connect:
As a result of connecting these patches, when the face tracker detects eyebrows are raised the Pulse patch will pass this message to the Counter patch.
The Option Picker is where we'll add information to set how much the texture should be rescaled.
Because we've used the Increase port in the Counter, each option will be triggered in ascending order each time the eyebrows are raised. In the Option Picker patch:
To set up the logic to switch between effects, select the Screen Tap, Switch and If Then Else patches from the menu in the Patch Editor.
Screen Tap will detect when someone has tapped the screen of their device. Connect:
This section of the graph should look like this:
The audio element adds an echo to the voice of the person using the effect that gets stronger as the face zoom increases.
All Spark AR Studio projects include an object called the microphone. It captures the voice of the user and plays it back into the effect. You’re going to take the sound captured by the microphone and use the Audio Delay patch to add the echo. This will be passed to a patch representing a speaker, which will render the sound in the scene.
Start by adding the speaker:
To create a patch for the speaker:
A yellow patch will be added to the Patch Editor:
Next add a patch for the microphone:
A purple patch will be added:
From the menu in Patch Editor, select the Audio Delay patch.
Here’s how the graph will look:
Next make some adjustments to the values in the Audio Delay patch. Start with the Active 1 and Active 2 values. Editing these will make the delay effect more intense:
Editing the Feedback value will cause the input to repeat multiple times. For this effect set it to 20.
Use the Mouth Openness port in the Mouth Open patch to send a signal to this section of the graph. Connecting the Mouth Openness port to the Audio Delay patch through the To Range patch will make the echo more prominent as the user opens their mouth wider. You’ll connect the To Range patch to the Wet input in the Audio Delay patch, which controls the intensity of the effect.
An Exponential Smoothing patch will make sure the sound doesn’t change too abruptly.
Create the To Range and Exponential Smoothing patches using the menu in the Patch Editor.
To finish, set:
Here’s how the finished section of the graph will look:
There are lots of ways to customize the audio element of this effect. For example, change the values in the Audio Delay patch, or swap it for another audio effect patch like the Pitch Shifter.