Adding the camera texture

To add the texture to the scene, you'll need to apply it to a material and apply the material to an object. In this effect the object is a rectangle, which will create a flat surface.

Create a rectangle and material

To add a rectangle:

1. Click Add Object.
2. Select Rectangle.

It'll automatically be added as a child of a canvas, so it scales with the screen of the device.

Change the size of the rectangle so it fills the whole screen. You can do this in the Inspector:

1. Click each box next to Size.
2. Select Fill Width, and Fill Height.

To create a material for the rectangle, click + next to Materials in the Inspector:

The material will be listed in the Assets panel as material0.

You'll need to make a couple of changes to the material's properties in the Inspector:

1. Change the Shader Type to Flat. This is a performant shader that can be used for materials that don't need to reflect or omit light - like 2D surfaces.
2. Instead of applying a texture under Diffuse, click the arrow next to Texture to create a patch and open the Patch Editor.

We'll use the Patch Editor to apply the camera texture to the material.

Your project should look like this:

Applying the camera texture

Start by creating a patch to represent the camera texture. Click on cameraTexture0 in the Assets Panel, and drag it into the Patch Editor - you'll see an orange patch.

Next, right-click in the Patch Editor, and select a Texture Transform patch from the menu.

The Texture Transform patch can be used to apply lots of different transformations to textures in your scene.

To connect these patches:

1. Connect the RGBA output in camera texture patch to the Texture input in the Texture Transform patch.
2. Connect the Texture output in Texture Transform patch to the Diffuse Texture port in the material patch.

Adding the Mouth Open patch

In the finished effect, the rectangle gets bigger in response to an open mouth, creating a zoom effect.

Create the patches

Building this logic starts with a face tracker, to detect the someone's facial movements:

1. Click Add Object and add a face tracker to the scene - it'll be listed as facetracker0 in the Scene Panel.
2. Drag the face tracker into the Patch Editor.

A series of patches will be created. We're going to use the final one in this effect, faceTracker0.

Next create a patch that will allow us to detect how open someone's mouth is - the Mouth Open patch. To do this:

1. Right-click in the Patch Editor.
2. Select the Mouth Open patch from the menu.

The Mouth Open patch has two output ports:

For this effect, you'll use the Mouth Openness port. This outputs a signal that indicates how open the mouth is.

To increase the size of texture in response how open the mouth is, you can use the Add patch. Select this patch from the menu in the Patch Editor.

The data type of the Add patch is set to Number by default - you can check this by selecting the patch. You'll see Number written in the blue box underneath the patch.

Click this blue box to change the data type. Select Vector 2 from the dropdown menu, because we're changing the scale of a 2D object, not a 3D object:

Finally, select a 2D Transform Pack from the menu. This patch can be used to transform 2D objects.

We'll use the Scale input in this patch to tell the texture to scale up or down.

Connect the patches

Start by connecting the Face port in the face tracker to the Face port in the Mouth Open patch.

Then connect:

1. The Mouth Openness port to either of the X inputs in the Add patch.
2. The Add patch to the Scale port in the 2D Transform Pack.

To apply the transformation to the texture, connect the 2D Transform output in the 2D Transform Pack to the Transform input in the Texture Transform patch we made earlier.

The texture is rescaling now, but it doesn't look quite right.

To fix this, we need to adjust some values in the patches.

Start with the Add patch. Change the X and Y values in the Add patch to 1. The scale is set to 0 by default, so this will tell the texture to double in size depending on how open the mouth is.

Next adjust the Pivot values in the 2D Transform Pack to 0.5. 0.5 is the center of the frame, so this sets the texture to scale from the center.

Here's how your effect should look now:

Responding to raised eyebrows

The logic in this part of the effect is slightly different from the logic we used with the Mouth Open patch. That's because the texture rescales each time the eyebrows are raised, rather than in response to how much they're raised.

Counting how many times the eyebrows are raised

Start by creating the patches that'll do this. In the Patch Editor, select:

1. An Eyebrows Raised patch.
2. A Pulse, Counter and Option Picker patch.

Change the Type of the Option Picker to Vector 2 - again because we're rescaling 2D objects. Then, connect:

1. The Face port in the face tracker patch to the Face port in Eyebrows Raised.
2. The Eyebrows Raised output to the Pulse patch.
3. The Turned On port in the Pulse patch to the Increase port in the Counter patch.

As a result of connecting these patches, when the face tracker detects eyebrows are raised the Pulse patch will pass this message to the Counter patch.

Next:

1. In the Counter patch, set the Maximum Count to 3 to detect three raised eyebrows and then start counting again.
2. Connect Counter to the Option input in Option Picker.

Rescaling the Texture

The Option Picker is where we'll add information to set how much the texture should be rescaled.

Because we've used the Increase port in the Counter, each option will be triggered in ascending order each time the eyebrows are raised. In the Option Picker patch:

1. For the first option, set X to 1 and Y to 1 - to double the texture.
2. For the second option, set X to 1.5 and Y to 1.5.
3. For the third option, set X to 2 and Y to 2.

Switching between effects

To set up the logic to switch between effects, select the Screen Tap, Switch and If Then Else patches from the menu in the Patch Editor.

Screen Tap will detect when someone has tapped the screen of their device. Connect:

1. Screen Tap to the Flip port of the Switch patch - to create the trigger to switch between effects.
2. Switch to If Then Else.

To finish the effect:

1. Disconnect the Add patch from the 2D Transform patch.
2. Connect the output of the Option Picker to the Then input of the If Then Else patch.
3. Connect Add to the Else port of the If Then Else patch.
4. Connect the If Then Else port to the Scale input of the 2D Transform patch.

This section of the graph should look like this:

Adding audio

The audio element adds an echo to the voice of the person using the effect that gets stronger as the face zoom increases.

All Spark AR Studio projects include an object called the microphone. It captures the voice of the user and plays it back into the effect. You’re going to take the sound captured by the microphone and use the Audio Delay patch to add the echo. This will be passed to a patch representing a speaker, which will render the sound in the scene.

Start by adding the speaker:

1. Click Add Object.
2. Select Speaker from the menu.

To create a patch for the speaker:

1. Select speaker0 in the Scene panel.
2. Go to the Inspector.
3. Click the arrow to the left of Audio in the Inspector.

A yellow patch will be added to the Patch Editor:

Next add a patch for the microphone:

1. Select Microphone in the Scene panel.
2. Drag it into the Patch Editor.

A purple patch will be added:

The echo effect

From the menu in Patch Editor, select the Audio Delay patch.

Connect:

1. The output of the Microphone patch to the Audio input in the Audio Delay patch.
2. The output of the Audio Delay patch to the Audio input in the speaker patch.

Here’s how the graph will look:

Next make some adjustments to the values in the Audio Delay patch. Start with the Active 1 and Active 2 values. Editing these will make the delay effect more intense:

• Check the box next to Active 1 and set Time 1 to 200.
• Check the box next to Active 2 and set Time 2 to 400.

Editing the Feedback value will cause the input to repeat multiple times. For this effect set it to 20.

Triggering the echo

Use the Mouth Openness port in the Mouth Open patch to send a signal to this section of the graph. Connecting the Mouth Openness port to the Audio Delay patch through the To Range patch will make the echo more prominent as the user opens their mouth wider. You’ll connect the To Range patch to the Wet input in the Audio Delay patch, which controls the intensity of the effect.

An Exponential Smoothing patch will make sure the sound doesn’t change too abruptly.

Create the To Range and Exponential Smoothing patches using the menu in the Patch Editor.

Now connect:

• The Mouth Openness port in the Mouth Open patch to the top input in the To Range patch.
• The output of the To Range patch to the Input input in the Exponential Smoothing patch.
• The output of the Exponential Smoothing patch to the Wet input in the Audio Delay patch.

To finish, set:

• The Max value in the To Range patch to 100. This will define the volume range.
• The Damping value in the Exponential Smoothing patch to 500.

Here’s how the finished section of the graph will look:

There are lots of ways to customize the audio element of this effect. For example, change the values in the Audio Delay patch, or swap it for another audio effect patch like the Pitch Shifter patch.