Clear
Back

As a result of the coronavirus (COVID-19) outbreak, we have a reduced capacity to review effects and delays in publishing are expected. Learn more.

Tutorial: Light Leak

This AR effect uses visual shader patches to create the appearance of a realistic beam of light falling across the face.

Background segmentation separates the person using the effect from the background of the scene, so the light appears on the face and not the background:



In this guide, you'll learn about:

Download the sample content to follow along with this tutorial. If you open the unfinished project in the sample content folder, you'll see we've already imported the texture you'll need - it's listed in the Assets panel as lightLeak_tex.

Adding the custom texture, camera texture and segmentation

In the finished effect, a beam of light shines across the user's face. It isn't visible in the background because we've separated the foreground and background using segmentation.

Materials in both the foreground and background also have the camera texture applied to them. In the foreground, we'll combine segmentation and the camera texture with the custom texture and a gradient effect too. In the background, we'll just add a gradient.

Adding segmentation

In Spark AR Studio, segmentation effects are created using a texture called a segmentation mask texture. Start by inserting the 2 rectangles that will render the textures in the scene.

  1. Click Add Object, and select a Rectangle. It'll automatically be inserted as a child of the canvas.
  2. Right-click the canvas, and select a Rectangle.

To help you keep track, rename:

  1. The first rectangle in the list background.
  2. The second rectangle in the list user.

Your Scene panel should look like this:



Next, create a material for each rectangle:

  1. Select each rectangle in the Scene panel.
  2. Click + next to Material in the Inspector.


Like before, it's worth renaming the materials. Rename them user_material and background_material, to correspond with the rectangles they're applied to.

Next resize the rectangles so they fill the screen of the device:

  1. Select the rectangles - you can select them both at the same time by holding command on your keyboard if you're using a Mac, or control on a Windows machine.
  2. Click in the first box next to Size in the Inspector, and select Fill Width.
  3. Click in the second box next to Size, and select Fill Height.

Now create the segmentation mask texture:

  1. Select the Camera in the Scene panel - it's listed underneath Device.
  2. In the Inspector, click + next to Segmentation.


You'll see the texture in the Assets panel, listed as segmentationMaskTexture0.

Apply this texture to both the materials you've created:

  1. Select the user_mat in the Assets panel.
  2. In the Inspector, tick the box next to Alpha.
  3. Click the dropdown next to Texture, and select segmentationMaskTexture0 from the list.
  4. Follows these steps for background_mat.

Your project should look like this:

The camera texture

The camera texture is a video of what's being captured by the camera when the effect is open on a device. Applying the camera texture will show the user and background in the effect again.

To create the camera texture:

  1. Select the Camera in the Scene panel.
  2. In the Inspector, click + next to Texture Extraction.

You'll see the texture listed in the Assets panel as cameraTexture0.

You'll finish creating this material in the Patch Editor, so you can combine multiple textures in the same material. Open the Patch Editor by selecting View in the menu bar, then Show/Hide Patch Editor.

Next create a patch to represent the Camera Texture. You can do this by dragging it from the Assets panel, into the Patch Editor.



You'll also need to create a patch to represent the Diffuse Texture property of each material. You can do this by:

  1. Selecting each material in the Assets panel.
  2. Clicking the arrow next to Texture in the Inspector:


You'll see two new patches in the Patch Editor:



Next, connect the RGBA output in Camera Texture to the Diffuse Texture port in background_material.



If you change the video to your own camera, you'll see the background has appeared in the Simulator.

Adding the texture

Now it's time to complete user_material, by combining it with the camera texture and custom texture. Use an Add patch to do this.

The texture is listed in the Assets panel as lightLeak_tex. Drag it into the Patch Editor to create a patch.

Right-click in the Patch Editor, and select an Add patch from the menu. You'll need to to change the data type of the Add patch to Vector 4 - because there are 4 values to pass into the user material - R, G, B and A:



Connect:

  1. The RGBA port in the CameraTexture and lightLeak_tex patches to the Add patch.
  2. The output in the Add patch to the Diffuse Texture port in user_material.

The graph should look like this:

You'll see the texture in the Simulator and Viewport:

Adding a color gradient

Now let's add the gradient effect. You'll need to break some of the connections between patches you've already made. It's simple to do this - just click on a cable connecting one patch to another, and press delete on your keyboard.

Disconnect:

  1. The connection between the camera texture and the Diffuse Texture port in background_material.
  2. The connection between the Add patch and the Diffuse Texture port in user_material:


Now create the patches you'll need for the gradient effect. Right-click in the Patch Editor and select:

  1. A Gradient and Gradient Step patch.
  2. 2 Multiply patches.
  3. An Add patch.


You'll use:

  1. The Multiply patches to multiply the strength of the color gradient.
  2. The Add patch to combine the gradient with the camera texture and custom texture.

So, connect:

  1. The output of the Gradient patch to the input of the Gradient Step patch.
  2. The Output port in the Gradient Step patch to the top input port in each Multiply patch.
  3. The output of one Multiply patch to the Diffuse Texture port in the background material patch:


To complete the gradient effect in the background of the scene, you'll need to make some edits to background_material. In the Inspector:

  1. Uncheck the checkbox next to Alpha.
  2. Under Render Options change the Blend Mode to Add.
  3. Change the Opacity to 30%.

Next make some adjustments to the Gradient Step patch:

  1. Change the color next to Previous Step to a pale yellow and the color next to New Color to white. You could pick different colors for a different effect.
  2. Set the Start Range to 0.6.

The shape of the gradient will be set to Horizontal by default. Change the shape of the gradient:

  1. Click on the patch.
  2. Select Circular from the menu.

Then connect the output of the Add patch connected to the two orange texture patches, to the bottom input in the Add patch you've just created:



Then, connect the output of the Add patch connected to the two texture patches, to the bottom input in the Add patch you've just created:



Now connect:

  1. The output of the second Add patch to the Diffuse Texture input in user_material.
  2. The output of the remaining Multiply patch to the input in the second Add patch.


The color effect is super bright:



Change the values in the Multiply patches to fix this. Change:

  1. The value in the Multiply patch at the top of the graph to 0.7.
  2. The value in the Multiply patch at the bottom of the graph to 0.2.

We experimented to work out the best values for the Multiply patches. You could make the color stronger or weaker by adjusting these numbers.

At this point, your project should look like this:



At the moment the texture looks flat and unrealistic. To add realism, we'll make the light leak texture bend and distort relative to the camera texture, so it appears to fall in three dimensions across the face.

Distorting the Texture

In the finished effect we've grouped the collection of patches that distort and transform the texture. It's called Texture Distortion Shader:



When it's connected to the rest of the graph it combines and distorts the camera texture and custom texture, adding realism to the effect. You can take a look at the patches that make up the Texture Distortion Shader by clicking the icon in the right corner of the patch group.

This part of graph works by using Vertex Attribute patches to read UV values, which define how a texture is laid across a surface. In this case, the surface is the rectangles our materials are applied to. This graph will take those values - normalized from 0 to 1 horizontally and vertically. It will then offset them using RG and RGB values from the camera texture - using a Multiply patch to amplify the effect.

These UVs, distorted by the camera texture and applied to the light leak texture, will causes the beam of light to distort relative to the camera texture, by looking up pixels in the light leak texture using the Texture Sampler patch. This gives the effect of the light rolling across a 3D face, even though the distortion is happening based on the RGB values, not actual depth or 3D information.

So you can quickly build this part of the effect, we've already added most of the patches you'll need for the Texture Distortion Shader and connected them in a graph. The graph is listed as an asset in the Assets panel, under Patches. It's called Distortion Shader:



Drag and drop the Distortion Shader from the Assets panel into the Patch Editor. Right-click on the patch group, and select Ungroup. The graph will look like this:



We need to adjust the values in these patches and connect them to the textures and materials, so you're still going to learn exactly what each patch is doing.

You'll need to create a couple more patches too. Right-click in the Patch Editor and select:

  1. 2 Vertex Attribute patches.
  2. A Pack patch.

Change:

  1. The value in the Vertex Attribute patches to Texture Coordinates, to read the texture coordinates from the rectangles.
  2. The Type in the Pack patch to Vector 2.


Connect the output of one Vertex Attribute patch to the UV input in the Texture Sampler at the start of the graph:



Now connect the output of the other Vertex Attribute patch to input in the Add patch that's connected to the UV input in the second Texture Sampler:



Then connect the output of the Multiply patch at the end of the graph to the input in the Pack patch:



Next we'll connect these patches to the rest of the graph.

Connecting the camera texture and custom texture

Like before, you'll need to break some connections again to do this. Disconnect the CameraTexture and LightLeak_tex patches from the rest of the graph.

You can delete the Add patch that was connected to lightLeak_tex - we don't need it anymore:



Instead, connect:

  1. The RGB port in lightLeak_tex to the Texture port in the Texture Transform patch.
  2. The RGB port in CameraTexture to both the Texture input in the Texture Sampler patch, and the top input in the final Multiply patch in the graph:

Connecting the materials

Go over to the right end of the graph, and connect the output port in the Pack patch to the remaining input port in the remaining Add patch. The end of the graph should look like this:



Now it's time to adjust the values in the patches.

Adjusting values

Start with the Pack patch at the end of the graph. Change the input value in the second row to 1:



You'll see the color in the scene, but the textures are really dark.



So you can see the texture, adjust the value in the Multiply patch at the bottom of the graph. Change it to -0.1. Then, change the value in the Swizzle patch to xy.



Let's adjust the brightness now. Do this by changing the input value in the Multiply patch that sits between the Texture Transform and Add patches to 1.5.

Change the value in the Add patch connected to it to 0.5. The Add patch lifts the black levels of the light leak texture, so when it's multiplied over the camera texture, it doesn't darken it too much.



Next let's flip the custom texture over. To do this, adjust the rotation and scale of the texture in the 2D Transform Pack:

  1. Next to Translation, change X to -0.09 and Y to 0.013.
  2. Next to Scale, change X to -1.1.
  3. Change the Rotation to 180.
  4. Next to Pivot change X to -0.5 and Y to 0.5.

And there we have it - you've used visual shaders to combine and distort the camera texture and a custom texture, creating the appearance of a beam of light:

Normalizing values

Now you have the texture in the scene, you're ready to add interaction - so the texture reacts to the user's face and moves when the screen pans. You'll also add animation to make the beam of wobble slightly from side to side.

This part of the effect will use the face tracker to detect the position of the face, the Screen Pan patch to detect screen interaction, and the Loop Animation patch to animate the texture.

The values these patches generate are high. If you connected, for example, the Screen Pan patch directly to the rest of the graph, the tiniest touch of the screen would make the texture move very dramatically.

To solve this problem, you'll normalize these values using a Divide patch. This will divide the values outputted by these patches by a high number, so they're much smaller.

You'll also create Add patches, which you'll use later to connect the animation patches and the Screen Pan patch to the rest of the graph.

Right-click in the Patch Editor and select 2 Add patches and a Divide patch.

Change the Type of the Add patches to Vector 2, because these patches will move the texture in 2D space - on the X and Y axes.

Connect them, so your graph looks like this:



Then:

  1. Connect the output of the Divide patch to the Translation input in the 2D Transform Pack patch.
  2. Change the value in the Divide patch to 1000.

Adding the face tracker

The face tracker will detect the movement of the user's face. This data will be used to drive the movement of the beam of light.

First you'll need to add a face tracker to your project:

  1. Click Add Objectt.
  2. Select Face Tracker from the menu.

To create a patch representing the face tracker, drag and drop it into the Patch Editor.

You'll also need to select an Unpack, Pack and Multiply patch from the menu.

You'll use Unpack to split the X,Y and Z data stream representing the 3D Position of the face into individual channels. The Pack patch converts this data in to a Vector 2 data stream - X and Y. This data can then be read by the 2D Transform Pack patch.

You'll edit the Multiply patch to amplify the effect of the movement of the face, causing the light leak texture to move a bit faster in line with the face's movement.

Like with other patches in this effect, change the Type of the Pack and Multiply patches to Vector 2.

Connect the patches, so your graph looks like this:



Finally, change the bottom values in the Multiply patch to -5 and -7, to amplify the movement of the face.

Adding the animation

Next we'll add extra interest to the effect with an animation that makes the beam of light move slightly.

From the menu in the Patch Editor, select a:

  1. Loop Animation patch.
  2. Transition patch.

Change the Type of the Transition patch to Vector 2.

Connect:

  1. The Progress port in the Loop Animation patch to the Progress port in the Transition patch.
  2. The output in the Transition patch to the lower input in the Add patch.

The section of the graph should look like this:



To set up the animation, adjust the values in the Loop Animation and Transition patches:

In Loop Animation:

  1. Check the box next to Enable.
  2. Change Duration to 3.
  3. Check the box next to Mirrored.

In Transition, adjust the values to set the position on the screen that the light beam will move between, and how it will move:

  1. Next to Start, set X to -50 and Y to -120.
  2. Next to End, set X to 50 and Y to -150.
  3. Change the Curve to Quadratic In-Out.

Here's how the patches should look:



The light beam should now move slightly across the face.

Adding interactivity

Right-click in the Patch Editor and select a Screen Pan patch. This patch will detect when a user moves their finger along the screen in one continuous motion.

Connect the 2D Offset port in Screen Pan to the remaining port in the second Add patch:

Adding audio

Finish the project by adding subtle, ethereal music playing on a continuous loop:

Something Went Wrong
We're having trouble playing this video.


You’ll use a free audio file from AR Library. It’s listed in the Assets panel as arl_mus_RelaxingMeditationSpace_lp_01.m4a.

Adding audio patches will alter the sound slightly, adding extra interest.

Adding the audio clip

To render sound in any effect, you’ll need a scene object called a Speaker. To add it:

  1. Click Add Object.
  2. Select Speaker from the menu.

Because the sound in this effect is supposed to be subtle, reduce the volume of the speaker:

  1. Select speaker0 in the Scene panel.
  2. In the Inspector, change the Volume to 75%.

The network of patches you’ll build will send a signal to a patch representing the Speaker, playing the sound in the scene.

To create a patch for the Speaker:

  1. In the Scene panel, select speaker0.
  2. Go to the Inspector.
  3. Click the arrow to the left of Audio.

A yellow patch will be added to the Patch Editor:



Now create a patch representing the audio clip. Select arl_mus_RelaxingMeditationSpace_lp_01.m4a in the Assets panel and drag it into the Patch Editor. An orange patch will be added:



To connect the audio file to the speaker, use a Pulse, Single-Clip Controller and Audio Player patch. You can create these using the menu in Patch Editor.

Connect:

  • The Turned On output of the Pulse patch to the Loop input in the Single-Clip Controller patch. This will tell the audio to play on a loop as soon as the effect opens.
  • The Controller input in the Single-Clip Controller patch to the Audio input in the yellow speaker patch.
  • The output of the orange patch representing the audio file to the Audio Clip port in the Audio Player patch.

You’ve now set up the audio file to play on a loop. Here’s how the graph will look:

Processing the sound

Three audio effect patches will change the sound slightly to better fit the atmosphere in the effect:

  • A Pitch Shifter patch will reduce the audio pitch by 4 semitones, making it even more subtle and meditative
  • A Parametric Equalizer patch shapes the output of the Pitch Shifter patch, so it sounds more natural.
  • A Reverb patch adds reflections to the sound as if it were being played back in a large space, contributing to the ethereal vibe.

Create these patches using the menu in the Patch Editor:



Add the patches to your graph, between the Audio Player and speaker patch:



To complete the sound effect, make some adjustments to the audio effect patches:

  1. In the Pitch Shifter patch, set the Semi value to -4 to reduce the pitch.
  2. In the Parametric Equalizer set the High Frequency value to 2000, to fade the sound into the background by removing some of the higher frequencies.

In the Reverb patch:

  • Set the Mix value 100, to make the reverberation effect more prominent.
  • Set the Reflectivity High value to 15, reducing higher frequencies and making the sound less abrasive.

Learn more

Join the Spark AR Creator's community

Find inspiration, see examples, get support, and share your work with a network of creators.

Join Community

Join the Spark AR Creator's community

Find inspiration, see examples, get support, and share your work with a network of creators.

Join Community

Frequently asked questions

Have a specific question? Maybe it's been answered.


Read FAQs
Was this document helpful?