In this tutorial you'll learn how to use segmentation and the camera texture to create an effect that separates someone from their background.
You'll use a segmentation mask texture to divide the foreground and background of the scene. You'll then apply the camera texture to a material in the foreground - to stream a video of the person using the effect back into the scene. You can then transform the environment behind them.
You don't need any custom assets to create this effect. All you need to do is open Spark AR Studio to get started.
Start by creating the segmentation mask texture. This texture separates the foreground - with the user in it - from their background.
To do this:
You'll now see the texture in the Assets panel - it's called segmentationMaskTexture0.
Next create the camera texture:
You'll see a texture called cameraTexture0 in the Assets panel.
To render the user in the scene and create the background, you'll use two 2D objects called rectangles.
One rectangle will have a material with the camera texture and segmentation mask texture applied to it, creating the foreground. The other will have a single material applied to it, creating a background.
Rectangles are always children of a canvas. In this effect, this will mean the rectangles rescale with the screen of the device.
Start by making the rectangle that will form the background:
It's a good idea to rename the rectangle, to help you keep track. To do this just:
In the Viewport you can see that the rectangle is quite small, and positioned in the middle of the scene. We want it to fill the whole canvas and therefore the full screen of the device.
To do this, go over to the Inspector panel. Next to Size you'll see two options. You can use the first one to set the width of the rectangle, and the second one to set the height. Click in the first box and select the option for Fill Width.
Do the same for the height option:
Next create a material for the rectangle. At the bottom of the Inspector:
The material will be listed in the Assets panel. Right-click on the material and rename it - we suggest background_material.
You can achieve all kinds of effects with segmentation - for example using your own custom textures for the background. For this tutorial, we're going to keep it simple and pick a color.
To do this, select the material. In the Inspector panel, change the Shader Type to Flat.
Then, under Diffuse:
Now let's build the foreground, with the user in it.
Right-click on the canvas and add another rectangle. Like before, rename the rectangle - go with something like user_material. In the Inspector, again select Fill Width and Fill Height next to Size.
Create a material for the rectangle and select it in the Assets panel - it'll be listed as Material0. Like before, rename it. You could choose something like user_material. Over in the Inspector change the Shader Type to Flat
Then, under Diffuse:
The user has appeared in the scene, but we can't see the background. This is because we haven't applied the segmentation mask texture yet.
To apply the segmentation mask texture: