As a result of the coronavirus (COVID-19) outbreak, we have a reduced capacity to review effects and delays in publishing are expected. Learn more.
Use segmentation in Spark AR Studio to create an AR effect that separates someone from their background. You can then transform the environment behind them using textures and objects. Learn about:
You can also follow this video tutorial to build your own segmentation effect.
Segmentation in Spark AR Studio:
You'll need to add 2 rectangles to your scene. One will be used to render the user in the foreground of your scene, and the other will create the background.
Start by adding a canvas:
The canvas is always the same size as the device that's using the effect. You'll insert the rectangles as children of the canvas, so they'll always be the right size for the device. To do this:
Repeat these steps, so you have two rectangles in your scene. It's worth renaming the the rectangles to help keep track. To rename an object in Spark AR Studio:
Your Scene panel should look like this:
Next change the size of the rectangles, so they're the same size as the canvas:
Your project should look like this:
The background rectangle needs to be added to another layer, so it renders after the foreground rectangle. To do this:
In the Layers tab you can adjust the order of the layers to change the order they render in. The new layer will be set to render last by default, so you won't need to make any changes here.
You'll apply a segmentation texture to the material on the user rectangle. You can add a color or your own textures to the material on the background rectangle.
To create the materials:
Repeat these steps for the background rectangle. This time, rename the material background_material.
The segmentation texture separates what the camera can see in the foreground, from the background. To create it:
Extracting the camera texture allows you to use the video captured by the camera as the effect plays as a texture in the effect. You'll add this texture to the user_material, to render the user in the scene. To create it:
Select the user material in the Assets panel. In the Inspector:
You should see the user in your scene:
Select background_material. You can make any edits you want to this material - for example, import your own texture or change the color. Below, we've edited the color under Diffuse:
When the segmentation texture is selected in the Assets panel, you can make changes how it works in your effect in the Inspector.
Segmentation creates an outline around the person identified in the scene. Use Edge Softness to blur or soften this outline.
Adjust the outline of the person in the scene.
We've found that effects with both segmentation and the face tracker don't perform well. It's best to avoid using these capabilities in the same effect.
If there are no 3D objects in your segmentation effect, you can remove the ambient and directional light from the Scene panel. These lights are included in Spark AR Studio projects by default to provide realistic lighting to 3D objects.