Articles
People Effects
People Effects Overview

Introduction to People Effects

The effects that you can create in Spark AR Studio fall into 2 broad categories:

  • People effects — People effects are effects that respond to someone’s movement. Capabilities typically used in people effects include face, hand and body trackers, person segmentation, and deformation.
  • World effects World effects add virtual objects into real world environments. Unlike people effects, world effects usually use the back facing camera. You make world effects using the plane tracker or target tracker.

In this article, we introduce the different ways you can create people effects

The face tracker

You can use the face tracker to create effects that respond to the position and orientation of someone’s face.

You can combine a face tracker with a face mesh to create a surface that detects facial movements and expressions:

The face mesh, as seen through the Simulator and Viewport.

This example (which you can learn how to build) combines the face tracker and the face mesh to add custom stickers to a person’s cheeks:

The face mesh, as seen through the Simulator and Viewport, with the sticker texture applied.

To get started, add a face tracker to the Scene panel by selecting Add Object. Alternatively, select a face tracking effect template from the Spark AR Studio welcome screen. Face Mask, Head Decoration and Face Distortion are just a few of the templates that make use of the face tracker.

Once you’re confident with the basics, try experimenting with effects that:

Segmentation

Segmentation is used to detect and separate parts of the camera input. For example, coloring someone’s hair without changing anything else in the scene.

There are 3 types of segmentation:

  • Person segmentation — Separates the person using the effect from their background.
  • Hair segmentation — Separates a person's hair from the rest of the scene.
  • Skin segmentation — Separates any skin detected by the camera from the rest of the scene.

This effect (which you can learn how to build) uses hair segmentation and the Screen Tap patch to change someone’s hair color:

The user's hair can be seen changing color.

To get started, select Camera in the Scene panel. The camera’s properties appear in the Inspector. Click the + next to Segmentation to add either a person, hair or skin segmentation texture to the Assets panel.

Alternatively, select a segmentation effect template from the Spark AR Studio welcome screen. Background, Color Portrait and Hair Color are just a few of the templates that make use of the segmentation.

Once you’ve practiced the basics, try experimenting with effects that:

The hand tracker

Use the hand tracker to create effects that respond to the movement and position of a person’s hand. There are 2 ways to create hand tracking effects:

  • Using the hand tracker object — Objects added to your scene as a child of the hand tracker move with the user’s hand. Only one hand can be tracked in this way.
  • Using a hand bounding box — Plots an invisible rectangle around a selected hand calculating 2D coordinates for all four corners and the center of the rectangle, as well as its height and width. Yoi can track both hands with this method.

This effect (which you can learn how to build) uses the bounding box method to create a shape that appears between a user’s hands:

A hand tracking effect where a user holds a rectangle between their hands.

One of the best ways to get started with the hand tracker is to start a project using the Hand Movement template. This template is already set up to track both hands using the bounding box method.

Once you’ve built your first hand tracking effect, try experimenting with effects that:

The body tracker

Body tracking effects use body landmark patches to track features and objects to specific parts of the body. There are 2 ways to create body tracking effects:

  • Body landmark patches — Body landmark patches enable the camera to track 20 unique points on a person’s body. Objects and effects can then be anchored to those points.

  • Body bounding box patch — The body bounding box patches allow you to draw an invisible rectangle around a person’s body, calculating 2D coordinates for all four corners and the center of the rectangle, as well as its height and width. This rectangle resizes and moves based on the person’s position.

This simple effect (which you can learn how to build) uses body landmark patches to apply red squares to each trackable point on a person’s body:

specific points of the user's body are tracked as they move in front of the camera

This next effect (which you can also learn how to build) uses the Body Bounding Box and the Bounding Box 2D Unpack patches to draw a rectangle around a person’s body:

As the user moves in front of the camera, the rectangle resizes to keep all parts of her body in its boundaries.

One of the best ways to get started with body tracking is to start a project using the Body Movement template. This template is already set up to track a person’s body using the landmark patches method.

Once you’ve built your first hand tracking effect, try experimenting with effects that:

Was this tutorial helpful?