Tracking the 3D objects to the eyes

Open the Patch Editor and drag faceTracker0 from the Scene panel into the Patch Editor. You'll see a group of 3 patches: Face Finder, Face Select and faceTracker0.

The Face Finder patch is usually used to detect and follow the entire face. In this effect we need to capture the positioning and orientation of the eyes. To do this, you’ll need to attach an Eyeball patch:

1. From the Patch Editor menu, select Face Landmarks.
2. Select the Eyeball
3. Click Add Patch.
4. Connect the Face output port of faceTracker0 to the Face input port of the Eyeball patch.

Here's how your graph will look:

The Eyeball patch has 6 outputs. For this effect you'll use the following ports:

• Left and Right Eyeball Center Position: the position of the center point of the eyeball (i.e. the center of the sphere), relative to the face tracker.
• Left and Right Eyeball Rotation: the 3D rotation of the eyeball (e.g. if the person is looking down, then this rotation will be rotated with a positive angle around the X axis).

You'll need to connect these ports to patches that represent the Position and Rotation properties of the left and right eyeball 3D objects:

To do this:

1. Select the eyeball_left and eyeball_right objects in the Scene panel.
2. In the Inspector, click the arrows next to both their Position and Rotation properties.

This will create 4 yellow patches, to represent these values:

The Eyeball Position ports in the Eyeball patch to the 3D Position ports of their corresponding object patches.

The Eyeball Rotation ports in the Eyeball patch to the 3D Rotation ports of their corresponding object patches.

Your finished graph will look like this:

The 3D eyeball objects should cover and track the eyes:

Applying textures to change the appearance of the eyes

To apply the the texture included in this tutorial:

• Select eyeball_mat in the Assets panel.
• Select the dropdown to the right of Texture in the Inspector.
• Select the eyeball texture.

Your effect will look like this:

Using the Eyeball Shader

In the Assets panel there’s a patch asset called Eyeball Shader. This is made of several visual shader patch graphs. The patch asset can be used to apply a more complex texture to the material covering the eyeball 3D objects.

You can easily customize this texture by editing the values in the Eyeball Shader to change how the eyes look.

Removing the Eyeball texture

Start off by removing the eyeball texture already applied to the material.

1. In the Assets panel, select eyeball_mat.
2. In the Inspector select the dropdown next to Texture and select Remove.

Connecting the patch asset to the material

In the Patch Editor:

1. Drag the Eyeball Shader into the Patch Editor.
2. Select eyeball_mat and under Diffuse in the Inspector, click the arrow next to Texture to create a patch representing the diffuse texture of the eyeball material.
3. Connect the Texture output port to the input port of the Diffuse Texture patch.

Your graph will look like this:

Customizing the material

You can easily customize your effect by editing the values in the Eyeball Shader to change attributes such as Iris Color, Pupil Color and Opacity. For example, to change the Iris Color:

1. In the Eyeball Shader click the rectangles next to the Iris Color A, B and C values.
2. Choose a color from the color wheel for each value and click OK when you’re happy with how your effect looks.

If you choose to keep the original values in the Eyeball Shader your final effect will look like this!