Animating the 3D Object

Follow these steps to add procedural animation to the teddy using the Patch Editor.

Moving up and down

You'll use the Position properties in two of the joints - jack_base and spring_mid - to make the teddy move up and down:

1. Select jack_base and spring_mid in the Scene panel. You can select them both at the same time by holding down control on your keyboard.
2. In the Inspector, click the arrow next to Position.

A blue patch will be created for the Position property of both joints. We chose the position property, because we'll use a Transition patch to set the values that the position of the joints move between, and the Loop Animation patch to animate this movement.

Right-click in the Patch Editor to open the menu. Select 2 Transition patches and a Loop Animation patch. Then connect the patches, so your graph looks like this:

You'll see the teddy is moving slightly. For the object to move correctly, you'll need to adjust the properties in the Transition and Loop Animation patches.

In the Loop Animation Patch:

1. Check the box next to Mirrored - this will mirror the movement of the teddy, so it moves up and then back down again.
2. Change the Duration to 0.25 - to speed the animation up.

In the Transition patch connected to jack_base:

1. Next to Start, change Y to 0.08.
2. Next to End, change X to 0.01, Y to 0.1 and Z to 0.01
3. Change the Curve to Sinusoidal In-Out - to set the pacing of the movement.

It'll now move up and down from the base. You could experiment with adjusting these values to make the movement more or less dramatic.

In the other Transition patch:

1. Next to Start, change Y to 0.1.
2. Next to End, change x to 0.01, Y to 0.114 and Z to 0.01.
3. Change the Curve to Sinusoidal In-Out.

Here's how the movement will look:

Moving side to side

You'll use the Rotation properties in sping_mid and jack_neck to make the puppet move from side to side.

1. Select spring_mid and jack_neck in the Scene panel.
2. In the Inspector, click the circle next to Rotation.

Again, create 2 Transition patches and a Loop Animation patch. You'll should see these 5 patches:

You're going to connect a patch representing the face tracker to this part of the graph. This will make the movement of the teddy mirror the movement of the person using the effect, like a puppet. To create this patch:

1. Click Add Object at the bottom of the Scene panel.
2. Select Face Tracker.
3. Drag the face tracker from the Scene panel into the Patch Editor.

Three patches will be added:

For this part of the graph you'll need to add 4 more patches from the menu in the Patch Edition: A Delay patch, a Multiply patch and and 2 Add patches.

You'll use the Delay patch to add a slight delay to the movement of one of the joints - to create a more natural effect. The Add patches will add the data from the face tracker to the movement of the teddy, while the Multiply patch will control how dramatic the movement of the teddy is.

Start by connecting:

1. The Progress output in the Loop Animation patch to the Delay patch.
2. The Delay patch to the Progress input in one of the Transition patches.
3. The output of the Transition patch to the second input in the Add patch. You'll need to set the data type of the Add patch to Vector 3, using the menu at the bottom of the patch.
4. The output of the Add patch to the 3D Rotation input in jack_neck.

Here's how the graph will look:

Now edit the values in the patches:

1. In Loop Animation, check the box next to Mirrored.
2. In the Transition patch, next to Start set X to -15, and next to End set Z to 15.
3. Change the Curve to Sinusiodal In-Out.

The teddy's head will be moving from side to side:

Next connect the remaining Transition and Add patches to the Loop Animation patch and the patch representing spring_mid joint.

In the Transition patch, set:

1. The Z value next to Start to -10.
2. The Z value next to End to 10.
3. Curve to Sinusiodal In-Out.

Finally, connect the face tracker. Connect the 3D Rotation output in the face tracker patch to the first input in the Add patch connected to jack_neck.

Then connect the 3D Rotation output in the face tracker patch to the first input in the Multiply patch. Like with the Add patches, make sure the Multiply patch is set to Vector 3. To finish this part of the graph:

1. Change all 3 values at the bottom of the Multiply patch to 0.5.
2. Connect the Multiply patch to the top input in the remaining Add patch.

Here's how the new section of the graph will look:

The teddy's head will now match the movement of the person using the effect. You test this out on yourself my changing the video in the Simulator to your own camera.

Raising the teddy's arms

This section of the graph will animate the movement of the teddy's arms.

It uses the Rotation property in the joints called wrist_r, wrist_l, arm_l and arm_r. These joints are children of the null objects listed under jack_base in the Scene panel.

So, like we did before, start by creating patches to represent the Rotation property of wrist_r, wrist_l, arm_l and arm_r.

From the menu in the Patch Editor, select:

1. A Loop Animation patch.
2. A Multiply patch.
3. A Pack patch.
4. 2 Transition patches.

Connect them, so your graph looks like this:

Again, make a few changes to the Transition patches:

1. In each Transition patch, set the Curve to Sinsusoidal In-Out.
2. For the Transition patch at the top of the graph, the End position to -20 next to Z.
3. For the Transition patch at the bottom of the graph, the End position to -30 next to Z.

In the Loop Animation patch:

1. Check the box next to Mirrored.
2. Set the Duration to 0.25.

In the next part of this tutorial, we'll connect this part of the graph to a patch that will detect an open mouth. As a result, this animation will become more dramatic when the user of the effect opens their mouth. That's why we need the Multiply patch - to multiply the signal from the Mouth Open patch, creating a more dramatic effect.

For now, in the Multiply patch change the lower input value to 30.

Responding to Facial Movement

Now let's make the movement of the teddy's face match the movement of the user.

Responding to an open mouth

This part of the graph will make the teddy's mouth open, at the same time as the mouth of the person using the effect. It uses the Rotation property in the joint called mouth_lower, which is a child of jack_neck in the Scene panel:

1. Select mouth_lower in the Scene panel.
2. In the Inspector, click the circle next to Rotation to create a patch.

From the menu in the Patch Editor, select:

1. The Mouth Open patch.
2. The Multiply patch - you'll need to change the Type to Vector 3.

Connect the patches, so the section of the graph looks like this:

The final step is to add values to the Multiply patch - to set how much the mouth should open. Set X on the bottom row of the Multiply patch to 20.

The teddy's mouth should open at the same time as the user:

Now we'll connect the Mouth Open patch to the part of the graph that animates the teddy's arms. Connect the Mouth Openness port to the top input in the Multiply patch created earlier. We've selected the patches you'll need to connect in the image below:

Making the teddy blink

Start by selecting an Eyelid patch from the menu in the Patch Editor. This patch captures data on the position of someone's eyelids - so it always needs to be connected to the Face port in the face tracker patch.

Next:

1. Create patches to represent the Scale properties of the joints eye_l and eye_r. The Teddy's eyes will disappear - because their scale is now being controlled by the patches.
2. Select a Subtract and Pack patch from the menu in the Patch Editor.

Connect the patches, so the section of the graph looks like this:

You'll notice we've only used the Left Openness port in the Eyelid patch. We've then used Subtract and Pack to make this signal drive the scale of both eyes. This is because if we used both the left and right eye openness ports, the eyes would likely appear to blink at very slightly different times, in sync with the person using the effect. We felt that for this effect making both eyes move at exactly the same time would look better.

You'll need to make some changes to the values in the patches:

1. Set the Subtract input to 0.1. This will make the teddy's eyes completely close when the person viewing the effect blinks. If you didn't do this, they'd stay a tiny bit open.
2. Set both Pack inputs to 1. This will maintain a uniform scale on the X and Z axis.

The teddy's eyes should blink at the same time as yours.

Moving the eyebrows

This graph will make the teddy's eyebrows move at the same time as the person using the effect.

Select an Eyebrow patch from the menu in the Patch Editor. This patch captures data on the position of someone's eyebrows.

Next:

1. Create patches to represent the Position properties of the joints brow_r and brow_l.
2. Select two Add patches from the menu in the Patch Editor.
3. Change the Type of the Add patch to Point 3D - because we're working with a 3D object.

Connect the patches so the section of the graph looks like this:

For each Add patch, set the Y value to 0.07 and the Z value to 0.012. This sets the co-ordinates that each eyebrow moves between. You could experiment with changing these values for a more or less dramatic effect.

The teddy's eyebrows should move at the same time as yours!

To add some extra interest, let's also use the 3D Rotation port in the facetracker patch to drive the movement of the teddy's body. Connect the 3D Rotation output in the face tracker to the input in the Multiply patch you added earlier. In the image below, we've selected the 4 patches that should be connected:

Now we can move on to the teddy's outfit!

Changing the Teddy's Outfit

This effect uses a patch group to change the number on the teddy's vest and hat, and the color of the outfit when the user taps the screen. You can test this out in the finished effect, by selecting Simulate Touch in the Simulator:

The different numbers that appear on the teddy's vest are textures, imported as a sequence. They're listed in the Assets panel under Animations, as inventory_tex.-animation.

The patch group is listed in the assets panel under Patches, it's called Teddy_outfit.

Drag the patch group into the Patch Editor to create a patch:

You can click the button in the right corner of the patch to expand the group.

If you want, you can change the colors in the Option Picker to choose different colors for the effect:

You'll notice the colors in the option picker are darker than the colors you can see in the finished effect. This is because the texture we'll apply to the teddy will be combined with these colors. The texture has a light area covering the teddy's shirt and hat, which makes the outfit lighter.

Click Back to Main in the top left of the Patch Editor to return to the rest of your graph:

The patch group has 2 input ports:

• Tex Input - this is where you'll connect the texture.
• Decal - this is where you'll connect the animation sequence. The logic in the patch group will cause this to change when the screen is tapped.

It has 2 outputs:

• Output - you'll connect this to a patch representing the Diffuse Texture property in the material.
• Decal Frame Output - you'll connect this to a patch representing the Current Frame in the animation.

From the Assets panel, drag teddy_tex and inventory_tex into the Patch Editor. Two orange patches will be created, representing the texture and the animation sequence:

To create a patch representing the material we want to apply the texture to, select teddy_outfit_mat in the Assets panel. In the Inspector:

1. Go to Diffuse under Shader Properties.
2. Click the arrow next to Texture.

Next we need a patch representing the Current Frame property of the animation sequence. Select inventory_tex.-animation in the Assets panel. In the Inspector:

1. Go to Advanced.
2. Click the arrow next to Current Frame.

Connect the patches, so your graph looks like this:

If you tap the teddy in the Simulator, you'll see the outfit and color changes.

Adding Audio

To complete the effect, add a little extra fun and interest with an audio clip. You’ll set it to play on a loop in the background of the effect:

Something Went Wrong

We're having trouble playing this video.

Learn More

We’ve already added the clip you’ll need, it’s listed in the Assets panel under Audio as Skeleton_Animation_Groove.m4:

Use an object called a speaker and an asset called the audio playback controller to render it in the scene. First add the speaker:

1. Click Add Object.
2. SelectSpeaker from the menu.
3. In the Inspector, set the Volume of the speaker to 50%, so it’s not too prominent in the effect.

Next add the audio playback controller:

1. Click Add Asset.
2. Select Audio Playback Controller from the menu.

To render the audio clip in the scene, connect the audio file, audio playback controller and speaker:

1. Select speaker0 in the Scene panel.
2. In the Inspector, click the dropdown next to Audio and select audioPlaybackController0 from the menu.
3. Select audioPlaybackController0 in the Assets panel.
4. In the Inspector, click the dropdown next to Audo and select Skeleton_Animation_Groove.m4a from the menu.

The audio playback controller is where you’ll set how the audio file plays in the effect. To make it loop continuously:

1. Select audioPlaybackController0 in the Assets panel.
2. In the Inspector, check the box next to Loop.

The audio clip will now be playing! You might need to click the Refresh button in the toolbar to hear it.

You’ve now completed the effect!