Clear
Back

Tutorial: Skeleton Animation

In this effect we've used the Patch Editor to animate a 3D object with a skeleton and joints. The object looks like a teddy.

We've connected the object to the face tracker, so the teddy moves its body and face in sync with the movement of the person using the effect - like a virtual puppet.

We've also added an extra layer of interactivity, so the clothes the teddy is wearing change color and style when the screen is tapped:



Download the sample content to follow along with this tutorial. You can adapt any of the assets in this effect in your own projects.

Learn about:

  1. What's in the unfinished effect - we've already added the 3D object, and a background.
  2. Animating 3D objects with skeletons and joints.
  3. Making the teddy's face move in sync with the user's facial movement.
  4. Using interaction to change the teddy's outfit.

In the Unfinished Effect

If you want to build this effect yourself, open the unfinished effect in the sample content folder. We've already imported the assets you need, to help you get started quickly.

We've added a simple gradient to the background of the scene, using shader patches. You can change the color of the background by adjusting the Previous Step and New Color values in the Gradient Step patch.

We've also instantiated the 3D object - teddy - in the scene, as a child of a null object. We've adjusted the Transformations in the null object, to position the teddy correctly.

The teddy contains a skeleton with several joints. You can see the skeleton inside the object in the Viewport, it's looks like grey lines.

Finally, we've applied a texture and material to the teddy. The project looks like this:

The 3D Object

Before you start building the effect it's worth taking a look at the 3D Object. It's listed as teddy in the Scene panel:



The skeleton it contains is made up of:

  1. model_grp - underneath this null object are several meshes. They create the shape of the teddy. We've already applied a texture and material to some of them. You'll add a sequence of textures to the remaining meshes, to create the appearance of the outfit.
  2. skel_root - these are the joints. You'll use patches to control how these joints move, to animate the body and the face of the teddy.

Animating the 3D Object

Follow these steps to add procedural animation to the teddy using the Patch Editor.

Moving up and down

You'll use the Position properties in two of the joints - jack_base and spring_mid - to make the teddy move up and down:

  1. Select jack_base and spring_mid in the Scene panel. You can select them both at the same time by holding down control on your keyboard.
  2. In the Inspector, click the arrow next to Position.

A blue patch will be created for the Position property of both joints. We chose the position property, because we'll use a Transition patch to set the values that the position of the joints move between, and the Loop Animation patch to animate this movement.

Right-click in the Patch Editor to open the menu. Select 2 Transition patches and a Loop Animation patch. Then connect the patches, so your graph looks like this:



You'll see the teddy is moving slightly. For the object to move correctly, you'll need to adjust the properties in the Transition and Loop Animation patches.

In the Loop Animation Patch:

  • Check the box next to Mirrored - this will mirror the movement of the teddy, so it moves up and then back down again.
  • Change the Duration to 0.25 - to speed the animation up.

In the Transition patch connected to jack_base:

  1. Next to Start, change Y to 0.08.
  2. Next to End, change X to 0.01, Y to 0.1 and Z to 0.01
  3. Change the Curve to Sinusoidal In-Out - to set the pacing of the movement.

It'll now move up and down from the base. You could experiment with adjusting these values to make the movement more or less dramatic.

In the other Transition patch:

  1. Next to Start, change Y to 0.1.
  2. Next to End, change x to 0.01, Y to 0.114 and Z to 0.01.
  3. Change the Curve to Sinusoidal In-Out.

Here's how the movement will look:



Moving side to side

You'll use the Rotation properties in sping_mid and jack_neck to make the puppet move from side to side.

  1. Select spring_mid and jack_neck in the Scene panel.
  2. In the Inspector, click the circle next to Rotation.

Again, create 2 Transition patches and a Loop Animation patch. You'll should see these 5 patches:



You're going to connect a patch representing the face tracker to this part of the graph. This will make the movement of the teddy mirror the movement of the person using the effect, like a puppet. To create this patch:

  1. Click Add Object at the bottom of the Scene panel.
  2. Select Face Tracker.
  3. Drag the face tracker from the Scene panel into the Patch Editor.

Three patches will be added:



For this part of the graph you'll need to add 4 more patches from the menu in the Patch Edition: A Delay patch, a Multiply patch and and 2 Add patches.

You'll use the Delay patch to add a slight delay to the movement of one of the joints - to create a more natural effect. The Add patches will add the data from the face tracker to the movement of the teddy, while the Multiply patch will control how dramatic the movement of the teddy is.

Start by connecting:

  1. The Progress output in the Loop Animation patch to the Delay patch.
  2. The Delay patch to the Progress input in one of the Transition patches.
  3. The output of the Transition patch to the second input in the Add patch. You'll need to set the data type of the Add patch to Vector 3, using the menu at the bottom of the patch.
  4. The output of the Add patch to the 3D Rotation input in jack_neck.

Here's how the graph will look:



Now edit the values in the patches:

  1. In Loop Animation, check the box next to Mirrored.
  2. In the Transition patch, next to Start set X to -15, and next to End set Z to 15.
  3. Change the Curve to Sinusiodal In-Out.

The teddy's head will be moving from side to side:



Next connect the remaining Transition and Add patches to the Loop Animation patch and the patch representing spring_mid joint.



In the Transition patch, set:

  1. The Z value next to Start to -10.
  2. The Z value next to End to 10.
  3. Curve to Sinusiodal In-Out.

Finally, connect the face tracker. Connect the 3D Rotation output in the face tracker patch to the first input in the Add patch connected to jack_neck.



Then connect the 3D Rotation output in the face tracker patch to the first input in the Multiply patch. Like with the Add patches, make sure the Multiply patch is set to Vector 3. To finish this part of the graph:

  1. Change all 3 values at the bottom of the Multiply patch to 0.5.
  2. Connect the Multiply patch to the top input in the remaining Add patch.

Here's how the new section of the graph will look:



The teddy's head will now match the movement of the person using the effect. You test this out on yourself my changing the video in the Simulator to your own camera.

Raising the teddy's arms

This section of the graph will animate the movement of the teddy's arms.

It uses the Rotation property in the joints called wrist_r, wrist_l, arm_l and arm_r. These joints are children of the null objects listed under jack_base in the Scene panel.

So, like we did before, start by creating patches to represent the Rotation property of wrist_r, wrist_l, arm_l and arm_r.

From the menu in the Patch Editor, select:

  1. A Loop Animation patch.
  2. A Multiply patch.
  3. A Pack patch.
  4. 2 Transition patches.

Connect them, so your graph looks like this:



Again, make a few changes to the Transition patches:

  1. In each Transition patch, set the Curve to Sinsusoidal In-Out.
  2. For the Transition patch at the top of the graph, the End position to -20 next to Z.
  3. For the Transition patch at the bottom of the graph, the End position to -30 next to Z.

In the Loop Animation patch:

  1. Check the box next to Mirrored.
  2. Set the Duration to 0.25.

In the next part of this tutorial, we'll connect this part of the graph to a patch that will detect an open mouth. As a result, this animation will become more dramatic when the user of the effect opens their mouth. That's why we need the Multiply patch - to multiply the signal from the Mouth Open patch, creating a more dramatic effect.

For now, in the Multiply patch change the lower input value to 30.

Responding to Facial Movement

Now let's make the movement of the teddy's face match the movement of the user.

Responding to an open mouth

This part of the graph will make the teddy's mouth open, at the same time as the mouth of the person using the effect. It uses the Rotation property in the joint called mouth_lower, which is a child of jack_neck in the Scene panel:

  1. Select mouth_lower in the Scene panel.
  2. In the Inspector, click the circle next to Rotation to create a patch.

From the menu in the Patch Editor, select:

  1. The Mouth Open patch.
  2. The Multiply patch - you'll need to change the Type to Vector 3.

Connect the patches, so the section of the graph looks like this:



The final step is to add values to the Multiply patch - to set how much the mouth should open. Set X on the bottom row of the Multiply patch to 20.

The teddy's mouth should open at the same time as the user:



Now we'll connect the Mouth Open patch to the part of the graph that animates the teddy's arms. Connect the Mouth Openness port to the top input in the Multiply patch created earlier. We've selected the patches you'll need to connect in the image below:

Making the teddy blink

Start by selecting an Eyelid patch from the menu in the Patch Editor. This patch captures data on the position of someone's eyelids - so it always needs to be connected to the Face port in the face tracker patch.

Next:

  1. Create patches to represent the Scale properties of the joints eye_l and eye_r. The Teddy's eyes will disappear - because their scale is now being controlled by the patches.
  2. Select a Subtract and Pack patch from the menu in the Patch Editor.

Connect the patches, so the section of the graph looks like this:



You'll notice we've only used the Left Openness port in the Eyelid patch. We've then used Subtract and Pack to make this signal drive the scale of both eyes. This is because if we used both the left and right eye openness ports, the eyes would likely appear to blink at very slightly different times, in sync with the person using the effect. We felt that for this effect making both eyes move at exactly the same time would look better.

You'll need to make some changes to the values in the patches:

  1. Set the Subtract input to 0.1. This will make the teddy's eyes completely close when the person viewing the effect blinks. If you didn't do this, they'd stay a tiny bit open.
  2. Set both Pack inputs to 1. This will maintain a uniform scale on the X and Z axis.

The teddy's eyes should blink at the same time as yours.

Moving the eyebrows

This graph will make the teddy's eyebrows move at the same time as the person using the effect.

Select an Eyebrow patch from the menu in the Patch Editor. This patch captures data on the position of someone's eyebrows.

Next:

  1. Create patches to represent the Position properties of the joints brow_r and brow_l.
  2. Select two Add patches from the menu in the Patch Editor.
  3. Change the Type of the Add patch to Point 3D - because we're working with a 3D object.

Connect the patches so the section of the graph looks like this:



For each Add patch, set the Y value to 0.07 and the Z value to 0.012. This sets the co-ordinates that each eyebrow moves between. You could experiment with changing these values for a more or less dramatic effect.

The teddy's eyebrows should move at the same time as yours!

To add some extra interest, let's also use the 3D Rotation port in the facetracker patch to drive the movement of the teddy's body. Connect the 3D Rotation output in the face tracker to the input in the Multiply patch you added earlier. In the image below, we've selected the 4 patches that should be connected:



Now we can move on to the teddy's outfit!

Changing the Teddy's Outfit

This effect uses a patch group to change the number on the teddy's vest and hat, and the color of the outfit when the user taps the screen. You can test this out in the finished effect, by selecting Simulate Touch in the Simulator:



The different numbers that appear on the teddy's vest are textures, imported as a sequence. They're listed in the Assets panel under Animations, as inventory_tex.-animation.

The patch group is listed in the assets panel under Patches, it's called Teddy_outfit.

Drag the patch group into the Patch Editor to create a patch:



You can click the button in the right corner of the patch to expand the group.

If you want, you can change the colors in the Option Picker to choose different colors for the effect:



You'll notice the colors in the option picker are darker than the colors you can see in the finished effect. This is because the texture we'll apply to the teddy will be combined with these colors. The texture has a light area covering the teddy's shirt and hat, which makes the outfit lighter.

Click Back to Main in the top left of the Patch Editor to return to the rest of your graph:



The patch group has 2 input ports:

  • Tex Input - this is where you'll connect the texture.
  • Decal - this is where you'll connect the animation sequence. The logic in the patch group will cause this to change when the screen is tapped.

It has 2 outputs:

  • Output - you'll connect this to a patch representing the Diffuse Texture property in the material.
  • Decal Frame Output - you'll connect this to a patch representing the Current Frame in the animation.

From the Assets panel, drag teddy_tex and inventory_tex into the Patch Editor. Two orange patches will be created, representing the texture and the animation sequence:



To create a patch representing the material we want to apply the texture to, select teddy_outfit_mat in the Assets panel. In the Inspector:

  1. Go to Diffuse under Shader Properties.
  2. Click the arrow next to Texture.

Next we need a patch representing the Current Frame property of the animation sequence. Select inventory_tex.-animation in the Assets panel. In the Inspector:

  1. Go to Advanced.
  2. Click the arrow next to Current Frame.

Connect the patches, so your graph looks like this:



If you tap the teddy in the Simulator, you'll see the outfit and color changes.

You've now finished the effect!

Join the Spark AR Creator's community

Find inspiration, see examples, get support, and share your work with a network of creators.

Join Community

Join the Spark AR Creator's community

Find inspiration, see examples, get support, and share your work with a network of creators.

Join Community

Frequently asked questions

Have a specific question? Maybe it's been answered.


Read FAQs
Was this document helpful?