Patch Editor: Working With 3D Objects

You can use the Patch Editor in Spark AR Studio to add interactivity, logic and animation to your effects, without needing to know how to write code.

You'll do this by creating patches that represent objects, assets and their properties. You'll then connect them together in what's known as a graph.

Getting started

We're going to demonstrate the basics of using the Patch Editor using a simple 3D object. If you want to try this out yourself, download the sample content and open the unfinished project.

In then unfinished project we've added a 3D object shaped like a heart to the scene. It's listed as heart in the Scene panel.

The 3d object is seen highlighted in the Scene panel.

You can achieve all kinds of techniques with the Patch Editor - from tracking objects to precise points on the face, to creating materials. In this tutorial we'll start with the basics, making the object:

  • Appear in response to interactions.
  • Change color.
  • Animate.

Opening the Patch Editor

To open the Patch Editor:

  1. Select the Workspace icon in the Toolbar.
  2. Select Show/Hide Patch Editor.

The Patch Editor will open at the bottom of the screen:

The Patch Editor can be seen at the bottom of Spark AR Studio.

Inserting patches

Most objects, assets and properties in Spark AR Studio can be represented as patches. Let's take a look at the different ways to create them.

Objects in the Scene Panel can be dragged into the Patch Editor.

To try this out, insert a face tracker:

  1. Click +, select Face Tracker.
  2. Then, just drag it into the Patch Editor.

 Dragging the face tracker from the Scene panel to the Patch Editor.

The face tracker produces a signal in response to the movement of the face. Because of this, the patch we've created is known as a 'producer patch'.

You can use the Inspector to create patches representing the properties of an object or asset. When a property has a small circle with an arrow in next to it, it means you can make a patch to represent it.

Now create a patch representing the visible property:

  1. Select the heart 3D object in the Scene panel.
  2. Click the arrow next to Visible.

The Inspector is shown with the Visible boolean highlighted.

The patch that's been created can be used to control whether or not the heart is visible in your effect.

Patches created to represent object properties will always be at the end of the graph. This is because they consume signals created by all the other patches, to make something happen in your scene.

For this reason, they're known as consumer patches. They're always yellow.

The 'visible' consumer patch is shown in the Patch Editor.

You can create the patches that control exactly what happens in your effect in the Patch Editor itself. To do this, click Add Patch at the bottom of the Patch Editor. A menu will appear:

The 'add patch' menu is shown.

You can also right click anywhere in the Patch Editor to show this menu.

Each of the options in this menu represent a different type of patch. For example:

  1. Face Landmark patches can be connected to the face tracker patch to detect the precise position of facial features.
  2. Audio patches can be used to make sound play in response to interaction, or distort audio in your effect.
  3. Shaders can be used to build materials and apply textures.

For this example, we’re going to use an interaction patch. Interaction patches detect interactions from the person using your effect. To add one to your project:

  1. Select Interaction.
  2. Select the Mouth Open patch.
  3. Click Add Patch.

You’ll see a gray patch has been added to the Patch Editor:

The mouth open patch can be seen in the Patch Editor.

When it’s connected to the face tracker, the Mouth Open patch will detect whether the person using your effect has an open mouth or not. It'll also detect how open the person's mouth is.

Patches in Spark AR Studio

Let's take a quick look at the patches you've just created. The circles on either side are used to make connections to other patches. They're known as ports. The circles on the left are known as input ports, and on the right — output ports.

The labels on the patch give you an indication of what data the ports can pass and receive when they're connected as part of a graph:

The mouth open patch can be seen in the Patch Editor.

The Mouth Open patch can receive data from the Face port in the face tracker patch. If you select the patch itself, the labels next to each output port show what signals they can pass on to the next patch.

In this case, the Open State port will indicate whether the mouth is open or not, and the Openness port will indicate how open it is.

Connecting patches

Now let's connect the patches. You can do this by clicking and dragging a connection from an output port to an input port. Follow these steps to connect the patches:

  1. Select the faceTracker0 patch.
  2. Connect the Face port in the face tracker patch to the Face port in the Mouth Open patch.
  3. Connect the Mouth Open port in the Mouth Open patch to the Visible port in the patch representing the heart object.

You'll see the heart has disappeared. It'll now only be visible when an open mouth is detected.

Here's how the graph will look:

A patch graph with five patches

Data types

Not all ports can be connected to one another. This is because you can only connect ports with compatible data types.

You can find the data type, as well as information about what each port does, in the Patch Library:

The Spark AR Studio patch library UI.

For the Mouth Open patch, you can see that the Open State port outputs a boolean signal.

A boolean signal is always either true or false - in this case, the mouth is either open or closed. Because an object is either visible or not, these patches can be connected to each other directly.

You'll be unable to connect the Openness port to the input in the Visible patch. This is because Openness outputs a number signal, detecting how open a mouth is. It needs to drive a property that can read this signal - like a single channel of scale. You can test this by:

  1. Breaking the connection between Open State and Visible - clicking a port and pressing delete on your keyboard.
  2. Connect it to Openness instead - You'll be unable to make a connection and see an error message.

Using the Transition patch

To try out using the Mouth Openness port, create a patch representing the scale of the heart object, by clicking the arrow next to Scale in the Inspector.

The 3D scale consumer patch has been added to the Patch Editor.

We can't connect the Mouth Openness port directly to the 3D Scale port in the patch we just created, because we need to scale the heart using values relative to the values generated by the Mouth Openness port.

We can use the Transition patch to do this. To add a Transition patch:

  1. Click Add Patch in the Patch Editor.
  2. Go to Animation.
  3. Select the Transition patch from the menu.

The Transition patch can be used to convert a progress value between a start and end range of your choice. In this case we'll use it change the scale of the heart along its X-axis, Y-axis and Z-axis. Theses three axis of the 3D object are shown as arrows in the Viewport:

heart object with azis highlighted in Viewport

First, connect:

  1. The Mouth Openness port to the Progress port in the Transition patch.
  2. The output of the Transition patch to the 3D Scale input.

A patch graph with seven patches.

To finish, set the co ordinates that we want the heart to scale between, by changing the values in the Transition patch. To do this, just click the value you want to change.

For this effect edit the values next to End. Set X, Y and Z to 9. When the mouth is fully open, the heart will be 9 times bigger than its original size.

The heart is shown appearng and expanding as the user opens their mouth.

Adding logic

When making effects, you might want something to only happen sometimes, or for different things to happen based on different interactions. This is called adding logic. A basic example of this would be to show one object if someone raises their eyebrows, and a different object if someone opens their mouth.

You can combine all kinds of patches to add complex logic, but to make a fun effect that's easy for people to use and understand, it's often a case of the simpler the better.

The logic we're going to add now will make the heart change between two colors, each time it appears.

You'll change the color of the material applied to the heart using a patch that represents the material's Diffuse property. Start by creating the patch that represents the Diffuse property:

  1. Select heart_mat in the Assets panel.
  2. Click the arrow next to Texture in the Inspector.

Adding a texture consumer patch.

Now create the patches you'll use to set up the logic. In the Patch Editor menu select a:

  • Pulse patch.
  • Switch patch.
  • If Then Else patch.

A patch graph with three patches.

When connected to the Mouth Open patch, the Pulse patch will send a signal to the next patch in the graph, each time the mouth opens.

You're going to connect the Pulse patch to the Switch patch. As a result, each time the mouth is opened a switch will be turned either on or off.

You'll connect the Switch patch to the If Then Else patch, which is where you'll add the two different color options to switch between.

Changing the data type

You can change the data type of many patches in Spark AR Studio, to use them in different ways. By default the If Then Else patch's type is set to Number. This means it'll count single values.

Because we're using this patch to change the color of the material, change its Type to Color.

To do this, click the patch and a dropdown box will appear at the bottom of the patch — here you'll see a list of the different data types you can choose from. Select Color.

Changing the data type of an If Then Else patch.

Now let's take a closer look at the two inputs in the If Then Else patch.

Like we said earlier, the first time the mouth opens, the switch will be turned on. This will trigger the color we set next to the Then port to appear in the effect. The next time the mouth opens, the switch will turn off, triggering the color we set next to the Else port to appear.

To set the colors, just click in each box and select a color.

Applying colors to our If Then Else patch.

For this tutorial we’ll select pink and blue. You can pick any colors you want!

Now you're ready to connect the patches. Connect:

  1. The Open State output of the Mouth Open patch to the Pulse patch.
  2. The Turned On port at the top of the Pulse patch, to the Flip port in the Switch patch.
  3. The output of the Switch patch to the Condition input port in the If Then Else patch.
  4. The If Then Else Port to the Diffuse Texture port in the material.

A patch graph containing four patches.

Now, each time the mouth is open, the heart will change color.

Adding animation

Let's now animate the object using the Loop Animation patch. The Loop Animation and Transition patches can be used together to add a continuous looping animation between the start and end values set in the Transition patch.

From the menu in the Patch Editor, select:

  1. A Loop Animation patch.
  2. A Transition patch.

Next create a patch for the property we want to animate in the 3D object. Let's go with Rotation — so we can make the heart rotate from side to side. To create this patch:

  1. Select the heart object in the Scene Panel.
  2. Click the arrow to the left of Rotation in the Inspector.

A consumer patch for the 3D heart's 3D rotation values.

Now connect:

  1. The Progress output in Loop Animation patch to the Progress input in the Transition patch.
  2. The output of the Transition patch to the 3D Rotation port in the patch we just made.

Here's how the graph will look:

A patch graph with three patches.

Now adjust the values in the Transition patch, to set how the animation moves. It's usually a case of experimenting until you get the effect you want.

Let's make the heart rotate slightly from side to side, by editing the Y values:

  1. Next to Start, change Y to 60.
  2. Next to End, change Y to -60.

The animation is now moving from one side to the next, then starting again. If you select the box next to Mirrored in the Loop Animation patch, the heart will move back and forth.

It can be fun to experiment with the Curve option — to change how the animation moves. At the moment it's set to Linear. If you change it to say, Exponential Out, the movement will change.

When you've finished editing the patches, your graph will look like this:

The finished patch graph. It contains a loop animation patch, a transition patch and a consumer patch for 3D rotation.