2D Interactive World Effect

The user is seen in a polaroid photo frame. When a tap is simulated, a different color filter is applied.

In this tutorial you'll build a world effect, visible through the back camera of a mobile device. In the effect, a 2D photo with dynamic text shows the date and location. You'll use the Patch Editor to add interactivity, so the frame changes color and a clicking sound plays when it's tapped:

Download the sample content to follow this tutorial. You can adapt any of the assets in this effect in your own projects.

To get started open the unfinished effect in the sample content folder. We've already imported the assets you need, to help you get started quickly. We've also removed the directional and ambient lighting from the scene. This is a good idea when you're creating effects with 2D objects, as they don't need lighting to add depth to objects.

Creating the photo frame

This project uses two rectangles and a 2D Text object to create the frame. One rectangle contains the custom texture and the other contains the colored overlay.

Adding the 2D objects

We're going to add all the objects at the same time and then assign them to layers so you can clearly see what you're building.

Insert:

A canvas. You'll use this to group the other objects, so you can transform them all at the same time.
2 rectangles, both as children of the canvas.
A 2D text object, again as a child of the canvas.

Your project should look like this:

Untextured rectangles and a 2d text object can be seen in the Viewport and Simulator.

It's worth renaming the objects to help you keep track. Rename:

The first rectangle rect_frame.
The second rectangle rect_overlay.
The text object dynamic_text.

Layers

You'll need 2 layers in this effect, to control the order objects render in.

There will already be 1 layer in the project, named layer0. In the Layers panel, create 1 more.

To keep track of layers, it's a good idea to rename them. We've called:

The first layer frame_layer.
The second layer color_overlay_layer.

Back in the Scene panel, assign:

canvas0, dynamic_text and rect_frame to frame_layer.
rect_overlay to color_overlay_layer.

Editing the canvas

First you'll need to make a couple of edits to the canvas. Because the rectangles are children of the canvas, they will take on any changes made to it.

In the Inspector, change the Mode from Camera Space to World Space.

The mode setting is highlighted in the Inspector.

Because the canvas is in World Space it will no longer scale with the screen of the device. This allows you to change its size and position. In the Inspector, change:

The Position to -0.013 on the X axis - the canvas will move to the side slightly.
The Scale to 0.0005 for the X, Y and Z values - the canvas will get slightly smaller.
The Rotation to 5 for the X, Y and Z values - the canvas will tilt slightly.

The canvas and its child objects have been moved and rotated.

Editing the rectangles

Editing the rectangles will make them the right size, and position them in the correct place. Start by setting Flexibility, so the rectangles scale with the screen of the device:

Select the rectangles in the Scenel panel.
In the Inspector, set Flexibility for both height and width.

To adjust the size of the rectangles:

Go to Width and Height in the Inspector.
For rect_frame, set the Width to 450 and the Height to 555.
For rect_overlay, set the Width to 375 and Height to 432.

The rectangles are the correct size now:

The rectangles now cover the user's face.

At the moment, one rectangle covers the other. To adjust this, In the Inspector change the Position values:

For rect_frame, set Y to 10.
For rect_overlay, set X to 36 and Y to 40.

Creating materials

To build the appearance of the frame, start by creating a material for each rectangle:

Select a rectangle in the Scene panel.
In the Inspector, click + next to Material.

In the Assets panel, rename:

The material you've made for rect_frame, frame_mat.
The material you've made for rect_overlay, overlay_mat.

Editing the materials

Set the Shader Type to Flat for both materials, because we don't need them to omit or respond to light. It's more performant than a standard shader, so it helps keep the size of the effect down.

For frame_mat:

Below Diffuse, click the dropdown next to Texture, apply the texture frame.
Check the box for Alpha Test and adjust the Cutoff slider to make some semi-transparent parts of the texture transparent. We set it to 50%.

The alpha settings are highlighted in the Inspector.

Now select overlay_mat. In the Inspector:

Go to Render Options.
Set Opacity to 30%, to create the semi-transparent effect.

The frame is taking shape now:

The user can now be seen through the center of the frame.

Overlay_mat changes color when the screen is tapped. You'll use the Patch Editor to do this. Start by creating clicking the arrow next to Texture in the Inspector to make a patch representing the Diffuse Texture property of the material.

The material's texture settings are highlighted in the inspector.

We'll add the other patches later.

Adding the dynamic text

The text in this effect is a dynamic text object. Dynamic text takes data from a user's device, like their location or the time, and represents it as text.

To add the dynamic text:

Select dynamic_text in the Scene panel.
In the Inspector, delete the text in the Text Box.
Next to Dynamic Text, click Insert.
Insert 2 dynamic text tokens from the menu - Current City and Date (Medium).

There are lots of other types of dynamic text to choose from when creating your own effects.

Next change:

Font to Facebook Script Light.
Font Size to 40.
Line Spacing to 3000.
Position to 38 on the X axis and 450 on the Y axis.

You'll see the text is in the middle of the frame. To fix this, change the Width and Height values:

Go to the Inspector.
Set the Width to 375 and Height to 50.

Pinning the text means it will always keep the same distance from the canvas's border. Next to Pinning in the Inspector, pin the text to the top, bottom and right side of the canvas. Make sure the rectangle isn’t pinned to the top or bottom of the canvas. This will maintain a relative distance to the center of the canvas on the Y axis, keeping the text under the overlay.

Adding Interactivity and Logic

Now we'll make the color of the frame change in response to a tap on the screen of the device.

You've already created the frame_color_mat patch. Create the other 3 patches by right-clicking anywhere in the Patch Editor, and choosing them from the menu. Select:

A Screen Tap patch, to detect when the screen is tapped - you could use another interaction patch on the menu.
A Counter patch.
An Option Picker patch.

Adding the colors

You'll add the colors to the Option Picker. First change the Option Picker's data type to Color:

Select the Option Picker.
Select Color from the menu at the bottom of the patch.

To add the colors, click the squares in the Option Picker. Choose a color for each one from the menu:

Colors are being added to the options picker.

Connecting the patches

Connect:

The Tap output in the Screen Tap patch to the Increase input in the Counter patch.
The output of the Counter patch to the Option input in the Option Picker.
The Option Picker to the patch representing the material.

You can test the effect in the Simulator, by clicking the gear and selecting Simulate Touch:

As taps are simulated, the filter changes color.

Adding audio

To finish the effect, add a sound that plays each time the screen is tapped. This is known as 'one-shot' audio:

Something Went Wrong

We're having trouble playing this video.

Learn more

This audio effect is easy to create, all you’ll do is add 4 more patches to your graph.

We’ve already added the audio file you’ll need to the project - it’s listed as arl_sfx_UITick_os_01.m4a in the Assets panel

Add a speaker

To render sound in any effect, you’ll need a scene object called a speaker. To add it to the project:

Click Add Object.
Select Speaker from the menu.

The network of patches you’ll build will send a signal to a patch representing the speaker, playing the sound in the scene.

To create a patch for the speaker:

In the Scene panel, select speaker0.
Go to the Inspector.
Click the arrow to the left of Audio.

A yellow patch will be added to the Patch Editor:

Adding the one-shot audio effect

To do this you’ll connect the Screen Tap patch to the speaker patch, using a Multi-Clip Controller patch and Audio Player patch to play this audio file through the speaker.

The Multi-Clip Controller patch can play the same sound multiple times, overlapping with previous playback instances. With longer audio clips, it’s worth experimenting with the Single-Clip Controller instead, which will stop the previous playback instance before playing a new one.

To create these patches, right-click in the Patch Editor to open the menu. Select the:

Multi-Clip Controller patch.
Audio Player patch.

You’ll also need a patch representing the audio file. To create it, just select arl_sfx_UITick_os_01.m4a in the Assets panel and drag it into the Patch Editor. An orange patch will be added:

Now connect the patches. Connect:

The Tap output of the Screen Tap patch to the Play input in the Multi-Clip Controller patch.
The output of the Multi-Clip Controller patch to the Controller input in the Audio Player patch.
The output of the Audio Player patch to the Audio input in the yellow patch representing the speaker.
The output of the orange patch representing the audio clip to the Audio Clip input in the Audio Player patch.

Here’s how the graph will look:

You’ve now added the audio clip to your effect! You can test it by clicking in the Simulator. You might need to click the Refresh button in the toolbar to hear it.

Tapping the screen in this effect is a subtle interaction, so the sound shouldn’t stand out too much. Complete the effect by adjusting the volume:

In the Scene panel, select speaker0.
Go to the Inspector.
Change Volume to 50%, using the slider.

Learn More

INTERMEDIATE

10 MINS

Patch Editor: Use Face Movements to Control Animation

Use interaction patches animate objects in your effect.

INTERMEDIATE

1 HOUR

Immersive Floating Particles

Combine segmentation, particles and interactivity to create a world effect.

INTERMEDIATE

10 MINS

Patch Editor: Using the Runtime Patch

Make something happen when your effect has been open for a set amout to time.