If you’ve followed our basic audio tutorial, you’ll have experimented with adding a looping sound using the audio playback controller and speaker.
In this tutorial you’ll use the Audio Analyzer and Energy Meter patches to add complexity and interest to an effect with audio:
These patches will take information from an audio clip, outputting signals to animate 2D objects in the scene.
You could create this effect using the Instagram Music patch instead of an imported audio file, to let people using your effect choose the track.
Download the sample content to follow along. You’ll get an unfinished finished effect. In the unfinished effect we’ve already added the objects you’ll animate. You’ll add the audio component in this tutorial.
To get started open the unfinished project. In the Viewport and Simulator you’ll see we’ve already added, positioned and resized 9 rectangles:
We’ve added a flat material with a color to each rectangle for visual interest. They’re children of a canvas in the Scene panel, which means the rectangles can be set to scale up or down with the device screen.
The animation of:
It’s easy to customize this effect by swapping the rectangles for your own objects and textures.
To render the audio clip so it plays in the scene, connect it to a scene object called a speaker. The simplest way to do this is with an audio playback controller. For this tutorial though we’re going to use the Audio Player and Single-Clip Controller patches instead.
Unlike the audio playback controller, the Audio Player and Single-Clip Controller patches provide outputs that can be connected to the Audio Analyzer and Energy Meter.
Add the speaker
To add a speaker:
It’ll be listed as speaker0 in the Scene panel:
Add the speaker patch
This patch will receive signals from the rest of the graph, including a patch representing the audio clip. As a result audio will play in the scene. To create a patch representing the speaker:
The Patch Editor will open and a yellow consumer patch will be created.
Add the audio clip patch
The audio clip is listed in the Assets panel as arl_mus_FunkyDrums_lpl_03.m4a.
To create a patch representing this audio clip, drag it from the Assets panel into the Patch Editor:
Add the Single-Clip Controller, Audio Player and Pulse patches
Right-click in the Patch Editor to open the menu. Select:
Check the box in the Pulse patch, to turn the signal on:
Connect the patches
You should read the following instructions very carefully to make sure each patch output is connected to the correct input of the correct patch or you won't get the correct result.
Here’s how the graph will look:
The audio clip will be playing on a loop. You might need to click the Restart button in the toolbar to hear it:
Next we’ll add the Energy Meter and Audio Analyzer patch, which will connect the output from the audio player to the rectangles.
To add the Energy Meter patch:
The patch has 4 outputs. The Audio output is always connected to an Audio input port, either in a patch representing the speaker or another audio patch.
The Energy output signals the power estimated on the most recent time window of the mono audio, or 0 in stereo mode. Energy Left and Energy Right output the same signal for the left and right channels respectively.
Connect it to a rectangle through a patch representing it's Scale property. The scale will change in response to the output from the Energy Meter, animating the rectangle. To create this patch:
Set positions for the rectangle to scale between using a Transition patch. To create this patch:
Because you’re working with 2D objects, change the Transition patch to Vector 2 using the menu at the bottom of the patch:
Because audio files can vary in dynamic range and peak volume, with any audio effect you’ll often need to adjust output values using math patches to achieve the desired effect.
In this tutorial, a Multiply patch will be used to adjust the output from the Energy Meter, and a Square Root patch will emphasize smaller output values from the audio source. Together they’ll scale the output more linearly with perceived loudness. To create these patches:
To add the Energy Meter patch to the graph you’ll need to break the connection between the Audio Player and the speaker patch, highlighted in the image below:
Once you’ve broken the connection, connect:
Adjust the values in the Transition patch. Set:
In the Multiply patch, set the Second Value in the bottom of the patch to 50. You could play with the values in the Multiply and Transition patches depending on the audio you’re working with, to create a stronger or more subtle effect.
Here’s how the graph will look:
In the Simulator, you'll see the rectangle at the bottom of is animating in time with the drum loop, responding to volume changes in the audio playback.
As a final touch use a patch called Exponential Smoothing to change the output from the Energy Meter more gradually. This will create a smoother movement in the animation.
To create the patch:
To add this patch to the graph:
You may need to add a different value to the Exponential Smoothing patch if you're using a different audio file.
Here’s how the graph will look:
The movement of the rectangle is smoother.
Audio Analyzer patch splits incoming audio into 8 bands.
Like with the Energy Meter patch, each output from the Audio Analyzer patch will be connected to a Square Root, Exponential Smoothing and Transition patch, and a patch representing the Scale of a rectangle.
Start by creating the patches:
Make make sure the data type in the Transition patch is set to Vector 2.
The Audio Analyzer has 9 output ports, as in the image below:
The Audio output is always connected to an Audio input port, either in a patch representing the speaker or another audio patch. The other outputs each produce a scalar signal indicating the acoustic energy present in a specific frequency band of the audio clip.
To add it to the graph:
Here’s how the new section of graph will look:
Now to connect the Audio Analyzer patch to the rectangle. Connect the:
In the Transition patch, set:
In the Exponential Smoothing patch, set the Damping value to 20.
Here’s how the new section of the graph will look:
You’ll see the rectangle in the scene is animated:
Connecting the remaining rectangles to the Audio Analyzer
To complete the effect, repeat the steps you followed to connect the Audio Analyzer patch and BandRectangle1, for the remaining rectangles. You can copy and paste patches to speed this up. The values in the Exponential Smoothing and Transition patches should be the same.
Your your graph will look like this:
In the sample content finished effect, you'll see we've turned this section of the graph into a patch group named Analysis Visualization. This is a good way to keep your project tidy.
The final effect looks like this: