Inside the finished project

Inside the project, you’ll see we used the following patch graph to create the effect:

The graph starts with a Screen Tap patch. Each tap on the screen:

• Modulates the Oscillator’s pitch (via vertical tap position).
• Triggers the volume envelope.

This means that instead of just generating a constant flat tone, we create an effect where a different musical note is played for a brief period of time. We’ll explain this in more detail below.

Setting the pitch

Each vertical tap on the screen changes the pitch generated by the Oscillator patch. There are three key operations we need to perform in the Patch Editor to do this:

1. Capture the position of the user’s finger on the Y axis of the device screen.
2. Transition the Y positional value to a value that’s within the MIDI (Musical Instrument Digital Interface) note value range. MIDI notes correspond to ascending keys on a piano keyboard: 50 is D, 60 is middle C and so on.
3. Transform the MIDI note value into a corresponding frequency measured in Hz. This output is then used as the pitch input for the Oscillator patch.

The first two operations (point 1 and 2) are performed by the patches in the Screen Tap Position → MIDI Note Value box:

First, the Screen Tap patch outputs the 2D Position of the finger on the screen. Then the patches connected to this output:

• Divide the 2D position by the Device’s Screen Size. This remaps the 2D position to a range between 0-1 (i.e X=0 at the far left side of the device screen, X=1 at the far right)
• Unpack the 2D position into its X and Y coordinates. This isolates the Y axis, so we can use the vertical tap position to control the pitch.
• Transition the Y value (from 0-1) to a value within the 30-60 MIDI note value range.
• Round the output of the transition patch, so that only MIDI values that occur on a piano keyboard are output, instead of any values landing between the notes.

The rounded value is then connected to the MIDI to Pitch patch group in the MIDI Note Value → Pitch (Hz) box. You can download this patch group from AR library to use in your own projects.

The MIDI to Pitch patch group contains the math patches that carry out point 3 — transforming the MIDI note values into a frequency. This provides the frequency value that’s used as the Pitch input of the Oscillator patch.

Shaping the volume envelope

At this point, connecting the output of the Oscillator patch to a speaker would generate different continuous notes, changing each time the user taps. For our synthesizer we want to emulate individual musical notes that start at volume zero, transition to an audible volume for a short period of time, then return to volume zero. This transition is called a volume envelope.

The patches in the Screen Tap Event > Gain Amount box shape this volume envelope:

When the screen is tapped:

• The first Animation patch takes the volume from 0-1 in 0.1 seconds.
• As soon as this transition from 0-1 is complete, this triggers the second Animation patch to take the volume back down from 1-0, in 0.4 seconds. This process starts again the next time the screen is tapped.
• The Subtract patch calculates the correct value (between 0-1) to pass on to the Gain patch, at each stage of the process — from screen tap to release.

Applying the pitch and volume

The final part of the graph — the patches in the Audio Generation & Processing box —outputs the pitch and volume defined by the earlier parts of the graph. The Oscillator handles the pitch and the Gain applies the volume.

Customizing the synth sound

With the synthesizer set up, it’s easy to experiment with different synth sounds. The Oscillator patch has five different waveforms to choose from, each with its own distinctive waveshape and sound. Select an option from the dropdown to listen to each one:

You could also connect the audio output of the Oscillator through any of Spark AR's audio processing patches to further shape the sound. For example, try adjusting the tone of your synth by adding an effect like the Ring Modulator patch between the Oscillator and the Gain as in the example below:

Alternatively, add a time-based effect like Reverb or Delay between the Gain and the Speaker to add a sense of space or depth to each note played.