back to all projects

Instrument Design

Ringtone

A custom electronic musical instrument with capacitive touch controls and an LED-based UI

Performing with Ringtone.

Ringtone is an original musical instrument featuring a circular body, capacitive touch controls, and a minimal LED user interface. I conceptualized and assembled Ringtone in Spring 2020 as my final project for the MIT class Digital Musical Instrument Design (21M.370).

Performing with Ringtone. (0:48)

Watch Demo on YouTube

The Instrument

Ringtone's body consists of a 7.5" diameter plate constructed from plastic takeout lids. Lining the inside edge of the plate is a strip of 32 LEDs, and along the outside edge are 12 capacitive touch sensors cut from copper tape. The 12 touch sensors form two sets: 8 "note" sensors on the right hand side, and 4 "button" sensors on the left.

Attached to the back is the 21M.370 class PCB, an ESP32 microcontroller, and an IMU, which collects XYZ acceleration and gyro data. The ESP32 connects via USB to a computer (not pictured), where the instrument's sound is generated.

Ringtone diagram

A diagram of Ringtone.

Ringtone: 4 button touch sensors Ringtone: 8 note touch sensors

left: B, -, +, and K buttons. right: 8 “note” touch sensors, mapped to pitches configured in software.

Ringtone: instrument back

Back of Ringtone.

Playing Ringtone

To play Ringtone, simply press the 8 note sensors as you would keys on a piano. Pressing a sensor starts its note, and releasing the sensor stops the note.

Ringtone: piano

Play Ringtone's note sensors like piano keys.

Ringtone also has two special features to enhance expressiveness while playing: tone control and an octavized bass note.

Tilt to Tweak Tone: You can tweak the tone and volume of Ringtone's sound by tilting the instrument in 3 axes as you play notes. This alters the blend of Ringtone's oscillators (see Sound Generation in Pd for more details).

Octavized/Bass Note: If you hold the [B] button, Ringtone will octavize the next note you play by also triggering the note an octave below it. The higher note will stop when you let go of the note sensor, as usual, but the lower note won't stop until you let go of the [B] button. One octave is a limited range, so this feature is useful for adding bass notes to achieve a fuller sound.

Ringtone: playing

Two features while playing Ringtone.

Configuring Ringtone

By default, Ringtone is set to the key of C, and the 8 note sensors are mapped to the major scale: C4, D4, E4, F4, G4, A4, B4, and C5. You can configure the offset, key, octave, and scale of this pitch mapping using different combinations of the button sensors.

Rolling the Scale: Press the [+] or [-] buttons to roll the scale to the left or right, respectively. This offsets each note sensor's position in the scale. For example, pressing [+] would scroll the C major scale to the left, resulting in D4, E4, F4, G4, A4, B4, C5, and D5.

Changing the Key: While holding the [K] button, press the [+] or [-] buttons to transpose to a higher or lower key, in semitone increments. For example, holding [K] and pressing [+] would transpose the C major scale to C#/Db major, resulting in Db4, Eb4, F4, Gb4, Ab4, B4, Cb5, and Db5.

Changing the Octave: While holding the [B] button, press the [+] or [-] buttons to transpose to a higher or lower octave. For example, holding [B] and pressing [+] would transpose the default C major scale up an octave, from starting at C4 to starting at C5.

Shake to Cycle Between Scales: Shake Ringtone to cycle through a list of different scales. The available scales are major, blues, natural minor, harmonic minor, and melodic minor.

Designing Visual Feedback

While configuring Ringtone, it can get difficult to keep track of what pitch mapping you have. Without feedback, it's also hard to tell if your actions have been recognized or if your desired changes have been made. To solve this, I designed a minimal LED-based interface to help you visualize the pitch mappings as you change them.

Each of the 12 touch sensors are positioned directly above an LED. The note sensor LEDs are color-coded to indicate what pitches they are assigned. The button sensor LEDs are off by default and light up when the corresponding buttons are pressed. Together, these LEDs provide a visual representation of state and great visual feedback for user actions:

The 20 LEDs that are not linked to touch sensors take on the color of the current key and run an infinite LED train animation.

How It Works

Ringtone's I/O Chain

Ringtone is built on top of the I/O chain that we used for 21M.370 class assignments, which includes 3 components: ESP32 code, Python code, and a Pd patch.

Ringtone: I/O chain

Ringtone's I/O chain

The ESP32 code (C++) collects input from the touch sensors and the IMU, and it sends output to the LED strip. The ESP32 communicates with Python via USB serial.

The Python code takes sensor data from the ESP32 and processes it. It then uses the data to perform various actions, such as changing the pitch mapping (see State Machine) and updating the LEDs (see LED User Interface). Python then forwards the processed touch and IMU data, as well as musical parameters like MIDI pitches, to the Pd patch via OSC.

The Pd patch accepts values from Python and uses them to synthesize Ringtone's audio output.

State Machine (Python)

Ringtone's primary logic is implemented in Python as four modes of operation, or states 0-3. Ringtone begins in State 0 (Play Mode) and switches between states based on which buttons are pressed or released. The states are described below:

* handled in Pd
0 - PLAY MODE
    8 down/up:  note on/off*
    B up:       note off (8va), clear bass pitch
    + down:     roll +1 scale degree
    - down:     roll -1 scale degree
    B down:     enter bass select mode (1)
    K down:     enter key select mode (2)
1 - BASS SELECT MODE (B down)
    8 down:     change bass pitch, note on (8vb)*, enter play mode (0)
    B up:       note off (8vb)*, clear bass pitch, enter play mode (0)
    K down:     enter octave select mode (3)
2 - KEY SELECT MODE (K down)
    8 down/up:  note on/off*
    + down:     increase key 1 semitone
    - down:     decrease key 1 semitone
    shake:      change scale (major/minor, etc.)
    K up:       enter play mode (0)
    B down:     enter octave select mode (3)
3 - OCTAVE SELECT MODE (K and B down)
    8 down/up:  note on/off*
    + down:     roll +1 octave
    - down:     roll -1 octave
    K up:       enter bass select mode (1)
    B up:       enter key select mode (2)

LED User Interface (Python)

To create the LED UI color coding, I mapped pitch to color in the HSV (Hue, Saturation, Value) color space. The pitch class (0-12, calculated by MIDI pitch mod 12) corresponds to evenly spaced hues between 0º to 360º. The octave is then mapped to saturation, and the value is fixed. I then converted the color from HSV space to RGB space to send to the LED strip.

As the player changes Ringtone's pitches, Python regularly recalculates the LED colors and sends them to the ESP32 to output to the LED strips. Python also responds to press and release events from the button sensors, turning the corresponding LEDs on and off when they are pressed and released.

Ringtone: Pitch to hue

Pitch class to hue mapping in Ringtone.

Sound Generation in Pd

In the final step of the I/O chain, a Pd patch collects values from Python via OSC to synthesize Ringtone's output audio.

Ringtone listens at these OSC addresses: /pitchX for the 8 note pitches, /capstateX for the 8 note sensor states, /basspitch and /bassnote for the bass note, /gyro for XYZ rotational positions, and /accel for XYZ accelerations.

Ringtone: OSC inputs to Pd

OSC inputs to Pd.

There are 9 subpatches, one for handling each of the 8 notes, and one for the bass note. The outputs of these subpatches are fed through a low pass filter, a graphic EQ, and then a reverb and stereo delay module before they reach the mixer, which outputs the audio to the computer speakers.

Ringtone: Main Pd patch

Ringtone's main Pd patch.

In each subpatch, there are 3 basic oscillators from Automatonism — one saw, one sine, and one triangle — each with their own Wavefold and VCA modules. The fold parameter and VCA gain parameter are controlled by the gyro X, Y, and Z values, so that tilting and rotating Ringtone will blend between different waveforms and change the tone. Each note’s onset and release are controlled by an ADSR envelope and the state of the capacitive touch sensors.

The relatively long sustain and release in the ADSR envelope, plus plenty of reverb and delay at the end of the chain, create Ringtone's "dreamy" sound.

Ringtone: Pd subpatch

Inside each Pd subpatch.

Design Evaluation

After performing with Ringtone, I identified the features I found the most useful, as well as a number of areas for improvement.

Reflection

I’m delighted with Ringtone as an instrument! I had lots of fun both making it and performing with it. Through this project, I learned a great deal about music technology, electronics, and interaction design. 21M.370 was my first venture into instrument design, and I was amazed by the sheer breadth of the field. I hope I'll get more opportunities to explore this area in the future.

Ringtone: instrument back

Back of Ringtone.
back to all projects