The vibrotactile technology that we use converts musical sounds that can be heard, into musical vibrations which are felt through the skin as vibrotactile feedback. It’s an assistive technology which has been described as a form of “hearing through the skin”.
Vibrotactile technology works best via the ‘glabrous’ skin on the palms of the hands and soles of the feet. Different musical pitches from C1 to G5 can be perceived in this way. In the Musical Vibrations system, up to six different vibrotactile elements can be used at the same time: (left hand, right hand, left heel, right heel, left forefoot, right forefoot).
The project builds on previous research into the tactile perception of music by D/deaf musicians which was inspired by Dame Evelyn Glennie, the highly-renowned solo percussionist who is classified as profoundly deaf with residual hearing at high amplification.
Using vibrotactile feedback can support people who are D/deaf in their music making and music appreciation in several ways:
Music education, rehearsal and performance:
- To assist two or three musicians in playing together as an ensemble
- To assist solo practice in time to a click track / electronic metronome
- To assist musicians in playing in time with a previous recording (multitrack recordings are preferable as the mix of instruments being fed to the shakers can be customised to provide the best vibration cues).
- To assist a deaf singer to rehearse/perform within a choir without needing overt visual cues.
Because six different vibrotactile elements are available, up to six different musical signals, for example from different instruments within a band, can be sent to different shakers. So, separate elements within the ensemble, for example, kick drum and snare drum, can be more clearly defined than would be possible through by touching a single loudspeaker which is playing a CD.
For musician, it’s still possible to ‘listen’ to four different signals by using the heels and toes of both feet, whilst leaving the hands free.
Compatible musical instruments
The system is easiest to use with anything you’d normally expect to plug into a mixer:
- Vocal microphone
- Electric or electroacoustic guitars and ukuleles
- Bass guitars
- Keyboards, electric pianos and synthesisers
- Digital audio workstations (DAW’s) + audio interface
- CD and MP3 players.
Any instrument which can be mic’d up or used with a pickup e.g. drums, violin, cello is also feasible:
Music appreciation by music fans who are D/deaf
- To appreciate music at a live performance as an addition to captioning and performance signing
- To appreciate recorded music
Because six different vibrotactile elements are available, up to six different musical signals, for example from different instruments within a band, can be sent to different shakers. These are derived from the main venue ‘FOH’ sound system but are selected to provide the most helpful vibrations. So, separate elements within the band, for example, bass guitar and snare drum, can be more clearly defined for the user than would be possible through touching a single loudspeaker connected to the venue sound system.
We tested this concept recently at the Shuffle Down music festival in Scotland.
Vibrotactile technology may facilitate music production by musicians and producers who are deaf by replacing auditory cues.
Separate audio outputs from a digital audio workstation (DAW) can be sent (via an audio interface) to the vibrotactile shakers, enabling ‘vibrotactile monitoring.’
We tested this concept recently with a D/deaf producer, writer and sign-song rapper called SignKid. Read more here.