American Innovations in Electronic Musical Instruments
Wearables (Active Clothing)
Yet another set of musical controllers has appeared over the last couple of decades. These are essentially “wearable” interfaces, where a sensor system is affixed to the body or clothing of a performer. A very early example of this is from composer Gordon Mumma, who inserted accelerometers and wireless transmitters into dancers’ belts for performances dating from 1971. Berlin-based artist Benoit Maubrey has been designing “electro-acoustic clothing” since the early 1980’s; apparel that incorporates tape recorders, synthesizers, samplers, speakers, and optical sensors for very expressive performance. Some of the best-known examples of wearable controllers were likewise used during the 1980’s by the New York performance artist Laurie Anderson, who pulled the triggers off a set of electronic drums and built them into a suit, enabling a percussive performance by tapping different sections of the body. Commercial companies started making such garments in the mid-80’s, for instance, the “Brocton-X Drum Suit”, with attachable Velcro percussion sensors and heel-mounted triggers. In the early 90’s, Mark Coniglio, of Troika Ranch, designed MidiDancer, a body suit instrumented with 8 sensors, measuring motion and bend at various joints. Data is wirelessly offlinked and converted to MIDI. In the mid-90’s, Yamaha introduced its Miburi system, consisting of a vest hosting an array of resistive bend sensors at the shoulder, elbows, and wrist, a pair of handgrips with two velocity-sensitive buttons on each finger, and a pair of shoe inserts with piezoelectric pickups at the heel and toe.
In contrast to fully-instrumented body suits, some intriguing musical controllers are independently cropping up in various pieces of apparel. Laurie Anderson, again a pioneer in this area, performed a decade ago with a necktie that was outfitted with a fully functional music keyboard. Here at the MIT Media Lab, we have recently built “musical jackets”, with a touch-sensitive MIDI keyboard embroidered directly into the fabric using conductive thread. We have also made a set of “expressive footwear”; retrofitting to a pair of dance sneakers with a suite of 16 sensors to measure several dynamic parameters expressed at a dancer’s feet. These shoes require no tether; they are battery powered for up to 6 hours and offload their data right from each shoe via a wireless link. We have used this system for interactive dance performances (e.g., recently at the 1999 American Dance Festival), with the dancers’ foot motion launching and modifying real-time musical events.
Other researchers have attached electrodes directly to the body, using neurological and other biological signals to control various sound sources in some very unusual performances. Some of the best-known such works were produced by composer David Rosenboom of Mills College during the 1970’s. These “biofeedback” pieces generated sounds as a function of the performers’ biological states, including heart rates, GSR (skin resistance) data, temperature probes, and of course, EEG (brainwave) data. In most of these pieces, a computer system would monitor these features and direct the sonic output as a function of their physiological states and their correlations. Academic work continues in this area, for example with the “Conductor’s Jacket” by Teresa Marrin at the MIT Media Lab, a vest with a host of biosensors that measure respiration, heart rate, temperature, skin conductance, and electromyography (EMG) for each bicep and tricep. Teresa has logged and analyzed data from conductors wearing the jacket during live orchestral performances and has conversely used the jacket as a musical controller for electronic music performance. This has now become a commercial business of sorts, with products appearing on the market aimed partially at musical control applications. For example, a system marketed by IVBA Technologies of Norwalk, CT consists of a sensor headband, which purports to measure brainwaves of various kinds. Another, more general, device is the “Biomuse”, produced by BioControl Systems, a Stanford University spin-off started by Hugh Lusted and Benjiman Knapp. The Biomuse is able to gather EMG (muscle), EOG (eye movement), EKG (heart), and EEG (brainwave) signals. It also has MIDI output and mapping capability, and has been used in musical research projects. Another such product is the Bodysynth, designed by Chris Van Raalte and Ed Severinghaus. Although the controllability and bandwidth of some of these parameters (particularly the brainwaves) is a topic for argument, new musical applications won’t lurk far behind as researchers at various institutes progress in extracting and identifying new and more precise bioelectric features.
The different sensors developed for motion capture in the computer graphics and Virtual Reality communities have been rapidly pushed into musical applications; various music researchers have worked with the magnetic tracking systems (such as the well-known products by Polhemus and Ascension Technologies), electro-mechanical body suits (e.g., the Gypsy by Analogous) and datagloves upon which the VR world was built. For example, Jaron Lanier, a well-known pioneer in this field, is still a very active musician, incorporating a host of different VR and musical interfaces (along with traditional and ethnic acoustic instruments) into his live shows. Gloves, in particular, have appeared in many musical performances. One example, composed by Tod Machover here at the Media Lab, is “Bug Mudra”, where an Exos “Dexterous Hand Master” was worn by the conductor, who had complete dynamic control over the audio mix and synthesis parameters through finger positions. Many other composers, such as Richard Boulanger at the Berklee School of Music in Boston, have used Mattel’s “Power Glove” (the low-cost brother of the original VPL DataGlove, intended for the home gaming market) as a controller in several pieces.
Some of the most interesting glove and hand controllers have come from STEIM, the Dutch center for electronic performance research in Amsterdam. Michel Waisvisz’s “Hands”, first built at STEIM in 1984, consist of a pair of plates strapped to the hands, each equipped with keys for fingering and other sensors that respond to thumb pressure, tilt, and distance between the hands. Waisvisz has written many pieces for this expressive controller, and still uses it in performance. Laetitia Sonami, now at UC Berkeley’s CNMAT Lab, has also built a device at STEIM called the “Lady’s Glove”, an extremely dexterous system, with bend sensors to measure the inclination of both finger joints for the middle 3 fingers, microswitches at the end of the fingers for tactile control, Hall sensors to measure distance of the fingers from a magnet in the thumb, pressure sensing between index finger and thumb, and sonar ranging to emitters in the belt and shoe.
From American Innovations in Electronic Musical Instruments
by Joseph A. Paradiso
© 1999 NewMusicBox