Enlarge

Sounds from Saturn?People don’t have to go to outer space to make music in new ways. Technologies using computers and sensors are being created that will let people do that right here on Earth. NASA/JPL

The musical instruments of the future may be right in front of your eyes and on the tables, walls and windows around you. All it takes to use them is the right hardware, and a little imagination.

In Switzerland, a team of scientists and artists are working together on new technology that can transform almost any surface into a musical instrument. The technology is called MUTE, short for Multi- Touch Everywhere. Using MUTE, a person can use a computer to translate taps on different parts of a table or a wall as different sounds.

For example, you may record and save different sounds on a computer — anything from a snare drum or trumpet to clapping hands or a sneeze. Then, you program your computer to play one of these recorded sound snippets whenever you tap a certain spot on a table top or wall. The left side of a table might play snare drum beats, the right side a melody on a trumpet. If you tap the two sides at the same time, you’ll hear both sounds come together as a song. The system uses a camera and lasers to see where you’ve tapped on the table.

What’s more, the programmed surface doesn’t even have to be solid — it can float right in front of you, explains musician and MUTE developer Alain Crevoisier. “It can even work in the air,” he says. “Since the lasers are creating a plane of light, what we actually detect is when you cross this plane with either the hands or sticks or mallets.” Imagine, for example, a virtual piano hovering in front of your face.

When the MUTE system is installed on a surface, it also uses acoustic sensors to track the location of a performer’s tap. (For more information on acoustics, see the sidebar below story, “What is acoustics?”) “When you tap the table you generate vibrations,” says Crevoisier, a researcher at the Music Conservatory of Geneva. The vibration travels through the surface as an acoustic wave, and when the vibration strikes the sensor, the sensor sends an electric signal to the computer.

The device is not a musical instrument in the way we normally think about instruments. But that’s part of the beauty of it, says Crevoisier. It allows a person to be creative. “It’s more like we are providing a means for people to design their own instruments,” he says. His system adds a layer of music to already existing sounds. On a regular drum, for example, a drum beat is just the sound of the drum. But on a drum outfitted with MUTE technology, a drum beat could be both the sound of the drum and a control for some other sound layered on top of that.

Crevoisier’s work on new musical instruments grew out of his participation in a project called TAI-CHI (pronounced ty-chee), which stands for “tangible acoustic interfaces for computer-human interactions.” An interface is a device, or a lot of devices working together, that allow people to communicate with machines. A tangible object is one that you can touch. The keyboard of a personal computer, for example, is a tangible interface between you and your computer. So is your mouse.

In the TAI-CHI project, Crevoisier and his colleagues showed that acoustics, or the science of sound, could be used to turn any surface into a musical instrument. They also showed that the technology could lead to a new kind of interface.

Here’s how: The sensors pinpoint the place on a surface where a person taps.

One way to do this requires at least three sensors on the surface. When a person taps the table, the sound waves travel to the sensors, and each sensor records the exact time when the waves reached it. By knowing where the sensors are located and what the surface is made of, a computer program can use the waves’ arrival times to figure out exactly where the surface had been tapped.

Enlarge

Tap screen This is the computer program used to assign different sounds to different parts of a surface. Sensors tell the computer when someone taps on one of the colored regions.Alain Crevoisier / Future Instruments

Another method to pinpoint a tap uses only one acoustic sensor, but it is more complicated. A user needs to fine-tune the device very carefully, and provide the computer with lots of information about the surface material itself.

Acoustic sensors could be used to build new kinds of computers that look nothing like traditional desktop models. Unlike a keyboard and mouse, which require a user to remain in front of a computer screen, acoustic sensors would allow a user to interact with the computer almost anywhere. You could use your fingers to draw a picture on a wall, for example, and record the drawing with your computer.

Or, imagine a restaurant owner, who could glue menus to the top of his tables and install acoustic sensors underneath. Diners could then order simply by tapping on the menu. The vibration from the tap would be picked up by the sensors, which would be able to figure out where the tap came from. A computer could match that location to a dish on the menu and send the order to the kitchen.

In another example, perhaps someone in a wheelchair could mark a spot on the wall, a table top or the arm of a chair to serve as a switch. It might be for turning on or off a light, for turning up or down the volume on a television or even for sending out a distress alarm. A simple tap on the spot could trigger the sensors, which could relay the information to a computer. The technology would allow people to make any surface into an interface to control some action.

Crevoisier isn’t the only one looking at ways to use acoustic sensors in future devices. Another European company, for example, is finding ways to use them to make a “smart apartment,” where any surface — mirrors, tables, counters, walls — can be used to interact with the house computer, to do tasks like change the lighting, turn on the television or raise the temperature.

Other researchers around the world are developing other kinds of new tangible interfaces, though not all of them use acoustic sensors. The computer company Microsoft has developed a device called TouchWall, for example, which converts almost any surface to a computer interface by using sophisticated laser trackers, cameras and a projector.

Look around again. The future of computing and of musical instruments may be all around you.

Sidebar: What is Acoustics?

Acoustics [uh-KOO-stix] is the science of sound. Stop reading for a moment and listen to the sounds around you. What did you hear? The whirr of a computer fan, people talking, the noise of cars going by? When scientists want to understand more about how sound moves and behaves, they study acoustics.

Enlarge

Highs and lowsSound waves are measured by height, called amplitude, and width, called frequency. At top, the sound waves have a lower frequency and, to our ears, a lower pitch. The bottom image shows a sound wave with a higher frequency, which we would hear as high-pitched notes.NOAA Ocean Explorer

When you drop a rock into a pond, you can see waves move across the surface, away from the rock. The sounds that you hear are also made of waves, except that sound waves can travel through air or through solids. In fact, sound waves travel faster in solids like wood or plastic than they do in the air.

Think about the sound made by a guitar. When a person strums a guitar string, the air around it begins to vibrate. This vibration moves through the air as a wave, and it can bounce off walls (which can cause an echo) or be absorbed by other materials. It can also go into your ear, which will send a signal to your brain. Your brain interprets the wave as the sound of a guitar.

Understanding sound waves is important for many fields. Doctors use ultrasound waves, which are so highly pitched that we can’t hear them, to see inside the human body. When an architect designs a concert hall, he or she must think about how the sound waves travel through the air to make sure everyone hears the same sounds. Sound waves can also be used to detect objects that are underwater, like fish or submarines, with a technique called sonar: sound waves are sent out through the water from a source, and the object reflects waves back to the original source of the sound waves called a transmitter.

Scientists who study acoustics try to understand all the parts of sound: how the sound is made, how it travels, and how it is detected and interpreted.

Power Words

Interface: the place at which independent and often unrelated systems meet and act on or communicate with each other

Tangible: capable of being perceived especially by the sense of touch

Enlarge

Acoustic bloomsThis device, built with TAI-CHI technology, is called the Sound Rose because when a person taps on the table, a colorful flower appears. A person’s taps are tracked using acoustic sensors, and the images are projected from the ceiling.Alain Crevoisier / Future Instruments

Acoustics: a science that deals with the production, control, transmission, reception and effects of sound

Vibration: a periodic motion of the particles of an elastic body or medium in alternately opposite directions from the position of equilibrium when that equilibrium has been disturbed (as when a stretched cord produces musical tones or molecules in the air transmit sounds to the ear)

Sonar: a method or device for detecting and locating objects especially underwater by means of sound waves sent out to be reflected by the objects

For Teachers

Question Sheet: Music of the Future

SCIENCE

Before Reading:

--How would you describe a musical instrument?

--How does it make sound?

--Sometimes computers are attached to musical instruments, like a piano. What might be the advantage of that?

--Computers have inputs and outputs. Outputs include your computer’s speakers, monitor and electrical signals that control other devices such as a printer. But how do you input commands to your computer, telling it what you want it to do? What device or devices do you use?

During Reading:

--Mute usually means to utter no sound. But in this story, the MUTE technology does what?

--A computer keyboard is a type of sensor. But in MUTE and TAI-CHI other types of sensors are used. What do they sense?

--These sensors rely on acoustics. What is that?

--What’s a Sound Rose?

--How might TAI-CHI speed a restaurant order?

After Reading:

--How might systems based on acoustic sensors change the idea of what a musical instrument is?

--How might the same idea be applied to improve the lives of people with disabilities or the elderly? Where would you install them to simplify your life?

--What are the advantages of a standard computer mouse as an input device, compared to acoustic sensors?

--What are the risks of making a spot on your desk or the kitchen table into a computer-input device? (Hint: What if you put a book or milk glass down hard?)

SOCIAL STUDIES

--In a few short years, computers went from being room-sized machines to machines as small, portable as a notebook or, in some cases (such as iPods or Blackberries), a deck of cards. Discuss how acoustic-input devices might expand the way we think of computers and where we use them.

--How would the MUTE and TAI-CHI systems change the way we make music? Is that a good thing or would you prefer the old acoustical instruments — strings, winds, drums and horns? Explain your preference.

--How might these acoustic-sensor inputs to computers save energy in homes or businesses?

LANGUAGE ARTS

--Imagine a computer-controlled world without keyboards. Describe a day in your life in this world.

--You’re going to make a MUTE-based musical instrument, recording sounds that your instrument will later associate with various spots on a tabletop. Describe in two paragraphs what sounds you’d collect and why. Imagine a band made out of five such MUTE systems. Describe in up to three paragraphs the challenges you might encounter — and have to overcome — in writing or orchestrating music for this MUTE combo. (HINT: What would happen if this band travels a lot.)

--Imagine that your neighbor was injured in an auto accident and confined to a wheelchair with no use of her legs. Describe how, as a MUTE or TAI-CHI engineer, you would set up their bedroom, living room and kitchen to maximize that neighbor’s use of those rooms and what’s in them.

• Print
• |
• Comment