>> Toshio Iwai, Ryuichi Sakamoto, "Music Plays Images x Images Play Music", 1997

This piece won the Golden Nica in the Interactive Art category in 1997's Prix Ars Electronica

from Ars Electronica: http://www.aec.at/en/archives/prix_archive/prix_projekt.asp?iProjectID=2494

"Music Plays Images x Images Play Music" is a multi-media concert that uses the system of Toshio Iwai's piano piece and visualizes a musical performance by Ryuichi Sakamoto in real time.

The idea for this collaborative performance has its origins in Toshio Iwai's installation "Piano ­ as image media" created at the ZKM, Karlsruhe, Germany in 1995. Conceived upon Iwai's return to Japan when he met Ryuichi Sakamoto, "Music Plays Images x Images Play Music" is a multi-media concert that uses the system of Iwai's piano piece and visualizes a musical performance by Sakamoto in real time. Over the course of a year, Iwai, who lives in Tokyo, often corresponded by e-mail with Sakamoto, who lives in New York. During their correspondence, the concept for the collaboration expanded rapidly, developing from the simple idea of "Sakamoto playing Iwai's 'Piano ­ as image media'" to a performance that links, in various ways, musical sounds with visual images. The actual performance was given on December 16, 1996 at the Art Tower Mito in Mito City, Japan. The result of the collaboration was more than a "novel combination of music and image." Many elements including the actual presence of Sakamoto/Iwai, the machinery that is called the piano, the acoustic sounds produced by the piano, and the virtual images generated from computers were linked and developed in various ways. From the feedback produced, a radically new space of musicality-visuality was created. In addition, the performance was transmitted over the Internet and relayed world-wide. This experiment, the first of its kind, was an attempt to utilize the interactive medium of the Internet. People on the other side of the Internet were able to play the piano located in Mito by remote control and in turn Sakamoto had a session with their playing.

The Set-Up of the Stage
Two concert grand pianos are placed on the stage. A semi-transparent screen 8 meters in height and 10 meters in width is suspended above them. The lids of both pianos are removed and their interiors and keyboards clearly illuminated. The screen is irradiated with a beautiful shade of blue. A black and white video camera is placed on each of the pianos in order to film the movements of the performer's fingers and the piano keys. The video cameras project enlarged images of each piano keyboard onto the screen on either side of the pianos. Several computers are set up on a long narrow table placed on the right side of the stage.

The Content of the Performance
1) Sakamoto does not initially appear on the stage. Amidst the silence, the piano begins to play on its own without a player. An image of Sakamoto playing the piano gradually comes into view on the screen to the side of the piano. Sakamoto's image slowly disappears as the music ends.

2) Sakamoto appears on the left side of the stage. He begins to play the small keyboard placed there. As he plays this keyboard, the piano starts to play. A faint light gradually emanates from the piano. At a certain point, Sakamoto stops playing. However, the last musical phrase repeats itself automatically, slowly fading out.

3) Sakamoto plays the piano directly. The light-images that visually represent each note of the musical piece begin to rise from within the piano. The images capture not only the timing of the notes, but every aspect of the music, including the length of the notes and their loudness. They are represented in a way that the eye can recognize.

4) During Sakamoto's performance, Iwai gradually modifies the image. The lights, which correspond to each note of the piece, are transformed over time in various ways. They do not merely rise from within the piano and fade into thin air. They radiate from the center of the screen out in all directions or revolve in a whirling pattern, eventually disappearing into a vortex. The metamorphoses of the projected images influence Sakamoto's playing as he watches the various transformations.

5) The image-objects, composed of 88 elements ­ the same number as the number of keys on a piano ­ appear on the center of the screen. Each of the elements coincides with the keys of the piano and the objects are continually transformed according to Sakamoto's piano performance.

6) Clicking a computer mouse, Iwai places dots of light onto the screen. Whenever the dots of light fall on the piano, the piano begins to play. The dots of light then rebound to their original position, striking the keyboard at fixed intervals. Depending on where the dots of light are placed, the pitch of the notes and the patterns of repetition change. The number of dots of light are increased. The dots of light bouncing from the piano are all shifted back and forth from left to right. The whole musical scale changes according to how the dots of light are moved. The dots of light are transformed into rays and then polygonal figures.

7) The video camera films Sakamoto's entire body. The computer captures the image in real time. Sakamoto's silhouette is pixelized and then projected onto the screen above the piano. When Sakamoto moves and the image of his body overlaps with the piano, the piano plays. When Sakamoto moves toward or away from the camera, or when a close-up image of his hands is projected, the sound is changed dynamically. The images of Sakamoto's hands and body are what plays the machine called the piano. Iwai implements further modifications by zooming in on and rotating Sakamoto's image on the screen thereby slowly modifying the sound. Sakamoto continues to move his body while looking on.

8) A chessboard-like grid appears on the screen and begins to rotate. Sakamoto and Iwai take turns placing glowing dots of light onto the grid, thereby creating a repetition of short melodic patterns. As the music is created through this collaboration, Iwai transposes the musical pattern, changing the octave and altering the sounds and the images. Sakamoto continues to create new patterns. Finally, with the completed musical phrases as a backdrop, Sakamoto improvises on the piano. Iwai continues to modify the performance piece.

9) The music played by Sakamoto is turned into an image. The image that springs from the first piano and lands on the second piano creates a parabola in its fall. The second piano plays according to this image. The figure of Sakamoto performing on the first piano is projected next to the second piano with a delay that coincides with the imaged music arriving at the second piano. It appears as if there are two Sakamotos playing the two different pianos.

10) The short musical phrase performed by Sakamoto and the resulting image loop between the two pianos, repeating again and again. Sakamoto continues to add more sounds to the loop. The repeating images and sounds, flowing forth like a waterfall between the two pianos, grow in layered richness and complexity.
(translated from Japanese by Dano O'Neill)