>> phonesthesia

from: http://www.arts.gla.ac.uk/SESLL/EngLang/LILT/phonaes.htm

Phonaesthesia occurs when certain sounds become associated with certain meanings, even though they do not attempt to imitate the sound (as in onomatopoeia). For example, it could be argued that is a phonaesthetic combination of sounds (or phonaestheme) in English in words such slip, slippery, slide, slither, sloppy, slimy, sleazy. The meanings are associated with wetness or greasiness, and gradually take on unpleasant connotations. You could probably add more words to the list (but you could also think of words, such as slant, which do not share this feature).
Notes


1. Writers such as Charles Dickens sometimes exploit phonaesthesia in the names they give their characters, such as Scrooge. Are there other names of characters in literature which predispose the reader to like or dislike the character? It is also exploited in names for products such as breakfast cereals.

2. Extended examples are given in David Crystal, The English Language (1988).



Concept
Phonaesthesia is generally thought to be specific to particular languages. Related languages may exploit the same patterns, but there are also differences.



Figures of speech
See also Onomatopoeia

>> Erkki Huhtamo, "Elements of Screenology", 2001

from: http://lizard.artun.ee/~raivo/imke/texts/huhtamo_WRO%2001Elements%20of%20Screenology.htm

A covered framework, partition, or curtain, either movable or fixed, which serves to protect from the heat of the sun or of a fire, from rain, wind, or cold, or from other inconvenience or danger, or to shelter from observation, conceal, shut off the view, or secure privacy; as, a fire-screen; a folding-screen; a window-screen, etc.; hence, such a covered framework, curtain, etc., used for some other purpose; as, a screen upon which images may be cast by a magic lantern; in general, and shelter or means of concealment.

Definition of 'screen', The Century Dictionary and Cyclopedia, 1911 (1889)

Considering the centrality of screens in contemporary media culture, there have been surprisingly few attempts to define their "essence". True, in spite of their ubiquitous presence screens are strangely evasive, hard to grasp. They are constantly metamorphosing, appearing in new places and new forms. There are "Big Screens" and "Small Screens". Some are flat, some fat, attached to a box. Some are like the sun - active, radiating "life" of their own - while others are like the moon, passive, reflecting light projected at them. There are screens observed from a distance, and others touched and interacted with, held in one's hand. How to formulate a definition that would embrace them all? Does it even make sense to ask such a question?

This article is a preliminary investigation toward a historical phenomenology of the screen, or what I call "screenology". My treatment of the topic is based on one main premise: in spite of their ubiquity, screens have a history, which should be traced. Although there has been work done on specific areas (for example Siegfried Zielinski's research on the relationship between cinema and television and Lev Manovich's studies on the archaeology of the computer screen) the general history remains largely unwritten.[1] However, simply writing a chronicle of different kinds of screens would not make much sense. Screens should not be studied in isolation of the apparata they are part of. The notion of apparatus comes from cinema studies: it comprises not only the technical system, but also the elements of the viewing situation, including the relationship between the screen and the viewer, which is both physical and imaginary.[2] The viewer is physically related to the screen in the (viewing) space, and simultaneously mentally related to the space on the screen. The notion of the screen changes in time, and so does this relationship.

For historical reception studies the viewing experience has usually been a difficult challenge. Save in some rare cases, we don't have documented evidence about what went on in the viewers' heads in front of the screen. The viewers' attitudes have to be reconstructed indirectly, through secondary source material, as Miriam Hansen has demonstrated in her studies about early film spectatorship.[3] Although we cannot enter the individual viewer's head, we can at least try to understand the general conditions that prevailed in different situations and influenced each viewing experience. We can, for example, look at the constitution of the apparatus itself, including the design of its elements, for hints about the kinds of experiences it may have triggered. We can also use "projective" material, like literary texts, popular cartoons and other forms of ephemera to provide more clues and to verify our hypotheses. Still, the aim is to reconstruct "frameworks of possibilities", rather than try to determine the actual readings by individual viewers and audiences.

The ultimate goal is the history of "screen practice(s)", to adopt a concept used by Charles Musser in his studies of early and pre-cinema.[4] Such a history should comprise not only the evolution of different kinds of screens and the interconnections between them, but also account for their uses as part of different media apparata and within changing cultural, social and economic settings. This article provides a first step toward such a synthesis by identifying and discussing some of the key ingredients of such a history. The basic questions are simple, but the answers are difficult: how were our 20th century notions of the screen anticipated in earlier times? What connections, if any, are there between these "screens" from different times and places? As should be clear by now, this article does not look for an immutable "essence" of the screen; if the screen has an essence, it lies only in the sum total of all the historical manifestations of different screen practices, not beyond them like some Platonic idea.

Two litle Skrenes

According to the Oxford English Dictionary, the foremost authority on the history of the English vocabulary, the word "screen" first appears in texts from the 14th and the 15th centuries, but its etymological origins remain unclear.[5] In the 16th century, and probably earlier, it was used to refer to a "contrivance for warding off the heat of fire or a draught of air". The screen meant, above all, a floor-standing piece of furniture, consisting of a sheet of lighter, often translucent material (paper, some kind of fabric, etc.) stretched in a wooden frame. There were also smaller handheld versions for ladies; a text from 1548 speaks about "Two litle Skrenes of silke to hold against the fier".[6] In addition to their main purpose, the often richly decorated hand-screens were also objects of fashion, aesthetic pleasure, and erotic play. Gradually the screens gained new connotations. Beside the natural elements, they were said to provide protection from "other inconvenience or danger, or to shelter from observation, conceal, shut off the view, or secure privacy", as the Century Dictionary and Cyclopedia (1911, orig. 1889) stated.[7] Whether from heat, cold or an intruding gaze, the screen was above all seen as a surface that protects a person by creating a barrier against something uncomfortable or threatening.

It was during the early 19th century that the word "screen" began to attain meanings that anticipated its current uses within media culture as a means of displaying and transmitting images. The earliest such occurrence recorded in the Oxford English Dictionary comes from 1810 and reads: "To make Transparent Screens for the Exhibition of the Phantasmagoria". This represents a clear departure from the domestic sphere and entry into the world of public entertainment. Phantasmagoria was a show, which enjoyed great popularity around the turn of the 18th and 19th century. It was a variant of the older magic lantern projections, but with its own characteristics. In Phantasmagoria the audience was shown images projected from behind the screen with a highly mobile magic lantern (often mounted on wheels and moving along rails). One of the aims was to create a total sensory experience. This goal was served by the hidden technology. Phantasmagoria showmen did their best to keep their machinery secret; they pretended that their show had nothing to do with the old magic lanterns. They even made efforts to hide the presence of the screen itself by plunging the audience in total darkness and opening the curtains only then. The projected figures were presented as "apparitions" flying freely through the hall. To achieve this, inventing ways to make the screen semi-transparent - the easiest of which was making it wet - was crucial.

Of course, such an explanation of the emergence of the screen as a projection surface is too simplistic. The word may not have been used in such a meaning before 1810 (I have some doubts about this), but "screen practice" as a phenomenon goes certainly much further back in time. Phantasmagoria was based on earlier traditions of showmanship involving screens. Not only was it a further development of the travelling magic lantern show, it also built on the shadow show. Although shadow theatre seems to have originated in Asia (found in many places from Turkey to India, China and Indonesia), it became popular in Europe in the 17th and 18th centuries.[8] Most versions of the shadow theatre were based on essentially similar arrangements of the apparatus as Phantasmagoria. The audience sits in front of the screen, while the performers operate their shadow puppets behind it, between the screen and the light source.[9] The audience only sees the moving shadows on the screen, not the "machinery" creating them. In Phantasmagoria the use of shadow puppets was replaced by "fantascopes" (special magic lanterns) and projected lantern slides.[10]

In the magic lantern shows given by travelling showmen from the late 17th century on, the apparatus had been arranged differently: the audience frequently gathered around the showman and his magic lantern, which was placed fairly close to the screen. This arrangement was partly necessitated by the weakness of the illuminants available (candles or simple oil lamps), but it also emphasized the traditional role of the showman as a storyteller, who illustrated his stories with projected images. For people not familiar with such shows the presence of the mysterious "projection box" hardly diminished the "magic" of the event. Indeed, it may have served as an extra attraction. In similar fashion, early film audiences often admired the cinematograph as a technological marvel as much as the moving pictures it produced. Such a novelty easily wears off. By hiding the magic lantern behind the screen the Phantasmagoria showmen managed to re-create the lost mystery, utilizing to the full the possibilities of the new Argand lamp, a greatly improved oil illuminant. Yet in time also the Phantasmagoria lost its appeal, and the magic lantern became visible and attractive again, re-designed as a gorgeous instrument, a marvel of Victorian science.

Eliza on the Screen

During the 19th century the connection between magic lantern shows and projection screens became semantically well established. As just one example among many, a text from 1846 stated: "Magic lantern is a species of lucernal microscope, its object being to obtain an enlargened representation of figures, on a screen in a darkened room." (1846)[11] Throughout the 19th century the size of the screen, the auditorium and the projected image grew larger. This was made possible by the development of new, more powerful illuminants (oxy-hydrogen limelight, electric carbon-arc), yet the social force motivating this development was the increasing demand for entertainment and visual instruction among the new mass audiences, particularly in cities. In the late 19th century, especially in America, the magic lantern was even taken outside to project huge advertisements and election results on public buildings, now re-defined as gigantic projection screens.[12] In less than a century, the word screen had taken on totally new meanings, in line with the emergence of the urban, technological media society. If it had been a thing that protected a single person from something unwanted, it now exposed a whole group of people to the visual extravaganza of capitalist commodity culture.

Against this background it is anything but surprising that the word "screen" was assimilated into the early film culture. As a screen practice, early silent film showmanship was in many ways (although not exclusively) a continuation of the magic lantern show. Until well into the 1910s (and even later) most film presentations were actually hybrid forms of films, lantern slides, phonograph concerts and live stage performances. As film production and exhibition consolidated their roles as major new entertainment industries from the 1910s on, the other attractions gradually faded to the background: the center was reserved to the pleasures of the screen. With this development, the word often came to be used metonymically, meaning the film culture itself, written with capital letters: The Screen. Already in 1910 the Moving Picture World wrote that "people like to see on the screen what they read about", referring to their film preferences. And when Mrs. P. Campbell stated in 1920 that she felt "much too aged for Eliza on the Screen", she of course referred to acting in the movies, working in the film industry.

At some point the word "Big" was added in front of the "Screen". When and why this happened needs some further research. I suspect it had something to do with the appearance of new competing screen practices after the Second World War, particularly the television. The "Small Screen" promised to bring "the events as they happened" directly into the living room. As a response to this challenge, the film industry promised even more magnificent spectacles. Stretching the cinema screen to gigantic dimensions was the solution offered by Cinerama, Todd-AO, Cinemascope and various other systems in the 1950s. The most extreme form was the curved giant screen of the Cinerama, on which films were projected from three projectors simultaneously. With Cinerama, the expansion of the screen reached a paradoxical conclusion: by covering the spectators' total field of vision the screen in a sense disappeared; there was no sense of frame marking the border between the real and the imaginary. The screen turned into an environment which enveloped the audience completely. This anticipated more recent spectacles like IMAX theatres and virtual reality. Something similar had been, however, already achieved by the panorama, another large scale visual entertainment, a hundred years earlier.

Moonlight transparencies

What about the Small Screen? Can we locate its etymological origins? The answer remains more speculative than in the case of the big screen. First of all, one might want to recall the fact that from early on the fire-screens were often embellished with images. During the Victorian era the large folding screens used at homes for various purposes often became real collages of all kinds of printed images, recalling the countless "scrap books" created by women and children as their pastime (and even the contemporary habit of covering the door of the refrigerator with postcards, photos and little magnets). Although the images served primarily a decorative function, such screens anticipated the future development of media culture by displaying the enormous proliferation of cheap mass produced images in the 19th century, made possible by advances in printing and image reproduction technologies (lithography, photography, etc.). Indeed, the habit of decorating screens with images became so common, that mediocre artworks were sometimes compared by critics with such banal screens.

Already in the late 18th century the idea of the fire-screen was adapted to the purpose of displaying transparent paintings in new and stunning ways. Such paintings, "moonlight transparencies" or "diaphanoramas", like those by the Germans Georg Melchior Kraus and Franz Niklaus König, were mounted on floor-standing wooden frames.[13] They really only came to their right when illuminated from behind, glowing in brilliant colours. In the 19th century forms of such back-lighted images proliferated, ranging from "lithophanes", porcelain images displayed on lamp-shades or in decorative wooden or metal frames, to domestic viewing machines like the massive Megalethoscope, designed for the viewing of large albumen photographs with hand-coloured filters attached to the backside.[14] Peering into a viewing "hood" and simultaneously opening a door at the back of the device, the black and white photographs were transformed into fabulous coloured spectacles. These, and many other kind of "screens" anticipated the future role and placement of the television screen, although their potential for transmitting visual information or depicting movement was limited.

Going through dictionaries, we also find other meanings that have connected small screens with media images. In the 19th century the word was used to refer to upright frames for displaying photographs, both privately and in public exhibitions. In 1888, for example, a person wrote about "some of the most delightful panel screens for photographs I ever set eyes on".[15] More interesting, however, is the connection with the photographic camera itself. The "focusing screen", or the "screen of ground-glass" (1879) was defined as "a flat piece of glass on which the image formed by a camera lens is focused prior to making the exposure".[16] This common principle was actually inherited from an earlier device, the camera obscura, which anticipated the photographic camera and influenced its construction. In the camera obscura, known already in the middle ages, an image of the outside world is formed inside a darkened box, by means of rays of light entering it through a tiny hole.[17]

Especially since the Renaissance, after one had learned to place a lens into the "pinhole" for sharper image, camera obscuras became widely used both as artists' tools and as popular pastime. From the point of view of the development of the small screen this device is extremely interesting, although very few observers have noted the relationship so far, probably because the camera obscura is merely seen as a primitive precedessor of the (still) photographic camera. In smaller camera obscuras the image was often directed by means of an internal mirror (45 degrees) to a ground glass on top of the device. By placing a transparent sheet of paper on the ground glass, the artist was able to sketch the outline of the landscape. There were also room-sized camera obscuras, often situated at well-known tourist locations, by the seaside or on hilltops. The image of the outside world was directed by means of a lens and a mirror from the top of the room onto a horizontal table in its center. Visitors stood around the table and admired the moving scenery from the outside, often pointing at details with their finger. Both the ground-glass and the table functioned essentially as framed screens.

Screen-Plays

It is important to remember that all kinds of camera obscuras transmitted a live image and displayed it on a framed surface. Although technically simple and involving neither electronics nor antennas, they clearly anticipated the principle of the television, defined in 1926 by Nature: "Every possessor of a 'televisor' will be in a position to see on his screen the performers in operas and plays as well as hearing them."[18] Indeed, in 1879 a cartoonist working for the British magazine Punch and envisioning the tele-vision technology of the future, captioned his creation as "an electric camera obscura", purportedly invented by Thomas Edison.[19] The visionary cartoon showed a panoramic flat screen, mounted on the wall above a fireplace - a situation which has not been realized yet, although plasma screen technology now finally promises to fulfill the expectations. The screen in Punch also provided two-way communication, another fantasy which has never been fully made its breakthrough, in spite of innumerable predictions (and working prototypes!). Most screens still serve one-way traffic, although the proliferation of the computer screen is quickly changing the situation.

Beside its resemblance to television, the camera obscura also anticipated the computer screen by encouraging a tactile relationship. The image of the camera obscura was not meant to be just observed from a distance - it could be touched, by the tip of the pen or simply by one's finger. In its time, this created an ontologically interesting and novel situation: a kind of tele-touching, caressing living and moving entities from a distance, by means of a technical apparatus. Although this situation seems alien to the television spectatorship, it was encouraged in the 1950s in one of the early experiments of interactive television, the American series Winky Dink and You. Children were encouraged to draw on the television screen (actually on a sheet of transparent plastic attached to the screen) by "Magic Pens" according to the instructions given by the host John Barry while the program was running. The activity of the child in this situation is not all that different from that of an 18th century artist sketching a landscape with the help of his camera obscura. The spectatorial model proposed by Winky Dink and You never became a standard in the rigid world of TV broadcasting, but it has been in a way been realized by the introduction of paint programs and edutainment multimedia for personal computers.[20]

This article has not been meant as an exhaustive treatment of the history of screen practices. Rather, it has only hinted at the wealth of material and approaches available. Connections need to be established and new data uncovered, especially about the metaphorical uses of the word. As the Oxford English Dictionary demonstrates, the word screen has been given a large number of meanings, only a few of which have been dealt with here. How are these meanings connected? Is there any meaningful link between an 18th century fire-screen and a 20th century cathode-ray tube, other than the fact that both are "lighted" or "heated" from behind? Wasn't the traditional screen meant to isolate the person, to protect him/her from heat or a gaze, to increase his/her comfort and privacy? Isn't the function of the television screen the opposite, to expose the viewer to the "heat" and "obscenity" of commercial media culture, and to invite the public sphere to invade the private? It might be claimed, however, that the relationship between issues like private/public is never so clear-cut. While blocking from view, the traditional screens also raised curiosity and desire towards the other side (best demonstrated by the countless Japanese wood-block prints showing people observing the shadows of others cast on the paper screens serving as walls). The television screen also provides privacy by offering a safe voyeuristic vantage point to observe the event "on the other side". While exposing it also protects.

There would be other intriguing parallels not mentioned in this article. For example the history of the mirror and the discourses surrounding it should be taken into consideration. Observed from a cultural and mental historical point of view, the mirror has never been seen merely as a device reflecting your own image here and now; for centuries, it has been a vehicle for intricate spatial, poetic and erotic "screen-plays". In literature and in pictorial traditions it has often been treated as a kind of screen - a fantastic communication device capable of telling visual stories, displaying the future or virtually uniting people separated by physical distance. It is enough to think about a well-known fairytale like the Beauty and the Beast with its enchanted mirrors - this connection did not escape the attention of Jean Cocteau, whose film La Belle et le Bete (1946) used mirrors as information and communication screens in poetic and imaginative ways. A history of screen practices should not overlook such fantasies and discourses, which are often intertwined with more "real" - meaning: more material, more tangible - phenomena in strange ways. So perhaps it was not a coincidence, after all, that Cocteau's film appeared on the "Big Screen" exactly at the moment when television broadcasting was beginning its triumphal march into the living room.

Š Erkki Huhtamo 2001


1. See Siegfried Zielinski: Audiovisions. Cinema and Television as entr'actes in history, translated by Gloria Custance, Amsterdam: Amsterdam University Press, 1999 (orig. in German 1989); Lev Manovich: "Towards an Archaeology of the Computer Screen", Cinema Futures: Cain, Abel or Cable? The Screen Arts in the Digital Age, edited by Thomas Elsaesser and Kay Hoffmann, Amsterdam: Amsterdam University Press, 1998, pp. 27-43.

2. See The Cinematic Apparatus, edited by Teresa de Lauretis and Stephen Heath, London and Basingbroke: Macmillan, 1980.

3. Miriam Hansen: Babel & Babylon. Spectatorship in American Silent Film, Cambridge and London: Harvard University Press, 1991.

4. See particularly Charles Musser: The Emergence of Cinema. The American Screen to 1907, New York: Charles Scribner's Sons, 1990. It should be noted that in the title of his book Musser uses the word "Screen" as synonymous with the institution of the cinema, which reflects an actual historical usage.

5. Much the same goes for the French "écran", which most dictionaries, including The Oxford English Dictionary, see as "closely corresponding with" the history and the meanings of"screen". All references to the Oxford English Dictionary (OED) are to the II edition, edited by J.A. Simpson and E.S.C. Weiner, Oxford: Clarendon Press, 1989.

6. OED, vol XIV, "screen".

7. The Century Dictionary and Cyclopedia (CDC), New York: The Century Co, revised and enlarged edition, 1911 (orig. 1889), Vol VIII, "screen".

8. See Olive Cook: Movement in Two Dimensions, London: Hutchinson, 1963.

9. An interesting exception was late 19th century "Ombromanie", the art of hand shadows. Here the shadow artist stood in front of the screen and revealed his "machinery" (his own hands) to the audience. Demonstrating the skill and mastery of the performer was as important as the end result.

10. Although the shadow theatre was imported to Europe from the East, by late 18th century it enjoyed widespread popularity. Its influence can be felt in certain aspects of Phantasmagoria, and some shadow showmen are known to have experimented with the magic lantern as well. The true synthesis of these screen practices, however, took place in Japan. The Japanese Utsushi-e show, which emerged in the early 19th century, is an original form of popular media theatre, in which hand-held, highly mobile magic lanterns have taken the role of shadow puppets.

11. OED, vol XIV, "screen".

12. Such projections were often pictures on the front pages of popular newspapers like Frank Leslie's Illustrated Newspaper (see f.ex. Nov. 23, 1872 and Oct. 25, 1884).

13. For examples, see Sehsucht. Das Panorama als Massenunterhaltung des 19. Jahrhunderts, Bonn: Kunst- und Ausstellungshalle der Bundesrepublik Deutschland & Stroemfeld/Roter Stern, 1993, pp.198-199.

14. Massive public spectacles like Daguerre's and Bouton's Diorama, which displayed gigantic slowly transforming paintings by means of manipulated back-lighting, were based on similar principles. All kinds of tiny toy versions, like the French Polyorama Panoptique, were also made.

15. OED, vol XIV, "screen".

16. Item.

17. The most complete history of the camera obscura is John Hammond: The Camera Obscura. A Chronicle, Bristol: Adam Hilger Ltd., 1981.

18. OED, vol XIV, "screen".

19. See Émmanuelle Toulet: Cinématographe, invention du siecle, Paris: Decouvertes Gallimard & Reunion des musees nationaux, 1988.

20. One reason why Winky Dink and You failed may have been the simple fact that children who did not own the magic pens and the drawing screen began to draw directly on the cathode ray tube, obviously destroying the TV set!

>> Golan Levin, "Dialtones", 2001

from the project's website: http://www.flong.com/storage/experience/telesymphony/index.html

Dialtones ented sonic phenomena and musically interesting structures. Moreover, by directing our attention to the unexplored musical potential of a ubiquitous modern appliance, Dialtones inverts our understandings of private sound, public space, electromagnetic etiquette, and the fabric of the communications network which connects us.

Dialtones was presented in two consecutive concerts in September, 2001, as a co-production of Golan Levin and the Ars Electronica Festival, and in seventeen performances in May/June 2002 at the Swiss National Exposition.

Dialtones begins with a brief preparation phase prior to its performance, during which the members of the audience register their wireless telephone numbers at a cluster of secure Web kiosks. In exchange for this information, the participants receive seating assignment tickets for the concert venue, and new "ringtones" are then automatically downloaded to their handsets. During the concert itself, the audience's mobile phones are brought to life by a small group of musicians, who perform the phones en masse by dialing them up with a specially designed, visual-musical software instrument. Because the audience's positions and sounds are known to the Dialtones computer system, the performers can create spatially-distributed melodies and chords, as well as novel textural phenomena like waves of polyphony which cascade across the crowd; these musical structures, moreover, are visualized by a large projection system connected to the performers' interfaces. Towards the end of its half-hour composition, Dialtones builds to a remarkable crescendo in which nearly two hundred mobile phones peal simultaneously. It is hoped that the experience of Dialtones can permanently alter the way in which its participants think about the cellular space we inhabit.


ARTISTS' STATEMENT

Wireless telephony has quickly become an indispensable aspect of modern life. Today, one out of ten people on the planet possesses a mobile phone; over the next three years, according to the industrial analysis firm The Gartner Group, this market is expected to increase by almost a billion new users [1]. Ironically, the astonishing eagerness with which we have adopted mobile phones is matched by our almost equal repulsion on the occasion of a cell phone's ringing. Mobile phones now infuse our theaters and public spaces with the least welcome details of our neighbors' intimacies, and perforate our private lives with the sonic machinery of electronic commerce. Our emotional reactions to these interjections can even outstrip the veneer of our professional identities: when ringing mobile phones interrupted keynote speakers at a recent telecommunications conference in Finland, the conference manager became enraged and threatened to get a radio-frequency scrambler to silence the din [2]. Caught between adoration and irritation, we have come to regard our intimate communications apparel with a deep ambivalence.

In the hype, hate and hypnosis surrounding the mobile phone, its potential as an ingredient of art has been largely overlooked. As with the proverbial fish who would never discover water, we take for granted that we are immersed in cellular space, our imaginations dulled by the extraordinary ubiquity of our wireless devices. Announcers at every modern-day concert command us to turn off our cell phones, but what Cagean aesthetic possibilities might we discover in leaving them on? What deranged beauty might we find, or what might we learn about our interconnected selves, in their high, pure tones? The mobile phone's speakers and ringers make it a performance instrument. The buttons make it a keyboard and remote control. Its programmable rings make it a portable synthesizer. Yet, although no sacred space has remained unsullied by the interruptions of mobile phone ringtones, there is no sacred space, either, which has been specifically devoted to their free expression. In the context of this lack, and in the context of our society's contradictory attitudes towards wireless communication technologies, Dialtones is proposed.

If our global communications network can be thought of as a single communal organism, then the goal of Dialtones is to indelibly transform the way we hear and understand the twittering of this monumental, multicellular being. One of Dialtones's strategies for doing so is the musical reification of this organism's sprawling and enveloping omnipresence. By placing every participant at the center of a massive cluster of distributed speakers, Dialtones makes the ether of cellular space viscerally perceptible. In a rejoinder to the eminent electronic composer Iannis Xennakis— who once complained that all electronic music sounded alike, because it would inevitably emanate from the same pair of speakers —Dialtones's radical surround-sound is at once musically and phenomenologically unique.

In an appropriate acoustic environment, the sporadic triggering of calls to mobile phones can evoke the placid chirps and trills of crickets, cicadas, frogs and birds. If hundreds or even thousands of mobile phones were to ring simultaneously, by contrast, the result would be an unimaginably seething, engulfing cacophony. Between these two textural extremes lies an enormous terrain of more musically familiar possibilities: gently shifting diatonic chord progressions, distributed and aggregate melodies, roving clouds of spatialized sound-clusters, and pointillistic hyper-polyphonies. Over the course of its half-hour duration, Dialtones explores sequences and combinations of each of these possibilities, scaffolded throughout by a set of recurring harmonic themes and slowly-evolving melodic phrases. Ultimately, the exact composition of Dialtones is a function of both the scored performance produced by the project's staff, and the specific settings of the phones brought by the concert's attendees.

In Dialtones, the phones, and not their owners, speak to one another. By summoning a communication between communications technologies in which there is no interlocutor, Dialtones invites its participants to perceive an order in what is otherwise disorganized public noise, and ratify it as a chorus of organized social sound. Thus the overdetermination of the world of Work is countered with an equally determined Play, as the ringing of mobile phones—ordinarily, the noise of business, of untimely interruptions, of humans enslaved to technology—is transformed into a sound of deliberate expression, startling whimsy, and unconventional beauty.

TECHNICAL REALIZATION

Dialtones' technical realization is broadly divided into three distinct software subsystems: (A) the means by which the audience's mobile phones were registered (prior to the performance) into a networked database; (B) the means by which the audience's cell phones were computationally dialed (and thereby performed) during the concert itself; and (C) the telephony middleware which communicated the dialing requests from the performance system to the infrastructure of the local Mobile Switching Center. In addition, two special optical subsystems added visual and diagrammatic dimensions to the performance: (D) a vertical video projection system, in which spots of light were cast from above onto actively ringing audience members; and (E) an assembly of autonomous miniature lights which visually augmented the audience's highly-localized cellular activities. In this section, each of these mechanisms is treated in turn.

A. Prior to the Dialtones concert, audience participants register their mobile phone numbers (and model numbers) at special Web-based terminals placed outside and around the performance venue. ASP-based CGI scripts are used to store this information in a SQL database. At the same time, the scripts also use a ticketing algorithm to issue the audience member an assigned seat in the concert auditorium. Depending on the make and model of the participant's phone, it can be possible to programmatically modify its ringtone at this time; if so, a specially-composed ringtone is encoded in the RTTTL (ringtone text transmission language) data format and transmitted to the user's phone as an SMS message. The Dialtones staff composed more than 100 customized ringtones for the concert, wrote special software to convert these ringtones from MIDI sequences into RTTTL, and created a special CGI system to transmit these tones automatically to the audience's phones. In the Linz performances, a small number of preconfigured phones were also available as temporary loans for phoneless participants.

B. After the participants' phone numbers and models have been collected and stored in a database, Dialtones itself is performed live on a custom software instrument which makes use of this database. This performance system consists of an interactive graphical software interface, which represents each mobile phone in the audience as a spatialized cell in a visual grid. During the performance, the performers place "animated paint" into specific cells in the visual grid; these actions trigger the ringing of the corresponding mobile phones in the audience. It is important to emphasise that all of the phone dialings are executed "by hand"; that is, they are set into motion by the direct action of a human performer. The performance instrument is implemented as an OpenGL-based Windows application.

C. The Windows-based performance software transmits dialing requests over a TCP/IP LAN to a nearby Linux-based telephony server. This machine uses an Aculab telephony card to convert these requests into actual phone calls. The Aculab card transmits the phone calls over two dedicated E1 lines (primary-rate 2Mbit ISDN connections), directly into the Mobile Switching Center (MSC) of the local mobile service provider (in Linz, A1 Mobilkom Austria; at the Swiss National Exposition, Swisscom Mobile). The local Base Transceiver Station (BTS, or cell antenna) in the location of the concert venue was specially modified by the provider in order to allow at least 60 simultaneous signalling channels.

D. The audience-orchestras at the two Linz performances each consisted of 200 participants, who were arranged in a 20x10 seating grid. The performers' grid-based graphical interface is projected onto the audience from above, and carefully registered with their seats. As a result, each participant is lit up by a personal spot of light whenever their handset is rung. The concerts performed in Linz used a 12000 ANSI Lumen Barco ELM R12, with a special wide-angle lens, for the projection of these spotlights.

In order to more clearly show how the spots of light play across the audience— indicating which people are ringing at any instant— a very large, multi-panel Mylar mirror (6x12 meters) is erected at an angle above the crowd.

Of the 200 participants in Linz—whose phones hailed from 13 different countries—as many as sixty could be dialed at any one instant. Each performance lasted approximately 28-30 minutes, and entailed the placement of more than 5000 phone calls. The seventeen concerts presented at the Swiss National Exposition used orchestras of 99 participants [9x11], of whom any 60 could be dialed simultaneously.

E. The last visual subsystem consisted of a set of two hundred small keychain lights, which were distributed to the audience-participants at the time of their pre-concert registration. These inexpensive and autonomous devices, which are sensitive to energy in the 800-1900 mHz radio band, illuminate a small red LED when they are within one meter of a ringing mobile phone. Despite their small size, the darkness of the concert hall made it possible to observe the flashing of these lights, which were also reflected in the large suspended mirror.

Generally speaking, these keychain lights would flash about two seconds prior to the ringing of a nearby mobile phone, while the overhead video projection system would typically enable its corresponding spotlight within half a second after the phone had begun to ring. These minute differences in timing had the effect of diffusing events over time, creating micro-anticipations and multilayered syncopations between the light and sound of the performance.

The combined effect of the telephone rings with their synchronized visual phenomena was to render each participant as an audio-visual pixel, a twinkling particle in an audio-visual substance—and the visitors, as a group, could at once be audience, orchestra and (active) score.

THE COMPOSITION

The Dialtones composition consists of three major subsections, or "movements", each approximately ten minutes long. The first section is produced entirely through the ringing of the mobile phones of the 200-person audience; these phones were completely unamplified by any means. The second section, a "solo" movement, is performed by Dialtones staff member Scott Gibbons on ten amplified (but otherwise unmodified) mobile phones. In the third section, the soloist plays together with the ensemble.

The goal of Dialtones' three-part structure is to introduce the contrasting aesthetic possibilities of virtuosic real-time cellphone performance ("mobile phone jockeying") on the one hand, with coordinated-ensemble handheld-music on the other. In addition to yielding a variety of sonic contrasts, this structure also allows for the exploration of a broad range of musical interaction-models: from the deeply practiced (e.g. Gibbons' solo performance), to the entirely visual (e.g. the graphical interface controls used by Levin and Shakar to interactively perform the audience phones), to a lightweight model of consumer participation (e.g. through one's selection/purchase of a phone model, negotiation of its ringtone, and manner of displaying it during the performance).

Within each of the three movements, the composition is structured as a sequence of sound-textures. These texture-segments are realized as interestingly distinct combinations of ringtones; while the sound of one texture resembles a forest full of twittering birds, another consists of pure drones, and recalls the sound of a pipe organ. There are about fifteen sound-textures in all, each approximately two minutes long. Although the order and duration of these sound-textures is explicitly scored, the moment-to-moment details within each texture are left to the improvisation of the Dialtones performers.

The final movement of the Dialtones concert concludes with a climactic crescendo involving both orchestra and soloist. During the course of this section, the Dialtones soloist Scott Gibbons initiates the "ringing" of a phone's vibrator, transduced by a flat piezoelectric microphone and amplified by a subwoofer. At the same time, increasingly greater numbers of phones are introduced until the maximum possible number of simultaneous rings (60) is achieved. At this point, louder phones are swapped with quieter ones, and the selection of rings shifted around the orchestra until, within the space of a few seconds, all 200 of the audience phones have been triggered.

>> Golan Levin, Zachary Lieberman, "Messa di Voce", 2003

"A performance and installation in which the speech, shouts and songs produced by two vocalists are augmented in real-time by custom interactive visualization software." (Golan Levin, Zachary Lieberman)


from the project's website:

"overview:
Messa di Voce (Ital., "placing the voice") is an audiovisual performance in which the speech, shouts and songs produced by two abstract vocalists are radically augmented in real-time by custom interactive visualization software. The performance touches on themes of abstract communication, synaesthetic relationships, cartoon language, and writing and scoring systems, within the context of a sophisticated, playful, and virtuosic audiovisual narrative.

Tmema's software transforms every vocal nuance into correspondingly complex, subtly differentiated and highly expressive graphics. These visuals not only depict the singers' voices, but also serve as controls for their acoustic playback. While the voice-generated graphics thus become an instrument which the singers can perform, body-based manipulations of these graphics additionally replay the sounds of the singers' voices — thus creating a cycle of interaction that fully integrates the performers into an ambience consisting of sound, virtual objects and real-time processing.

Messa di Voce lies at an intersection of human and technological performance extremes, melding the unpredictable spontaneity and extended vocal techniques of two master composer-improvisers with the latest in computer vision and speech analysis technologies. Utterly wordless, yet profoundly verbal, Messa di Voce is designed to provoke questions about the meaning and effects of speech sounds, speech acts, and the immersive environment of language.


background:
Messa di Voce, like our previous augmented-reality artworks RE:MARK and Hidden Worlds of Noise and Voice, is concerned with the poetic implications of making the human voice visible. In Messa di Voce, the core technology which makes this possible is a custom software system which integrates real-time computer vision and speech analysis algorithms. Specifically, a computer uses a video camera in order to track the locations of the performers' heads. This computer also analyses the audio signals coming from the performers' microphones. In response, the computer displays various kinds of visualizations on a projection screen behind the performers; these visualizations are synthesized in ways which are tightly coupled to the sounds spoken and sung by the performers. Owing to the head-tracking system, moreover, these visualizations can be projected such that they appear to emerge directly from the performers' mouths. In some of the visualizations, projected graphical elements not only represent vocal sounds visually, but also serve as a playable interactive interface by which the sounds they depict can be re-triggered and manipulated by the performers.

Our group's interest in phonesthesia, or phonetic symbolism, is at the heart of the Messa di Voce project. According to this idea, the sounds of words tend to reflect, to some extent, associated connotations from other perceptual domains such as shape or texture. A classic illustration of the phonesthetic principle can be found in Wolfgang Köhler's pioneering psychology experiment from 1927, in which he asked subjects, "which of the figures below represents the sound maluma, and which one represents the sound takete?" Nearly all viewers respond with the same answer — suggesting rich research opportunities for artmaking and cognitive psychology alike.
Messa di Voce brings together findings from phonesthesia research with our group's shared interests in abstract language, extended vocal techniques, interactive systems, and live audiovisual performance. In our concert, these ideas take shape in a series of twelve brief vignettes which explore different symbolic, tactile and audiovisual aspects of phonesthetic relationships. The entire Messa di Voce performance generally runs 30 to 40 minutes in length."

>> Toshio Iwai, "Electroplankton"

the product's website:
http://electroplankton.nintendo-europe.com/enGB/index.html

electroplankton is in a way Toshio Iwai's portfolio, a portfolio that invites people to play with. It is a special edition of the nintendo gameboy that features ten different soundobjects Iwai developed. By playing electroplankton, users can create their own melodies. the term "play music" gets a new connotation by that.

>> Toshio Iwai, "Tenori-on", 2001-2007t

>> tenori-on:


from the yamaha-website (for which iwai developed the tenori-on):

Tenori-on captures the imagination of so many different people. From the creative and professional media artist to non-musicians alike, this new form of musical interface is opening many new avenues of musical exploration and creativity.

A recent independent dissertation by UK-based Graham Wynne is a fascinating document as he explores the extent to which instruments such as Electroplankton and Tenori-on are overcoming barriers to creativity in modern music production. With related quotes from Toshio Iwai, Brian Eno, Steve Reich and Norman Fairbanks you can read Graham's dissertation HERE

more images can be found here: flickr.com/photos/kirstykomuso/1980268675/

"Toshio Iwai, the Tenori-on's inventor in conjunction with Yamaha, is a world renowned Japanese designer (who has also designed the Electro-plankton innovative interactive music game with Nintendo). The tenori-on device embodies visualisation of music, auditory and visual display and an intuitive interface for layering and performing sounds. Its basic operation can be sunstantially augmented by a sound expert by loading banks of your own samples, thereby enriching its auditory scope to a potentially limitless and individual set. Additional memory allows you store and record creations for later performance. As my research looks at digital means to augment conventional music devices and hyper instruments, aesthetic digital display, as well as the bimodality of auditory and visual display, this device encompasses all of those concerns. I am interested to use it, to study its usability and investigate the interaction of the visual and auditory output. It also ties into the 'gestural interaction with sound' theme of my ARC research.

Concept: Media artist Toshio Iwai and Yamaha have collaborated to design a new digital sonic interface for the 21st century, TENORI-ON.
A 16x16 matrix of LED switches allows users to play music intuitively, creating a "visible music" interface. The TENORI-ON is a unique 16 x 16 LED button matrix interactive sound device with a stunning matrix visual display.

Operation: It is simultaneously a performance input controller and display. By operating and interacting with the LED buttons and the light they produce you gain access to the TENORI-ON's operation modes. These include Score Mode, Random Mode, Draw Mode, Bounce Mode, Push Mode and Solo Mode.

TENORI-ON layers can be thought of as “performance parts” or “recording tracks.” The TENORI-ON has a total of 16 layers. Separate notes and voices can be assigned to each layer, and all layers can be played together in synchronization.
The 16 layers are divided into six performance mode groups as shown in the illustration below. The six modes have different note entry methods and operation. Up to 16 layers created using different modes can be combined for rich, complex musical expression.
A completed set of 16 layers is called a “block.”

The TENORI-ON can store up to 16 programmed blocks (16-layer groups) in memory, and you can switch from block to block instantly during performance.

You could, for example, create a musical composition in one block, then copy that composition to another block and edit it to create a variation of the original composition. Or you can load a number of previously-created compositions into separate blocks from an SD Memory Card and switch between them to create variation during playback.
See links above for full specs and info or to watch a Youtube video demo>/i> (mine coming soon to my blog) "

Toshio Iwai - Tenori-On from watz on Vimeo.

>> Toshio Iwai, Ryuichi Sakamoto, "Music Plays Images x Images Play Music", 1997

This piece won the Golden Nica in the Interactive Art category in 1997's Prix Ars Electronica

from Ars Electronica: http://www.aec.at/en/archives/prix_archive/prix_projekt.asp?iProjectID=2494

"Music Plays Images x Images Play Music" is a multi-media concert that uses the system of Toshio Iwai's piano piece and visualizes a musical performance by Ryuichi Sakamoto in real time.

The idea for this collaborative performance has its origins in Toshio Iwai's installation "Piano ­ as image media" created at the ZKM, Karlsruhe, Germany in 1995. Conceived upon Iwai's return to Japan when he met Ryuichi Sakamoto, "Music Plays Images x Images Play Music" is a multi-media concert that uses the system of Iwai's piano piece and visualizes a musical performance by Sakamoto in real time. Over the course of a year, Iwai, who lives in Tokyo, often corresponded by e-mail with Sakamoto, who lives in New York. During their correspondence, the concept for the collaboration expanded rapidly, developing from the simple idea of "Sakamoto playing Iwai's 'Piano ­ as image media'" to a performance that links, in various ways, musical sounds with visual images. The actual performance was given on December 16, 1996 at the Art Tower Mito in Mito City, Japan. The result of the collaboration was more than a "novel combination of music and image." Many elements including the actual presence of Sakamoto/Iwai, the machinery that is called the piano, the acoustic sounds produced by the piano, and the virtual images generated from computers were linked and developed in various ways. From the feedback produced, a radically new space of musicality-visuality was created. In addition, the performance was transmitted over the Internet and relayed world-wide. This experiment, the first of its kind, was an attempt to utilize the interactive medium of the Internet. People on the other side of the Internet were able to play the piano located in Mito by remote control and in turn Sakamoto had a session with their playing.

The Set-Up of the Stage
Two concert grand pianos are placed on the stage. A semi-transparent screen 8 meters in height and 10 meters in width is suspended above them. The lids of both pianos are removed and their interiors and keyboards clearly illuminated. The screen is irradiated with a beautiful shade of blue. A black and white video camera is placed on each of the pianos in order to film the movements of the performer's fingers and the piano keys. The video cameras project enlarged images of each piano keyboard onto the screen on either side of the pianos. Several computers are set up on a long narrow table placed on the right side of the stage.

The Content of the Performance
1) Sakamoto does not initially appear on the stage. Amidst the silence, the piano begins to play on its own without a player. An image of Sakamoto playing the piano gradually comes into view on the screen to the side of the piano. Sakamoto's image slowly disappears as the music ends.

2) Sakamoto appears on the left side of the stage. He begins to play the small keyboard placed there. As he plays this keyboard, the piano starts to play. A faint light gradually emanates from the piano. At a certain point, Sakamoto stops playing. However, the last musical phrase repeats itself automatically, slowly fading out.

3) Sakamoto plays the piano directly. The light-images that visually represent each note of the musical piece begin to rise from within the piano. The images capture not only the timing of the notes, but every aspect of the music, including the length of the notes and their loudness. They are represented in a way that the eye can recognize.

4) During Sakamoto's performance, Iwai gradually modifies the image. The lights, which correspond to each note of the piece, are transformed over time in various ways. They do not merely rise from within the piano and fade into thin air. They radiate from the center of the screen out in all directions or revolve in a whirling pattern, eventually disappearing into a vortex. The metamorphoses of the projected images influence Sakamoto's playing as he watches the various transformations.

5) The image-objects, composed of 88 elements ­ the same number as the number of keys on a piano ­ appear on the center of the screen. Each of the elements coincides with the keys of the piano and the objects are continually transformed according to Sakamoto's piano performance.

6) Clicking a computer mouse, Iwai places dots of light onto the screen. Whenever the dots of light fall on the piano, the piano begins to play. The dots of light then rebound to their original position, striking the keyboard at fixed intervals. Depending on where the dots of light are placed, the pitch of the notes and the patterns of repetition change. The number of dots of light are increased. The dots of light bouncing from the piano are all shifted back and forth from left to right. The whole musical scale changes according to how the dots of light are moved. The dots of light are transformed into rays and then polygonal figures.

7) The video camera films Sakamoto's entire body. The computer captures the image in real time. Sakamoto's silhouette is pixelized and then projected onto the screen above the piano. When Sakamoto moves and the image of his body overlaps with the piano, the piano plays. When Sakamoto moves toward or away from the camera, or when a close-up image of his hands is projected, the sound is changed dynamically. The images of Sakamoto's hands and body are what plays the machine called the piano. Iwai implements further modifications by zooming in on and rotating Sakamoto's image on the screen thereby slowly modifying the sound. Sakamoto continues to move his body while looking on.

8) A chessboard-like grid appears on the screen and begins to rotate. Sakamoto and Iwai take turns placing glowing dots of light onto the grid, thereby creating a repetition of short melodic patterns. As the music is created through this collaboration, Iwai transposes the musical pattern, changing the octave and altering the sounds and the images. Sakamoto continues to create new patterns. Finally, with the completed musical phrases as a backdrop, Sakamoto improvises on the piano. Iwai continues to modify the performance piece.

9) The music played by Sakamoto is turned into an image. The image that springs from the first piano and lands on the second piano creates a parabola in its fall. The second piano plays according to this image. The figure of Sakamoto performing on the first piano is projected next to the second piano with a delay that coincides with the imaged music arriving at the second piano. It appears as if there are two Sakamotos playing the two different pianos.

10) The short musical phrase performed by Sakamoto and the resulting image loop between the two pianos, repeating again and again. Sakamoto continues to add more sounds to the loop. The repeating images and sounds, flowing forth like a waterfall between the two pianos, grow in layered richness and complexity.
(translated from Japanese by Dano O'Neill)