Body≈Sounds: an emergent sonic practice
As a performer with a highly physical practice, the body and its capacity for movement, including its ability to generate and disrupt rhythmic structures, are of central concern. In this article, I discuss a number of body≈sound interfaces, which, collectively provide an overview of my approach to sonically extending the body. My aim with these works has been to augment a performer’s expressive abilities through the addition of musical (or other sonic) abilities. Each of the different approaches embodies different interrelationships between body, identity, interface, the aesthetics of sound output in relation to both interface and context, and the movement explorations afforded and supported. In each case the sonic extension magnifies or extends aspects of our embodied-ness. This amplification defamiliarises, or "makes strange," prompting us to focus our attention and question our assumptions, to see and experience our bodies anew.
'Making strange' or defamiliarisation is the artistic technique of forcing the audience to see common things in an unfamiliar or strange way, in order to enhance perception of the familiar. (Otto 1964; Shklovsky 1965) The literary theorist, Uri Margolin, claims it brings us to an awareness of the artistic procedures that influence the way we read and comprehend events and experiences. (Margolin 2005) It is a key concept of Russian Formalism, has been used as a basic strategy in artistic expression (Danto 1981) and is a basic satirical tactic and a central concept to both Surrealism and Dada, epitomized in the surrealist slogan of "making the ordinary extra ordinary". (Lefebvre, 1991)
Defamiliarisation is a key strategy in my own works, which I discuss below -- FaceClamps (1998), Boxes (1998), ange (2003), hipDisk, the hipdiskettes (2007-2009) and the gesture≈sound experiments (2007). These works highlight the importance and impact of gestural, mechanical and sensorial extension, in my practice, as well as demonstrating how my thinking about pairing technology with the body in relation to sound has evolved over the years. All of the interfaces discussed are body-worn except Boxes, which is a carried device that provides virtual extension of memories, reflection and breath, through sound and object. Before I turn to my own works, as case studies of practice-based research, I will first background and contextualise them -- drawing out themes from a number of related works that approach similar concerns, including gestural control of sound, interaction design, performance, wearables and physical engagement.
Performance (extension, distortion and turning)
In theatre and performance research there has long been interest in the idea of extending, as well as distorting or turning gesture in order to focus attention. According to Anne Bogart, Agnes de Mille described the use of distortion, or turning, in dance, for example, as the extension of effort, the prolongation of stress beyond the norm, saying that it can be arresting or remarkable, and it can help fix a gesture in memory. (Bogart 2001) Russian Formalist, Victor Shklovsky, wrote that everything around us is asleep and that it is the function of art to awaken what is asleep, to turn it slightly until it awakens, and that by doing so the resulting defamiliarisation prompts a reassessment of the object or situation in question. (Shklovsky 1965) Brecht in his articulation of Alienation (the A-effect, or, from the German Verfremdungs-, or V-effect), developed theories about making the strange familiar and the familiar strange stating that "What is 'natural' must have the force of what is startling. This is the only way to expose the laws of cause and effect." (Otto 1964)
These ideas suggest that a body-sound device can focus our awareness on a gesture if it renders it startling or remarkable, that the wearer or observer of the device will then be able to fix the gestures in memory, and the underlying mechanisms and motivations can thus be exposed. These ideas are integral to the work discussed here, works that often embody a heightened level of style and privilege exaggerating aspects. A powerful example of extension that exposes underlying mechanisms is Kei Kagami's Head Holder, a dress that explores dynamic structures by means of rods and strings that cause exaggerated movement of the garment by the motion of the wearer (Kagami 2006 ). My own works described below differ somewhat as they are not dynamic. They relate a little more easily to the costumes of Leigh Bowery that exaggerate his body through abstracted, fixed physical extension. (Bowery and Greer 2005).
Gestural control of sound
A comprehensive review of gesture-controlled sonic interfaces is provided by Marcelo M. Wanderley. (Wanderlay 2001) Amongst the performance practices that Wanderley describes, those of Atau Tanaka (EMG based gestural control) and Michel Waisvisz (The Hands) are particularly relevant here in that both of these artists extend fundamental functionality of their bodies through the use of sound. They magnify nuances to bring our attention, through the sound, to aspects of the body in motion (arm muscles and fingers, respectively). By doing this they "make strange" or defamiliarise, prompting us to see the body in new ways. That the performers’ movements are translated into sonic output allows us to clearly differentiate between input (the body) and output (sound) and so to experience the sound as an extension of physical expression. In contrast to Tanaka and Waiswizs, my own recent research is focused on creating performances that engage the whole body. Doing so avoids dependence on hand-based sensor input or "interface artefacts" which can problematically draw the attention of the viewer (and performer) away from the body to the artefact. In this regard my work relates perhaps more closely to Tomie Hahn and Curtis Bahn's "Pikapika" character who embodies movements from the bunraku (Japanese puppet theater). (Hahn 2002) The "Pikapika" character is highly theatrical. The body-worn devices I create share this characteristic and many of them have been created for theatrical performance.
Bangarama, a head-banging interface developed by Laszlo Bardos, Stefan Korinek, Eric Lee and Jan Borchers is also inherently theatrical (Bardos 2005). Bangarama uses the body’s physical affordances to generate sound in a surprising way--surprising because we don’t normally use our heads to interface with a computer system. Bangarama, which goes beyond digit- and limb-triggered interaction, just as many of my hip-controlled interfaces do, provides another example of how "making strange" can cause us to experience the body anew.
Unlike the plethora of interactive dance systems that Hahn and Bahn discuss, (Bahn et al. 2001) the goal of my the projects, which I discuss below, is not to compose performances for dancers, but rather to give expressive sonic capabilities to the body in motion, or through physical engagement (with the exception of Boxes, where the technology must be dealt with independently – as an object that is engaged with independent of the body). This is a crucial distinction since it frames the body as sonic rather than dancing. Nonetheless, there is strong resonance for my work with Bahn and Hahn’s discussion of the integration of the dancing body and the musical body as well as with Todd Winkler's idea of "allowing the physicality of [human] movement to impact on musical material and processes." (Winkler 1995)
Interaction Design – considering physical pleasure and emotion
William Gaver emphasises the need to consider people’s values, aspirations, fears and desires when developing interactive devices. He cautions that "we do not just engage with the world in the form of problems to be solved and tasks to be pursued. We are also playful creatures." (Gaver 2002, 2006) Besides playfulness, humour, and ambiguity are key factors in producing pleasure and emotion for the viewer/auditor. Though quite different, ambiguous situations require people to participate in making meaning because they thwart easy interpretation. The more ambiguous a work, the more it sets the scene for meaning making. The work of making an ambiguous situation comprehensible can be both inherently pleasurable and lead to a deep conceptual appropriation of artefact. (Gaver 2003) Anthony Dunne and Fiona Raby extend this to speculate about the cultural function of electronic products in interaction design, as expressed, for example, in Prescription Products: Designs for Fragile Personalities in Anxious Times, (Dunne) and Placebo, a collection of electronic objects which explore mental well-being in relation to domestic electro-magnetic fields (Dunne and Raby 2001). Their speculation is relevant here, because they point out how physical pleasure and emotion are integral to experience, no matter how idiosyncratic, and that making room for idiosyncrasy is important. Finally, Brendan Walker’s work at Aerial is focused on measuring and replicating the thrill experienced on high-velocity fairground rides. (Walker, 2005) Walker describes his practice as "a design practice specialising in the creation of tailored emotional experience."
The work of Gaver, Dunne and Raby, and Walker suggests that another kind of relationship is possible with interactive objects – other than one driven by utilitarian concerns or straight forward pleasure principles. (Snyder, 2007) Their work privileges the individual within the experience. While none of their projects result in body-worn devices unless we consider Walker’s monitoring devices to be an outcome, they all raise the importance of a consideration of the inner life of the user, and in Walker’s case focus expressly on physical immersion and visceral experience, the importance of which is discussed below.
Wearables and physical engagement
Moving the core of our body feels radically different to moving our digits or limbs. By provoking or inspiring core-body interaction, we can generate different relationships to the body in movement, as well as different feelings—both sensations and emotions. Projects that share a concern with core-body interaction include Rinotchild’s Laughing Swing, (Rinotschild 2005) Bernie Lubell’s Cheek to Cheek (Lubell 1999) and Grace Kim’s Twirl Skirt. (Kim 2005) While Laughing Swing is not wearable it is a tangible, physically engaging interface that enhances a physically immersive experience that uses sound to encourage the user to extend themselves. Laughing Swing is essentially a modified swing. It has a speaker embedded in the seat and a sensor that can measure the rate of acceleration of the user. As the user begins to swing the swing begins to laugh (laughter is heard, and felt, through the speaker in the swing’s seat). As the swing accelerates so does the laughter – building in amplitude, density, engagement and volume. This creates a feedback loop as the person swinging also laughs with the swing – louder and faster and higher as they swing faster and higher and the swing’s laugh responds. The nature of the laughter changes in direct relation to the user’s physical engagement. This aims to affect the user’s inner life by the laughter generating associated feelings. Both humour and physical engagement are key tactics here.
Cheek-to-Cheek does not work with sound, and is only partially wearable, but again it is a physically immersive experience that can intimately relate to the physical, visceral and inner life of the wearer. With Cheek-to-Cheek the user sits on a stool that is connected with pneumatic tubing to a headpiece. By moving their buttock cheeks the users can momentarily inflate cushions that are held in place against their facial cheeks, so they "dance with themselves". When one buttock cheek moves, one cushion against a facial cheek is filled momentarily with air. The relationship is very direct, creating an intimate feedback loop in the body. Grace Kim’s Twirl Skirt extends these ideas into a fully wearable space. Twirl Skirt, an augmented woman’s skirt, is an autonomous, completely wearable system. It is essentially a skirt with an accelerometer in the waistband and three electro-luminescent panels on the lower section of the front of the skirt. These panels illuminate in response to how fast the wearer is spinning. Twirl Skirt seems to inspire people to spin and twirl in ways that they usually haven’t since they were children through a direct coupling of feedback and visceral pleasure with the velocity of the wearer’s spin. The nature of immersive experience generated by these works resonates strongly with my own works, to which I now turn.
The projects that follow can be grouped into two sections, Early Work (faceClamps, boxes and ange) and Current Research: Sounding the Phenomenological Body (hipDisk, hipdiskettes and gesture≈sound experiments and developments). The diachronic journey is not necessarily direct, though current research is indelibly informed by the early projects. As mentioned above, I present these projects as practice-based research, works where I explore body≈sounds as emergent sonic practice.
faceClamps and boxes emerged from abacusParts, an umbrella project aimed at examining the introduction of digital space into a performance context. ange, was realised during an MA in Interaction Design at the Royal College of Art in London. Each of the projects reflect and embody different conceptual concerns, but are connected by the desire to extend the body sonically to bring our awareness to aspects of our embodied-ness.
We move the mouth to speak, yet how do we speak when moving the mouth? FaceClamps physically extrude the face to magnify and measure the gesticulations of our mouth along two axes, transforming these movements into rich textural soundscapes that offer an alternative to verbal communication. As the mouth moves, the sonic output starts and restarts, shifts register, falters, hesitates and starts again. Language breaks down, loops and becomes abstract textures interwoven with algorithmically controlled sounds. Seemingly unpredictable, the evolving soundscape prompts the performers to extend and exaggerate their mouth movements in an attempt to gain control. Frustration and determination are at times etched in their distorted faces, as misunderstandings seem inevitable : interpretation of the sonic output is completely open to subjective response and there seems very little basis for common ground, or control.
Figure 1: faceClamps
The crude metal and plastic constructions that extend the face effectively act as a kind of translation-pantograph, translating and exaggerating from movement to sound in real-time. They mechanically magnify the movements of the mouth, ignoring subtle permutations and expressions and increasing both visual and mechanical resolution. The change in resolution defamiliarises as outcomes are other than expected, which leads to an exaggeration of the wearers’ use of their jaws and mouths, which in turn further "makes strange" for both user and observer.
The work was created in response to my conviction that language is an inadequate tool for communication. Many people struggle with language, yet it’s common knowledge that body language is a highly expressive component of communication. Formalised understandings and interpretations of body language seem somehow stilted and sterile, lacking in emotive texture. The mapping from mouth movement to sound control is, arguably, arbitrary, yet the mouth is an intimate element of verbal communication. The rich sonic textures that result with the use of faceClamps reflect the underlying communicative struggle and are suggestive of the rich complexity of the communication process. The sonic output provides a stark contrast to traditional forms of body language. As the faceClamps "makes strange" they prompt us to be present to the body in action. They are difficult to wear, just as it is sometimes difficult to communicate succinctly. They thus embody the complexity of the experience of verbal communication.
Figure 2: boxes
Boxes shift the way of thinking about extending the body in that there is no literal, physical extension; rather the notion of extension is embodied in an independent object with which one can interact. The Boxes ‘hold’ memories that are recounted and reflected upon as "things that are commonly, even repeatedly, lost". They raise the notion that memories may somehow be held or saved for later access; yet by allowing these memories to be erased, or recorded over they nonetheless suffer the same fate as memories we hold in our hearts, brains, imaginations. When the performer records over an existing memory with their own breath, the audience are invited to make a correlation between the fragility of breath (and the autonomic processes that keep the cycle of breathing in play) and the way memories are often replaced in a seemingly involuntary, uninterruptable cycle. By equating memory with breath and the body as container of identity the boxes embody fundamental aspects of our body-selves in identical, portable, technologically augmented objects. The virtual extension provided by the boxes thus brings out attention to our bodies once again, through the magnification and defamiliarisation achieved through displacement.
"When you get sick you go where? You go, oh, oh, oh…"
"I can’t get my clothes all hung up…"
"I knew there was something wrong with me…"
"I kept losing my job and I’d go out and get another one…"
(excerpts of anonymous interviews with Alzheimer’s patients)
In the case of the performance of boxes prepared for Counting Laps each box contains a fragment of an interview with an Alzheimer’s patient, discussing what it’s like to forget what they’ve forgotten. The cycle of forgetting is thus further amplified. Fragments of identity can be seen to be embodied in language and these fragments of language are, in turn, embodied in the boxes that glow with the voices as they discuss their confusion. Each box seems to speak or breathe with a life of its own as an internal light is calibrated to the volume of output, so the modulations of the voice are amplified and also "made strange".
As each box is a stand-alone device, the performers are also able to create different spatial juxtapositions between themselves and the voices and sounds emitted by the boxes, simply by crossing the stage. By weaving in and out of each other a rich tapestry of embodied voices is created, this again focuses the attention of the audience and engages them anew with processes that they may be taking for granted.
Within the performance, the boxes take the audience from one autonomic process to another as the memories "that replace each other" by being recorded over with new recollections are themselves replaced with the performers’ breath. The performers literally record their breathing on top of, replacing, the recorded voice, and leave the disembodied breaths to flicker in the space. Thus the correlation between memory and breath is clearly signified through literal demonstration.
The boxes provide a complex, multi-layered way of examining experiential bodily processes, through extension. Breathing is used to signify parameters of the self and we are prompted to ask what is ultimately left of a person, once memory has been stripped away?
The final work of the early cycle of work I would like to discuss is Ange. Ange was inspired by Ange Anatomique (1746), an 18th Century medical engraving prepared by Jacques-Fabien Gautier d’Agoty for the French École de Médecine. Ange Anatomique both attracts and repels – the sensual pull of the flesh is conflicted by the knowledge that the work is an engraving of a cadaver. Philippe Comar, in Les Images du Corps, cites Gautier d’Agoty as saying:
we used the cadaver of a woman as the muscles are more delicate and take up less space. This allowed us to give greater expanse to the form. We left the head on for pleasure. (Gautier d’Agoty in Comar 1993, translation, D. Wilde)
Figure 3: Ange Anatomique
In the engraving the buttock skin is peeled back like delicate lingerie, the back muscles opened up like wings, the neck exposed like an Edwardian collar, the spine, the skeleton, seems distended. The result is both sensual and troubling. The cadaver also wears a soft blue hairband and her cheeks are ruddy as if freshly made up. "We left the head on for pleasure." The image is beautiful, captivating, troubling.
Figure 4: Ange (bodice)
Ange echoes the attraction and repulsion of the original by turning the inside outside and exposing the flesh of the wearer through a solid, transparent, rose-tinted carapace. The insides of the wearer’s body is thus extended for consideration, and the body defamiliarised. The "rib-cage" of the interface consists of 24 curved metal prongs distorted to echo the skeletal structure of the original engraving, further challenging assumptions. The transparent bodice brings the body of the wearer to our attention through this extended anatomy. The entire ensemble effectively "makes strange". The "ribs" provide the opportunity to control 24 different sound samples. This allows wearer and/or interactor to examine the musicality and rhythm of touch in an extended way. Physical and emotional responses of the wearer are suggestively embodied in the resulting musical/sonic extension. The overall aesthetic is influenced by the Japanese ideals of iki, or hidden beauty, (Pincus 1996) though everything is paradoxically exposed.
Figure 5: Ange (full costume, rear)
Ange allows us to explore through sound the idea that when we touch each other in an intimate context, when we let ourselves go, we initiate and play a series of vibrations in, through and with the body. The physical and sonic extension draws our attention to this notion as touch triggers sound, and the nature of touch determines the nature of the sonic output. The player can compose rhythmically and sonically, touching and "playing with" where the wearer’s body might end or begin. The wearer’s body can thus be played, just as a body can be played sensually.
Amongst the samples accessible through the interface are sounds relating to breath and voice, including a poem about the body by E. E. Cummings; (Cummings 1904-1962) laughter, breathing and barely-there whispers; abstract sounds such as water and scratching and other textural noises; as well as sounds that we would traditionally think of as "musical" samples – a drum loop, a gong loop, a number of oboe mixes and other traditional musical sounds. With this collection of samples a wide range of sonic output can be generated by the player, reflecting the body’s capability to generate infinite rhythms and textures that we can experience akin to sound.
Figure 6: Ange (video still)
Ange is presented in both performance and exhibition, as a body-worn device – literally worn, or as a playable interface on a stand. When the transparent bust sits on a stand, people are free to come and caress and press each of the ribs and to create music of their own making. In performance the audience is invited to play, but they rarely dare to. Rather the highly stylized performer, reminiscent of the almost untouchable beauty of a geisha keeps the audience at bay. And like a lonely doll, she is left to play (by) herself. Attracted and repelled, the audience clamber to listen and view, hold themselves back as if repulsed when the performer draws near, then rush to the interface to play once it’s again on the stand. The extension embodied in the interface, and the resulting attention that it draws to the body and the repercussions of touch, seem compelling. Nonetheless, Ange is essentially a failed attempt to get people to explore intimate touch in public through the body and sound even though it clearly places the viewer in the presence of their feelings about doing so.
Ange is a dangerously serious and highly stylised work – it is far too confronting for a casual audience at a gallery to engage with when worn. Pressing some of the ribs brings the fingers close to the wearer’s nipples. Despite the 6mm rose-tinted plastic barrier the presence of the body cannot be ignored. Added to this, the interface is like a carapace, inviting but preventing any kind of touch at all – contact with the skin underneath is impossible, and the wearer cannot move or articulate their body in natural, responsive ways, and can only physically respond within very constrained parameters.
Ange is confronting and in a way ambiguous. It totally ignores, hampers even, the body’s capacity for movement. It exposes touch yet prevents it from truly taking place, thus echoing the attract and repel dichotomy of the original 18th Century engraving.
Figure 7: Ange
SUMMARY: EARLY WORK
The three projects discussed above, faceClamps, boxes and ange, extend the body in different ways. FaceClamps, through physical and sonic extension, provokes an extension or exaggeration of the wearer’s facial expressions. It "makes strange" the actual movement that controls the sounds and prompts further exaggeration. A feedback loop of defamiarisation draws our attention to the wearer’s attempts at communication in ways that are highly compelling. Boxes embodies inner, reflective processes through virtual extension, "making strange" and drawing our attention to the fragility of memory by equating it with breath. Ange extends the body mechanically and sonically, as well as physically – literally and metaphorically bringing the inside to the outside. The resulting defamiliarisation invites us to examine and reflect upon the repercussions of touch.
The sonic output in each of the projects also differs. In FaceClamps the interface allows algorithmic control of soundsamples; Boxes allows record, playback and erasure of looped samples; Ange allows individual triggering and limited control of a selection of 24 individual sound samples, each mapped to an independent controller. The nature of the physical engagement with the different interfaces is thus different. In the following section, I discuss current research, where I give greater focus to the impacts and opportunities of physical engagement.
CURRENT RESEARCH : SOUNDING THE PHENOMENOLOGICAL BODY
The projects discussed in this section are part of the Swing That Thing: moving to move. (Wilde) Swing That Thing… is a practice-based investigation that arose out of a desire to further examine the body’s capacity for movement as actuator of digitally controlled events. (Wilde) The research is grounded in embodiment theory, (Dourish 2001) and places the phenomenological body central to a theoretical and practical investigation of the role of body dynamic and extension in experience, examined through interactive concept garments and other wearable technologies. These projects make particular use of sound to extend and emphasise the body in motion.
Figure 8: hipDisk
hipDisk exploits changing relationships between hip and torso to actuate simple tones. By physically and visually extending the body, the interface enables a sonic extension and demarcation of gesture. In doing so it demands, provokes, requires and seems to inspire gestural extension, (Wilde 2008a) just as it shifts the way the body is experienced and viewed. The hipDisk was born out of a desire to encourage people to explore and extend the range of movement in their hips. The intention was not, from the outset, to create a musical instrument, yet once the interface was conceptualised it seemed logical that a hipDisk-ed body generate sound. Sound accents the highly visible, movement-based input, supporting rather than diluting the visual clarity of the extending disks. The result is a self-contained wearable sonic output system for performance and play.
The sonic output of hipDisk is unrefined, electronically primitive, harsh, reedy and simplistic. Surprisingly, from a musical perspective, in the overall experience of the work this is perceived as a positive attribute. The hipDisk demands an inordinate amount of effort to play, resulting in comparatively feeble, almost ridiculous sound output. The inherent contradiction seems to be both humorous and engaging, and is further augmented by the lack of restraint shown by the wearer as they, seemingly willingly, put their body into bizarre positions in order to hit the notes, as well as by the costume. All of this combines to make even stranger that which each element individually already defamiliarises (see (Wilde 2008a) for more).
Despite the lack of sonic refinement, the ability to make sound with hipDisk is essential – both for wearer and audience, as the sound provides a demarcation of extension, as well as a goal worthy of pursuit. A hipDisk-ed performer is free to improvise and explore – both physically and sonically, just as they are able to intentionally generate recognisable tunes. It is perhaps important to note though that the generation of recognisable tunes has proven to be quite difficult, requiring a level of mastery not yet successfully achieved. The hipdiskettes, below, examine this further.
hipDisk, like the other body-worn interfaces discussed in this article, is tightly coupled to the body.Such interfaces cannot be put down, moved through, or otherwise decoupled. They provide a kind of body-artefact environment that must be navigated by the wearer through their physicality, especially when physical engagement is key. This places the body at the epicenter of experience, and is in accord with Aaron Levisohn’s argument that the body should be an integral element of interactive systems. (Levisohn 2007) It conflates the defamiliarisation afforded by the interface with our perception of the body and, experientially, affords a direct examination of the phenomenological notion that the body is at the foundation of our existence in the world. (Heidegger 1962; Merleau-Ponty 1962)
Heidegger, in his seminal work Being and Time, posits the notion of objects being present-at-hand or ready-to-hand, depending on whether our attention is on the object itself or the task we are undertaking with it, in which case the object effectively disappears as we incorporate it momentarily into our body’s functional boundaries. (Heidegger 1962) In the case of physically engaging body-worn devices such as hipDisk, interaction is initiated, and engagement realised through the body. Even when a wearable is used to trigger distal phenomena the coupled nature of the interface to our body renders the experience immediate, proximal. (Polanyi 1967) In Heideggerian terms the body itself thus is present at hand or ready-to-hand as the wearer focuses on their actions and the results of their actions, rather than on (or through) an object or interface. (Wilde 2008a) As a result, as the wearer’s focus shifts, observer and enactor are both prompted to reposition themselves in relation to the body and assumptions are repeatedly put into question.
hipDisk requires intense physical engagement, and results in total immersion in the act of playing. Performers don’t censure their physical expression as they are necessarily focused on hitting the notes. hipDisk thus brings us, experientially, through sound, into the presence of our bodies.
Figure 9: the hipdiskettes
The hipdiskettes iterate the hipDisk through time and space, and foreground the interconnection between choreography and composition through a more complex sound than a single hipDisk-ed performer can achieve alone. A group of hipDisk-ed performers can collectively play chord structures, harmonize, provide counterpoint, and play a range of notes in rapid succession, and thus create rhythmically complex works. They can also create visually complex works as they shift their spatial relationships and extend their bodies to create sound.
The hipdiskette performers have a range of musical and movement skills. Their goal is to develop and perform my special arrangement of "The Girl From Ipanema" a classic Bossa Nova tune that crosses cultural boundaries and resonates with the swimming costumes that hipDisk-ed performers wear. Arriving at a performable rendition has proven to be incredibly challenging; however, surprisingly, the process has provided a platform for understanding how people learn through, and acquire knowledge about, their bodies. (Wilde 2008b) The actual struggle of trying to play the piece seems to enhance the strangeness of the entire experience and render the work even more compelling as we must engage with the body’s and the performer’s struggle in each moment.
Figure 10: Speed Harmonics and Head Scrape
GESTURE≈SOUND EXPERIMENTS & DEVELOPMENTS
The gesture≈sound experiments explore how one might successfully mesh gestural/physical and sonic composition by extending the moving body through sound. The aim was to develop systems that support kinaesthetic-auditory synchresis, where human body motion is mapped into sound in such a way that sound production becomes an inherent and unavoidable consequence of moving the body. The intention was to engage both performer and audience in a fluid experience of the relation between performed sound and gesture.
The approach taken for the initial experiments, undertaken at STEIM, the Studio for Electro-Instrumental Music, in Amsterdam (STEIM) in July 2007,was multifaceted, reflecting the diverse interests of the collaborators. Considerations included: physicality in space, sonic and compositional form, structure and aesthetics, conceptual semantics, sensor technologies and applications. We used Nintendo Wii Remotes (Nintendo 2006) as a plug-and-play wireless sensing device, combining it with Max (Cycling74) and Audiomulch (Bencina) software to facilitate real-time manipulation of sound. A range of patches were prepared, that responded differently to different parts of the body in motion. Exercises were developed that began with: the voice, naturally occurring sounds, recorded sounds, sounds collected from the Freesound database, (Freesound) and sounds from the body in motion. We aimed to challenge our assumptions about the relations of gestures to sound and to shift habitual practice to arrive at unexpected outcomes. (Bencina, 2008)
Figure 11: Leg Ratchetts
Extending the body sonically with an interface that sits close to the body, or is embedded in a garment, affords a different kind of examination than if the interface is visibly and mechanically extended. The body is prompted to move in response to the sonic output, but otherwise not constrained in any way. This kind of extension affords a different kind of "making strange" as the body appears normal to our gaze, yet the behaviour of the performer becomes exaggerated and affected, and the sonic extension underlines or strengthens their movement choices by adding coherency. Thus we have a feedback loop between body-movement and sound that compounds, that prompts us to see the body anew.
SUMMARY: CURRENT RESEARCH
hipDisk and the hipdiskettes use sonic extension to emphasise and draw attention to physical and gestural extension of the performer, as well as to their use of the mechanical extension of the interface. The interface constrains movement and intensifies the strangeness of the entire experience thus compounding the defamiliarisation process to bring our attention to the body in a highly directed way. In contrast, gesture≈sound has a hidden interface and the performer’s movements are not constrained by it in any way; rather their movement is free to respond to the sonic output and to explore the relationships between movement and sound. These projects embody different ways of "making strange" through extension, and offer many avenues for future research in relation to extension and defamiliarisation through sound.
Each of the projects that I have discussed approaches the question of extending our understanding of the body through sound from a different perspective. Integral to all is the notion of placing the body central to a process, where technology facilitates conceptually driven body-centric exploration. Using sound to reflect, respond to and reiterate inherent features of our bodies, and our embodied state affords shifts in perception. It allows for complex suggestion and juxtaposition without having to tie down the meaning of a work and, further, can be used to emphasise or compound the propensity of an interface to defamiliarise and prompt re-engagement. Body≈sound works are thus an ideal medium through which to challenge our assumptions and bring us into the presence of the body.
ACKNOWLEDGEMENTS & CREDITS
Many people have been involved in the development of the different projects discussed in this article. Full credits are available at http://www.daniellewilde.com or by contacting the author.
All photographs © the author except:
Figure 1 © Romy Achituv, Danielle Wilde and Alison Bradley. Photographed by Alison Bradley.
Figure 3 Ange Anatomique by Jacques Fabien Gautier-d’Agoty (1746), in ARTstor [database online]. [cited 3 September 2009]. Available from ARTstor, Inc., New York, New York.
Bahn, C., Hahn, T. and Trueman, D. (2001) "Physicality and Feedback: A Focus on the Body in the Performance of Electronic Music." in Proceedings of International Computer Music Conference (ICMC) Havana, Cuba: ICMA
Bardos, L., Korinek, S., Lee, E., and Borchers, J. . "Bangarama: Creating Music With Headbanging. " in Proceedings of the 2005 International Conference on New Interfaces for Musical Expression (NIME05) Vancouver, BC, Canada: ACM
Bencina, R. [online] http://www.audiomulch.com Accessed 3 September 2009
Bencina, R., Wilde, D. and Langley, S. "Gesture ≈ Sound Experiments: Process and Mappings" in Proceedings of The 8th International Conference on New Interfaces for Musical Expression (NIME 08) Genova, Italy (2008) 197 – 202: ACM
Bogart, Anne. (2001). A Director Prepares, Seven Essays on Art and Theatre. London: Routledge.
Bowery, Leigh and Greer, F. (2005). Leigh Bowery Looks. London: Violette Editions.
Comar, Philippe. (1993). Les Images du Corps. Paris, France: Découvertes Gallimard Sciences.
Cummings, E. E. (1904-1962). Complete poems: 1904–1962: Liveright Publishing Corporation, USA: 1923–1991, the Trustees for the E. E. Cummings Trust: 1976–1979, George J Firmage: 1997–2003, The Academy of American Poets.
Cycling74 [online] http://www.cycling74.com Accessed 3 September 2009
Danto, Arthur C. (1981). The transfiguration of the commonplace : a philosophy of art. Cambridge, Massachusetts: Harvard University Press.
Dourish, Paul. (2001). Where the action is: the foundations of embodied interaction. Cambridge, Massachusetts: MIT Press.
Dunne, A., Raby F., Anastassiades, M. [online] Prescription Products: Designs for Fragile Personalities in Anxious Times http://www.dunneandraby.co.uk/content/projects/71/0 Accessed 3 September 2009
Dunne, Anthony and Raby, Fiona. (2001). Design Noir: The Secret Life of Electronic Objects. London: Birkhauser.
Freesound [online] http://freesound.org Accessed 3 September 2009
Gaver, William. (2002). "Designing for Homo Ludens " in 13 Magazine 12.
Gaver, William. (2006). "Curious things for curious people." In Press.
Gaver, William, Jake Beaver, Steve Benford. "Ambiguity as a resource for design" in Proceedings of CHI’03 Conference on Human Factors in Computing Systems Fort Lauderdale, FL, USA: ACM
Hahn, T., Bahn, C. . (2002). "Pikapika - The collaborative composition of an interactive sonic character" Organised Sound, Cambridge: Cambridge University Press 7(3): 229-238.
Heidegger, Martin. (1962). Being and Time. New York: Harper & Row.
Kagami, Kei. (2006 ). "Head Holder." In The Fashion of Architecture: CONSTRUCTING the Architecture of Fashion, ed. Bradley Quinn. New York: AIA Center for Architecture.
Kim, Grace [online] http://www.iamgracie.com/twirl/ Accessed 3 September 2009
Lefebvre, H. (1991) Towards a leftist cultural politics: remarks occasioned by the centenary of Marx’s death. 1988. In NELSON, C., Grossberg, L. (eds.) "Marxism and the Interpretation of Culture". Urbana and Chicago: University of Chicago Press, p. 75-88.
Levisohn, A. "The Body as a Medium: Reassessing the Role of Kinesthetic Awareness in Interactive Applications" in Proceedings of ACM Multi Media (ACM MM’07) Augsburg, Bavaria, Germany: ACM
Lubell, B. [online] http://blubell.home.att.net/03cheekinstall.htm Accessed 3 September 2009
Margolin, Uri. (2005). "Russian Formalism." In The Johns Hopkins Guide to Literary Theory and Criticism. (Second Edition), ed. Martin Kreiswirth Michael Groden, and Imre Szeman. Baltimore, Maryland: The Johns Hopkins University Press.
Merleau-Ponty, Maurice. (1962). Phenomenology of perception. London: Routledge & Kegan Paul.
Nintendo. (2006). Wii™ hardware and computer program. Redmond, USA.
Otto, L. (1964). "Interview with Luth Otto " In Brecht on Theatre: the Development of an Aesthetic, ed. J Willett. New York: Hill and Wang.
Pincus, Leslie. (1996). Authenticating Culture in Imperial Japan: Kuki Shûzô and the Rise of National Aesthetics. Berkeley: Unversity of California Press.
Polanyi, Michael. (1967). The Tacit dimension. New York: Doubleday.
Rinotschild: Rinott, M., Rothschild, M., Weinstein, L. [online] http://www.michalri.net/laughingswing/ Accessed 3 September 2009
Shklovsky, Victor. (1965). Art as Technique. In Russian Formalist Criticism: Four Essays. Lincoln: University of Nebraska Press.
Snyder, C. R., Lopez, Shane J. (2007). Positive Psychology: The Scientific and Practical Explorations of Human Strengths. Thousand Oaks, CA, USA: Sage Publications, Inc.
STEIM [online] http://steim.org Accessed 3 September 2009
Walker, Brendan [online] http://www.aerial.fm Accessed 3 September 2009
Walker, Brendan (2005). The Taxonomy of Thrill and Thrilling Designs: Chromo11 Volumes One and Two. . London: Aerial Publishing.
Wanderlay, Marcelo M. (2001). "Gestural Control of Music." in International Workshop Human Supervision and Control in Engineering and Music. Kassel, Germany: IEEE and the Journal of New Music Research
Wilde, Danielle [online] http://www.tinyurl.com/SwingThatThing Accessed 3 September 2009
Wilde, Danielle. (2008a). "hipDisk: using sound to encourage physical extension, exploring humour in interface design." in the International Journal of Performing Arts and Digital Media (IJPADM) 4(1):7-26. Bristol, UK: Intellect
Wilde, Danielle. "The hipdiskettes : Learning (through) Wearables " in Proceedings of The Australasian Computer Human Interaction Conference (OZCHI 08) Cairns, Australia: ACM
Winkler, Todd "Making Motion Musical: Gesture Mapping Strategies for Interactive Computer Music. " in Proceedings of The 1995 International Computer Music Conference. San Francisco, CA, USA: ICMA