1. Introduction
Understanding the role of sound qualities in inter-cultural improvisatory dialogue is crucial for deciphering how sound artists and musicians communicate in an opaque networked environment. Approaches to negotiating culturally unfamiliar sound worlds and musical progressions (e.g. timbres, tonalities, modes and rhythms), suggest that nuances of sound become ‘signs’ (Lakoff and Johnson Reference Lakoff and Johnson1980) for the dissemination of structure, form and mood in telematic improvisation. The studies suggest that ‘images of embodiment’ (Cumming Reference Cumming2000: 16) are key to a musician's comprehension of musical signs, both in terms of representation and their intentions and responses to them. Iyer (Reference Iyer2008: 273) contributes further to this notion, suggesting that ‘music perception and cognition are embodied activities … and actively constructed by the listener’. Likewise, fluidity in networked musical dialogue requires extended listening practices, replacing visual referents in facial expression, body language and the physical marking of time. Information carried in texture, intonation and note or phrase duration indicates as much as harmonic cadences or coda, when navigating improvised soundscapes. Viewed through a framework of Distributed Cognition, it is the ‘functional relationships of elements’ (Hollan, Hutchins and Kirsh Reference Hollan, Hutchins and Kirsh2000: 175) such as technology, environment and culture that shape our cognition and response to musical signs. The authors challenge previously assumed boundaries between semiotics and cognition, arguing as De Haan that ‘the semiotic and the natural are both part of the same “system”, as is the case in the Distributed Cognition thesis, versus opting for a Cartesian separation of both’ (De Haan Reference De Haan2002).
2. Ethernet Orchestra: Live Networked Improvisation
Ethernet Orchestra is an international inter-cultural ensemble of skilled musicians and sound artists who improvise through network interfaces in different combinations of personnel and instruments, including voice.Footnote 1 The ensemble's primary concern is with developing new practices of improvisation through networking musicians from diverse improvisatory traditions. Although each musician brings his or her own improvisatory aesthetic to the collaboration, no one form or style of playing ever dominates the interaction. This could be described as ‘free improvisation’, though without the constraints or connotations that this may suggest regarding expectations of musical style. While Ethernet Orchestra does not adhere to any one style or genre of improvisation, whether it is, as Bailey describes, ‘non-idiomatic’ (Bailey Reference Bailey1993: xi) remains up to the listener to decide. What it does have in common with ‘free improvisation’ is a sense of experiential interaction, in what Eddie Prévost describes as ‘dialogical heurism’ (Prévost Reference Prévost1995: 3). Indeed, it is identification of musical signs and relational transferences in the ensemble's improvisations that inspire this research. Ethernet Orchestra collaborates remotely and occasionally performs live to audiences through the Internet, terrestrial radio and gallery events. The two improvisations evaluated in this paper occurred in distinct scenarios with different combinations of performers, in which the one of the authors is a participant in both. Notably, the employed network platform eJamming is a dedicated audio-only interface that achieves very low latency through compression algorithms that shrink the file size for high packet flow.Footnote 2 Its peer-to-peer configuration also allows it to route the signal directly to players rather than via external servers, and collaborators communicate via microphone or typed text. While the visual streaming of collaborators can be achieved by incorporating tele-conferencing platforms, there are inherent latency problems with synching sound to picture as well as a drain on the available bandwidth for maintaining fast high-fidelity audio transmission. This results in the absence of traditional visual cues that will be discussed further.
3. Musical Prosody and Nuanced Sound
A useful starting point for thinking about how networked musicians communicate through sound is to examine analogies in the prosodic features of instrumental signalling of tone and drum languages. The manguaré drum language of the north-west Amazonian Bora people is relevant because it is not only ‘an acoustic substitutive system for the spoken form of the Language’ but also one that is used to communicate over distance, based on the ‘different phonological and prosodic features of the corresponding spoken language’ (Seifart and Meyer Reference Seifart and Meyer2010: 1). Nettl also describes forms of distance communication in Melanesia, and Central and West Africa, as ‘instrumental signaling’, or ‘tone language’, in which syllabic pitch differences intone differentiation in the lexicon (Nettl Reference Nettl1956: 9–10). Many of the same principals apply to networked improvisation. Remote collaborators (connected across distances of thousands of miles) cannot see each other, so the embodied, visual referents of gesture, and their relationship to sound or ‘mapping’ (Johnston Reference Johnston2009: 21), are not available to mediate the experience. Bora drum messages contain ‘special syntactical and lexical features’, which contribute to their ‘intelligibility’ (Seifart and Meyer Reference Seifart and Meyer2010: 6). Applied to networked improvisation, these could be designated as facets such as timbre, pitch, amplitude and metre, which require the improviser to listen, interpret and react to these signifiers as they arise sonically. This does, however, require a degree of training, not of the ear, but of the ‘brain body’ (Oliveros Reference Oliveros2010), echoing Barthes’ notion that ‘hearing is a physiological phenomenon; listening is a psychological act’ (Barthes Reference Barthes1985: 246). Acclaimed musician and telematic improviser Pauline Oliveros describes her own theory of listening, as ‘inclusive listening’, where ‘many places at once are treated as one rather than many, … assembling a narrative out of disparate elements, lending significance to the relational and associative connections found between the many’ (LaBelle Reference LaBelle2006: 158–9).
Concrete examples of ‘associative connections’ in networked musical interaction can be found in the ‘psychophysical dimensions of music’ that Balkwill and Thompson define as ‘any property of sound that can be perceived independent of musical experience, knowledge or enculturation’ (Balkwill and Thompson Reference Balkwill and Thompson1999: 44). Timbre, amplitude or tempo become signifiers of mood, dynamics or intention. These are also present in collocated improvisation, however they are supported by gesture, which contributes significantly to their ‘semiotic value’ (Van Leeuwen Reference Van Leeuwen1999: 6). George E. Lewis (Reference Lewis1993) corroborates this notion of semiotics and learning in improvisation in the liner notes of his ‘Voyager’ CD:
Improvisation is about … interaction and behavior as carriers for meaning. On this view, notes, timbres, melodies, durations, and the like are not ends in themselves. Embedded in them is a more complex, indirect, powerful signal that we must train ourselves to detect.
Lewis is referring to collocated improvisation, but it is suggested that this is further heightened in the networked scenario, in which musicians develop extended listening practices through interacting in a spatially and temporally dislocated networked environment. Chafe (Reference Chafe2009) makes an analogy between this experience and ‘under water acoustics’, entering into ‘different sound worlds with our ears, the properties of water or Internet media give them a sonic imprint all of their own’ (Chafe Reference Chafe2009: 415). As qualitative and circumstantial – that is, situated or contextual data – these ‘properties of Internet media’ are integral to technological user evaluation and for design purposes. Underlying this, and perhaps of greater interest to the networked improviser, are the ethnomusicological implications arising from what these ‘different sound worlds’ mean to different cultural worlds.
4. Timbral Semantics and Signifiers
Timbre deserves particular attention as a signifier and semantic carrier of meaning in the context of inter-cultural networked improvisation. Whilst timbre inherently includes syntactical and lexical features specific to its cultural context, the training of the performer and the idiom of the instrument performed, as well as particular to musical training and derivative from a body of traditional repertoire, it is also possible to use timbre as a signifier and semantic dimension of musical communication that is not dependent on, or bounded by, understanding of these culturally specific syntactical characteristics. According to Moisala (referring to Shepherd Reference Shepherd1992): ‘Music evokes … both cognitive and corporeal meanings. Meanings of music evolve primarily through the process and texture (including quality and timbre) of music instead of the syntactical elements of music’ (Moisala Reference Moisala1995: 12). In the case studies, the musicians interviewed referred to timbral mimesis and timbral listening when describing their approach. They also alluded to the figurative and narrative semantic potential of timbre, independent of pitch modality and time, making timbre a useful musical dimension in inter-cultural communication. The musical analysis indicated timbral transference and imitation that transcended discrete instruments and cultures, through blending, recall and mimetic tone. Within the context of collaborating diverse cultures and heterogeneous instrumental ensembles (e.g. computer, guitar, percussion, trumpet, bowed strings), and without visual cues, timbre functioned as a blending element, a method of creating homogeneity without syntactical constraint, both for the musicians and for listeners: ‘something to hold on to’ (Landy Reference Landy1994: 51) in the dense layers of new sounds.
In many non-Western cultures, especially in Asia and some Eastern European musical traditions, timbre is more foregrounded than pitch, harmony and rhythm, and it is significant in the carrying of meaning and expression through non-pitched noise effects, as well as transformations of tone colour (e.g in Japanese biwa, shamisen and shakuhachi, Chinese pipa, Mongolian throat singing, Bulgarian harmonic singing, Tibetan chant). Ethnomusicologists have long been aware of the paucity of Western methods of transcription and analysis in capturing timbral detail and importance, especially as Western notation gives precedence to pitch and rhythm. ‘Timbre is a slippery concept and a slippery percept, perceptually malleable and difficult to define in precisely arranged units’ (Sawer Reference Sawer2003: n.p.). ‘The relative lack of perceptual studies in ethnomusicology prevents an informed cross-cultural cognitive theory of music from being developed’ (Serafini Reference Serafini1995: 137 (quoting Tolbert Reference Tolbert1992)).
Contemporary composers and improvisers, especially since the 1950s have been increasingly using the scope of timbral transformation through extended instrumental techniques to broaden their creative palette (e.g. Xenakis, Sciarrino, Scelsi, Truax, Derek Bailey and Evan Parker). In electronic music – through electroacoustic music, musique concrète, synthesis, sample processing and filtering techniques and so on – timbre is integral to the creation, and arguably the ‘medium is the message’ (McLuhan Reference McLuhan1964) in which timbral effect both forms the primary material and embeds itself symbiotically in the meaning, influencing its perception. In the current era of ubiquitous microprocessor synthesis and computer technologies for creating and manipulating sound, an expansive horizon of music is now conceived from the micro-material level up: organising ingredients of sound (the subatomic anatomy of tones) using timbre and experimental sonorities to create music (Roads Reference Roads2001; Manning Reference Manning2004).
In auditory analysis, definitions of timbre include: ‘an attribute of auditory sensation in terms of which a listener can judge two sounds similarly presented and having the same loudness and pitch [to be] dissimilar’ (ANSI 1994: n.p.) and an attribute whereby ‘a listener can judge that two sounds are dissimilar using any criterion other than pitch, loudness and duration’ (Pratt and Doak Reference Pratt and Doak1976: 317). Timbre is characterised by a group of static and dynamic sound properties (Loureiro, Paula and Yehia Reference Loureiro, Paula and Yehia2004). In music and speech, this provides the means for distinguishing between different types of voices (Houtsma Reference Houtsma1984; Monahan, Kendall and Carterette Reference Monahan, Kendall and Carterette1987; Beerends and Houtsma Reference Beerends and Houtsma1989; Kendall and Carterette Reference Kendall and Carterette1993; Houtsma Reference Houtsma1997; Hajda, Kendall, Carterette and Harshberger Reference Hajda, Kendall, Carterette and Harshberger1997; Kendall, Carterette and Hajda Reference Kendall, Carterette and Hajda1999); in auditory display it is a representational dimension (Bregman Reference Bregman1990) that allows categorisation, distinguishing and grouping of auditory streams. To some degree, timbre can be described by temporal and spectral characteristics, through the use of short-duration fast Fourier transform (FFT) sequences (Song Reference Song2010). Sensations of ‘brightness’ or ‘sharpness’ can be readily distinguished in sustained or consistent tones (Von Bismarck Reference Von Bismarck1974: 215; Zwicker and Fastl Reference Zwicker and Fastl1999). Beerends and Houtsma (Reference Beerends and Houtsma1989) correlate timbre with four main variables: spectral power distribution, temporal envelope, rate and depth of amplitude and frequency modulation, and the degree of inharmonicity. Timbre is also affected by perceptual and (dynamic) acoustical features (Hajda et al. Reference Hajda, Kendall, Carterette and Harshberger1997; Hajda Reference Hajda2007).
From a musical standpoint, composers and improvisers use timbre as ‘part of the musical expression’ and as ‘a main element in musical aesthetics’ (Hourdin, Charbonneau and Moussa Reference Hourdin, Charbonneau and Moussa1997). As composer Toru Takemitsu points out, attentiveness to timbre is far more accentuated and significant in music of his native Japan than the importance usually given to timbre in Western training. Timbral listening is an alternative to the Western way of listening to and interpreting music, in which the ear focuses on texture and colour of the music, instead of on pitch and harmony. Landy's analysis of the koto composition ‘Tayutai’ by Shinohara states that ‘noise effects’ are ‘seen by the composer to be a “natural extension” of pitched, highly timbral material in Japanese music in general’ (Landy Reference Landy1987: 76). Timbre is critical to understanding certain kinds of non-Western music on its own terms and, in this context of inter-cultural improvisation, timbre functions as a facilitator, signifier and semantic carrier.
Thus an integrated triangulated methodology is required to capture a useful understanding of semantic and physical attributes. Drawing on other research by the authors (Figure 1), compilation of a lexicon of qualitative timbral terms used by participants in interviews is being developed in the continuing series of networked improvisation case studies, in order to cross-examine the signifier and semantic value ascribed to timbre alongside musical and spectral characteristics. Corroborating the need for verbal descriptors, Fales writes: ‘As scholars, indeed as listeners, we have a difficult time describing timbre. Though we can talk about it in large generalities, as though it were a conceptual abstraction … it is only by deliberate effort that we conceptualise it as a distinctly on-going, dynamic feature of music with the same clarity as pitch or meter … We have a peculiar amnesia in regard to timbre, but we're not deaf to timbre: we hear it, we use it – no one has much trouble telling instruments apart – but we have no language to describe it’ (Fales Reference Fales2002: 57).
Framework for timbral analysis integrating qualitative and quantitative methods, including the compilation of a verbal lexicon of (situated, contextual) timbral descriptors that is applied here to networked improvisation.
Existing research developing descriptors to present sensory evaluation of sound stimuli falls broadly into categories of specialist lexicons for particular instruments, lexicons developed in response to everyday (not musical) sounds, and ranking timbral samples relative to pairs of Ballas’ descriptors (Ballas Reference Ballas1993; Kendall and Carterette Reference Kendall and Carterette1993; Bonebright Reference Bonebright2001; Rioux Reference Rioux2001; Štěpánek and Moravec Reference Štěpánek and Moravec2005; Štěpánek Reference Štěpánek2006). These previously developed systems have not been tested in the context of music, especially in improvisation. Hence, the researchers’ methodology develops a lexical system to assess the carriage of meaning in relation to inter-cultural communication through timbral signifiers in a non-visual networked environment.
5. Signifiers of Expression in Sonic and Musical Contexts
Since Charles Pierce (the nineteenth-century logician and philosopher) first wrote about the sound of ‘the scarlet trumpet’ (Smith Reference Smith1972: 28), literature pertaining to signification in musical expression has broadly related to qualities of sound, gesture (Lidov Reference Lidov1987) and metaphor (Cumming Reference Cumming2000) as indicators of emotion and musical progression. In Piercian terms, the way in which a musician plays a melodic phrase (e.g. timbre, pitch or volume) is a ‘representamen’ of its ‘object’ (creative expression). For the networked improviser, ‘representamens’ may also be indicators of changes in tempo, dynamics or tension and resolution by fellow collaborators. However, are these musical signs universally understood across cultures?
Cumming suggests that just as sound is capable of carrying ‘connotations … of human subjective qualities’, these start as ‘terms used by musicians and those in the musical community, to describe music as signifying, and what is “heard in” the sounds, …, has been heard according to a learned code of recognition’ (Cumming Reference Cumming2000: 16–17). Picking up on Norman Bryson's terminology, ‘the “code of recognition”, situates the musical sound within a tradition of musical experience, distinguishing it from an un-interpreted aural phenomenon’ (Cumming Reference Cumming2000: 18).
Reflecting on qualities of sounds in her own violin practice, Cumming argues: ‘It does not make sense, [then] to say that interpretations of tonal “warmth” or “coolness” are “merely subjective,” as if they have been derived from psychological states of the listener, because these terms have been socially negotiated, through time, as having a designated range of meanings’ (Cumming Reference Cumming2000: 18).
Lakoff and Johnson (Reference Lakoff and Johnson1980) describe notions of warmth or coolness or high or low as ‘ontological’ and ‘orientational’ metaphors that traverse all cultures; however, ‘which concepts are orientated which way and which orientations are most important, vary from culture to culture’ (Lakoff and Johnson Reference Lakoff and Johnson1980: 24). Advocating that communication of emotion in music is embodied, and closely related to that of vocal expression, Juslin and Laukka argue that ‘emotions influence physiological processes’ (Juslin and Laukka Reference Juslin and Laukka2003: 770). To emphasise this view, they draw on what nineteenth-century philosopher Herbert Spencer said was ‘the general law that feeling is a stimulus to muscular action’ (Juslin and Laukka Reference Juslin and Laukka2003: 770). Somewhat critical of the cognitive psychological conclusions drawn by the dualists, Cumming substantiates this point, arguing that the ‘musical mind’ is not separate from the body, and, bringing it back to the ontological metaphor, claims that ‘Images of embodiment appear as common modes of musical understanding’ (Cumming Reference Cumming2000: 16). The implications for networked improvisation is that expression of emotion or intention in music emerge from shared inter-cultural metaphors as acknowledged parameters of embodiment, providing a valuable starting point for musicians collaborating for the first time.
6. A Distributed Framework
The practice of networked improvisation is distributed and multi-faceted, covering aspects of technology, ethnomusicology and cognition, and therefore requires a synthesis of these distinct strands of investigation. The research situates itself in a framework known as distributed cognition, a theoretical hybrid (Rogers Reference Rogers1997) conceived by Edwin Hutchins and colleagues at the University of California, San Diego in the mid-1980s (Hollan et al. Reference Hollan, Hutchins and Kirsh2000). As a theoretical perspective, it has been instrumental in human computer interaction (HCI) design research in collaborative work place systems and practices (Rogers 1997). Echoing Cumming's ‘images of embodiment’, Hollan et al. (Reference Hollan, Hutchins and Kirsh2000) argue that ‘cognition is embodied’ and that our perception of the world is a complex interaction of internal and external human resources, for example relationship between mind and body, and the artefacts of our immediate environment, which themselves become part of our cognitive processes. ‘Just as a blind person's cane or a cell biologist's microscope is a central part of the way they perceive the world, so well-designed work materials become integrated into the way people think, see, and control activities, part of the distributed system of cognitive control’ (Hollan et al. Reference Hollan, Hutchins and Kirsh2000: 178). Analogous to the examples from HCI evaluation, networked improvisation is embodied, situated and contextual, and involves a similar inter-relation between human factors, technology and multi-tasking cognitive activities. The innate constructionist outlook of distributed cognition also allows for a hybridisation of epistemology and theory that encompasses the technological and empirical aspects of the research, ‘recognising that cognition and intelligence are not properties of an individual person but rather arise from interaction of a person with the surrounding environment’ (Resnick Reference Resnick1996: 281). O'Neill contrasts this idea to Heidegger's notion of consciousness or ‘being’ as ‘always embedded in our surroundings’ (O'Neill Reference O'Neill2008: 34). As our environment and everything in it is a multi-modal mixture of perceptual signs – or, as Gibson would have it, ‘affordances’ – ‘the more information that can be pushed towards the affordance end of our embodied semiotic spectrum, the less cognitively demanding will it be to find our way round’ (O'Neill Reference O'Neill2008: 115). The correlation here is between networked musical interactivity through distributed artefacts (network, instruments, environment and culture) and the fluid production and perception of signs in music.
7. Method
Underlying Hutchins’ theory is ‘ the nature of representations and the way that people use representations to do work’ (Hollan et al. Reference Hollan, Hutchins and Kirsh2000: 185). Employing methods of participant observation, interviews and informal post-performance conversations, data was gathered focusing on the use of metaphor as representation, and the musician's responses to musical signs as they appeared in the improvisation. The frames of reference for the interviews are based on the musician's individual scenario, examining features such as awareness of representation and corresponding musical interactions, responsiveness to changes of tempo, timbre and tonality. Variables such as time zone, climate, room size, and technical and instrumental configurations were also scrutinised to contrast their impact on the musician's perceptions and their recorded cognitive responses.
8. Data Collection
Field-mixed audio recordings of both performances provided the major source of data. The rest was from informal post-performance discussions, which supplied a second recording of the musician's immediate responses. Reviewing both recordings, the author compiled transcript interviews that drew detailed attention to instances of communicative behaviours between participants focusing on ‘metaphor’ in musical signs, transferences and responses. Correlations were also made between what the musicians said in the post-performance conversations and musical events identified in the recording of the performance.
9. Analysis
The analysis involved the reduction of the two forms of data into a descriptive narrative of each performance, identifying key musical events on a time line. A preliminary draft was forwarded to the musicians for participant feedback, and further responses were incorporated into the final description. A second tier of analysis was then applied, as the performances were broken down event-by-event along the timeline. This created new questions to which each musician qualified their responses until a ‘saturation of categories’ (Glaser Reference Glaser1998: 157) was complete.
10. Case Study 1 – Networked Radio Performance
10.1. Performance description
The first case study was a 35-minute improvisatory performance, involving five networked musicians and broadcast live on fbi Radio Sydney, Australia (table 1). It encompassed three musicians in a ‘hub’ studio at the University of Technology, Sydney and musicians in studio locations in Londrina, Brazil, and Braunschweig, Germany (excerpt in Sound example 1). Employing the multi-user network interface ‘eJamming’, the audio output from the interface was broadcast via P2P (peer-to-peer) streaming server NicecastFootnote 3 to the radio station fbi Radio via the Internet. Other than the musicians in Sydney, collaborators cannot see each other, and communication is via microphone and text bar. A telephone line was used for communication between the radio studio and the hub studio (Sydney). Apart from a technical rehearsal, it was the first time that this group of musicians had performed together for a live event. The performance serves to illustrate the musicians’ recognition of the musical ‘object’ (Pierce and Burks Reference Pierce and Burks1958: 210) in improvisatory dialogue, demonstrating their understanding of these objects as signifiers in musical expression. The map (figure 2) illustrates the locational network configurations of each performance.
Case study 1 – musicians, instruments, locations and time zones
Location of networked performers for each event.
The instrumental make-up of the group is one that reflects the authors’ interest in combining cross-cultural groups of skilled musicians whose practices reflect a variety of improvisatory traditions. Apart from providing an eclectic and complementary set of instrumental timbres, this inter-cultural diversity exemplifies the proposition of ‘ontological’ and ‘orientational’ metaphors traversing different cultures (Lakoff and Johnson Reference Lakoff and Johnson1980). Significant in these performances is the way in which such metaphors act as signifiers, both in terms of representation and the musicians’ intentions and responses to them. This was observed in both of the featured studies, and occurred in what Sansom describes as a ‘dialectic-interplay in the distinction between participant and musical object’ (Sansom Reference Sansom2007: 11).
Illustrating this in the opening 5 minutes of the improvisation, guitar, Turkish oud and Mongolian horse fiddle enter with elongated notes in A minor, over which electronic musician Martin Slawig contributes percussive textures, vibrating a cymbal in tandem with a programmed Max/MSP patch. Describing his intention and expectation here, he reported: ‘My plan was to start with a nail and sand on a cymbal, and use a preset of my max patch that I know will give me a certain atmosphere, which is also open enough to interact with the others … I expected a starting where everybody is bringing in more open sounds to develop a collective launching ramp’ (5:40). Slawig's notion of a ‘collective launching pad’ now serves as orientational metaphor and, together with the Moorish tonality and timbre of Turkish oud and Mongolian horse fiddle, contributes further representation on which the other musicians build. An interpretant ‘flamenco-esque’ melodic phrase played on trumpet (by Roger Mills) follows, acting as representamen of musical object in this metaphor-laden passage. Without the musicians’ prior knowledge, these first few improvised lines set up an atmosphere of the ‘East’ that serves as a recurring metaphor all the way through the improvisation. This is corroborated by guitarist Chris Vine, who reflects on where the improvisation had reached several minutes later as the horse fiddle led with chordal sequences reminiscent of a more structured form. ‘Well it is very expressive playing and the tone of the fiddle does conjure up images of the Steppes’ (12:49).
Horse fiddle player Bukhchuluun Ganburged contributes his own metaphorical association to this. When asked about the content of his improvisation, he replied: ‘the electronic sounds made me think of being near a river, which inspired my improvisation based on the playing techniques of the traditional “long song”’. The vast rivers of the Mongolian steppes are a well-known feature of this landscape, and interestingly Vine was unaware that Ganburged had made that admission, hinting at the power of the musical object in delivering meaning as an interpretant. Pegg (Reference Pegg2001) describes the gestural roles of instrumental playing techniques in Mongolian music as dependent on the type and construction of the instrument, as well as on the context in which they are played. An example of this is the physical manipulation of the instrument by the Eastern Mongolian player known as the Daringa Saaral. When melodically imitating the gait of a horse, the musician ‘reaches under the lower string with all three fingers of the left hand, applying gentle pressure, laterally, to the top string with the tips of his fingers’ (Pegg Reference Pegg2001: 71). In this example, the player adjusts his physical technique to play the instrument in a gestural way to produce evocative, imitative sounds – in other words, transforming the symbolic into representative.
The authors argue, as does Cumming (2010), that ‘qualities of sound’ carry ‘symbolic metaphor’ of gesture, and that this contributes to its interpretation. Cumming also suggests that ‘bodily motion – may account for the felt significance of sound’ (Cumming Reference Cumming2000: 134). As with the mental image of finger gesture on the staccato attack on violin or the trembling hand resonating vibrato on a guitar fret board, images of gesture help to codify qualities of sound in the minds of the musician and audience alike.
This is significant in the non-visual networked environment as improvisers draw upon images of embodiment to support their interpretation of sound in a non-visual environment. In a setting of diverse ethnic instruments, it is likely that unfamiliar gestures will be associated with their playing; however, what emerges is that physical movements demanded for creating qualities such as attack, staccato, vibrato and note duration are shared visual parameters of gesture. This is apparent in Uydu's post-performance interview, where he comments (at 6:53) on his consciousness of gesture during his oud playing: ‘I come in with trembling oud plucks and Roger followed it with a similar response, then I repeated the same action a couple of times’. Here the repeated action of ‘trembling’ illustrates the metaphor of gesture. Ganburged corroborates this by saying, ‘I could hear Yavuz plucking with very quick fingers. I wanted to contrast this with long notes.’
Ganburged's most observable trait was one of setting up complementary musical responses or oppositions, and recalling them as metaphors during the interviews. He often used analogies of gesture as a way of describing sequences of the performance. This is illustrated at 17:50, where he begins playing a repetitive conjunct progression from B to C, that the trumpet responds to with an F-sharp to G processed by a harmoniser set to an interval of a 5th. This gives it an unstable, lilting feeling that Ganburged interprets as ‘slipping between notes as on ice’, illustrating another example of using metaphors of gesture to interpret and describe a musical object. Similarly in the final 10 minutes of the performance, Slawig contributes a response to a passage of breathy trumpet flutters and syncopated percussion, which he describes using a postcard metaphor: ‘Fading in and out sounds, feeling like waves on a sea of plastic waste or flapping rotten sunshades at a long abandoned tourist place’ (27:50).
A notable observation was the cyclical nature of the group collectively building tension, resolution and deconstruction. This occurred in a pattern of rhythmic or textural layers over which musicians would create melodic and chordal constructions. These would build to a climax, and then fragment as a zenith in intensity was reached. Again this will be experienced in collocated improvisation; however, significant to networked improvisation is the way that this is negotiated through sound rather than visual gesture. It is the moment of release and conclusion that is most striking, as timbre appears to play the dominant role in communicating this between musicians. On each occasion tension is created by a combination of amplitude, dissonance and a sense of urgency in longer note durations, where the timbre of the main melodic instruments becomes grainy to the point of release. This was a recurring attribute of both case studies.
11. Case Study II – Live Networked Gallery Performance
11.1. Performance description
The second case study was a 40-minute improvisation between six musicians performed live to a gallery audience in Braunschweig, Germany (Table 2). Three of the six musicians were collocated in the gallery, and the other three networked from Sydney and New York (excerpt in Sound example 2). The network technology is again the multi-user interface eJamming. Four of the musicians were participants in the first improvisation, and the other two had never performed with this group. While many of the themes discussed in the first improvisation were present in this second case study, what emerges as a distinguishing feature is the effect of the musicians’ environments on their performances, and how this shaped their creative cognition during the event. The musicians performing at the hub location (Braunschweig) also noted that the presence of an audience and technical issues affected their performance. Slawig highlights the additional set-up involved in including an audience: ‘This time there were more technical things to organise … so this took nearly all my attention, and I played only very little parts and sometimes not that concentrated [sic].’
Case study 2 – musicians, instruments, locations and time zones
Utermöhlen's impression was of the impact on the intimacy between musicians: ‘The presence of audience gave this on-stage character to the session that took part of the intimacy between the musicians away.’ The suggestion here is that tension produced by including a collocated audience in a networked performance may alter a musician's perception musical signs. This could also be seen as an issue of technical organisation rather than necessarily the presence of an audience, but it does hint as Sansom argues that between improvisers ‘the relationship with aspects of the environment is strongly associated with the anticipation of the event’ (Sansom Reference Sansom2007:10).
Performing from New York at 5:30 am, Lainhart describes the positive effect that the technical issues had for him. ‘I had set everything up the night before and tested it all, so I knew it was all working … but once it was clear we had a problem getting everyone connected, I had to switch to technical mode … which I find concentrates my thinking in an interesting way. I've performed many times, several thousand at this point, and have on occasion been nervous for various reasons, but have always found that having to deal with technical issues immediately resolves such thoughts and frees my thinking from negative creative energies.’
In post-performance discussions it emerged that the musicians also felt that climate was a contributing factor to their creative cognition and interaction. Speaking from a northern winter, both Slawig and Lainhart acknowledged that temperature and available light affected their emotions and in turn their playing. Slawig reflected, ‘when it is dark and winter you don't give [sic] much time to look around, and when it is summer you feel much more relaxed, and this is also the same with the way I play’. Lainhart acknowledged that ‘The warm months definitely lead to more languid feel in my playing, which is more obvious in timbre than tonality’. Interestingly, Lainhart delimits timbre from tonality, which one might hypothesise would be equally affected by such variations. He contributes further, ‘I think tonality is more a function of what is going on in a group sound … timbre and level of activity perhaps.’ Here we see an awareness of individual timbre and collective tonality, and the effect of elements such as temperature on the emotional states of musicians. Through the lens of distributed cognition, aspects such as climate and environment become categories in the evaluation process and contribute to the ethnographic and musicological analysis. Interestingly, Krumhansel (Reference Krumhansel1997) has illustrated that listening to composed music with emotional content can contribute to physiological changes in body temperature. Whether this is the same for improvised music is another question and perhaps one for future investigation. What it does highlight, however, is the distributed nature of metaphorical association in networked improvisation and the potential influence of external circumstances on the perception of signification in sound.
Inherent in this performance, and in contrast to the first, was the instrumental differences between the two ensembles. The inclusion of two rhythm instruments, a drum kit at the hub location (Braunschweig) and the Turkish bendir in Sydney, led to rhythm dominating much of the semiotic representations and ensuing improvisatory exchanges. This is best illustrated in the opening three minutes as Uydu imposes a bendir part in 8/8 over a 4/4 rhythm initiated by drummer Edward Phillip, leading to the eventual synchronisation of both. This again draws the improvisation into a realm of Moorish or Eastern influences. Indeed, Maurice Ravel used this rhythm in much the same way in his Piano Trio in A minor to hint at the Basque dance rhythms of his paternal homeland. Uydu explains how this carries signification in an improvisatory context: ‘I feel this rhythm gives the feel that something is about to happen, shaking things up and something is about to go wrong. (Not in the music though, I mean the real-life feeling of the beat)’ (3:13–3:20).
On this occasion rhythm is the dominant signifier; however, timbre isn't completely absent; drummer Phillip goes on to express an interest in the different qualities in his drum sounds at this point. Reflecting on his desire to include a roto-tom in his kit, he reported: ‘I wanted to have a bright sound colour. I also used a towel on my snare drum, as it makes the sound darker and softer.’ Here we are reminded of Pierce's ‘scarlet trumpet’ in the way that the instrumentalist is describing the timbre he wants to convey (Smith Reference Smith1972). Lainhart substantiates this from his own perspective, contrasting variations of timbral states in his guitar playing with those of the Buchla synthesiser. He documents these, citing ‘ontological metaphors’ as a temporal representation of his playing throughout the performance:
• 5:50 – warm: sustained guitar with pedal
• 12:15–18:15 – warm: lap steel guitar, mostly as single lines and trying to echo the oud lines to some extent
• 19:20–21:15 – cold/harsh: Buchla synthesiser noise sweeps, which sound almost like symbols at times.
Cross-referencing these ‘states’ in the recording, the authors observed similarities between these timbral variations and their impact on the spectrum of improvisatory responses. In each example there are subtle changes in the qualities of sound that elicited further timbral and musical exchanges by musicians. Most notable among these were the rhythmic responses to Lainhart's timbrally ‘cold / harsh noise sweeps’ on the Buchla synthesiser. These sweeps expand and contract in amplitude over these 50 seconds of the performance, attracting interactive responses from the drums and bendir. Here a musicological analysis confirms an increase in tempo of the rhythmic exchanges as each sweep pulsed in volume. It is Lainhart's metaphor of a granular noise sweep sounding ‘almost like symbols’ that adds further signification to this interaction. As Utermöhlen comments, ‘because we had never played with Richard before, this meant a different level of concentration to recognise sounds and find their intentions’. It is this ‘recognising of sounds’ in timbrally complex layers where the sound of a synthesiser emulating a symbol can suggest different improvisatory responses. Likewise, at 36:54 Uydu in Sydney felt that ‘the textures in the sound changed the mood of my playing in an abstract way’. He is describing the sudden deconstruction of a driving polyrhythmic exchange between drums and bendir that is interrupted when ‘glassy’ Buchla sounds permeate the interaction. The timbre of these sounds have the effect of unlocking the rhythm and casting it adrift, requiring the players to respond in a more ‘abstract way’ without sounding out of time. This suggests that the sound qualities described by Lainhart equate with reflections made by Utermöhlen and Uydu, as with the aforementioned softer snare-drum sound soliciting more gentle responses from the other musicians at those moments.
12. Discussion
The evaluation of these two case studies serves as an introduction to the semiotic underpinning of strategies that cross-cultural musicians use to mediate improvisatory performance in the non-visual network interface. The analysis also illustrates the role of metaphor and embodiment in the perception of musicians creating and responding to musical signs in networked synchrony. This highlights the need for a more integrated qualitative analytical methodology for analysing qualities of sound to understand and fully appreciate the semantic and physical attributes of timbre. As shown in the example of Uydu's collaborative imposition of an 8/8 metered part on bendir, over a 4/4 drum kit rhythm in case study II, it is not only qualities of sound that musicians must learn to negotiate but rhythm and tonality as well. The inter-cultural nature of this exchange suggests that musicians negotiate unfamiliar musical forms in what Thompson describes as ‘the micrological pedagogical process of musical negotiation’ (Reference Thompson2007: 1). This is reflective practice in action, in which musicians ‘reveal a capacity for reflection on their intuitive knowing in the midst of action’ and ‘use this capacity to cope with the unique, uncertain, and conflicted situations of practice’ (Shön Reference Shön1995: 8).
Although the focus of this study has been on metaphor in musical signification, in what was probably the most poignant of the musicians’ responses we find an example of what Lakoff and Johnson refer to as ‘experiential gestalts … that play the role of a background in metaphorical understanding’ (1980: 178). In this instance, it is oud player Uydu who responded, ‘At 14:50 there is a part played by Elke or Martin, which took me away and gave me the idea of what I could do with the oud. That was a very painful feeling, especially at 15:39 and onwards. It reminded me of war or a child that lost his/her parents and left alone. Those moments are real dark places for me.’ Here we see a much deeper level of metaphorical association and creative cognition. It reveals the multiple layers of meaning contained within a networked improvisatory experience, underscoring Hollan et al.'s analytical theory of distributed cognition in ‘the interplay between human memory, external representations and the manipulation of objects’ (Blandford and Furniss Reference Blandford and Furniss2005: 3). In the context of networked improvisation, this ‘interplay’ is an arrangement of complex layers in flux, shifting between different levels of representation. The authors argue that it is by understanding these multiple levels of semiosis that creative dialogical exchanges can be developed and fully explored.
This application of semiotic evaluation to networked improvisation is intended to provide new insights for the interpretation of how musicians communicate and engender meaning in a dislocated environment without visual cues. The research opens the way for further investigation of semiotic, semantic, cognitive and timbral analysis in order to understand the range of inter-cultural musical dialogues facilitated by network technologies for real-time improvisatory collaboration. As presented, this includes expanding the lexicon of qualitative timbral terms used by musicians to describe their awareness of timbre and its semantic value, alongside ethnomusicological and spectral characteristics.
The next phase of this research will involve further case studies that make direct comparisons between musicians’ perception and cognition in scenarios of collocated and tele-improvisatory, networked environments. It will examine the strategies employed by musicians of the same and different cultures to mediate unfamiliar sound worlds, focusing on the role of signification in the form and structure of cross-cultural improvisatory traditions. This article has applied a triangulated semiotic and distributed cognition approach to analysing networked musical encounters. These studies have underscored the importance of timbre and musical gesture as signifiers in the non-visual networked environment for communication between tele-collaborative musicians.