1 INTRODUCTION
In a way, we are all experts at different forms of interaction, with both our physical and our social environment. From the moment we are born, we evolve cognitively and emotionally throughout life in relation to other beings, both human and non-human. Perhaps due to this ‘expertise’, the ubiquity of relatedness, or the hegemonic scientific paradigm, it is easy to downplay the value of collecting idiographic accounts of experiencing performance interplay mediated or assisted by computer technology. There is a constantly growing body of research in the music community on interactivity, and descriptive approaches are commonly used. However, phenomenological accounts of what performing with interactive music systems actually feels like, where an affective dimension also is included, are still relatively rare.
A few examples: Elizabeth McNutt (Reference McNutt2003) describes being ‘trapped between the bar lines’ when performing with interactive technology. Jean Penny (Reference Penny2011) emphasises the disembodiment of performing with live electronics, and Franziska Shroeder (Reference Schroeder2006) asserts the instrument as an extension of the self. Luciani, Florens, Couroussé and Castet (Reference Luciani, Florens, Couroussé and Castet2009) encompass felt experience, discussing the ‘ergotic gesture-sound’ with emphasis on tangibility of interfaces. However, their work is based on laboratory environment, thus lacking ecological validity.
Schroeder and Rebelo emphasise the body not as object, but as flux (2009: 136), which is similar to my approach. Furthermore in my work I strive for a seamless – cohesive – unity between singing and sound synthesis, although this does not mean per se that the relationship always will be experienced as seamless. Discontinuities, like Schroeder and Rebelo (Reference Schroeder and Rebelo2009) argue for, as well as resistance, may still appear experientially for the performer. My approach is also similar to that of Johnston, Candy and Edmonds (Reference Johnston, Candy and Edmonds2008), to devise no extra control unit apart from the live input.
There is a fine line between exploration/discovery and repeatability in the kind of responsiveness a system exhibits (e.g. Mulder 1994; Paine Reference Paine2002; Drummond Reference Drummond2009). Is it a matter of surrendering control that is at stake? Or is control pivotal for interactivity? The notion of control is indeed disputed. The community of music scholars seems to be divided in two: there are those arguing for a move beyond control (Broadhurst Reference Broadhurst2006; Frisk Reference Frisk2008; Johnston et al. Reference Johnston, Candy and Edmonds2008; Paine Reference Paine2009) at the same time as there are others defending it as one cornerstone for interactivity (Drummond Reference Drummond2009; Eigenfeldt Reference Eigenfeldt2011).
Moreover, there is a plethora of approaches towards mapping: neural networks, linear, behavioural objects, metaphor and ‘control mapping’ to mention but a few. Many times direct mapping is considered not to provide enough exploratory qualities in interactive systems (Paine Reference Paine2002; Drummond Reference Drummond2009). Mapping through metaphor is said to be particularly suited for enabling embodied knowing (Antle, Corness and Droumeva Reference Antle, Corness and Droumeva2009), and is an approach applied in one of my previous works PS. I will be home soon! (Einarsson and Friberg Reference Einarsson and Friberg2015) and by Essl and O’Modhrain (Reference Essl and O’Modhrain2006), even though the latter, although discussing metaphor as weak sensorimotor integration, never make reference to Lakoff and Johnson (Reference Lakoff and Johnson1999) or Johnson’s (Reference Johnson2008) seminal work on the matter.
Musicologist Eric Clarke (Reference Clarke2005) presents an ecological approach to the perception of musical meaning, drawing upon the work of James J. Gibson. Unfortunately, he only spends a handful of pages discussing listening-when-playing, that is, the performer’s perspective. There he hints at the particularities of the performance situation, where the separation between the object (the music) and the subject (the musician) tends to blur: ‘In playing music, the object really is within the control of the subject, because perception and action – held apart for listeners in concert culture – are in dynamic relation with one another’ (ibid.: 150–1). According to Clarke, the type of instrument performed plays a role in determining how the particular relationship between action and perception is expressed in the particular situation. As I will be suggesting, in singing this conflation is taken to the extreme.
2 AIM AND METHOD
When the computer is conceptualised in the discourse of mixed works, that is, works where an acoustic sound source and an electronic sound source are used in combination, it is usually considered as either an extension of the musician’s instrument or a separate entity, an ‘other’. Simon Emmerson, for example, writes in The Oxford Handbook of Computer Music: ‘How does the instrumentalist respond – literally and musically – to the disembodied “other” (even when it is a mirror image)? It is the duality that is the focus here’ (Emmerson Reference Emmerson2009: 168).
Now what do the accounts from singers performing a mixed work reveal? How do they conceptualise the computer? And how can their accounts be understood when situated in a framework of radical embodied cognition and ecological psychology (Clarke Reference Clarke2005; Chemero Reference Chemero2009)? Interactivity provides an opportunity for noting aspects of performing music through ‘an event that bring [sic] corporeal and cognitive awareness’ (Morse Reference Morse2003: 18). In the following, it is the experiences of the ‘self’ and the listening, singing and relating to fellow musicians mediated by the computer technology as well as the relationship towards the computer as such that are collected and highlighted. Through singing when standing, walking and even in elevation, emphasis was put on how embodied activity would influence this experience.
The investigation is a continuation of composing and working with the chamber opera PS. I will be home soon! (Einarsson Reference Einarsson2015; Einarsson and Friberg Reference Einarsson and Friberg2015) and takes place in the work Metamorphoses, a responsive work for four laptops and four vocalists, where singing voice features affect the subsequent sounding electronics. The word ‘responsive’ is used to denote a work facilitating an exploration of different ways of conceptualising the computer, less occupied with control and more concerned with different ways of responding (Kozel Reference Kozel2007: 21). Data is collected with semi-structured interviews and analysed with the method of Interpretative Phenomenological Analysis (IPA) as a touchstone. In brief, IPA is intended not only to generate descriptions but also to add a hermeneutic layer. Holmes and Holmes write: ‘phenomenological analysis and interpretation can reveal an idiographic relationship between the subjective conscious awareness and the cognitive landscape of the individual’ (2013: 74), and they recommend the choice of IPA when doing explorative research on music performance experiences (ibid.: 80). IPA was also chosen due to the focus of lived experience and how performers try to make sense of their experiences, with an emphasis on a particular context (i.e. Metamorphoses) as well as its proximity to embodied cognitive approaches (Smith, Flowers and Larkin Reference Smith, Flowers and Larkin2009: 198–200).
3 EMBODIED COGNITION
Embodied cognition, sometimes referred to as ‘second-generation cognitive science’ (e.g. Shapiro Reference Shapiro2007), is a vast and diverse field of research. It has grown out of a reaction against the traditional computational model of cognition, according to which human cognition is considered to work as a computer program and where cognition is primarily seen as an abstract manipulation of (meaningless) symbols (internal representations), and where the body is superfluous (Shapiro Reference Shapiro2007; Johnson Reference Johnson2008; Lindblom Reference Lindblom2015).
Roughly speaking, two branches can be said to exist within embodied cognition. One is embraced more by the Artificial Intelligence (AI) community and Rodney Brooks, where a robot is embodied due to having a body and is situated since it exists in the world (Shapiro Reference Shapiro2007: 139), an approach also denoted mechanistic embodiment (Sharkey and Ziemke Reference Sharkey and Ziemke2001). The other branch, building upon Gibson ecological psychology which is also the one I adhere to in the following, arrives in radical embodied cognitive science pairing Gibson’s ecological psychology with dynamic system theory (Chemero Reference Chemero2009; Käufer and Chemero Reference Käufer and Chemero2015) and has the living body as a cornerstone, phenomenal embodiment (Sharkey and Ziemke Reference Sharkey and Ziemke2001).
There is a growing body of interdisciplinary research in support of a radical view of embodied cognition where sensory-motor processes play a vital role. Groundbreaking research on mirror neurons, Antonio Damasio’s and Joseph Ledoux’s research on emotion and, of late, new findings on embodied social cognition and embodied simulations (see Lindblom Reference Lindblom2015) seem very convincing and in extension also provide interesting perspectives on performance practices in mixed work with computer technology.
According to embodied cognitive science, the mind is not independent from the world (Johnson Reference Johnson2008). The subject/object division is considered an abstraction (ibid.), and interacting with computer technology is very different from the idea of an external manipulation of a tool or an object but instead a coupled activity. Meaning in interactions between organism and environment arises from movement, emotions, and metaphor, all of which are firmly rooted in bodily experience (ibid.).
Furthermore, it is through repeated firing of neurons – ‘neurons that fire together wire together’ – that learning is made possible. This repeated stimulation, which results in adaptation largely due to brain plasticity, is in ecological terms referred to as perceptual learning (Clarke Reference Clarke2005: 22–4) and part of the coupling between organism and environment to which ecological psychology has paid considerable attention.
Finally, a few words on the relationship between the capacity of an organism and features in the environment, so-called affordances. The concept is so watered down from its original sense that it has almost lost its meaning altogether (Torenvliet Reference Torenvliet2003; Susi and Ziemke Reference Susi and Ziemke2005). In the following, affordance will be used as denoting action possibilities in the environment, that is, a direct relationship between organism and environment (Chemero Reference Chemero2009). As Käufer and Chemero (Reference Käufer and Chemero2015) point out, Gibson’s affordances have a peculiar ontological status for they are neither a property of the organism nor the environment, but relational. There may be sequential affordances, where we perceive one to then perceive the whole (Gaver Reference Gaver1991). This makes sense in terms of computer programs realised during performance. One action may in turn open up a range of additional action possibilities once the response is provoked. The microphone is, on the other hand, another story, since there is not really anything about the microphone that suggests ‘singing-into’ affordances. This displays a cultural dimension, possibly encompassed in what Gibson would call ‘second-hand knowledge’ (Gibson in Rambusch and Susi Reference Rambusch and Susi2008). That culture is part of what we perceive is acknowledged by Clarke (Reference Clarke2005), but not something Gibson paid particular attention to. What humans use as guiding principles to select between affordances is still to be uncovered, but one interesting suggestion about the agency is the concept ‘invites’ (Withagen, de Poel, Araújo and Pepping Reference Withagen, de Poel, Araújo and Pepping2012). Also perceptual learning may assist in differentiating between information pointing towards affordances in the organism’s environment. According to Gaver (Reference Gaver1991), perception of affordances may be ‘highlighted’ by aspects such as experience, culture, social setting and intention, yet this constitutes a departure from Gibson’s original claim that perception is direct. Subsequent research also confirms how social knowledge impacts what we perceive in many different ways (see Lindblom Reference Lindblom2015).
Affordances do not cause us to act; they are opportunities for action. Neither are they about appearance only of a device or object, which is a common confusion in the matter (Rambusch and Susi Reference Rambusch and Susi2008).
4 THE WORK METAMORPHOSES
The work Metamorphoses is a responsive work for four laptops and four singers. Jörgen Dahlqvist (Teatr Weimar) wrote the text, drawing upon the lives of four characters whose lives have been transformed in one way or another. The piece consists of three successive parts, where the first part containing the sections ‘the small legs’ and ‘I was there’ is dominated by granular synthesis with different add-ons, the second part a poly-synthesiser sensitive to incoming pitch and the third part containing the section ‘holds up, stop’ has a build from the IRCAM object psychoirtrist as its major component. The way technology is implemented is further described below.
4.1 Technical description
4.1.1 Singing voice feature analyses
The feature analysis applied in the work was a translation of Anders Friberg’s application for Pure Data (PD), the cue-extractor (Friberg, Schoonerwaldt and Juslin Reference Friberg, Schoonderwaldt and Juslin2007) to Max/MSP. The extracted features are amplitude, pitch (yin~ from IRCAM), mean duration, tones, articulation, soundlevel (SL), attack, spectral slope and onset.
4.1.2 Mapping
Inspiration for mapping was drawn from the approach multiple-layered mapping as described by Hunt, Wanderley and Paradis (2013). On a conceptual level it is a one-to-one mapping. However, as the authors of the article acknowledge, many low-level parameters have been cross-coupled. In this piece there are three layers: Performance features → Perceptual feature → Sound synthesis.
4.1.3 Sound synthesis
The main sound synthesis engines were as follows. Part one executed granular synthesis by the rogs~ object from IRCAM. Part two consisted of a poly~ synthesiser/sampler tuned by preset values stored in a table. Part three was a combination of different forms of synthesis: mosaicking~ and psychoirtrist~ by IRCAM and comb filters.
4.2 Singers
The participating singers all had a solid background in jazz and improvisation, and some to moderate experience of singing contemporary music. The singers had little to no prior experience with performing with live electronics. The author and composer of the work also participated as a singer.
4.3 Data collection
Two major data collections were made. On the first occasion the singers were first asked to write down their experiences from the day, to facilitate recall and avoid regression to mean. Second, an hour-long open structured interview with all three singers took place. During the whole day testing was recorded and the resultant discussions taking place while testing were also transcribed.
For the second data collection, an interview schedule was constructed to cover areas of interest. It drew on the earlier work with chamber opera PS. I will be home soon! and writings of Simon Emmerson. Separate interviews were conducted; each interview lasted about one hour. They were recorded and later transcribed verbatim.
4.4 Data analysis
The analysis phase followed the following stages.
Five superordinate themes were identified from the process of repeated reading, where each of the 4th (and final) interview transcriptions were juxtaposed finding psychological conceptualisations at slightly higher levels: (1) singing, body and emotion; (2) listening and seeing; (3) relating and interplaying; (4) the process towards the work; (5) the score. This article focuses on the themes (1)–(3) presented in the subordinate themes:
-
∙ The computer as extension or separated
-
∙ The computer as a fickle playmate
-
∙ Body and gesture
-
∙ Participation and uncertainty: influence by culture
-
∙ An expanded sphere
-
∙ A sense of whole
-
∙ Listening in dynamic dialogue with the situation at hand
-
∙ Experiencing in an altered bodily awareness
-
∙ Looking for a response
-
∙ The character of the response
-
∙ Situational factors.
It might be worth noting that, just as IPA is a method with emphasis on interpretation, interpretations drawn are informed by the interviewer as a participating vocalist in the work and collaboration towards the work through workshops and rehearsals. On the one hand, this threefold role of interviewer/composer/co-performer could be considered problematic. On the other hand, this facilitated a shared cultural understanding, something someone from the ‘outside’ can hardly acquire.
5 RESULTS
The singers’ accounts will be discussed from three main points of departure: first, conceptualisation of the computer; second, experiences mediated or moderated by the computer when singing, listening and relating; and third, possible mechanisms behind these experiences such as the characteristics of the response, the effect of the response and the impact of the dynamics between performer and environment during performance.
5.1 Conceptualisation of the computer
5.1.1 The computer as extension or separated
What denotes the interplay with the responsive technology in Metamorphoses? Seemingly there is a shift in the experience throughout the work between the computer as an extension of the singer’s instrument and the computer as a separate entity. Speaking in a very tentative and almost mysterious way, the word ‘spirit’ is suggested, and singer one arrives at depicting the computer as follows: ‘Perhaps more spirit than extension, but a bit of an extension as well.’ Singer two displays a similar ambiguity: ‘I don’t see this as something outside of myself, even though it is.’ The singers provide examples from the work where either one dominates. The section referred to as ‘stannar upp, stanna (holds up, stop)’ serves as an example of an extension of a singer’s voice. The application mainly active in the section is a build on psychoirtrist~ from IRCAM. It anticipates the singing voice feature vibrato, its presence or absence, which affects the transposition of the electronics. Pressed voice, derived from the quality parameter in yin~ together with amplitude, also affects the electronics, altering its frequency.
The section ‘jag var där’ (‘I was there’) is referred to as an example of the computer as separated. One singer visualises someone residing inside the loudspeaker: ‘One almost visualises someone living inside the speaker.’ In this section there was a combination of comb-filtered granular synthesis and feedback delay. The singing voice features pitch and duration evoked the responsiveness. Pitch alone affected timbre by controlling interpolation between two different settings for a spectral filter, and in combinationFootnote 1 with amplitude it affected the number of grains in the response.
Experimenting with the singers adjusting the length of the granular response in the section ‘dom små benen’ (‘the small legs’) during one of the workshops, it was also notable how, depending on the length of the response, singers experienced the live electronics as separated or extended, although there was no pivotal point in common for everyone. The degree to which it is possible to anticipate the behaviour of the computer also influences whether it is perceived as separated or extended, as well as how quickly the computer response follows the sung input. Moreover, all these parameters act in concert with other situational cues, such as the balance in sound level between voice and live electronics, proximity of loudspeakers and positioning of fellow musicians.
Drawing upon the majority of the singers’ interviews, the distinction between extension and separation indeed seems permeable, or at least they do not act in opposition, but rather as a complementary principle, almost like yin and yang: ‘Also because it plays with you, and lives its own life and such. So it is easy to form oneself around it. And it forms itself around me.’ What is left is a web of relations high in complexity where everyone has their own ‘extended arm’: ‘It becomes somehow more because everyone has an extended arm, everyone has another dimension or whatever you call it. So it becomes like a richer soundscape.’
5.1.2 The computer as a fickle playmate
One singer discusses the great number of preconceptions she has towards the computer. She makes use of the computer as a tool for writing scores quite a lot, and the frustration that gets stirred up by a malfunctioning computer program is an association close at hand for her. She prefers talking about the computer as a fickle person, and she is more prone to address the computer as separated from her. She primarily assigns this inclination to the ability to – as she put it – have the computer as a playmate.
5.1.3 The body and the gesture
It is almost as if the physical presence or absence of another body – musician or computer – is of less importance. The prime concern is the musical gesture. Is it playful? Is it imitative? The imitative quality in the electronics brings out a sensation of being connected. Singer two:
Some musicians or some contexts, one feels that one is not playing together; so it is this I mean, that I feel like I am playing together with the electronics, and therefore we are together, in some way. And then I do not feel that it is a separate layer. And since it also imitates the sounds one makes oneself, and like that, then it becomes easy to feel a connection with it.
This issue is brought up by Marc Leman, that ‘mimicking may enforce liking and social bonding’ (Leman Reference Leman2008: 144–5). Perhaps some degree of imitation may be a good thing if a composer wishes to enhance the feeling of connectedness with the computer technology.
5.2 Experiences of listening, singing, relating
5.2.1 Participation and uncertainty: influence by culture
Different aesthetics and compositional approaches give birth to different responses on the part of the computer, which will naturally elicit different experiences in the singer when performing the mixed work. Quite contrary to the idea that it makes no difference whether the relations between electronics and singer are real or inferred, a singer with previous experience of performing with fixed media emphasises the shift in the sense of participation when working with responsiveness: ‘I have never before worked with any sorts of effects or such on my singing, apart from working with NN, but then it was that it – sort of – worked with me. More than that, I was like not really part of it.’
She then contrasts that experience with the work at present, where there is an exploratory approach and an element of waywardness in the relationship towards the computer, something she returns to throughout the collaboration as something valuable: ‘one sees that the singer sings and generates a electronic sound, and perhaps next time something else results. So, since it’s not really totally set, that there are differences in the electronics part creates a certain … insecurity … which one is totally used to if one is used to improvisation.’
This latter pinpoints how culture constitutes part of the bricolage we perceive, and as Eric Clarke writes, ‘Culture and ideology are just as material (in the concreteness of the practices that embody them) as are the instrument and human body that generate this performance, and, as perceptual sources, they are just as much a part of the total environment’ (Clarke Reference Clarke2005: 61).
5.2.2 An expanded sphere
When the response from each one of the singers consists of both the sounding voice and the computer response, this renders the voice alone less significant. The singers describe in slightly different ways an experience of a sort of branching out, of experiencing having an expanded sphere, which impacts singing as well as listening. As for singing, the unusual and slightly awkward experience of being responsible for sound not chosen by the singer herself (the electronic sounds) was brought up. In sharp contrast with the high degree of control exerted by singers and the very intimate and corporeal relationship towards the voice production, the computer technology was suddenly part of their vocal sphere.
5.2.3 A sense of a whole
Emphasis is given to the sense of a whole when listening: ‘One is more “one”, a mass, instead of different musicians’, ‘I feel that my role is different, that when the sounds I choose to make are supposed to be a part of the whole … um … that is supposed to be mixed together, instead of running off somewhere.’ A sense of distance is expressed, even though everyone was positioned very close in a semi-circle when performing in concert. She continues picturing an almost contemplative listening when describing how ‘the ears sink outwards’.
5.2.4 Listening in dynamic dialogue with the situation at hand
The experience of listening to a whole is echoed in the second singer, but a whole containing multiple layers. Directing her attention is a cornerstone, with inclusion and exclusion of different aspects of the performance situation, such as ‘listening’ and ‘un-listening’. There is also a listening for what is not yet there, for the music’s potential and the composer’s intention. ‘listens … according to what you think that you want the direction of the music to take’. There is no one way of listening, she exemplifies, but a listening in dialogue with the situation: ‘one always listens differently, in almost all one’s concerts. That is, in all contexts where one takes part.’ Since the situation encompassing responsive technology comprises many layers, there is an oscillating movement between the singer’s own expanded sphere, the others’ expanded spheres and the resultant whole:
I am always listening to what the electronics are doing. And try to play with it. So I am listening, trying to analyse it, and we have done that together too, and you have sometimes also told us what the different things do. So I play with it as if it was a musician. So, so it is a way of listening, while I am at the same time listening to what I am doing, because that also becomes a part of it then: what one does oneself and what one can do with the electronics is one thing, and then what you are doing and what you do with the electronics is another, so all this becomes a whole that is a listening.
Throughout the work she is assertive to possible affordances in all these different layers mentioned. Technically there are different parameters for analyses and synthesis in different sections of Metamorphoses, so the computer will respond differently at different moments in time, as the sounding environment changes. Not only does this impact the musical action throughout the work; even more so, it changes its meaning dynamically, since affordances in an ecological perspective are tightly connected to meaning-making. The computer responsiveness may give birth to different sets of meaning for different singers at the same time-event, but also quickly or dynamically change how singers make sense of the computer and their interplay when performing.
5.3 A look at possible mechanisms
5.3.1 Looking for a response
During the process of working with Metamorphoses, especially during the first two workshops, the singers tended to fall into a pattern of looking for a computer response. Why is this so? For one it may be just a novelty effect, which would go away over time. There is a notion that comprehension of the relationship towards the computer evolves over time: ‘that one gets a better understanding of how we communicate [over time]’. Listening strategies may also develop as time goes by. In the following the singer describes how she attends to the response: ‘One is used to singing in one’s own body, to be totally present in what comes out. One makes a sound to hear the reaction, that which NN spoke of earlier.’ The same singer also comments that the relationship towards the device takes time to grow into: ‘It takes some time for them to grow together into music-making. It can take quite some time.’
Another way to look at it is as a bringing something forward, putting it on display in a ‘look what I found’ sense. The actual meeting is then not with technology but with each other through technology. Something is being uncovered for a fellow musician or an audience,Footnote 2 real or imaginary. Yet in the first workshop, one account was how the looking for a response happened at the expense of the relationship towards the fellow musicians.
Voc 1: It is quite easy to not think musically, at least for my part. It feels like [brr, brrr, helloooo].
All: [Laughs]
Interviewer: Ok?
Voc 2: To not think musically?
Voc 1: No but, it doesn’t feel as if we have made music together. I am more interested in, if, how … It’s more on my part, if one can get past it.
Interviewer: (Laugh) … if one can get past it?
Voc 3: I think so!
Voc 1: One must be able to. There has to be some discipline. But it’s fun to play too.
Then, quite contrary to meeting each other through technology, the relationships are isolated islands, singers separated from each other.Footnote 3
But maybe the question should be put differently? Perhaps the looking for a response in the mixed work is a necessity? For what would be the case otherwise? The texture would be extremely dense and the function of the electronic sounds as a partner in dialogue and impulse-giver would almost disappear. The live electronics would solely be assigned the task of background. So, tentatively speaking, maybe this aspect of looking for a response in the computer technology is mandatory, but in interplay with all the material and people contained in the situation. This puts high demands of flexibility on the performer, but would harmonise with the idea of something developing over time.
5.3.2 The character of the response
The character of the response shapes what is sung, it constitutes a situation for improvisation and interplay. The granular synthesis structure is commented on in the following:
And it is rather dense, I was thinking. Or at least somewhat dense. And what, what was nice were the rests … But it wasn’t anything negative, so … not the way I thought of it, per se. That causes one to relate to it. In order to take it in and understand it you need to be quiet. A bit.
In the above application the electronic sounds follow the length of the sung input. The response elicits a way of singing where there is space for the computer response. Following this, the character of the response impacts the improvisation to become more fragmented than it otherwise would have been with having the score only. As a consequence, there is a layer in the work that is rather implicit and brought forward only when the work is performed:
For example, this ‘springande springa’ (sings) and it’s like that, and then reactions stem from that. And then I know that this way, that there came a funny sound when I made the ‘r’s or how it was like so I went on, then it generates more ‘r’ from me to see what will happen with it. So of course it affects.
5.3.3 The situational factors
The particularities of the situation are not negligible. The responsive system encompasses four computers and four vocalists, which brings on a completely different situation from a solo singer performing with responsive electronics in an acoustic ensemble. Somehow it normalises the situation and also impacts listening. In addition it contributes to a sense of communion. Not least if the shared situation also comprises elements of challenge as in the concert performance, where triggering was a delicate action performed, preferably in synchronisation, at predetermined locations notated in the score. Another factor mentioned is the presence of the composer, resulting in a clearly felt relation to the composer’s intention in the technology.
5.3.4 Experiencing in an altered bodily awareness
To reiterate, affordance is a relation between the capacities of the individual and features of the environment (Chemero Reference Chemero2009). Then how is the experience if the capacity is restrained, as when singing in elevation? The assumption was it would be drastically altered, but as it turns out, the answer is both yes and no. The impact on the performance aspect of the voice production is almost none. The learned component of voice production in professional singers is profound; and all singers agreed it is such a strong corporeal practice. And at the same time, being a practice so firmly rooted in the body, would not an alteration of the bodily awareness leave any imprint? Well, the audience fades from the awareness, or at least there is a sense of distance in my experience. The others’ bodies are of less importance. They can no longer be used for visual cues, nor be experienced kinaesthetically; they are just too far away and/or too restrained. When singing in the harness uplifted in the ceiling, the channel for interplay is focused to the singing voice, the sung gestures. Listening is accentuated; that is where the interplay with the others occurs. It wanders in between the different layers of the sounding – the others’ emotional expression, the cues and responses from the electronics from each and every one, hearing one’s own voice somewhat from a distance, but also meanwhile a heightened awareness towards the bodily sensation and the feelings that rise and fall with the situation: tense, relaxed, active, passive.
6 DISCUSSION
What happens when the electronic layer is added to the acoustic layer in the mixed work? There is a shift in attention – of course – since something is being added. And the question emerges among the singers as to which one of the sounding tails is their response as a consequence of the web of interrelations being expanded.
What happens though is not as simple as to say that A has been added. There is a change in impulse-giving, which in turn affects how the singers perform. Certain applications provide certain responses, generating a push in a particular direction. For example, the implementation of the object psychoirtrist~ from IRCAM was very responsive to [v]- and [f]-sounds, as also was suggested in the score. When the singers experimented with inviting new sounds into the improvisation, the general comment was that there was a pull back to these [v]- and [f]-sounds, which elicit the strongest response. Maybe it is similar to empathic confirmation (validation) inside a dialogue (Rogers Reference Rogers1975). What brings out resonance in the listener is elaborated on, what is un-echoed is dropped (Miller and Rollnick Reference Miller and Rollnick2012); hence a flow or an unspoken direction is set out.
But what about this change in the sounding situation, in what way does this differ from, for example, overhearing the sound of a truck being parked with the engine still on outside the house when performing? Maybe the difference lies in the pre-composed rule system where a relation between what is sung and what the subsequent electronics do is decided beforehand. Talking about attention, the performer may decide to encompass the truck in her field of attention and thereby establish a relationship, that is, direct her attention. Nevertheless this will form a static relationship in the sense that no matter how the performer sings, it will have no effect on the sounding electronics. It remains detached. The computer on the other hand is capable of assuming different roles. It may assume the role of the ‘truck’ having a process completely separated from the performer, at an instant switch to denote the opening of a new section and form a sort of way mark, then mould into an extension of the performer’s musical expression by some preconceived parameter. Thus the computer is characterised by its versatility.
Turning to the performers’ interplay with the computer, one recurring line of thought throughout the accounts of experiences from the work Metamorphoses is the importance of the situation; the computer is assigned different roles and ascribed different sets of meaning depending on what sounds it is generating. What was possible in one situation is no longer a possible course of action in the next, and this interplay is constantly evolving and reshaping the situation. As previously discussed, affordance is tied with meaning-making, which may help explaining the changes experienced of the computer being on the one hand an extension and on the other a separate entity. It is not only the computer that transforms but also the person in interplay with the computer. It is not only the situation and the computer, or the performer’s way of singing, but also the two intertwined that reach into and transform the performer’s experience of self. Herein also lies the core of direct perception as described by Chemero (Reference Chemero2009). The object in the situation is neither separate nor an extension, but both.
Parallel to the work’s momentary processes is the more overarching notion of perceptual learning, as described in section 3. As the singer studies the work, participates in workshops and learns the principles for interacting with the computer, a transformation is also taking place in her, especially in such a process as Metamorphoses, lasting for more than a year. This transformative potential sheds some new light onto the process towards a work, bringing some clarity, but also adding yet another layer of complexity to the interplay. The performer’s ability is not static but highly dynamic.Footnote 4 Johnston et al. (Reference Johnston, Candy and Edmonds2008) write that the single biggest factor differentiating various modes of interaction with a virtual instrument in their study is the issue of control. These findings serve as an important reminder that perceived control is not static and set once and for all.
According to Clarke (Reference Clarke2005), culture is part of what we perceive. For one of the singers, the computer (in particular the Mac computer), constitutes part of her office working environment, and pertinent to this its ‘working-partner’ qualities become a salient aspect of the environment to her. In relation to listening, the formal training a musician has is yet another cultural component. What are the individual’s listening habits? In Metamorphoses there was the notion of attending to a whole, much more than on other occasions.
Furthermore working with jazz singers, the experience of insecurity about what to expect from the computer response was totally natural.Footnote 5
In the beginning of the article it was suggested how the conflation between perception and action is taken to the extreme in the practice of singing. Drawing upon my experience, this is due in part to the very mechanism of singing, where the experience simultaneously is of the sounding reaching the eardrum, vibrations through the skull and muscular sensations in the larynx, and the experience of the full bodily state as described by Damasio (Reference Damasio2003). One vocalist in Metamorphoses describes this confusion, or conflation, expressing a sensation of constituting one mass, dissolution of the subject, depicting the metaphor of being a body moving as a school of fish. Singers describe both being and having their voice. So even towards theirs own voices there is a relationship that is at the same time extended and separated.
During the performance, the affordances appear in several layers simultaneously – in the staged setting as well as in the sounding (music) setting. There are interesting indications of an amplification of the sung expression when these are pulling in the same direction, which may be understood in relation to intermodality, how senses inform each other.
According to Paine (Reference Paine2002), a detailed treatment of the nature of the relationship, that is, mapping, is crucial to perceiving a correlation between the quality of the input and the quality of the output for those who engage with the interactive sound system. Indeed Paine has a point, but there is also a need to emphasise the small word ‘quality’, because the issue is not only what connections are devised but also how they behave will decide how the relationship is experienced. Affordances alone do not aid embodiment, as proposed by Paine (Reference Paine2009), at least not affordances in the Gibsonian sense, because they do not reveal anything about what actions are chosen from multiple opportunities or how the action chosen is carried out.
Composer James Andean (Reference Andean2011) has attempted to apply the ideas of Gibson to listening to electroacoustic music with the idea that the physical body has precedence over other sources of information on the musical scene. He describes a scenario where the listener first scans the situation for humans and then proceeds to ‘reading’ the situation in a particular order. This order of events is not supported by any empirical findings in the article and rather bares traces of a hierarchical processing of information, something a more radical interpreter, like Chemero (Reference Chemero2009), would hardly agree on. Perceiving and experiencing is the same according to Chemero (Reference Chemero2009), and there are no representations or cognitive calculations to mediate what is perceived. Turning to experiences from Metamorphoses, would sounds generated by a physical visible body/the performer have precedence in the way Andean suggests? The experiences suggest otherwise. The physical body is of less importance, at least for the performers. The attention wanders between voices and electronic responses. Speaking in ecological terms, what causes shifts in attention is rather changes in affordances. Marc Leman makes the claim that we always listen for what is causing the sound and, if anything, human beings in the mixed work are a given. If anything, attention is drawn to ambiguities (Windsor Reference Windsor2000). The structure of the sounds, their behaviours, qualities and placement are all important facets to consider when trying to determine what causes the singer’s readiness to act and shift of attention.
7 CONCLUSION
To summarise, the practice of singing is an embodied activity, in which body and mind are not separated but conjugated. Embodiment is a vehicle for being situated, and, as I have shown, using theories from ecological psychology and radical embodied cognition, the coupled interplay with the situation as it unfolds through a musical work is of great importance. When the computer technology is incorporated forming the mixed work, as in Metamorphoses, it constitutes part of both the embodied practice of singing and the situation.
Despite the limited number of participants and observations, the described accounts from singers may nevertheless help in raising questions and deepen our understanding of the relationship towards computer technology in a mixed work, and perhaps to some extent be extrapolated to how technology is related to everyday life. This may also lay foundations for future compositional tools and/or performance practices.
