1. Introduction
I have explored interactive systems extensively in my own creative sound art practice, inspired by their potentials to facilitate liquid and flexible approaches to creating dynamic sonic temporal structures and topographies while still maintaining the integrity and overall identity of an individual work. Just as a sculpture can change appearance with different perspectives and lighting conditions, yet a sense of its unique identity is still maintained, so too an interactive sound installation or performance may well sound different with subsequent experiences of the work, but still be recognisable as the same piece. However, the term interactive is used widely across the field of new media arts with much variation in its precise application (Bongers Reference Bongers2000; Paine Reference Paine2002). This liberal and broad application of the term interactive does little to further our understanding of how such systems function and the potentials for future development. The description of interactive in these instances is often a catchall term that simply implies some sense of audience control or participation in an essentially reactive system. Furthermore, with specific reference to interactive sound-generating systems, there is considerable divergence in the way they are classified and modelled. Typically such systems are placed in the context of Digital Musical Instruments (Miranda and Wanderley Reference Miranda and Wanderley2006), focusing on interface design, gesture sonification (Goina and Polotti Reference Goina and Polotti2008) and mapping, defining a system in terms of the way inputs are routed to outputs, overlooking the equally important and interrelated role of processing. However, the term interactive still has relevance, as it encompasses a unique approach to compositional and performative music-making, hence the need for this paper, drawing together both historical and contemporary practice.
An interactive system has the potential for variation and unpredictability in its response, and depending on the context may well be considered more in terms of a composition or structured improvisation rather than an instrument. The concept of a traditional acoustic instrument implies a significant degree of control, repeatability and a sense that with increasing practice time and experience one can become an expert with the instrument. Also implied is the notion that an instrument can facilitate the performance of many different compositions encompassing many different musical styles. Interactive systems blur these traditional distinctions between composing, instrument building, systems design and performance. This concept is far from new. Mumma (Reference Mumma1967), in developing his works for live electronics and French horn, considered both composing and instrument building as part of the same creative process. For Mumma, designing circuits for his cybersonics was analogous to composing. Similarly, the design of system architectures for networked ensembles such as The Hub (Brown and Bischoff 2002) and HyperSense Complex (Riddell Reference Riddell2005) is integrally linked to the process of creating new compositions and performances.
1.1. Shared control
A different notion of instrument control is presented by interactive systems from that usually associated with acoustic instrument performance. Martirano wrote of guiding the SalMar Construction – considered to be one of the first examples of interactive composing instruments (Chadabe Reference Chadabe1997: 291) – through a performance, referring to an illusion of control. Similarly, with respect to his own interactive work Chadabe (Reference Chadabe1997: 287) describes sharing the control of the music with an interactive system. Schiemer (Reference Schiemer1999: 109–10) refers to an illusion of control, describing his interactive instruments as improvising machines, and compares working with an interactive system to sculpting with soft metal or clay. Sensorband performers working with the Soundnet (Bongers Reference Bongers1998) also set up systems that exist at the edge of control, due no less in part to the extreme physical nature of their interfaces.
1.2. Collaboration
Interactive systems have recently had wide application in the creation of collaborative musical spaces, often with specific focus for non-expert musicians. Blaine’s Jam-O-Drum (Blaine and Perkis Reference Blaine and Perkis2000) was specifically designed to create such a collaborative performance environment for non-expert participants to experience ensemble-based music-making. This notion of the tabletop as a shared collaborative space has proved to be a powerful metaphor, as revealed by projects such as the reacTable (Kaltenbrunner, Jordà, Geiger and Alonso Reference Kaltenbrunner, Jordà, Geiger and Alonso2006), Audiopad (Patten, Recht and Ishii Reference Patten, Recht and Ishii2002) and Composition on the Table (Blaine and Fels Reference Blaine and Fels2003). Interactive systems have also found application providing musical creative experiences for non-expert musicians in computer games such as Iwai’s Electroplankton (Blaine Reference Blaine2006).
1.3. Definitions, classifications and models
The development of a coherent conceptual framework for interactive music systems presents a number of challenges. Interactive music systems are used in many different contexts including installations, networked music ensembles, new instrument designs and collaborations with robotic performers (Eigenfeldt and Kapur Reference Eigenfeldt and Kapur2008). These systems do not define a specific style – that is, the same interactive model can be applied to very different musical contexts.
Critical investigation of interactive works requires extensive cross-disciplinary knowledge in a diverse range of fields including software programming, hardware design, instrument design, composition techniques, sound synthesis and music theory. Furthermore, the structural or formal musical outcomes of interactive systems are invariably not static (i.e., not the same every performance), thus traditional music analysis techniques derived for notated western art music are inappropriate and unhelpful. Not surprisingly, then, the practitioners themselves are the primary source of writing about interactive music systems, typically creating definitions and classifications derived from their own creative practice. Their work is presented here as a foundation for discussions pertaining to the definition, classification and modelling of interactive music systems.
2. Definitions
2.1. Interactive composing
Chadabe has been developing his own interactive music systems since the late 1960s and has written extensively on the subject of composing with interactive computer music systems. In 1981 he proposed the term interactive composing to describe ‘a performance process wherein a performer shares control of the music by interacting with a musical instrument’ (Chadabe Reference Chadabe1997: 293).Footnote 1 Referring to Martirano’s SalMar Construction and his own CEMS System, Chadabe writes of these early examples of interactive instruments:
These instruments were interactive in the same sense that performer and instrument were mutually influential. The performer was influenced by the music produced by the instrument, and the instrument was influenced by the performer’s controls. (Chadabe Reference Chadabe1997: 291)
These systems were programmable and could be performed in real-time. Chadabe highlights that the musical outcome from these interactive composing instruments was a result of the shared control of both the performer and the instrument’s programming, the interaction between the two creating the final musical response.
Programmable interactive computer music systems such as these challenge the traditional clearly delineated western art-music roles of instrument, composer and performer. In interactive music systems the performer can influence, affect and alter the underlying compositional structures, the instrument can take on performer-like qualities, and the evolution of the instrument itself may form the basis of a composition. In all cases the composition itself is realised through the process of interaction between performer and instrument, or machine and machine. In developing interactive works the composer may also need to take on the roles of, for example, instrument designer, programmer and performer. Chadabe writes of this blurring of traditional roles in interactive composition:
When an instrument is configured or built to play one composition, however the details of that composition might change from performance to performance, and when that music is interactively composed while it is being performed, distinctions fade between instrument and music, composer and performer. The instrument is the music. The composer is the performer. (Chadabe Reference Chadabe1997: 291)
This provides a perspective of interactive music systems that focuses on the shared creative aspect of the process in which the computer influences the performer as much as the performer influences the computer. The musical output is created as a direct result of this shared interaction, the results of which are often surprising and not predicted.
2.2. Interactive music systems
Rowe (Reference Rowe1993) in his book Interactive Music Systems presents an image of an interactive music system behaving just as a trained human musician would, listening to ‘musical’ input and responding ‘musically’. He provides the following definition:
Interactive computer music systems are those whose behaviour changes in response to musical input. Such responsiveness allows these systems to participate in live performances, of both notated and improvised music. (Rowe Reference Rowe1993: 1)
In contrast to Chadabe’s perspective of a composer/performer interacting with a computer music system, the combined results of which realise the compositional structures from potentials encoded in the system, Rowe presents an image of a computer music system listening to, and in turn responding to, a performer. The emphasis in Rowe’s definition is on the response of the system; the effect the system has on the human performer is secondary. Furthermore the definition is constrained, placed explicitly within the framework of musical input, improvisation, notated score and performance.
Paine (Reference Paine2002) is also critical of Rowe’s definition with its implicit limits within the language of notated western art music, both improvised and performed, and its inability to encompass systems that are not driven by instrumental performance as input:
The Rowe definition is founded on pre-existing musical practice, i.e. it takes chromatic music practice, focusing on notes, time signatures, rhythms and the like as its foundation; it does not derive from the inherent qualities of the nature of engagement such an ‘interactive’ system may offer. (Paine Reference Paine2002: 296)
Jordà (Reference Jordà2005) questions if there is in fact a general understating of what is meant by Rowe’s concept of ‘musical input’:
How should an input be, in order to be ‘musical’ enough? The trick is that Rowe is implicitly restraining ‘interactive music systems’ to systems which posses the ability to ‘listen’, a point that becomes clearer in the subsequent pages of his book. Therefore, in his definition, ‘musical input’ means simply ‘music input’; as trivial and as restrictive as that! (Jordà 2005: 79)
However, Rowe’s definition should be considered in the context of the music technology landscape of the early 1990s. At this time most of the music software programming environments were MIDI based, with the sonic outcomes typically rendered through the use of external MIDI synthesisers and samplers. Real-time synthesis, although possible, was significantly restricted by processor speed and the cost of computing hardware. Similarly, sensing solutions (both hardware and software) for capturing performance gestures were far less accessible and developed in terms of cost, speed and resolution than are currently available. The morphology of the sound in a MIDI system is largely fixed and so the musical constraints are inherited from instrumental music (i.e., pitch, velocity and duration). Thus the notions of an evolving morphology of sound explored through gestural interpretation and interaction are not intrinsic to the system.
2.3. Composing interactive music
Winkler (Reference Winkler1998) in his book Composing Interactive Music presents a definition of interactive music systems closely aligned with Rowe’s, in which the computer listens to, interprets and then responds to a live human performance. Winkler’s approach is also MIDI based with all the constraints mentioned above. Winkler describes interactive music as:
a music composition or improvisation where software interprets a live performance to affect music generated or modified by computers. Usually this involves a performer playing an instrument while a computer creates music that is in some way shaped by the performance. (Winkler Reference Winkler1998: 4)
As is the case with Rowe’s definition, there is little direct acknowledgment by Winkler of interactive music systems that are not driven by instrumental performance. In discussing the types of input that can be interpreted, the focus is again restricted to event-based parameters such as notes, dynamics, tempo, rhythm and orchestration. Where gesture is mentioned, the examples given are constrained to MIDI controllers (key pressure, foot pedals) and computer mouse input.
Interactive music systems are of course not ‘found objects’, but rather the creation of composers, performers, artists and the like (through a combination of software, hardware and musical design). For a system to respond musically implies a system design that meets the musical aesthetic of the system’s designer(s). For a system to respond conversationally, with both predictable and unpredictable responses, likewise is a process inbuilt into the system. In all of the definitions discussed, to some degree, is the notion that interactive systems require interaction to realise the compositional structures and potentials encoded in the system. To this extent interactive systems make possible a way of composing that at the same time is both performing and improvising.
3. Classifications and Models
3.1. Empirical classifications
One of the simplest approaches for classifying interactive music systems is with respect to the experience afforded by the work. For example, is the system an installation intended to be performed by the general public or is it intended for use by the creator of the system and/or other professional artists? Bongers (Reference Bongers2000: 128) proposes just such an empirically based classification system, identifying the following three categories:Footnote 2
(1) performer with system;
(2) audience with system; and
(3) performer with system with audience.
These three categories capture the broad form and function of an interactive system but do not take into account the underlying algorithms, processes and qualities of the interactions taking place. The performer with system category encompasses works such as Lewis’ Voyager (2000), Waisvisz’s The Hands (Reference Waisvisz1985), Sonami’s Lady’s Glove (Bongers Reference Bongers2000: 134) and Schiemer’s Spectral Dance (1999: 110). The audience with system category includes interactive works designed for gallery installation such as Paine’s Gestation (Reference Paine2007), Gibson and Richards’ Bystander (Richards Reference Richards2006) and Tanaka and Toeplitz’s The Global String (Bongers Reference Bongers2000: 136). Bongers’ third category, performer with system with audience places the interactive system at the centre, with both performer and audience interacting with the system. Examples of this paradigm are less common, but Bongers puts forward his own – The Interactorium (Bongers Reference Bongers1999), developed together with Fabeck and Harris – as an illustration. The Interactorium includes both performers and audience members in the interaction, with the audience seated on chairs equipped with ‘active cushions’ providing ‘tactual’ feedback experiences and sensors so that audience members can interact with the projected sound and visuals and the performers.
To this list of classifications I would add the following two extensions:
(4) multiple performers with a single interactive system; and
(5) multiple systems interacting with each other and/or multiple performers.
Computer interactive networked ensembles such as The Hub (Brown and Bischoff Reference Brown and Bischoff2002), austraLYSIS electroband (Dean Reference Dean2003) and HyperSense Complex (Riddell Reference Riddell2005) are examples of multiple performers with a single interactive system, exploring interactive possibilities quite distinct from the single performer and system paradigm. In a similar manner the separate category for multiple systems interacting encompasses works such as Hess’s Moving Sound Creatures (Chadabe Reference Chadabe1997) for twenty-four independent moving sound robots, which is predicated on evolving inter-robot communication, leading to artificial-life-like development of sonic outcomes.
3.2. Classification dimensions
Developing a framework further than just simply categorising the physical manifestations of interactive systems, Rowe (Reference Rowe1993: 6–7) proposes a ‘rough classification system’ for interactive music systems consisting of a combination of three dimensions –
(1) score-driven vs. performance-driven systems;
(2) transformative, generative or sequenced response methods; and
(3) Instrument vs. player paradigms.
For Rowe, these classification dimensions do not represent distinct classes; instead, a specific interactive system would more than likely encompass some combination of the classification attributes. Furthermore, the dimensions described should be considered as points near the extremes of a continuum of possibilities (Rowe Reference Rowe1993: 6).
3.2.1. Score-Driven vs. Performance-Driven
Score-driven systems have embedded knowledge of the overall predefined compositional structure. A performer’s progress through the composition can be tracked by the system in real-time, accommodating subtle performance variations such as a variation in tempo. Precise, temporally defined events can be triggered and played by the system in synchronisation with the performer, accommodating their performance nuances, interpretations and potential inaccuracies.
A clear example of a score-driven system is demonstrated by score following (Dannenburg Reference Dannenburg1984; Vercoe Reference Vercoe1984)Footnote 3 in which a computer follows a live performer’s progress through a pre-determined score, responding accordingly (figure 1). Examples of score-following works include Lippe’s Music for Clarinet and ISPW (Reference Lippe1993) and Manoury’s Pluton for piano and triggered signal processing events (Puckette and Lippe Reference Puckette and Lippe1992).
Model of a score-following system, adapted from Orio, Lemouton and Schwarz Reference Orio, Lemouton and Schwarz2003.
Score following is, however, more reactive than interactive, with the computer system typically programmed to follow the performer faithfully. Score following can be considered as an intelligent version of the instrument and tape model, in which the performer follows and plays along with a pre-constructed tape (or audio CD) part. Computer-based score-following reverses the paradigm, with the computer following the performer. Although such systems extend the possibilities of the tape model, enabling real-time signal processing of the performer’s instrument, algorithmic transformation and generation of new material, the result from an interactive perspective is much the same, perhaps just easier for the performer to play along with. As Jordà observes,
score-followers constitute a perfect example for intelligent but zero interactive music systems. (Jordà 2005: 85)
A performance-driven system, conversely, has no pre-constructed knowledge of the compositional structure or score and can only respond based on the analysis of what the system hears. Lewis’ Voyager can be considered an example of a performance-driven system, listening to the performer’s improvisation and responding dynamically, both transforming what it hears and responding with its own independent material.
3.2.2. Response type
Rowe’s three response types – transformative, generative or sequenced – classify the way an interactive system responds to its input. Rowe (Reference Rowe1993: 163), moreover, considers that all composition methods can be classified by these three broad classes. The transformative and generative classifications imply an underlying model of algorithmic processing and generation. Transformations can include techniques such as inversion, retrograde, filtering, transposing, filtering, delay, re-synthesis, distortion and granulating. Generative implies the system’s self-creation of responses either independent of, or influenced by, the input. Generative processes can include functions such as random and stochastic selection, chaotic oscillators, chaos-based models and rule-based processes. Artificial-life algorithms offer further possibilities for generative processes, for example flocking algorithms, biology population models and genetic algorithms. Sequenced response is the playback of pre-constructed and stored materials. Sequence playback often incorporates some transformation of the stored material, typically in response to the performance input.
3.2.3. Instrument vs. player
Rowe’s third classification dimension reflects how much like an instrument or another player the interactive system behaves. The instrument paradigm describes interactive systems that function in the same way that a traditional acoustic instrument would, albeit an extended or enhanced instrument. The response of this type of system is predictable, direct and controlled, with a sense that the same performance gestures or musical input would result in the same or at least similar, replicable responses.
The player paradigm describes systems that behave as an independent, virtual performer or improviser, interacting with the human musician, responding with some sense of connection to the human’s performance, but also with a sense of independence and autonomy. Lewis (Reference Lewis2000: 34) defines his Voyager system as an example of a player paradigm, with the system both capable of transformative responses and also able to generate its own independent material. For Lewis, an essential aspect of Voyager’s system design was to create the sense of playing interactively with another performer.
3.3. Multidimensional models
Others have proposed multidimensional spaces to represent interactive systems. Spiegel (Reference Spiegel1992) proposes an open-ended list of some sixteen categories intended to model and represent interactive musical generation. Spiegel considers the representation model an alternative to an Aristotelian taxonomy of interactive computer-based musical creation consisting of ‘finite categories with defined boundaries, usually hierarchical in structure’ (1992: 5). Spiegel’s categories include the system’s mappings, the nature of the interactions and expertise required, how formal musical structure is defined and engaged with, and system responsiveness. Addressing interactive digital musical instruments, Pressing (Reference Pressing1990), Piringer (Reference Piringer2001) and Birnbaum, Fiebrink, Malloch and Wanderley (Reference Birnbaum, Fiebrink, Malloch and Wanderley2005) also propose multidimensional representation spaces. Recurring throughout these representation models are notions of control, required expertise, feedback, expressivity, immersion, degrees of freedom and distribution.
3.4. System responsiveness
The way an interactive music system responds to its input directly affects the perception and the quality of the interaction with the system. A system consistently providing precise and predictable interpretation of gesture to sound would most likely be perceived as reactive rather than interactive, although such a system would function well as an instrument in the traditional sense. Conversely, where there is no perceptible correlation between the input gesture and the resulting sonic outcome, the feel of the system being interactive can be lost, as the relationship between input and response is unclear. It is a balancing act to maintain a sense of connectedness between input and response while also maintaining a sense of independence, freedom and mystery; that the system is in fact interacting not just reacting. A sense of participation and intuition is difficult to achieve in designing interactive systems and each artist and participant will bring their own interpretation of just how connected input and response should be for the system to be considered interactive.
3.5. Interaction as a conversation and other metaphors
Chadabe offers the following three metaphors to describe different approaches to creating real-time interactive computer music (Chadabe Reference Chadabe2005):
(1) Sailing a boat on a windy day and through stormy seas.
(2) The net complexity or the conversational model.
(3) The powerful gesture expander.
The first of these poetic images describes an interactive model in which control of the system is not assured – sailing a boat through stormy seas. In this scenario interactions with the system are not always controlled and precise but instead are subject to internal and/or external disturbances. This effect can be seen in Lewis’s use of randomness and probability in his Voyager system:
the system is designed to avoid the kind of uniformity where the same kind of input routinely leads to the same result. (Lewis Reference Lewis2000: 36)
The second metaphor depicts an interactive system in which the overall complexity of the system is a result of the combined behaviour of the individual components. Just as in a conversation, no one individual is necessarily in control and the combined outcome is greater than the sum of its parts. Examples of this type of system include the work of networked ensembles such as The League of Automatic Composers and The Hub.
A number of artists have drawn comparisons between this model of information exchange presented by a conversation and interactive music systems. Chadabe has used the conversation metaphor previously, describing interacting with his works Solo (Chadabe Reference Chadabe1997: 292) and Ideas of Movement at Bolton Landing (Chadabe Reference Chadabe1997: 287) in both instances as ‘like conversing with a clever friend’. Perkins compares the unknown outcomes of a Hub performance with the surprises inherent in daily conversation (Perkis Reference Perkis1999). Winkler, likewise, makes use of the comparison, noting that conversation, like interaction, is a:
two-way street … two people sharing words and thoughts, both parties engaged. Ideas seem to fly. One thought spontaneously affects the next. (Winkler Reference Winkler1998: 3)
A conversation is a journey from the known to the unknown, undertaken through the exchange of ideas. Paine similarly considers human conversation a useful model for understanding interactive systems, identifying that a conversation is:
• unique and personal to those individuals
• unique to that moment of interaction, varying in accordance with the unfolding dialogue
• maintained within a common understood paradigm (both parties speak the same language, and address the same topic). (Paine Reference Paine2002: 297)
Chadabe’s third metaphor, the powerful gesture expander, defines a deterministic rather than interactive system in which input gestures are re-interpreted into complex musical outputs. This category includes instrument oriented models such as Spiegel (Reference Spiegel1987) and Mathews’ intelligent instruments, Tod Machover’s (Machover and Chung Reference Machover and Chung1989) hyperinstruments and Leonello Tarabella’s (Reference Tarabella2004) exploded instruments.
4. System Anatomy
4.1. Sensing, processing and response
Rowe (Reference Rowe1993: 9) separates the functionality of an interactive system into three consecutive stages – sensing, processing and response (figure 2).
Rowe’s three-stage system model.
In this model the sensing stage collects real-time performance data from the human performer. Input and sensing possibilities include MIDI instruments, pitch and beat detection, custom hardware controllers and sensors to capture the performer’s physical gestures. The processing stage reads and interprets the information sent from the sensing stage. For Rowe, this central processing stage is the heart of the system, executing the underlying algorithms and determining the system’s outputs. The outputs of the processing stage are then sent to the final stage in the processing chain, the response stage. Here the system renders or performs the musical outputs. Possibilities for this final response stage include real-time computer-based software synthesis and sound processing, rendering via external instruments such as synthesisers and samplers, or performance via robotic players.
This three-stage model is certainly concise and conceptually simple. However, Rowe’s distinction between the sensing and processing stages is somewhat blurred. Some degree of processing is needed to perform pitch and beat detection; in other words, it is not simply a passive sensing process. Furthermore the central processing stage encapsulates a significant component of the model and reveals little about the possible internal signal flows and processing possibilities in the system.
4.2. The system model expanded
Winkler (Reference Winkler1998: 6) expands Rowe’s three-stage model (figure 2) of sensing, processing and response into five stages:
(1) Human input, instruments
(2) Computer listening, performance analysis
(3) Interpretation
(4) Computer composition
(5) Sound generation and output, performance.
Figure 3 reveals the similarities between the two models. Winkler’s human input stage is equivalent to Rowe’s sensing stage. This is where the performer’s gestures or instrumental performance, or the actions of other participants, are detected and digitised.
Winkler’s five-stage system model compared to Rowe’s three-stage model.
Winkler separates Rowe’s central processing stage into three parts – computer listening, interpretation and computer composition. The computer listening stage analyses the data received by the sensing stage. Winkler (Reference Winkler1998: 6) defines this computer listening stage as the analysis of ‘musical characteristics’, such as timing, pitch and dynamics. The interpretation stage interprets the data from the previous computer listening process. The results of the interpretation process are then used by the computer composition stage to determine all aspects of the computer’s musical performance. Winkler’s final sound generation or performance stage corresponds to Rowe’s third and final response stage in which the system synthesises, renders or performs the results of the composition process, either internally or externally.
Winkler’s model clarifies the initial sensing stage by separating the process of capturing input data (musical performance, physical gesture, etc.) via hardware sensors from the process of analysing the data. However, the separation of the processing stage into computer listening, interpretation and computer composition is somewhat arbitrary. The exact difference between computer listening and interpretation is unclear. The computer composition stage can conceivably encompass any algorithmic process while providing little insight into the underlying models of the system. Furthermore Winkler’s descriptions of the processing are still constrained as ‘musical’.
4.3. Control and feedback
Focusing on the physical interaction between people and systems, Bongers (Reference Bongers2000: 128) identifies that interaction with a system involves both control and feedback. In both the aforementioned Rowe and Winkler interactive models there is little acknowledgement of potentials for feedback in the system itself or with the actual performers interacting with the system. Bongers outlines the flow of control in an interactive system, starting with the performance gesture, leading to the sonic response from the system and completing the cycle with the system’s feedback to the performer.
Interaction between a human and a system is a two way process: control and feedback. The interaction takes place through an interface (or instrument) which translates real world actions into signals in the virtual domain of the system. These are usually electric signals, often digital as in the case of a computer. The system is controlled by the user, and the system gives feedback to help the user to articulate the control, or feed-forward to actively guide the user. (Bongers Reference Bongers2000: 128)
System-performer feedback is not only provided by the sonic outcome of the interaction, but can include information such as the status of the input sensors and the overall system (via lights, sounds, etc.) and tactile (‘haptic’) feedback from the controller itself (Berdahl, Steiner and Oldham Reference Berdahl, Steiner and Oldham2008). Acoustic instruments typically provide such feedback inherently: for example, the vibrations of a violin string provide feedback to the performer via his or her finger(s) about its current performance state, separate to the pitch and timbral feedback the performer receives acoustically. With interactive computer music systems, the strong link between controller and sound generation, typical of acoustic instruments, is no longer constrained by the physics of the instrument. Virtually any sensor input can be mapped to any aspect of computer-based sound generation. This decoupling of the sound source from the controller can result in a loss of feedback from the system to the performer that would otherwise be intrinsic to an acoustic instrument and as a result can contribute to a sense of restricted control of an interactive system (Bongers Reference Bongers2000: 127).
Figure 4 presents a model of a typical instance of solo performer and interactive music system, focusing on the interactive loop between human and computer. The computer system senses the performance gestures via its sensors, converting physical energy into electrical energy. Different sensors are used to capture different types of information – kinetic energy (movement), light, sound or electromagnetic fields, to name a few. The actuators provide the system’s output – loudspeakers produce sound, video displays output images, motors and servos provide physical feedback. The sensors and actuators are defined as the system’s transducers, enabling the system to communicate with the outside world.
Solo performer and interactive system – control and feedback, adapted from Bongers Reference Bongers2000.
Similarly, the human participant in the interaction can be defined as having corresponding senses and effectors. The performer’s senses (inputs) are their ability to see, hear, feel and smell, while the performer’s effectors (outputs) are represented by muscle action, breath, speech and bio-electricity. For artists such as Stelarc, the separation between human and machine interface becomes extremely minimal, with both machine actuators and sensors connected to his own body, leading to the concept of Cybernetic Organisms or Cyborgs. For example, Ping Body (Stelarc 1996) allowed participants using a website to remotely access, view and actuate Stelarc’s body via a computer-interfaced muscle-stimulation system.
4.4. Mapping
Connecting gestures to processing and processing to response are the mappings of the system. In the specific context of a digital musical instrument (Miranda and Wanderley Reference Miranda and Wanderley2006: 3), mapping defines the connections between the outputs of a gestural controller and the inputs of a sound generator. Figure 5 depicts a typical and often cited example of such a system (Wanderley Reference Wanderley2001).
Mapping in the context of a digital musical instrument (Miranda and Wanderley Reference Miranda and Wanderley2006).
In this model a performer interacts with a gestural controller’s interface, their input gestures mapped from the gestural controller’s outputs to various sound generating control parameters. While a performer may be described as interacting with the gestural controller in such a system, the digital musical instruments represented by the model are intended to be performed (and thus controlled) as an instrument and consequently function as reactive, rather than interactive, systems.
In the context of an interactive music system, mappings are made between all stages of the system, connecting sensing outputs with processing inputs and likewise processing outputs with response inputs. Furthermore, the connections made between the different internal processing functions can also be considered as part of the mapping schema. Mappings can be described with respect to the way in which connections are routed, interconnected and interrelated. Mapping relationships commonly employed in the context of digital musical instruments and interactive music systems are (Hunt and Kirk Reference Hunt and Kirk2000; Miranda and Wanderley Reference Miranda and Wanderley2006: 17):
• one-to-one
• one-to-many
• many-to-one
• many-to-many.
One-to-one is the direct connection of an output to an input, for example a slider mapped to control the pitch of an oscillator. Many inputs can be mapped individually to control many separate synthesis parameters; however, as the number of multiple one-to-one mappings increases, systems become more difficult to perform effectively. One-to-many connects a single output to multiple inputs – for example, a single gestural input can be made to control multiple synthesis parameters at the same time. One-to-many mappings can solve many of the performance interface problems created by multiple one-to-one mappings. Many-to-one mappings, also referred to as convergent mapping (Hunt and Kirk Reference Hunt and Kirk2000: 7), combine two or more outputs to control one input, for example a single synthesis parameter under the control of multiple inputs. Many-to-many is a combination of the different mapping types (Lazzetta Reference Lazzetta2000).
4.5. Separating the interface from the sound generator
Mapping arbitrary interfaces to likewise arbitrarily chosen sound-generating devices creates the potential for the interrelated physical and acoustical connections between an instrument’s interface and its sound output – which are typically inherent in traditional acoustic instruments – to be lost. For traditional acoustic instruments the sound-generating process dictates the instrument’s design. The method of performing the instrument – blowing, bowing, striking – is inseparably linked to the sound-generating process – wind, string, membrane. In the case of electronic instruments this relationship between performance interface and sound production is no longer constrained in this manner (Bongers Reference Bongers2000: 126). Sensing technology and networked communication methods such as Open Sound Control (Wright, Freed and Momeni Reference Wright, Freed and Momeni2003) allow virtually any input from the real world to be used as a control signal for use with digital media. The challenge facing the designers of interactive instruments and sound installations is to create convincing mapping metaphors, balancing responsiveness, control and repeatability with variability, complexity and the serendipitous.
5. Summary
This article has discussed the differing approaches taken to the definition, classification and modelling of interactive music systems encompassing both historical and contemporary practice. The interactive compositional possibilities explored by early practitioners still resonate today, for example – the concept of shared control, intelligent instruments, collaborative conversational environments, and the blurring of the distinctions between instrument building, performance, improvisation and composition. The term interactive is applied widely in the field of new media arts, from systems exploiting relatively straightforward reactive mappings of input-to-sonification through to highly complex systems that are capable of learning and can behave in autonomous, organic and intuitive ways. There has also been a recent focus on describing interactive systems in terms of digital musical instruments, concentrating on mappings between gestural input and sonification. However, interactive systems can also be thought of in terms of interactive composition, collaborative environments and conversational models.
Interactive systems enable compositional structures to be realised through performance and improvisation, with the composition encoded in the system as processes and algorithms, mappings and synthesis routines. In this way all aspects of the composition – pitch, rhythm, timbre, form – have the potential to be derived through an integrated and coherent process, realised through interacting with the system. The performance becomes an act of selecting potentials and responding to evolving relationships. The process of composition then becomes distributed between the decisions made during system development and those made in the moment of the performance. There is no pre-ordained work, simply a process of creation, shared with the public in performance.
