1. Introduction
Nothing dates faster than people’s fantasies about the future. This is what you get when perfectly decent, intelligent, and talented men [sic] start thinking in terms of space rather than place; and single rather than multiple meanings. It’s what you get when you design for political aspirations rather than real human needs. (Hughes Reference Hughes1980)
1.1. Preamble: speculation limitations
This twentieth anniversary edition of Organised Sound is an opportunity to reflect on an aspect of electroacoustic music practice, and where it might best explore next. In doing this, I am mindful of Robert Hughes’ view of the building of the city of Brasília, but also of how far things can evolve in unexpected directions consequently. Like any conjecture, mine is time bound and can be quickly undermined by subsequent inventions and events. And like any speculation, the further out on a limb it goes, the more divorced it becomes from the comforts of evidence.
I start from the western secular assumption that musical and sound-making activity is a reflection and part of how people, who inhabit both bodies and complex environments, conceive, respond to and interpret their world through particular tools/techniques. Music and sound art are a means to systematically imagine, explore and/or impose order and significance on experiences: past, present or predicted. Guido of Arezzo’s notation system, Bach’s equal temperament and Smalley’s notion of spectromorphology provide examples of this.
Exploring any changing physical, technological or social environment provides people with an opportunity to see where music/sound-making activity fits in. And many early experiments start from applying known techniques to new situations. With new musical instruments, for example, the electric guitarist first copying saxophone solo lines, or fortepiano compositions first replicating older harpsichord practices, demonstrate this. Yet each new exploration may eventually make unique contributions.
In experimenting to find a ‘best fit’ we often overshoot or underestimate through trial and error. Exploration is a fine balancing act – particularly between the need to experiment and the need to engage interest and participation in sonic outputs. Experimental and communicative concerns rarely match up, and often only time tells what was important from experiments, or where a practice might find popular engagement. With this limitation in mind, my concern in this paper is largely focused on experimental issues, while some aspects of communicative concerns are taken into account.
1.2. eResearch: trends, projection and focus
Developments in electroacoustic music cannot be separated from technological developments generally, and this two-way relationship through ‘where things fit’ experiments is on going. A summary of aspects of this relationship is found in the recent Oxford Handbook of Computer Music (Dean Reference Dean2009), for example.
Of the many current grand themes in eResearch apart from electroacoustic music, four are of most concern to telematic electroacoustic music. These include i) connectivity and the increase in bandwidth, ii) processing power in areas like high-performance computing (HPC) and smart/embedded technology, iii) artificial intelligence and automated decision making, and iv) data streams and how to make knowledge out of information.
If we anticipate these eResearch trends continuing, it is likely there will be increasing convergence between them. So my focus here is on how Internet2 music-making might take part in this convergence over at least the next five years or so. Given limited space, my view is presented in very board brushstrokes.
1.3. Internet2 music and conjecture
With the development of high-speed Internet2, it was little surprise that electroacoustic music practice first adopted and built on older LAN-centric approaches in its early explorations. Extending the field more recently, as music studios and traditional concert venues have become more globally connected through high-speed broadband, telematic electroacoustic music practice has rapidly increased. It includes using multiple bi-directional video, audio and graphics channels, intelligent-agent performers and integrated Open Sound Control (OSC) data control allowing for real-time electroacoustic music composition/performance synchronously within the limits of network latency (see Fields and Whalley Reference Fields and Whalley2012).
To further develop the practice, however, we need to see ‘where it fits’ beyond copying or simply better facilitating older practices. My argument here is that what is unique about the telematic medium, beyond the creative use of network latency or many conventional electroacoustic music practices, largely remains to be explored artistically.
My suggestion to address this is through telematic electroacoustic music’s exploration of affective sonic art practice, and its further integration with intelligent-agent/decision-making technology when amalgamated with complex data streams such as the Internet of Things (IOT). And I argue that we can make sense out of these data streams through affective data sonification made possible by machine learning. Through doing this we can explore our place in increasingly complex networks of multiple relationships that typify contemporary life.
2. Some limitations in networked music theory/practice
As noted, many current real-time telematic electroacoustic works replicate what one could do if players/machines were in the same space – but, by adding remote connections, they facilitate what practitioners could do with sufficient time, equipment and funds in one place, however improbable this might be. Consequently, much of the initial discourse in telematic music has centred on notions of what space, live performance and the extended network add to the practice.
In a recent paper mapping a decision arena of telematic electroacoustic music from an artistic perspective, I noted that the primary concerns are about the means/methods of production, the nature of relationships between people and people/machines, and the amalgamating of sonic languages when blending diverse music traditions (Whalley Reference Whalley2012). This extended a prior overview of the role of intelligent-agent software in real-time performance and composition that could be used in telematic work (Whalley Reference Whalley2009).
In putting forward a schema of the telematic electroacoustic music field (Whalley Reference Whalley2012), a limitation was in not considering an affective dimension of electroacoustic music, although it was covered in the earlier survey on intelligent-agents software. Moreover, the survey I undertook in 2012 did not consider the integration of the telematic medium with real-time data streams apart from machine/human players, and particularly data streams from non-natural environments.
In addition to these surveys and although Internet2 music-making has made increasing progress over the last decade, there are limitations in the practice when judged against some of the common theoretical paradigms in the field. Further, there are limitations in these paradigms themselves that need addressing to expand the practice.
2.1. Traditional distribution modes
Much telematic electroacoustic music is still dominated by traditional music performance modes of delivery in making works that are time-bound and require linked visual performers to fulfil concert programmes for physical audiences at networked venues. Yet this reflects a small part of the possibilities of the field. Follmer (Reference Follmer2005), for example, sets out a three-dimensional matrix to show the potential scope of NMP (networked music practice). It includes interplay with network characteristics (or how the structural characteristics of the network shape the work), interactivity and openness (or the extent that users/audiences can join in with and influence outcomes), and complexity and flexibility (or how musically/sonically flexible the outcome can be). Accordingly, his suggestion is that NMP is best suited to hypermusic (new hybrid instruments), sound installations combining real and virtual space, and algorithmic/generative installations.
Hybrid instruments controlled by distributed input and sound object manipulation are an area that has received a good deal of attention in practice and in the literature as people look to build new ‘instruments’ suited to this new environment. But other areas remain to be extensively explored. In addition, a limitation of Follmer’s analysis and recent practice is in not considering a wider range of means by which network space can be connected – or what can be plugged in to network space to take part in creative works, like non-performance-based data streams such as Twitter feeds.
A further limitation in many current electroacoustic telematic works is that they largely centre on Weinberg’s (Reference Weinberg2005) notion of musical structure or composer-controlled goal approaches, where participants (agents and/or humans) fill prescribed musical or performance outcomes. This is in contrast to his notion of processes-based approaches that explore possibilities or fulfil goals through collaboration or competition and where user experiences might differ with sessions.
Given these theoretical overviews, the practice of rendering telematic electroacoustic music into concert hall formats seems somewhat unsuited to exploring many of NMP’s wider possibilities. As performances are able to be streamed live to worldwide audiences through digital broadcast, for example, the NMP medium may be better rendered as Internet only, with works played over longer periods, with multiple participants (real and virtual), and through structures that are more flexible.
2.2. Relationships: theory and practice
In current telematic electroacoustic music practice, many relationship models of production are still grounded in older methods of composer/performer chains of implementation, or projected dramaturgy where one node acts as the author and other nodes contribute or project to it (Rebelo Reference Rebelo2009). Less common is his notion of directed dramaturgy where authorship remains with a group or individual who takes on a director’s role only – or distributed dramaturgy where the authorship is retained by each node but they contribute expertise and content to shared production, giving more flexible outcomes. Yet Rebelo’s model as a schema of relationships is also limiting as it does not accommodate real-time aggregates of collective behaviour, or information that might become useful parts of works beyond current human and/or machine players.
Finally, receiving attention in networked music endeavours, given non-trivial machines and OSC communication between individuals/machines and groups through Internet2, is the role of intelligent–agent technology in composition/performance (Whalley Reference Whalley2009). Some approaches include: a multi-agent system that is self-contained (generative model), or one that can include external input (machine or human) to make a generative improvisational model using intelligent machines. Yet in terms of a wider relationship model, the theoretical focus in this work was largely on machine/machine and machine/people relationships. A limitation in the schema going forward in terms of telematic electroacoustic music is that machines also need to be considered as interacting with external and non-human data streams, rather than just be extensions of existing composition/performance practices, or partners with human players at or between telematic nodes.
2.3. Coordinate and cooperate, and eResearch
A significant change in contemporary life facilitated by the application of digital technology, is a shift from command and control paradigms to coordinate and cooperate ones (Friedman Reference Friedman2007). Associated with this is a change in emphasis to one’s connections within a network of relationships and with multiple non-linear relationships, rather than in a hierarchy of command lines.
From a networked electroacoustic music practice perspective, one consequence of this is that communicative issues such as the phenomenology of sound making and reception, music/sound as a means to explore real/virtual space, and more flexible social interaction in music-making become more prevalent. Logistically, interruptions as part of creative processes, varying time lengths for works, diverse controllers, scattered time zones and the need to combine real and virtual spaces are also things to increasingly consider.
The question then, particularly with the need to also accommodate developments in eResearch that will further enhance coordinate and cooperate paradigms, is how to cope with this complexity in NMP. In addressing an aspect of this here I largely set aside HPC/embedded computing for the sake of brevity, and concentrate on increases in artificial intelligence, affective computing, and how to make knowledge out of increasing information presented in data streams.
3. Data streams and the Internet Of Things
3.1. More complex relationships
It should be acknowledged that Internet2 music-making has added to the number of people we could make music together with from remote locations by making complex ensembles drawing on specialists at each node. And using machine-based intelligent-agents at singular or distributed network nodes has also increased the complexity of relationships between human/machine and machine/machine players. The same could be said of deploying network distributed and controlled digital instruments.
Yet independently of NMP practice, we are often embedded in multiple real-time data streams from non-environmental sources such as political news and financial barometers as part of contemporary life, presenting a wider view of possible NMP relationships.
Metaphorically, people are often receptive ‘players’ in data streams, and part of something of a collective limbic system as a consequence. While these streams may often be rendered visually or through words, we constantly respond to them affectively through interpreting collective moods and kinetic gestures. And crowd reactions reflected in data streams like sentiment indicators with their associated feedback loops are likely to be influenced by affective concerns as much as reasoned thinking.
A broader consideration of networked life may then be a better ‘natural’ environment for telematic electroacoustic music practice, reflecting a contemporary aesthetic, particularly if it explores the medium in an affective way. An ongoing research question is then how we might incorporate data streams in NMP, or extend electroacoustic musicianship to make these streams part of our artistic landscape.
3.2. Internet of Things
Extending current data streams, real-time information feeds are likely to rapidly increase with the expansion of the Internet of Things (IoT). This is made possible by increasing network bandwidth and the expansion of IPv6 addresses. While we may presently think of the Internet as connecting people through desktops, laptops, phones and so on, the Internet of Things will significantly change this idea over the next decade or so. Gartner (2013), for example, projects that there will be 26 billion devices on IoT by 2020, and ABI Research (2013) projects 30 billion devices will be wirelessly connected by 2020.
Beyond IoT as just tagging objects that plug into the Internet allowing tracking based on machine-to-machine communication (M2M), the medium will expand into far wider applications and domains if current developments are anything of a future guide. This will involve significant increases in embedded technology through miniaturisation, associated smart computing, and connectivity allowing widespread data collection and monitoring (Höller, Tsiatsis, Mulligan, Karnouskos, Avesand and Boyle Reference Höller, Tsiatsis, Mulligan, Karnouskos, Avesand and Boyle2014). This enhances the possibilities of distributed ‘intelligence’, were embedded technology facilitates autonomous actors to reduce complexity, and where parallel computing can distribute the processing load.
Our human-focused view of time/space/location in relationship to networks is then likely to become secondary to information-centric networks that interface with human needs and networks inhabited by ubiquitous intelligent machines that can interact with people and other machines.
Accordingly, we will need to rapidly adjust our sense of place in contemporary networked life. Furthermore, we will need to make sense of more complex sets of relationships. And a most pressing concern from a human-centric perspective is how we will deal with the increasing amounts of information that is produced within networks.
4. Intelligent Machines And Automation
All this of course first raises the question of what ‘intelligent’ machines actually are.
In the popular consciousness, machine intelligence is often conceived of in comparison to people, and robotics is a main focus. Science fiction films like I Robot, Bladerunner and Terminator are examples. Yet in the immediate future such multi-tasking, cross-domain, multi-modal, parallel-processing and self-reflecting machines seem to be some way off. The concept of sentient artificial entities that self-program to set their own goals or sub-goals requires a significant leap in machine imagination. And questions will always remain as to its desirability.
4.1. Machine intelligence and automation
Apart from this, there is little doubt of the impact of artificial intelligence in algorithmic decision-making and pattern recognition. I deal with decision-making first here, and then look at pattern recognition when discussing machine learning in a later section.
Currently, algorithms for automated decision-making are a common aspect of networked life. For example, in finance, high-frequency trading (HFT) accounted for a quarter of all trades in public stock markets in 2005, and by 2008 this had risen to 65 per cent (Lewis Reference Lewis2014: 107). Over 2006 to 2008, the HFT share of US stock-market trading doubled from 26 to 52 per cent and has been about 50 per cent since. And on increasing automation generally, The Economist (2014), for example, recently noted when discussing recent changes in higher education that 47 per cent of current occupations are likely to be automated over the next few decades. Decision-making technology, often based in intelligent-agent technology, is increasingly deployed in many mission critical areas of everyday life. We can then at least predict automated decision-making by algorithms continuing and expanding, and intelligent-agent technology to be increasingly participatory, across many fields.
Decision-making technology is also increasingly used in computer music/electroacoustic performance and composition. For example, I have previously put forward a theoretical framework for the use of intelligent-agent technology in creating music and sound art works, and an overview of common approaches (Whalley Reference Whalley2009). Furthermore, over the last few years, the field has also expanded, as illustrated in the Musical Metacreation (MUME) Conference.
5. Affective Data Sonification
The question is then how we make sense of increasing data streams from a sonic perspective. This is where affective sonification, translating complex information streams into something understandable in human terms, is worthwhile to explore. Well ahead of computing machine intelligence, we can use and extend many people’s innate parallel processing capabilities, their sonic musicianship, and their ability to process affective information, to help with this.
5.1. Affective production
Sound-based art facilitates experiences that are difficult to replicate by other means, and people can discern patterns and relationships in sound that are sometimes difficult to realise visually. In a parallel example in speech, words may convey one sense of idea, but the tone of delivery provides additional and complex affective and gestural information.
Recently, and particularly in the music information retrieval community (ISMIR), affective computing has become widespread as a topic of interest, at least from a reception perspective based on tonal music. Much of this work augments expanding interest in affective computing generally emerging in the last decade or so (Calvo, D’Mello, Gratch and Kappas Reference Calvo, D’Mello, Gratch and Kappas2014). The field of music and emotion is also one that has also been explored extensively over a number of years (see Juslin and Sloboda Reference Juslin and Sloboda2001, for example), as have musical gestures encompassing sound, music and meaning (Godøy and Leman Reference Godøy and Leman2010).
Affective concerns in electroacoustic music have also been explored recently (Whalley Reference Whalley2009; Kendal Reference Kendall2014; Wanderley and Winters Reference Wanderley and Winters2014). In attempting to extend intelligent-agent deployment in sound art, for example, I surveyed affective computing and music literature and put this forward toward exploring a new hybrid model that integrated non-linear, generative and conversational perspectives on interactivity (Whalley Reference Whalley2009). Further, Kendal (Reference Kendall2014) recently developed a detailed schema in The Feeling Blend: Feeling and Emotion in Electroacoustic Art. Here, he widely considers emotions and puts forward the concept of mental layers in support of the multi-levelled nature of meaning where experience is felt and comprehended. He goes on to extend blend theory by Fauconnier and Turner (Reference Fauconnier and Turner2002) in developing the notion of ‘feeling blend’, covering mental space and things that trigger a blend in relationship to some electroacoustic music examples. Kendal also notes that feeling blend is both an intrinsic part of human experience and necessary to understanding artistic practice.
In terms of implementing an affective and gestural schema in telematic electroacoustic music production, I recently developed GNMISS – a Graphic Networked Music Interactive Scoring System (Whalley Reference Whalley2014). This is based on human performers’ perspectives. It allows distributed co-improvisers, mainly using a directed dramaturgy method in early tests of the system, to communicate affectively. GNMISS has four visual layers that illustrate the dramatic structure of works, each represented on a circle with parts for each ‘player’ to follow. One layer maps emotions to colours based on associated words as a primary basis for gesture and timbre representation. This is based on Plutchik’s (Reference Plutchik2001) circumplex scheme of emotions that collates a range of prior studies, and is a model often used in affective computing. A second score layer gives sonic motives and the general frequency range for participants to follow. A third layer allows for more detailed indications of gesture and sound archetypes through representative symbols that were purposed design for the system, to signify types of sound gestures to be played over the emotion line by each participant. A score layer in the middle of the circle represents key centres for any spectral information that a player might need to adjust to.
Apart from affective rendering, this facilitates process-based methods of composition, as net-distributed scores can be built by a composer or by a team (players or otherwise) in real time between digital partners. And it can also be used to break down the division between audiences and performers, as people might take part in the narrative in some sections, but be recipients in another.
Affective sonification in GNMISS relies on the practical musicianship and real-time control that sound-makers and performers bring to renditions of the score, much as one film-music composer might interpret a mood differently from another, but within a similar broad remit. Renditions of a score can then have different outcomes depending on participants assembled for a performance, their interpretation of score parts and their interaction with other players’ interpretations of parts.
6. Data sonification and affective rendering
The GNMISS approach presents the possibility of how external narratives in data streams might be included in, and influence, the interactive affective dialogue of players, and how real-time data-streams as amalgams of crowd behaviours, or collective summaries of information, might be sonified in a meaningful way to extend our networked embedded experiences.
The link is that data streams are often affectively interpreted through embodied cognition collectively, similar to how music and sonic art interpretation can be explored and produced by individuals. This provides the meeting point for telematic electroacoustic music and real-time data streams. There are two areas that need to be addressed to explore this: data sonification generally and collective affective mapping of this.
6.1. Sonification: approach and problems
Aspects of the recent scope of data sonification are well covered in recent literature such as The Sonification Handbook (Hermann, Hunt and Neuhoff Reference Hermann, Hunt and Neuhoff2011), the International Conference for Auditory Display (ICAD), and a recent edition of Organised Sound (Schedel and Worrall Reference Schedel and Worrall2014).
Vickers and Hogg (Reference Vickers and Hogg2006) provide an overview of the aesthetic issues of the field: particularly the relationships between sonification (ars informatica) and music and sound art (ars musica), as well as abstract and concrete approaches to data sonification. They represent this schema on a circular diagram to show the continuity between these four concepts.
From this, a suitable to solution of sonifying most data streams, in the same way that GNMISS has, points to an ars infomatica approach but one that leans toward the metaphorical mapping of data – particularly because it can incorporate an embodied sense of data streams.
In terms of sonifying data streams generally as one example, financial information has received ongoing attention in the literature. Worrall (Reference Worrall2009) gives a summary of approaches to doing this, and the difficulties in terms of direct mapping because these markets do not resonate according to acoustic laws. His proposed solution is transforming the data so that it resonates, but the approach is not well suited to the metaphorical translation needed to address affective states.
A significant issue to address in sonification is how we embody data to make sense of it in human terms. Here, Grossman (Reference Grossman2010) puts forward a productive starting point regarding sonification used for knowledge generation as an extension of auditory sensibility toward previously imperceptible properties of our environment. Accordingly, and based on the nature of our bodies that are mediators between a shared exterior and individual interior world, he suggests a model that covers three problematic areas of bodily extension: the cognitive, the physical and the extended. The value of what is proposed is that it makes thinking in embodied terms essential to sonification, central to the notion of both individual and group embodiment of data and to the relationship between these two.
In these terms, bringing collective electroacoustic musicianship to the interpretation of data streams is essential to convey cognitive, gestural and affective outcomes. Accepted here is that notions about embodiment generally are subject to ongoing debate (Stapleton Reference Stapleton2011).
Reinforcing this and associated with it, is addressing mapping input to sonic outputs that is an ongoing concern in the field. The need for subtle mapping that will make sense in an embodied way to audiences is also a continuing concern in the literature. Worrall (Reference Worrall2011), for example, puts forward a method called a gesture-encoded sound model (GESM) toward synthesiser parameters being mapped to data in a way that makes sense to general listeners, influenced by performing musicians’ input to aid this mapping.
Extending this, Worrall (Reference Worrall2013) again notes the need to understand micro-gestural inflections in parameter mapping of sonifications, arguing that many computer music software tools used for data sonification are unsuited to the task, due to their being based on adaptions of computer music tools used for playing notated scores. And he notes that, as recent research suggests (Godøy and Leman Reference Godøy and Leman2010), listening is embodied and invisibly enacts internal gestures accordingly (Godøy Reference Godøy2006, Reference Godøy2010), so sonification using parametric mapping should incorporate micro-gestural inflections to enhance the intelligibility of aural translation from the data.
Electroacoustic musicianship then has a central role to play in this data sonification. And in tandem with this, in extending proprioceptive interfaces as part of the rendering process (Worrall Reference Worrall2013), careful consideration of mapping physical gestures through haptic interfaces to sound is also needed.
6.2. Affective mapping of data streams
In terms of the affective mapping of data streams more directly, considered here as an extension of embodied musicianship as illustrated in the GNMISS system, two recent articles have begun to address the area generally.
First Kirke and Miranda (Reference Kirke and Miranda2012), again using a capital market data stream as example, put forward a system that utilises ‘melodies’ representing affective states, noting that affective computing is an increasing part of artificial intelligence developments, and how their work aligns with the progress of human–computer interaction work looking at the comprehension of affective outputs to users/engineers. The leap made here is in connecting affective computing with metaphorical data representation (mood) in real time, although using a simpler map of emotions than that adopted by GNMISS. A limitation here is in not embedding the sonic output in performer-based experience, its traditional notation basis, and in the small range of sonic representations of affective outputs in comparison to Plutchik’s emotion schema.
A recently released paper providing a systematic way of evaluating the relationship between sonification and emotional representation is Wanderley and Winters’ (Reference Wanderley and Winters2014) ‘Sonification of Emotion: Strategies and Results from the Intersection with Music’ – again in line with affective computing developments. After identifying favourable contexts for the auditory display of emotion and surveying the state of the research field, they reinforce the conditions needed for sound to meet the requirements of sonified emotions. Accordingly, strategies for display are given that use acoustic and structural cues intended to trigger selected auditory-cognitive mechanisms of musical emotion. In applying the outcome to sonification to convey a selected range of emotions – using one approach designed ecologically and another computationally and then evaluating the outcome – they illustrate that while the computation design performs better, an ecological design is better suited to emotional communication. They then point to developments of computational design/evaluation for future work.
This seminal work provides a means of more accurately linking sonic production to intended emotional response in artistic work that been articulated intuitively by musicians using GNMISS, and their ongoing work may provide a more systematic basis for translating collective aspects of any real-time data stream aurally in ways that will make emotional sense to wider audiences.
7. Integration: Machine Learning, Mapping, Automation
Of course, much remains yet to be done to bring this together. What is missing in the literature is a comprehensive map of electroacoustic music to emotion parameters, a means of mapping micro-gestures to expand the subtlety, and a means to automate both the reading of increasing data streams and the relationship between these streams. In looking at possible ways to do this, my conjecture here becomes increasingly speculative.
7.1. Machine learning
Assuming that work mapping electroacoustic music to affectively meaningful sonic parameters collectively is further explored, I first concentrate on ways of reducing the complexity of the process.
Independently of electroacoustic music, machine learning as a technique is currently used to address translating vast data sets and real-time data streams into accessible human knowledge – WEKA (Witten, Eibe and Hall Reference Witten, Eibe and Hall2011), for example, is one of the most widely used tools for this, and has been applied across a number of disciplines.
Translating data-stream responses into sensible gestural/affective parameters may best be undertaken at a macro-level first by using a variety of electroacoustic music real-time players applying diverse sound objects. Their gestures can be grasped through machine learning, and player efforts checked against the further development of sonic emotional schema such as that of Wanderley and Winters (Reference Wanderley and Winters2014). The collective efforts of many players may then yield a common electroacoustic music database of responses to interpreting data streams – a basis for ongoing mapping based in the phenomenology of electroacoustic music production and reception.
Once broader gestures have been established, we might address more subtle sonic performance mapping from performer-embodied interpretations of data.
7.2. Micro-gesture and micro-sound
Parameter mapping of micro-gestures to micro-sounds needs rapid dynamic multi-dimensional non-linear connections to be made. Yet many of these connections may be beyond what a single instrumental player might be able to achieve in performance. It requires building a complex relationship between inputs and outputs that are as subtle and dynamic as developing a new language that is fit for purpose. And like any sustainable language, it is likely to be collectively discovered, built and owned.
Key here is the mapping of micro-performance gestures to micro-sound in real time – and machine learning can again be used to learn the dynamic connections that are needed here (Fiebrink, Trueman and Cook Reference Fiebrink, Trueman and Cook2009). It is made possible also because through digital technology, we can capture and control a wide range of rapidly changing small parameters that are often typical in electroacoustic music performances.
Finally, micro-sound (Hofer Reference Hofer2014) provides a means to cope with the subtlety of language needed for the required sonic output, and is an area that electroacoustic music practice is well advanced in. Micro-sound sees sound as having a mass or matter found in much twentieth-century experimental music composition. Contemporary digitally based micro-sound as a language is also increasingly shared between sound art and experimental electronica, so provides the basis of the beginnings of adopting a wider language that is commonly understood.
7.3. Automation
As the basis of affective mapping to sound is increasingly comprehended, the mapping process is likely to be automated over the next five years or so. With ubiquitous machines that can parallel process and intelligent agents that communicate and machine learn, we may see the development of sonic decision-making technology deployed with similar capacity to that which is currently found in financial instruments’ HFT.
But, as important, by reducing single streams to affective knowledge to make sense of complexity through sound, the same approach might be used to examine the relationship between multiple affective data streams. And in doing this, we have a way for performers, including people and machines, to join in and make networked-based performances and compositions.
8. Conclusion
I began by noting some of the limitations of current telematic electroacoustic music work and also attempting to accommodate aspects of recent trends in eResearch into its development.
The scenario put forward here in response to extend the practice tries to accommodate changing human needs in a non-linear, interactive and networked world. This will include more flexible relationships and more complex meanings for participants.
With our environment being increasingly mediated by networked data streams, both slow and fast moving, the relationships between people and the Internet of aggregated information warrant further exploration. Building on current work and aided by intelligent machines, affective sonification of these streams and the relationships between them provides a way for electroacoustic music practice to navigate and make sense of aspects of networked life. And it is this experience, beyond analytic means (see Wolfe Reference Wolfe2014), that may provide more integrated experiences of contemporary life.
In doing this we may also find new patterns in data that are only possible from an aural perspective by turning complex networked information into useful knowledge, while also continuing to inhabit bodies and local physical environments as telematic participants. And exploring this multi-dimensional activity may allow us to reimagine our wider changing networked presence.
