Sonification can be used for purely scientific purposes, or as the basis for musical composition. There is a continuum between auditory display, which attempts to faithfully reproduce data in audio, and composition in which creativity can be king. In each case, aesthetic decisions must be made in order to translate data into the auditory domain – either to bring out or hide a quality in the data, or to massage the sound of the data in order to fit musical goals. One early article on aesthetics (Vickers and Hogg Reference Vickers and Hogg2006) was presented at the 2006 International Conference on Auditory Display (ICAD); Vickers and Hogg posited that attention to aesthetics facilitated comprehension of the data. Yet most publications on sonification have focused on why sonification works or how the mapping from data to sound was performed. While the techniques and methodologies of sonification have been explored in depth, this themed issue of Organised Sound aims to probe and document musical practices and innovations in the aesthetics of sonification. How can electroacoustic music studies advance this interdisciplinary field? Sonification is not limited to electroacoustic music, and in fact sound has played a role in scientific inquiry for centuries, from Pythagoras to Galileo (Drake Reference Drake1980). However, electroacoustic music has a natural connection to science and computation; our authors explore this topic from personal, historical and theoretical approaches.
The issue opens with an article co-authored by a name familiar to most readers of Organised Sound, Natasha Barrett. Barrett has composed several works using scientific data, but she wanted to couple her music to the scientific process more tightly. ‘Aftershock: A science–art collaboration through sonification’ details the results of her collaboration from 2009 to 2012 with geoscientist Karen Mair at the University of Oslo's Centre for the Physics of Geological Processes (PGP). After taking an auditory display approach, the pair decided to focus on musical goals. Specifically they used data reduction to highlight interesting emerging features, refined the scale ranges, and explored new mapping rules. Yet the authors were aware that, ‘In reality, many results from the initial sonification work were already coloured by artist choices to explore musical expressions of process, form and internal structure’. This kind of subtle reasoning is exactly what we were hoping our authors would contribute to this issue, increasing awareness of the aesthetic choices made when sonifying data.
Volker Straebel and Wilm Thoben also tease out the subtleties in sonification by writing about the ‘chain of decisions, operations and technical devices that … constitute the technique of sonification’ in Alvin Lucier's Music for Solo Performer (1965). Most critics mark this work as the proper beginning of Lucier's compositional career. To create the piece, Lucier attached electrodes to his scalp in order to detect brain waves; as the amplified waves flowed through speakers, percussion instruments were physically resonated. Straebel and Thoben's ‘Alvin Lucier's Music for Solo Performer: Experimental music beyond sonification’ fills a gap which exists between the discussion of sonification and conceptual music. They expose the myth of Lucier's direct sonification of brain waves through a detailed historical and technical analysis which will surely become the seminal article on this important piece.
Another musical genre where the discussion of sonification has been sorely lacking is rock and roll. Mark Ballora begins to rectify this in his article ‘Sonification, Science and Popular Music: In search of the “wow”’. Detailing his collaboration with the musicologist and Grateful Dead drummer Mickey Hart, for the Berkeley Center for Cosmological Physics, his article describes their work Rhythms of the Universe”: an audio and video montage that speculates on humankind's innate desire to understand the cosmos. Ballora describes the techniques he used to sonify Schumann Resonances, the movement of planets in our solar system, earthquakes and black holes. In addition to the multi-media presentation these sonifications also go into Hart's sound library, from which he draws during live performances. Ballora clearly articulates his aesthetic decisions based on the mission of entertainment and scientific outreach.
Florian Grond and Thomas Hermann also develop recommendations for sonification in relation to different design purposes. ‘Interactive Sonification for Data Exploration: How listening modes and display purposes define design guidelines’ takes a top-down approach to the development of guidelines for sonification assembled from taxonomies of listening, such as those by Schafer, Chion and more recently by Tuuri and Eerola. They argue that by conceptualising sonifications along two poles where sound serves either a normative or a descriptive purpose, design guidelines can be developed and the role of aesthetics can be fruitfully discussed in a way that is proper to listening modes.
David Worrall uses recent research in listening and embodied cognition to argue for inclusion of micro-gestural inflections in sonification. He shows that listening is not simply a passive ingestion of organised sounds but rather is an embodied activity that invisibly enacts gestures of what is heard. This embodiment is lost in much computer-synthesised music and, through tool inheritance, in parameter mapping sonifications (PMSon), in which parameters or features of the data are mapped to sound parameters, such as physical (frequency, amplitude), psychophysical (pitch, loudness) or perceptually coherent (timbre) complexes. PMSon is recognised as a valuable sonification method, because of its flexibility and the high number of acoustic attributes available, but he finds it limiting because of its co-dependence, or lack of orthogonality in the psychophysical parameter space. His article title asks ‘Can Micro-Gestural Inflections Be Used to Improve the Soniculatory Effectiveness of Parameter Mapping Sonifications?’ Worrall is reluctant to make his title a declamatory statement because no empirical study has been undertaken to examine the effects of incorporating micro-gestural models in the process of transforming data structures to PMSon; however, his article makes a compelling argument to include them.
Michael Winters and Marcelo Wanderley go beyond aesthetics and call for the field of sonification to consider the representation and communication of emotion more seriously. In ‘Sonification of Emotion: Strategies and results from the intersection with music’ the authors discuss affective computing, musical emotion and sonification. They frame present research, by ‘identifying contexts where sonification has thus far been used, its relationship to aesthetics, and the conditions that qualify a technique as a “sonification of emotion”’. After much contextualisation the authors present two of their own sonification mapping strategies that use auditory cognitive mechanisms of brain-stem reflex and emotional contagion cues to display arousal and valence. This is a thoroughly researched paper supported by a strong methodology and analytical appraisal of computational results, which is difficult to achieve when writing about emotion.
Stephen Roddy and Dermot Furlong return to embodiment in ‘Embodied Aesthetics in Auditory Display’. They use Lawrence Zbikowski's embodied music cognition framework to design mapping strategies that include embodied processes of conceptual metaphor, conceptual blending, prototype theory and conceptual model building. There have been significant contributions to the literature in understanding of how schemata relate to relationships within a data set. However, an understanding of how schemata can be modelled in the auditory domain remains a little-studied topic. The research described in this paper focuses on employing embodied schemata theory as a design framework for sonification in an auditory display. With these relations as a guideline, the authors posit that an auditory display designer can coordinate mappings to show or emphasise different embodied schematic structures and relationships within a sonification.
Ryo Ikeshiro also posits a new type of sonification, self-similar sonification (SSS). This refers to the use of the same data as sound at multiple time-scales: 1) audio rate, audifications or modifications thereof; 2) control rate, rhythms and pitches; and 3) at the structural level, relating to sections and their order within the work. He describes his own audification and non-standard synthesis techniques in Construction in Self, a generative music work based on the Lorenz dynamical system, which is a representation of forced dissipative hydrodynamic flow and convection currents. Ikeshiro writes intelligently about sonification and synthesis, combining the historical, technical and aesthetic in an engaging essay which encompasses more than his own work.
Michael Filimowicz uses his own work in progress – sonifications of the quantitative component of five years’ worth of student course evaluations – to relate compositional choices to an aesthetic field. In ‘Peircing Fritz and Snow: An aesthetic field for sonified data’, he uses the personas of Fritz, Data-in-Itself, and Snow, Listener-for-Itself, to set up a fascinating dialectic between the poles of music composition and auditory display mentioned in his introduction. He examines three constitutive sources of audible parameters in data: parameters ascribed to features of the data itself, parameters ascribed to the structural organisation of the data (e.g. a matrix, array or table structure), and parameters attributable to the specific means of sonification from the two perspectives (data-centric and listener-centric). Of course the two perspectives only create a line; other points and vectors are needed to fill out the area of the field under consideration and construction. Filimowicz examines other dialectical tension such as determinacy and chance, and what he terms cognitive support or sabotage. This nuanced article is dense with references and relies heavily on Pierce's ideas of semiotics, but rewards even the naive reader with clear examples of sometimes difficult concepts.
While data sonification is a relatively new field of modern scientific enquiry, its use in music inherits many of the concerns of procedural composition. The breadth of articles about data sonification in this issue illustrate that the relationship between these two uses of what in modern parlance is called ‘sound design’ is ongoing, dynamic and largely positive. Data sonification has inherited tools designed for synthesising musical sounds, while music benefits from the call for more objective understanding of performance sound affect, and both are concerned with a deeper understanding of the requirements for effective listening to organised sound.
