3.1 Different listening modes

A sonic environment can be defined, in a trivial way, by all the sounding objects or elements that belong to one’s daily surroundings. Grounded on a more academic basis, this notion of sonic environment can be linked to the concept of soundscape developed by R. M. Schafer in his seminal book The Tuning of the World (Schafer Reference Schafer1977). In this publication, Schafer makes a clear distinction between hi-fi and lo-fi soundscapes, the former obtaining a ‘good’ signal-to-noise ratio (SNR) – that is, high enough to deliver an auditory perspective and make possible the distinction of several sources – and the latter being, on the contrary, a low SNR environment in which all sources seems to be mixed together and covered by background noise. Another way to define – and illustrate – the notion of sonic environment could also borrow from Nelken and de Cheveigné’s definition of a general sound scene, linked to the functioning of auditory system in the brain: ‘a skeleton of events on a bed of texture’ (Nelken and de Cheveigné Reference Nelken and de Cheveigné2013). This last proposition converges towards the understanding of a sonic environment as a composition of singular sound sources and extended noises perceived more or less easily by human beings.

That being said, it has been well established that the perception of a sound signal coming from any source can be processed in differerent ways and at different levels of meaning. Initiating the musique concrète current, Pierre Schaeffer redefined the notion of listening by formalising four modes, each of them corresponding to a singular perceptual attitude:

• ∙ perceiving (ouïr) means listening to the sound without questioning, like an elementary, raw and passive perceptual action;

• ∙ hearing (entendre) means applying reduced listening, that is, selecting in the sound signal some structural elements useful for the characterisation of the percept;

• ∙ listening (ecouter) means looking for cues and indices in the sound that will inform of its possible origin, its source and the nature of the event that has caused it;

• ∙ understanding (comprendre) means arriving at the meaning of the sound by considering it as a sign within an entire language.

A similar terminology was proposed by W. Gaver (Reference Gaver1993) with regards to environmental sounds as defined by Vanderveer (‘any possible audible acoustic event which is caused by motions in the ordinary human environment ... Besides 1) having real events as their sources ... 2) [they] are usually more “complex” than laboratory sinusoids ... 3) [they] are meaningful, in the sense that they specify events in the environment ... 4) the sounds to be considered are not part of a communication system, or communication sounds, they are taken in their literal rather than signal or symbolic interpretation’ (Vanderveer Reference Vanderveer1980)):

• ∙ musical listening focuses on sound properties;

• ∙ everyday listening focuses on the source that produced the sound.

The term musical is nevertheless possibly ambiguous because, as quoted by Tuuri, Mustonen and Pirhonen (Reference Tuuri, Mustonen and Pirhonen2007), it may be possible to have a musical listening of a non-musical sound and, conversely, to focus on the sources (origins) of a musical signal – for instance, by trying to separate some instruments within the orchestra.

Combining theses notions, in his work dealing with image/sound relationships, Michel Chion defines three listening modes, or auditory perception attitudes, which constitute a sort of synthesis of Schaeffer and Gaver’s approaches (Chion Reference Chion1994):

• ∙ reduced listening, taken from Schaeffer’s terminology;

• ∙ causal listening, similar to Gaver’s ‘everyday listening’;

• ∙ semantic listening, related to the meaning of sound.

Since that time, some scientific studies have aimed at demonstrating – even partially – the mental structure associated with the perception of environmental sounds that has been established more or less empirically by their respective authors. In particular, within the framework of the European collaborative research project CLOSED (Closing the Loop of Sound Evaluation and Design; Susini, Misdariis, Lemaitre, Houix, Rocchesso and Polotti Reference Susini, Misdariis, Lemaitre, Houix, Rocchesso and Polotti2006), Lemaitre, Houix, Misdariis and Susini (Reference Lemaitre, Houix, Misdariis and Susini2010) have collected and studied a sound corpus related to a specific daily environment – the kitchen – and have experimentally discovered two main listening levels, similar to those defined by Chion:

• ∙ acoustic level – analytic listening of the abstract properties of a sound;

• ∙ causal level – listening of the sound production phenomena.

Moreover, this study also showed the interaction of two main factors on these listening modes:

• ∙ identifiability (i.e., the ability of the sound to be identified by the listener) – the less the sound is identifiable, the more the acoustic listening mode is involved;

• ∙ expertise – the more the listener is inexperienced, the more the causal listening mode is involved.

3.2 Warning sounds

In the specific case of warning sounds, these various ways of interpreting the auditory perception mechanisms can be found again through different formal approaches and methodologies for designing auditory warnings. Simply sketched, on the one hand, Patterson (Reference Patterson1982) proposed a structure for alarms based on a hierarchical decomposition of the signal into elementary components (from shortest to longest): pulses, bursts and warnings. A pulse is defined as an element of noise or vowel-like signal of a few milliseconds in length, a burst consists of multiple pulses, and a warning of multiple bursts. In constructing this taxonomy, Gaver attempts (with limited success) to bridge the gap between the atomic level and the semantic level of an auditory warning. Despite its reductive nature, Gaver’s formalisation within the framework of auditory icons (Gaver Reference Gaver1986) led to a recommendation to use environmental sounds for warnings (or at least auditory displays) able to be easily understood and associated with a particular object or context. A third way to consider the development of warning sets, in reference to Stanton and Edworthy’s (Reference Stanton and Edworthy1998) formalisation, rests upon a user-centred approach and the notion of auditory affordances.

This latter notion was borrowed by the authors from Gibson’s theory of affordances, tied to the ecological approach that he advocates to explain perceptual mechanisms (Gibson Reference Gibson1979). This theory provides a precise specification of the objects of sensory stimulation that are available for the human organism. According to this theory, perception is active, linked to movement, and picks up the variations and structural stability of a flux of information received by the sensory organs. In contrast to top-down models of perception, the idea of an internal mental representation is alien to Gibson’s, as information is already contained in the structure of the sensation (for instance, of light energy or sound timbre). The perceptual mechanisms are thus focused on the perceived object and its intrinsic data (data-driven), thereby obviating the need to process information coming from the sensory modalities for building a representation of the world.

To paraphrase Gibson, the notion of affordance can be defined as the potentially useful and relevant information for a person to take action (Gibson Reference Gibson1977).Footnote ⁴ Moreover, Stanton and Edworthy have extended and applied this notion to the perception of auditory warnings, by proposing a ‘theory of auditory affordances’, and consequently a methodology for designing auditory warnings, that assumes that ‘people understand sounds in the world in terms of their potential for action’ (Stanton and Edworthy Reference Stanton and Edworthy1998: 393–4). For instance, in an Intensive Treatment Unit (ITU) medical context, the sound from a syringe pump should confer the action of replacing the drug (ibid.: 392). More precisely, the associated methodology and the corresponding pilot study described by the authors examined and tested several options of sound design, ranging from nomic to symbolic, including metaphoric propositions for new auditory warnings in an ITU setting (for instance, heartbeats for cardiac monitoring, music for an infant warmer device or bubble sounds for syringe pump) (ibid.).

Schalkwijk (Reference Schalkwijk2017) addresses associations between two classes of auditory warnings and their respective vehicle and aviation referents. The two classes of auditory warnings are identified as follows:

• ∙ auditory icons – iconic representations (or caricatures) of sounds occuring in everyday life situations and, again, transposed in virtual environments to embody objects they contain or actions they suggest (Gaver Reference Gaver1986);

• ∙ abstract melodic alarm signals (i.e., earcons) – arbitrary sounds linked to elementary actions in a virtual environment, and in some cases, using hierarchical rules for creating complex and composite messages (Blattner, Sumikawa and Greenberg Reference Blattner, Sumikawa and Greenberg1989).

As such, the efficacy of music-element-based metaphorical alarms is systematically examined to a greater extent than has been the case in previous studies. The issue of prior associations/mental models of subjects for the presented abstract melodic alarm stimuli did emerge when conducting experiments. However, the demonstrated ability to form such associations that are not easily verbalised by subjects suggests their utility in improving non-verbal communication in human–computer interfaces (HCI).Footnote ⁵

The formal approaches described above fit quite clearly into the scope of the alarm/will/sound project, especially by considering the way in which new synthetic auditory warnings may induce human actions/reactions against the sounding object (the vehicle, in this case).

3.3 Iconic vs abstract sounds: sound causation identifiability experiment (1)

The main objective of this first perceptual experiment was to determine levels of sound source identifiability among selected items of the sound catalogue (see Section 2 and Figure 1). Given the size and scope of the catalogue, only a subset of the Industrial/Mechanical category was chosen as stimuli. This category was preferentially selected because of its large number of sub-categories and the varying levels of source abstractness relative to the other Individual Sound categories. The expected result of this experiment was to assume the form of an abstractness-iconicity scale constructed across the corpus that would also help to further determine salient semantic and acoustical attributes of the sounds (see Section 4). Such a scale could be formally related to one of the the two fundamental concepts of earcons and auditory icons, as defined in Section 3.2.

The experimental framework was an online test in which subjects were asked to listen to each stimulus, provide a brief description of sound causality and indicate a confidence rating of causality identification on a standardised five-point Likert scale (see Figure 2 for a depiction of the experimental user interface).

Figure 2

Experiment 1 user interface.

Figure 3 presents the confidence ratings of the 39 tested stimuli, averaged over the 15 usable participant data sets and listed in increasing order (from low to high). After careful analysis of the (useable) data, it became evident that there were no instances of a high confidence level being correlated with an incorrect or vague sound identification, and vice versa. Given this finding, it was determined that, for the purposes of this experiment, confidence rating was a viable measure of sound iconicity. As such, a significant difference t-test was performed to compare the response vector associated with each stimulus and those of the highest- and lowest-ranked sounds. This enabled the authors to localise the points along the confidence rating scale at which the difference between the upper-bound and ith sound, and symmetrically, the ith and lower-bound sound became significant. Fortunately, these points were identical. It was therefore possible to demarcate the threshold between iconic (strongly identifiable) and non-iconic sounds (represented by the thick vertical line in Figure 3).

Figure 3

Experiment 1 confidence ratings for 39 stimuli. Mean scores, as well as maximum and minimum standard deviations are indicated. The thick black line indicates the iconicity threshold.

An analytic view of Figure 3 can lead to the following assumptions: the sounds that caused the most confusion have either contextually obscure sources (wooden gears or milling machine), impulsive features (grenade blast, marbles in vase, or a ball-ricochet) or are steady-state and drone-like in nature, with motor sources (air conditioner, microwave oven, boat motor or food processor). By contrast, the iconic sounds are quite context-specific (grandfather clock bells, antique train or machine gun), time-varying (e.g., helicopter, airplane or train passing), or common (e.g., corkscrew opening a bottle or alarm clock sounding).

The results of experiment 1 in alarm/will/sound project allowed the authors – as expected – to situate the sound stimuli along an abstractness-iconicity gradient and to determine salient semantic characteristics of each sound of the corpus. It was then concluded that sounds falling to the right of the iconicity threshold (see Figure 3) would not fit the auditory warning context, being potentially too closely attached to specific sources that may exist in the vehicle’s direct environment (e.g., airplane, train). The definition of this threshold also allowed the authors to define an operational sound corpus for the subsequent phases of the project.

4.1 Acoustic description of the sound corpus

On the basis of specific acoustic parameters that had been discovered in the Misdariis et al. (Reference Misdariis, Minard, Susini, Lemaitre, McAdams and Parizet2010) study to be critical to an efficient description of heterogeneous sets of environmental sounds, the sound corpus used in the alarm/will/sound project – and more precisely, the subset of 31 Industrial/Mechanical sounds tested in experiment 1 that fell below the iconicity threshold – has been characterised in a two-dimensional physical space, where each dimension, representing a specific acoustic feature, has been demonstrated to be significant in the perception of environmental sounds.

The two features defining this physical space are perceptual spectral centroid (PSC) and harmonic/noise energy ratio (HNR). These parameters were computed in the Timbre Toolbox framework by means of IrcamDescriptor v2.7 routines (Peeters Reference Peeters2004). PSC and HNR are defined as follows (see Misdariis et al. Reference Misdariis, Minard, Susini, Lemaitre, McAdams and Parizet2010 for details):

• ∙ PSC (perceptual spectral centroid) – amplitude-weighted arithmetic frequency average of the signal spectrum, computed on each channel of a gammatone filterbank based on the ERB (equivalent rectangular bandwidth)-rate scale (Slaney Reference Slaney1998).

• ∙ HNR (harmonic/noise energy ratio) – relative amount of harmonic energy and noise energy contained in the signal. Once both parts (harmonic/noise) of the signal are extracted, the feature simply consists of the ratio of their respective sound levels, that is, energy computed with regards to the ERB-rate scale, as above.

Ideally, a control experiment consisting of a dissimilarity listening test would have been useful to verify the coherence between the a priori physical space and the actual perceptual space. In other words, this control test would assess the fact that the perceptual dimensions underlying this sound corpus can actually be explained by the chosen physical parameters PSC and HNR. At this stage of the process, the authors assume, on the basis of Misdariis et al.’s meta-analysis conclusions and the comparable nature of the sound stimuli used in the 2010 study, that these features are significatively relevant regarding the perception of the selected sound corpus.

4.2 Populating the physical space: source, hybrid and reference sounds

After this computational stage, the subset of 31 stimuli, obtained by considering the iconicity threshold from experiment 1 on the initial 39-sound corpus, could be situated inside the PSC–HNR physical space, as depicted in Figure 4. In order to fill empty or under-represented areas of this physical space determined by the two acoustic features, hybrid sounds were constructed via applying constant cross-synthesis and LPC amplitude and phase source-filter synthesis algorithms to pairs of one-second samples of extant sounds in AudioSculpt.Footnote ⁶ Decisions for each sample pair regarding which synthesis algorithm to employ, as well as constant cross- and source-filter analysis settings (e.g., LPC order, in the case of source-filter synthesis) were made based upon analyses of the combined signals, as well as a process of trial-and-error. PSC and HNR values for candidate hybrid out-signals were computed using the methodology described above. Those synthesised sounds whose PSC or HNR values fell roughly half-way between the values of the input signals from which they were generated were retained and integrated into the acoustic features space. Given that the objective of this procedure was merely to populate gaps in the descriptor space across the PSC range, and within the band of HNR ≤1.5 in which the vast majority of source sounds fell, it was neither necessary nor productive to apply stricter hybrid selection criteria.

Figure 4

Acoustic space built upon two perceptually relevant acoustic features taken from (Misdariis et al. Reference Misdariis, Minard, Susini, Lemaitre, McAdams and Parizet2010): perceptual spectral centroid (PSC) and harmonic/noise energy ratio (HNR). Black numbers beside black dots=source sounds; black numbers beside empty circles=six existing car alarm sounds; grey numbers beside empty squares=hybrid sounds.

Finally, for sake of homogeneity and curiosity, a subset of real car alarms collected during the initial step of constitution of the sound catalogue were also added to the physical space so as to be able to compare existing textures (timbres) used in current auditory warnings with the textures proposed and realised with the scope of the alarm/will/sound project. This comparison can be considered as the first step towards the innovative design of car alarms targeted by the project, and can be used as a first informal representation of the authors’ ability to extend the current domain of auditory warnings.

4.3 From source/hybrid sounds (timbre) to auditory warnings (alarms)

After having preferentially focused in the previous step on the notion of texture (see above) via the timbre characterisation of a selected set of sounds within the initial corpus, the work was then continued by considering more precisely the temporal shape of the new warnings. This approach was centred around attending to two high-level sound parameters that could be considered musical, to a certain extent: 1) the temporal morphology of the energy and 2) the rhythm, in the form of a very specific attribute for alarms, the inter-onset interval (IOI). Finally, the combination of source texture, energy profile and rhythm/rate lead to the construction of an original corpus of new types of alarms which were evaluated and assessed in a second sound perception experiment (experiment 2: see below).

4.3.1 Temporal morphologies of environmental sounds

Regarding the question of temporal morphologies, the authors’ point of departure was Peeters and Deruty’s study of an (automatic) morphological description of sounds (Peeters and Deruty Reference Peeters and Deruty2008), that has been thereafter perceptually reinforced and augmented thanks to an original protocol based on subjects’ graphical production on a WACOM Intuos tablet (Minard, Misdariis, Houix and Susini Reference Minard, Misdariis, Houix and Susini2010). In the latter study, 55 sound stimuli of varying duration were presented: 12 sounds ranged between 350 ms. −1 second, 18 were between 1 and 3 seconds, and 25 between 3 and 5.15 seconds. These two studies formalise a taxonomy of six perceptually distinct environmental sound morphologies: stable, decreasing, increasing, single impulse, pulse-train and rolling, whose symbols – extracted from an ‘average profile’ of all subjects’ drawings in the Minard et al. study – are depicted in Figure 5. In the context of alarm/will/sound, these six specific energy envelopes have then been modelled as BPFs (break-point functions) and have been used as amplitude modulation envelopes to create 186 (6×31) specimens of alarms combining different textures (timbre) and temporal profiles.

Figure 5

Taxonomy of temporal morphologies of environmental sounds (from Minard et al. Reference Minard, Misdariis, Houix and Susini2010).

4.3.2. Existing car alarm inter-onset intervals (IOIs)

The inter-onset interval (IOI) feature has been theorised, among others, by Patterson in the context of proposing guidelines for the creation of auditory warnings. In so doing, Patterson defines a standard structure of an alarm: a series of bursts made of a series of noise pulses (Patterson Reference Patterson1982). The time between the bursts in the series is identified as the inter-onset interval, a leading attribute of an auditory warning, responsible for creating a sense of perceived urgency (Sueid, Susini and McAdams Reference Suied, Susini and McAdams2008).

In order to explore this fundamental feature, the inter-onset intervals of six standard car alarms, derived from the Car Alarm Prototypes category of the sound corpus depicted in Figure 1, and situated in the acoustic features space (Figure 4), were calculated in the MatlabFootnote ⁷ environment. For each of them, an elementary component – the ‘burst’ in Patterson’s formulation – was manually isolated and used to detect the periodicity of the alarm that directly provides the IOI value (see Figure 6 for a depiction of this principle). Given the similarity in IOI of alarms 1 and 2, 3 and 5, and 4 and 6 (respectively) to each other, each pair of IOIs was averaged, producing three distinct IOI mean durations: 300 ms (named ‘short’), 536 ms (named ‘medium’) and 2,082 ms (named ‘long’). These three different IOI values were used to create 558 (3×6×31) specimens of alarms combined with the previous combination set of different textures (timbre) and temporal profiles.

Figure 6

Six car alarm signals (1 to 6 from top to bottom). For each, the grey segment of the curve is the elementary component used for computing the IOI of the whole auditory warning signal.

6.1 detritus|reconstructions: interactive electronics improvisation

The structured laptop improvisation detritus|reconstructions (2013) incorporates, analyses, processes and modulates samples derived from the Paris field recordings falling under the Recorded Auditory Scene sub-category of the sound catalogue (see Figure 1), as well as the Industrial/Mechanical Individual Sound sub-category. These processed samples are presented at varying levels of recognisability – ranging from iconic to thoroughly glitched. Using the insights and abstractness-iconicity gradient obtained from the experiment 1 results as a reference, frequency, speed and direction shifting, amplitude modulation and delay/feedback are applied to the selected source-sounds at varying levels, such that sound causation (and context) identifiability is distorted and ultimately minimised. Through the layering of transformed field recording samples, synthetic auditory scenes are constructed, and identifiability of individual sources is further diminished. It is intended that, as the Auditory Scene taxonomy expands, samples will be selected on the basis of performance location.

6.2 Future Creatures: fixed media audiovisual work

A fixed audiovisual media work created using the Python-based concatenative synthesis progam AudioGuide,Footnote ¹⁰ Future Creatures (2013) was produced in collaboration with Korean animation artist Eunjung Hwang (Movie example 1; Sound example 1). The sound segments and targets to which segments were mapped in the concatenative synthesis process were derived in part from the Industrial/Mechanical, Vocal Individual Sound (Human and Animal) and Car Alarm Prototype categories of the aforementioned catalogue. In so doing, both the (granulated) surfaces and (spectro-temporal) structure of the source sounds become the core material of the audio. These sound categories were selected in response to the characters, objects and events represented in the animation.

Given the garish nature of the colour palette of the visuals, and lack of ‘glitch’ elements therein, Industrial/Mechanical and Car Alarm Prototype sounds situated in the acoustic features space depicted in Figure 4 with relatively high PSC and HNR values were chosen.Footnote ¹¹ The decisions as to the segment-size of the Industrial/Mechanical source sounds were determined based upon their respective positions along the aforementioned abstractness-iconicity gradient. That is to say, the more identifiable the source-sound, the smaller the segment-size and the larger the segment density. Echoing the frenetic and densely layered nature of the visuals,Footnote ¹² these surface fragments and sound morphologies are situated in polyphonic textures, thereby creating synthetic auditory scenes, in which the sources stand in complex and unstable relationships with each other. The auditory events alternately reinforce and conflict with the sense of place conveyed by the visuals.

By dint of the sheer event-density of both the visual and the auditory domains, which saturate the senses, changes in the audio tend to trigger attentional shifts in the visual field, and vice versa. It is these cross-modal relationships on the part of the spectator that the composer wished to explore, as opposed to establishing a fixed translation between visual and audio elements. Thus far, no systematic study of the attraction/repulsion levels and internal properties of the audio elements that cause the greatest attentional shifts among viewers of the video piece has been conducted. However, as a supplement to experiment 2, such an investigation is planned for the near future. This type of empirical study would influence both further research into synthetic auditory warning attraction vs repulsion, and subsequent creative works in which sound sources of a similar nature to those found in Future Creatures are employed.

6.3 fcremap and atrocity exhibition: ensemble + live electronics electroacoustic compositions

The extraction and application of spectro-temporal features of synthetic alarm source-sounds in the creation of Future Creatures was taken a step further in fcremap (2014–16), a series of works for instruments, live electronics and visual media, based upon the video and audio of Future Creatures. With respect to raw material in the electronics, entries in the original sound catalogue and the hybrid timbres populating the acoustic feature space described above (and employed in experiment 2) were incorporated. Throughout the cycle, the timing and direction of physical actions of performers, which trigger routines in the electronics, are influenced by the morphologies and IOI durations of the synthetic alarms used as stimuli in experiment 2. These morphologies and IOIs were thus treated as abstractions, transposed onto a diverse array of instruments (e.g., various percussion, piano and mixed chamber ensemble), at multiple time-scales and degrees of polyphony.

In the cases of the pieces fcremapercremaflvln (for alto flute, violin, electronics and video)Footnote ¹³ and fcremap’dB (for ensemble of quasi-variable instrumentation, animated score and electronics, see Movie example 2, Sound examples 2 and 3),Footnote ¹⁴ these abstractions were employed as filters and linear (visual) elements imposed upon the still images comprising the video projection (in the former case) and the animated score (in the latter).

In fclremap (for improvising solo clarinetist and electronics) and fcremap’dB, the performer is presented with a graphic score, consisting of a series of modified images derived from Future Creatures, and a set of guidelines as to how to interpret the graphics on their respective instruments. The electronics thus function as a kind of ‘aural score’, which is to be reconciled with visual data. The application of six alarm IOIs to the electronics effectively creates a sequence of six temporal grids, which the performer may resist, use as temporal reference points or (try to) ignore entirely. For fclremap, the clarinetist is also asked to approximate the timbral qualities of the audio segments. The surface and structural features of the alarms, coupled with relatively stable image-to-sound mappings between Future Creatures video elements and the instrumental and electronics material of the fcremap pieces, form a connective tissue for the composer and listener, binding the constituent pieces of the cycle together, despite contrasting performance scenarios and aesthetic characteristics (Figure 11 and 12).

Figure 11

An excerpt of the first movement of the fcremaperc score, scored for tam-tam, resonating metal sheet and live electronics. The pitch contours and IOIs of synthetic alarms utilised in experiment 2 are transposed onto the right-hand and left-hand actions of the performer.

Figure 12

An excerpt of the fcremap’dB score, with synthetic alarm temporal patterns and pitch contours inscribed.

Scored for horn, electric violin, electric guitar, bass guitar, prepared piano and live electronics, the basic material elements in the electronics and the instrumental parts of atrocity exhibition (2013) stem from Industrial/Mechanical, Animal Call (under the Vocal Individual Sound category) and Car Alarm Prototype entries in the sound corpus (Sound example 4). Both the sound morphologies and their respective spectral contents were employed to generate the rhythmic and harmonic structures, as well as playing techniques in the instruments, and the spectral content of the electronics. In the case of the piano, the sources of a few of the industrial/mechanical sounds (ratchet, glass shards, metal chain, metal rod and plastic plectrum) – each of which represents a unique physical material or mode of sound production – are utilised on the piano strings. Furthermore, the electronics are projected through a pair of small transducers placed inside the piano (see Figures 13 and 14). As such, the boundaries between the production and projection of instrumental and electronic sound become blurred. This grey area between mechanical instrument and transducer activation also extends to the violin, electric guitar and bass guitar, whose output is distorted via effects pedals and amplified.

Figure 13

Small transducer (back) utilised in atrocity exhibition and Minotaur.

Figure 14

Small transducer (front).

The range of influences and representations of the sound catalogue entries on the musical materials – as sources analysed for spectral and temporal content, as processed samples in the electronics and as models of physical causalities in the instruments – engenders an abstraction-iconicity gradient, along which these materials are situated. This investigation of levels of abstraction, iconicity and sound causality identification reflects the objectives of experiment 1 (outlined in Section 3.3). In constructing our sound catalogue taxonomy, the perceptual issue of listening modes (acoustic/causal/semantic) arose. Through presenting myriad manifestations across multiple musical dimensions of the original sound sources, and blending acoustic and electronic modes of sonic transmission, it was intended in composing atrocity exhibition to explore and reveal the complexities behind sound source perception and identification, and the preservation, loss or transformation of sonic semantic associations.

6.4 Minotaur: interactive media installation

An ongoing audio–visual–kinetic performance-installation collaboration Minotaur (2012–current) brings together music/sound design, video, dance, sculpture, live painting and set design. With respect to alarm/will/sound, it transports the sound categories and features discussed above into the gestural domain, serving as a testing ground for a set of interaction design models that may be incorporated into alarm system prototypes emerging from our research and development activities.Footnote ¹⁵ Based upon twentieth-century re-castings of the myth of the Minotaur,Footnote ¹⁶ the locus of activity is a labyrinth constructed out of Plexiglas panels. Inside the labyrinth, dancers’ movements are detected via microphones and sensors, which are processed in the electronics. In turn, audio output is transmitted to an array of transducers (similar to the model used in atrocity exhibition), affixed to the Plexiglas as well as to hollow sheet-metal sculptures at the centre of the labyrinth (see Figure 15). Consequently, the sculptures, which symbolise the primal desires and tendencies of the Minotaur, function as resonators, and the audio produces Plexiglas vibrations, which influence the dancers’ actions. A feedback loop between sound generated inside the labyrinth and audio produced external to it is thus set in motion. Adaptable for gallery, concert and even telematic performance contexts,Footnote ¹⁷ sounds recorded directly outside of the venue during the performance/exhibition may be integrated into the audio, thereby localising the constructed labyrinth environment.

Figure 15

Sheet-metal sculptures (with transducers attached), dancer, and Plexiglas projections, in one version of Minotaur.

Audio events may be self-triggered at random intervals, manually activated by the composer or triggered in response to activity within the labyrinth. In all cases, the temporal patterning of the synthetic alarm sounds used in experiment 2 has governed the number and rate of iterations of triggered sonic elements, with sounds intended to repel dancers from their current respective locations assigned medium and short-duration IOIs.Footnote ¹⁸ However, changes in physical parameters within the labyrinth cause shifts in the spectral and spatial properties of the triggered sound materials over time. Such a gesture-sound relationship introduces a degree of unpredictability to the electronics.

Given the feedback loops between labyrinth-internal and labyrinth-external audio, this unpredictability is compounded, and impacts not only the auditory, but also the tactile and kinetic dimensions of the constructed labyrinth environment. Consequently, the interaction model becomes multi-modal, multi-directional and time-varying. In the gallery context, audience members have the opportunity to enter the labyrinth and engage with the sensor/actuator system. To an extent, the installation version of Minotaur serves as a model for an eventual interactive public presentation of our alarm systems under development, as described below.