1. MUSIC–TEXT–FILM

In 2002 I started exploring a form of multimedia that involved projecting text alongside synchronised sound. At first this was a way of exploring what would occur when meaning or narrative is imposed on a musical structure, where the music itself had no pre-defined narrative intention. The idea was that the onscreen words are not being used to either amplify or translate what is being spoken, sung or visualised, but rather to interfere with the viewpoint on the music. It was conceived as a form of subtitling (or surtitling), minus the foreign voice, which was in this case the music. I called these pieces, for want of a better phrase, ‘music–text–films’. From these initial experiments two related questions arose, and eventually became an interesting point of reflection and further development: Who is narrating? And where is the voice?

The concept of narrative voice has been widely discussed in narrative theory over the last century, with the main focus being on literature. The French literary theorist Gérard Genette, one of the principal thinkers in the field of narratology, takes the Aristotelian concept of ‘diegesis’ and develops it to ask precisely the question about the location of voice (Genette Reference Genette1988). In literature, when we read words on a page, whether the narration is ‘intradiegetic’, that is coming from a character in the story world, or ‘extradiegetic’, told from outside the narrated world, things are relatively straightforward. Even in music where an actual voice is concerned, in song or in opera, one can locate the focal point of the narration. Matters begin to get complicated when the idea of narration is removed from a discernible voice, when there is no clear embodiment of the storyteller, or when the idea of voice becomes split between different media.

The second question, which in truth is very similar to the first one, became more urgent after I noticed a strange phenomenon occurring in performances of these music–text–films. When we read text in time with music we become very aware of our inner voices. We hear ourselves reading.Footnote ¹ The role of the viewer/listener is crucial in the subvoicing of each of the phrases, words or phonemes that appear onscreen, precisely because they are presented in synchronicity and through the surrogate voice of the music. In a sense, the voice, formed by the combination of music and projected text, becomes embedded in the audience’s lips, as each person projects onto it their own inner voice. The absence of a spoken or sung voice that correlates exactly to what is being projected onscreen is one of the defining features of my definition of music–text–film. This concerns not only the idea of a redundancy of medium, a performative voice being doubled with our own voices reading the text, but also with the idea of absence, and the invitation to the viewer/listener to look for a surrogate voice within the framework of the music.

When we read a book, we are not required to overtly vocalise the text, because most advanced readers have learned to ‘chunk’ syntactic groups and process them more efficiently. (‘Chunking’ is the grouping of words in a sentence into shorter meaningful phrases that prevents a reader from having to read each word on its own.) In my music–text–films, because the onscreen words appear one at a time, mostly at a slower pace, and are generally synchronised with an ongoing musical syntax, I have observed two phenomena occurring in parallel. The first is that the sound of the text continues to echo in the mind of the listener/reader as they wait for the completion of the sentence (that will eventually make these words meaningful). The second is that these suspended words, awaiting syntactic completion, form parallel semantic structures with the music.

The effect of hearing one’s own voice in the music becomes an added dimension in the composition that I had not initially predicted when I set out to explore music–text–films. It was a discovery that had many consequences in the way that I subsequently approached composition and ideas about listening. Regarding the subjectivity of the listener, it is not only the extent to which a musical narrative is interpreted in a different way by each individual, but also how the listener might even use their own inner voice to filter and echo the sonic information being received by the senses. Maybe there is a case to be made that we use our inner voice to test out certain sounds we hear for syntactical/melodic structures; we map these sounds onto our own imagined vocal apparatus, a silent imitation of sorts that we might have used as a developmental tool in childhood, that still remains as a way of experiencing the world by imitation.Footnote ²

I eventually came to define music–text–film as the use of projected text together with music, where the words are not being used to either amplify or translate what is being spoken, sung or visualised. This latter point is vital in that it eliminates its predominant use in film, pop video and theatre, where there is a doubling or redundancy occurring in the overlaying of a text that is already being expressed in another medium. This often results in an over-emphasis of meaning, or a ‘sandwiching’ of a medium between two others, that might be said to constrain or force a single interpretation. Similarly, with surtitling in opera, the text is projected both visually and via the voice of the singer onstage, as a result of which the audience does not have to look for the narrative source; it is being reinforced through all the layers of media in front of us.

In my music–text–film forms, there is a discernible dynamic between the media. Where, for instance, the text functions as an independent voice, which Nicholas Cook defines as ‘an independent dimension of variance’ (Cook Reference Cook1998: 83), this separation is exactly what highlights the question of who is narrating in a piece of music. The text is read by the inner voice of the viewer/listener at the composed pace, so that it becomes amalgamated with the music. Sometimes the speed of text is too slow or too fast for any realistic utterance, so that the text becomes suspended or attaches itself to whatever layers of music are being heard at that moment in time. In this way, meaning comes and goes. It is constructed and deconstructed without forcing a clear-cut interpretation. The avoidance in most cases of a clearly perceivable voice to which the text is attached is one of the strategies that open up the space for the music to become a surrogate voice. The voice-like properties of music, and the semantic arsenal that it carries, give a sense, or perhaps an illusion, that something is being said in the music.

In the text that follows, I will cite three music–text–film works of mine, written in the last ten years – which coincidentally have a reference to or use the subject of dreams – as different examples of how words, video and music interact to produce meaning: how the different media can support, highlight, complement, compete, or problematise each other.

Subliminal: The Lucretian Picnic (2003) deals with the type and quality of information we process at the so-called ‘subliminal’ level. This exploration focuses on the psychological idea of ‘priming’: how an indirect exposure to a stimulus influences an emotional response.Footnote ³ I attempt to use it as a metaphor for how meaning is negotiated in a multimedia work – partially on a conscious level, partially on a subconscious, undetected one. Dreams of the Blind (2007) begins with the question of how the dreams of blind people differ from those of sighted dreamers, and goes on to explore the question of medial transference. In The Arrest (2010), the cinematic model of narrative with diegetic sound is explored, albeit in a poetic or associative way; concrete sound is used next to instrumental musical lines to imply location or scene, but because it exists in a dream context, this is shifting and metaphorical, and not to be trusted.

2. SUBLIMINAL (THE LUCRETIAN PICNIC)

Subliminal: The Lucretian Picnic (Movie example 1), composed in 2003, is a 30-minute work for large ensemble, video and soundtrack. The initial intention behind Subliminal was to explore the powerful connection between images, text, sound and music that lie on the threshold of perception: how they affect each other’s expressive potential and resultant meaning, and how our perception changes depending on the type and amount of information it is fed. The piece takes the form of a ‘dream’ essay, where fragments of film, soundtrack, music and ‘psychoacoustic’ electronic sounds are combined in constantly shifting polyphonic textures to produce an overall disorientating effect.

My fascination with what I have termed ‘music–text–film’ originally sprang from an idea about subliminal messaging, perhaps even out of a misconception: how split-second flashes of hardly perceptible text would interfere with the audience’s experience of the music. However, the level of subliminal effect that I had first imagined was very difficult to achieve with the standard computer–video–beamer set-up; one would need a ‘tachistoscope’, a projector with the capability of opening and closing at shutter speeds of about 1/100th of a second or shorter. Furthermore, there was the lingering artistic issue of whether something lying so deep under the radar of consciousness could significantly influence our perceptions. In practice, it was more interesting for me to work on the conscious level of visual text perception, and deal with the subconscious level through the combination of the visual and aural.

Studies of how so-called ‘action priming’ on the subliminal level – how a stimulus would lead a participant to actively do something – have had conflicting results, but accounts of the first use of subliminal techniques in advertising are part of the folklore of modern media history. In 1957 James Vicary, a market researcher, claimed that over a six-week period, 45,699 patrons at a movie theatre in Fort Lee, New Jersey, were shown two advertising messages – ‘Eat Popcorn’ and ‘Drink Coca-Cola’ – while watching the film Picnic (directed by Joshua Logan in 1955). According to Vicary, a message was flashed for 3/1000th of a second once every five seconds. The duration of the messages was so short that they were never consciously perceived. Despite the fact that the customers were not aware of perceiving the messages, Vicary claimed that over the six-week period the sales of popcorn in the theatre rose 57.7 per cent and the sales of Coca-Cola rose 18.1 per cent (O’Barr Reference O’Barr2005: 4).

These claims later turned out to be false, when Vicary himself admitted that the figures had been invented as a marketing ploy. Nevertheless, the idea of the techniques he had supposedly used caused nationwide unrest at the time. People became fearful of being susceptible to this kind of subliminal manipulation, especially with regards to the new media, such as television, that was beginning to permeate the social landscape. This fed into the general climate of paranoia, in part stoked up by the political climate of the time, but also because of the changes that were taking place within society (Packard Reference Packard1957).

The composition Subliminal: The Lucretian Picnic is split between four basic levels of media: two visual – text and image, and two aural – live ensemble and electronic soundtrack.

The visual, cinematic material of the composition Subliminal consists of direct quotations from the actual film Picnic. This was a commercially and critically successful romantic drama staring William Holden and Kim Novak, about a drifter arriving at a mid-western town on Labour Day, and falling for a girl who is set to marry a less charming local.

The manipulation of the source film material in Subliminal is threefold. First, the narrative order is reversed, in order to disorientate the conventional narrative reading, to disengage the diegetic causality, so that it does not dominate. In the second scene, for instance, we see the hero descending from a moving train and running backwards to meet a woman whose hands he grips passionately. In actual fact this is the penultimate scene of the film, where we see the lovers parting, and the hero running to jump on the moving train. The second form of visual manipulation is the rhythmic editing that marks the temporal flow of the footage. Frame rates are manipulated and made to oscillate between positions on the timeline in order to create a visual rhythmic polyphony. The third and final form of manipulation, and the one that has the most important visual impact, is the dimension in which the film is projected. The film is cropped to a ratio of 10:1, and is subsequently projected onto a 10m x 1m screen hanging in front of the ensemble, so that what is eventually visible is predominantly a framing of hands and feet, a narrative which is restricted to bodily appendages. The idea behind this was twofold: to undermine the completeness of the original material, in order to bring it into dialogue with the music and the text, and to make a reference to and comment on the visual manifestation of subtitling or surtitling.

The ‘Lucretian’ part of the title refers to the text used throughout the work, which is taken from the Rolfe Humphries translation of De Rerum Natura by the first-century Roman poet and philosopher Lucretius (1968). Lucretius’s text is used as commentary on what we see and hear. The texts used allude to sensory experience, how the world is perceived, and the nature of dreams, emotions and thoughts, with a strong emphasis on an Epicurean philosophical ideology. This becomes a fitting answer to the sense of hysteria conjured up by Vicary’s experiment. At times the text addresses this head on, with notions about collective fear and about the nature of dreams, reality and identity:

The type of text animation used in Subliminal varies enormously. There was a certain exploratory approach to the many rhythms and layouts used. Nevertheless, the central premise was the short fast-blinking text, which appears a number of times during the piece, for example in the first visual scene of the train and bus where a text about perception appears, blinking at an on/off rate of 1:12 at the speed of 120 bpm:

Later in the piece, two or three consecutive texts might appear at the same time, superimposed, in parallel, or scrolling quickly; the intention was to put stress or weight on the mental ability to process information in one medium against another. In fact this was one of the principal strategies I explored in the piece: experimenting with the perceptual borders of text, image and sound, in order to investigate how much it is possible to assimilate these at any given moment, when the perspective and interrelation between the media is always shifting. The viewer has to constantly reassess the perspective towards the media, the focalisation point of what they are watching in each new section.

The type of text narration in this regard is an extra-diegetic commentary, which does not have an explicit connection to the visual or aural information in the work, but is there to suggest a possible interpretation of the other media. That is to say, we know that the text does not directly refer to the images or the music, but it is up to the viewer to make the connections, draw the conclusions. Because it is never clear where the voice is located in the narrative – who is speaking – the audience is compelled to switch back and forth between the media in search of it.

The music itself is divided into multiple layers, which come across in differing hierarchies. There is the instrumental music, which contains quotations from the film score, and various electronic and instrumental pulses, which are there to emphasise or disorientate visual rhythms. There are subliminal uses of voices from the film, and resonances of these that appear within the ensemble or through the electronic soundtrack. The music develops in polyphonic blocks, switching between different points of view, and only occasionally finds a sense of momentum, most notably towards the final parts of the piece. This formal approach, which can be said to frustrate the audience’s ability to be immersed in the narrative, is akin to a ‘Verfremdungseffekt’, the distancing effect associated with Berthold Brecht, and found in many of the films of Jean Luc Godard, where text, image and sound are constantly obstructing one another (Monaco Reference Monaco1976: 136). Yet because, in this case, there is no explicit voiced narration, and what text there is is only intermittent, we are constantly going back to the music or the visual as a clue for possible meaning.

The concept of the hidden voice is a recurring theme in many of these music–text–films, because it is bound up with the voice of the viewer, the thoughts of the viewer becoming entangled with the projected words during the act of reading. This state of dynamic perspective is enhanced by strategies of avoiding completion. An incomplete or partially hidden medium can draw the spectator from one point of view to another. This conception of a multimedia model, where the media problematise each other rather than reinforce or duplicate the intended meaning, is at the heart of my approach.

3. PERSPECTIVE AND FRAMES

In the musical space, the distinction between the physical sounds we hear and the imaginary world and meanings they set in motion is of cardinal importance, as those differing levels of ontology are exactly what create the idea that a narration is taking place. For classical narration to exist, there needs to be a hierarchical relationship between the world of the narrator and the ‘narrated’ world, in which the latter is embedded in the former (Vervaeck and Herman Reference Vervaeck and Herman2001: 83). One can understand this in terms of the idea of frames, where a frame is created on one level to expose something within it on another level. The important point here is that the frame is created already within one diegetic space, using the attributes and expressive potential of that particular world. In this use of the concept of frames – borrowed from Lawrence Barsalou’s idea of recursive attribute-value structures (Barsalou Reference Barsalou1992) – a model can be constructed in order to show how different media interact with each other to create meaning.

The purpose of using the model of frames in order to understand how narration and multimedia relationships occur is that it conveys the sense that we always look through something onto something else. We need the understanding and sense of one concept to understand another. We need to be able to distinguish variant from invariant for that understanding to occur; the insight we gain manifests itself in the form of a narrative.

In the case of music, because we can distinguish between notes and instrument, between words and voices, between rhythm and sound, we begin to differentiate between levels of ontology, the worlds or frames that are embedded within each other. In this way a multiframed work can exist just as much in a musical form, music within music, as it can in a work of literature, story within story. And yet within music’s fuzzy grammar exist multiple frames, which more often than not mesh into each other, often making it difficult to draw a clear distinction between these micro-frames.

If narration takes place when a space is created between differing ontological frames, then it is nowhere more pronounced than in multimedia work. Here there is a dynamic and metaphorical exchange between levels of meaning that is more apparent and conscious to the perceiver than, say, in a mono-media work (if there can ever be such a thing), because the difference between the media can be clearly demarcated and compared. The splitting of a work into multiple ontological levels, which is the characteristic of multimedia, creates the space for the construction of narrative time, which in turn creates an observable distance. Works with a high degree of narration bestow a self-reflexive quality that, as experienced by the spectator or listener, opens up a viewpoint to walk into the compositional space and explore it on different levels.

This is the case in point in ‘music–text–films’, where the relation between perspective and frames of view becomes more complex. However, while the possible complexity is increased, the opportunity to become aware of these perceptual frames is also greater.

4. DREAMS OF THE BLIND

Dreams of the Blind (Movie example 2) is a suite of five pieces for an ensemble of nine instruments, electronic soundtrack and text film. It was written in the summer of 2006 for the Dutch Ensemble MAE. The five movements of the piece are each based on dream accounts of blind people, collected by the ‘Dream Bank’ research site of the University of California, Santa Cruz.Footnote ⁴ Three of the five have direct references to music – Supermarket Guy, Car Radio, Winter Funeral – while the other two – Floating Table and Hairdresser – are informed by a combination of touch and hearing.

While Subliminal: The Lucretian Picnic deals with a form of extra-diegetic narration, Dreams of the Blind is very much ‘intradiegetic’, as it explores the idea of the voice in the first person, recounting a narrative taking place in the most intimate of places, in the mind of a sleeper. The relation between the absent voice and the instrumental music, which here becomes a surrogate voice of sorts, is very clearly pronounced. The music here deliberately shadows the hypothetical rhythm and melody of the narrator, giving it some form of vocal expression, and in so doing takes on the function of voicing the text.

What is also explored, in a more general sense, is the question of medial transference – how the absence of one medium can place more weight of meaning onto another. In its absence, voice is replaced by text and music, and the absence of image is highlighted by the material of the dreams, which, because they originate in the minds of blind dreamers, rely predominantly on the senses of hearing and touch rather than sight.

In the context of Dreams of the Blind we can describe the following sequence: sensations of touch and hearing are transferred to narrative situations and emotions by the dreamer, which are then put into spoken words by the dreamer’s account and documented as a written text by the lab researcher. I take this text, set it in some way to music (as notation for musicians and electronic sound), and, with an ensemble of musicians, perform the music and project the words on screen. The audience hears the music and reads the text (senses of hearing and sight), which leads to a recreation of a voice (an imagined voice) and perhaps a visual recreation of the imagined dreamer’s narrative.

The metaphor of the blind dreamer is one that can be taken to generally represent how meaning is conveyed through different sense perceptions, both in the mind of the dreamer and in how these dreams are understood by a sighted reader. There exists here an inherent asymmetry in the sensorial communication, because while we usually associate the idea of dream narratives in terms of images, the narratives of blind dreamers are not informed by the visual, but, just as in their waking lives, are predominantly fed by the senses of hearing and touch.

An interesting observation in the process of reading dream accounts is how the ‘absent’ voice acts as a bridge between two barely conscious private spaces: the mind of the dreamer and that of the reader. Just as the voice of the dreamer has been used to convey and understand the images that he or she has experienced, the inner voice of the audience undergoes a transference in the act of reading; the voice of the dreamer is stripped of its original vocality and emotional undertones and takes in its place a mediated instrumental one. The imagined vocal presence of the dreamer embeds itself into the voice of the audience; the inner voice of the reader/audience becomes possessed by the narrator.

In terms of musical narrative, Dreams of the Blind follows two distinct paths: there is an instrumental layer, which reinforces, interprets and embodies the projected text; and there is a constant drone-like electronic track, made up of complex synthesis sounds, which is an attempt to metaphorically represent the space where the dream takes place. In a poietic sense, the drone takes on the form of the dreamer’s body, reacting to or being affected by what is being dreamt. There is a classic ‘figure and ground’ relationship between instrumental sound and electronics, which, as I have explained previously, reinforces the perspective of narration, because it creates the sense of two separate ontological frames.

This also highlights the different types of listening experience between the two media, between a syntactic one and a more directly sensory one, words vs body. These two strands of function are intertwined in as much as the words are intertwined with the music. In fact it is the instrumental ‘voice’, as suggested above, that bridges the space between the sensorial/emotional experience of the electronic sound and the projected words. In many parts of the piece, the ensemble acts clearly as the medium between words and body, something like a voice; a voice existing in parallel with the listener’s own inner voice.

Taking as an example of how the perspective shifts between text, musical syntax and sound, I will analyse an excerpt of Dream IV ‘Car Radio’. The narrative opens with the dreamer describing the location of the dream and whom she or he was with in the dream.

The music is composed of a drone layer of electronic sound that acts as an unstable pedal point for the instrumental phrases that are played. The instruments seem to synchronise the beginnings and endings of the text phrases with short notes, positioning the first-person voice within the realm of the music. The longer notes on violin and recorder, which accompany the text, help give an expressive, emotional and vocal quality to the words. The flow of text is occasionally interrupted by electronic sounds from the drone layer, which seem to have the effect of moving the story along, as if the sleeping body, occasionally stirring into life, directs the course of the dreamer’s narrative.

When the text says:

we hear the guitar and the singer’s voice for the first time. The guitar takes on a more prominent role in voicing the text of Virginia, and the entry of the female voice in the ensemble somehow suggests a voice of ‘the other’. Eventually the instruments overlap with each other, so that the sense of who is saying what becomes blurred.

In the next section the recorder and bass clarinet set the extreme pitch range of the instrumental sound, in a repeated motif that is intended as a sonic representation of the engine of the car the dreamer is sitting in. These instruments assume the role of scene painting. Virginia switches the radio on, and the dreamer tries to describe the music she or he is listening to, and the apparent embarrassment felt at having to like what the friend is playing for her:

The music here switches from ‘voicing’ what the dreamer’s persona and friend are saying, to suggesting a kind of music that is heard from the car radio, over the sound of the engine:

As the dream comes to an end, and the music is stripped down to the bare electronic drone, the dreamer, now reflecting on the dream, reminds us that she/he is blind:

This reminds us that any images we might have seen with our mind’s eye, as an audience reading the text, are very far removed from the world of sensations that the dreamer occupies. We remain in the realm of sound, because images and seeing are never suggested, even though, as an audience, the way we gain insight into this unsighted world is, ironically, with our eyes.

5. DREAM NARRATIVES AND METAPHOR

It is perhaps a coincidence that much of the material for these explorations is taken from a direct or indirect reference to dream narratives. Yet there is clearly something of a correspondence, or at least a metaphorical connection, between the cross-modal activity of dreaming and the exploration of inner narratives that this type of multimedia implies.

One of the strongest associations between dream experience and the cross-modal processing that is involved in the experience of art is in the concept of metaphor. Neurologist V. S. Ramachandran has, amongst others, studied how the inferior parietal lobe is involved in the processing of speech metaphors as well as creating cross-modal links between images and sounds (Ramachandran and Hubbard Reference Ramachandran and Hubbard2001). One could say that meaning in dreams is created by the cross-modal interaction of personally significant associations that appear to us as visual metaphors.

Metaphor has been cited by many writers on multimedia art as a principal concept in the ways we can understand the function of different media. Metaphor is a figure of speech that describes one set of ideas in terms of another, unrelated one (Lakoff and Johnson Reference Lakoff and Johnson1980: 5). The two parts of a metaphor have an unequal relationship: one borrows attributes from the other to highlight some meaning, and this inequality is underlined by their cross-domain correlation. It is precisely the idea of inequality that I find a fascinating key to understanding our perception of multimedia. In my opinion, one of the misconceptions in understanding the cross-modal in much audiovisual work is that there has to be some kind of equality between the media for the work to be valid and convincing.

Without oversimplifying what is a hugely complicated cognitive process, I would like to suggest that this unidirectional transfer of meaning is one way we can understand what might be happening under the hood of consciousness when we are exposed to information on multiple sensory and cognitive levels. The fact that it is unidirectional does not undermine the idea that we might be experiencing this in terms of a constantly updated feedback loop of cognition, but it does underline the principle that a source and a target in metaphorical terms are never equal. For example, in the metaphor ‘Time is money’, attributes of money are mapped onto our understanding of time, not the other way around. We do not see money in temporal terms, we see time in financial terms (Lakoff and Johnson Reference Lakoff and Johnson1980: 8).

That is to say, there is always a hierarchy in how source and target (the terms given to parts of a metaphor) are defined.Footnote ⁵ In language, this is created by the syntax, the order of the words, but how is this manifest in a non-linguistic idea of metaphor?

The hierarchy of relationships between different media depends on many factors. Primarily, the entrenched historical form of a given genre is what mostly determines our point of view. These conventions play a large part in determining our point of view in the experience of multimedia. In much new audiovisual art the relationships between the media are not as entrenched as, say, in mainstream cinema, theatre, or even commercial music video, so there can be a greater dynamic in how the hierarchies are constructed.

In the music–text–films cited in this article, in which there are at least three distinct media interacting in our minds, the metaphoric hierarchy of these relationships – what is the source and target – play an important role in how the meaning of the work is eventually constructed. We might understand the music through an idea in the text, or the text through an idea in the music, or the music through a visual idea, etc. Furthermore, these relationships are always in a state of flux.

I have observed several ways in which these metaphorical hierarchies are constructed and modulated in the context of the music–text–films, which can be applied to audiovisual forms in general:

1. 1. Structure: a medium might establish itself as primary when it gives a clear signal about its formal consistency. Form or structure can make a medium seem self-sufficient, and thus to exist in its own inner logic. A countdown, an alphabetical series, a scale or other more complex large-scale forms will always underline a strong inner order that gives meaning unto itself.

2. 2. Rate of information: the rate and density of information of a given medium often determine how much of the focus it will draw. An active visual field or a dense musical score might tip the scales, and determine it as a source or target of a metaphoric relation.

3. 3. Junctures: points at which hierarchies change, where one medium that has been dominant is removed or replaced by something else. Moments where there is an eruption of sound or a disappearance of sound, a text which suddenly appears out of nowhere, a visual close-up, a synchronisation point, a black out and so on. Anything that disturbs or changes the previous order will create a juncture point, where a change in point of view comes under negotiation, and therefore the relation of target to source may change.

4. 4. Scale: the dominance of a medium could simply be a question of scale. That which is bigger, closer, or louder tends to occupy more cognitive space in our consciousness.

5. 5. Voice: in both a physical and a narrative sense, voice tends to hog the foreground and become ‘target’ rather than ‘source’. This may be connected with how we prioritise vocal or linguistic communication above other types of information (Latinus and Belin Reference Latinus and Belin2011:143), as well as with how our minds construct narrative as a way of understanding the world.

These attributes are some of the ways in which hierarchies can be established and then shift during the course of a piece. Ultimately there is a large element of subjectivity at play; there is no hard and fast rule for the way we experience music or any audiovisual work, but understanding some of the ways in which music, language and visuals interact with each other can give clues to ways in which we can approach creating a space that is dynamic, open and charged with signification.

6. THE ARREST

Change of media perspective is one of the underlying themes in The Arrest (Movie example 3), a composition for ensemble, electronic soundtrack and text film written again for Ensemble MAE in 2010. Taking another aspect of the dream scenario of how music is filtered through the sleeping mind and given some kind of signification, The Arrest uses sound in a cinematic and metaphorical sense to signify something that comes into a meaningful relation with the text.

It is also an example of how mimetic properties of concrete sound are used in the service of narrative, what one could call the cinematic model. Sounds which tend towards a mimetic type of listening are used alongside instrumental musical lines to imply location or scene, but because the concrete sounds are understood in a dream context, they have the tendency to be read as metaphorical rather than literal, and subsequently become subsumed into the narrative or the diegesis.

The text of The Arrest is from a dream narrative by Oulipian writer Georges Perec, found in a collection of 150 dream narratives that he published under the title La Boutique Obscure (Perec Reference Perec1973). The particular dream used in The Arrest is typical of one of Perec’s recurring nightmares: being stopped and arrested by the police, a fear that he had been said to carry from his mother’s experience, originally a Polish Jew, who was captured, deported from France and murdered during the holocaust. This fear of persecution is brought to the fore at a certain point in the text in reference to his Jewish heritage, and the ambiguity he felt regarding his pro-Palestinian political standpoint.

The layers of the music are set up in almost the opposite way from Dreams of the Blind, in that the instrumental music, rather than articulating the narrative voice, serves as a more fixed image, the ground, fluctuating in longer lines. It acts as a voice without content, much like the electronic sounds in Dreams of the Blind, creating tension and stretching out the canvas of the narrative. The cinematic samples of found sound – such as dogs barking, a motor, street voices, a helicopter – act as a window to the dream narrative, sometimes in contradictory and sometimes in complementary relation to the text. For instance, when we read:

the sound of a motor comes into focus, giving us a contradictory idea about what could be an idyllic landscape. Or, when the narrator references his own guilt about the Israeli–Palestinian conflict, we hear the barking of dogs (who have been in the background at several other points in the piece) come sharply into focus, a self-conscious metaphoric relation but one that can exist within the logic of dreaming.

This last example of the barking dog could be said to be a juncture point (as described above) where the metaphoric relation becomes reconfigured. The soundtrack, which had been more subdued in the previous few minutes, suddenly reappears with this sound of a barking dog. This event underlines the text:

The metaphor of the Jewish–Arab conflict as a dogs’ barking match comes clearly to the fore. The soundscape takes on a strongly metaphorical significance, because it is pushed into the foreground. As the text fades we are left with the soundscape of the dogs in the distance and the appearance of new text aligned to the music. The changed situation underlines the instrumental score as the inner voice, because words and musical phrases are synchronised ever more closely. Finally, an urban soundscape drifts into hearing, perhaps highlighting the difference between inner and outer worlds, which signals the end of the dream.

This way of manipulating the sound images in relation to the text has its precedence in some of the great films of Robert Bresson or Ingmar Bergman, in the way meaning and emotion can be suggested by a carefully chosen sound object. What I found interesting to explore in this work was how one can slip between what we perceive as sound information and what we perceive as musical information. Whether one is aware of the exchange between these layers or not, whether it is a conscious process or a process that occurs below the surface, this will still affect how we understand the meaning of the narrative – not in terms of manipulating the meaning, but in terms of providing junctures of reflection, paradoxical moments where one can observe the workings of music and narrative.