Outside school, students are immersed in a rich entertainment media environment that has been transforming the ways in which they listen and respond to music. In a recent study Marian Dura notes how the powerful influence of film, music video, multimedia music software and video games has ‘created a new popular view of music as a visual, as well as aural, experience’ (Dura, Reference Dura2002: 3). Dura calls for a music education that offers learners a ‘multi-domain experience of music’ involving ‘bodily-kinesthetic and visual-spatial’ capacities (ibid: 2). In this article I discuss a cross-media approach to classroom listening and analysis activities. Cross-media listening takes into account students' screen engagement and increasing screen literacy as well as their physical responses to music; it considers music from (variously) aural, visual, spatial and kinaesthetic orientations. More specifically, cross-media listening centres on visual musical works, pieces from moving image genres – short films, feature film extracts, trailers, television commercials, music videos, Flash and other animations, even home made videos – that are structured according to the codes, materials, devices and processes of music (see Vernallis, Reference Vernallis2004: 291, n1).

My research for the article has involved identifying a repertoire of such pieces which, for the purposes of classroom study at various levels, both effectively integrate music and moving image and offer opportunities for participatory performance-related activities. I begin by referring to the broader musicological context of cross-media listening and provide a working definition of a visual musical work. Next, I outline my theoretical framework and analytical methods, which are derived predominantly from music video studies and popular music studies, and I briefly describe how I adopt and adapt Patricia Campbell's (Reference Campbell2004) three-phase ‘listen to learn’ pedagogical model. I then turn to a detailed discussion of specific visual musical works as employed in actual cross-media listening and trialled in my own recent secondary and tertiary teaching. To conclude I propose ways to explore cross-media listening further and look at some of the implications of the approach.

Merging images with music opens up new possibilities of meaning and invites alternative musical interpretations. As the title suggests, YouTube and other video sharing sites are assisting in the proliferation and dissemination of such material, and can, I believe, be a source for the stimulation of creative and artistic activity from which music education could benefit. It has been noted that visual material accompanying music ‘may direct audiences to certain meanings and not others’ (Negus, Reference Negus1996: 92). For this reason cross-media listening should build on and supplement, not replace, other kinds of structured listening. Cross-media listening explores ways to expand common practice listening activities with the intention of enlivening the experiences and enriching the meanings of music for students in classroom settings.

Multi-domain experience of music

Nicholas Cook (among others) has recently advocated an alternative musicology that emphasises the performative dimension of music. Such a musicology rehabilitates performers and listeners in the division of musical labour and identifies a ‘continuum between experiencing music as process and as product’ (Cook, Reference Cook2001: 4). Cook explains that this musicology emerges (in part) from the ever expanding repertoire that now falls within a musicological purview, which both results from and contributes to ‘contemporary trends in multicultural education’ (ibid: 2). Not unrelated is Dura's point that ‘the current interest in multicultural [and I add popular] music education suggests the need for an understanding of views of musical experience where movement is a much more integral component than in Western concert-going and classroom listening contexts’ (Dura, Reference Dura2002: 3).

In his work on musical multimedia analysis Cook points out that music is never ‘alone’ and he proposes a general theory of how music works together with other media including words, images and dance to create multimedia (Cook, Reference Cook1998: 268). Cook details how the rise of the public concert led to ‘narrowing of the doors of musical perception,’ a process completed by mechanical reproduction (radio, long-playing records, CDs), which ‘filters out everything except the sound’. ‘Social practice,’ he writes, ‘has worked overtime to put back what technology has taken out’ (ibid: 266). Cross-media listening can contribute to music education's growing awareness of and responses to such practice and encourage interdisciplinary study in schools with film and media studies programmes.

Cook writes of the discrepancy between listeners' experiences of music and the way it is described in theoretical terms. He contrasts ‘musical listening’ (everyday, effortless, non-analytic listening) with ‘musicological listening’ (sophisticated ways of imagining music that rely heavily on metaphor) (Cook, Reference Cook1990: 152). Both kinds of listening are valid; both can provide genuine, pleasurable experiences of music (and inversely, neither kind of listening guarantees such an outcome). It is also important to consider in this context Kassabian's thinking on the new phenomenon of ‘ubiquitous listening’, listening which takes place simultaneous to other activities and involves ‘sequences of music neither chosen by their listeners nor actively listened to in any recognizable sense’ (Kassabian, Reference Kassabian, Hesmondhalgh and Negus2002: 131). Kassabian draws on Stockfelt (Reference Stockfelt, Kassabian, Svendsen, Schwarz, Kassabian and Siegel1997) who argues that ‘modes of listening, listening situation, and musical style co-produce each other’ (Kassabian, Reference Kassabian, Hesmondhalgh and Negus2002: 133), that is, how you listen and where you listen ‘conditions the music itself’ (Stockfelt, Reference Stockfelt, Kassabian, Svendsen, Schwarz, Kassabian and Siegel1997: 133). In designing classroom listening activities we do well to consider as significant the listening competencies that students acquire through various everyday learning processes (see Stockfelt, Reference Stockfelt, Kassabian, Svendsen, Schwarz, Kassabian and Siegel1997: 132).

Simon Frith emphasises that the different musical activities of listening, performing, dancing and composing ‘produce different aesthetic objects’ (Frith, Reference Frith1996: 267). Further, in relation to listening, Frith explains that the

Classroom music analysis activities involve discovering the meanings of musical works, and as Frith notes, ‘to ascribe meaning to a musical work is to provide a certain sort of description’ (Frith, Reference Frith1996: 263, emphasis in original). David Brackett refers to music analysis as describing ‘details of musical style’ and he sees the meta-language of musicological analysis as a resource to enrich popular music studies (Brackett, Reference Brackett, Hesmondhalgh and Negus2002: 65). ‘Explanations of why music sounds the way that it does’, Brackett writes, ‘are not tangential to aspects of experiencing music that non-musicologist, non-musician listeners find meaningful: sonic patterns have social ramifications’ (Brackett, Reference Brackett, Hesmondhalgh and Negus2002: 79). With cross-media listening, a particular kind of (filmic) description or explanation is already ‘built in’ to the visual musical works selected for study. We will now turn to a brief discussion of the formal and stylistic qualities – the aesthetics – of visual musical works (see Fraser, Reference Fraser2005: 28).

Aesthetics and the visual musical work

Carol Vernallis' theorising and analysis of music video in her study, Experiencing Music Video (Reference Vernallis2004), is apposite in relation to visual musical works, which as already noted, are pieces drawn from a range of moving image genres including music video. The music video differs in intention from the visual musical work (except where the latter is a music video) since, as Vernallis specifies, it ‘must sell a song’ (see also Goodwin as quoted by Fraser [2005: 8]). Visual musical works, like music videos, variously employ narrative and non-narrative visual structures, depict musical performance, dance, and other forms of performance spectacle, as well as incorporate elements of advertising. The image content of visual musical works ranges in degree of realism or abstraction, and the ‘genre’ incorporates a wide range of musics including art music, jazz, popular music, folk music, indigenous music, and experimental music (see the Appendix).

Unlike the ways films usually employ music, visual musical works derive directly from and are strongly responsive to the music they set, that is, the music determines the images designed by the director (see Vernallis, Reference Vernallis2004: x; xi). The car ride scene at the opening of Gus Van Sant's film, Gerry (Van Sant, Reference Van Sant2002), or the ‘Victor's piano solo’ scene from Tim Burton's animated feature, The Corpse Bride (Burton & Johnson, Reference Burton and Johnson2005), can become visual musical works since they operate as kinds of music video. Like music videos, visual musical works reflect the music's structure and ‘pick up on specific musical features in the domains of melody, rhythm and timbre. The image can even be seen to imitate sound's ebb and flow and its indeterminate boundaries’, Vernallis explains, and it ‘gives up its autonomy and abandons some of its representational modes. In exchange, the image gains in flexibility and play, as well as in polyvalence of meaning’ (Vernallis, Reference Vernallis2004: x).

As Pete Fraser explains, ‘how the camera is used and how images are sequenced has a significant impact on meaning’. In the study of music video, he continues, ‘camera movement, angle and shot distance all need to be analysed’ (Fraser, Reference Fraser2005: 29). Vernallis details how in music video, editing, camera movement and camera placement ‘enable relations’ between the music and the images (Vernallis, Reference Vernallis2004: 28). While an understanding of such technical and aesthetic elements is critical to the construction and interpretation of musical meaning in visual musical works, it is not possible to undertake here a detailed discussion of the grammar of shots and edits as used in film and music video. Vernallis provides such a discussion that is invaluable to greater understanding of the potential of visual musical works in cross media listening contexts. She demonstrates how music video editing can ‘elucidate aspects of the song, such as rhythmic and timbral features, particular phrases in the lyrics, and especially the song's sectional divisions’ (Vernallis, Reference Vernallis2004: 27). Vernallis makes a further point that the way we listen to the music ‘shapes the way we perceive the image’, but equally, ‘what we attend to in the image helps to determine how we hear the music’ (Vernallis, Reference Vernallis2004: 51). When music is coherently merged with moving images the result can be much more than a way to ‘sell’ the music.

In cross-media listening visual musical works are considered within a broader instructional framework. Patricia Campbell's ‘listen-to-learn’ pedagogical model as articulated in Teaching Music Globally (Campbell, Reference Campbell2004: 54–55) is ideal, being especially designed for holistic listening experiences. Before providing analyses of particular visual musical works and suggesting ways these can be approached in the music classroom, I will outline how Campbell's model can be adapted to accommodate cross-media applications.

A framework for cross-media listening

Campbell proposes that in the context of music education (where an individual's ‘valuing of music as an important part of daily life’ is deepened) there are ‘three central features that help to develop a sound awareness of music – instruments [including voices], elements, and context’ (Campbell, Reference Campbell2004: 54). These are ‘folded within instructional processes that lead to ‘attentive’, ‘engaged’, and ‘enactive’ listening’ (ibid: 55). Campbell's model strives to lead students into ‘deeper levels of musical understanding’ through exploration of the ‘explicit features’ of various musics as well as discovering the ‘contexts in which people make music meaningful and useful in their lives’ (ibid: 55).

In the first or Attentive phase of Campbell's model, listening is focused on ‘musical elements and structures’ and is ‘guided by the use of specified points of focus or diagrams (including notation and ‘maps’ that lead a listener from one musical event to the next)’.Footnote ¹ The second, Engaged phase requires ‘the active participation by a listener in some extent of music making’ (such as singing or playing a part of the music, or ‘moving eurythmically or in an actual dance pattern’ while the music, recorded or live, is sounding). The third, Enactive phase involves ‘intensive listening to every musical nuance’ as part of recreating the music in performance ‘in as stylistically accurate a way as possible’ (Campbell, Reference Campbell2004: 55).

Jøran Rudi, a composer of visually arresting computer music video works, points out that ‘new possibilities for narrative arise when other forms of representation and expression are added to sound’ (Rudi, Reference Rudi2005: 36). In cross-media listening, such ‘narratives’ extend the principle of the ‘graphic devices’ employed in the Attentive and Engaged listening phases of Campbell's model which are employed as ‘ancillary techniques for getting to the heart of music's structures and meanings’ (Campbell, Reference Campbell2004: 91). Further, the image content of a visual musical work prompts imaginative thinking and provides ideas for participatory responses related to those proposed in Campbell's Engaged listening phase.

Cross-media listening, then, is an approach to music comprised of two phases closely related to (and which can articulate specifically with) the Attentive and Engaged phases of Campbell's model. While the order of these phases is reversible depending upon the actual media work involved, the more ‘cognitive’ approaches to listening and viewing might better be undertaken after at least some sort of purposeful physically participatory approach to a given visual musical work has been explored.

The style-structure-context phase involves looking at and listening to the visual musical work. Preparation for such viewing can be in the form of thoughtfully considering analytical descriptions of selected elements of musical style, structure and context, such as those provided for my examples below (or have some of these pointed out during a stop-start viewing after the work has been seen right through at least once). Alternatively, students may discover some of the aspects analysed unaided, and teachers may choose to guide conversations about the work under consideration in the light of these analyses. Some analytical aspects are relatively sophisticated however, relying on a technical vocabulary, and will certainly need to be explained to students. Such details and insights should make room for further observations and speculations by students and teacher.

The performance participation phase involves some form of imaginative participatory ‘response’ to the works and suggestions for these are provided below as starting points. With Campbell I commend such ‘multiple encounters’ with music since they ‘necessitate the careful and conscious listening that is far beyond the passive, armchair style of consumer listening’ (Campbell, Reference Campbell2004: 92). Thoughtfully creative, performed responses require thorough familiarity with the music's structures, devices and processes. Marian Dura proposes developing ‘entire curricular units based upon multisensory integration’ which could teach students how to use ‘kinesthetic, visual, and other metaphorical referents to increase music vocabularies and enhance listening experiences’ (Dura, Reference Dura2002: 257). Visual musical works often bring such referents to the fore and provide ideas for physical involvement. In the context of its simple narrative that provides dramatic tension, the 2001 Swedish short film, Music for one apartment and six drummers, for example, prompts a wealth of ideas for rhythmically exploring beats and grooves on household, ‘found’ objects.Footnote ²

Such units as Dura proposes might include: making computer animations, short music videos, shadow plays or flip books; choreographing movement sequences (or seeking inspiration from rhythmic gymnastics or drills from games such as basketball or football); building Lego or other models to visualise musical structures; creating and playing ‘interactive scores’ of themes from Mozart's Symphony No. 40 by roller blading past water tuned bottles,Footnote ³ or learning to juggle-perform music (Chris Bliss' virtuosic, musically ‘juggled’ interpretation of the Beatles' songs, ‘Golden Slumbers/Carry That Weight/The End’ is inspirational to this end).Footnote ⁴

Yo-Yo Ma's Inspired by Bach film series offers another direction to pursue, involving as it did such creative enterprises as designing and creating a garden embodying aspects of the structure of J.S. Bach's Suite No. 1 for Unaccompanied Cello. Yet another idea is to capitalise on students' growing interest in foley, the practice of recreating and recording all of the sounds for a film – atmospheres, sound effects, actors' movements and dialogue. An ensemble such as The Spaghetti Western Orchestra have turned foley into a performance art form and inspiration for classroom work can be found in a video of one of their performances at http://www.ennios.com/swo.htm. Students could create, for example, a foley performance interpretation of the soundtrack to Chris Gavin's short film on evolution, Evo (Gavin, Reference Gavin2006).

In the description and analysis of specific instances of cross-media listening that follows I am suggesting a framework for deepening conversations about music in the classroom, rather than providing a schema with predetermined, finite analytical conclusions or prescribed ways to participate in performance. Analytical observations and findings and participatory involvement should ideally arise from specific classroom contexts. I have endeavoured to select works that meet various levels of competence and which represent contrasting music styles and moving image genres (the Appendix lists further works for exploration).

Cross-media listening in practiceFootnote ⁵

1. 1. TV commercial/music video – ‘Vertigo’ by U2 (2004). iPod version and Temple Bar Mix.

Conclusion – music analysis down the (You) tube?

In this article I have proposed ways that music education can benefit from considering new modes of listening that are becoming prevalent globally. In increasingly diverse and imaginative forms, sound and music is being aligned with moving images and being shared via sites such as YouTube. Cross media listening can contribute to the ongoing process of reconceptualising music education. The emerging field of music video studies is a valuable ally here and together with film music studies will become increasingly more significant to contemporary music education.

The application of Patricia Campbell's listening model to reconfigured musical forms such as the visual musical work, and the incorporation of extended classroom applications such as those suggested, can assist in shaping music education into a multi-domain experience. Ultimately there may be some implications for teacher education, for example, in order to make viable a full dimensioned performance participation approach such as is recommended in the article, some classes might be better taught utilising the workshop approach of drama and theatre arts teachers. In addition, for competent teaching of visual musical works in the classroom, some theoretical knowledge of film, video and other screen forms is vital.

One area for development is in the adaptation of examination and assessment approaches to take into account visual musical material. Indeed, my own cross-media thinking about music gained impetus from the process of creating with a colleague an experimental aural/musicology exam for upper secondary school students that incorporated what we called at the time, visual listening. With such an approach it is possible, without great difficulty, to incorporate into the teaching of elements of musical style and performance, aspects of performance practice, music transmission processes, and other kinds of related social, cultural and historical knowledge.

Several years ago in this journal Leif Finnäs identified a need to

Such a study is certainly compatible with the aims and concerns of cross-media listening, that is, to enhance students’ musical experiences and to seek ways to prise back open the doors of musical perception (Cook, Reference Cook1998: 266). By drawing on the non-formal ways in which students listen and respond to music, cross-media listening can assist in making classroom music analysis a more multidimensional, imaginative and vital educational experience.