What is the framework for the embodied and mediated voice through sensor interfaces in human-computer interaction? This is a key question in Franziska Baumann’s book Embodied Human-Computer Interaction in Vocal Music Performance. The author explains the answer in the introduction:
When a vocalist performs on stage with technology such as gesture-controlled live electronics, an expanded field of connections between body, gesture and mediated sounds is generated. What we see and hear is no longer based on the traditional understanding of sound linked to kinetic energy, such as a violin bow moving across a string. Instead, we are exposed to a holistic experience in which the boundaries between the human voice and the computationally manipulated voice are blurred. Thus, the relations between the voice, the visible gesture and the sound associated with it can no longer be recognized. (1)
This offers the reader a brief description of human-computer interaction. On the one hand, there is literature on gesture sound-mapping and motion capture techniques, but there remains a gap in the literature regarding how vocalists interact with gestural interfaces. On the other hand, although analysts have focused on excitement and modulation, providing definitions of sound-producing gestures and their functional aspects, a closer understanding of the body’s gestures is needed. The book aims to contextualise vocalists and the various aspects that arise in interface-based performance practice and explains the creative practices between computers and humans in live music performance. This is a framework grounded in an embodied relationship within such systems. Thus, the question is, ‘What is the framework for the embodied and the mediated voice through sensor interfaces in human-computer interaction?’ is explored through the underlying and mediated practices, sensor technologies and principles that occur in embodied human-computer interaction in music performance. Overall, Franziska Baumann’s contribution to enriching the phenomenon of vocal performance enhances our understanding of the interaction among performers, music and the audience, as well as the role of technology.
Comparing Baumann’s book to other contributions in the field of performance and music technology reveals both parallels and distinctions. Firstly, there is a parallel between Iris Garrelfs and Jan Kirby’s (Reference Garrelfs and Kirby2025) Embodied Dialogues: An Improvised Duet between Sensor Dress and Voice and Baumann’s; as both examine the interactions between movement, technology and voice. Practical applications and case studies are central to Baumann’s work and provide important insights into integrating technology into vocal music performance, whereas this is a gap in books such as ‘Introduction to Computer Music’ by Nicolas Collins (Reference Collins2010), focusing on performance and music composition technologies. A distinction can be drawn between Alex Ross’s (Reference Ross2007) The Rest is Noise and Baumann’s, in that Ross focuses on the evolution of 20th-century music and omits the vocal music performance that Baumann presents as a contemporary phenomenon, including its physical aspects. Furthermore, Baumann’s book addresses contemporary issues in vocal performance, making an important contribution to the literature on human-computer interaction; the case studies and the contemporary user-centric designs are key variables for technologists, musicians and any reader interested in vocal music performance.
The three chapters, including a short introduction to Embodied Human-Computer Interaction in Vocal Music Performance, cover aspects of meaning, practical experiences and the technical and phenomenological intersection of mediated sound, gesture and body. These are based on Baumann’s research and practice, as well as the work of various researchers and vocalists. The author draws on 20 years of performances and concerts around the world and practice-oriented research, specialising in electroacoustic composition and sensory live electronics in contemporary vocal performance. This has clearly informed this valuable resource for musicians, composers, researchers, vocal performers, human-robotic researchers, dramaturgs and actors, as well as for those engaged in the theory and design of new instruments for musical expression.
Chapter 1, ‘The Embodied Voice’, refers to the acoustic voice but also the multivocal voice, which is not a conventional definition of vocal performance but a voice with a globalised character, employing extended techniques that emerged as singing transformed by moving away from European traditions. In a contemporary environment, the extended use of the voice, stylistic and ethnic colours, Western bel canto and vocal utterances are treated equally. Baumann considers these to be at the same hierarchical level for vocal expressions, arguing that the multivocal voice represents a communication platform beyond cultural idioms (9). Baumann also makes connections with the Dada movement, the non-idiomatic voices of female American vocal performers in the 1960s and Pierre Schaeffer’s and Denis Smalley’s developed concepts of listening. Thus, in this chapter, the focus is on abstract listening with a spectromorphological ear, in which the listener has a different perception from that of the composer-performer-improviser and the vocal sound is detached from its sound source. Hence, the process involves blocking the instrumental-extrinsic threads to focus on sound qualities and refining spectromorphological details. Thus, the mediated voice becomes an abstract vocal terrain that the author examines in the next chapter.
Chapter 2, ‘The Mediated Voice’, draws on images from abstract vocal terrain, myths and everyday disembodied voices, such as speech assistants and telephones, and conjures imaginary bodies for the disembodied voices surrounding us (6). The mediated voice forms separate from the body, where the ‘I’ becomes ‘From Me’ and the 1st person becomes a 3rd person. The author uses interviews to trace a discussion of how meaning is generated in embodied human-computer interaction. Key areas of the questions focus on how practitioners phenomenologically implement individual aesthetics with their technology and on the practitioner’s aesthetic motivation, as well as questions such as ‘What are the qualities of their gestures assigned to sound, what is their experience with embodiment and voice, what role do their props play?’ (6). Thus, in Chapter 1, key areas of discussion included reduced listening to radical musical vocality as well as the potential of spectromorphological analysis, whereas in this second chapter, the more abstract vocal terrain of the mediated voice is discussed.
Chapter 3, ‘Embodied Interface Performance with Gestural Systems’, seems the most interesting to me because it offers readers the opportunity to discover ways to compile and compose strategies for creativity between computers and human embodied interaction. Baumann presents several perspectives on the embodied and the disembodied voice, for example, between the ‘I’, the ‘From Me’ and the ‘Clones of Myself’. These examples include aspects such as dress, wearables, interface appearance and gender, which affect the sonic aesthetic. This chapter seems to provide the best practical examples of sound and the gesture vocabulary involved, as well as the creation of a feedback loop of embodied perception. This phenomenon is then constantly reorganised in our perception of what we hear and what we see. The chapter offers an appropriate example of the embodied practice of seven co-players and a visual metaphor with seven limbs, which creates meaning in interaction. This offers an important and useful method for creative practitioners, musicians and composers in their understanding of embodied human-computer interaction in vocal music performance, as well as importance for voice-model and artificial machine-intelligence researchers and human-robotic researchers who would like to develop gestural systems with a more targeted focus on embodied interaction approaches. Overall, Baumann offers an excellent balance between theory and practice in this chapter.
Baumann’s book contributes significantly to human-computer interaction, a perspective that aims to link artistic expression and technological innovation. The investigative process implies the detachment of the sound source from the vocal source, a process in which instrumental-extrinsic threads are set aside to redefine spectromorphological details. This complication highlights the challenges of the abstract, relatively objectified process of a microscopic, intrinsic listening (23). The author connects an abstract vocal terrain to the mediated voice, whose four domains, linguistic, affect, identity and articulation, are never clearly separated but mutually dependent and whose boundaries are sometimes difficult to delineate. This complication is not necessarily a weakness; the author emphasises that, whether the vocal utterance is spoken, sung, experimental or some other vocal type, each of the four domains can highlight meaning in relation to the others. The author draws a parallel between the machine and the human, in which the machine and the performer form a unified process to express music and vocal sound. Thus, culturally diverse designs for performers could be implemented to eradicate outdated body images. Baumann cautions that if men and women and diverse beings compose, design and physically perform gestural systems, they should be responsible for the body images. ‘Designers need to envision an alternative future to replace outdated and dismantled social conventions’ (Mainsbridge Reference Mainsbridge2022).
The book could be further developed, as emphasised in the book’s last paragraph: ‘We are not moving towards a solid new practice. I rather feel that we are moving away from the predominance of linear concepts between movement and sound towards multiple new embodied human–computer interactions and inherent meanings waiting to be explored’ (79). This book is an interesting and complex volume grounded in the theory and practice of vocal music, linked to the history of multivocal voice in Chapter 1, to the electrified space of vocal expression and vocal identities in Chapter 2 and to human-computer interaction as co-players in the creation of meaning in Chapter 3. This book is not just about music performance but offers a fascinating narrative that reflects on the connection between technology, cultures, society and art.
