2. ON NETWORK TOPOLOGIES

In a recent review of some of our research prototypes, we introduced the notion of interaction topologies (Matuszewski, Schnell and Bevilacqua Reference Matuszewski, Schnell and Bevilacqua2019). Starting from the point of view that music is a collective experience, occurring either as co-located or dis-located in time and space, some of the research effort has concerned the design of mediation technologies (Leman Reference Leman2007). As stated earlier, in the context of this article, collective experience is understood as a series of ‘actions’ emerging between human and non-human ‘actors’ (to be understood as in the Actor-Network Theory).

The use of web technologies, as a means to investigate these relations between all these actors, is critical in many of the discussed projects because each audience member has a device in their pocket that can communicate over a network, as well as generate, receive and display information and media through a web browser. This choice means that no additional application needs to be downloaded to the audience member’s device, which may not have sufficient free space, or the user may not accept the possible security risk. It also means that the bar for access is minimal as all phones/tablets have a web browser and can connect to Wireless Local Area Networks (WLAN) through Wi-Fi.

For the purpose of the works discussed in this article, a dedicated ‘Local Area Network’ (LAN) is often created as a wireless local network that is not connected to the outward facing internet. These ‘technological networks’, considered as mere tools, are used in order to contribute to our interaction design aims. Importantly, there is a meaningful mismatch between the actual ‘technological network’ links and nodes and a more global network describing the complete map of interactions that could be described as a ‘perceived network’ among the actors.

This is of course similar to what happens when interacting with communication technologies, which succeed in making the physical information transmission invisible to the user(s). For example, exchanging messages with another person is perceived as a two-person interaction. The segmentation of the signal into packets travelling through several servers, maybe taking non-trivial routes, is hidden and unperceivable to the users. Only the general transmission quality might be perceived, but no agency is provided to understand where physically in the world the bandwidth bottleneck may be. At best, an informed guess, based on knowledge and experience can provide an answer.

This gap between the physical communication network and the perceived network of interaction is important and must be fully recognised. As researchers or artists (or both) in music, we have unique opportunities to act on both levels using our own built system(s), creating creative ecosystems that may not be possible with commercial, closed solutions. We design and create networks and interaction considering both perceptual (experience) and technical points of views. Thus, two types of network topologies should be considered: the technological ones and the ones describing the human/communal experience. The technological network does not represent a goal in itself, it is only developed and set up to enable musical experience. This experience is the result of a complex set of interactions between humans, objects and sound. The complete topology of such an experience is mostly an emerging property, based on a set of material, rules and human (action-)perception. Several experimental stages are necessary to observe and grasp the full emerging properties, constantly making adjustments at various levels, human and technological.

3. EXAMPLES

The following examples presented all rely on a technological framework, which originates from a collaborative project with several institutions called CoSiMa ^{Footnote 2} and which enabled us to develop collaborative and collective interaction based on mobile and web technologies (Robaszkiewicz and Schnell Reference Robaszkiewicz and Schnell2015). The formats we explore include interactive sound installations, augmented reality experiences, participatory concerts, flash mobs and collective performances.

The possibility to use web standards such as HTML/CSS, Web-Sockets, Web-RTC, Web-GL, Web Audio API as well as Geolocation, DeviceOrientation and DeviceMotion enable implementation of complex network systems, which can be deployed either on local networks or on the internet. Precisely, the deployment of prototypes and pilots as web applications allows for instantaneously publishing applications worldwide as well as for very rapidly setting up experiences on local networks, inviting audiences of virtually any size to participate. More recently, this technological framework has been generalised to gather large ensembles of mobile and embedded devices, such as smartphones/tablets, microcontrollers and nano-computers (e.g., Raspberry Pi) (Matuszewski Reference Matuszewski2020b).Footnote ³ Importantly, it also exposes the possibility of sharing a common clock and thus synchronise these heterogeneous devices, which is of particular importance for several music contexts (Lambert, Robaszkiewicz and Schnell Reference Lambert, Robaszkiewicz and Schnell2016). Various developed open-source JavaScript libraries have been expanded beyond the initial CoSiMa project and are currently available. ^{Footnote 4}

4. On participation and mutual listening

As is illustrated in the preceding examples, over the last decade we have experimented with interactive and participative approaches of sound installations and collective concert experiences in which the audience is actively involved in the production and induction of digital sound and imagery. In some experiences, participants interacted with digitally enhanced machinery or everyday objects, in others they just pulled their smartphones from their pockets to participate. These experiences ranged from concerts, where the sound of solo musicians had been rendered through the audience’s smartphones, to collective improvisations where nobody but the audience collectively performed on web-based networked instruments. All these experiences had in common that – as is true with other interactive and participative works – they were designed with a particular attention to the actions of instrumented participants.

It may seem justified to argue that these experiences are particular because the audience here is more actively involved than in traditional concert performance practices of Western culture. One may claim that while traditionally the audience is ‘just listening’ without being involved in the production of the music itself, in participative works, the audience is ‘really’ and ‘actually’ involved in making the music.

However, after many years of creating experiences that merit their classification as ‘interactive’ and ‘participative’ due to their technical set-up, we have not succeeded in clearly distinguishing the previously mentioned ‘active’ participation of an audience from any other kind of participation. In general, the audience’s participation in a musical experience is manifold and hard to quantify, and this distinction does not seem to sufficiently describe the complex entanglements of the interactions involved – neither actually nor metaphorically.

The crucial difference, participation, introduces to the composition as well as the reception of the works a fresh and different perspective: it affords a significant shift of attention from strictly metaphorical or exemplary interactions musical works usually display towards actual interactions that involve the participating recipients.

Christopher Small, in the introduction of his book Musicking: The Meanings of Performing and Listening, argues that the ‘meaning of the act’ of making music lies in the relationship ‘between the people who are taking part … in the performance’ (Small Reference Small1998). From this point of view, a different degree of activity in an audience’s participation does not seem to make a notable difference. For instance, how do we justify the significant effort of instrumenting audiences (performing, listening, gesturing, being consciously part of an ensemble) as significant interactions beyond ‘just listening’.

However, within the context of networked music-making, Small’s argument helps us notice the different direction in which the listeners’ attention is focused and the meaning therein. In the participative approaches, instead of listening to featured musical performers, the attention of the participants (the audience) is turned towards each other and to the collective they form. Each participant is listening to the others and this mutual attention is a critical distinguishable quality of these approaches and reflects the deeper meaning ‘of the act’ – actually and metaphorically.

Even in the case that not all members of the audience participate with the same skills or commitment, or where the participants may be assigned different roles in the performances, they are not participating in completely different categories, as would be the case for performers and the audience at a classical or rock concert, for example. Some of the experiences we designed featured performers who were clearly distinguished from the audience (e.g., on stage). In these experiences, the audience’s contribution could be partly understood as dialoguing with the performer(s), partly as supporting their performance. Even if these two functions – or attitudes – may essentially be the same as for any other performance practices involving an audience, the experiences required and allowed for an enhanced mutual attention of all participants, individually as well as collectively.

An additional factor that enhances the perception of participation in many experiences we have designed is the distribution of sound within a physical space through a network of sound generators formed by the participants. The participants can usually clearly perceive the sound produced by their own device and the sound of individual participants around them while being immersed in the sound collectively produced by all participants together. The presentation of sound through tens or hundreds of devices produces a unique spatiality that the audience is immersed in and collectively represented by, namely, a soundfield that has remarkable depth and mass.

Ultimately, the perspective of participation also applies to works designed for interacting devices such as Biotope and Poly briefly introduced previously. Even if the works’ interactions do not involve the recipients in the same way, they instantiate a very similar metaphoricity featuring the idea of mutual attention.

Involving either people or digital agents, or both, works that instantiate participation and mutual listening open new avenues of expression to composers and designers. While some may reject the idea that the musical outcome of each performance depends on the participants’ interactions, others have embraced this idea as a desired factor of the overall creative process.

5. SUMMARY

Designs such as those discussed previously break down many of the traditional expectations of a musical concert presentation or art installation. The recipients actively participate in its realisation. In some works, this could be undertaken at any time of the day and in any place across the globe. The traditional, formal structures of presentation, curation and hierarchies of performer/artist and audience are therefore largely re-framed.

The ad hoc organisation whereby each individual becomes a formational agent also implies that anyone, regardless of existing skill or training, can become a musical agent, interacting with other musical agents in an act of mutual attention. Simultaneously, this individual experience is a communal one, forming a shared objective, and of course a shared experience of being both an individual creator and a community of actors and listeners.

It can therefore be argued that these changes of focus produce different experiences of artistic creation and performance, in forms of both presentation and engagement. The network then becomes the key medium that acts as an interaction platform supporting individual as well as collective expression and mutual attention. Interaction, here, is more than a set of metaphorical relationships displayed in sound on stage, as it is often the case in common musical practices, but it is actually instantiated through the recipients’ own participation. Accordingly, the metaphoricities implied by such experiences are shifted from representations to embodied relationships acted out and mediated through the network.

Therefore, this type of work goes much further than replicating more traditional performances over networks, a format we saw greatly increasing in prevalence during the 2020–21 COVID-19 crisis. Artworks transmitted through network technologies, mainly used as ‘communication channels’, have often been reported as unsatisfactory in terms of collective experience, reproducing only very partially onsite interaction. The works presented here are fundamentally based on different approaches, considering the physical network as material in a complex and interleaved human(s) and object(s) interaction. Importantly, in the cases we considered, the collective interactions are designed as situated interactions, occurring in a shared physical space. As advocated here, these complex situations, superimposing social and technological interactions, objects and digital media, should be described and comprehended from a holistic perspective.