1. Introduction
The motivation behind much of the current laptop-centred networked performance seems to be the construction of enhanced musical relationships within a system comprising performers and instruments. Often, due to the logistics of performing with laptops, where information is displayed on a sizable screen, and the laptop is usually placed on a table along with peripherals, such as soundcards and controllers, the scope to facilitate gestural anticipation, recognisable visual cues, or meaningful physical movements is much more reduced than with performances using traditional instruments. As Seddon observes, ‘when jazz musicians play together they have at their disposal verbal communication, non-verbal communication (e.g. eye contact, aural cues and body language) and musical communication’ (Seddon Reference Seddon2005: 47). When a percussionist hits a drum, the other performers and the audience have a clear idea of the causal agency between the action and the resultant sound. However, in the field of electronic music, and moreover laptop performance, it becomes more difficult to rely on physical gesture to convey, for example, the onset of one's sound to an audience, or to communicate with other performers. This may be due either to the nature of the interface being used – in the case of mice, keyboards or other devices involving micro-movements of the hand – or to the complexity of the sonic outcome, where it may be unclear how a sound has been produced, or indeed who has made it.
1.1. Networked music performance
Along with recent technological possibilities, these issues of performance and communication have driven musicians and sound artists increasingly to explore various types of networked music performance (NMP). A large portion of this research deals with high-quality uncompressed audio streaming. For example, the SoundWIRE project, at CCRMA, Stanford, examines methods of creating networks over the Internet as a means of extending the realm of computer music performance. In fact, one particular concert spanned a geographical distance of over 6,000 miles (Cáceres, Hamilton, Iyer, Chafe and Wang Reference Cáceres, Hamilton, Iyer, Chafe and Wang2008). Such NMPs were emulated in the Apart Project, undertaken at the Sonic Arts Research Centre, Belfast, in order to ‘better understand conditions for performance that are created, facilitated and suggested by geographically displaced network performance environments’ (Schroeder, Renaud, Rebelo and Gualdas 2007). Various scenarios were constructed in which performers in dislocated situations received audio and video feeds of each other, both with and without latency, so as to help better understand the complex effects of musical cues. Whilst thoroughly technically descriptive, this project clearly alludes to the power of networks in relation to social concepts, such as community. Moreover, it concludes that rather than trying to recreate that which occurs on the stage, one should ‘rather take advantage of the network itself as a medium for performance’ (Schroeder et al. Reference Schroeder, Renaud, Rebelo and Gualdas2007: 139).
This idea of using the network as an agent for performance has been realised with the emergence of numerous laptop orchestras, such as PLOrk, based at Princeton University. Using local wireless networks and focusing on data transmission of parameter and timing control, rather than audio-streaming (although not precluding wired audio networks), this group has developed strategies for performing with laptops and localised speakers by applying techniques for real-time synchronisation, cueing, scheduling and non-bodily visual communication (Trueman, Cook, Smallwood and Wang 2006). Here, the notion of connectivity within the digital realm is furthered by the introduction of a ‘conductor machine’. This can guide the piece by, for example, sending simple text instructions to the performers, or by directly affecting specific parameters such as tempo (Trueman et al. Reference Trueman, Cook, Smallwood and Wang2006).
Two further projects that draw on this idea of a virtual conductor, but remain within the realm of improvisation, should be mentioned. Anne La Berge and Robert van Heumen's duo Shackle consists of a local network between two laptops, over which a series of cueing commands are sent (La Berge and van Heumen Reference La Berge and van Heumen2006). These directions include ‘aspects of restriction, either in sound material, timing, dynamics or other musical parameters’ (La Berge and van Heumen Reference La Berge and van Heumen2006), and are presented in a somewhat abstracted form to the audience on a projection screen. Additionally, the players may skip past a particular state, if so desired. The ensuing performance presents the two musicians indeed shackled, but clearly toying with and struggling against the imposed restrictions. Similarly, external direction is given to a group of four performers in Eric Lyon's Selected Noise Quartets, where instructions are generated in real time and sent to each performer via a laptop screen (Lyon Reference Lyon2011). Here, it is noise itself that guides and creates the structures behind the improvisations; the performers must be able to react quickly to the often highly unpredictable changes. Again, a struggle may arise as the instructions are frequently unfamiliar, and may in fact be technically impossible to carry out. Yet, through all the exertion ‘the voice of each musician is heard; and behind it, the voice of noise’ (Lyon Reference Lyon2011:98).
1.2. The reintroduction of haptic sensation
The Selected Noise Quartets, which are performed on acoustic instruments and electric guitar, demand a great deal of dexterity from the players. However, it is arguable that, compared to what can be achieved with conventional instruments, the level of physical sensitivity and control required for such deftness is absent in digital musical instruments (DMIs). Most traditional instruments are constructed to be played with the mouth or hands, where the largest number of sensory receptors in the body can be found (Rovan and Hayward Reference Rovan and Hayward2000). Performing with these instruments provides the player with a wide range of physical forces and vibrations, which create an embodied knowledge about the nature of the sound being produced. Vibrations felt through a percussionist's hands from the mallets and through the legs from the bass drum pedal, as well as the bounce that the taut drum-skin offers, all inform the performer about, for example, the dynamic, timbre or shape of the sound that is being produced. Hence what is heard through the ears is supported by this physical feedback mechanism, which creates a closed loop of ongoing listening and sensing, playing and readjusting. This all occurs before, and whilst, making each subsequent sound. Thus, by introducing artificial vibrotactile feedback to DMIs, some attempt may be made to restore this vital sensory information.
Generally, interfaces for digital musical offer minimal haptic feedback. They rarely reveal to the performer any tangible information in themselves about the qualities of the sound being made. Working with specially designed haptic interfaces, such as Claude Cadoz's Modular Feedback Keyboard (Cadoz, Lisowski and Florens Reference Cadoz, Lisowski and Florens1990), physical forces, including resistance and pressure, can be carefully introduced to enhance our interactions within the digital realm. Furthermore, as the sensing nerves on our skin are capable of detecting extremely complex patterns of data (Gunther and O'Modhrain Reference Gunther and O'Modhrain2003), additional vibrotactile feedback can be added to DMIs by way of actuators, such as motors. Marshall and Wanderley, at CIRMMT, McGill University, measured the effects of embedding vibrotactile stimuli in DMIs, with varied results (Marshall and Wanderley Reference Marshall and Wanderley2011). They noted that while adding vibrotactile feedback may improve the feel of the instrument, the extra sensory load caused some participants to feel less in control of their playing. However, this may in fact be beneficial in terms of creating a challenging instrument that could be mastered over time. Indeed, it is generally accepted that haptic feedback can assist learning processes (Davidson Reference Davidson1976), and, as discussed above, it is undoubtedly significant in the role of building a performer's perception of sound. Moreover, this experience is uniquely private to the performer, forming an intimate relationship between musician and instrument.
2. Background
This section gives a brief contextual summary of the musical activity surrounding this project. The Networked Vibrotactile Improvisation System (NeVIS) arose out of a two-year-long collaboration between the authors, both composer/performers, combining piano and live electronics (Hayes) with percussion and live electronics (Michalakos). Both being practitioners of digital augmentation and hybridisation of our chosen acoustic instruments, we inevitably began to develop strategies that attempted to tackle some of the issues related to performing with augmented acoustic instruments in a collaborative environment. We also both engage in a variety of extended techniques. To give some information about the systems being used: typically, the acoustic percussion or piano sound is amplified, and is also converted to a digital signal for further processing. Analysis of the incoming sound occurs continuously, in real time, and the various parameters derived, including pitch information, density and dynamics, are used to drive assorted processes within Max/MSPFootnote 1 and Max for Live.Footnote 2 Additional controllers such as foot pedals, sensors and MIDI interfaces are often also employed; these provide hierarchical control over various parameters within the software. While our individual approaches have both favoured hybridisation that makes use of machine listening techniques, we have found ways to integrate these additional devices without losing any sense of flow or agility. In fact, being able to dynamically control the level of one's digital sound is something that we have both found to be a necessary feat.
The creation of the NeVIS project emerged both from ideas developed throughout our experience as the improvising duo Můstek (Figure 1), as well as through individual research exploring on the use of vibrotactile feedback as a performance tool (Hayes Reference Hayes2011). Involvement with large-scale ensembles, such as Edimpro (see Edimpro 2009), a free improvisation group consisting largely, but not exclusively, of students and staff from the University of Edinburgh Music Department, also raised questions about communication strategies within group improvisation. Further influence came from participation in workshops dedicated to the performance of electronic music and improvisation, hosted by, notably, Fred Frith and Christophe Fellay, and the Converging Objects workshop by Anne La Berge and Robert van Heumen (Reference La Berge and van Heumen2010). Lastly, both authors partake in the yearly roving-researcher-led Laboratory for Laptop and Electronic-Audio Performance Practice (LLEAPP), founded at the University of Edinburgh in 2009 by members of Sound Lab Edinburgh (2007), with support from the Roberts’ Fund for Researcher-Led Initiative; this workshop aims to highlight and tackle many of the issues related to the live performance of electronic music (see LLEAPP 2009). Now in its third year, LLEAPP provides a framework for collaboration and discussion among postgraduate researchers and music-makers from around the UK.
Můstek performing Socks and Ammo at Sonorities Festival, SARC, Belfast, 2011.
3. Motivations
The main threads that emerged from the various aforementioned scenarios and improvisations revolved around:
• strategies for structuring improvisations
• strategies for communication between performers
• novel interaction between performers.
One of the most exciting yet possibly troublesome aspects of group improvisation is that, rather than a single-person-led evolvement, ideas may be put forward by any agent present (Edwards Reference Edwards2010). Moreover, new material may be emergent, appearing only as a result of everything that has previously been put forward by the present assemblage. We began to consider new methods that might challenge these characteristics of improvisation, which appeared to be ubiquitous. Furthermore, after performing together for a significant amount of time, performers begin to predict or expect what their well-known partners might contribute in any given situation. On one hand, this is of course an advantage in long-term collaborations, as players become familiar with the sonic worlds of their peers; but, in some cases, it can also lead to a lack of spontaneity, or at least spark a desire for a freshness of sorts. In our case, introducing a third, unpredictable agent into the system was certainly something that was appealing, not only for the sake of newness, but moreover because we would be able to consider its role in the construction of the sound and musical form.
A further motivation for imposing this system was noted during the post-concert discussion of the LLEAPP workshop of 2010 at Newcastle University's Culture Lab. The improvisations of electronic music presented on that occasion generally seemed to settle on an average of fifteen minutes’ duration: as participants, it appeared that we were neither daring by performing extremely short works, nor confident in demanding more time, where needed. It was felt that different approaches to structure, in terms of duration, would have helped significantly in nearly all of the pieces presented.
Naturally, the role of structure within improvisation is a complex issue. Analogies have been drawn between improvisation and vocalising, either as monologue, emulating the flowing nature of singing (Sudnow Reference Sudnow1978), or as conversational dialogue (Healey, Leach and Bryan-Kinns Reference Healey, Leach and Bryan-Kinns2005). The latter types of comparisons suggest that a certain pattern of interaction occurs within group free improvisation, whereby one person provides a new idea and the others listen and respond to it, just as may happen with a new choice of topic in conversation. Analysis of group improvisational sessions seems to confirm this (Healey et al. Reference Healey, Leach and Bryan-Kinns2005). However, this phenomenon suggests a certain structure and development to the music: it does not readily facilitate synchronisation points or cues between players. Just as would not typically happen in conversation, a group of performers would not usually move synchronously to a set of new ideas, unless a clear cueing command was given. This was one of the main motivations for creating the imposed framework: these cue points would be suggestions to simultaneously move to new material, without implying a fully pre-composed piece. Just as Bavelas and Chovil give extensive evidence that certain ‘nonverbal acts are an intrinsic part of language use in face-to-face dialogue’ (Bavelas and Chovil Reference Bavelas and Chovil2006: 110), we claim that within musical improvisation, too, nonverbal communication is fundamental, and worth exploiting beyond merely the gestural domain.
While traditional visual cues continue to exist between performers using electronics, there is increasingly a need to find other solutions to some of the problems that arise when working with new technologies. Dislocation of the sound source and the loudspeakers means that stage layouts become complex and often confused, another point raised, and clearly evident, with regard to some of the performances at LLEAPP 2010, and subsequently at LLEAPP 2011, at the University of East Anglia, Norwich. Naturally, the origins of the acoustic sounds of augmented instruments are situated within the body of the instrument. Ideally, any approach to positioning loudspeakers should involve a conscious effort to integrate the electronic audio. However, due to the nature of a particular space or the availability of equipment, often loudspeakers are not placed proximally to the acoustic component of the hybrid system, and so discerning what each player is actually doing can become a difficult task. Similarly, it can be difficult to always situate instrument stations so as to maintain an adequate line of sight between performers. As well as engaging with the acoustic instrument, performers will often be manipulating other devices, such as foot pedals or MIDI controllers; this may expend the amount of time available to watch out for cues from the other players. The individual systems that we used for this project were designed to enable as much freedom from the constraints of looking at laptop screens or focusing on interfaces other than the original acoustic instruments, which were employed both as sound sources and as controllers. Nevertheless, especially in the early stages, we often felt consumed by the operation of our hybrid instruments.
4. Methodology
The NeVIS framework was developed in response to the aforementioned issues.
4.1. Vibrotactile device
At the core of the system is a novel device that transmits haptic feedback in the form of vibrations onto the skin of the performers. First used as a solo performance tool for piano and live electronics, and designed to signal sections within a score and rhythmic information, the system's development is fully documented by Hayes (Reference Hayes2011). The device was built from an ArduinoFootnote 3 microcontroller and three small Samsung disk coin-type pager motors (each 1.5 V, 70 mA and measuring less than 1 cm in diameter). These were connected directly across the Arduino's ground and pulse-width modulation/digital pins. The motors were fixed to a glove made of a thin elasticised material, which the performer wore on her left hand. Two of the motors were positioned on either side of the back of the hand, and the third was positioned directly underneath, on the wrist. In this way, the performer, even whilst playing the piano, could accurately perceive three discreet channels of information. The extremely small and light nature of the vibration motors meant that the performance would not be impeded in any way by the device, as no extra noticeable weight would be added to instrumentalist's hands.
Long-length wires were run from the motors to the Arduino, which was connected to a laptop using a standard USB cable (Figure 2). Information was sent to the three motors via the Arduino using Max/MSP, the same software environment that was being used for the digital signal processing (DSP). This allowed the system to be easily integrated with the pre-existing performance patches. By simply toggling between on and off states, vibrotactile pulses were created; but, by additionally using the pulse-width modulation feature, a clearly noticeable increase in intensity of vibration could be experienced. In duplicating the device for duo performance, we found that using a glove was not suitable for the percussionist: when positioning the motors on the hand, perceivable changes in vibration were ambiguous due to the natural feedback felt from hitting the drums. To rectify this, a further device was created, which was worn on the upper left arm of the percussionist. Here, the motors were positioned around an elasticised armband. Again, we experimented with the positioning of the motors, and concluded that placing them equidistant around the circumference of the arm gave the most discreet and discernable results.
Lauren Hayes wearing the vibrotactile feedback device.
4.2. Structure by suggested cues
A cue-based framework, the core of which is timed event-points, drives the system. That is to say, before a performance, the participants must decide on the number of sections that the piece will consist of, and the duration of each section. This information must be entered within the Max/MSP patch; alternatively, there is an option to allocate an arbitrary number of sections, or to have these sections assigned random durations of a minimum of thirty seconds. Enforcing a minimum section length was an aesthetic choice, used to allow for a moderate amount of time for propagation of musical ideas. The total duration of the piece is also displayed within the graphical user interface, mainly as a guide for concert situations where a predetermined piece length is required. The performers may allocate names to each state, although, as discussed later, this is merely ancillary and optional. The section changes are simply predetermined cues, but the performers do have the option to pause the time-line, by pressing a button on a MIDI controller, and to remain for longer within a certain section if desired.
One of two laptops acts as a conductor, sending the timing cues and other information, which will be outlined below, as OSCFootnote 4 messages over a local network connection to the second laptop. The sections and their corresponding durations are shown on the laptop screens (Figure 3). More important, however, is that this cue list should be perceived cutaneously, through the surface of the skin of the performers. Just as how, within a notated or graphic score, different symbols or instructions signal particular musical events, so too is it the interpretation of the different vibrational sensations felt by the performers that is crucial here.
Graphical user interface of the NeVIS interface and cueing system.
As mentioned above, the first layer of the system consists of the section cues; these being points in time during which major musical cuts may occur throughout the improvisation. These changes can be textural or rhythmical, and either in the acoustic or electronic sound worlds. Although the specific moment when these events occur within time-line is predetermined by the cue list (unless, of course, one of the performers freezes the state of play), it is left up to the players to decide whether to acknowledge and respond to these suggested prompts. For this reason, the different sections are also annotated with generic names, which may suggest a musical description understood by both performers (such as sparse), but are vague enough to apply to various situations. For example, who should play sparse? Is it rhythmically sparse, or a sparing use of pitches? It should be noted that looking at the screen for such information is optional, and is something that we constantly try to move away from, or completely avoid. Hence, to recapitulate: how changes are made and what is changed in the musical progression is entirely up to the performers. They will merely receive a signal telling them to change, along with a textual suggestion, should they decide to look at the laptop screen.
The only other quasi-predetermined parameter is tempo. This is sent in the form of a pulse, and is included not so as to enforce strict time-keeping, but rather to serve as a foundation around which possible interlocking between parts can be created. Of course, this can occur naturally within playing, but again this is an additional suggestion to be integrated, or not, as desired by the individuals present. The performers must predetermine the tempos of each section within the patch (zero, if undefined), but they do have the option to stop receiving the pulse if it becomes either too distracting, or unfitting to the current state of play. We both use the small and discreet Korg NanoKontrol Footnote 5MIDI controller as part of our extended instruments, and so, by simply pressing a button on the interface, we can turn off the tempo vibrations. This was easy to add to the pre-existing systems, and seemed the most logical way to control this parameter: since we were already adept at using these controllers it was simply a case of repeating an action that was already learned. An alternative would have been to attach a further button or sensor to the wearable part of the device, but this could have potentially interrupted play. Of course, the situation may transpire where one performer is playing along with the pulse, and the other may have turned it off; this is just one scenario where the voice of the system itself may become noticeable. Another advantage of the suggested pulse is that we often slipped into a few standard tempi during previous improvisations, and so the conducting provided a gentle prod towards fresh ideas. Moreover, transitioning synchronously to a new tempo is virtually impossible without some form of direction. This would have to come either from one of the players, or, as in this case, from the cueing system.
4.3. Communication between performers
During performances, we found that we would often drift into a state of semi-isolation, focusing on, gauging and reacting to the specifics of the individual augmented instruments (something also commonly observed within the larger improvisation group, Edimpro). Thus it was often difficult to attract the attention of the other player for visual cues. In order to remedy this, a nudge function was built into the system, which served as a tool to enable visual communication. This is initiated by pressing a button on one of the MIDI controllers, sending a burst of three short pulses to the arm of the other musician, over a duration of 1,800 milliseconds. This would simply alert us to make eye contact: the meaning or intent of the actual visual cue given after contact was made would depend, of course, on the ensuing gestures, glances or signals. However, this nudge function certainly helped to enable these exchanges.
As an artistic choice, we have always performed in close proximity to each other. However, this system could certainly be used across greater distances and locations providing that a low-latency network could be established. Parker describes a networked performance across three different cities where not only audio, but also control data was exchanged, with a latency low enough to enable real-time performance (Parker Reference Parker2006). This is an example of what Gil Weinberg terms as ‘the Bridge approach’ (Weinberg Reference Weinberg2005), whereby performers in distant locations attempt to play as if they were spatially together. Certainly NeVIS could be tested in more extreme situations, but our aims were to investigate the effects of the system on the structural outcome of the music, and to enhance our already established communication practices.
4.4. Musical parameters
Although, as stated, the role of the network was limited to signalling and other simple forms of communication, an additional element was added to allow for the exchange of parametric data. The densities of the individual acoustic instrumental sounds, already being calculated in both our patches for internal processing, were sent over the network and mapped to various modules (in the other performer's patch), thus influencing the overall musical texture. Similarly, the spectral centroid of each performer's final output, after all DSP, was also swapped and used in a similar manner. The aim was to explore how parameters less easily perceived than, for example, pitch or amplitude might be useful for affecting the electronic processing. Additionally, each cue point can be assigned up to eight numbers, which will enable or disable sound processing modules within Max/MSP as the piece progresses, creating a more fluid, less disjointed performance during the improvisation. Selections that are displeasing to the performers can easily be overridden using the MIDI controllers. Due to differences in the hybrid instruments, only one of the performers chose to utilise this feature, as it was more conducive to their particular approach.
4.5. Choice of modalities
As mentioned earlier, issues of privacy, feedback and creating intimacy with the instrument are just some of the benefits of working with the haptic sensory channels. Furthermore, when working with laptops, using haptics may help to free performers from the constraints of looking at the monitor for visual feedback (Hayes Reference Hayes2011). With this in mind, we decided to use vibrotactile feedback as the method for communicating the signals and cues discussed so far. Within electroacoustic music, in the absence of a conductor, the use of click-tracks for performers is often deployed, whereby a metronome pulse is heard through headphones. Stockhausen's Helikopter-Streichquartett (Reference Stockhausen1995) is an extreme, although perhaps apt, example of this phenomenon: four performers play from inside four helicopters, spatially distanced from each other, and directed only through click-tracks on headphones. It was felt, however, that the use of headphones reduces the ability to listen clearly to the overall sonic results: indeed, simply discerning distinct electronic parts in a group setting can be difficult enough. Furthermore, the tempi to be transmitted across the network were only suggestions, which could be easily disabled, and so a constant audible sound would be too distracting. Thus, short continuous pulses (lasting 75 milliseconds) were transmitted to the hands and arms of the performers, allowing the tempo of each section to be adequately perceived.
Similarly, having a visual nudge or alert represented on the laptop screens would not suffice, as we tend not to fixate on the monitor while performing; a signal of three short bursts of vibration was transmitted instead. For the same reason, the section cues were indicated by a vibrotactile sensation that was short enough to trigger an impulsive reaction from performers, but that would also give just enough time to prepare any electronic changes that might be necessary. A ten-second approach signal was used, which increased in intensity over the duration. This length was chosen as it gave adequate time for any musical changes to be made, yet preserved enough of the spontaneity that we wished to arise from the appearance of the synchronisation points.
5. Results
The system (Figure 4) was developed over a six-month period, and the resulting work, Socks and Ammo (sound example 1), was performed at various festivals and conferences, which included Sound Thought (Glasgow), Sonorities Festival of Contemporary Music (Belfast), Soundings Festival of Sonic Art (Edinburgh) and NIMEFootnote 6 (Oslo). With the exception of Soundings, where a guest double-bass player participated in the improvisation without using NeVIS, the system was used in the same format. Not only its usefulness, but also the musical character of the project became apparent the more that it was adopted in performances. The nudge function was immediately utilised as a simple communication tool, and helped to improve our general communication on stage. Moreover, as this is a private method of interaction, it arguably helped to give the audience the illusion of a more integrated and polished performance. We certainly found that it helped to quickly rouse us from the states of absorbed isolation that sometimes occurred, and re-establish any required visual contact. Indeed, at the Sonorities concert in SARC's Sonic Lab, it was noted from audience feedback that a very strong sense of integration and coherence between the performers was evident in the music. Of course, this sense of connectedness may be attributed to our collaborative history, but in comparison to performances that we have given without using NeVIS, we believe that this is indeed due to the implementation of the network.
Signal flow of the NeVIS system.
The section changes raised several points worthy of discussion. The knowledge that there was an imminent change very often resulted in a state of self-awareness and anticipation, rather than an engaged performance or progression with musical ideas. This was particularly noted when testing the system with other performers at LLEAPP 2011. Here, participants initially found the system restrictive and counterintuitive to their former improvisation practice, but after some time they began to comment on the potential usefulness of it. Indeed, after performing with the system many times over a period of several weeks, the anticipatory nature of our own responses receded. At this point, NeVIS became more clearly useful as a tool to shape the improvisations. This can be attributed to the fact that, when we had performed extensively with the vibrotactile feedback, it became an integral part of the performance; and after a certain degree of familiarity was achieved the musical output became the main focus once again. The perceived instructions and suggestions would be taken into account only if they served the already established musical material and direction, and, whilst they were very often acknowledged, they could be and were ignored too.
Of course, due to the autonomy of the performers within the framework, all cues could potentially be ignored and therefore be rendered meaningless. However, this never happened in performances as we wished to understand how the voice of the system might be heard amidst our own playing. Certainly, it was mentioned in the audience feedback at NIME that the system could clearly be perceived to be influencing the direction of musical progression in ways that would not otherwise have arisen naturally: this was illustrated mainly by the synchronised and often abrupt changes in direction. This feedback was encouraging, considering the context of this performance, as we were unaware to what extent an informed audience (here with explanatory programme notes) would be able to discern the effect of the system. Similar comments were received at Edinburgh performances, from audience members who had heard us perform in both scenarios, with and without NeVIS. When an element of unpredictability and surprise was introduced, the performances, conversely, appeared to take on a stronger sense of direction.
6. Future Directions
To date, the NeVIS project has been used with a relatively static predetermined cueing structure. Further developments will focus on creating a real-time non-randomised suggestion system based on machine listening techniques. This will be realised by creating a database of pre-recorded musical gestures, whereby the system will become more familiar with the individual performers the more gestures that it learns. IRCAM's Gesture Follower Footnote 7 and Fiebrink, Trueman and Cook's Wekinator Footnote 8 are possibilities that we have started to explore for this purpose. It is hoped that, with the implementation of these systems, the selection of processing modules within each section may become more meaningful. Rather than being selected at random by the system, or predetermined by the users, the instrument will respond to what is being played. We hope to explore how the emergence of the cues over time might function in a similar manner.
Further exploration into the vibrotactile representation of audio will be undertaken by examining more complex models of analysis of the resultant sonic output. We will investigate which types information can be successfully and usefully integrated into the vibrotactile network. Schroeder et al. created visual avatars from analysis of the incoming signals; the data was recreated as an abstract image from an amalgam of distinct parameters, and was used to assist the improvisations (Schroeder et al. Reference Schroeder, Renaud, Rebelo and Gualdas2007). We will attempt to establish whether similar information can be represented in the form of haptic feedback, and if this enhanced perception of the sound will in any way aid the improviser. Finally, the system will be expanded to multiple wearable devices and be made wireless. It will then be tested with a larger group of performers.
Acknowledgements
We would like to thank Michael Edwards and Martin Parker for their continued support of and advice on our endeavours, and for thorough commenting and counsel on previous drafts of this work. We would also like to thank Peter Nelson for his suggestions on the text.
