2.1. Control data networks and virtuosity

The difficulty of categorising control types is compounded to the extent that the virtuosity demonstrated by acoustic instruments is thought as a model. Is this difficulty introduced with the notion of virtuosity well understood?

The paper ‘The Importance of Parameter Mapping in Electronic Instrument Design’ (Hunt, Wanderley and Paradis 2002) precedes us in defining the essence of an instrument by reference to the control domain. The ‘mapping between input and system parameter … defines the very essence of an instrument … by altering the mapping, even keeping the interface and the sound source constant, the entire character of the instrument is changed’ (Hunt et al. Reference Hunt, Wanderley and Paradis2002: 1). We build upon this premise that the essence of a digital instrument lies in the control data domain. The above authors add to the prior discussion of the control data dimension by introducing the idea of mapping layers. The mapping of one-to-one control and its enrichment by introducing not only mapping layers but also cross-coupling of control data streams (Hunt et al. Reference Hunt, Wanderley and Paradis2002) allows for the interaction of control-data streams amongst themselves. Points of contact within the control domain itself are nodes. With the introduction of control domain nodes, a typology of control would have to admit yet more complex relationships between control-data streams. A multiplicity of control-data nodes means a control network. If complexity within the control domain is the norm, and the condition for this complexity is a multiplicity of control nodes, then configuration with the control domain is always a network configuration. The essence of a digital instrument lies in the configuration of its control-data network. Understanding of network configuration enables optimised controller design and therefore would presumably facilitate virtuosic performance.

Virtuosity is in part extraordinary micro-athletic ability, and its best exemplars are still acoustic instrumentalists. Rate and resolution are clearly two dimensions of expressivity, and the problem of enabling athletic virtuosity is thus aided from the two poles of sensor development and software processing power. Sensors now are robust, and output at high resolutions and speed, and anything understood as an imperfection in the control output (e.g. data jitter), can usually be corrected with additional data wrangling. From the other end, new software such as ChucK promises greater time-responsiveness through strongly timed ‘on-the-fly’ (Wang and Cook Reference Wang and Perry2003: 1) implementations of controller data. However, sensor resolution and processing power do not solve the problems of simulating the interaction between the acoustic instrument's control gestures and sound production. The Cilia is modelled on the performance gestures of the Japanese shakuhachi (a traditional vertical bamboo flute). It inherits its control gesture technique, doesn't produce sound itself, and models itself on complexities in the interaction between the control gestures and the same instrument's sound production. Much of the weight of reproducing this complexity can be placed on the imaginative use of mapping within the control data network.

Our kernel definition of instrument remains focused on the configuration of control-data networks. A survey of the field of artistic endeavour where musicians have contended with differing definitions of instrument through music-making will sharpen this focus.

2.2. Some precedents in network instrument making

The ensemble practice of The Hub was oriented to ‘emergent phenomena’ resulting from data sharing amongst independent algorithmic instrumentalists (Brown Reference Brown2007). The ensemble prioritised ‘network behaviour over the actual sounds employed in the realization of each piece’ (Brown Reference Brown2007: 2). There are few better examples of network-instrument making. Hub performances are about processes that are dependent upon multiple agencies each responsible for the continuous generation of control data, whether or not they are collaborating or contending. A basic ‘data exchange protocol’ enables ‘free association’ and the demonstration that ‘surprising instances of odd correlation’ happen with a certain regularity, indifferent to any attempts to determine them (Bischoff Reference Bischoff2007: 11–12). Our focus on configuration of control-data networks is not affected whether the control sources are dispersed and indeterminate, or unilateral and determining.

Could one performer manage a Hub performance? Inspired by the early Hub experiments, the Bridges application suggests the affirmative: ‘there can be zero or more performers on the network’ (Wyse and Mitani Reference Wyse and Mitani2009: 2). Performance could also presumably be automated. The number of performers involved does not change the definition of net instrument. Bridges enables the network performer to play with routing, scaling and mapping of control data freely between the control sources and the control destinations. It provides for a ‘dynamic’ ‘unrestricted reconfiguration capability’ (Wyse and Mitani Reference Wyse and Mitani2009: 4). Even if the routing of control data is itself thought as purely formal material to be played (e.g. as routing presets), this still needs to be instrumentalised. The focus on configuration of control data networks is not affected when implemented in purely formal ways, as when a network instrument is configured to route routings.

Netochka Nezzvanova, in introducing ‘URL synthesis’ with the Nebua.m81 implementation (Nezzvanova Reference Nezzvanova2000), was amongst the first to specifically use the Internet as a network instrument. Search actions (automated and ‘non-human’, whether stochastic or deterministic) on URLs yielded data (numbers and text) that were then sonified. It is clear in such implementations that human agency is not especially important. Yet clearly control-data networks are continuously being configured. Network instruments can be human- or machine-driven.

Atau Tanaka's and Kasper Toeplitz's Global String was an instrument ‘destined for concert performance, but also to occupy public space as an installation’ (Tanaka Reference Tanaka2005: 280). To some degree performance requires a passive listening audience willing to witness the piece as a process, and installation, an audience willing to be physically engaged with the piece as an object. Whether or not the ‘string’ (a series of sensors mounted on a cable) was played by ‘virtuoso’ performers in a concert format, or stimulated in an almost ambient way by a sound-performance naïve public, a measure of control configuration activated the physical modelling routine. Certainly a tele-instrument can be played in ambient non-expert fashion.

Others have focused on consequences of issues related to signal transmission. Chris Chafe finds materiality in the acoustics of transmission such that they become music composition and improvisation parameters: ‘path delays themselves can also be used to constitute network sound objects in a new breed of synthetic, distributed musical instruments’ (Chafe Reference Chafe2009: 414). Whether or not the Internet itself is thus put into the foreground as material, the practical performance question is still how to instrumentalise such sonic phenomena. Whether or not sonic content is derived from tele-transmission, the instrument maker will still have to configure control data to play it.

This quick survey allows us to step through a series of reductions that corroborate our focus on configuration of control data networks. It also compels us to supplement it. With the exception of the work of Chris Chafe, the above examples are network instruments that are not primarily concerned with the question of presence.

2.3. Presence and virtuosity

The topic of presence is covered at length by the tele-media industry in understandably one-sided discussions of their products. If presence is defined as proximity, then the general problem is to simulate proximity. But what is proximate? Teleconferencing extends collaborating remote musicians’ vision and audition into their respective rehearsal rooms. There are synaesthetic, kinaesthetic and semantic factors that cannot be transmitted. It is imperfect simulation, but serviceable given the right expectations. But this is a poor definition of presence. It is also an opportunity to segue into a discussion of another dimension of virtuosity.

Matthew Wright has argued that signal transmission can never substitute for being there: ‘The point isn't that physical proximity is inherently superior to technologically mediated communication, but that we should use computer networking to provide new and unique forms of musical communication and collaboration’ (Wright Reference Wright2005: 193). Tele-media cannot substitute for being there; this is not a bad thing, simply a different form of representation, and there are more interesting things to do. The medium presents opportunities to understand presence anew. The focus on networked control transmission is one way. It would seem that the author of OSC had this in mind. Furthermore, it is clear that, in speaking of the action driving tele-instruments, the problem undergoes a transposition of sorts. Presence is as much presenting, as it is to be present. It has both a verbal and nominal character. To use transmission as a performance instrument for generating new contents, this is already presence.

There is a traditional position that harbours the same reservations concerning what proximity simulation alone can accomplish. A performing artist is interested in a specific kind of presence that facilitates engaging an audience such that they have a cathartic emotional experience, one that they could not have in any other way. Virtuoso musicians are capable of extraordinary technical feats, but no less extraordinary are their emotional enactments. Arguably the difference between a performing artist and an athlete is the empathetic ability required only of the former. Incidentally, it is also one difference between human and machine performers, the tentative premise being that machines are not capable of empathy. An audience is compelled because they bear witness to the authenticity of the performing artist. Good instruments optimise capacity to respond to the widest range of expressive gestures, from deceptively simple distillation to exuberant athleticism. Whatever the aesthetic purpose, a human performer executes extraordinary micro-athletic and empathetic feats in order to compel an audience into a cathartic emotional narrative experience. When this is successful, it demonstrates efficacy in the projection of presence.

Therefore, design of the control data network should optimise responsive capacity. It is clear that signal transmission uniquely contributes to performance presence especially to the extent that transmitted signal content bears a generative relation to the control data actions. Thus the verbal and nominal aspects of presence are optimised.

3.1. Control and abandon

Control-data networks are complex. Complexity is always an implicit topic. In performance practice, complexity begins with a contradiction that resides within the term controller, between control and abandon. All experienced musicians know this, whether improvising or interpreting. There is a feedback continuum between the immediate production of sound and the responsive adjusting of technique. There is a moment of abandon as the player puts the sound out there, then in the response a moment of control as the resistances of the instrument are negotiated. The resistances of good instruments are complex, always providing some nuance beyond the player's control. Experienced players negotiate quickly and efficiently. There is a poise that is part of the athlete's psychology that enables intense focus on movement and reaction. The back and forth feedback happens so fast and constantly, a listener can't parse it. The sound shimmers, and even if one can't parse it, this shimmering is still audible. Shimmering is another way of expressing the temporally complex meaning that past, present and future are equi-primordial.

The sustained action of shimmering requires athletic virtuosity. Presence in tele-performance is not so much a matter of resolving signal path problems (such as the removal latency jitter) as it is of encouraging shimmer, as in any other performance medium. How can one facilitate shimmering in design? We propose next to translate it into exacting control domain terms.

3.2. Unidirectional control types

Discrete control means direct implementation of triggers (e.g. simple switches or sequence tracking routines) and continuous control of data streams (e.g. from capacitive sensors such as track-pads). Both instantiate direct control. They directly affect a signal path. This has been indicated by others as one-to-one mapping, but would also include ‘divergent mapping’ or isomorphic parsing (Rovan et al. Reference Rovan, Wanderley, Dubnov and Depalle1997). Direct routing includes one-to-many and many-to-one mappings.

There are four types of fingerings on the Cilia. They are all second-order direct discrete controllers as they are derived from primary controllers (raw pad data) read for discrete state control (is the finger off or on the pad?). The combinations of states from the different pads yield the second-order controllers. Non-vented and vented fingerings are woodwind players’ terms. The former describe continuous closures from the top of the tube down to the bottom. The latter describe non-contiguous closures. Numeric fingerings describe the number of pads whose state is on. They are possible only on an electronic controller. When non-vented fingerings are combined with ratio threshold values from the mouthpiece (there are two thresholds, therefore three possible discrete ratio values), what is produced is analogous to a three-octave range. Four non-vented fingerings become a series of 12 basic notes.

Two of the track-pads’ continuous controllers are all that is needed to divide the pads into quadrants, providing four state detectors per track-pad. The Cilia is thus capable of playing homophonically (i.e. non-parallel chord sequences). Each combination of quadrant state values maps onto intervallic structures (how chords are defined). It should be noted that multi-tasking of the pads on several levels is the norm.

Dynamic control happens when controllers directly control other controllers. This is a simple example of what has been called ‘convergent mapping’ (Rovan et al. Reference Rovan, Wanderley, Dubnov and Depalle1997). A formal theory of types would admit discrete and continuous dynamic controllers, and of first and second orders.

Indirect dynamic control occurs when data is routed through an additional mediating controller. In the Cilia work, the mediating controller is invariably a dynamic controller or another indirect dynamic controller. As with dynamic controllers, other sub-distinctions can be anticipated formally. They have not been implemented in the Cilia programming.

Consider the control category implementations in figure 2. If controller 3 went directly to the VCA it would demonstrate simple direct continuous control. As controller 4 modulates its upper limit, it functions as a dynamic continuous controller. Similarly, controller 1 is a dynamic continuous controller as controller 2 is setting its upper limit. Controller 4 parses and, because its values are scaled, provides an example not only of complex control, but also of indirect dynamic control. It affects controller 1 through manipulating its dynamic controller, controller 2. Complex control is demonstrated in the parsing of controller 3. When not modulated by dynamic controller 4, it routes directly into a time function, ‘summing’ with the already modulated frequency to make frequency ramps (glissandi).

Figure 2

Demonstration of unidirectional control types.

This pedagogical patch enables performance with a sine wave. The aesthetic objective is to avoid having very loud activity at higher frequencies, and to do so in a way that is not entirely predictable and that a performer can control. Complex multiple referent controllers perform deterministically, even if statistically within ranges of values. Such behaviour, such inexact predictability, aids in making a performance lively.

Complexity begins in control data configuration with the parsing of a single control stream, and in its non-isomorphic implementations.

3.3. Reverse analysis of the control dimension of a shakuhachi

Reverse analysis here means use of electronic music concepts to understand the control dimension of the shakuhachi anew. It begins with common practical knowledge that any player can attest to.

Breath and finger actions are two types of physical gesture used on the shakuhachi. There are four breath actions parameters. The angle of the vertical flute to the lips changes the ratio of air in and out of the tube. Force is the amount of pressure applied by the stomach muscles to push the diaphragm and force air from the lungs. Air volume is the shape of the embouchure hole: the more horizontally oblong the hole is, the more breathy the sound; the less so, the denser the timbre. Velocity is the size of the embouchure hole: the smaller it is, the faster. Fingerings change not only the length of a standing wave, but also the impedance configuration, the resistance that creates instabilities in the standing wave. Any combination of closed and open holes counts as a fingering. Covering only part of a hole (shading) enables many other techniques. The percentage of area covered is one parameter; so is the shape of the shading, whether it is covered from the top of the hole down or from the side.

Early and continuing work at Waseda University in the development of a robotic flutist parsed the flute embouchure into six parameters (Takanishi and Maeda Reference Takanishi and Maeda1998). Engineering laboratory analyses can clearly augment the practical intuitive knowledge of the player. However, as reverse engineers with a different set of categories and objectives, the additional parsing of parameters does not aid us in understanding the virtual control domain, nor affect our conclusions.

At node one (figure 3), the breath parameter angle acts on force, or conversely breath with consistently less force enables greater angling (greater pitch and timbre spectrum). If the angle or ratio value is closer to 0 (the breath is almost all in the flute or out), then less force will be possible. This is an example of continuous dynamic control.

Figure 3

Reverse analysis of the control structure of a shakuhachi.

This is repeated at node 2, except with greater complexity as two more parameters are acting as dynamic controllers.

Direct continuous control is seen in the movement of the fingers over the keys (affecting pitch) (node 3), and in the increasing of breath affecting both amplitude and timbre in a continuous spectrum (force).

At node 5, velocity as a discrete threshold controller acts on the fingerings by changing the frequency register, octave shifts. The fingerings function progressively differently from the norm of non-vented fingerings the higher the register. This demonstrates discrete dynamic control.

If the value of any breath parameter is 0, then all are zeroed. Whatever else is going on at node 6, this shows multiple dynamic control and interdependence.

All fingerings produce unique frequencies in combination with timbre variations. In this way the shakuhachi demonstrates direct discrete control.

The one instance of indirect dynamic control can be seen by tracing force from node 1 to node 5. Insofar as it effectively changes the interior of the instrument (by over-blowing), it affects the impedance configuration.

This description also reveals limits in the paradigm. Indirect dynamic control is hardly present; the control routing cannot become as complex as one can easily imagine with electronic controllers. Furthermore, the control routing configuration is fixed on an acoustic instrument. The Cilia is an interface for a universal machine and need not be so limited.

What design lessons are here?

1. 1. The control that we see demonstrated on the shakuhachi involves the summing of several control sources to create other levels of control (dynamic and indirect dynamic). Richness of control is achieved by such summing.

2. 2. At node 1, force is parsed and routes to two different purposes; it contributes to the breath signature, and to the impedance configuration. The degree of complexity of control yields a higher measure of nuance and depth in the sonic outcome.

3. 3. Combinations of the five fingers on the shakuhachi yield fingerings, each of which yield a new sonic possibility. Higher-order controllers can be developed through tracking of combinatorial outcomes of simpler controller mechanisms. Higher orders of control enable manipulation of higher orders of musical content, such as scales.

4. 4. The shakuhachi demonstrates how certain control parameters are bound together, mutually dependent. It is important for instrumentalists to have thresholds and limits. Designing such dependencies with consistency aids the instrumentalist in developing a performance practice.

5. 5. Electronic controllers offer far greater complexity of control design than is possible on acoustic instruments. The ease with which the control limits of acoustic instruments can be surpassed in net instrument design suggests a massively undeveloped field.

3.4. Reflexive control

Reflexive control types are not unidirectional. A preliminary typology of this type has not been attempted. However, a more generalised implementation enables the following account.

Figure 4 charts a complex sub-routine within a much larger application for the Cilia. The intention is first to bear witness to the potential complexity of control routing, and then through a reduction of the sub-routine to introduce a novel control category.

Figure 4

Complex control.

There are seven original control sources: two microphones functioning at sub-gain levels as amplitude sensors in the mouthpiece, and five track-pads outputting three continuous control streams each. They are indicated at the top of the diagram. Solid circles indicate points of dynamic control; empty circles, indirect dynamic control. Signal processing does not begin until just above the numbers 3 and 4. The final signal is output after 6. Everything above 3 and 4 is control processing and routing. Control streams split and at some later point in the configuration flow re-join with themselves. The numbers 1–6 indicate points where this re-joining or ‘self-referencing’ in the control path happens. The original control shape undergoes modification and delay after parsing. Still, something of its original shape is retained at the time of re-joining.

Figure 5 reduces and exposes these reflexive moments in the control routing. Starting from the simple split air column, four of the derived control streams are traced. They eventually all meet again but after modification and variable delay.

Figure 5

Reflexive control.

Something of the original air column gestalt is preserved in all these derivations indicated in the diagram. They split off from the original, but in various ways continuously modify the original by interacting with other control streams or by being modified through implementation in a signal path. Sedimentation of variation accrues, the variants also being marked by different delays. It is very much a reflexive control structure. It finds a sonic analogy in heterophony.

If one were to conduct physics laboratory analyses on the standing-wave phenomena inside the shakuhachi and were able to show how one's abstracted data charts of rudimentary control gesture data are affected by the sound generation, perhaps it would look something like figure 5. Would the micro-delays, micro-echoes and micro-amplifications caused by asymmetries of resistance along an interior bamboo wall find successful simulation with reflexive control-data configurations? Successful or not, it would presumably be as complex as what was modelled here speculatively and creatively. Additionally, if basic conditions for resonance are delay and repetition, then virtual resonance is achieved in the above instance. There is precedent in the cases others have made for the resonant acoustic space of the Internet. ‘Sound waves propagated through Internet acoustics behave just as in air, water or along a stretched string’ (Chafe, Wilson and Walling Reference Chafe, Wilson and Walling2002: 1).

Reflexive control is not another magnitude of mediation between control nodes, but describes a different type of control structure. A threshold is passed when a control data network becomes self-referential. Reflexive control is a condition for any simulation of resonance, a quality of all good acoustic instruments. It is also a common if intuitive measure by which players attribute independence to their instruments, as though the instrument itself speaks. The anthropomorphising of one's instrument is not unimportant to shimmering: musicians frequently need to know their instruments as their other.

3.5. Control destination types

A control-data stream always resolves into a control destination. Control efficacy is determined by how well data is routed to the whole or a part of a sound-generating routine. What can be said of such wholes and parts? Let parts that are dependent upon one another be called elements. Pitch, amplitude and duration are such parts. No sound generation is possible when one is absent. The first level of an independent part will be called an individual. A simple note wherein no demarcation can be distinguished is an example. A plenum is observed when several such individuals are sounded simultaneously as in a chord or density. Control can be directed to any level of whole and part.

View the categories together (figure 6) and it is clear that the design potential is enormous. Figure 6 could be described as a toolbox. It proposes control-data categories to be used in digital instrument design, and suggests through the use of a routing matrix that the mapping potential is enormous. Such a toolbox is required in any systematic design approach.

Figure 6

Classification of control categories.

3.6. Multi-lateral cross-control: Calgary Intervention

Multi-lateral cross-control is a control type located between the control source and the control interface.

3.6.1. Calgary Intervention

This is an application composition first performed in a Calgary–Beijing tele-performance in March 2009, and again in January 2010. Jeremy Brown was the Calgary-based improviser (soprano saxophone) in the first performance, David Eagle (flute) in the second. The author was improvising on Cilia from Beijing in both performances. The same sonic results were projected to both locals.

Figures 7 and 8 describe the routing of two presets in the MaxMSP application. In all instances, the matrix object indicates 8 possible control inputs and 13 routing destinations. Practically, there are four outputs as control streams are merged. Each player exercises frequency control, double amplitude envelope generation and pitch-bend scrolling. In figure 7, the Calgary player controls frequency, but the Cilia player controls amplitude envelopes and bend (which here controls a timbre function). Frequency, amplitude and timbre are all dependent parts, elements. Two players are required to produce one electronic sonic result. The partial loss of control is at first disorienting for a player. That is by design. The disorientation affects the Cilia player more than the acoustic player as the acoustic player is still generating autonomous sound from his acoustic control instrument. The Cilia player is silent without generated sound.

Figure 7

Calgary Intervention multilateral cross-control 1.

Figure 8

Calgary Intervention multilateral cross-control 2.

Both players control frequency in the second preset (figure 8), generating the mean of the frequency inputs. If one player is silent, then it is effectively a transposition. Something similar happens with the scroll function. The Cilia assumes total control of amplitude. If the Cilia player does not blow, no matter what the Calgary player does no generated sound will ensue. The pop-up menu in figure 8 indicates the complexity of the larger patch within which the routing is embedded. ‘MIDIpitchin2’ is routed to nine sound-generating routines (the tenth is an interface monitor). It indicates a breadth of sound-generating routines. A maximum of five would ever be used simultaneously. Rehearsal time would ideally be spent learning to hear how this dispersed control behaves in different sound-generating configurations.

This piece provided an opportunity to experiment with a performance practice where authority is dispersed, shared and collaborated. The more control is dispersed and cross-directed to elements and not individuals or plena, the more a performer senses that another performer is controlling part of their performing voice.

The first toolbox (figure 6) is expanded in figure 9. The simple addition of a second player or control source makes the routing possibilities yet more complex. A very different kind of performance practice develops where the player is required to relinquish control in order to learn how to affect a higher-order collective control.

Figure 9

Calgary Intervention multilateral cross-control 2.

3.7. Interactive scoring: Graphic Cilia

At the end of the first part of this article we suggested that the nominal aspect of presence is optimised in tele-instrumentation when the ostensible signal channel content bears a generative relation to the control activities. In musical performance the audio transmission is self-evident and enhanced largely through good sound systems. Graphic Cilia proposes a synaesthetic partnership in which video transmission can be directly relevant to the control activity, and in a reflexive way that interacts with the musicians and their audio output.

Conventional music notation enables musicians to coordinate tempo, rhythm, pitch and harmonic context. It provides instructions that prescribe what musical actions should take place. Sonic contents that are not represented in conventional musical scores can be shown graphically. The arbitrariness of mapping from ostensible audio control data to visual output is quickly demonstrated by the differences in the numerous visualisation applications that accompany software media players. One recent application goes beyond this by providing flexible interactive mapping for the purpose of descriptively enhancing performance, while also enabling a visual performer to interact. Veldt allows ‘arbitrary text, images, video and geometric models’ to be triggered by musical control events (Davidson, Kapur and Cook Reference Davidson, Kapur and Cook2004: 2). Graphic Cilia conjectures further that an interactive score can facilitate greater presence of musical intention by facilitating a manifestation of complex temporality at the level of improvisational choice.

The premiere on 8 June 2011 again featured soprano saxophonist Jeremy Brown with guitarist Russell Broom tele-casting from the University of Calgary, and the author on Cilia from the Central Conservatory in Beijing. Although an interface was set up to derive control data from the two acoustic players, for practical reasons the performance went ahead unilaterally, the Cilia exclusively generating control that generated and engaged the graphic score.

The scoring concept behind Graphic Cilia is to remain an instructive graphic language. If these graphic renderings become painterly, it is incidental, but not irrelevant insofar as a performer may become fixated by them. In this instance, sonic generation becomes secondary to the desire to manipulate the image.

The effort to facilitate structurally self-aware performance practice has a number of precedents. For example, Nick Didkovsky's Zero Waste begins with prescribed stochastically generated music that a pianist endeavours to sight-read. The pianist is tracked as a MIDI performer and what she plays then is introduced into the notation, which is again sight-read off a laptop computer (Hajdu and Didkovsky Reference Hajdu and Didkovsky2009). Such scores are prescriptive and descriptive at the same time. There is a reporting that happens simultaneously to the generating of instructions that influence the musicians.

Graphic Cilia endeavours to go a step further by cueing the player into an awareness of complex temporality.

First note that the score (figure 10) is in colour, and control is routed into red-green-blue (RGB) values in various ways. The small, dark, rounded square just off centre is a rendering of what the player is doing in time present. Low and high notes render up and down, a timbre spectrum is rendered as left and right. As a player moves the square about, a noise is generated in the form of pixilation. It appears in this screenshot as a kind of messy grid. The grid is read along horizontal lines and rendered as conventional rhythm (a pixilation renders an attack; no pixilation, a rest). The rhythm is gradually built up as the square moves, until a maximum number of whole-note measures is reached. The rhythms remain unchanged until they are covered by pixel noise, or the entire screen is reset. The control instrument determines harmonic background, the upper right column indicating pitch classes. The bold note name with the octave designation indicates the pitch that the control instrument last played. The planes of gradated colour are created in the wake of the small square's movement. They trace where it has been, leaving sedimentations of volumes, colours and shadings. The greater the square's movements, the greater are the plane volumes. The more disjunct the square's movements are, the greater the layering. The more punctuated the playing is with silences, the greater the shadings. Finally, the line-drawn rectangles on the upper right pulsate as a predetermined tempo guide.

Figure 10

‘Graphic Cilia’ screenshot.

The player can inscribe as well as be inscribed by the descriptive routines. He can control the rectangle's movements, colour palettes, folding, and noise textures. Once the rhythm is completely inscribed the player can become more expert by limiting himself to the given rhythms. It becomes clear after longer periods of playing that the player can never engage the whole of the score. When the focus is on playing the rhythm with the inscribed pitches, the movement of the square becomes incidental. The player loses track. If the player becomes immersed in the colours and shading, then the pitches may become secondary.

Through reading, the musician's time awareness becomes multi-directional. The activity of the small square locks the musician into a present. It is also true with the generating of rhythm cells, as it is the playing of notes that triggers their appearance. All conventional notations orient the player to the future as potential contents to be used at some later point. The shaded colour volumes and foldings represent the past: traces, resonances and sedimentations. The same is true of the noise pixels. At any moment whatever part of the score is being attended to may avail itself to a forward-oriented glance. It happens when some content is regarded not as something that has been produced, but as it is itself, and therefore warranting a renewed question: what does this mean?

In all these ways, the graphic score cues the player into a complex temporal awareness. Furthermore, in this way visual transmission contributes to the facilitating of performance presence. It doesn't do so isomorphically as do algorithmic visualisations, but in a circular interactive manner.