1. Introduction
When playing music, acoustic musicians face, among others, two types of constraints: physiological and acoustical. These constraints define a range of possibilities that musicians must master to achieve expressive performances.
Similarly to other studies (Leman Reference Leman2008), our working hypothesis is that joint investigation of the musician’s physical movements and the resulting acoustic sound helps to formalise fundamental concepts fruitful for designing digital synthesis control, and more generally approaches in electroacoustic music.
This approach is related to other recent studies on instrumental gesture, in particular studies investigating the relationships between gesture and sound characteristics in actual playing situations (De Poli, Rodà and Vidolin Reference De Poli, Rodà and Vidolin1998; Dahl Reference Dahl2000; Goebl, Bresin and Galembo Reference Goebl, Bresin and Galembo2004).
In this paper, we focus on string bowing motion and its relationship to sound. Beyond the acoustical point of view, bowing corresponds to a challenging problem for sound control. First, bowing can be seen as a continuous and simultaneous control of parameters, most notably bow speed and pressure. Second, bowing can produce attacks and articulations as well, which are also of prime importance. We propose here to pay attention particularly to this second point through the study of bow stroke transitions and their relationships to transient noise. Specifically, we aim for our study to point out limitations of note-based approaches in sound synthesis techniques.
Generally, musicians call articulation the manner of merging successive notes or more generally groups of notes. On self-sustained musical instruments, like winds or bowed strings, going from one tone to another implies going through a transient phase between two nearly periodic regimes. In the case of bowed strings, this transient phase is characterised by an irregular motion of the string, where one Helmholtz motion stops and a new one develops. Through years of training, string players develop an in-depth control over this transient phase. Adjusting the main bow parameters, expert players are able to vary the way transient noise sounds: from smooth and light to harsh and crunchy. From this control, they therefore can produce different kinds of articulations.
Research in music has shown a vivid interest in transient parts of sound, around the idea that they contain key expressive elements. Perception works demonstrated that transients play a predominant role in instrument categorisation and recognition (Grey Reference Grey1975; McAdams, Winsberg, de Soete and Krimphoff Reference McAdams, Winsberg, de Soete and Krimphoff1995). Sound quality of audio signal synthesis improved thanks to dedicated work on the transient parts, especially techniques based on signal models (Serra and Smith Reference Serra and Smith1990; Verma, Levine and Meng Reference Verma, Levine and Meng1997; Dannenberg and Derenyi Reference Dannenberg and Derenyi1998; Röbel Reference Röbel2003). Acoustic studies investigated the origins of transient parts and their contribution to the instrument’s acoustic signature (Askenfelt Reference Askenfelt1993; Goebl et al. Reference Goebl, Bresin and Galembo2004). Simulations and experiments on bowing machines were performed, and the influence of constant bowing parameters on non-periodic string motions was investigated (Guettler and Askenfelt Reference Guettler and Askenfelt1997; Guettler Reference Guettler2004; Woodhouse and Galluzzo Reference Woodhouse and Galluzzo2004). Nevertheless, there are still very few studies on the gestural control of transients (Demoucron, Askenfelt and Caussé Reference Demoucron, Askenfelt and Caussé2008).
This paper investigates the relationships between bow movements and sound properties in actual playing situations, paying a particular attention to the temporal behaviours of these multimodal components. The paper is structured as follows. First, we recall important concepts of sound and gesture. Second, we describe the methodology for specific experiments we performed on bowing articulation. Third, we describe the results. Finally, we discuss how these results can provide us with particular perspectives on the control of digital instruments.
2. Sound and gesture articulations in the violin
From a sound standpoint, the irregular string motion that occurs during the transient phase results in a broad-band pulsed noise (Chafe Reference Chafe1990). For musicians of bowed string instruments, this typical noise is well known; they usually learn to control it explicitly or implicitly for expressive purposes. Violin pedagogue Ivan Galamian, talking about sound production on a violin, alludes to this noise saying that percussive sounds like consonants are necessary to shape the melody line formed by the ‘vowel’ sounds (Galamian Reference Galamian1962). This sound analogy between bowed string instruments and the voice, and more generally between music and speech, is also often drawn in music acoustics (Chafe Reference Chafe1990; Godoy Reference Godoy2004; Wolfe Reference Wolfe2007). In particular, works led by Wolfe (Reference Wolfe2002, Reference Wolfe2007) investigated clues supporting this analogy and especially brought forward common issues of timing and energy in voice and instrument sounds. Such comparisons are particularly insightful for the study of players’ control of sound articulations.
From the point of view of control movements, players of bowed string instruments often consider transitions between strokes as important as the strokes themselves. Musicians continuously control bow motion and sound to achieve different expressive cues as shown through the study of bowing techniques by Rasamimanana, Fléty and Bevilacqua (Reference Rasamimanana, Fléty and Bevilacqua2006) or through the analyses of different performance versions by De Poli et al. (Reference De Poli, Rodà and Vidolin1998). However, making a transition between two strokes requires as much control skill as sustaining a sound, as shown by violinist Ami Flammer (Flammer and Tordjman Reference Flammer and Tordjman1988), and can itself be considered as a constitutive part of bowing (Menuhin Reference Menuhin1973).
In bowed string instruments, these two sonic and gestural points of view on articulations are actually summed up in the concepts of bowing techniques. Learning and performing bowing techniques indeed concern both sound and gesture. On the one hand, the names of bowing techniques often refer to an ‘action’ (e.g. martelé: ‘hammered’), on the other hand, the end goal is to achieve sound with specific characteristics (e.g. percussive). Qualities of transitions in gesture and sound traditionally result from playing with different bowing techniques. However, it may require years for student players to fully master a bowing technique and use it in a musical context. For these reasons, bowing techniques offer a fertile ground and a structured basis for studies on both sound and gesture control in string playing. In the following, we present a study of two fundamental bowing techniques from a sound and gesture point of view, with the aim to derive more general principles on articulations.
3. Methodology
The methodology followed in this paper is inspired by works of Guettler and Askenfelt (Askenfelt Reference Askenfelt1989; Guettler Reference Guettler2004) on bow transitions. However, our study is based on actual musicians’ performances, instead of controlled acoustics experiments. Our approach is therefore similar to Goebl’s study of the piano (Goebl, Bresin and Galembo Reference Goebl, Bresin and Galembo2005). Besides, we analyse the sound as emitted by the whole instrument, instead of only the movement of a sole string. In our case, we take into account the resonance effects from the violin body, which is closer to the musician’s perception.
We outline in this section the experimental procedure with a brief description of the bowing techniques considered, the measurement setups and recorded movement parameters, and finally the audio analysis.
3.1. Procedure
Eight violin players participated in the study. They were all advanced-level violinists, with 9 to 20+ years of practice. To measure different articulation qualities, they were asked to perform a one-octave, ascending and descending D-major scale with the détaché bowing technique, then the same scale with the martelé bowing technique. Both scales were recorded at a tempo of 80 bpm and the dynamic level forte. To reduce possible accelerometer bias due to gravity, subjects were asked to remain on the D string, therefore minimising the angle variations of the bow. All violinists were asked to perform on the same violin and bow, to guarantee common conditions for all measurements.
3.2. Bowing techniques
Détaché is the most common bowing technique. Each note is performed on a separate bow, hence the name. The sound is kept relatively constant during one stroke and there is no break between notes. In Détaché, the articulation corresponds to the transition from one stroke to the other. This transition can be achieved with different degrees of smoothness or harshness but generally remains smoother than martelé. For this kind of stroke, transitions can be compared to liquid consonants such as ‘l’. As opposed to détaché, martelé strokes are incisive and sound almost percussive, hence the name. Strokes are generally short, with a harsh beginning and ending. In martelé, the articulation corresponds to the transition between stop (no motion/silence, end of previous stroke) to the beginning of the next stroke. Such transitions can be compared to plosive consonants such as ‘t’. Galamian (Reference Galamian1962) actually describes these two bowing techniques as being important poles in bow mastery from which violinists can compose other bowings.
3.3. Bowing measurements
As stated by Guettler (Reference Guettler2004), bow acceleration is one of the essential parameters influencing sound articulations. Moreover, we previously found (Rasamimanana et al. Reference Rasamimanana, Fléty and Bevilacqua2006) that bow acceleration is a particularly salient parameter to characterise the two bowing techniques détaché and martelé: differences and similarities between both techniques were characterised using features derived from bow acceleration profiles. Motivated by these previous results, we assume that acceleration is an essential motion parameter for bowing, in particular during bow stroke transitions.
The system used to record players’ bowing movements consists of two parts. The first part is a module that measures bow acceleration with two accelerometers (Analog Device ADXL202). This module is mounted at the bow frog with a carbon clip. The placement of the two accelerometers is adjusted to measure bow dynamics in three directions: along the bow stick, along the strings and at right angles to the stick. Accelerometer data are digitised at the sampling rate frequency = 333 Hz with a resolution of 16 bits and are sent wirelessly with a radio frequency (RF) transmitter powered with batteries. This module is shown in figure 1. The second part consists of a computer interface (Fléty, Leroy, Ravarini and Bevilacqua Reference Fléty, Leroy, Ravarini and Bevilacqua2004) with a dedicated card receiving data from the RF transmitter. Data are sent through an ethernet connection to a laptop for recording using the Open Sound Control protocol. Accelerometer data is median filtered with a window of eight samples to remove eventual acceleration peaks due to high frequency (HF) transmission errors. The total weight of the system is 14 grams at the frog: although the bow is perceptively heavier, the participants reported that it was easily playable. This system is similar to the one used in Bevilacqua, Rasamimanana, Fléty, Lemouton and Baschet (Reference Bevilacqua, Rasamimanana, Fléty, Lemouton and Baschet2006).
Module placed at the frog of the violin bow to measure players’ bowing movements. It consists of two accelerometers and a RF transmitter, powered with batteries.
3.4. Sound analysis
As explained before, we consider the resulting sound of the whole instrument. To do so, we recorded the violinists’ performances with a microphone clipped behind the violin bridge (DPA 4021). Sound is digitised at 44100 Hz, in 16 bits. It was recorded simultaneously to acceleration data using Max/MSP.
The sound analysis consists of extracting transient noise. The general approach is to use signal processing techniques assuming a signal model with deterministic and stochastic components. The extraction of the transient noise can then be performed using analysis/synthesis techniques. The general procedure is to estimate the parameters of a signal model describing the deterministic components (analysis) and to generate a new signal on the basis of this model (synthesis).
Subtracting this modelled signal from the original signal, we get a residual signal that contains the transient noise. We subsequently estimate the quantity of transient noise by computing the energy of the residual signal.
Because of the short time span of transient parts, between 50 ms and 90 ms in playing situations (Guettler and Askenfelt Reference Guettler and Askenfelt1997), the chosen model in this paper is based on the formalism of High Resolution Methods (HRM). In HRM, the deterministic components are modelled as exponentially modulated sinusoids. This actually gives HRM a higher frequency resolution than Fourier, especially on short windows, therefore enabling a more precise estimation of sinusoid parameters. The method applied in this paper is based on previous works on the use of HRM in audio signal analysis (Laroche Reference Laroche1989; Badeau Reference Badeau2005), using ESPRIT for the estimation of the sinusoid parameters (Badeau, Richard and David Reference Badeau, Richard and David2005) (see Annex for details).
4. Experimental results on bowing
Recorded waveforms and bow acceleration are shown in figure 2 for a series of strokes performed détaché (top) and, martelé (bottom). The residual energy profile, corresponding to transient noise, is also plotted (see computation details in Annex). As expected, in case of détaché, the transient noise peaks lie at transitions between strokes. In the case of martelé, the transient noise peaks are mainly located at the start and end of strokes, which correspond to moments when periodic string vibrations are initiated and stopped. Moreover, as already noted in a previous study on similar bowstrokes (Rasamimanana et al. Reference Rasamimanana, Fléty and Bevilacqua2006), each détaché stroke is characterised by one acceleration peak, while two acceleration peaks (acceleration and deceleration) occur in each martelé stroke.
From top to bottom: Détaché audio signal waveform, residual energy (transient noise), bow acceleration, and Martelé audio signal waveform, residual energy (transient noise), bow acceleration. Vertical bars delimit the analysis segments.
For statistical analysis, we built a dataset by isolating bow articulations for each of the bowing techniques. The segmentation is performed in two steps. First, we achieve a manual segmentation to select moments corresponding to articulations in the sound files. Second, an automatic process adjusts the segments’ limits based on the transient noise and the acceleration profiles: limits are determined by the energy and acceleration standard deviations. Vertical dotted lines delimiting the analysis segments are shown in figure 2.
For both gesture and sound, we observe that each articulation presents specific temporal distributions, as shown in figure 3 for détaché and martelé. Note that, for clarity, the distributions are normalised (maximum is equalled to one). For each articulation, acceleration and transient noise profiles exhibit different bell shapes. Interestingly we can observe small time shifts between the two profiles, which actually vary on the bowing techniques. To quantitatively assess these shifts, the first order moment of the profiles are computed for acceleration and transient noise:
Normalised temporal distributions of residual energy (dark), representing the sound transient noise, and acceleration absolute value (light) for one articulation in Détaché (top) and one articulation in Martelé (bottom). 
and 
respectively designate first order moments for residual energy and bow acceleration.
where sn and tn are respectively the digitized signal and the samples index.
The time shift Δtm, quantified by the difference between moments of the residual energy distribution and acceleration distribution:
is found to be positive for détaché and negative for martelé. We further examined such temporal features in different cases and players.
Figure 4 shows the succession of articulations on the scale exercise for both détaché and martelé bowing techniques (player 8). All 13 stroke transitions in the scale played détaché show positive time shifts, while the 14 strokes in Martelé show negative time shifts. Precisely, the ensemble of détaché transitions is characterised by a Δtm median value of 15 ms and an interquartile of 12.3 ms, while for martelé the Δtm median value is −18 ms with an interquartile of 10.8 ms. These values show that there is a statistically relevant timing difference between the two bowing techniques. These values also seem to reveal that the different bowing techniques imply distinct motion–sound relationships.
Δtm computed for one violinist articulations on the scales in Détaché (dark Δ) and Martelé (light X). Each symbol corresponds to a stroke transitions. Boxplots give synthetic views for each scale.
We now extend the analysis to eight violin players. In spite of player idiosyncrasies, we can see that the average time shift Δtm remains positive for détaché and negative for martelé, as shown in figure 5. Quantitatively, over all players, détaché articulations are characterised by a Δtm median value of 19 ms and an interquantile of 15 ms. Martelé articulations are characterised by a Δtm median value of −20 ms and an interquantile of 21 ms. This actually confirms on a broader statistical level that temporal motion–sound relationships can be specifically related to articulation types. As expected, some inter-player variability can be found. Such variability could be interpreted as possible differences in articulation ‘pronunciations’: some players ‘uttered’ tones in a globally more distinct way.
Δtm for eight violin players for each scale in Détaché (dark) and Martelé (light). Each boxplot represents one acending – descending scale.
5. Discussion and perspectives on sound control
We presented an experimental study on violin articulations, focusing on both bowing motion and sound properties. Particularly, bow acceleration and the noise component of the sound were measured. These parameters constitute key elements for the sound control in violin playing. We especially looked into note and gesture transitions, which have been understudied. Interestingly, from this angle, we found that bow acceleration and ‘noise’ are not in a direct causal relationship, even if acceleration is recognised as an important acoustic parameter that directly influences transient noise (Guettler Reference Guettler2004; Woodhouse and Galluzzo Reference Woodhouse and Galluzzo2004). Precisely, we found that transient noise could appear either before or after acceleration peaks and this actual time offset is dependent on the particular bowing technique. This can be partially understood considering that transient noise for détaché and martelé always peaks after the note onset. However, in upbow–dowbown detaché, the acceleration peaks appear exactly between two separate continuous strokes, while it appears slightly after the attack of martelé. Thus, the role of past gesture is fundamental for the correct interpretation of the acceleration data. This aspect can be regarded as gesture co-articulation (Rasamimanana Reference Rasamimanana2008, Rasamimanana and Bevilacqua forthcoming).
Of course, a complete physical model including all parameters (e.g. at least complete bow position, velocity and pressure temporal profiles) could explain these results. Nevertheless, our point here is that time relationships between control and sound parameters are complex. Considering the possible consequence on mapping strategies, our results show that simple strategies directly linking motion values to sound parameters on a frame-to-frame basis could not replicate the type of articulations considered in this study. To avoid such shortcomings, it seems important to consider the approach schematically illustrated in figure 6a, which is a generalisation of our experimental figure 3. This figure illustrates that we need to separate, at the signal level, a raw gesture level and the actual signal used at the synthesis level. Importantly, our point is here to propose explicitly a temporal approach in the transformation between these two levels. Each phrase is transformed through a specific temporal process (such as time convolution in case of linear process). These temporal processes overlap and depend on both the previous, current and forthcoming processes. This principle is illustrated by comparing the figures 6a and 6b, where the order of the gesture sequence is changed: A, B, C in Figure 6a and C, B, A, in Figure 6b. The effect of this permutation should fundamentally change the morphology of the sound objects. We elaborate on this approach with the three points below that seem to us essential in sound synthesis control to overcome the limitations of a note approach.
Temporal mapping schema: gesture data time profiles are transformed to input control profiles for sound synthesis. Gesture sequence order is reversed between (a) and (b): the morphology of the sound objects is changed as an effect of the gesture permutation.
First, gesture data should be considered as temporal functions instead of a stream of data. As a matter of fact, gesture data are most often processed as individual data frame, as this is directly handled in several programming environment (e.g. Max, Pd). For example, the MIDI protocol is clearly based on a ‘note approach’ with a single parameter to model the attack (velocity). If continuous control can be achieved through the use of ‘aftertouch’ parameters, complex articulations cannot be properly reproduced.
Second, the control parameters should consider both the previous and following notes. Such a phenomenon is analogous to co-articulation found in speech, and we propose here that its transposition to gesture, known as gesture co-articulation (Ortmann Reference Ortmann1929; Palmer Reference Palmer2006; Rasamimanana and Bevilacqua forthcoming), should be considered. Transitions cannot be simply taken into account in note-based approaches such as MIDI, for example.
Third, gesture-to-sound mapping should contain intrinsic dynamic time behaviour. Such mechanisms, as incorporated in physical modelling (Henry Reference Henry2004), could model adequately the types of temporal shifts found in our results. Such a model can actually encompass fine articulation mechanisms as found in bowing.
Our recent work on ‘gesture following’ (Bevilacqua, Guedy, Fléty, Leroy and Schnell Reference Bevilacqua, Guedy, Fléty, Leroy and Schnell2007), could incorporate these different aspects. In this processing system, the time profiles of gesture data are analysed. The time index of these input profiles can be arranged so that they correspond to other profiles that are the actual input for sound synthesis, as illustrated in figure 6. These two levels of time profiles can be set either manually or using appropriate algorithms. Compared to other mapping strategies that operate principally on spatial relationships, the mapping strategy we propose is therefore in the time domain, and could take into account articulations as measured in the study reported here. Such ‘temporal’ mappings are currently being experimented with.
Acknowledgements
The authors would like to acknowledge Roland Badeau for his help on High Resolution Methods. The authors also thank Matthias Demoucron, Julien Bloit, Emmanuel Fléty, Norbert Schnell and René Caussé for fruitful discussions and support. The authors particularly thank Anne Mercier, Florence Baschet and the students of the Dijon Music Conservatory for their participation in this study. This work has been partially supported by the European Commission 7th Framework Programme SAME project (no. 215749)
ANNEX A. High-Resolution model and estimation
The model used most often is derived from Fourier spectral analysis where the deterministic components are represented as a sum of sinusoids with variable amplitudes, frequencies and phases. Because of the short time span of transient parts, between 50 ms and 90 ms in playing situations (Guettler and Askenfelt Reference Guettler and Askenfelt1997), the chosen model in this paper is based on the formalism of High Resolution Methods (HRM). In HRM, the deterministic components are modelled as exponentially modulated sinusoids. This actually gives HRM a higher frequency resolution than Fourier, especially on short windows, therefore enabling a more precise estimation of sinusoid parameters. The method applied in this paper is based on previous works on the use of HRM in audio signal analysis (Laroche Reference Laroche1989; Badeau Reference Badeau2005), using ESPRIT for the estimation of the sinusoid parameters (Badeau et al. Reference Badeau, Richard and David2005).
A.1. Signal model
The deterministic components are modelled as a sum of exponentially modulated sinusoids.
For all t∈
,
where K∈
* is the order of the model. αk∈
* are complex amplitudes. zk∈
* are distinct complex poles. zk can be written as 
, with δk∈
the sinusoid modulating factor and fk∈
the sinusoid frequency.
The observed signal is then represented as the combination of the deterministic component model s(t) and an independent, centred, white gaussian noise w(t) with a given variance sigma.
A.2. Parameter estimation
In this paper, estimation of the parameters (i.e. amplitudes and poles) is based on a property of the modelled signal covariance matrix Rss(t): the rank of Rss(t) is exactly K, the number of distinct poles, if it is of size n > K and computed from l > K observations. This has a direct consequence on the observed signal covariance matrix: a study of its rank allows the observed signal space to be separated into two orthogonal subspaces: the signal space spanned by the exponentially modulated sinusoids and its orthogonal complementary, the noise space. Namely, the eigenvalues of the observed signal covariance matrix Rxx(t) are:
The poles are computed using the K first eigenvectors of Rxx(t) combined with the property that the signal space is actually spanned by the poles. This is done with the ESPRIT algorithm (Badeau et al. Reference Badeau, Richard and David2005), based on the rotational invariance property of the signal space. The poles’ amplitudes are finally estimated with a least square regression.
A.3. Application to a violin recording
Previous studies showed that the ESPRIT algorithm provides an accurate estimation of the frequency of the deterministic components under the condition of an additive white noise (Badeau Reference Badeau2005). To optimise the performance of the parameter estimation, the recorded audio signals are cut into eight frequency subbands of equal width. The analysis is then carried out independently on each subband, assuming a constant noise power for each of them. The window size used to perform the analysis is 128 samples at Fsa = 44100 Hz, i.e. 2 ms. The number of exponentially modulated sinusoids K is usually unknown, although it plays a key role in the algorithm performances. For this study, K is set to 20 sinusoids per subbands. This value actually overestimates the theoretical value of 18 for D4 (294 Hz), but ensures a correct estimation of the poles and their amplitudes (Laroche Reference Laroche1989).
