1. INTRODUCTION
Performance instructions for live electronics frequently give indications for actions to be carried out by musicians but rarely offer references to the quality of the sound resulting from these actions. These references can be considered as a prerequisite facilitating an engaged music performance. Individuals working in a group show an amazing ability to agree on an in situ vocabulary for sound, as can frequently be observed in the joint exploration of musical material in rehearsals. This vocabulary is often present as latent knowledge within this group and is rarely assembled and communicated in writing as part of music scores. A structured approach to elicit this vocabulary could facilitate the dissemination of electronic music repertoire for musicians who were not present during the creation and premiere of such works. It could furthermore serve as an aid in the subsequent analysis of musical material. Such an approach is attempted in this contribution through an exemplified elicitation of verbal descriptions for sound. Using tools from psychology and music perception research, the vocabulary of eight individuals in the context of a contemporary music composition with live electronics is considered.
2. SOUND AND TIMBRE
In electronic music, the transformation of acoustic sources through electronic processing is commonplace, either as a means of generating sound material for a composition or as a musical activity in a concert performance. Sound transformation cannot be seen as a separate field from sound generation for aesthetical and technical reasons. For example, frequency filters or delay lines can be turned into signal generators once their feedback gain is increased, waveshapers share similar input–output relations with wavetable oscillators and convolution can likewise be considered as a filtering or a sample playback mechanism. Nevertheless a helpful distinction can be drawn with regard to the temporal relationship between input and output of a transformation. A relationship featuring a large time delay is most likely to be perceived by listeners as multiple separate instruments or voices playing. An instantaneous temporal response then again can give the impression of an acoustic instrument unified with its transformation, perceived rather as a transformation of instrument timbre itself.
2.1. How many timbres?
An interesting differentiation imposes itself in the discussion of musical timbre, especially relevant when sound is considered that has little or no reference to its original source as in acousmatic sound. Schaeffer (Reference Schaeffer1966: 232) alerts us to the ambiguity of timbre being the quality that allows us to identify the identification of different sounds as originating from the same instrument but nevertheless to differentiate between these two sounds in the first place. In summing up multiple contributions in the field, Siedenburg (Reference Siedenburg2016) proposes three conceptual categories for timbre but points out that ‘it is misleading to suggest that one sound-producing object or instrument yields exactly one timbre. Contrary to parlance of “the timbre of the clarinet”, there is no single timbre that fully characterises the clarinet’ (Siedenburg Reference Siedenburg2016: 15). In the work presented here, this timbre plurality is circumnavigated by restricting the research to relative timbre changes between 12 sound recordings originating from comparatively long excerpts of a music performance featuring multiple playing techniques and articulations.
2.2. Review of verbal description of timbre
The first referenced mention of timbre is already an attempt to describe it verbally, as found in Rousseau’s articles on Tymbre in the 1765 Encyclopédie and in the 1768 Dictionnaire de Musique (Muzzulini Reference Muzzulini2006: 248). In the latter, Rousseau explains it as ‘par métaphore, cette qualité du son par laquelle il est aigre ou doux, sourd ou éclatant, sec ou moëlleux’ (‘metaphorically, that quality of sound by which it is sour or sweet, dull or bright, dry or soft’) (Muzzulini Reference Muzzulini2006: 249, translation by author). Rousseau relies on metaphorical language and uses adjectives from senses other than hearing. Helmholtz (Reference Helmholtz1863/2009: 112) described the effect of different strengths of partials to ‘answer to such species of musical quality as are called, soft, piercing, braying, hollow or poor, full or rich, dull, bright, crisp, pungent, and so on’. An overview of research building upon Helmholz’s work up to the late twentieth century is given by Kendall and Carterette (Reference Kendall and Carterette1993a: 447). In von Bismarck’s study (1974) on the verbal attributes of timbre, subjects rated steady sounds on 30 scales of opposite verbal attributes. These German-language attributes were pre-selected by the subjects and the experimenter. Von Bismarck showed that the musicians among his subjects were able to describe the sounds using the four scales dull–sharp, compact–scattered, full–empty, colourful–colourless with high inter-subject agreement. He related the first scale to the energy concentration in the spectrum of his sounds and the second scale to the discrimination of tone and noise but with high inter-subject scatter. No scale other than dull–sharp showed a common relation to a physical property of the sounds and he concluded that ‘language does indeed provide groups of subjects with some verbal attributes … which are used consistently in the same manner to describe a considerable portion of timbre’ (von Bismarck Reference von Bismarck1974: 157). Letting their subjects rate ten sounds of pairwise blended wind instrument tones on eight of the scales from the von Bismarck study, Kendall and Carterette (Reference Kendall and Carterette1993a) showed that these scales failed to differentiate their sounds. They questioned the oppositeness of the attributes used and repeated their experiment with the opposite poles derived through negation such as sharp–not sharp. The results gave a clustering of the scales complex, heavy, hard and loud, and a clustering of pure, compact and dim. Interestingly, the scale sharp is absent from these two clusters. In their follow-up study, 21 adjectives derived from Piston’s Orchestration were used as scales for the same sounds again and the three dimensions strong–weak, nasal–rich and simple–complex were identified through principal component analysis (Kendall and Carterette Reference Kendall and Carterette1993b). From about 1,500 French words describing timbre from free verbalisations, Faure, McAdams and Nosulenko (Reference Faure, McAdams and Nosulenko1996) selected the 23 most frequently occurring ones and, together with their negations, used them as semantic scales on which subjects rated 12 synthesised sounds. They related the ratings to perceptual dimensions derived from a previous dissimilarity study on the same sounds. Correlations were found between perceptual dimensions and semantic scales for attack time and the scales plucked, blown and fast attack, and between spectral centroid and scales discriminating high/sharp/shrill from deep/low among others. Faure et al. point out that the spectral centroid dimension, being based on the frequency axis of the spectrum, can indeed exhibit a low or high property not unlike musical pitch. In another study, Zacharakis, Pastiadis, Reiss and Papadelis (Reference Zacharakis, Pastiadis, Reiss and Papadelis2012) analysed musical timbre semantics having tested 41 subjects with a predefined vocabulary of 30 English adjectives, allowing subjects to provide up to three adjectives on their own in case they considered the provided ones unsuitable. They described three salient dimensions from their test under the conceptual labels luminance, texture and mass.
The direct applicability of the results from timbre studies using relatively short and stable tones with normalised pitch and loudness to the discipline of sound composition poses a challenge at the much-quoted intersection of research and creation. This quest is undertaken here by an analysis of musical excerpts of comparatively long duration and high variability, accepting a loss of general validity of the results for further use in timbre research, while hoping to gain a structured vocabulary for sound from the musically trained experiment participants.
3. SOUND TRANSFORMATION AS PERSONAL CONSTRUCT
While a single transformation can be adequately described as a signal processing algorithm, it can have drastically different musical results. An audio delay can produce a rhythmical pulse for brief input signals on the one hand but an intermodulation for longer inputs featuring pitch glissandi on the other hand. Verfaille, Guastavino and Traube (Reference Verfaille, Guastavino and Traube2006) give different approaches to classification of digital audio effects based on underlying technology, type of control or perception. Using a data set derived from their subject’s usage of digital effect plugins, Stables, De Man, Enderby, Reiss, Fazekas and Wilmering (Reference Stables, De Man, Enderby, Reiss, Fazekas and Wilmering2016) analyse the semantic description of timbral transformations and show among other things that equalisation and compression share a common vocabulary while reverb and distortion have different description schemata. In the composition of electronic music sound transformations can have a more fundamental role than that of mere audio effects. Since such transformations might furthermore comprise multiple interconnected processing algorithms, a structural analysis of their results is difficult. But it is especially the transformation’s strong dependency on the input sound that inhibits a general and straightforward description of the resulting output. Following the aforementioned example of the audio delay, a frequency filter is highly dependent on the spectral content of the input signal, up to a point where its effect can be remain inaudible. Likewise, a big range of different musical results can be achieved by applying temporal amplitude modulation to stable or fluctuating input signals. Sound transformations can therefore not be described without reference to their input and this is the motivation to conduct this research using an actual compositional example, studying relative timbre changes between different kinds of transformations. Such a study poses a constantly ongoing search and, in the context of the present article, also a personal enquiry. A theory resonating with such an enquiry is offered by George Kelly in his Personal Construct Theory and described by him as ‘a notion about how man may launch out from a position of admitted ignorance, and how he may aspire from one day to the next to transcend his own dogmatisms … It does not say what has been or will be found, but proposes rather how we might go about looking for it’ (Kelly Reference Kelly1970/2003: 3). In this theory Kelly postulates that ‘The meaning of an event … is anchored in its antecedents and its consequent’ and his Individuality Corollary states that this construction of events differs among persons. That ordinal relationships do exist between constructs and that these relationships are embraced in each person’s evolving construction system is expressed in the Organization Corollary (Kelly Reference Kelly1970: 9). In order to investigate sound transformations, these construction systems, with respect to eight individual’s verbal description thereof, are considered central elements for the following step.
4. EXPERIMENT
An experiment was set up to study the verbal description of sound in the context of contemporary music with electronics. Eight participants, all students or professionals in the field of music composition, performance or research, participated in the experiment, which was conducted in English. The participants were asked to complete a questionnaire in advance regarding their music education and activities and therein identified themselves as (performing) musicians in all but one case. All six non-native English speakers indicated fluent English language proficiency. Two participants were interviewed and tested remotely using teleconferencing software and were provided with high-resolution soundfiles and listened through quality headphones or loudspeakers. The six on-site participants listened to the sounds through a single near-field loudspeaker at moderate volume and in an acoustically treated room.
4.1. Stimuli
The sounds used in the experiment were derived from a 46-second recording of Marij van Gorkom performing the opening bars of the composition action for jackson for bass clarinet and electronic by Roland Dahinden (2009) (see Figure 1). This piece is written in staffless notation with notes in five different degrees between noise and pitched sounds and instructions to use ad libitum electronic effects in the tradition of stomp box pedals. The entire recording was alternately subjected to 12 different sound transformations devised and implemented by the author as real-time signal processing in the Pure Data computer music language (Puckette Reference Puckette1996). The 12 transformations were designed not to add a musically perceivable delay to the input sound in order to prevent the creation of a second voice following or lagging behind and to make the transformation an integral part of the bass clarinet sound itself. With its range of pitched and unpitched notes as well as their phrasing and ornamentation, the bass clarinet part of action for jackson serves as ideal input signal with high entropy to the sound transformations used in the experiment. The different transformations are described in Table 1 and were applied to the entire duration of the recording. All transformations were designed to not exhibit a primary sonic characteristic that can be traced back to a specific and therefore nameable DSP algorithm and participants had no knowledge of their names and implementation. All 12 resulting soundfiles were normalised in loudness according to the EBU R 128 standard using the ebur128 software (Adriaensen Reference Adriaensen2011). Each with a length of 46 seconds, these sounds are considerably longer than most of the stimuli used in the musical timbre studies outlined above. In the experiment, all participants were able to familiarise themselves with the untransformed bass clarinet recording before listening to the modified ones (Sound example ‘original’ and Sound examples e1–e12).Footnote 1
Beginning of action for jackson with playing instructions for bass clarinet: noise – pitch.
Sound examples and respective transformations constituting the stimuli in the repertory grid technique experiment
4.2. Method
The repertory grid technique (RGT), based on Kelly’s Personal Construct Psychology, was chosen as the method for this experiment, with the aspiration of reducing the experimenter’s influence by allowing the participants to articulate their own verbal descriptions rather than having to rate stimuli on pre-defined verbal scales. This method was proposed or used in related fields of study for the assessment of product sound quality (Kjeldsen Reference Kjeldsen1998), of spatial performance of sound reproduction systems (Berg and Rumsey Reference Berg and Rumsey1999) or spherical microphone arrays (Avni Reference Avni2010), or to study the perception of distorted guitar timbres (Atsushi and Martens Reference Atsushi and Martens2005) and of textural sounds (Grill, Flexer and Cunningham Reference Grill, Flexer and Cunningham2011). The RGT consists of a two-staged approach for each participant. The first stage features an interview setting in which each participant listens to multiple pairs of stimuli and is asked to describe the differences in each pair with two words which then form a bipolar personal construct. In the experiment described here, these constructs were formed by two adjectives describing musical timbre. The second RGT stage consists of a rating task, in which all participants rate sounds on their own construct scales. All participants undertook the first stage in individual sessions with the author and the second stage on their own.
4.2.1. Elicitation of constructs
Out of the four different ways to elicit constructs from participants described by Fransella, Bell and Bannister (2004: 30), the Dyadic Opposite method was used, which seemed appropriate as the differences between stimuli were quite well perceivable. Out of the 12 transformed bass clarinet sounds, four stimuli (Sound examples e2, e7, e10 and e11) were presented in six unique pairwise combinations and in randomised order. This subset of four stimuli was chosen to confine the time required to take this first phase of the experiment, as all 12 stimuli would have asked for 66 different pairings. At the presentation of each pair, the participants were asked to delineate the differences between the two transformations with two adjectives describing timbre. In addition to thus providing a total of six bipolar construct scales, the participants were asked to evaluate the suitability of each of their adjectives for the description of the relative timbre change on a four-point numeric scale.
4.2.2. Rating of transformations on constructs
Several weeks after the initial experiment, the same participants rated all 12 sound transformations on their own six construct scales. For each given stimulus, they were asked to supply an integer number from 1 to 4 specifying if the sound transformation corresponds rather to the left (1) or to the right (4) pole of their construct with 2 and 3 serving as intermediate values. All ratings then constitued the participant’s repertory grid, one of which is given as an example in Table 2. The comparatively low number of participants is not uncommon to RGT studies and is partially outweighed by the large number of ratings collected, amounting to 576 ratings (eight participants rating 12 stimuli on six construct scales each) in the present case. The participants furthermore evaluated the suitability of the construct scale at hand on a four-point numeric scale for each rating.
Example of a repertory grid from one participant
5. Interpretation
The interpretation of the experiment’s results was performed with respect to three types of data: first from the provided constructs and their pole’s suitability values; second from the suitability values indicating if a participant found a construct usable for rating; and third from the ratings of the sound transformations on the participant’s constructs. The repertory grid technique allows the use of software tools for the analysis of the ratings and the numerical computing language GNU Octave (Octave community 2016) as well as the statistical computing language R (R Core Team Reference Core Team2016) with the OpenRepGrid package (Heckmann Reference Heckmann2016) were employed for this task.
5.1. Construct grouping
The 48 constructs provided in total showed commonalities across all participants that allow their grouping as presented in Table 3. Therein, 22 constructs describe physical properties of material which are generally observable by humans using touch or vision. Ten constructs are concerned with qualities of physical change over time observable by touch or vision or made observable by excitation. Seven constructs specify characteristics of actions by living creatures that seem related to their internal states, consciousness or intentions. Six constructs have poles spanning across the aforementioned categories. Of three constructs not fitting into above categories two describe the intensity of the transformation.
Groups of qualities described by constructs
5.2. Suitability values
The indication of suitability values allowed the participants to express to what extent they found a construct to be a useful verbal measure for the assessment of a sound transformation. Comparing the suitability values of adjectives verbalised first with those of adjectives verbalised second indicated no significant bias towards either end. This suggests that the second adjectives verbalised under the demand to be opposites represent usable poles for the participants. Berg and Rumsey (Reference Berg and Rumsey1999) question the extent of construct poles to be orthogonal in applications of the repertory grid technique.
The comparison of the mean suitabilities for ratings of the four stimuli already present in the first phase with those of the stimuli introduced in the second phase of the experiment gave only a small difference. It can be concluded that constructs elicited from a limited set of stimuli are considered usable by the participants for a larger amount of unknown stimuli at a later point in time.
Large variations in the ratings can be suspected to be caused by either confident use or confusion with a given construct scale. For constructs with high variation, the suitability values of three participants correlated highly (>0.5) with their rating standard deviation. One participant’s data showed high negative correlation (−0.91) therein suggesting that variation might be an effect of unsatisfying constructs. Four participants showed smaller correlation (<0.5).
Variations in suitability values were tested to serve as indicators for different characteristics of adjective usage. Given the scope of the experiment, suitabilities above mean seem to be of high usefulness for the participants. In order to further differentiate this usefulness, high or low standard deviations of ratings on these constructs were suspected to indicate either specific or general adjective usage. No commonalities among such specific or general adjectives could be delineated from the data provided by the participants.
5.3. Ratings
The analysis of the repertory grid ratings was motivated by the search for synonymous adjective usage as well as for underlying dimensions on which each participant rated the transformations. In order to comprehensively show the different construct usage, repertory grids of participants were visualised as biplots (Greenacre Reference Greenacre2010) with the first two components from a principal component analysis (PCA) aligned along the x and y axis. The relative values of every stimulus on a given construct can be determined by vertical projection of its point onto the construct axis (Heckmann Reference Heckmann2016).
Biplots for four participants P1, P2, P5 and P6 are shown in Figures 2–5 respectively. The P1 biplot (Figure 2) shows a first component on its horizontal axis accounting for 45.4% of all the variance in the grid. The construct jagged–fluid, which is attributed to the physical property construct group (P), corresponds strongly with this first component. The vertical axis accounts for 27.5% of the total variance and the constructs edgy–smooth (also P) and hesitant–confident (action group A) are close to it. It is important to line out the synonymous usage of these two constructs despite their attribution to different groups. This makes the construct pole confident appear similar to smooth, and pole edgy similar to hesitant. Two other constructs are used in related ways as schizophrenic–compact (group X) is aligned with scattered–congealed (P), implying a likewise meaning of schizophrenic and scattered in this context. Finally the construct hazy–clear (P) is situated halfway between the two primary axes. Visual grouping of this participant’s constructs can indicate the presence of three underlying dimensions and their usage in the rating task. As perpendicular projection of a soundfile stimulus onto a construct gives the strength of this stimulus’s rating, transformations e7 and e6 show high ratings on the poles delineated by schizophrenic, scattered and jagged. The algorithms producing these transformations were an irregular amplitude modulation and a 225 millisecond spectral freeze respectively.
Biplot of the repertory grid for participant P1.
Biplot of the repertory grid for participant P2.
Biplot of the repertory grid for participant P5.
Biplot of the repertory grid for participant P6.
Ratings from participant P2 result in a biplot with two underlying dimensions well aligned with the horizontal and vertical principal component axes (Figure 3). Most soundfile stimuli were rated on the positive half of the horizontal axis, describing the severity of modification to the sonic material. Similar use of the construct poles separated, resonant and rugged as well as of their antipodes is apparent. The constructs rugged–even and separated–integrated (both group P) are used along resonant–immediate from the physical change group (C).
Another threefold similarity can be seen in the biplot from participant’s P5 data (Figure 4). Construct poles submerged, muffled and dull (all group P) are used alike, as are their antipodes playful, thin and floral. The remaining constructs seem more independent, suggesting a total of four underlying dimensions. Soundfile stimulus e10, created with an audio delay whose 30 milliseconds delay time is modulated by lowpass filtered noise, is rated as corresponding the most with the left-hand primary axis.
The biplot for participant P6 (Figure 5) indicates three underlying dimensions and similar usage of omni–multi, the only construct not fitting any construct group, with distant–present (group P). The three constructs along the horizontal axes can be considered to be alike, relating construct poles dull, fuzzy and tight with each other as well as their opposite poles open, contoured and bright. A large number of soundfile stimuli align with the left side of these constructs for participant P6.
The use of biplots to analyse construct systems of single participants in the experiment presented here allows access to and interpretation of the constructs and ratings of each individual. A global analysis of all participant grids in combination did not give reliable principal components nor did it allow a clear grouping of constructs. It is left open to determine if these commonalities would show for constructs elicited from a group of participants collectively.
6. CONCLUSION
This experiment shows that it is possible to align characteristics of sound transformations with verbal bipolar constructs provided by musically trained subjects in a relative comparison test. The long duration and highly variable content of the soundfile stimuli limit the general validity of the results but the method can nevertheless serve as a promising procedure investigating the sound material of a given musical example. The Repertory Grid Technique employed here proves adequate to elicit a vocabulary for sound transformations from individuals. Personal constructs of this vocabulary were shown to be groupable in categories, of which the primary ones describe physical properties, qualities of temporal changes and actions attributable to internal state, intentions or consciousness of living beings. The category describing qualities of temporal changes reaches out for an investigation in relation to the concepts of Facture (Schaeffer Reference Schaeffer1966) as well as of Spectromorphology (Smalley Reference Smalley1997). The recording of the participants’ suitability rating for constructs and ratings alike permits a validation of bipolar adjective scales as being useful despite the absence of true opposites. Suitability ratings furthermore showed the applicability of existing constructs for newly introduced stimuli. The stimuli used in this work are made accessible to others for possible further investigation. The procedure presented here seems promising and useful in the structured exploration of new sonic material and in the description and transmission of musical intentions derived from sound transformations.
