3.1 Early models: dynamical systems

Modelling of vowel systems goes back to the early days of modern computing, when Liljencrants & Lindblom (Reference Liljencrants and Lindblom1972) placed points in a 3D formant-frequency space, imposed the constraint that points should try to ‘contrast’ (i.e. be as far apart as possible in perceptual space) and simulated the resulting dynamical system. The resulting arrangements of points achieved a reasonable level of match – judged impressionistically – with real vowel systems, although with a number of discrepancies: for example, predicting [ɨ] rather than the more frequent [ø] in seven- and eight-vowel systems. The conclusion was that ‘contrast’ is an important constraint determining the shape of human vowel systems.

This early model already demonstrates the importance of a key factor in all mathematical modelling of real-world systems: how abstract shall we be, and why are we making various abstractions, and does the resulting loss of fidelity in the modelling vitiate the conclusions? Is it safe to assume a spherical cow? Liljencrants & Lindblom make several abstractions that are repeated in the later work discussed below. I list some of them here, together with prima facie criticisms, which I examine more carefully in what follows.

(i) Vowels are represented by the first three formant frequencies. Despite debates on formant vs. whole-spectrum perception, the issues of speaker variation, etc. (see e.g. Johnson Reference Johnson, Pisoni and Remez2005), there is, and was, enough evidence for the key role of formants in vowel perception that this decision is likely to be accepted by most.

(ii) The boundaries of vowel space are defined by constructing a model of the vocal tract and basic articulators (Lindblom & Sundberg Reference Lindblom and Sundberg1969), to match ‘a typical male speaker’, and computing the possible formant frequencies emitted by this model. This is not justified over the alternative of using data from actual speakers; it was perhaps simply easier at that time to use their existing model to generate extremes than to use humans.

(iii) The acoustic vowel space is mapped to a perceptual space by converting Hertz to mels. The justification is that mels are by definition constructed to match the response of the human auditory system, and this should also apply to our perception of formants. However, over the range of frequencies that formants take, the non-linearity of the mel scale is not very marked, so it is not clear that this step had any significant effect on the results. (There was, indeed, existing work (Stevens Reference Stevens1952) suggesting that a linear scale might anyway have been better.)

(iv) The authors need a measure of how widely dispersed – how contrastive – a set of vowels is, in the perceptual space. They choose the measure in (1), where rij is the Euclidean distance between vowels i and j.

1. (1)

   

This is justified by analogy with physics, where many problems about bodies interacting via forces are solved by minimising the potential energy induced by the forces. In fact, their analogy fails, since the potential energy is the integral of the forces – contrary to their stated intention, they are using a 1/r3 force, penalising close-by vowels very strongly. It is not discussed whether the choice of a 1/r measure instead of a 1/r2 measure would have a significant effect.

(v) ‘To make computations somewhat easier’ (Liljencrants & Lindblom Reference Liljencrants and Lindblom1972: 841), the 3D space is projected down to a 2D space using F1 and a linear combination of F2 and F3 (still in mel space). On the face of it, this should make a dramatic change: confined to a 2D circle, the minimum energy configuration of four points is a square, whereas, confined to a 3D sphere, it is a tetrahedron, which might project to a quadrilateral, or to a triangle with an internal point, depending on the projection.

(vi) Finally, the results of letting this system evolve until it appears to have reached equilibrium are compared with the distribution of real-world vowel systems. However, these systems are of course generally reported only at the coarse level of one or maybe a half division on the standard IPA vowel chart. Matching of the ‘predicted’ system to real systems is impressionistic.

This short and incomplete list illustrates several of the reasons for making abstractions, and also several of the dangers. (i) is a simplification of complex reality, but with good evidence that the simplification should preserve behaviour. (ii) and (iii) are cases where a more detailed realisation is chosen for the sake of matching ‘reality’ better, but without an argument that the increased fidelity is needed. (iv) represents a poorly motivated choice for what may or may not be a critical function needed by the model. (v) shows a simplification purely to make the problem more tractable, with some justification from psychological reality, but one that may considerably change the behaviour of the system. (vi) demonstrates the difficulty of comparing even a very abstracted mathematical model with data obtained by others for their own, descriptive linguistic, purposes.

The model is conceptually simple, and aims simply to demonstrate that human vowel systems try to maximise contrast (learners of Danish may be forgiven scepticism). In fact, it is sufficiently conceptually simple that the entire numerical simulation set-up is unnecessary. One can reason analytically that, given a 1/r2 potential and the convex boundary shape, as the number of vowels increases, they will distribute around the boundary until the centre becomes nearer to an edge than the distance between vowels on the edge. Indeed this applies to a 1/r potential, or even any potential corresponding to a repulsive force, which shows that their accidental mismatch with physics was not important. At this point, centralised vowels appear, and typically there will be many minimal configurations – in a circular space, this happens when the sixth and seventh vowels are added. Which one Liljencrants & Lindblom's (deterministic) simulation arrived at will have depended on arbitrary factors of precision and rounding. With modern computing power, variation due to such factors is easy to explore; with the resources of 1970, it was not – their calculations, which would now take a fraction of a second, ran overnight.

It is, then, not clear that the entire paper says much more than ‘if vowels are subject to a pressure to contrast, they will be arranged to maximise contrast’, which is almost a truism. More importantly, the reverse implication, which is what the paper is aiming to establish, does not follow. To show what was claimed, the authors would need to have shown that if there is no pressure to contrast, then human vowel systems do not arise. It is trivial that without contrast or any other constraint, their model would allow random vowel systems; but it is not trivial to show that there is no other constraint than contrast which might lead to the human configurations.

I do not, of course, doubt that maximising contrast is a major factor in the shape of vowel systems, but Liljencrants & Lindblom (Reference Liljencrants and Lindblom1972) provided no substantive support for that belief, which rested quite adequately on informal considerations of communicative efficiency.

3.2 An agent model

As modern computing developed, it became possible to make effective use of models based on many interacting particles (e.g. nuclei) or elements (e.g. cells of atmosphere, or portions of an aircraft wing). The social sciences were quick to experiment with such models, with linguistic simulations appearing as early as Klein (Reference Klein1966). In such models, the term ‘agent’ refers to a process modelling some kind of human (or animal) actor, and which typically interacts with many other agents all running independently. In this subsection, I consider one of the best-known fairly recent phonological examples, again looking at vowel systems.

De Boer (2001) created a model, not of abstract synchronic vowel systems where one is looking for a defined ‘optimal’ system, but of speakers transmitting vowel systems through the generations. The basic framework is the ‘imitation game’ described by Steels (Reference Steels1997), which is an instantiation of a framework going back at least to early 20th-century work in population genetics. As applied by de Boer, the key points, and key abstractions are:

(i) A population consists of a number of agents.

(ii) Each agent possesses an inventory of ‘vowels’, identified as points in a 3D ‘articulatory’ space modelling height, backness and rounding.

(iii) Agents interact by events in which: (a) A chooses a vowel V from its inventory; (b) A ‘says V to B’: transforms V into a 3D ‘acoustic’ space (modelling the first three formants), and transmits it to B; (c) B transforms the acoustic signal to a 3D ‘perceptual’ space (essentially formants modified by mel and other psycho-acoustic results), and matches it against B’s own inventory (by ‘saying its vowels to itself’ and seeing which is the best match, according to the Euclidean distance in perceptual space), identifying a vowel V′; (d) B says V′ back to A; (e) A matches V′ against its own inventory, and signals ‘extralinguistically’ to B whether it heard V, the vowel it first sent; (f) if there was a match, B marks a successful communication against V′, and moves V′ somewhat in the direction of V (that is, reinforcement learning); if there was no match, B marks a failure against V′.

(iv) From time to time, agents spontaneously add a new randomly situated vowel to their inventory, remove repeatedly unsuccessful vowels, and merge vowels that are perceptually ‘too close’ together.

(v) Some random noise is added at all places where it can be: in the positions of the selected vowels, and at transformations between spaces.

Given this setting, a population of a dozen or two agents is allowed to interact for a few hundred or thousand transactions, and then the inventories of the agents are examined to see how well they match human systems – by impressionistic comparison of pictures. It is claimed that plausibly human-looking vowel systems emerge from this procedure.

Following the discussion in the previous subsection, it will be apparent that this work also contains some decisions that might be ‘bad abstractions’, or ‘bad non-abstractions’, as well as ‘good abstractions’ or ‘good non-abstractions’ (by non-abstraction, I mean a deliberate decision to retain some detail). De Boer does discuss some of these issues at some length, though without, in my view, complete success (see also Donegan's Reference Donegan2004 review for a critical view). One of de Boer's most interesting sections occurs only in his original thesis (Reference Boer1999: §3.1): he first tried to use a fairly concrete articulatory model, similar to that of Liljencrants & Lindblom, to generate not only the boundaries of the vowel space, but also the formant values. Moreover, he also tried to model consonants and distinctive feature phonology. This (hugely ambitious) project resulted in sound systems with ‘no relevance whatsoever to understanding human sound systems’ (1999: 27), and he scaled his ambitions back to the simpler model outlined above. Why did the first model fail? I am confident it was because of the uncontrollable interactions of bad abstractions and bad non-abstractions.

De Boer (2001) has been influential. It also sparked my own interest in simulations in phonology, and consequently I decided to examine it more closely, by re-implementing his framework and attempting to tease apart which abstractions and non-abstractions were good or bad. This I discuss in the following subsections.

3.3 Detail, parameters and abstraction in simulation

I re-implemented the framework of de Boer (Reference Boer2001) in Java. Thanks to de Boer's careful descriptions, this was for the most part straightforward.

One aspect of simulation models that I have not yet mentioned explicitly is the role of parameters. Almost every model will end up containing a number of numerical values or functions which encode choices not determined by the underlying theory. In the Liljencrants & Lindblom (Reference Liljencrants and Lindblom1972) model, examples include the strength of the contrast measure, the details of the projection from 3D to 2D, the precise details of the numerical hill-climbing algorithm, and so on. De Boer was unusually careful about mentioning these, but nonetheless missed one (at least; I added only those parameters essential for the model). There are a dozen parameters, and there is no really principled choice of value for most of them.

Figure 1 shows two example runs of my implementation of de Boer's (Reference Boer2001) original model, plotting the vowels of 20 agents, each starting with a single random vowel, after a few thousand interactions. In Fig. 1a, ‘standard’ parameter settings (copied from de Boer) have been used. The community has more or less converged on a five-vowel /i a ɨ ~ ə o u/ system. For the particular parameter settings used, four or five vowels are typical; by tweaking parameters, fewer or more vowels can be induced to emerge. However, in Fig. 1b, one parameter, artEpsilon, has been increased a little; this governs the amount by which successful listeners accommodate their vowel to what they heard. Here, /i u/ are reasonably clear, but there is no convergence in the lower half of the vowel space, and indeed more detailed inspection shows speakers with anywhere from four to ten vowels in their inventory.

Figure 1

Two runs of the de Boer (2001) algorithm, with different parameters.

This set-up, with its repeated, extensively randomised interactions, is not easily amenable to direct analysis in the way that Liljencrants & Lindblom's was. It is possible that hybrid system techniques or continuous approximation techniques could be applied, but this has not been done. The set-up has been treated as a type II model, whether or not it is one. Despite this, and although de Boer was explicit in avoiding the teleological ‘contrast-maximising’ inherent in Liljencrants & Lindblom (Reference Liljencrants and Lindblom1972), we might still ask whether anything can be learned from the simulation. The explicit biases in the model are to converge with one's peers, and to maintain the identity of phonemes until they become too close together. Implicitly, however, the convergence, together with deletion and merging, fairly directly have the effect of biasing toward contrast. As for the shape of the vowel systems, that these match (somewhat) with human vowel systems is a simple consequence of contrast being a function of distance in the vowel space, which makes vowels disperse just as much as Liljencrants & Lindblom's explicit potential function. In some ways, this model would be more interesting if it failed.

3.4 When are details necessary?

Having established the original model, we can ask which of its details are necessary to the results. In particular, the model has a fairly detailed model of transformations between (pseudo-) articulatory, acoustic and perceptual space. Is this detail of any importance? That is, when we've modelled a basically spherical cow, is there any point in modelling its several stomachs, rather than just having one tube? To investigate this, I produced a simplified version of the model. I threw away almost all the phonetic detail: vowels become simply points inside the unit height/backness/rounding cube, and are transmitted to the listener with no change other than the addition of noise. The only concession was to compress the perceptual front–back dimension of lower vowels, and reduce the size of the rounding dimension, to match the shape of the usual vowel chart, and so make merging more likely in low vowels.

Figure 2 shows a run of the simplified system; parameter settings are mostly the same as in Fig. 1a, save that the perceptual parameter has been adjusted for the new space. One can see that this population has mostly settled on a five-vowel system, /i ε ɑ ɔ u/, which is both more tightly grouped and more human-looking than that in Fig. 1 (though there's a noticeable [a–ɑ] split in the /a/ vowel).

Figure 2

A run of the simplified model.

The sceptical reader will for some time have been asking a rarely addressed question: are these pictures cherry-picked? Let me be honest. Of course they are, as they almost certainly are in every simulation study ever published (apart from those who do carefully present composite data on many simulations). However, they are reasonably representative of typical runs, judged impressionistically. The sufficiently interested reader can run the programs to see – source code is available in online supplementary materials.Footnote ¹

With more experimentation, it turns out that, at least to my eyes, the simplified model produces ‘more human’ vowel systems not just for five-vowel systems, but for higher numbers of vowels also. It is tempting to conclude, therefore, that de Boer's fairly detailed articulatory–acoustic–perceptual model was a ‘bad non-abstraction’ – retaining fidelity to reality gives less persuasive results. This is a somewhat disturbing conclusion, as indeed de Boer must have been disturbed when his first, ambitiously detailed, model, gave very poor results. We must hope that the cause lies in inaccuracy of the details, rather than in the decision to retain details.

However, an alternative to being disturbed is to ask whether we can learn something from the more abstract solution giving a better fit. This leads on to my second case study.

4.1 Learning prototypes or features?

Boersma & Chládková (Reference Boersma and Chládková2010) were interested in whether people learn vowels as prototypes – that is, /e/ is a point in phonetic space which identifies the ‘best’ /e/ – or whether it can be argued that people learn to discriminate categorical features, so that /e/ is indeed a feature bundle.

Boersma has for some years been developing a sophisticated phonological grammar, one slogan of which is ‘bidirectionality’: the grammar is layered into modules, which feed up and down to each other. The grammar is constraint-based, in the style of Optimality Theory (Prince & Smolensky Reference Prince and Smolensky1993), though some of the constraints used are rather different from those of typical OT phonology: for example, so-called ‘cue constraints’ allow the specification of a vowel by weighting its formants against a large number of points on the frequency spectrum, and thereby allow vowels to be modelled as diffuse, distributional entities rather than points.

Boersma & Chládková (Reference Boersma and Chládková2010) set up a simulation framework in this grammar, with agents learning a five-vowel system, by re-weighting constraints on the basis of input. They report that when they simulate learners who are learning prototypes, the perceptual boundaries of the agents (as determined by testing them on an array of vowels) are drawn ‘diagonally’. On the other hand, if the learners are learning to associate vowels with [±high], [±back], etc., then the boundaries are horizontal and vertical. This is illustrated schematically in Fig. 3.

Figure 3

Vowel-perception boundaries in (a) prototype and (b) feature models, after Boersma & Chládková (2010).

Savela (Reference Savela2009) carried out extensive experiments testing the perceptual boundaries of vowels for speakers of many different languages, using artificial stimuli. The results appear to support horizontal/vertical divisions, rather than diagonal divisions. Thus, Boersma & Chládková say, their simulations support the claim that people learn features, not prototypes.

There is also a more specific claim about two languages. Spanish and Czech both have standard five-vowel systems. It is, however, claimed that Spanish phonology makes /a e/ [central], while in Czech /a/ is [back] and /e/ [front]. Running simulations learning these feature specifications predicts different perceptual boundaries, shown in Fig. 4. Boersma & Chládková conducted perception studies on Czech and Spanish speakers, and their results matched Fig. 4. (Clearly, they are assuming ternary height and backness features.)

Figure 4

Predicted boundaries in (a) Spanish and (b) Czech, after Boersma & Chládková (2010).

4.2 Analysis and response

We can make two criticisms of the argument, both of which echo criticisms made above of Liljencrants & Lindblom (Reference Liljencrants and Lindblom1972). The first is that the simulations add nothing to the argument. If agents model vowels as prototypical points, and incoming vowels are matched by Euclidean distance to the nearest such point, the boundaries in Fig. 3a are natural. Similarly, if they learn features that are defined to be horizontal and vertical divisions of vowel space, the boundaries in Fig. 3b are natural. The second is that, even granting the argument that the empirical results are consistent with feature learning, there might be other plausible explanations that would give similar results. In particular, could a different ‘prototype’ learning model account for the facts? I explored the latter by refining the simplified de Boer model introduced above. Recall that in this model, vowels were essentially points in abstract vowel space, with no subtleties of articulation or acoustics, and a minor adaptation for perception to compress low vowels. To explore the question, I made two changes to the model.

(i) Because we wish to study the evolution of the realisation of a stable vowel system, rather than the development of vowel systems, I split the population into adults (whose systems do not change) and children, who learn their vowels by interaction with adults according to the imitation game. As adults age and die, children replace them to maintain the population, and then become adults.

(ii) I enriched the notion of vowel, loosely inspired by exemplar theory (Pierrehumbert Reference Pierrehumbert, Gussenhoven and Warner2002). However, rather than exemplar clouds for words, or even phonemes, I proposed a simple model in which, for a vowel V, agents have an articulatory prototype for V, but also a convex polygonal region of space encompassing vowels they have heard and recognised as V by nearest matching. The prototype is nudged by the examples they hear; but the perceptual recognition region only expands.

(iii) I ignored the rounding dimension for this simulation.

I also went back to the literature to investigate Spanish and Czech vowels (Harmegnes & Poch-Olivé Reference Harmegnes and Poch-Olivé1992, Ekštein Reference Ekštein2004, Volín & Studenovský Reference Volín and Studenovský2007, Chládková et al. Reference Chládková, Escudero and Boersma2011). I did not find strong support for the claim that Czech /a/ is [back]; indeed, some argue it is [central] (Aleš Bičan, personal communication). I did, however, find that a typical realisation of Czech /a/ is somewhat backer than Spanish /a/, that Spanish /e/ is typically higher than Czech /e/, and that the participants in Savela (Reference Savela2009) showed a similar slight difference in their judgement of the ‘best’ examples. Figure 5 shows schematically perceptual boundaries and best examples from Savela (Reference Savela2009), and Fig. 6 shows Fig. 4 adapted to the usual vowel quadrilateral.

Figure 5

Perception of (a) Spanish and (b) Czech vowels, adapted from Savela (2009). Synthesised vowels are plotted in acoustic F1, F2 space; the symbols indicate how humans classified vowels at each point. The grey circles mark the locations judged to be the ‘best’ representatives of the five vowels.

Figure 6

Results from Boersma & Chládková (2010), redrawn on the standard vowel quadrilateral: (a) prototype learning; (b) feature learning. The shaded areas indicate the perceptual regions after training. The white circles show the centres of the vowel distributions used in training.

I converted average formant data from the sources to positions on the abstract vowel chart, and used these positions to seed the articulatory prototypes for ‘Czech’ and ‘Spanish’ populations. After running the simulation for a few generations, I measured the perceptual spaces of the agents. The results, together with the original prototypes, are shown in Fig. 7.

Figure 7

Agent simulation of vowel learning in the prototype and perceptual region model, with (a) Spanish and (b) Czech prototypes. The grey circles mark the locations of the prototypes in the initial populations. The shaded squares demarcated by black lines show the perceptual regions of an instance of a new adult after a few generations.

It is immediately noticeable that the rather small difference in the positions of the prototypes has a noticeable effect on the perceptual regions that agents acquire; that, in particular, the apparent slope of some region boundaries is changed from fairly diagonal to fairly horizontal; that, in general, the shape of these boundaries is not clearly identifiable as more similar to simple prototype-learning predictions than simple feature-learning predictions; and that the results match the empirical results of Savela (Reference Savela2009) rather well, particularly given the purely geometrical notion of vowel I use.

I have to admit that these results surprised me. The prototype-and-perceptual-region model is sufficiently complex that a simple qualitative analysis is not obvious, but my intuition was that the perceptual polygons would approximate circles, and the perceptual boundaries would end up in much the same place as before.

To summarise, I have here argued that two simulations with contrary intents give similar results, so that the original simulation does not itself argue strongly for feature-based perception. Of course, there are other ways to argue for it, and Chládková et al. (Reference Chládková, Boersma and Benders2015) have recently presented an elegant perception discrimination experiment which provides direct evidential support for the perception of features, so vindicating their original contention.