2.1 Data and analysis

As reported by Moreux (Reference Moreux1985, Reference Moreux2006), Rizzolo (Reference Rizzolo2002) and Eychenne (Reference Eychenne2014), certain dialects of Southern French have a synchronic process which tenses mid vowels in open syllables, while laxing them in closed ones (as well as open syllables followed by a schwa syllable; we will disregard this complication, and refer the reader to Selkirk Reference Selkirk1978, Moreux Reference Moreux1985 and Rizzolo Reference Rizzolo2002 for potential explanations of this). This is conventionally called the loi de position (‘law of position’), and is illustrated in (3) (examples from Rizzolo Reference Rizzolo2002 and Eychenne Reference Eychenne2014).

1. (3)

   

Tensing/laxing is made opaque (is counterbled) by resyllabification across word boundaries, at least in some cases, as shown in (4b), from Rizzolo (Reference Rizzolo2002: 51), where the lax vowel //œ// is retained in the surface open syllable [pœ].Footnote ¹

1. (4)

   

In the simulation reported here, we investigated whether our learner could deal with this simple case of opacity. This also serves as a simple case to illustrate in more detail the functioning of the learner. The data that we offered the learner consisted of exceptionless loi de position as well as exceptionless opacity through resyllabification, as in (5).

1. (5)

   

The constraint set that we used for this subset of real French was maximally simple, and is given in (6). Correct syllabification was taken for granted, in order to keep the constraint and candidate sets as small as possible, so that the interactions yielding opacity are easy to see.

1. (6)

   

The original formulation of Stratal OT (Bermúdez-Otero Reference Bermúdez-Otero1999, Kiparsky Reference Kiparsky2000) allows for three levels of derivation: a stem level, a word level and a phrase level, which all have the same constraints, but different rankings or weightings. For this problem, we will only consider two levels – a word-level and a phrase-level grammar. The word-level grammar evaluates each word individually, without regard to its neighbours in the phrase. By contrast, the phrase-level grammar, which operates on the output of the word-level grammar, does evaluate entire phrases together.

With this set-up, then, opaque interaction is obtained when word-level *[+tense]/closed and *[–tense]/open have high weight and word-level Ident(V) has low weight, while the opposite holds at the phrase level. This corresponds to activity of the loi de position at the word level, and its inactivity at the phrase level.

This latter scenario is the only option to derive this opaque interaction. As can be seen in tableau (8) below, phrase-level markedness disprefers surface [sε.ta], so that the mapping from /e/ to [ε] cannot be derived at the phrase level. At the same time, word-level markedness prefers the word-level output that does map /e/ to [ε], //sεt#a//, because the first vowel is in a closed syllable at that level, since the phrasal context is invisible. This can be seen in tableau (7a).

Activity of loi de position at the word level can be expressed with the weights in (7) (found in successful runs of the learner described in §2.2).

1. (7)

   

At the phrase level, giving high weight to Ident(V) and zero weight to both markedness constraints results in a probability of 1.00 for all faithful mappings. This is illustrated in (9) for the phrase-level derivation that takes place if the word-level form output is //sεt.#a//: phrase-level outputs retain the [ε] that was created by closed-syllable tensing at the word level with a probability of 1.00.

1. (8)

   

The single-stratum mappings shown in (7) and (8) are assembled into derivational paths that lead from an underlying representation (UR) to a surface representation (SR). The probability of a single derivational path is obtained by multiplying the probabilities of every step in the path, as illustrated in (9). This is because every derivational step in Stratal OT is by definition independent of earlier or later derivational steps (Bermúdez-Otero Reference Bermúdez-Otero1999, Kiparsky Reference Kiparsky2000).Footnote ²

1. (9)

   

The expected probability of a surface form (more accurately, a UR/SR mapping) is the sum of the probabilities of all derivational paths that lead from the input to that surface form, as in Table I.Footnote ³ This table shows that, given the weights in (7) and (8), the desired output candidates (cf. (5)) are generated with a probability of 1.00.

Table I

Computation of expected probabilities of UR/SR mappings by summing over all derivational paths.

This serial approach does not have look-ahead capabilities: the phrase-level grammar cannot influence which mappings will be preferred at the word level, in accordance with Bermúdez-Otero (Reference Bermúdez-Otero1999). A closely related model that does have look-ahead capabilities is explored by Boersma & van Leussen (Reference Boersma and van Leussento appear), whose predictions for the same dataset would also be interesting to explore in future work.

Now that we have explained and illustrated the generation of probabilities over UR/SR mappings using the successful weights in (7) and (8), in §2.2 we will explain the structure of the learner that arrived at these weights. In §2.3 we will show how often this learner arrives at this successful set of weights within a sample of 100 runs, and what happens within that same sample when the learner does not find this analysis.

2.2 Learning

Staubs (Reference Staubs2014b) provides an implementation in R (R Core Team 2013) of a variant of the approach to learning serial grammars discussed in Staubs & Pater (Reference Staubs and Pater2016). We modified it minimally to allow different violations of the same constraint at different derivational stages, which is sometimes necessary in a stratal framework.

For instance, the constraint *[+tense]/closed in the Southern French simulation will be violated at the word level by a sequence [set#a], because even though [ta] could form a separate syllable, the word boundary between [t] and [a] makes it necessary to syllabify [t] as a coda. However, the same constraint remains unviolated at the phrase level for the same sequence, since the grammar can now look beyond word boundaries and evaluate entire phrases, so that [ta] does form a syllable: [se.ta].

Expected probabilities of UR/SR mappings are computed given sets of candidates, violation vectors and constraint weights, as described and illustrated in §2.1. In addition, the learner is also given observed probabilities of UR/SR mappings. Since we are working with categorical data, all UR/SR mappings found in the data were given a probability of 1.00, and all others a probability of 0.00. The learner then minimises the Kullback-Leibler (KL) divergence (Kullback & Leibler Reference Kullback and Leibler1951) of the expected probabilities from the observed probabilities, which is a measure of how closely the grammar has been able to fit the data; see (10).

1. (10)

   

Minimisation of this objective function is achieved by the L-BFGS-B method of optimisation (Byrd et al. Reference Byrd, Lu, Nocedal and Zhu1995). This is a pseudo-Newtonian batch-optimisation method, which uses approximations of the first-order and second-order derivatives of the objective function to take incremental steps towards an optimum. The ‘optim’ implementation of this method in R was used, specifying a minimum of zero on all constraint weights.

To place an upper bound on the objective function, an L2 regularisation term (Hoerl & Kennard Reference Hoerl and Kennard1970) with μ=0 and σ2=10,000 was added. This regularisation term penalises weights as they increase, keeping the optimal solution finite, and driving a constraint's weight down to zero whenever it is not helpful in the analysis.

Summarising, our algorithm calculates probabilities of derivations in Stratal MaxEnt, sums over them to get the expected probabilities of overt forms and fits the observed distribution of UR/SR mappings by minimising KL divergence.

2.3 Results

The learner described in §2.2 was run 100 times on the French loi de position data, with every run starting from random starting weights for each constraint, drawn with replacement from a uniform distribution over [0, 10]. The results are given in Table II.

Table II

Results for 100 runs for the Southern French dataset.

Thus opaque loi de position was learned successfully for an overwhelming majority of start weights. However, there is still a slight chance of incorrect learning, which we defined as a probability of 0.50 or less on at least one surface form that has a probability of 1.00 in the learning data (see also §1). The four unsuccessful runs yielded grammars which had free variation between tense [e[ and lax [ε], as in Table III.

Table III

UR/SR mapping probabilities for the local optimum.

This is a local optimum that arises when all constraint weights are set to zero. KL divergence from the actual data (our measure of error) is smaller for these weights than for any other weights within the weight space explored by the learner.

As will be explained in §4.1, an initially incorrect weighting at the word level leads to a tendency for the weight of phrase-level faithfulness to be lowered. At the same time, when phrase-level Ident(V) reaches zero weight, any effect of closed-syllable laxing from the word level cannot be transmitted to the final output. Once this happens, 0.50 probability for all candidates is the best possible fit to the data, and the all-zero weight version of that solution minimises the penalty on the regularisation term, which prefers smaller weights.

It could be argued that this weight set is not a local optimum in the strict sense. There is a locally available solution that is better than all-zero weights: assigning minimal weight (0+ε for any positive ε) to word-level *[–tense]/open and *[+tense]/closed, as well as phrase-level Ident(V), while keeping all other constraints at 0, will lower the objective function with respect to all-zero weights, as this will nudge the probabilities of both /set#a/ → [sε.ta] and /se#ta/ → [se.ta] slightly above 0.50. However, for the reasons outlined above, the learner cannot see this possibility when it retreats to all-zero weights.

3.1 Data

Our second case study is a classic case of opacity, attested in Canadian English (Joos Reference Joos1942, Idsardi Reference Idsardi2000 and references therein, Pater Reference Pater2014): the interaction between Canadian raising and flapping. The low-nucleus diphthongs [aɪ aʊ] raise to [ʌɪ ʌʊ] before voiceless consonants, as in (11a), and the coronal oral stops /t d/ become a voiced flap [ɾ] in a variety of contexts (De Jong Reference De Jong, van Oostendorp, Ewen, Hume and Rice2011) – among others, between two vowels if the first is stressed and the second is not, as in (11b).

The two processes display a counterbleeding interaction: the fact that flapping cancels the voicelessness of underlying /t/ does not prevent that /t/ from triggering raising. This is illustrated in (11c).

1. (11)

   

In addition, raising is restricted to the word domain, as illustrated in (12a), while (b) shows there is no evidence of such a restriction for flapping.

1. (12)

   

This fits with a stratal analysis in which raising applies only at the word level, and flapping only at the phrase level, as in (13).

1. (13)

   

The transparent counterpart to this pattern, claimed to be spoken by some Canadian English speakers (Joos Reference Joos1942), is a language which also has both raising and flapping, as in (11) and (12), but in which the application of flapping blocks the application of raising, as in (14), since the flap [ɾ] is not voiceless. However, the existence of this transparent dialect is disputed in later literature (Kaye Reference Kaye, Mascaró and Nespor1990). In our simulations, we use this (perhaps hypothetical) transparent dialect as a non-opaque baseline against which to compare the mainstream, opaque dialect of Canadian English.

1. (14)

   

In the next subsection, we investigate the learnability of the opaque and the transparent versions of the Canadian English data. We consider the contrast between opacity and transparency, and between various possible datasets.

3.2 Simulation setup

Canadian English provides at least two pieces of independent evidence regarding the stratal affiliation of the opaque raising process: evidence for its application outside the flapping context, as in (11a), and evidence for its word-boundedness, as in (12a). Based on Kiparsky (Reference Kiparsky2000), we predict the availability of such evidence to make the opaque interaction more learnable. To investigate this, we considered the four datasets in Table IV, each of which contains a subset of the relevant mappings observed in Canadian English, and provides various degrees of independent evidence for the stratal affiliation of the opaque process.

Table IV

Opaque datasets for Canadian English.

Mitre–cider provides no independent evidence regarding the opaque process – its synchronic activity and stratal affiliation must be inferred from the opaque interaction. Mitre–cider–life provides evidence for the opaque process applying outside the interaction, since the [f] in life is not subject to flapping: raising must be synchronically active at some stratum. Mitre–cider–lie for provides evidence for the non-application of raising at the phrase level, since lie for does not undergo raising, despite having [f] after the diphthong across a word boundary. Finally, mitre–cider–life–lie for combines both pieces of evidence. These datasets were investigated with both an opaque and a transparent interaction between raising and flapping.

Our learner had access to the four constraints in (15). As in the Southern French case, only two derivational levels were used: a word level and a phrase level.

1. (15)

   

Whenever *aɪ/[–voice] is sufficiently higher-weighted than Ident[low], the raising process will be active. The flapping process is active when the weight of *V́TV is sufficiently greater than that of Ident[son].Footnote ⁴ The opaque interaction is captured if there is raising but no flapping at the word level, and flapping but no raising at the phrase level (see §1 on the impossibility of capturing the interaction within one stratum). This is reflected in the weights found at successful learning trials for mitre–cider–life–lie for. At the word level, *aɪ/[–voice] is weighted far above Ident[low], and *V́TV far below Ident[son]; the opposite is true of the phrase level, as shown in Table V.

Table V

Sample successful weights for opaque mitre–cider–life–lie for.

The word-level tableau in (16) shows that the weights above ensure raising but no flapping at the word level.

1. (16)

   

The candidates considered at the phrase level are the same as those at the word level.Footnote ⁵ As can be seen in the phrase-level tableaux in (17), it is essential for the generation of these probabilities that /maɪtɚ/ exhibit raising at the word level, since the phrase level must preserve diphthongs faithfully, otherwise /maɪtɚ/ will not surface with both raising and flapping (i.e. as attested [mʌɪɾɚ]).

1. (17)

   

The surface-form probabilities resulting from the weights in Table V match the actual opaque Canadian English data extremely closely, as shown in Table VI.

Table VI

Surface form probabilities generated by Table V.

As opposed to the opaque interaction, the transparent interaction does not need to order flapping after raising. In fact, for transparent mitre–cider and mitre–cider–lie for, raising need not be represented in the grammar at all, because the raised diphthong is absent from the data – see the lack of weight on *aɪ/[–voice] at both levels in Tables VIIa and b. For transparent mitre–cider–life, raising can be represented either at the word level, by giving high weight to word-level *aɪ/[…voice] and zero weight to word-level Ident[son], as in (c) – or at the phrase level, by giving high weight to *V́TV and *aɪ/[–voice] at the phrase level, as in (d). Transparent mitre–cider–life–lie for, however, does require raising to take place at the word level, because the non-raised diphthong in /laɪ#fɔɹ/ → [laɪ fɔɹ] precludes raising at the phrase level. The weights found for this dataset are very similar to the second set of weights for mitre–cider–life, as shown in (e).

Table VII

Sample weights for successful runs of various transparent datasets.

Thus the opaque interaction of raising and flapping requires word-level raising and phrase-level flapping. The transparent interaction, however, only requires flapping at the word level, while raising can be represented at either level.

Finally, the addition of both life and lie for to the transparent dataset leads to the necessity of representing raising at the word level, even though this is not otherwise required for the transparent interaction.

3.3 Results

Simulations were run as described in §2.2, except that L2 regularisation was made stronger by setting σ2 to 9,000 instead of 10,000, to prevent the learner from considering constraint weights that tend towards infinity for the opaque mitre–cider dataset.

All four datasets were examined with both opaque and transparent interaction between raising and flapping. The same 100 sets of initialisations drawn with replacement from a uniform distribution over [0, 10] were used for all eight datasets. The results are given in Table VIII.

Table VIII

Results for 100 runs for the Canadian English dataset.

As can be seen, independent evidence regarding the opaque process yields a clear increase in learnability for the opaque cases. Evidence that the opaque process is word-bounded (lie for) has a stronger effect than evidence for independent activity of the opaque process (life).

For all transparent datasets except mitre–cider–life–lie for, performance is (almost) at ceiling. For mitre–cider–life–lie for, however, the opaque and transparent versions are learned at a near equal rate.

Whenever (opaque or transparent) Canadian English was not learned successfully (i.e. at least one attested SR was given a probability of 0.50 or less by the grammar), the learner ended up in one of the four local optima summarised in Table IX.

Table IX

Local optima observed for the various Canadian English datasets. Parentheses indicate words that are not shared between all datasets for which this local optimum is attested.

For each underlying representation the table lists the surface representations that have a probability of more than 0.00 in that local optimum. The symbol ‘~’ will be used as a shorthand for 0.50 probability on both surface representations shown, unless indicated otherwise. Weights generating each local optimum are given in Table X.

Table X

Sample weights for local optima.

These four local optima have in common that they try to represent both processes with minimal appeal to the interaction between levels. For instance, local optima I and IV are attempts to represent the data without appealing to underlying representations, by setting all faithfulness constraints to zero.

Local optimum II does not represent the raising process at the word level, as would be necessary for any dataset involving life with a raised diphthong and lie for with a non-raised diphthong (see §3.2). This means that the vowel contrast between life and lie for has to be modelled as within-word variation.

Finally, local optimum III is a consequence of representing both raising and flapping at the word level. Since applying raising and flapping at the same level leads to lack of raising in mitre, variation between raising and non-raising is created by lowering the weight of word-level Ident[low].

As in the case of Southern French, local optima I and IV are not local optima in the narrow sense: if the weights of word-level Ident[son], Ident[low] (in the case of local optimum I) and *aɪ/[–voice] and phrase-level Ident[low] are increased by even a little, this will decrease the value of the objective function. It is this ‘bottleneck effect’ (see §4.1) that leads the learner to land in this state. However, local optima II and III are true local optima, in the sense that any neighbouring values will increase the value of the objective function.

As a final note, none of these outcomes represent opaque analyses. Local optima II and III resemble opaque analyses, since they both assign a significant amount of probability to [mʌɪɾɚ], but they crucially ignore the difference between underlying /t/ in mitre and underlying /d/ in cider, and allow unmotivated raising in both words (because the constraints that regulate raising either have a weight of 0 or are tied with competing constraints).

3.4 Summary

We have found that independent evidence about the stratal affiliation of an opaque pattern can significantly improve the learnability of the opaque interaction – especially the addition of evidence that the process does not apply at the phrase level. Furthermore, the presence of both lie and lie for makes the transparent and opaque interactions equally learnable, and in fact destroys the transparent interaction's learning advantage.

Whenever the languages are not learned successfully, this is because either phrase-level faithfulness is given zero weight, making it impossible to transfer information from the word level to the phrase level, or raising and flapping are represented at the same level when they need to be represented at different levels. We will now turn to a discussion of the ‘bottleneck effect’ (see §3.3), an obstacle posed by the current hidden structure learning problem that leads the learner away from the global optimum, as well as ways in which evidence for stratal affiliation helps the learner overcome this obstacle.

4.1 Cross-level dependencies

The relative difficulty of learning opaque interactions found in our results seems to stem from the fact that the effectiveness of the weighting on each level depends on the weights on the other level. Specifically, high weight on phrase-level faithfulness is only effective when word-level constraints are weighted appropriately, while the result of word-level weighting can only be transmitted to the surface representation when phrase-level faithfulness has a high enough weight.

We will show here how the learner fails to reach the global optimum if it finds appropriate weights only for word-level constraints or only for phrase-level faithfulness, and we will show that this scenario is more likely to occur in opaque cases than in transparent cases. Furthermore, we will show how the types of evidence for stratal affiliation of the opaque process offered by the Canadian English datasets increase the likelihood that the learner will find the global optimum.

When the learner has not found weights that generate a desirable distribution at the word level, it gets closer to its objective by lowering the weights of phrase-level faithfulness, rather than raising them. For instance, consider the weighting in Table XI, which leads to a local optimum for opaque mitre–cider.

Table XI

Sample initialisation leading to local optimum for mitre–cider.

At the word level, this weighting gives a non-raised diphthong in mitre highest probability, because high-weighted *V́TV and low-weighted *aɪ/[–voice] leads to flapping, which blocks diphthong raising, as illustrated for the word-level tableau in (18).

1. (18)

   

At the same time, as shown in (19), it displays flapping and lack of raising on the phrase level, as desired for the opaque interaction (cf. §3.2).

1. (19)

   

In cases like this, KL divergence decreases (and fit to the data increases) as the weight of phrase-level Ident[low] approaches zero. As shown in (10) above, KL divergence is computed based on observed and expected probabilities. Table XII shows the observed probabilities for opaque mitre–cider, and Table XIII shows expected probabilities for various weights of phrase-level Ident[low] and the resulting KL divergence.

Table XII

Observed distribution for opaque mitre–cider.

Table XIII

KL divergence for grammars with word-level weights as in Table XI and various weights of phrase-level Ident[low].

If the learner has assigned zero weight to phrase-level faithfulness, moving towards appropriate weights at the word level does not lower KL divergence. This is illustrated in Table XIV. Word-level and phrase-level expected probabilities are shown only for mitre, but KL divergence is computed for both mitre and cider. Phrase-level constraint weights are as given in Table XI, except for Ident[low], whose weight is set to zero.

Table XIV

KL divergence for mitre–cider when phrase-level Ident[low] has zero weight.

As can be seen in Table XIV, raising the weights of word-level Ident[low] and *V́TV and lowering those of word-level Ident[son] and *aɪ/[–voice] leads to a desirable result at the word level: raising and no flapping (cf. (17)). However, this information is not factored into the distribution over UR/SR mappings if phrase-level Ident[low] has a weight of 0.

At the same time, phrase-level Ident[low] has a motivation to decrease its weight until word-level outputs with a raised diphthong gain a cumulative probability of at least 0.50. The lower the weight of Ident[low], the closer the winning candidate will be to having a probability of 0.50, since setting Ident[low] to zero means that the phrase level will assign equal probability to raised and non-raised diphthongs. As long as the current word-level weights are such that the winning candidate has less than 0.50 probability, this is an improvement. In the particular case discussed here, Ident[low] will be motivated to have non-zero weight only when the raised diphthong is more likely than the non-raised diphthong at the word level, so that the probability of [mʌɪɾɚ] increases as the weight of Ident[low] increases.

This is what yields the aforementioned bottleneck effect: if the current word-level weights lead to lower probability on the attested candidate compared to all-zero weights at the word level, then phrase-level faithfulness might be set to zero before a more desirable set of word-level weights is found.Footnote ⁶ Thus word-level information needs to travel through the bottleneck of the phrase-level faithfulness constraints in order to have an effect on the distribution over UR/SR mappings. The bottleneck effect is not limited to just this dataset, but applies to any dataset with dependence between levels, including the Southern French case in §2.3 above and other Canadian English datasets (see §4.2 below).

4.2 Advantage from evidence for stratal affiliation

As we showed in §3.3, the addition of various kinds of evidence for the stratal affiliation of the opaque process of raising dramatically increases the likelihood that the opaque interaction will be learned.

Adding life to the opaque interaction means that raising at either the word level or the phrase level will be rewarded, regardless of the weighting of the flapping constraint *V́TV. This means that increasing the weight of word level *aɪ/[…voice] leads to a sharper drop in KL divergence for opaque mitre–cider–life than for opaque mitre–cider–lie for, as shown in Table XV.Footnote ⁷

Table XV

Correlation between the weight of word-level *aI/[–vce] and KL divergence for mitre–cider and mitre–cider–lie for.

In this way, the presence of life in the dataset means a sharp increase in the gradient of the word-level constraint *aɪ/[–voice] in a position in the weight space that would otherwise have a low gradient for that constraint. We have not studied how the exact numerical increase in the gradient of this constraint, which depends on the relative frequency of data points like life in the total corpus (we only considered a minimal corpus here), relates to learnability rates – this is a matter for future research. However, we can confidently say that the categorical presence of raising outside its interaction with flapping has a positive influence on the learnability of the interaction.

Adding lie for to the opaque interaction, as in Table XVI, means that high weight on phrase-level Ident[low] is penalised less strongly when the word-level weights do not generate the desirable candidates with high probability.Footnote ⁸ This is because the word lie itself contains neither a flapping nor a raising context, so that flapping and raising constraints do not interact with the identity of the diphthong at the word level. It is not necessary to determine the mutual weighting of word-level *V́TV and Ident[son] or *aɪ/[–voice] and Ident[low] to give the desirable word-level output for lie for more than 0.50 probability – non-zero weight on word-level Ident[low] is sufficient.

Table XVI

Correlation between the weight of phrase-level Ident[low] and KL divergence for mitre–cider and mitre–cider–lie for.

When both life and lie for are present in the dataset, the grammar must represent the raising process at the word level in order to generate lack of raising in lie for (because word-level *aɪ/[–voice] cannot see that lie for has /aɪ/ before /f/, while phrase-level *aɪ/[–voice] can). As shown in §3.2, this introduces an additional dependency between word and phrase levels for the transparent datasets: while the transparent interaction between flapping and raising does not require raising to apply before flapping, the combination of life and lie for does.

This creates a bottleneck effect for the transparent dataset, which explains the decrease in learnability for transparent mitre–cider–life–lie for. The increase in learnability for opaque mitre–cider–life–lie for, on the other hand, can be seen as the cumulative effect of life and lie for on the opaque interaction, as reviewed above.