2.1 Measuring Orthographic Stem Distances

Baroni et al. (Reference Baroni, Johannes, Harald and Maxwell2002) presented an algorithm that, by taking a raw corpus as its input, produces a ranked list of morphologically related pairs as its output. The algorithm finds morphologically related pairs by looking at the degree of orthographic and semantic similarity between words from the input corpus. Experiments with German and English inputs gave encouraging results. In this study, orthographic similarity is calculated using edit distance, which is also known as levenshteindistance (Levenshtein Reference Vladimir1966). The Levenshtein distance between two strings is calculated as the “cost” of the total set of insertions, deletions, and substitutions needed to transform one string into another (Kruskal Reference Kruskal, Sankoff and Kruskal1999). The algorithm calculates orthographic distances by finding the minimum number of letters that need to be inserted, deleted, or replaced when changing the spelling of a word in one language into the spelling of the corresponding word in another language (see section 4).

Van Bezooijen & Gooskens (Reference Bezooijen, Charlotte, Doetjes and van de Weijer2005) considered orthography as an explanatory factor of intelligibility between Afrikaans and Dutch, and between Frisian and Dutch. They calculated orthographic distances by means of Levenshtein distance. The authors found that the orthographic distances of cognates that are related—directly or via a synonym—are much smaller between Afrikaans and Dutch than between Frisian and Dutch.

Zulu et al. (Reference Nicholas, Botha and Barnard2008) measured orthographic distances between 11 South African languages. Levenshtein distances were calculated using existing parallel orthographic word spellings in sets of 50 and 144 words from each of the 11 official languages of South Africa. These data were manually collected from various multilingual dictionaries and online resources. The authors concluded that statistical methods based solely on orthographic transcriptions are able to provide useful objective measures of language similarities.

Doetjes & Gooskens (Reference Doetjes and Charlotte2009) studied the role of orthography in the mutual intelligibility of Danish and Swedish spoken languages. They measured phonetic distances between the languages and took into consideration the help that the listeners can receive from the orthography when listening to the neighboring language. Both phonetic and orthographic distances were measured by means of Levenshtein distance. The authors concluded that Danish listeners indeed seemed to make use of the additional information that the orthography can provide.

2.2 Affix Distances

A simple way of measuring morphological distances between language varieties is counting the number of differences. Let us compare English morphology with Dutch morphology on the basis of a small sample of seven words in table 1 (see below). The affixes in the words appear in bold. We find that five out of seven words have different affixes in English and Dutch. The distance can be calculated as 5/7×100=71.4%. Reversely, the similarity can be calculated as 2/7×100=28.6%.

Table 1.

Comparing English and Dutch morphology.

Séguy (Reference Jean1971, Reference Séguy1973) and Goebl (Reference Goebl1982, Reference Goebl1984, Reference Goebl and Viereck1993) measured distances between local dialects at several linguistic levels in a similar way, namely, by simply counting the number of differences (Séguy) or similarities (Goebl) between two local dialects, considering a set of item pairs. Séguy focused on 154 local dialects, the data for which appeared in the Atlas linguistique de la Gascogne. Among other linguistic levels, he considered morphosyntax and verb morphology. Goebl analyzed l'Atlas Linguistique de l'Italie et de la Suisse Méridionale (AIS), compiled by Karl Jaberg and Jakob Jud in the first quarter of the 20th century. He measured lexical and morphosyntactic similarities.

Heeringa et al. (Reference Heeringa, Martijn, Boudewijn van den, Nerbonne, Lenz, Gooskens and Reker2009) used the same methodology in order to measure morphological distances between Dutch Low Saxon dialect varieties. They used data from the Morphological Atlas of Dutch Dialects (De Schutter et al. Reference De Schutter, Boudewijn van den, Goeman and Thera de2005, Goeman et al. Reference Goeman, van Oostendorp, van Reenen, Oele, van den Berg and van Reenen2009). Morphological dialect variation in the following domains was considered: plural sub-stantives, diminutives, possessive pronouns, verbs, participle prefixes, and verb stem alternations. The authors report that their data could be divided into four groups. This classification differs from the one on Daan's map (Daan & Blok Reference Daan and Dirk1969), the most recent traditional Dutch dialect map, which is based on the little-arrow method. On the map, places in which, according to the speakers, (nearly) the same dialects are spoken are connected by arrows. In that way, white strips arise where there are no arrows; these are the dialect borders. Daan's map shows nine groups.

In all of the examples mentioned above, categorical data were used. In order to find the right categories, historical knowledge may be required. In the dialect of Dordrecht, for instance, one can find the plural form huiz e ‘houses’. The plural suffix - e may be a realization of either the plural suffix -en or -er. By using historical knowledge we are able to decide whether the plural suffix -e is a reduction of -en or -er and thus whether it belongs to the category of words pluralized with the suffix -en or -er. In this case, the suffix is most likely a reduced form of -en, since the plural suffix -er is more commonly found close to the Dutch/German border.

Another example is the prefix in the past participle form of the verb werken ‘to work’. In dialects related to Standard Dutch it is gewerkt, whereas in other Dutch dialects it is ewerkt or werkt. In order to answer the question whether the prefixes ge- and e- are separate categories or not, one needs historical knowledge once again. The two examples show that determining morphological categories is not always easy.

It is our aim in this paper to model intelligibility. Since most language speakers are nonlinguists and, therefore, lack theoretical knowledge of morphology, we assume that intelligibility of affixes is not necessarily determined by historical classes motivated morphologically. Besides, by confining affix variation to a restricted set of classes we lose information that may play a role in intelligibility. For example, the double plural suffix in the English word child ren and the double plural suffix in the Dutch word kind eren historically belong to the same class. English -re is -er that underwent metathesis in English but not in Dutch, as seen in kind eren. English -n and Dutch -en have been added after the original plural suffixes (that is, -re and -er, respectively) had become unproductive. A Dutch nonlinguist, however, would not be aware of the fact that the English and Dutch suffix belong to the same class; rather she or he would perceive the English suffix -ren simply as a reduction of the Dutch suffix -eren.

A better way of modeling perceptual distances is using the Leven-shtein distance. This algorithm offers the most economical way of changing one string of sounds (for example, a word in a target language) into another (for example, the corresponding cognate word in the assumed reader's native language). This algorithm enables one to measure how much effort the reader needs to exert to understand the word in the target language. In order to find the most economical way of changing one string of sounds into another, the two words in question are aligned. When comparing the two suffixes, -eren and -ren, the algorithm finds one deletion, as shown in table 2.

Table 2.

Levenshtein distance: -ren versus -eren.

We assume that this approach better reflects the distance perceived by a reader than the one that relies on a reader's knowledge of historical facts, as only a minority of readers would have such knowledge.

Another example is English apostroph es versus Dutch apostrof s. The suffixes -es and -s belong to the same class, but when reading the words, the reader would likely notice that the English suffix has an extra e compared to the Dutch suffix. The Levenshtein distance method takes this difference into account. The Levenshtein distance would align the suffixes, as shown in table 3.

Table 3.

Levenshtein distance: -es versus -s.

In this paper, we use Levenshtein distance. Compared to earlier studies based on categorical data, our approach does not require historical knowledge in order to find the right categories, and it better reflects the distance perceived by a reader.

3.1 Selection and Alignment of Texts

The basis of our analyses is a set of four English texts at the B1/B2 level as defined by the Common European Framework of Reference for Languages (CEF).Footnote ² The texts were used as preparation exercises for the Preliminary English Test (PET). The diploma is offered by University of Cambridge ESOL Examinations, England. The texts used in the study were obtained at englishaula.com.Footnote ³

Each text was translated from English into Dutch, Danish, German, and Swedish by native speakers of those languages. The translations were subsequently proofread by two other native speakers. The number of words for each text in each language is given in table 4. There are no significant differences in the number of words across the five languages. In section 5.1, we show that our data set is sufficiently large for an analysis presented in this paper.

Table 4.

Number of words for each text, in each language.

The texts were then arranged in a table with five columns, one per language. Each column contains (the translation of) the four texts, where words are found below each other, each cell containing one word (see table 7). Generally, words are considered separate entities when they are separated by spaces. In case of compounds and verbs, however, groups of words in one language may be aligned with individual words in other languages. Some examples are given in table 5.

Table 5.

Alignment of compounds and multiword compound verbs.

When a group of words is found in one cell, we changed the order of the words to optimize the matching of a form in one language with the corresponding form in another language by the Levenshtein distance (see section 4.1). This is, of course, a somewhat artificial approach, since in the actual texts these words were presented to the reader in the original order. However, we assume that the Levenshtein distance obtained from the optimized order reflects the readers’ effort more realistically. Examples are given in table 6. The second row contains the forms as they appear in the original text. In the third row, the word order for English, Danish, Dutch, and German is changed, so that the forms can be optimally matched with the Swedish form when calculating Levenshtein distance.

Table 6.

Optimalization of word order for cells with multiple words.

Table 7 shows the first part of the table and contains the first part of the first sentence of the text “Child Athletes”. In the table, affixes appear in square brackets, which mark them as affixes for the algorithm and hence enable us to measure orthographic stem and affix distances separately (see section 4.3).

Table 7.

The first part of the table with the texts in the five languages aligned.

The complete table comprises the four texts, and is a large table that consists of 1259 rows. The table is stored as Microsoft Excel table.

3.2 Number of Word Pairs

Table 6 contains several empty cells. They are found throughout the table, since not every word appears in every language. For example, the English determiner a in the eighth row is not found in the Dutch column, since Dutch does not require a determiner in that position. The English word show has equivalents in Danish and Dutch but not in German and Swedish. The sentences translate less literally into German and Swedish, and, therefore, one finds the words haben and har, respectively, which generally have a different meaning. Therefore, these words appear in a separate row.

To calculate orthographic distances, our measurements are based on pairwise word comparisons within every pair of languages. Aligned data from five languages yield ten language pairs. However, due to the empty cells, not every word has a match. For example, the English word show (sixth row in table 7) does not have a German counterpart, and the German word haben does not have an English counterpart. If there is a pair, it is taken into account.

We also restrict our analysis to word pairs, whose members belong to the same word class. For example, in the text “Catching a Cold” the English phrase the best answer is translated into Dutch as het beste. Being originally an adjective, best is inflected as a noun in the Dutch sentence. It is tempting to match English best with Dutch beste, but since the English word is an adjective and the Dutch word is a noun, we do not match them. In this paper, we focus specifically on affix variation, but it would not be fair to compare affixes of words that belong to different word classes. Therefore, word pairs such as the English/Dutch pair best/beste were not included in the analysis. Table 8 shows the number of word pairs for each language pair.

Table 8.

The number of word pairs for each language pair.

For each language pair a large number of word pairs is found, varying from 534 (German versus Swedish) to 742 (Dutch versus English).

3.3 Number of Cognate Pairs

In our orthographic measurements, we distinguish between stems and affixes. When reading a text in a foreign language, the reader would match the words she or he is reading with cognate words in his or her native language on the basis of the stems. For example, when a native speaker of English reads the Dutch word handen, she or he might be able to match it with the English word hands on the basis of the joint stem hand, although the affixes in the two languages differ.

In contrast, when a native speaker of English reads the Dutch word kevers, she or he would not be expected to match it with the English word beetles, despite the fact that both the Dutch and the English word—having the same meaning—have the same plural affix -s. When both the stem and the affix are different—for example, English ducks versus Dutch eenden—it is less likely that an English reader would relate the Dutch plural suffix -en to the English plural suffix -s. Considering this, we decided to calculate orthographic distances on the basis of word pairs whose members are cognates.

Focusing on cognate pairs only further reduces the number of analyzed word pairs per language pair. For example, the fourth row in table 7 contains the Danish word børn and the Dutch word kinderen. Since the Danish and Dutch words are not cognates, this word pair was not included in the analyses. In contrast, Dutch kinderen and German Kinder are cognates, and the orthographic distances were measured for this word pair.

Table 9 shows the number of cognate pairs per language pair. The numbers are much smaller than the numbers in table 8 and vary from 203 (English versus Swedish) to 435 (Danish versus Swedish).

Table 9.

The number of cognate word pairs for each language pair.

Despite the smaller numbers and apparently large differences, consistent results can still be obtained on the basis of this data set. In section 5.1, we show that the number of cognate pairs for each of the language pairs is sufficiently large to yield consistent results for all of the measurements reported in this paper.

Table 10 shows the percentage of word pairs whose members are cognates. The distance between any two languages obtained on the basis of cognate word pairs is divided by the number of cognate pairs.

Table 10.

The percentage of word pairs, whose members are cognates.

The percentages reflect lexical similarity. We conclude that Danish and Swedish are lexically most similar as they share 70.5% of the words in our corpus. However, lexical similarity is not the topic of this paper.

4.1 Levenshtein Distance

Orthographic distances between two words are measured with the aid of the Levenshtein distance metric (Levenshtein Reference Vladimir1966). Recall from section 2.1 that the Levenshtein distance between two strings is calculated as the “cost” of the total set of insertions, deletions, and substitutions needed to transform one string into another (Kruskal Reference Kruskal, Sankoff and Kruskal1999). In our case, the strings to be compared are orthographic transcriptions of words. In table 11 (see below), we illustrate this algorithm by transforming the English word interest into the Swedish word intresse. This represents a native speaker of Swedish reading English or a native speaker of English reading Swedish, and trying to map the target word to its cognate in his or her native language. The two words are very similar, but in the fourth slot an e is deleted, in the eight slot a t is replaced by an s, and in the ninth slot an e is inserted. Therefore, the Levenshtein distance equals three operations. The alignment has nine slots. We calculate the normalized Levenshtein distance as 3/9×100 = 33%. Many different sequences of operations transform interest to intresse, but the Levenshtein distance always gives the cost of the cheapest mapping.

Table 11.

Levenshtein distance: interest versus intresse.

For each character, we distinguish between a base and a diacritic. For example, the base of é is e, and the diacritic is the acute accent. Two characters may differ in the base and/or in their diacritics. We weigh differences in the base as 1. For example, a versus e, p versus b. If two characters have the same base but different diacritics, we weigh this as 0.3, for example, e versus é and è versus é. We admit that this method is not based on empirical measurements and may appear arbitrary, but our choice is motivated by the idea that diacritical differences should weigh significantly less than base differences, since differences in the base usually confuse the reader to a much greater extent than diacritical differences. When the bases are different, the weight is 1, regardless of whether there are diacritical differences, since differences or similarities between diacritics are meaningless when the corresponding bases are different. Insertions and deletions are weighed as 1.

In German, the first letter of every noun is capitalized. If the Levenshtein distance counts lower and upper case letters as different characters, German would become disproportionally distant from the other Germanic languages, since readers do not really distinguish between lower case and upper case letters. Therefore, we do not distinguish between lower case and upper case letters. For example, the distance between the English word problem and the German word Problem is zero. We also ensure that the minimum cost is based on an alignment in which a vowel matches with a vowel, and a consonant matches with a consonant.

4.2 Aggregated Stem and Affix Distance

For each language pair, we calculated the aggregated orthographic distance by calculating the average of the normalized Levenshtein distances within the word pairs considered for that language pair. A small sample is given in table 12, which shows the comparison of Dutch and German on the basis of a set of five words. The fourth column shows the Leven-shtein distances. When the numbers in the fourth column are divided by the numbers of slots in the alignment (fifth column), the normalized Levenshtein distance is obtained (sixth column).

Table 12.

The aggregated distance is the average distance (0.27 or 27%).

We specifically focus on stem and affix distances. When calculating the stem distance, we considered the stem of the words only. This is shown in table 13 (see below), where the aggregated stem distance is 28% (the stems appear in bold).

Affix distances are found by considering affixes only. This is illustrated in table 14 (see below). The words in the fifth word pair each contain two affixes: a prefix ge- and a suffix -d (Dutch) or -t (German). The affixes were concatenated to ged (Dutch) and get (German), and the Levenshtein distance was computed between the two concatenations. Sometimes words do not have affixes. For example, the plural form of the English word sheep is sheep. When this word is transformed into, for example, Dutch schapen, which has plural suffix -en, the Levenshtein distance equals two (two insertions).

Table 13.

The aggregated stem distance is the average stem distance (0.28 or 28%).

Table 14.

The aggregated affix distance is the average affix distance (0.37 or 37%).

These examples show how orthographic language variation in whole words, stems, and affixes can easily be quantified by using Levenshtein distance.

6.1 First Hypothesis

If the Germanic languages emerged from one common root, that is, Proto-Germanic, it is reasonable to assume that diversification of the pronunciation of the Proto-Germanic stems likely does not run (completely) parallel to diversification of the pronunciation of the Proto-Germanic affixes (see discussion in section 1). We then hypothesized that orthographic stem variation among languages does not correlate with orthographic variation in inflectional affixes. In order to test this hypothesis we correlated the orthographic stem distances with the orthographic affix distances. Figure 3 shows a scatter plot in which orthographic affix distances are drawn against the orthographic stem distances. The correlation between the two measures is r=0.15.

For finding the significance of this correlation coefficient we used the Mantel test (Mantel Reference Nathan1967), a widely used method to account for distance correlations. Classical tests rely on the assumption that the correlated objects are independent. However, values in distance matrices are usually correlated in some way, and are not independent (Bonnet & Van de Peer Reference Bonnet and Yves Van de2002). By using the Mantel test, we found that p=0.36. Therefore our hypothesis is confirmed: Orthographic stem variation among languages does not correlate with orthographic variation in inflectional affixes.

Figure 3

The orthographic affix distances and the orthographic stem distances.

In figure 3, the two measurements do not correlate. In the scatter plot, the Danish/Swedish pair is found in the lower left corner. It is found distant from the other points. If this pair is excluded, the correlation between stem and affix distances becomes r= −0.24, which is still not significant (p=0.29).

We also calculated correlations per language pair. The number of cognate pairs in each language pair is given in table 8. For each cognate pair, both the orthographic stem distance and the orthographic affix distance were measured. The orthographic stem distances of a language pair are correlated with the corresponding orthographic affix distances. For example, for the Danish/Dutch language pair we have 282 cognate pairs. Therefore, the correlation is calculated between 282 stem distances and 282 affix distances, which is r=0.17. The correlations of each of the language pairs are shown in table 17: ** means p<0.01, *** means p<0.001, **** means p<0.0001. The significant correlations have a small effect size.

Table 17.

Correlations between orthographic stem distances and orthographic affix distances per language pair.

This table shows either nonsignificant correlations or the significant correlations that have a small effect size according to the guidelines of Cohen Reference Cohen1988. These results confirm our hypothesis that stem and affix distance are not correlated.

6.2 Second Hypothesis

In section 1, we assumed that the diversification of affixes in Proto-Germanic proceeded slower than the diversification of stems. Assuming that all present-day Germanic languages originate from Proto-Germanic, we expect the affix distances between those languages to be smaller than stem distances. In this section, we test the hypothesis that orthographic stem distances among languages are larger than orthographic affix distances.

The stem and affix distances are shown in figure 4. We compared stem and affix distances using a paired-samples t-test and found that stem distances are significantly larger than affix distances (t=5.10, df=9, p<0.001, r=0.86; large effect size).

Figure 4

Stem distances (dark) and affix distances (light) per language pair.

The differences between orthographic stem distances and affix distances are shown in figure 5 and table 18. The smallest distance is found for the Danish/Swedish language pair.

Figure 5

Differences between stem distances and affix distances, ranging from 1.9% (Danish/Swedish) to 32.9% (Dutch/English).

Applying a paired-samples t-test, we tested, for each language pair, whether the stem distances are larger than the affix distances. The p-values are given in table 18. For all language pairs, stem distances are almost always significantly larger than the affix distances, which further supports the hypothesis that stem distances are larger than affix distances.

Table 18.

Differences between stem distances and affix distances

(* means p<0.05, **** means p<0.0001).

For the Danish/Swedish language pair we do not find stem distances being significantly larger than affix distances. Figure 5 shows that the difference between the aggregated stem distance and the aggregated affix distance is smallest for the Danish/Swedish language pair, and in figure 4 it can be seen that the stem distance within this language pair is smallest. This raises the question whether there is a correlation between the differences between stem and affix distances (as shown in table 18/figure 5) and stem distances (as shown in table 15/figure 4). Do smaller stem distances within a language pair correspond with smaller differences between stem and affix distances? Indeed, we found a significant correlation (r=0.74, p<0.05).

We also considered the correlation between stem distances and the affix/stem distance ratio. To obtain the affix/stem distance ratio we divide an affix distance by its corresponding stem distance. The ratios are shown in figure 6. We found a significant correlation (r=0.66, p<0.05). This means that affix distances become proportionally larger when stem distances become smaller.

Figure 6

Affix/stem distance ratios, ranging from 0.92 (Danish/Swedish) to 0.39 (Dutch/English).

To summarize, we found that stem distances are significantly larger than affix distances, and that smaller stem distances within a language pair correspond with smaller differences between stem and affix distances and with proportionally larger affix distances.