1 Introduction
Part of speech (POS) tagging for English is often considered a solved problem. There are well-established approaches such as the Markov model trigram taggers (Brants Reference Brants2000), maximum entropy taggers (Ratnaparkhi Reference Ratnaparkhi1996), or Support Vector Machine (SVM) based taggers (Giménez and Màrquez Reference Giménez and Màrquez2004), and accuracy reaches approximately 97%. However, morphologically rich languages, such as Modern Arabic, still present unsolved challenges for POS tagging: Rich morphology generally results in a high number of word forms per lemma, which means that fewer word forms can be observed in a training corpus of finite size. For example, the Arabic lemma 
(ktb in Buckwalter transliteration, Eng. to write) can be found in over 400 different forms. As a verb, ktb can be realized as ktb, ktbt, ktbn, ktbnA, ktbA, and ktbwA in the past tense alone, and each of these can be preceded by proclitics and followed by pronouns, which are parts of the words. The same word can also be used as a plural noun, which can take similar prefixes and suffixes.
For morphologically rich languages, the POS tags often contain morphological information in addition to information about word classes because of the morphological complexity of words. For example, the word
(wsyktbwnhA, Eng. and they will write it) is assigned the following POS tag in the Penn Arabic Treebank (ATB) (Bies and Maamouri Reference Bies and Maamouri2003):
The POS analysis states that the word consists of a conjunction, a future particle, a verb prefix, an imperfective verb, a subject suffix, and an object suffix. Both factors, the rich morphology and the large tagset, contribute to data sparseness.
Previous approaches (cf. Diab, Hacioglu and Jurafsky 2004 for example) to POS tagging for Arabic have concentrated on a narrow definition of the task, i.e. they assumed gold standard tokenization, which cannot be assumed for naturally occurring Arabic. In this paper, we present the first approach to POS tagging of Arabic, as it occurs naturally in written texts, i.e. without assuming vocalization or tokenization. We assume that the task of POS tagging is assigning POS labels to full words. We answer the following questions:
(1) Is POS tagging of full words feasible when using the full tagset of the ATB? That is to say, for the word shown above, the full word would be presented to the POS tagger, and the task would be to assign the complex POS tag. A positive answer to this question also implies that POS tagging for Arabic can be performed successfully without prior morphological analysis.
(2) Can we improve the POS tagging results by segmenting the words first? In this case, the word above would be automatically split into its segments, and the individual segments would be tagged by the POS tagger. In a post-processing step, the segment tags would be assembled into a complex tag.
(3) Does vocalization, i.e. the restoration of word-internal diacritics, improve POS tagging results? Vocalization can be performed very reliably with word error rates (WER) of approximately 6.72%, and vocalized words are less ambiguous than unvocalized ones.
The decision between tagging full words or segments has far reaching consequences: Tagging full words, on the one hand, means that the tagset extracted from the treebank consists of 993 different complex POS tags. In theory, the size of the tagset is even larger, since some of the possible segment combinations do not occur in the ATB. Also, a large number of the POS tags that can be observed in the treebank occur only once in the data set, contributing to data sparseness. However, under this tagging approach, a high number of words are unambiguous. Tagging segments, on the other hand, results in a medium-sized tagset (139 segment tags), but the segments themselves are more ambiguous than full words. For example, the word 
(wbHsnAthm, Eng. and with their virtues) is not ambiguous as a full word. But after segmentation into w+b+Hsn+At+hm, we have 3 × 2 × 4 × 2 × 6 = 288 possible composite tags for the word. Thus, it is not immediately obvious which approach will perform better.
The remainder of the paper is structured as follows: Section 2 gives an overview of the most important characteristics of the Arabic language with regard to POS tagging, and it describes the main data source, the ATB. In Section 3, we discuss related work on POS tagging for Arabic, but also on segmentation and vocalization. Section 4 provides an overview of the data set and the experimental setup that we use, and Section 5 presents the results for a baseline and the full word POS tagger. In Section 6 we discuss the integration of segmentation and vocalization into POS tagging, and in Section 7 we present the results, including an error analysis. In Section 8 we investigate whether a reduction of the tagset size to the Extra-Reduced Tagset (ERT) by Habash and Rambow (Reference Habash and Rambow2005) results in higher accuracy. We also investigate the best conversion strategy. Finally, In Section 9 we conclude.
2 The Arabic language
Arabic is a Semitic language with a rich morphology, combining concatenative with templatic morphology. An Arabic word carries not only inflections but also clitics, such as pronouns, conjunctions, and prepositions. The word 
(wbHsnAthm, Eng. and with their virtues), for example, contains two proclitics w and b, the noun Hsn, the plural marker At, and the enclitic hm. Since there are many potential forms per word, segmentation or tokenization is required for many applications. We define segmentation as the task of determining inflectional affixes and clitics. We will use the term tokenization, in contrast, to describe the task of delineating parts of the word that have a syntactic function, such as conjunctions, prepositions, or pronouns. Thus, the word above would be segmented as w+b+Hsn+At+hm (the + sign signifying segment boundaries) and tokenized into w b HsnAt hm.
Arabic is written in an abjad, in which the long vowels are written while the short vowels can be expressed as diacritics. In most texts, however, the short vowels are not written. This leads to one written form having several pronunciations with each pronunciation carrying its own meaning(s). The word 
(mskn) is an example of a highly ambiguous word. Its possible pronunciations include maskan (Eng. home), musakkin (Eng. analgesic), masakn (Eng. theyfem. have held), or musikn (Eng. theyfem. have been held). As this example shows, restoring the short vowels, which is called vocalization, often disambiguates the POS of the word form, with musakkin being a noun and musikn a verb form. However, the short vowels also represent case endings, mood endings, and nunation (the syntactic indefiniteness marker for a noun or adjective). Since these are syntactic rather than lexical processes, this type of diacritics is often excluded from automatic vocalization.
The major data source for any approach to POS tagging and related topics is the ATB (Bies and Maamouri Reference Bies and Maamouri2003). The textual basis of the Arabic Teebank is a collection of newswire stories from three sources: Agence France Press, the Alhayat newspaper, and Ummah News. Apart from the syntactic annotation, which will not be used for the purposes of this paper, the treebank contains information about POS tagging, segmentation, and vocalization. The treebank is POS tagged on the segment level, assigning one out of 139 segment tags to each segment. If the words are taken as basic units of the language, however, each word is assigned a complex tag, with each part corresponding to a segment label. Thus, the word from Section 1, 
(wsyktbwnhA, Eng. and they will write it), is assigned the following complex POS tag in the ATB: [CONJ+FUT_PART+IV3MP+IV+IVSUFF_SUBJ:3MP_MOOD:I+PRON_3FS]. This leads to a highly complex tagset with thousands of possible values, out of which 993 are attested in the ATB. There is also a Reduced Tagset (RTS), which contains twenty-four major POS categories, similar to the English Penn Treebank tagset. The treebank distribution contains a script that allows the user to convert the complex tagset to the RTS.
The tightly controlled text source of the ATB leads to a rather low percentage of unknown words: Only 8.56% of the words are unknown given a five-fold cross-validation setup. This also has an influence on POS tagging accuracy, since tagging unknown words is more challenging than tagging known words, for which the ambiguity set (i.e. the set of all possible POS tags) is known. However, the rich morphology of Arabic words in combination with the rich information in the POS tagset leads to a low average ambiguity for words and segments: With regard to POS tags, 84.89% of the segment types and 92.97% of the complex word types are unambiguous.
3 Related work
In this section, we review related work on segmentation, tokenization, vocalization, and POS tagging for Arabic, but also work on Hebrew since the two languages are closely related. Note that most approaches for Arabic use different data sets or different versions of the ATB so that a comparison of the results is difficult. There is also work on parsing Arabic and Hebrew (cf. Chiang et al. Reference Chiang, Diab, Habash, Rambow and Sharif2006; Tsarfaty Reference Tsarfaty2006; Tsarfaty and Goldberg Reference Tsarfaty and Goldberg2008; Green and Manning Reference Green and Manning2010 and Marton, Habash and Rambow Reference Marton, Habash and Rambow2010), which we will not discuss here since we assume that parsing is only one of the many applications for which POS tagging can serve as pre-processing. The chapter is structured as follows: We first describe approaches to segmentation and tokenization (Section 3.1), then to vocalization (Section 3.2) and to morphology and POS tagging (Section 3.3), and finally approaches that combine several tasks in an integrated architecture (Section 3.4).
3.1 Segmentation and tokenization
In one of the first approaches to Arabic word segmentation, Darwish (Reference Darwish2002) used patterns of possible stems, prefixes, and suffixes in order to detect roots. These lists of affixes were derived automatically from word-root pairs. The system selected the most probable segmentation on the basis of a simple probabilistic model of different segments. Darwish assumed that a word would carry at most one suffix and one prefix, i.e. he grouped all prefixes and suffixes. For example, the word 
(wbHsnAthm, Eng. and with their virtues) was segmented as wb+Hsn+Athm, the + signifying proposed segment boundaries. Thus, the approach combined the prefixes w and b into one. Darwish reported an accuracy of 92.7% on a 9,606 word corpus with the system trained on 270,000 words from a corpus of Agence France Press (AFP) newswire text.
Lee et al. (Reference Lee, Papineni, Roukos, Emam and Hassan2003) used an affix list that provided the basis for possible segmentations, in combination with a trigram model for disambiguation between possible segments. To combat the high number of unknown stems resulting from a small training corpus, they used a bootstrapping approach to recover more stems. Lee et al. reported an accuracy of 97% on an early version of the ATB.
3.2 Vocalization
The first approaches to vocalizing Arabic defined the problem as word-based, in which the task is to determine for each word the complete vocalized form. Then the focus shifted to character-based models, where the models decide for each character whether it is assigned a diacritic and which one.
There are two word-based approaches, by Gal (Reference Gal2002) and by Kirchhoff et al. (Reference Kirchhoff, Bilmes, Henderson, Schwartz, Noamany, Schone, Ji, Das, Egan, He, Vergyri, Liu and Duta2002). Gal used a hidden Markov model (HMM) for vocalizing Arabic and Hebrew, in which the hidden states were vocalized words and the observations were unvocalized. Gal used the Bible and the Qur'an as data sources. He achieved a WER of 14% on Arabic and 19% on Hebrew.
Kirchhoff et al. (Reference Kirchhoff, Bilmes, Henderson, Schwartz, Noamany, Schone, Ji, Das, Egan, He, Vergyri, Liu and Duta2002) designed a simple maximum likelihood module for use in a speech recognition system on the basis of the LDC CallHome corpus of Egyptian colloquial Arabic (Canavan, Zipperlen and Graff Reference Canavan, Zipperlen and Graff1997). For unknown words, the most similar unvocalized word was used to determine the vocalization of the unknown word. Their approach reached a WER of 16.5% on conversational Arabic.
Nelken and Shieber (Reference Nelken and Shieber2005) proposed a hybrid model. They used four weighted finite-state transducers. For known words, morphological units based on the Buckwalter analyzer (Buckwalter Reference Buckwalter2002) were used for retrieving the vocalization, while unknown words were diacritized on the basis of a 4-gram character model. Nelken and Shieber reached a WER of 23.61% when case restoration was included and of 7.33% on word-internal vocalization only.
Zitouni, Sorensen and Sarikaya (Reference Zitouni, Sorensen and Sarikaya2006) were the first to treat vocalization as the classification of indivdual characters. They used maximum entropy modeling for classification; their features included lexical features, segmentation features, and POS features, as well as the previously assigned two diacritics. Zitouni et al. reached a WER of 17.3% on vocalization, including case endings, and 7.2% when case endings were not restored.
Habash and Rambow (Reference Habash and Rambow2007) performed full vocalization, including case endings and nunation. They used the Buckwalter analyzer to obtain all possible morphological analyses, including all diacritics. Then they trained individual classifiers to disambiguate between these analyses. Residual ambiguity was resolved via an n-gram language model. Habash and Rambow (Reference Habash and Rambow2007) reached a WER of 5.5% using Zitouni et al.'s data set. When they included case endings, the WER deteriorated to 14.9%.
Habash et al. (Reference Habash, Gabbard, Rambow, Kulick and Marcus2007) presented a study of case diacritics, which are often ignored in vocalization since those diacritics require syntactic information for successful restoration. Habash et al. compared handwritten rules and a machine learning approach based on a maximum entropy model. They used gold standard syntactic trees as information. Their results show that the machine learning approach reached higher accuracies. However, an in-depth error analysis led the authors to the conclusion that many of the errors of the handwritten rules are due to errors in the gold standard annotation, and that if those errors were accounted for, the rule-based approach outperformed the machine learning one.
Shaalan, Abo Bakr and Ziedan (Reference Shaalan, Abo Bakr and Ziedan2009) investigated a lexicon-based approach, an approach using word bigram statistics, and an SVM classifier using features from automatic segmentation, POS tagging, and chunk parsing. Shaalan et al. showed that the best results for vocalization were reached by combining all three approaches. They reached a WER of 12.16% on vocalization for Arabic, including case restoration.
In Kübler and Mohamed (Reference Kübler and Mohamed2008), we investigated vocalization using memory-based learning. We focused on determining the optimal context in a character-based approach. Our results show that the lowest WER was reached by using only word internal information. Adding context words increased the WER. In Mohamed and Kübler (Reference Mohamed and Kübler2009), we extended these experiments and showed that the inclusion of punctuation and other words that are never vocalized in the evaluation artificially increased the accuracy and potentially concealed the best method. We also showed that it is important to restore the hamza and alif in real world text, where they are often used interchangeably, in order to avoid discrepancies between the training data from the carefully edited ATB and the test data.
Haertel, McClanahan and Ringger (2010) used a hybrid Markov model to vocalize Syriac, Hebrew, and Arabic. The hybrid model used word sequences for known words, but backed off to consonant sequences for unknown words. The system had access to the prefixes and suffixes of previously vocalized words, the words' diacritics, n-gram, and positional information. Using Zitouni et al.'s (Reference Zitouni, Sorensen and Sarikaya2006) data set, Haertel et al. (Reference Haertel, McClanahan and Ringger2010) found that overall, a consonant-only model reached a better WER (15.02%) in comparison to the hybrid model (17.87%). For unknown words, however, the hybrid model reached better results with a WER of 47.85%.
3.3 Morphology and POS tagging
In most approaches to POS tagging, the systems are based on a morphological analysis. For this reason, we include early work on the morphological analysis of Arabic. Beesley (Reference Beesley1996) presented a reimplementation of a morphological analyzer for Arabic on the basis of earlier work (e.g. Beesley Reference Beesley1990). While the earlier version used the KIMMO-style two-level morphology, the reimplementation used the Xerox finite-state tools (Karttunen and Beesley Reference Karttunen and Beesley1992), which resulted in a more powerful, efficient, and robust system. Buckwalter (Reference Buckwalter2002) later documented a version of the morphological analyzer, which is considered the current state of the art. Habash, Rambow and Kiraz (Reference Habash, Rambow and Kiraz2005) and Habash and Rambow (Reference Habash and Rambow2006) presented a morphological analyzer and generator for Arabic dialects, which used finite-state technology in combination with a multitape representation.
Adler and Elhadad (Reference Adler and Elhadad2006) described an approach to POS tagging for Hebrew, which used an extension of an HMM. The input for the HMM consisted of different analyses of a morphological analyzer (Segal Reference Segal2000).Footnote 1 The HMM was capable of analyzing different segmentations of a word and selecting the best combination of segments and POS tags. Based on a Hebrew corpus of six million words, the POS tagger reached an accuracy of 92.32%. Bar-Haim, Sima'an and Winter (Reference Bar-Haim, Sima'an and Winter2008) investigated different segmentation schemes for POS tagging Hebrew on the basis of the same morphological analyzer. Their experiments show that a segmentation-based approach reached higher accuracies than a full-word approach. Their best approach, in which they tokenized the text and bootstrapped the algorithm, reached a per word accuracy of 89.59% on the Hebrew Treebank (Sima'an et al. Reference Sima'an, Itai, Winter, Altmann and Nativ2001).
Van den Bosch, Marsi and Soudi (2007) used memory-based learning for both morphological analysis and POS tagging of Arabic. They were the first to use a full tagset of the ATB in combination with tokens from ATB part 1 version 2 (166,000 words). Van den Bosch et al. reported a total accuracy of 91.5%, with 93.3% accuracy on known words and 66.4% accuracy on unknown words.
Diab (Reference Diab2007) introduced a new tagset, the Extended Reduced Tagset (ERTS), to be used with Arabic base phrase chunking. The ERTS comprises seventy-five POS tags, it is derived from the full tagset of the ATB, and enriches the RTS with definiteness, gender, and number information. The ERTS explicitly marks gender, number, and definiteness on nominals, namely nouns, proper nouns, adjectives, and pronouns, but it does not mark person or number for verbs.
In Mohamed and Kübler (Reference Mohamed and Kübler2010a, Reference Mohamed and Kübler2010b), we presented a first version of the experiments given in Sections 5 and 6. We showed that while POS tagging for Arabic based on gold standard segmentation resulted in the highest word-based accuracy, whole word tagging reached a higher accuracy than an approach using a state-of-the-art automatic segmenter.
3.4 Integrated approaches
All of the approaches described in this section, as well as the approach by van den Bosch et al. (Reference van den Bosch, Marsi, Soudi, Soudi, van den Bosch and Neumann2007) described above, are based on the tokenization present in the ATB, not on full words, thus assuming gold standard pre-processing.
Diab et al. (Reference Diab, Hacioglu and Jurafsky2004) presented a system to perform tokenization, POS tagging, and base phrase chunking for Arabic, using SVM classifiers. Diab et al. treated tokenization as an Inside-Outside-Beginning (IOB) tagging problem, where each character of the word is classified as beginning (B), inside (I), or outside (O) the prefix, the stem, or the suffix. They used a context of +/− 5 characters and the classifier's previous decisions as features on the ATB in its state of 2004 (140,000 words) and reported an accuracy of 99.77% and an F-score of 99.12%. For POS tagging, their feature set included the focus word in a window of +/−2 words, their alpha-numeric type, and previous tagging decisions for context words. Diab et al. used the English-like RTS, which contains twenty-four/twenty-five tags; they reported a POS tagging accuracy of 95.5% on the ATB. Diab (Reference Diab2009) described AMIRA, the second generation of this toolset. This version is flexible and can produce either the RTS (twenty-four tags) or the ERTS (seventy-five tags).
Habash and Rambow (Reference Habash and Rambow2005) presented an approach that performs tokenization, morphological disambiguation, and POS tagging ‘in one fell swoop’. They used a morphological analyzer, BAMA (Buckwalter Reference Buckwalter2002), to produce all the possible tokenizations of a word. Then they employed different SVM classifiers to select the best solutions among those provided in the morphological analysis. The classifiers were based on ten features, including the POS tags, gender, person, number, voice, and aspect, along with binary features regarding possible affixes. In a final step, a classifier trained on data from the ATB was used to select the most probable solutions. For tokenization, Habash and Rambow (Reference Habash and Rambow2005) reported a token accuracy of 99.6% and a word accuracy of 99.3% based on their tagset, which encompasses fifteen tags. They also showed that the use of the morphological analyzer helped to achieve better POS tagging accuracy and reported an accuracy of 97.6% on their fifteen-tag tagset.
Shacham and Wintner (Reference Shacham and Wintner2007) performed morphological disambiguation, including POS tagging, for Hebrew. They showed that an approach that uses a hierarchical setup of classifiers rather than a naive combination, such as the one used by Habash and Rambow (Reference Habash and Rambow2005), resulted in significant gains in terms of accuracy.
Roth et al. (Reference Roth, Rambow, Habash, Diab and Rudin2008) and Habash, Rambow and Roth (Reference Habash, Rambow and Roth2009) presented MADA and TOKAN. MADA is a system that performs a full morphological analysis, lemmatization, diacritization, and POS tagging on the basis of the BAMA lexicon. TOKAN uses the information produced by MADA to perform tokenization. Roth et al. (Reference Roth, Rambow, Habash, Diab and Rudin2008) showed that using lexemic features improved accuracy, especially for diacritization and lemmatization. Their optimized POS tagging reached an accuracy of 96.4%.
4 Data, experimental setup, and evaluation
4.1 Data
For all the experiments reported here, we use the ATB (Bies and Maamouri Reference Bies and Maamouri2003), and more specifically a combination of two sections of the ATB (P1V3 and P3V1). Our chosen data set contains approximately 500,000 words. In order to be a representative of real-world Arabic, we use the non-vocalized version of the treebank. Since previous approaches used different data sets, our results are not directly comparable. However, in order to compare our results to previous results by Habash and Rambow (Reference Habash and Rambow2005), in Section 8, we describe an experiment, in which we reduce the full tagset from our approach to their fifteen-tag tagset after POS tagging using the full tagset.
For all our experiments on segmentation and POS tagging, we modifiy the ATB representation of words in order to obtain the text as it would occur in newscasts. For this reason, all conjunctions, prepositions, pronouns, and any elements that constitute parts of the word (with the exception of punctuation) are re-attached to the word. The word 
(ltxbrh, Eng. in order to tell him), for example, is represented as three entries in the ATB, l, txbr, and h, but is treated as one single unit in our experiment. Another modification concerns the null element in Arabic verbs. Since Arabic is a pro-drop language, the ATB annotation includes a null element in place of the omitted subject plus the POS tag that it would receive. Since this information is not available in naturally occurring text, we delete the null element and its tag. For example, 
({i$otara¥+(null), Eng. he bought) and its tag PV+PVSUFF_SUBJ:3MS would occur as {i$otara¥ with the tag PV in our representation (but we additionally remove vocalization).
These modifications allow us to work on Arabic text as it would occur in newspapers or newscasts, without assuming knowledge about tokenization. However, since we use the words extracted from the treebank annotations rather than the original text, the text is cleaner than text as it occurs in such newspapers or newscasts because spelling errors, nonstandard spelling, as well as nonstandard uses of diacritics have been normalized in the treebank.
4.2 Experimental setup
For all experiments, the data set is used in a five-fold cross-validation setting, where the data is divided into five sections, and each fold serves once as test set, while the four others are used for training. The results reported below are averaged over the five folds.
All the experiments reported in this paper are based on memory-based learning. Table 1 gives an overview of different methods and settings for different tasks. Memory-based learning is a lazy learning paradigm that does not abstract over the training data. It has been shown to possess a learning bias suitable for a wide range of problems in Natural Language Processing (cf. Daelemans, van den Bosch and Zavrel Reference Daelemans, van den Bosch and Zavrel1999 and Daelemans and van den Bosch Reference Daelemans and van den Bosch2005). With regard to the experiments discussed here, memory-based learning has two important characteristics: (1) It uses an inherent smoothing mechanism for feature values (Zavrel and Daelemans Reference Zavrel and Daelemans1997), and (2) it is to a high degree insensitive to a fragmented class space with large label sets. As a consequence, it is well suited for POS tagging with a large tagset. In memory-based learning, training consists of storing all instances in an efficient way. During classification, the k nearest neighbors to a new instance are retrieved from the training data, based on their similarity to the new instance. The class assigned to the majority of these neighbors is then assigned to the new instance. For segmentation and vocalization, we use TiMBL (Daelemans and van den Bosch Reference Daelemans and van den Bosch2005; Daelemans et al. Reference Daelemans, Zavrel, van der Sloot and van den Bosch2007), an implementation of memory-based learning that differs from the original definition of the IB1 algorithm (Aha, Kibler and Albert Reference Aha, Kibler and Albert1991) by assigning weights to features. For POS tagging, we use MBT, a memory-based tagger (Daelemans et al. Reference Daelemans, Zavrel, Berck and Gillis1996). MBT uses TiMBL in the background; it offers the possibility to use words from both sides of the focus word as well as previous tagging decisions and ambitags as features. For words to the left of the focus word, previous decisons by the tagger can be used as features. For the words to the right of the focus word, the tagger has not made a decision yet; here, ambitags are used. An ambitag is a combination of all POS tags of the ambiguity class of the word. For example, the word 
(Drb, Eng. to hit, hitting, type) is ambiguous between noun and verb and would thus be assigned a combination of those two labels as its ambitag.
Different system and parameter settings for segmentation, vocalization, and POS tagging
For all experiments, parameter settings are determined in a non-exhaustive manual search using one of the folds. For the POS tagging experiments, the best results use the Modified Value Difference Metric as a distance metric and the number of nearest neighbors (or rather distances in TiMBL) k = 25. (For a description of the algorithms and parameters, cf. Daelemans et al. Reference Daelemans, Zavrel, van der Sloot and van den Bosch2007.) MBT uses different models for known and unknown words. For known words, the features include the two context words to the left along with their disambiguated POS tags (previously assigned by the tagger), the focus word itself, and one word to the right along with its ambitag. For unknown words, the features include the first five and the last three characters of the word, the presence of a hyphen in the focus word, the focus word itself, one context word to the left, its tag, one word to the right, and its ambitag. The IB1 algorithm produces the best results with known words, while the IGTree yields better results for unknown words.
Based on results from previous work (Kübler and Mohamed Reference Kübler and Mohamed2008; Mohamed and Kübler Reference Mohamed and Kübler2009), we choose to regard segmentation and vocalization as per-character classification tasks, i.e. the classifier decides for each character in a word whether it constitutes the end of a segment or whether it has a diacritic. For both segmentation and vocalization, the best results are obtained with the IB1 algorithm, with similarity computed as weighted overlap, relevance weights computed with gain ratio, and the number of k nearest neighbors equal to 1.
4.3 Evaluation
For all experiments, we report accuracy, but in contrast to previous work, we calculate accuracy in terms of words, not on the basis of tokens. For the POS tagging approach based on segmentation, this means that the segments are re-attached to whole words before evaluation.
To give a concrete evalutation example, for the phrase 
(jrydp AlHyAp AlsEwdyp, Eng. the Saudi newspaper Al-Hayat), the words plus annotations are shown in Table 2. Assume that the POS tagger assigns the tag DET+NOUN+NSUFF_FEM_SG to the word AlHyAp, instead of the simple NOUN_PROP. Then the tagging accuracy for this phrase is two out of three words, or 66%. Evaluation for segmentation and vocalization is also word-based.
A POS evaluation example
5 Baseline and full-word POS tagging
As a baseline for POS tagging, we use an approach that, for known words, assigns the most frequent POS tag from their ambiguity class. If the word does not occur in the training set, the most common tag, NOUN, is assigned. We compare this baseline with whole word tagging: In this experiment, words are treated as single units, and they are assigned complex POS tags. For example, the word 
(wbHsnAthm, Eng. and with their virtues) is used as one unit, and the correct POS tag is the complex CONJ+PREP+NOUN+NSUFF_FEM_PL+POSS_PRON_3MP. This tagging scheme is novel since previous approaches used the pre-tokenized text from the treebank. There are potential problems with this tagging scheme: Since Arabic is a morphologically rich language, many word forms are unknown. In our data set, the percentage of unknown words across folds is 8.56%. In addition, the number of complex tags is very large, there are 993 different tags in the treebank, and there are more theoretically possible tags consisting of segment combinations that are possible in the language but do not occur in the treebank. As a consequence of the large tagset and the limited training set, many of the complex POS tags occur only once in the training set: 228 out of the 993 (22.91%) complex POS tags. Both factors lead us to expect data sparseness. On the other hand, the ambiguity rate for full words is low, a word has on average 1.1 tags, and the maximum size of the ambiguity class is 7.
The results of both experiments are shown in Table 3. The baseline reaches a rather high accuracy of 77.02%, which is surprising given the morphological richness of Arabic and the large tagset. However, a look at the baseline performance for unknown words, i.e. words in the test set that do not occur in the training set, shows that only 3.67% are tagged correctly. This indicates that the good performance of the baseline is due to the high percentage of known, unambiguous words, which is corroborated by the high accuracy of 83.88% on known words. The extremely low accuracy on unknown words can be explained in terms of tag complexity and sparseness: While the tag NOUN is the most common tag in the training set, there are other nominal tags, such as DET+NOUN and NOUN+NSUFF_FEM_SG.
Results of the baseline tagger and the whole word tagger. We report the percentage of correctly tagged whole words
The results of the whole word tagger are even more surprising. Despite the morphological complexity and the size of the tagset, this approach reaches an accuracy of 94.74%, a highly significant improvement over the baseline.Footnote 2 The accuracy for known words reaches 96.62%. The accuracy for unknown words, however, is 20% points lower, and reaches 74.64%. The high accuracy on known words can be explained by the fact that 92.97% of all known words are unambiguous.
The low accuracy on unknown words leads to the question whether the results can be improved by alleviating data sparseness by first segmenting words and then POS tagging the segments rather than words. The number of segments is considerably lower than the number of word types in the training set: 19,019 segments versus 56,137 word types. Thus, data sparseness is less severe. Another question that needs to be answered is whether automatic vocalization can reduce ambiguity in the data set even further and thus provide a basis for higher POS tagging accuracy. These questions are addressed in the following section.
6 Integrating segmentation and vocalization into POS tagging
In this section we include segmentation and vocalization into the POS tagging process. For this reason, we first describe the segmentation and vocalization modules and evaluate them before we present the POS tagging experiments that use those modules as pre-processing.
6.1 Segmentation
Segmentation is defined as a classification task in which each character of a word is assigned information whether the character is the final character of a segment. Possible classes are ‘+’ for segment ends and ‘−’ for all other characters. In order to find the best feature set for the task, we perform three experiments:
(1) Basic. The basic experiment is the simplest experiment as it requires no information beyond the word itself. For each character of a word, we create a vector. Each vector consists of the focus character, the five preceding characters, and the five following characters. Since all the previous studies on segmentation used only the word-internal context, we expect competitive results from this experiment. As an example, the vectors for the word
(wAlmhndswn, Eng. and the engineers) are shown in Table 4 starting from column 2; the underscore represents nonexistent context at the beginning or end of a word. The final column shows the class to be learned, which was extracted from the training data. The first character, for example, is assigned the class ‘+’, which means that w is considered to be a segment.(2) Basic and previous decision. This is a two-stage experiment in which we first classify with the features from the basic experiment, then we add the classes predicted by the classifier to the training set. The purpose of this experiment is to give the classifier the possibility to learn from its mistakes. To illustrate, let us assume that the word wAlmhndswn above was segmented by the basic segmenter as w+Al+mhnds+w+n, where the penultimate character, w, is erroneously classified as a segment end and received a ‘+’ class. This class is then added to the feature vector. The new feature vectors for this word are the complete vectors shown in Table 4. The first feature of each vector is now the class previously assigned by the basic classifier. In the second vector from the bottom, the class previously assigned is different from the correct class: w has the basic class ‘+’, but the final class (‘−’) shows that it is not a segment end.
(3) Basic, previous decision, and POS}. For some words, several segmentations are possible based on the morpho-syntactic category of the word. For example, the word
(wDE) can be segmented as w+DE (Eng. and let, POS tag: CONJ+VERB) or as wDE (Eng. (to) bid farewell, POS tag: NOUN/VERB). Therefore, the POS tag of the word can provide useful information for segmentation. For this reason, we add POS tags for the word to the feature vectors. For example, the word wAlmhndswn would be assigned the POS tag CONJ+DET+NOUN+NSUFF_MASC_PL_NOM. We use the whole word tagger (cf. Section 5) to assign POS tags. This tagger is used without any previous linguistic pre-processing, thus avoiding a looped architecture.
Feature vectors for the characters of the word wAlmhndswn, including the previous classifier decision and a context of five characters to the left and right of the focus character as features
6.1.1 Segmentation results
Table 5 presents the results of segmentation experiments. It is obvious that segmentation can be performed very reliably: Even the basic experiment reaches an accuracy of 98.15% on words. Adding the previous decisions and the POS tags results only in a minor but significant increase to 98.23%. While this gain is minimal, it shows that identifying the morpho-syntactic category provides useful information in some cases. With the number of words in each fold averaging 96,649, the average number of errors per fold in the basic experiment is 1,788.6 compared with 1,715 in the best experiment. In other words, adding previous decisions and POS tags results in an error reduction of 4.26%.
Segmentation results
The most important features measured by gain ratio, as given by TiMBL, are the previous decision, followed by the character following the focus character, followed by the focus character itself, and then the last character of the vector. The POS feature is the least important of all the thirteen features (it adds 0.02% on average). This means that word segmentation can be largely viewed as both lexical and local.
One reason for the high accuracy of the segmenter is that the ATB is not very ambiguous with regard to word segmentation: The average number of segmentations per word is 1.01, i.e. the vast majority of words are unambiguous. This may be due to the nature of the ATB, which exclusively consists of news stories from a limited domain and thus has a limited vocabulary.
In order to estimate the performance of the segmenter on more varied texts, we had a closer look at unknown words. The average number of unknown words across the five folds is 8.68%. The average accuracy in the best experiment on unknown words is 82.22%, which is considerably lower than the accuracy on known words, with an average of 99.75%. Examining the nature of unknown words reveals that the majority of these are of the noun class. Nouns constitute 44.28% of all unknown words followed by proper nouns (21.32%) and adjectives (11.55%). However, all of these categories show inconsistencies in the ATB annotation, which makes it difficult for the segmenter to learn the correct regularities. One type of inconsistency can be found in the treatment of the feminine marker in nouns. In words such as 
(HyAp, Eng. life) or 
(mdrsp, Eng. school), the feminine marker is present, even though it does not mark the feminine form of a masculine noun. Nevertheless, these words are segmented as HyA+p and mdrs+p in the ATB.
Another inconsistency can be found in the treatment of proper nouns. Proper nouns in Arabic are regular nouns and do not have any formal distinguishing markers. In the ATB, proper nouns with the definite article Al are in some cases segmented as DET+NOUN_PROP and in others as NOUN_PROP. For example, 
(Alq*Afy, Al Gaddafi) is a proper noun that is segmented as Al+q*Afy in some cases and without segmentation, Alq*Afy in others.
6.2 Vocalization
In Arabic, short vowels are normally not written. Unvocalized words are potentially ambiguous. For example, the word 
(mSr, Eng. Egypt) can have at least five different pronunciations in Modern Standard Arabic, each pronunciation having its own grammatical category and meaning (cf. Table 6). It is therefore intuitively clear that a vocalized version of the word should provide a better basis for POS tagging than an unvocalized one.
Possible vocalizations for the word mSr
We define vocalization as the restoration of word-internal short vowels, i.e. it does not involve the restoration of case and mood endings, even when those are expressed by short vowels. Given the word 
(Almhnds, Eng. the engineer), the intended vocalized form is Alomuhanodis, where the last letter of the word, which is the case carrier, remains unspecified. Case and mood assignments are morphological and syntactic processes; attempting them with minimal knowledge would result in high error rates that make the results less than useful for POS tagging.
We define vocalization as a classification task, in which each character of a word is assigned a short vowel or the ‘−’ marker, which signifies that no short vowel is assigned. A more extensive documentation of the vocalization experiments, including a study on real-world data outside the ATB, can be found in Kübler and Mohamed (Reference Kübler and Mohamed2008) and Mohamed and Kübler (Reference Mohamed and Kübler2009).
We perform four experiments to determine the optimal feature set:
(1) Characters only. This experiment uses only characters as features, without any context beyond the word itself. We use a context of the five preceding and the five following characters, as well as the focus character. Table 7 shows the instances for the word
(Almthdp, Engl. the united).(2) Characters and word context. Here we add context words to the vector. Since unvocalized words can be ambiguous, the word itself often does not provide enough information, and the lexical context is necessary. The lexical context in this experiment includes the two previous words, the focus word, and the two following words. Thus the following features are added to all instances in Table 7:
(3) Characters and stemmed word context. Since using words as features leads to data sparseness, we modify the previous experiment and include stemmed words (cf. below for a description of our definition of stemming) instead of full forms. Thus, for this experiment, the following features are added to the instances in Table 7:
(4) Characters and stemmed words plus previous decisions. This is a two-stage process that first performs the previous experiment using stemmed words. In the second stage, we add the classifier decisions from the first stage to the feature vector. This allows the classifier to learn from its mistakes.
Vocalization instances for the word AlmtHdp
For stemming, we use a light stemmer that strips off combinations of prefixes and suffixes without using any dictionary of stems. The stemmer starts with the longest span of possible affixes and then backs off to shorter sequences. To illustrate the procedure, the stemmer checks whether the word starts with the combination Conjunction+Preposition+DefiniteArticle, i.e. with one of the prefixes wbAl, wkAl, wll, fbAl, fkAl, or fll. If the word does not match this criterion, the stemmer checks whether the word starts with the prefix Preposition+DefiniteArticle, i.e. one of the prefixes bAl, kAl, or ll. If the word does not have such a prefix, the stemmer checks whether it starts with the definite article. A similar process applies for suffixes. The stemmer does not strip off the feminine singular marker, but when a word ends with a feminine plural marker, it is normalized to the singular form.
6.2.1 Vocalization results
Table 8 presents the results for the four experiments described above. Vocalization based on context within the word reaches an accuracy of 92.31% on the word level. This shows that in most cases the vocalization of a word can be determined on the basis of only the word itself. Adding context words provides an improvement of 0.31% points, adding stems slightly more. Adding stems and previous decisions improves word accuracy to 93.28%. Although the differences are small, they are highly significant.
Vocalization results
In the experiment using stemming and previous decisions along with word-internal information, the most important feature, judging by gain ratio, is the previous decision. The second most important feature is the character following the focus character. This is not surprising since a character can be followed by a long vowel, in which case no short vowel needs to be inserted. The third most important feature is the focus character itself. This can be attributed to similar reasons, since many characters simply do not accept vocalization.
These experiments show that Arabic vocalization can be performed very reliably. However, when the evaluation for the experiment using the previous decisions is separated into an evaluation of known and unknown words, it becomes evident that the high overall accuracy is mostly due to the high accuracy on known words (96.59%). Unknown words only reach an accuracy of 57.82%. This means that vocalization will work less reliably on texts with a higher rate of unknown words. One reason for the low accuracy for unknown words is that a large class of these are proper nouns. Many, if not most, of these are foreign names that do not follow Arabic rules, and thus cannot be predicted correctly.
6.3 The integrated POS tagger
As the previous sections have shown, segmentation is necessary for segment-based POS tagging, and vocalization provides disambiguation for many ambiguous words. For this reason, we investigate whether integrating this information results in an increased POS tagging accuracy in comparison with whole word tagging as presented in Section 5. We have shown in the previous two subsections that both segmentation and vocalization can be performed very reliably with a word accuracy of 98.23% and 93.28%, respectively. Previous versions of the comparison between whole word and segmentation-based tagging can be found in Mohamed and Kübler (Reference Mohamed and Kübler2010a, Reference Mohamed and Kübler2010b).
We perform the following experiments:
(1) Gold standard segmentation. In this experiment, we perform POS tagging on segments rather than on words. That is to say, the word
(wAlmhndswn, Eng. and the engineers) is segmented as w+Al+mhnds+wn, and each segment is then assigned a POS tag. For evaluation, the segments and the segment POS tags are again combined into words. For this experiment, the segments are extracted from the ATB. Thus, the experiment provides an upper bound for the next experiment.(2) Automatic segmentation. This experiment uses the same routine as the first experiment, except that the segmentation is not the gold standard. Rather, it is provided by the segmenter that achieved the best results in Section 6.1, i.e. the segmenter using the word-internal context and previous decisions, as well as the POS tags from the whole word tagger.
(3) Gold standard vocalization. In this experiment we use the whole word POS tagger presented in Section 5. But here the input to the tagger consists of vocalized words. The vocalization is the gold standard provided by the ATB. This experiment provides the upper bound for the next experiment.
(4) Automatic vocalization. This experiment also uses whole and vocalized words as input for the POS tagger, but here the vocalized forms are provided by the best performing vocalizer from Section 6.2, i.e. the vocalizer using the word-internal context and the stems of context words, as well as previous decisions.
We did experiment with vocalizing the words first and then segmenting the vocalized versions or segmenting the words first and then vocalizing the segments. Both settings resulted in a severe degradation in accuracy of the resulting text so that we did not pursue these possibilities for POS tagging.
7 Results
Table 9 presents the results of integrating segmentation and vocalization into POS tagging.Footnote 3 The first row repeats the results of the whole word POS tagger presented in Section 5. When we compare the overall result of the whole word tagger (94.74%) with the results of integrating gold standard segmentation and vocalization, we see that using gold standard segments results in a small increase in accuracy (94.91%), but gold standard vocalization improves accuracy considerably to 95.72%. However, if we look at the results based on automatic segmentation and vocalization, we see that both result in a degradation in performance to 93.47% in the case of segmentation and to 92.59% for vocalization. This means that even though both pre-processing modules by themselves reach state-of-the-art performance, their accuracy is still too low to benefit POS tagging. All differences between the whole word approach and the other approaches, as well as between the gold standard and automatic approaches are highly significant.
POS tagging results for whole word tagging, segment-based tagging, and whole word tagging with vocalization
The results for segmentation-based POS tagging are somewhat surprising. We expected better results, since data sparseness should be alleviated in segment-based tagging. One reason for the low performance of the segment-based tagger is that while most segments have been observed in training, they are also much more ambiguous than whole words. In terms of segmentation-based POS tagging, 84.89% of the segments are unambiguous, while in whole word tagging 92.97% of the words are unambiguous. This means that while segmentation is necessary to overcome the data sparseness problem, it does introduce new problems.
In a next step, we investigate how whole word tagging and segment-based tagging perform on known and unknown words. For known words, gold standard vocalization boosts full word POS tagging accuracy to the highest accuracy for all approaches, i.e. 97.54%. As expected, when no gold standard vocalization is available, the best results for known words are reached by the whole word tagger (96.62%). The reason for this can be found in the fact that a very high percentage of Arabic words, 92.97% of word types, are unambiguous because of their complex morphology. As a consequence, if such words are present in the training data, they can be tagged very reliably.
For unknown words, however, our hypothesis was that segment-based POS tagging would perform more reliably because the tagger has access to segment information, which should have been observed in the training set even when the whole word is unknown. The results for gold standard segmentation corroborate our hypothesis: The use of gold standard segmentation results in an increase in accuracy of approximately 10% points, from 74.64% to 84.25%. The results for automatic segmentation, however, show that even though the segmentation reaches an accuracy of 98.23%, POS tagging for unknown words reaches an accuracy of only 71.06%, approximately 2.5% points lower than the whole word tagging accuracy on unknown words. The results show that in order to use this type of information successfully for POS tagging, we need near perfect segmentation results. However, remember that the POS categories for which the segmenter gives the lowest results are proper nouns, i.e. those words that do not obey Arabic segmentation regularities. They are also generally unknown to the POS tagger.
Next, we have a closer look at the results of the automatic vocalization experiment: If we consider only words that were correctly vocalized, tagging accuracy for such words is 96.59%, which indicates that correct vocalization indeed benefits POS tagging. For incorrectly vocalized words, in contrast, the POS tagging accuracy only reaches 49.09%. The percentage of words that are both correctly vocalized and correctly tagged is 88.46%. These results show that incorrect vocalization leads to incorrect POS tagging in more than 50% of the cases. Since our vocalization approach results in an error rate of 6.72% (42.18% for unknown words), we can conclude that the present vocalization reliability is not high enough to use it in POS tagging successfully.
7.1 Error analysis
We perform an error analysis for the best POS tagging approach, whole word tagging. Table 10 lists the ten most common confusion pairs.
The Most Common Errors in the whole word POS tagging approach
The table shows that all the top confusions involve nouns, adjectives, and proper nouns. The confusion between nouns and proper nouns has its source in the fact that Arabic does not designate proper nouns in a specific way. Also, the determiner proclitic Al is attached to nouns in general, whether they are proper nouns or not. Thus, both categories share the same distributional characteristics.
The noun–adjective distinction is not all that clear in Arabic since adjectives can be used as nouns, and they share the same morphological patterns: Adjectives receive the plural and feminine markers as well as case in the same way that nouns do. This corresponds to the observation by Diab et al. (Reference Diab, Hacioglu and Jurafsky2004) that in Arabic the two categories cannot be separated cleanly, which leads to inconsistencies in the treebank annotation.
There is only one POS tag in the list in Table 10 that is neither noun nor adjective: Perfect verbs (PV) are incorrectly tagged as nouns in 1.87% of all errors. This is a confusion that can potentially be solved by vocalization since many forms are ambiguous between noun and perfect verb, but differ in pronunciation. An example of this is the form 
(Drb), which can be a noun (Eng. hitting, type) or a verb (Eng. to hit). In the verb reading, possible vocalizations are Daraba and Duriba, and in the noun reading, it is vocalized as Darob.
8 Reducing the tagset size
The large size of the tagset raises the question whether Arabic can be POS tagged more reliably with a smaller tagset. For this reason, we investigate POS tagging with the ERT developed by Habash and Rambow (Reference Habash and Rambow2005). We conduct a set of experiments in which we convert our data to the ERT. This tagset is based on tokens and comprises fifteen tags, which are described in Table 11.
The Habash and Rambow (2005) ERT
The difference between segments and tokens consists in the treatment of inflectional affixes: Segmentation determines all segment boundaries, whether inflectional or clitic. Tokenization, in contrast, only delineates tokens that have syntactic functions, such as conjunctions, prepositions, or pronouns. The segmentation and tokenization for the sentence
(Eng. in the service of their own goals and their interests that are as narrow as their minds) are shown in Table 12. This sentence of seven words consists of fifteen segments or twelve tokens. The white spaces in the segmented text correspond to words, segment boundaries are marked by the + sign. In the tokenized text, tokens are separated by white spaces.
Segmentation vs. tokenization
Table 13 compares the annotation of the sentence
(w>$Ar byAn >Sdrh "mntdY >yAm mdn m$lwlp" <lY >n AldEwp >Tlqt" fy <TAr mqATEp Al{ntxAbAt Alr}Asyp fy 22 t$ryn Al>wl / >ktwbr"., Eng. A statement issued by the ‘Days of Paralyzed Cities forum’ pointed out that the call was made within the campaign to boycott the presidential elections to be held on October 22nd.) from the ATB using the full tagset and the ERT. In the latter tagset, all the inflection tags are removed from nouns and adjectives, the distinction between imperfect, perfective, and passive is no longer available, and both the preposition and the subjunctive conjunctions are now tagged as particle (PART).
A sentence annotated with both the full tagset and the ERT tagset
For the conversion to the ERT, all decisions are made so that the resulting data resemble the one by Habash and Rambow (Reference Habash and Rambow2005) as closely as possible. For most cases, the conversion is straightforward, involving the deletion of morphological information and the combination of related POS tags. In a small number of cases, we do not have enough information. For those cases we make decisions that are consistent with Habash and Rambow's overall strategy. One of these cases involves the POS tag for possessive pronoun (POSS_PRON). Since the RTS only has a PRO tag for nominal pronoun, we convert all POSS_PRON tags into PRO. While this conversion gives us data in a very similar format to that by Habash and Rambow, we note that the comparison is based on different training and test sets.
The main question that we pursue here is whether (a) the tagset reduction leads to better accuracy in POS tagging, (b) it is better to derive the ERT tagset through a conversion script after performing tagging with the full tagset, or (c) to convert first and then train the POS tagger on the ERT tagset. Thus, we present three experiments:
(1) Tagging ERT. In this tagging scheme, we first transform the original ATB to the ERT tagset, and then perform POS tagging on the new tagset directly using MBT with the whole word settings. Since we do not have a tagger optimized for tagging tokens, we have to choose between the segment-based tagger and the whole word tagger. We use the whole word tagger because tokens are more complex than segments and thus more closely resemble words. One drawback of this approach is that we do not obtain any tokenization information for the words so that we can only evaluate on the word level.
(2) Derive ERT from whole word tagging. In this experiment we POS tag the text using whole word tagging. We then convert the tagger output to the ERT. Again, we do not obtain any tokenization information for the words so that we evaluate on the word level.
(3) Derive ERT from segmentation-based tagging. In this experiment, we POS tag the text first using the segmentation-based approach. Then, we convert the POS tags to the ERT tagset. In doing so, we transform segmentation to tokenization using heuristics. For example, the word
(wnqAbAt, Eng. and syndicates) is assigned the segmentation w+nqAb+At and the composite POS tag CONJ+NOUN+NSUFF_FEM_PL. In ERT, it is tokenized as w+nqAbAt and is assigned tag CC+N. This makes it feasible to present and measure word-based as well as token-based accuracies.
Table 14 presents the results of these three experiments as well as Habash and Rambow's (2005) original results. The segmentation-based approach reaches the lowest word accuracy with 94.86%. The best results from our experiments are obtained in the setting in which we use the full tagset for training and tagging using the whole word tagger, and then convert afterwards. This approach reaches an accuracy of 96.00% as compared with 95.89% when using the ERT for training and tagging directly. This shows that there is no gain in using the ERT; tagging with the full tagset is as reliable. The reason for this may be that morphological complexity must be present in order to describe distributional features of words. For example, for the word 
(AlHsnAt, Eng. the alms), the DET+NOUN+NSUFF_FEM_PL may describe the word's characteristics better than a simple NOUN tag.
Experiments with the ERT tagset
A comparison of our results with those by Habash and Rambow (Reference Habash and Rambow2005) is difficult because they evaluate on tokens, while we are forced to use a word-based evaluation. However, if we accept that the difference between the word-based and the token-based evaluations is similar for different approaches, then our word-based results are comparable with those by Habash and Rambow even though we do not use any morphological analysis in the process.
9 Conclusion
In this paper, we have described two methods for POS tagging of Arabic text as it occurs naturally. By this, we mean text as it is normally written, i.e. without diacritics, and without pre-annotation in terms of tokenization from a treebank as used by most research on POS tagging Arabic. The two methods that we used comprise the word-based approach, which does not require any pre-processing, since the words are tagged as whole entities, and they are assigned complex POS tags. The other approach requires automatic segmentation, and the segments are assigned segment tags. For evaluation, words are reassembled, and the evaluation is performed on the word level with complex POS tags.
Our comparison shows that for the ATB, the whole word tagging scheme reaches higher accuracy than the segmentation-based tagging. Although segmentation is very reliable with a word accuracy of more than 98%, the results are not reliable enough to boost POS tagging accuracy.
The high accuracy of whole word tagging is partly due to the relatively low percentage of unknown words in the ATB, which in turn is due to the text genre of the ATB, newswire text. The analysis of unknown words shows that tagging accuracy is considerably lower for those words that lead us to the conclusion that text with a higher percentage of unknown words will be tagged less accurately than the ATB texts. However, segmentation is no cure for the problem of unknown words since it also performs worse on those words.
We have also investigated the use of vocalization in whole word tagging. While gold standard vocalization information provides an improvement of POS tagging results by 4% points, tagging based on automatically vocalized data results in a deterioration of the results, thus showing that vocalization is important information for POS tagging Arabic but it can only be used successfully when the vocalization information is extremely reliable. This is a possible venue for future research: If the quality of vocalization can be improved to a high level of accuracy, then integrating vocalization will improve POS tagging for Arabic.
In the final experiment, we investigated whether the large set of complex POS tags hampers the accuracy of POS tagging. We used the ERT proposed by Habash and Rambow (Reference Habash and Rambow2005) for comparison. Our experiments show that converting the ATB to a smaller tagset before training has a detrimental effect on POS tagging accuracy as compared with training and testing on the full POS tagset and converting the final result. This shows that morphological complexity in the POS tagset is not necessarily an obstacle for reliable POS tagging. On the contrary, if the morphological features are necessary to capture distributional characteristics of the POS tags, they improve tagging accuracy. This is encouraging, especially since applications using the POS tagger may require rich morphological information in the POS tags. In conclusion, we have shown that it is not only feasible to perform POS tagging for Arabic without previous segmentation/tokenization, morphological analysis, or vocalization but this approach also reaches the highest accuracy on the ATB.
