5.1.1 Fragment hierarchies

In XMG, elementary trees are made of common reusable tree fragments, which can be combined conjunctivelyFootnote h and/or disjunctively.Footnote i Each of these fragments are specified and encapsulated within classes (also called abstractions). These latter are defined by means of dominance and precedence constraints (Rogers and Vijay-Shanker Reference Rogers and Vijay-Shanker1994) which have the function of ensuring that the solutions generated are well-formed trees. Furthermore, XMG allows inheritance between classes. Thus, unlike its first version, ArabTAG V2.0 (Ben Khelil et al. Reference Ben Khelil, Duchier, Parmentier, Ben Othmane Zribi and Ben Fraj2016) is organized into hierarchically factorized structures. In fact, the first version consists of a set of elementary trees that are not interconnected. We have thus prioritized the hierarchical organization because it facilitates the extension of the grammar. Indeed, it is important to structure the grammar by involving various phenomena such as the inheritance structures or the hierarchy of tree patterns.

Verbal sentences . We described the syntax of verbal predicates in Arabic in a concise and modular way. We used the transitivity of the verb as a fundamental criterion to create a hierarchical organization between tree fragments. Then, we have combined these fragments together in order to obtain the three basic verb families (intransitive, transitive, and ditransitive). This organization is shown in Figure 2.

Figure 2.

Organization of the meta-grammatical description of verbal sentences.

The starting point of this organization is the abstraction “VerbalSpine.” It contributes to a fragment of a tree for the verbal spine. VerbalSpine is instantiated into two classes: The “MorphActive” class for the active form of the verbs and the “MorphPassive” class for the passive form.

We also represented two other forms of verbs with clitics. The first one describes verbs preceded by a verbal proclitic. As for the second one, it represents a verb attached with enclitic (for example an anaphor representing the object of the verb).

We introduced “ArgSpine” in order to attach the verbal spine to the tree descriptions of its arguments (the subject and the object(s)). A subject can be in elliptical, canonical, or relative form. Each of these forms is represented by a specific class.

Likewise, an object can be realized in different forms: noun phrase, prepositional phrase, verbal phrase, subordinate phrase (relative object), or an enclitic. Each of these variants is represented by a specific class.

We combined these fragments to obtain the three families of verbs (intransitive, transitive, ditransitive). Each of these classes captures the possible syntactic realizations between the different structures of the sentence. They are organized as follows: Morphactive class defines the elementary tree fragment that constitutes the verbal spine of the sentence. The Intransitive family is obtained as a result of a conjunction between Morphactive and the subject’s classes. The Transitive family is obtained by combining the Intransitive class and the Object class.

The latter represents a disjunction between the direct object (in canonical, relative, or clitic form) and the indirect object. Finally, we get the DiTransitive family by combining Transitive and the second object.

Nominal sentences . Tree fragments specific to nominal sentences are defined in the second part of our meta-grammatical description. The organization of these fragments is illustrated by Figure 3.

Figure 3.

Organization of the meta-grammatical description of nominal sentences.

In the same way as the verbal sentence, we defined the abstraction NominalSpine (parameterized by a color) that contributes to a tree fragment for the subject of nominal sentence. However, the subject is instantiated by two different classes SubjectSpine and SubjectVSpine. Indeed, we chose to distinguish these two classes in order to allow the representation of modified noun sentences (sentences that begin with a verb of existence or a verb of certainty).

“Subject” groups the set of possible realizations of themes for simple and modified nominal sentences. As a result, this class is parameterized by a variable of the case thus making it possible to define the nominative or accusative subject as well as its corresponding verbal spine.

“Predicate” has three parameters: the case of the subject, the case of the predicate, and the spine of the verb. This class includes all the possible realizations of the predicateFootnote j: adjective, noun phrase, etc.

Therefore, the family of nominal sentences corresponds to the combination of the verbal spine SubjectSpine, the nominative predicate and the nominative subject. While the modified nominal sentences are distinguished into two families: (1) family of nominal sentences modified by a verb of existence. The theme spine of the modified sentence is combined with “Subject” (nominative) and “Predicate” (accusative). As for (2), the family of nominal sentences modified by a “certainty” verb is defined by the conjunction of the subject spine of the modified sentences, “Subject” (accusative) and “Predicate” (nominative).

Phrases . Our meta-grammar defines different types of Arabic phrases, namely noun phrases $\big($ / al-murakabāt al-ismiyah $\big)$ , prepositional phrases $\big($ /al-murakabāt al-ḥarfīyah $\big)$ , and subordinate phrases $\big($ / al-murakabāt al-maūṣūliyah $\big)$ . Each category of these phrases is described by at least one elementary fragment.

5.1.2 Syntactic phenomena addressed by ArabTAG V2.0

In this section, we focus on linguistic phenomena specific to Arabic language which are represented in ArabTAG V2.0 (Ben Khelil et al. Reference Ben Khelil, Ben Othmane Zribi, Duchier and Parmentier2018). We cite the variation of the position of the syntactic components of the sentence, the optional complements, the agreement rules, the agglutinated forms, and the complex structures such as embedded structures.

Free word order . TAG is suitable to analyze natural language free word order such as Arabic. Through to its adjunction and/or substitution operations, we can combine tree structures without taking into consideration the order of combinations. Indeed, these two operations can be performed in a free order and consequently form sentences with multiple syntactic structures.

Moreover, by using XMG, we were able to represent this phenomenon within our meta-grammatical description. To ensure this, we avoided imposing priority constraints between nodes whose order change does not affect the consistency of the sentence. The generated combinations, following the conjunctions and/or disjunction of the defined tree fragments, will cover all the tree structures corresponding to all the possible positions of the nodes without precedence constraint.

Figure 4.

Tree fragments for adverbs management.

By these changes, our grammar provided the tree models. However, when checking these models, we found a problem of over-generating verbal sentences with two objects (ditransitive verbs). In order to prevent the redundancy of models and respect the order of precedence, we defined and integrated two new principles in XMG: “precedes” and “requires.” The first principle “precedes” controls the order of precedence between two nodes X and Y. While “requires” imposes the presence of a node X so that the node Y exists. Combinations of elementary fragments were more controlled with these two principles. The obtained tree models at the end of the compilation are those that fulfill both conditions, that is: the first object (object 1) always precedes the second one (object 2) and the latter (object 2) exists only if the first object (object 1) already present in the sentence.

Adjunction of adverbs and optional complements . Adjunction in TAG allows the insertion of a complete structure at an inner node of another complete structure. It is a very natural way of managing adverbs in natural language. In Arabic, adverbs and optional complement (circumstantial of time, circumstantial of place, cause, etc) can be freely placed between the constituents of the sentence.

We decided to provide, within our meta-grammatical description, two appropriate adjunction points for these complements: ADVG (left adverb) and ADVD (right adverb). The ADVG node of the model (C) of Figure 4 is an adjunction point allowing an adverb (conforming to model (A)) or an optional complement at the front of the clause, while ADVD (model (B) of Figure 4) allows the insertion of an adverb or an optional complement after a verb or an argument.

For example, we can add the adverb (kathīrān/ a lot) in the sentence (6)

1. (6)

2. Yanāmu

3. Ali sleeps

4. “Ali sleeps”.

before the subject:

1. (7)

2. Yanāmu kathīrān

3. Ali a-lot sleeps

4. “Ali sleeps a lot”.

or after the subject:

1. (8)

2. Yanāmu kathīrān

3. a-lot Ali sleeps

4. “Ali sleeps a lot.”

Insertion is performed at the adjunction node of the right adverb ADVD. In the same way, this adverb can be placed at the beginning of the sentence.

Through the adjunction node of the left adverb ADVG, (kathīrān/ a lot) is inserted before the verb (Yanāmu/ sleep). Thus, we obtain the sentence (9).

1. (9)

2. Kathīrān yanāmu

3. Ali sleeps a-lot

4. “Ali sleeps a lot”.

The syntactic trees of ArabTAG V2.0 corresponding to these three sentences are illustrated in Figure 5.Footnote k

Figure 5.

Inserting the adverb (kathīrān/ a lot) in the sentence (yanāmu / Ali sleeps).

Representation of agglutinated forms . TAG allows to manage the agglutination phenomenon thanks to its power of control. Indeed, the defined features within elementary trees can contain morphological and syntactic information. These features make it possible to assist the parsing process and remove ambiguities that may arise.

Figure 6 illustrates the agglutinated form of the sentence (10). We can define in one feature structure that the proclitic (sa/ will) is a particle of the future (Specified by the feature cat: proc_v). This kind of particles can only be attached to a verb in the indicative mode, which is indeed the case of the word (yaktubu/ he writes). The latter represents the indicative mode of the verb (ktb/ to write). Finally, the enclitic attached to the end of the verb (hu/it) represents the object of the verb (Specified by the feature fg: objet) whose case is accusative (Specified by the feature cas: acc).

1. (10)

2. sayaktubuhu

3. it-write-will

4. “he will write it”.

Figure 6.

Derived tree of the sentence (sayaktubuhu/ he will write it).

Subject Omission . ArabTAG V2.0 covers sentences with an elliptical subject and proposes the corresponding models to represent them. Figure 7 illustrates two sentences with different structures, but the same meaning: “he sleeps.”

Figure 7.

Derived trees of (yanāmu/ sleeps)/ (huwa yanāmu/ he sleeps).

The model (A) represents the derived tree of the sentence (yanāmu) composed of a verb and an elliptical subject, whereas, the model (B) represents the derived tree of the sentence (huwa yanāmu) composed of a pronoun (huwa/ he) and a verb (yanāmu/ sleeps).

Embedded structures . The representation of embedded structures is one of TAG’s strengths. It is possible thanks to the adjunction operation which allows to insert a complete structure into another structure. Let us consider the sentence (11) illustrated by Figure 8.

1. (11)

2. al-tilmīdhu tasalama al-jāizata

3. the-prize received the-student

4. “The student received the prize.”

Figure 8.

Example of handling embedded structures with TAG.

We can insert a subordinate phrase (13) between the subject (al-tilmīdhu/ the student) and his verb (tasalama/ received).

1. (12)

2. alladhī ḥaqqaqa al-nnajāḥ

3. success achieved who

4. “Who achieved success.”

The resulting sentence is:

1. (13)

2. al-tilmīdhu alladhī ḥaqqaqa al-nnajāḥ tasalama al-jāizata

3. the-prize received success achieved who the-student

4. “The student who achieved success received the prize.”

Crossed dependencies . TAG represents grammar rules (by elementary trees) of any depth, higher or equal to one.

Thus, it is possible to define an extended locality domain. This locality domain offers the possibility of describing local representation of syntactic dependencies. Thanks to the adjunction operation, we can represent some complex sentence structures such as sentences with cross dependencies.Footnote l This phenomenon occurs when dependency relationships between two sets of words cross each other.

Figure 9 illustrates the following example:

1. (14)

2. al-qalamu alladhī yaktubu bihi al-ṭālibu

3. the-student with-it writes that the-pen

4. “The pen that the student uses to write.”

Figure 9.

Example of handling crossed dependencies with TAG.

The management of cross dependencies is achieved by performing two adjunction operations in the structure “VS.” The requirement that at the end of these operations, the features of all nodes in the derived tree are unified.

Agreement rules . ArabTAG V2.0 deals with agreement rules of Arabic: the adjective-noun agreement (in definiteness, gender, number, and case), the subject-verb agreement in verbal sentence, and the agreement between elements of nominal sentences. These rules are handled in the grammar through morphosyntactic features specific to the involved nodes. By using these features, we were able to define the appropriate constraints to ensure all the agreement rules in our meta-grammatical description.

For example (Figure 10), the subject-verb agreement is ensured as follows: We define the equations of agreement within subject trees. These equations link the feature between the verb node and its subject. When a verb precedes the subject, the verb is singular, but agrees with its subject in gender. This is represented in ArabTAG V2.0 as follows: The gender agreement is realized with the equality between the gender features of the node of the verb, and the node of the subject (?V_gen =?NP_gen) and the verb is always singular with the feature ?V_num = sg.

Figure 10.

Agreement between the verb and the subject when the verb precedes the subject.

However, when the subject precedes the verb, the two distinguished cases are represented as follows (Figure 11):

(1) If the subject is non-human (?Hum = h_0) and plural (?NP_num = plr), the verb is in the feminine singular form with the following equations ?V_num = sg and ?V_gen = f. Otherwise, like in (2) and (3), the verb agrees in gender and number with these equations between features of the verb and its subject: ?V_gen =?NP_gen (for gender) and ?V_num = ?NP_num (for number).

Figure 11.

Agreement between the verb and the subject when subject precedes the verb.

5.1.3 Distribution of tree models in ArabTAG V2.0

The current version of ArabTAG V2.0 consists of 1074 generated trees from a description made of 29 classes (29 tree fragments or combination rules). The distribution of these models, that is the number of trees corresponding to each family of trees, is shown in Figure 12.

Figure 12.

Distribution of tree models in ArabTAG V2.0.

Thanks to a corpus of phenomena containing different types of Arabic sentences, we were able to verify that our grammar covers the structures already treated in the first version of ArabTAG, namely verbal sentences (active and passive form), noun sentences, subordinate phrases, prepositional phrases, and noun phrases. These latter have several categories: annexation phrase $\big($ / murakab idhāfī $\big)$ , adjectival phrase $\big($ / murakab na^ctī $\big)$ , corroborative phrase $\big($ / murakab tawkīdī $\big)$ , approbative phrase $\big($ / murakab badalī $\big)$ , state phrase $\big($ / murakab biḥāli al-mufradat $\big)$ , conjunctive phrase $\big($ / murakab al-^caṭf $\big)$ , and the semi-propositional phrase $\big($ / murakab shibh īsnādī $\big)$ . The coverage of the grammar has also been extended, compared to the first version by adding elementary trees for the representation of an elliptical subject as well as additional complements such as circumstantial complements of time, circumstantial complements of place and adverbs.

Moreover, ArabTAG V2.0 deals with more complex forms of sentences and different phenomena of the Arabic language, such as the semi-free order of words, agglutinated forms, embedded structures, and crossed dependencies and agreement rules.

However, these results are insufficient to conclude on the grammar’s quality. The manually defined corpus of phenomena is limited, that’s why we evaluated our grammar using a larger corpus of real texts (ref. Section 6).

5.1.4 Grammar validation process

In order to verify the grammar coverage, we set up a development environment (as a python script) while designing ArabTAG V2.0 with XMG. Besides our grammar, we defined manually syntactic and morphological lexicons following the 3-layer architecture of the XTAG project (XTAG 2001). The XTAG system consists of three sub-modules:

• A basis of tree schemas classified into families of elementary trees

• A lemma basis where each lemma is associated at least to a family tree

• A morphological basis in which each surface form is associated with a lemma and its appropriate morphosyntactic information.

The aim of this validation process (Figure 13) is to evaluate both under-generation and over-generation of the elementary trees of our grammar. The generated grammar must be able (1) to recognize valid sentences covering linguistic phenomena in Arabic (sentences in Arabic textbooks, newspapers, etc) and (2) to reject ungrammatical sentences. For this reason, we built a test corpus called: corpus of phenomena. It covers the most prominent and frequent syntactic phenomena in MSA.

Figure 13.

Validation architecture of ArabTAG V2.0.

The corpus of phenomena is enriched manually with grammatical and ungrammatical sentences whenever we introduce new syntactic phenomena within the meta-grammar. Ungrammatical examples have been added to check if the grammar can return incorrect syntactic trees. Each sentence of this corpus is associated with the number of expected syntagmatic analyzes (0, 1, or more).

The corpus of phenomena contains 212 examples of sentences (150 grammatical sentences and 62 ungrammatical ones). It is composed with 134 verbal sentences, 45 noun sentences, 32 noun phrases, and one prepositional phrase. Table 1 summarizes the different phenomena covered by ArabTAG V2.0.

Table 1.

Phenomena corpus

5.2.1 Type constraint hierarchy implemented in the meta-grammar

To build the hierarchy of constraints, we exploited the lexical resource “ArabicVerbNet” (Mousser Reference Mousser2010) It has 334 classes that contain 7672 verbs and 1393 frames. We reviewed all verb classes of ArabicVerbNet, and we gathered information about the described frames and the roles of their participants. For example, for a verb “to chase,” its semantic frame can have several lexical units such as “to pursue,” “to conduct,” “to run”. Its mandatory semantic roles are as follows: AGENT and THEME, and it can have the following descriptive roles: LOCATION, MODE, DURATION, etc. With this information, we created two hierarchies: the hierarchy of semantic roles and the hierarchy of frames types.

Hierarchy of semantic roles . The implemented hierarchy of semantic roles is illustrated by Figure 15. We were guided by the organization of the proposed roles within ArabicVerbNet. Verbs with similar syntax and semantic behavior are assigned to the same class group. A class group represents a hierarchy established by the semantic relations between its classes. This hierarchy is also inspired by VerbNet for English (Kipper et al. Reference Kipper, Korhonen, Ryant and Palmer2008).

Figure 15.

Hierarchy of semantic roles implemented in SemArabTAG.

This hierarchy makes it easier to assign the general roles for participants of different types of events (such as an actor who is the instigator of the event), to specify them (such as AGENT and CAUSE) or to distinguish them. Also, this distinction permits to differentiate the verb classes. For example, the roles “CAUSE” and “AGENT” are two actors who initiate an event, but the first one does it without intention contrary to the second one.

Hierarchy of types of frames . A verb may impose a set of restrictions on its argument roles. For example, it can require that a role must be a human and/or animated, etc.

Let us consider the following two sentences with their semantic interpretations:

1. (15)

2. yuḥibu ^calīyun fāṭimah

3. Fatima Ali loves

4. “Ali loves Fatima.”

5. EXPERIENCER / ^calīyun/ Ali+ PATIENT / fāṭimah/ Fatima.

6. (16)

7. yuḥibu al-kitābu fāṭimah

8. Fatima the-book loves

9. “The book loves Fatima.”

10. EXPERIENCER / al-kitābu/ the book+ PATIENT / fāṭimah/ Fatima.

The verb (āḥaba/ to love) is the predicate. Although both sentences are syntactically correct, the second one is semantically incorrect. The subject, who is the EXPERIENCER (al-kitābu/ book), cannot express feelings towards a human. Therefore, it is important to use the constraints specified for semantic roles to filter out sentences that are semantically incorrect. After examining the verbal class of (āḥaba/ loving), we noticed that the restrictions imposed by this class on the “EXPERIENCER” are “animated” and “human.” We further exploited the ArabicVerbNet classes, and we established a second hierarchy (Figure 16) that relates to the basic types of semantic frames. These types can also be used to impose restrictions on semantic roles.

Figure 16.

Hierarchy of frame types implemented in SemArabTAG.

During semantic analysis, this hierarchy will optimize the task of semantic role labeling and handle type constraints when unifying frames.

5.2.2 Meta-grammatical description of semantic frames

Kallmeyer and Osswald (Reference Kallmeyer and Osswald2013) formally define a semantic frame as base-labeled, typed feature structures. Elementary trees are paired with frames by using unification variables. Semantic roles and types are defined as attributes and values within frame description. We took another approach and defined different patterns of frames.

Using XMG2, we described within our meta-grammar (SemArabTAG) 27 semantic frames in typed feature structure format. These models of semantic frames are divided into two categories: the semantic frames of the predicate and the elementary frames. The lexicon of trees and frames are given separately, but paired during parsing. Semantic information (roles, types, and constraints) is provided by the lexicon. Thus, the syntactic tree(s) and semantic frames pair(s) are built during parsing according to information provided by the lexicon and the implemented hierarchy.

Semantic frame of the predicate . In accordance with the hierarchical modeling of the family trees of verbs (intransitive, transitive, and ditransitive), we have adopted a hierarchical organization between the descriptions of the semantic frames of the predicate (the verb). This organization is shown in Figure 17.

Figure 17.

Hierarchy of semantic frames of the predicate in SemArabTAG.

We can specify the event type of the predicate with the variable $<$ FrameType $>$ . The semantic roles corresponding to the participants of the event are defined by the FE attributes. The latter receives as value the variable $<$ FEType $>$ which designates the semantic role and its elementary frame $<$ sub-Frame $>$ . This elementary frame must fulfill the possible restrictions on the type of the semantic role ( $<$ constraint $>$ ). Thus, an event expressed by a verbal predicate, for example (nāma/ to sleep), belonging to the family of intransitive verbs requires a single participant designated by the attribute FE1. This attribute will receive as value the frame representing the semantic role “AGENT” which must have as a participant an animated entity. In the same way, an event expressed by a verbal predicate belonging to a family of transitive verbs (respectively ditransitive) requires two participants designated by the attributes FE1 and FE2 (respectively three participants: FE1, FE2, and FE3). For example, the semantic frame of the transitive verb (āḥaba/ to love) is composed of two participants. The first corresponds to the semantic role “EXPERIENCER” (the one who feels the event and which must be a human) while the second corresponds to the role “PATIENT” (the one who undergoes the event). Finally, the semantic frame of the ditransitive verb (ā^cṭá/ to give) is described by three participants: the animated “AGENT” who initiates the event, the “THEME” which is the center of the event and finally the “DESTINATION” which designates the animated physical location to which the “THEME” is directed.

Elementary frame . Within the meta-grammar, we described elementary frames corresponding to the frames of noun phrases. During the semantic analysis, these frames will be unifiedFootnote m with the frame of the predicate (and possibly with other elementary frames) to build the whole meaning of the sentence. For example, an elementary frame of a simple noun phrase is described in the FrameCommonName class. The described frame specifies the type of the phrase as well as the value of its lemma.

Figure 18 shows the syntactic tree of the noun phrase (al-shurṭīyu/ the policeman) and its corresponding elementary frame.

Figure 18.

Example of an elementary frame for a noun phrase defined in SemArabTAG.

5.2.3 Construction of the syntax-semantic interface

The syntax-semantic interface corresponds to the definition of a matrix of features for each class using the $<$ iface $>$ dimension. This matrix associates a global name (the feature) with a unification variable (the feature value) which syntactic feature is bound to where semantic feature.

Figure 19 provides an example of the syntax-semantic interface between a syntactic structure of a verbal sentence and its corresponding semantic frame. The link between these two dimensions is ensured by the feature E (for the event of the predicate) arg0 (for the first argument) and arg1 (for the second argument). The last two features receive, in the semantic dimension, the values of the semantic roles corresponding to the arguments of the predicate. Let us consider the following sentence:

1. (17)

2. ṭārada al-shurṭīyu al-liṣa

3. the-thief the-policeman pursues

4. “The policeman pursues the thief.”

Figure 19.

Description of the syntax-semantic interface within the meta-grammar.

Applying our approach, the process of constructing the meaning of this sentence is carried out as follows:

1. (1) At the lexical level, each element of the sentence is associated with its semantic frame and its syntactic tree family as follow: The elementary frames (Figure 20) are attributed to the phrases: (al-shurṭīyu/ the policeman) and (al-liṣa/ the thief). At the level of the predicate frame, the semantic roles and their constraints are defined by matching the verb (ṭārada/ to pursue) and ArabicVerbNet classes. At the end of this correspondence, the two roles attributed to the predicate are AGENT and THEME. Each of these roles admits type constraints. The “AGENT” can be “human” or “animal,” and the “THEME” must be “animate”. Thus, the verb (ṭārada/ to pursue) is given two possible combinations of semantic frames illustrated by Figure 21.

2. (2) The element “I” of Figure 22 represents the syntax-semantic interface. It allows sharing between the features of the nodes and the variables from the semantic frames.

3. (3) Substitution operations trigger the unification equations between these variables: [X1 = A] and [X2 = B] (Figure 23).

4. (4) During unification process, illustrated by Figure 24, the syntax-semantic parser has to deal with ambiguity in the semantic frames of the predicate. Thanks to the hierarchy of the constraints on the frame types, the parser checks if the imposed constraints by the roles “AGENT” (?X1) and “THEME” (?X2) are compatible with the type “human” of the two elementary frames [A] and [B].

   The implemented hierarchy distinguishes the “human” type from the “animal” type, which are both subtypes of “animate.” In other words, the “human” type of the (al-liṣa/ the thief) can be unified with the “animate” type of the role “THEME” of the two semantic frames of the verb ‘ (ṭārada/ to pursue). On the other hand, the type of the frame (al-shurṭīyu/ the policeman) is unified with a single frame of the verb (ṭārada/ to pursue). Indeed, the unification between “human” and “animal” fails, thus leading to the resolution of the ambiguity. As a result, only the first semantic frame of the predicate is selected.

5. (5) At the end of the unification, the elementary frames of (al-shurṭīyu/ the policeman) and (al-liṣa/ the thief) are inserted in the semantic frame of the verb (ṭārada/ to pursue). Thus, the final frame representing the meaning of the sentence is formed. The result of syntax-semantic analysis, namely the syntactic tree and its corresponding semantic frame, is illustrated by Figure 25.

Figure 20.

Elementary frames assigned to (the policeman) and (the thief).

Figure 21.

Semantic frames attributed to the verb (to pursue).

Figure 22.

The process of the syntax-semantic analysis of the sentence.

Figure 23.

Triggering the unification process of semantic frames.

Figure 24.

Unification of semantic frames.

Figure 25.

Result of the syntax-semantic analysis.

6.1.1 Test corpus

The test corpus (i.e., Grammatical Dataset) construction was done manually by extracting 1000 sentences from the Tunisian Basic Education reading textbook (grade 8). This textbook brings together a rich set of texts of Arabic literature. It describes the majority of the syntactic structure of sentences in MSA. So, we have selected 650 verbal sentences and 350 nominal sentences that are syntactically correct. The number of words in these sentences varies between 1 and 20 words. This selection has been made while considering the following phenomena: Negation, exclamation, interrogation, optional complements, types of phrases, active form, passive form, agreement, tense and aspect of verbs, coordination, and embedding. Unlike the corpus used during the validation process of ArabTAG V2.0 (ref. Section 5.1.4), this test corpus is composed of a wide variety of sentences and more complex structures.

In order to evaluate the capacity of our grammar to recognize ungrammatical sentences, we transformed a few sentences of this test corpus and created 250 grammatically incorrect sentences. This Ungrammatical Dataset consists of 167 verbal sentences and 83 nominal sentences. They cover the following errors:

• False agreement between the verb and the subject,

• False agreement in nominal sentences,

• False agreement between the verb of existence or certitude and its theme or subject,

• Incomplete agreement,

• Incorrect order within the phrase,

• Incorrect order of the words of the sentence.

6.1.2 Process of syntactic evaluation

We have developed a new tool dedicated to ensuring the analysis process (Figure 26). It allows to:

1. (1) Perform the morphosyntactic tagging of the input sentence.

2. (2) Generate the lemma basis and morphological basis of labeled words.

3. (3) Run the TuLiPA parser (Parmentier et al. Reference Parmentier, Kallmeyer, Lichte, Maier and Dellert2008).

4. (4) Store the result of the analysis (labeled tree(s)) in a new file in order to build a tree labeled corpus (in xml format).

Figure 26.

Parsing process.

In what follows, we detail each step.

The morphosyntactic tagging . We used a tool that we have developed, based on the POS-Tagger of Ben Othmane Zribi, Ben Fraj, and Limam (Reference Ben Othmane Zribi, Ben Fraj and Limam2017), to ensure and facilitate the morphosyntactic labeling. Each input sentence is divided into words. Each word is assigned its lemma, its corresponding morphosyntactic features and eventually its semantic frame.

Let us consider the following example:

1. (18)

2. ^calīyun yuḥibu fāṭimah

3. Fatima loves Ali

4. “Ali loves Fatima.”

This sentence is divided into three words: : (^calīyun/ Ali), (yuḥibu/ loves), and (fāṭimah/ Fatima). Then, each word receives its corresponding features. The validation of these data leads to the expansion of the lexicon database.

The lexical database consists of two well-defined files:

• The morphological basis: in which each line corresponds to a morphological item. The latter consists of the flexed form associated with a lemma and its appropriate morphosyntactic information. Figure 27 shows an extract from the morphological lexicon following the morphosyntactic labeling of the sentence (18).

• The lemma basis: analogous to the morphological lexicon. The lemma of each item is associated with its category, one (or more) family trees (e.g. NomPropre for Proper noun and TransitifActif for active-transitive verbs) and its semantic frame (FrameNomPropre for Proper noun and FrameTransitif for active-transitive verbs) defined in Arab TAG V2.0. An extract from this database is illustrated in Figure 28.

Figure 27.

Extract from the morphological lexicon: (^calīyun/ Ali), (fāṭimah/ Fatima) and (yuḥibu/loves).

Figure 28.

Extract from the lemma basis corresponding to: (^calīyun/ Ali), (fāṭimah/ Fatima) and (yuḥibu/loves).

Parsing . The TuLiPA parser (Parmentier et al. Reference Parmentier, Kallmeyer, Lichte, Maier and Dellert2008) receives as input, the sentence, the XML files of our grammar, the morphological basis, and the lemma basis.Footnote n Then, lexicalizationFootnote o is carried out as follows:

• Each word of the input sentence is associated with its morphological lexicon.

• The parser retrieves the corresponding lemmas from the morphological lexicon of the sentence.

• The parser tries to associate every lemma and its flexed form with the corresponding trees by means of feature unification.

• If the unification is done successfully, the parser attaches the flexed form to the lexical anchor of the tree template.

• The selected trees are combined to provide a valid derived tree.

• The result of the analysis is represented by the derived tree of the sentence and its derivation tree. We can have more than one syntax tree as a result. In this case, we talk about syntactic ambiguity of the sentence.

Let’s consider the previous example of the sentence (18). Initially, TuLiPA gets all the elementary trees associated with the following pairs of (lemma, flexed form): $\Big($ $\Big)$ / (^calīyun, ^caly) for Ali, $\Big($ $\Big)$ / (yuḥibu,āḥabba) for loves, and $\big($ $\big)$ / (fāṭimah,faṭmah) for Fatima. Then, the parser tries to associate these lexical units to their corresponding trees. The resulted trees (Figure 29) can be obtained once the feature of the tree nodes and the lexical units are unified. After checking manually whether the final result is correct, the derived tree of the sentence and its derivation tree are saved into an XML file: the Parsed-Corpus.

Figure 29.

Graphical output after parsing the sentence (^calīyun yuḥibu fāṭimah/ Ali loves Fatima).

6.1.3 Parsing results

Table 2 shows the parsing results of the Grammatical and Ungrammatical Datasets. The sentence is considered successfully parsed if the parser has succeeded in finding the correct result, that is the expected syntactic tree for the grammatical sentence or no tree for ungrammatical sentence. We used a manual reference to verify the parser output.Footnote p

Table 2.

Parsing results of the Grammatical (1000 sentences) and Ungrammatical Dataset (250 sentences)

As shown in Table 2, 90.22% of the sentences in these Datasets were correctly parsed. However, we measured 12.03% of the ambiguity rate (i.e., grammatical sentences having more than one correct resulting analysis). Figure 30 displays the number of resulting trees, according to the length of the 106 sentences.

Figure 30.

Performance in terms of ambiguities in relation with sentence length.

Most of the these sentences have two syntactic trees as a result. This is explained by (1) the ambiguity at the lexical level since a word can be associated with several families of trees and (2) the syntactic ambiguity of some sentences. Indeed, the component of a sentence can be a nominal or verbal phrase which is itself composed of several types of phrases. Consequently, the parser provided all matching result trees with the possible syntactic interpretations of this component. Let us consider, for example, the following sentence (19):

1. (19)

2. wadda^ca al-mudarrisa ^cadadun kabīrun min ṭalabati al-madrasati wa mudarisīhā

3. its-teachers and the-school students of number the-teachers bade-farewell

4. “A large number of students of the school and its teachers bade farewell to the teacher”.

The parser provided two matching trees. This ambiguousness is due to the semantic interpretations of the noun phrase (21):

1. (20)

2. ṭalabati al-madrasati wa mudarisīhā

3. its-teachers and the-school students

4. “students of the school and its teachers.”

The phrase, in the first tree (Figure 31), is considered as a conjunction between (ṭalabati al-madrasati/ students of the school) and (mudarisīhā/ its teachers). The semantic interpretation is different from the second tree. It is represented by an annexation phrase among (ṭalabati/ students) and (al-madrasati wa mudarisīhā/ the school and its teachers). Although these two interpretations are plausible, the first remains the most adequate. In certain cases, using semantic information can resolve syntactic ambiguities.

Figure 31.

Syntactic trees of the noun phrase (ṭalabati al-madrasati wa mudarisīhā/ students of the school and its teachers).

On the other hand, we obtained a 11.9% failure rate in parsing grammatical sentences. It represents 80 nominal sentences and 39 verbal sentences for which the parser did not find their corresponding syntactic trees. The majority of these cases are due to the non-recognition of some structures that are more complex than those described in our grammar such as the sentence:

1. (21)

2. ānā alladhī ^carafa mā fī al-īnsāni likathrati lahu

3. of-him my-observations because-of-multitude man in what knew who I-am-the-one

4. “I am the one who knew what man has because of my multitude observations of him.”

We have also underlined some unrecognized examples because of the imposed constraints of the structures which caused the under-generation of some models. For example, the parsing of sentence (22), which is grammatically correct, failed because we restricted the possibility of having a predicate that starts with an “existence” verb within the meta-grammar.

1. (22)

2. al-sururu al-khashabīyatu al-manqūshatu kānat min fanin mughriyī

3. alluring art of were the-carved the-woody the-beds

4. “The wood carved beds were alluring art.”

This finding was confirmed by the analysis of the Ungrammatical Dataset. In this instance, we check if the parser can correctly recognize the ungrammatical sentences. Since these input sentences are ungrammatical, none of them were wrongly declared by the parser to be grammatically incorrect. On the other hand, the parser did not detect the error and wrongly provided resulting trees for 33 sentences (16 sentences with incorrect agreement rules and 17 sentences with invalid structures).

This outcome can be explained by the insufficiency of the imposed constraints within the elementary trees descriptions. In addition, we have noted that it will be more relevant to refine the descriptions of some structures such as interrogative sentences or even nominal sentences.

6.2.1 Test corpus

In order to ensure a relevant comparison between our results (of the semantic analysis) with examples of semantically labeled sentences, we chose to build a corpus composed of verbal sentences extracted from ArabicVerbNet. Within this lexical resource, each type of semantic frame is associated with at least one example of sentence labeled by its semantic roles. Our aim is to ensure that the obtained results of our syntax-semantic analysis matches with those of ArabicVerbNet.

We have therefore extracted an example of sentence of each type of semantic frames described in ArabicVerbNet, and we have obtained a corpus of 460 sentences (Semantic Dataset): 30 sentences with intransitive verb (require a single argument), 201 with transitive verb (requires two arguments), and 229 with ditransitive verbs (requires three arguments). The length of these sentences varies between 2 and 14 words.

6.2.2 Process of semantic evaluation

Sentences of the Semantic Dataset undergo the phase of the morphosyntactic and semantic tagging as well as the syntax-semantic analysis (Figure 32). After the analysis, we compare the resulting semantic representations with those of ArabicVerbNet.

Figure 32.

Syntax-semantic analysis process.

First, each word of the input sentence is tagged and the semantic correspondence with the defined families of semantic frames (within the meta-grammar) is done. Following this step, the lemma and morphological basis are generated. Then, the syntax-semantic analysis is performed using a new version of the TuLiPA parser namely TuLiPA-frames (Arps and Petitjean Reference Arps and Petitjean2018). Finally, the resulted frame of the semantic analysis is compared to the semantic representation of the sentence within ArabicVerbNet.

Syntax-semantic analysis is performed using the TuLiPA-frames which provides the resulted syntactic trees and their corresponding semantic frame(s). TuLiPA-frames requires as input: the description of the syntax (SynArabTAG), the description of the semantic dimension (SemArabTAG), the hierarchy of semantic role constraints and semantic frame types, the lemma basis, the morphological basis, and finally the sentence (cut out). If the unification of the semantic frames succeeds, the parser provides one (or more) result(s) consisting of the derived tree(s) of the sentence, its derivation tree(s), and its final semantic frame(s).

Let’s take again the example of the sentence (18). Following the phase of semantic tagging (Figure 28), elementary semantic frames with type “human” are associated to (^caly/ Ali) and (faṭmah/ Fatima). The corresponding frame of the predicate (āḥabba/ to love) has two semantic roles: the “EXPERIENCER” and the “PATIENT.” These roles must be animated. During unification, the type of elementary semantic frames of (^caly/ Ali) and (faṭmah/ Fatima) must be compatible with the constraint “animate” otherwise unification fails. The implemented hierarchy of constraints allows the unification of a human with an animate. Thus, the final semantic representation of the sentence corresponds to the semantic frame shown in Figure 33.

Figure 33.

Semantic frame of the sentence (^calīyun yuḥibu fāṭimah/ Ali loves Fatima).

The semantic representation consists of a frame that has a type desire-event and two attributes FE1 and FE2. The first attribute is of type patient-undergoer-experiencer and is unified with the subframe of the subject (type concrete-natural-animate-human-selRestr) through the variable ?B0. The second attribute is of type patient-undergoer and corresponds to the subframe of the object (type concrete-natural-animate-human-selRestr) unified by the variable ?D0. Unification is triggered by substitution, and the insertion of the lexicon into the frame is triggered by lexical anchoring.

6.2.3 Result of the semantic analysis

As shown in Table 3, we were able to correctly analyze 95.43% of the Semantic Dataset.

Table 3.

Results of the semantic analysis (460 sentences)

In other words, semantic representations of 439 sentences matched with those of ArabicVerbNet. Among these analyzed sentences, 33 sentences have more than one semantic representation. This result can be explained by the lack of restriction defined in verb classes of ArabicVerbNet. Consequently, it is possible that a verb is assigned to several semantic frames without the possibility of filtering. The number of entries increases. Therefore, it increases the parsing ambiguity and generates several results that may be incorrect.

For instance, the analysis of the sentence (23) resulted in two semantic representations.

1. (23)

2. al-muṣallī ^calá al-masjidi

3. the-mosque to the-faithful perseveres

4. “The faithful perseveres in going to the mosque”.

The first, illustrated by Figure 34, corresponds to the unification of the elementary frames of (al-muṣallī/ the faithful) and (^calá al-masjidi/ to the mosque) with the frame of the predicate (/ to persevere) that requires the roles “AGENT” (animate) and “PREDICATE.”

Figure 34.

First semantic frame from the analysis of the sentence ( al-muṣallī ^calá al-masjidi/ The faithful perseveres in going to the mosque).

The second final semantic frame, represented by Figure 35, corresponds to the unification of the same elementary frames with another frame of the predicate (/ to persevere) that requires the roles “THEME” and “CAUSE.”

After a comparison between these results and the labeling of the example within ArabicVerbNet, we conclude that the first semantic frame is the correct one. However, the non-existence of restriction within the second semantic frame of the predicate, allowed obtaining an additional wrong result. The failure rate is 4.56%. The majority of the cases of failures are obtained at the level of parsing. The parsing failed for 17 sentences because their structures were not defined in ArabTAG V2.0. In contrast, semantic analysis failed only for four sentences. This result is due to the failure of the unification between the elementary frames defined for some phrases (approbative and corroborative phrases). These latter must be redefined in order to allow their unification.

Figure 35.

Second semantic frame from the analysis of the sentence (al-muṣallī ^calá al-masjidi/ The faithful perseveres in going to the mosque).