3.1 Arabic WordNet

AWN (Elkateb et al. Reference Elkateb, Black, Rodríguez, Alkhalifa, Vossen, Pease and Fellbaum2006; Rodríguez et al. Reference Rodríguez, Farwell, Farreres, Bertran, Alkhalifa, Martí, Black, Elkateb, Kirk and Pease2008) is a freely available lexical database for standard Arabic that has been constructed with the same methodology as EuroWordNet (Vossen Reference Vossen1998). AWN is one of the most promising resources. It is structured as a lexical semantic ontology and it can be used for several Arabic language processing applications. Like other WordNets, AWN was designed to represent the core unit in WordNet called the synset. A synset unit is a set of synonyms that are grouped together to describe the meaning (sense) of a word for a part of speech. Synsets are divided based on the parts of speech into four types: noun, verb, adjective, and adverb. Table 1 shows the number of synsets and words in AWN with their part of speech types. In this work, we focus only on the nominal synsets (with part of speech noun) because Wikipedia concepts are nouns. Arabic nominal synsets have been classified manually by lexicographers into NE synsets (1,147) and linguistic-term synsets (6,813).

Table 1.

Statistics for Arabic WordNet

The synsets in AWN are linked to each other through the different types of lexical relations that are used to represent knowledge as a semantic ontology. AWN contains 18,340 links, distributed across17 types of lexical relation such as hyponym, antonym, holonym, and related to. The hypernym (IS-A) is a hierarchical relation that is used to build the semantic taxonomy of the noun and verb synsets. In the semantic taxonomy, the concepts are organized according to the specificity notion (from general concept to specific concepts) as a directed acyclic graph for the noun or verb synsets. In path-based and information content methods for measuring the semantic similarity, the measures are computed between two nodes (synsets) based on the positions of these nodes in this semantic taxonomy. In this paper, the term non-hierarchical (NH) relations refers to all lexical relations (except IS-A) in the AWN database. Due to AWN being constructed in parallel with PWN, there is a bilingual semantic relation equivalence between an Arabic item and a PWN item, in which each Arabic synset is connected to a corresponding synset in English.

The gloss is a very important part of WordNet, containing a short textual definition of a synset. Unlike English WordNet, of 7,960 Arabic nominal synsets in AWN, only 189 (2.3 per cent) synsets have textual definitions (direct glosses). To overcome this limitation, for the other synsets (97.7 per cent), the glosses are extracted from the semantic taxonomy of AWN on the basis of the hierarchical relations among the synsets. For each node (synset), the gloss is generated by concatenating the lemmas in its related concepts (ancestors, children, and siblings) as the text (sequence of words). The ancestors of the synset are extracted as the nodes in the paths from the node that corresponds to this synset to the root of the taxonomy. The nodes at different levels in the semantic taxonomy have different numbers of possible ancestors, as anode in the lowest level has more ancestors than anode in the top level. Since anode can have more than one path to the root (multiple hypernym relations), the ancestors are extracted from each path. For example, the ancestors of the synset ‘atropine ⁿ ₁’ are ( ) (poison ⁿ ₁, antidote ⁿ ₁, alkaloid ⁿ ₁, mydriatic ⁿ ₁, antispasmodic ⁿ ₁, remedy ⁿ ₂, medicine ⁿ ₂, drug ⁿ ₁, organic compound ⁿ ₁, compound ⁿ ₁, agent ⁿ ₂, causal agent ⁿ ₁, substance ⁿ ₁, entity ⁿ ₁ ), which are extracted from the following paths from the node that corresponds to the synsets for the root in the AWN taxonomy:

The semantic associations between a node and its ancestors vary according to the position of the node in the taxonomy, in which a deeper node is more semantically related to its nearest ancestors than a shallower node. For instance, the direct hypernym of a shallower node (depth = 2) that corresponds to the synset ‘halogen ⁿ ₁’ is the synset ‘group ⁿ ₁’, which contains the general words . In the first path of the node , which contains the ancestors ( ), the lemmas in the nearest nodes and are considered general because they can be extracted for several synsets (e.g. the synset ‘beverage’ has the path ). On the other hand, the path from the node that corresponds to the synset ‘oxⁿ ₁’ to the root includes 15 ancestors as follows:

The nearest ancestors of the synset are the nodes , and , which contain the lemmas related to this synset. Therefore, the number of ancestors of anode is determined based on the depth of this node in the semantic hierarchy. For a node v with its paths to the root, the number of nearest ancestors from each path p that should be included in the gloss is defined by the following formula:

(1) $$\begin{equation} N_{{\rm ancestors}} \left( {p_v } \right) = {\rm Round}\left( {\frac{{{\rm Length}\left( {p_v } \right) \times 4}}{{{\rm Max}_{{\rm depth}} }}} \right) \end{equation}$$

where Max_depth is the depth of the AWN taxonomy,Footnote ¹ Length(p_v ) returns the length of the path p_v in terms of its number of edges, and Round(x) is a function that rounds a number x to the nearest integer (e.g. Round(1.5) = 2, Round(1.4) = 1). We used that number in the numerator in order to make the maximum of number of selected ancestors equal to four synsets which have strong semantic relationships with the specific node. For each path from each node to the root, this formula returns a number ranging between 0 and 4. The numbers of nearest ancestors of the paths of the synset are 1, 2, 2, 2, and 2, respectively. Thus, the nearest ancestors of this synset are poison ⁿ ₁ , alkaloid ⁿ ₁ , organic compound ⁿ ₁ , antidote ⁿ ₁ , remedy ⁿ ₂ , mydriatic ⁿ ₁ , drug ⁿ ₁ , and antispasmodic ⁿ ₁ .

The children of a synset are the nodes (synsets) that are directly subsumed (hyponymy relations) by this synset in the taxonomy. Since most nodesFootnote ² are leaves in the AWN taxonomy (they do not have children), the siblings of each node are extracted as the related concepts instead of the children. The siblings of a synset are the nodes with the same parent in the synset. There can be different numbers of children or siblings across the nodes in the taxonomy. Thus, each node is represented by at most four nodes from its children or siblings. For a synset that has more than four related nodes, based on the specificity of the nodes in the taxonomy, we select the top four nodes from its children or siblings to be the related concepts of this synset. The specificity of a node v is defined based on the length of the shortest path between the node and its leaves by the following formula:

(2) $$\begin{equation} {\rm specificity}\left( v \right) = \frac{1}{{\mathop {\min }\limits _{c \in leaves\left( v \right)} \left\lbrace {{\rm length}\left( {v,c} \right)} \right\rbrace }} \end{equation}$$

where the function length returns the number of nodes in the shortest path between two synsets in the taxonomy.

3.2 Arabic Wikipedia

Wikipedia is a goldmine containing a massive amount of structured and unstructured information that is increasing and being pruned continuously (Medelyan et al. Reference Medelyan, Milne, Legg and Witten2009). As a knowledge resource, the basic unit in Wikipedia is the article, which describes the single concept named by the unique title. The organization of Wikipedia addresses two problems due to the uniqueness of the title: polysemy and synonymy. Polysemy occurs when a word or phrase has more than one sense (meaning). For example, the word bar is ambiguous because it refers to several concepts: bar (law), bar (music), bar (counter), bar (establishment), and bar (landform). Synonymy is when different words or phrases refer to the same concept. For this problem, the redirect links and anchor hyperlinks can be considered to be the inventory of the synonyms and variation forms of the specific concept (Niemann and Gurevych Reference Niemann and Gurevych2011). For example, the Wikipedia concept Psychology is redirected from the following terms: hyde event, Human trait, Psychological, Pyhscology, Phsycology, and Psychological terms.

Wikipedia concepts are categorized in a hierarchical organization ranking from highly specific to highly generic classes. Each Wikipedia concept belongs to one or more categories. These categories can be represented as the semantic relations that describe the meaning of the Wikipedia concept in terms of the classifications. For example, the concept Chemistry belongs to three categories Chemistry, Natural sciences, and Physical sciences. Wikipedia has categories that are used for maintenance due to the crowdsourcing process of creating and editing articles. For example, the category: page need revise, Articles with dead external links, and Articles with inconsistent citation formats are considered non-semantic categories. These categories are excluded in the semantic representation of the Wikipedia concepts. Wikipedia also has across-lingual (interlanguage links) feature, which is used to link an article in one language to corresponding articles in other languages. In this paper, this feature is used to extract the title of the English concept that maps to the Arabic article.

Since its release in March 2012, the database dump of AWP has been used as a source of knowledge in the alignment between WordNet synsets and Wikipedia articles. Table 2 shows statistics for different components of AWP. Wikipedia contains several non-useful pages that cannot be linked to any WordNet synsets. The non-useful pages have the following types: articles titled by numbers (years, dates, or others), template pages, and disambiguation pages. Therefore, these pages are filtered out in the mapping.

Table 2.

Statistics for Arabic Wikipedia

Most Wikipedia concepts can be classified as NEs, which cover personal names, locations, organizations, and media. It seems to be obvious that linguistic-term synsets in WordNet cannot be linked to the NE concept in Wikipedia. The motivation for classifying the Wikipedia concepts before applying the mapping is illustrated in Table 3, which presents some examples of the NE articles that are retrieved for the linguistic-term synsets. Therefore, the identification of Wikipedia NEs in this study plays an important role in pruning the article candidates for a WordNet synset, thus, decreasing the degree of ambiguity. A rule-based approach (Saif, Ab Aziz and Omar Reference Saif, Ab Aziz and Omar2013) is used to classify Wikipedia concepts based on the labels of their categories into two types: NE concepts or linguistic-term concepts, as shown in Figure 1. This approach depends on the keywords that have been manually identified for each NE type. For instance, the keywords of a NE person contain terms (e.g. ‘people’) that can be used to form the patterns (e.g. *) of the categories for people, such as ‘Living people’, and ‘People from Isfahan’. Thus, when the categories of an article do not match the patterns of NE types, the article is classified as a linguistic term, otherwise it is classified as a NE.

Table 3.

Examples of NE candidates for linguistic-term synsets

Fig. 1.

Distribution of the types of Arabic Wikipedia articles.

5.1 Candidate extraction

This step extracts suitable articles from Wikipedia as the set of candidates for the given WordNet synset. It depends on a matching between the identifier of this synset and the identifier of each Wikipedia article. The identifier of the synset is represented by the monolingual and bilingual features as the set of keyphrases that includes the synonyms in AWN and the synset of the English WordNet that maps to the Arabic synset. The identifier of the article consists of the title and redirect titles (the words or expressions that redirect to the article) as the monolingual features, and the title with the redirect titles of the English article that links to the Arabic article by the interlanguage link as the bilingual features. The AWP database dump contains only the title of the corresponding article in another language; however, the redirect titles of the English article are extracted from the English Wikipedia database dump.

For a synset syn, each article in Wikipedia is extracted as a candidate of this synset if at least one element in its identifier exactly matches one element in the identifier of the article. The description tag in the title of the article that is used to address the ambiguity is ignored in the matching. Let W = {w ₁, w ₂, . . ., w _n } be the set of Wikipedia articles. The candidates of the given synset syn are defined as the following set:

$$\begin{equation*} {\rm Candidates}\left( {syn} \right) = \lbrace x:x \in W,{\rm matching}\left( {iden\left( {syn} \right),iden\left( x \right)} \right) > 0\rbrace \end{equation*}$$

where the matching function returns the summation of the intersection (the number of elements that occur in two sets) of monolingual and bilingual identifiers between the synset and the article. Consequently, the candidate set for each synset may contain extracted articles based on the matching of monolingual, bilingual, or monolingual–bilingual features. Figure 3 shows examples of candidate extraction for two synsets.

Fig. 3.

Example of candidate extraction of synsets.

5.2 Measuring semantic similarity

In this paper, we propose three measures that use the monolingual and bilingual features of the synset and article to compute the similarity between them. For the monolingual features, both WordNet and Wikipedia are exploited as the knowledge sources to define the unit-based measures (article-based and synset-based) that model the degree of association between the synset and the article. The unit-based (monolingual) measures are modeled based on Arabic features in two directions (from WordNet to Wikipedia and vice versa). For the bilingual features, the synonyms of English words are used to compute the similarity between the synset and the article. The following subsections describe in detail these three similarity measures.

5.2.1 Article-based measure

For this measure, Wikipedia is exploited as the source of knowledge to model the semantic relatedness by hybridizing the corpus-based and hyperlink-based methods. The synset is represented as a set of Arabic words extracted from the semantic information in WordNet. After that, the corpus-based technique is used to extract the core semantic units (concepts) of the text representation of a synset. For the bag-of-words set of the synset, the Wikipedia-based semantic representation method (Section 4) is used to construct the semantic vector of this set of words. The non-zero elements in the semantic vector contain the Wikipedia articles that have a semantic association with the synset (an article with higher weight is more related to the synset than another article). Then, the highest scoring elements (ten in this work) from the semantic vector are selected as the core semantic units for the given set of words (synset representation).

For the core semantic units of the synset, the hyperlink-based method is used to measure the semantic relatedness between each unit (concept) and the given article. This hyperlink-based method was inspired by the normalized Google distance (Cilibrasi and Vitanyi Reference Cilibrasi and Vitanyi2007), which models the semantic distance (SD) between two concepts based on the incoming links as:

(3) $$\begin{equation} SD\left( {c_1 ,c_2 } \right) = \frac{{\log \left( {{\rm Max}\left( {\left| {IL\left( {c_1 } \right)} \right|,\left| {IL\left( {c_2 } \right)} \right|} \right)} \right) - \log \left( {\left| {IL\left( {c_1 } \right)\mathop \cap IL\left( {c_2 } \right)} \right|} \right)}}{{\log \left( {\left| N \right|} \right) - \log \left( {{\rm Min}\left( {\left| {IL\left( {c_1 } \right)} \right|,\left| {IL\left( {c_2 } \right)} \right|} \right)} \right)}} \end{equation}$$

where IL(c_i ) is the set of concepts that link to the article c _i . Since the range of SD is between zero and infinity, this measure can be converted to determine the semantic relatedness rather than distance with the following formula:

(4) $$\begin{equation} Sim_{hl} \left( {c_1 ,c_2 } \right) = e^{ - 2 \times SD\left( {c_1 ,c_2 } \right)} \end{equation}$$

Finally, the semantic relatedness between a synset and an article is computed as the average of the similarity between the article as a unit and each concept in the core semantics of the synset as follows:

(5) $$\begin{equation} Sim_{WP} \left( {syn,{\rm Art}} \right) = \frac{{\mathop \sum \nolimits _{c \in CS\left( {syn} \right)} Sim_{hl} \left( {c,{\rm Art}} \right)}}{{\left| {CS\left( {syn} \right)} \right|}} \end{equation}$$

5.2.2 Synset-based measure

AWN is used as the source of semantics to model the similarity between synsets and articles. An article is represented by its features from Wikipedia (title, redirects, categories, out-links, and page definition) as a set of keyphrases (the preference is for multiword expressions). Then, this text representation of the article is transformed into the set of the synsets (the semantic core of the article) in AWN by adapting the PPR algorithm (Agirre and Soroa Reference Agirre and Soroa2009). This algorithm has been introduced for word-sense disambiguation or semantic measures, depending on the semantic graph of WordNet. For a set of key phrases, the output of this algorithm is the semantic representation vector of the structural relevance of AWN concepts to the given set of key phrases. This semantic representation vector contains the weight of each node (synset) in the semantic graph according to its semantic relatedness to the given text (a node with high weight is more related to the text than a node with low weight). In the synset-based measure, the top ten highest weighted nominal concepts in the PPR vector are selected as the core semantic units of the given set of key phrases of the article. Therefore, the semantic similarity between the synset and the article is modeled based on the relations between the core semantic units of the article and the synset using the taxonomical relations in WordNet.

The taxonomical relations are exploited to build the WordNet semantic hierarchy, which can be used to measure the semantic similarity between any two synsets. Based on the semantic hierarchy, measures can be classified as path-based measures (Wu and Palmer Reference Wu and Palmer1994; Li, Bandar and McLean Reference Li, Bandar and McLean2003) or information content measures, using either a corpus-based method (Resnik Reference Resnik1995; Leacock and Chodorow Reference Leacock and Chodorow1998) or an intrinsic-based method (Seco, Veale and Hayes Reference Seco, Veale and Hayes2004; Zhou, Wang and Gu Reference Zhou, Wang and Gu2008). In this paper, we use an intrinsic-based measure (Sánchez, Batet and Isern Reference Sánchez, Batet and Isern2011) where the specificity of a concept quantifies the amount of information it provides for different features that relate to the position of the concept in the semantic taxonomy. This measure of the information content of a concept is based on the leaves and the subsumers of the concept, which correspond to the degree of its generality and concreteness in the semantic taxonomy, respectively:

(6) $$\begin{equation} IC_{{\rm Sanchez}} \left( c \right) = - \log \left(\frac{{\frac{{\left| {{\rm Leaves}\left( {\rm c} \right)} \right|}}{{\left| {{\rm subsumers}\left( c \right)} \right|}} + 1)}}{{{\rm Max}_{{\rm Leaves}} + 1}}\right) \end{equation}$$

where Max_Leaves is the number of leaves in the taxonomy. According to Pirró and Euzenat (Reference Pirró and Euzenat2010), the semantic similarity between two nodes (synsets) in a taxonomy is measured by taking into account the information content of the least common subsumer of the nodes (lcs) as:

(7) $$\begin{equation} Sim_{IC} \left( {c_1 ,c_2 } \right) = \frac{{IC_{{\rm Sanchez}} \left( {lcs\left( {c_1 ,c_2 } \right)} \right)}}{{IC_{{\rm Sanchez}} \left( {c_1 } \right) + IC_{{\rm Sanchez}} \left( {c_2 } \right) - IC_{{\rm Sanchez}} \left( {lcs\left( {c_1 ,c_2 } \right)} \right)}} \end{equation}$$

Finally, the synset-based measure between the article and synset is computed according to the information content similarity of the synset and each concept in the core semantics of the article as:

(8) $$\begin{equation} Sim_{WN} \left( {{\rm Art},syn} \right) = \frac{{\mathop \sum \nolimits _{u \in CS\left( {{\rm Art}} \right)} Sim_{IC} \left( {syn,u} \right)}}{{\left| {CS\left( {{\rm Art}} \right)} \right|}} \end{equation}$$

5.2.3 Bilingual-based measure

The bilingual-based measure is introduced to exploit the English words in WordNet and Wikipedia that correspond to the Arabic synset and article using the cross-lingual links. This measure focuses only on the most important relation (synonyms) in English WordNet and the corresponding relation in Wikipedia (redirects) to compute the similarity based on the bilingual features. Therefore, the synset and article are represented as the set of key phrases of the English synonyms that correspond to the given synset and the title of the English article with its redirect titles that link to the Arabic article by the interlanguage link. To determine the degree of the importance of the elements in the bilingual representation sets for the synset and article, each lemma in the two sets is weighted based on the number of noun senses of the lemma in WordNet and the number of articles that contain this lemma in their English identifiers as:

(9) $$\begin{equation} {\rm Weight}\left( {{\rm lemma}} \right) = \left\lbrace {\begin{array}{c}\frac{1}{{\left| {{\rm Senses}_{{\rm noun}} \left( {{\rm lemma}} \right)} \right|}}{\rm where},\;{\rm lemma}\; \in \;{\rm synset}\\ {\frac{1}{{\left| {{\rm Articles}\left( {{\rm lemma}} \right)} \right|}}{\rm where},\;{\rm lemma}\; \in \;{\rm article}}\\ \end{array}} \right. \end{equation}$$

Finally, the bilingual measure is used to find the similarity between the synset and the article based on the weight of the lemmas in the intersection between the bilingual features of the synset and the article. This measure is normalized to the maximum length of the sets:

(10) $$\begin{equation} Sim_{BF} \left( {Syn,{\rm Art}} \right) = \frac{{\mathop \sum \nolimits _{w^{\prime } \in BF\left( {Syn} \right)\mathop \cap \nolimits BF\left( {{\rm Art}} \right)} {\rm weight}\left( {w^{\prime }_{{\rm synset}} } \right) \times {\rm weight}\left( {w^{\prime }_{{\rm Article}} } \right)}}{{\max \left( {\left| {BF\left( {Syn} \right)} \right|,\left| {BF\left( {{\rm Art}} \right)} \right|} \right)}} \end{equation}$$

5.3 Matching classification

Starting with AWN synsets, this step classifies each synset in AWN as mapped or not-mapped. The mapped class is when the synset has an identical concept in Wikipedia. The not-mapped class means the synset cannot be matched to any articles or the semantic associations between the synset and the matched article are very limited. Based on the candidate extraction step for each synset, there are two possible cases for the synset in Wikipedia: null-matched (no articles can be retrieved for the synset) and matched (at least one article matches the synset). The synsets that are null-matched are classified directly as not-mapped to all Wikipedia articles. For the synsets that have candidates in AWP, the semantic similarities are used to decide whether the synset can be linked to an article in Wikipedia.

To classify the matching, let Cand(syn) = {a ₁, a ₂, . . ., a_m } be the set of Wikipedia articles (candidates) for a given synset syn. Three semantic similarity vectors are constructed by measuring the similarity between the synset and each article in the candidate set using three measures: article-based, synset-based, and bilingual measures. Each vector contains the similarity scores of the given synset and its candidates obtained for one similarity measure. For instance, the similarity vector of the article-based measure (Sim_WP ) is defined as: SV_WP = {a ₁, v _wp1, a ₂, v _wp2, . . ., a_m , v_wpm } where v_wpi refers to the similarity score between syn and the a_i (Sim_WP (syn, a ₁)). Due to the differences between the granularity of the semantic contents in WordNet and Wikipedia, the matching classification is introduced to handle two important issues that can occur in the matching of identifiers for the synset. These issues reflect the variety of semantic associations between the synset and its candidates. In non-similar semantics, the semantics of all candidates are totally different from the synset, whereas in multi-similar semantics, the semantics of the synset are related to more than one candidate. In non-similar semantic synsets, the selection of the article with the highest similarity score among candidates (as in word-sense disambiguation) is not suitable. On the other hand, the challenge for a synset with multi-similar semantics is how to determine the most related article from the set of candidates.

To handle the non-similar semantic issue, the threshold-based classification technique (Niemann and Gurevych Reference Niemann and Gurevych2011) is used to detect unrelated articles (the candidates with similarity scores below the threshold). The threshold is determined for each similarity measure by searching for the optimum value from zero to one that maximizes the performance (in terms of the F-measure) of the mapping for the Arabic gold standard data set (Section 7.1). Based on the threshold of each similarity measure, its corresponding semantic similarity vector is used separately to assign the binary weight with respect to the similarity score for each article in the candidate set: 1 (identical or related to the synset) or 0 (unrelated). For the synset–article pair, syn–a _i is assigned to 1 if the similarity score v_wpi exceeds the threshold. Consequently, the binary weights of the synset–candidate pairs using three semantic similarity vectors can be combined together as:

where x_i , y_i , and z_i are the binary weights (0 or 1) that have been assigned to the syn − a_i pair depending on the semantic vectors of the article-based, synset-based, and bilingual-based measures, respectively. The c_i is the combined binary weight (0 or 1), which depends on the majority voting function,Footnote ³ c_i = x_iy_i + x_iz_i + y_iz_i , of three inputs (x_i , y_i , and z_i ). This combined weight of all pairs means that the synset can be semantically related to none, one, or more than one article (multi-similar semantics issue). For a synset with a combined weight of 0 for all candidates, the synset–article pairs are classified as not-mapped (none-similar articles for this synset). For a synset that relates to only one article a_i , the syn–a _i pair only is classified as mapped, and the other pairs are classified as not-mapped.

Since the aim of the proposed method is to map a synset to only one Wikipedia article, therefore, for each synset, the mapped class must be assigned to only one synset–article pair. From the candidates, the most related article is selected according to the highest similarity score of the integrated measure (the total score of the three measures). In spite of the similarity scores of the synset and article being ranged between zero and one for the monolingual measures, however, these scores are not balanced across any two measures because each measure models the similarity depending on different sources for the semantics. For this reason, the similarity vectors of monolingual measures (article-based and synset-based) are normalized to balance the similarity scores in two vectors. In each vector, the normalization function for each score depends on the maximum and minimum values in the vector as:

(11) $$\begin{equation} {\rm norm}\left( {v_i \in SV} \right) = \frac{{v_i - \min \left( {{\rm SV}} \right)}}{{\max \left( {{\rm SV}} \right)}} \end{equation}$$

Consequently, the synset–article pair where the article has the highest integrated similarity score is classified as a mapped label. Other synset–article pairs are classified as not-mapped, regardless of their similarity scores.

7.1 Test set

To evaluate the proposed synset–article mapping, a gold standard data set that contains the WordNet synsets and their corresponding Wikipedia articles is required. To our knowledge, there is no gold standard (available evaluation data sets) for the alignment of WordNet synsets and Wikipedia articles in Arabic. Therefore, we created an Arabic data set for evaluating the proposed method in addressing the ambiguity and synonymy issues in the alignment of WordNet synsets and articles in Wikipedia.

Words in AWN can be classified into two types: monosemous (the word has only one sense) and ambiguous words (each word has more than one sense). Since the proposed method has been introduced to address the ambiguity issue in mapping synsets to Wikipedia articles, the data set was created to encompass the ambiguous words in WordNet and Wikipedia. From WordNet, we randomly selected 50 ambiguous words, which returned 188 synsets with the part of speech noun, as the first sample. From Wikipedia, 115 articles that could be candidates for 30 synsets by matching their identifiers were selected as the second sample. The synsets in these two samples were selected to cover the ambiguous single words in the two sources, WordNet and Wikipedia. The synsets in the final sample were selected from WordNet using two factors: the degree of specificity of the synset and whether the extracted candidates had monolingual or bilingual features. To ensure this sample covers various degrees of specificity, 200 synsets are selected from different positions (based on the depth of the synset) within a semantic taxonomy of AWN. These selected synsets were distributed according to their candidates from Wikipedia to include the following cases: monolingual only (all candidates of a synset were extracted by matching Arabic features only), bilingual only, and monolingual–bilingual together (the candidates were retrieved by matching both Arabic and English features).

The manual mapping annotation for the compiled data set was accomplished by three human annotators (experts in computational linguistics). The semantic information of the synset (synonyms, related words, and gloss) and the article (title, category, and definition) were presented to provide the annotators with sufficient information to make a decision. For each synset–article pair, the annotators were asked to classify this pair as one of the following categories: mapped (1) or not-mapped (0). The mapped category is when the WordNet synset is identical or related to a large extent to the Wikipedia concept in the synset–article pair. The not-mapped category indicates the semantics of the synset are completely different from the semantics of the article or the relation between them is very limited. Finally, the synset–article pair was assigned to a category (mapped or not-mapped) if at least two annotators marked the pair as this category. Table 7 presents the statistics of the Arabic mapping data set.

Table 7.

Statistics for the mapping data set

7.2 Performance measures

The proposed method was evaluated for the manual data set by comparing the category obtained by the method to the category assigned by the human experts for each synset–article pair in terms of the performance measures (precision, recall, F-measure, and accuracy). The accuracy measure is used to determine the percentage of correctly identified mapped and not-mapped classes compared to the manual classifications in the data set. Precision (P) denotes the ratio of correctly mapped to all mapped classes assigned by the method. Recall (R) determines the ratio of correctly mapped to the manually mapped class in the data set. The F-measure is the harmonic mean of the precision and recall.

7.3 Experiments

7.3.1 Representation features

This experiment aimed to examine the performance of the method with different representations of the monolingual features from AWN and AWP. The text representation of a synset can be extracted as the bag-of-words set from the following semantic information in AWN: synonyms (S), (NH) relationsand glosses (G). On the other hand, the text representation of an article can be extracted from the following Wikipedia features: titles (T) (title of article with redirect titles), categories (C), out-links (L), and the page definition (D). Table 8 presents the evaluation results for different representations of articles and synsets for the data set, which shows that feature representations of the synsets and articles play an important role in the performance of the mapping method. The accuracy for the data set ranged between 85.5 per cent and 93.6 per cent for the different representations of synsets and articles. The representation R73, which combines all semantic features for two units, had the highest accuracy (93.6 per cent) in comparison to other representations. For the articles, the features can be ordered based on their degree of importance as: out-links (L), definitions (D), and categories (C). On the other hand, in addition to the synonyms, the gloss feature of the synset improved the performance of the method better than the NH feature.

Table 8.

Evaluation results for different representations of synsets and articles

7.3.2 Unit-based versus text-based methods

For Arabic features, the semantic similarity between a synset and article was computed using the unit-based measures (synset-based and article-based) that depend on the PPR and ESA semantic representation vectors. In these measures, the PPR and ESA representation vectors were pruned by selecting the top ten weighted concepts from each vector as the semantic core concepts of the synset and article. To investigate the effect of this pruning, the unit-based measures are compared with the text-based measures that use the direct semantic representation vectors of the PPR algorithm and ESA model. The latter measures are classified as text-based measures because the similarity between the synset and the article is computed as the similarity between two text fragments of the synset and article. Based on the semantics in AWN, the first similarity measure is defined by the cosine function between the PPR representation vectors of the synset and article as:

(12) $$\begin{equation} TB_{WN} \left( {{\rm Syn},{\rm Art}} \right) = \frac{{{\rm PPR}\left( {{\rm syn}} \right) \times {\rm PPR}\left( {{\rm Art}} \right)}}{{\left| {PPR\left( {syn} \right)} \right|\left| {PPR\left( {Art} \right)} \right|}} \end{equation}$$

The second measure uses the semantics in Wikipedia to compute the similarity between the synset and article. For two bag-of-words sets, the ESA vector is constructed for each set. Then, the similarity measure between the two vectors is defined by the Jaccard function as:

(13) $$\begin{equation} TB_{WP} \left( {Syn,Art} \right) = \frac{{ESA\left( {Syn} \right) \cdot ESA\left( {Art} \right)}}{{\left| {ESA\left( {Syn} \right)} \right|^2 \left| {ESA\left( {Art} \right)} \right|^2 - ESA\left( {Syn} \right) \cdot ESA\left( {Art} \right)}} \end{equation}$$

To evaluate the text-based technique (TB), the synset-based and article-based measures are replaced by the text-based TB_WN and TB_WP measures in the proposed mapping. For the evaluation data set, Figure 4 shows the accuracy of the mapping using the TB for the similarity measure (TB_WN and TB_WP ) compared to the unit-based technique for the similarity measures (synset-based and article-based) for the same representations of the synset and article. The comparison shows that the unit-based technique is more accurate than the TB. In the unit-based technique, the TB (building the semantic vectors) was used only to construct the semantic core of the synset in AWP and the semantic core of the article in AWN (as a feature selection task). The hyperlinking feature in Wikipedia and the specificity of the concepts were exploited in the unit-based technique to gather more semantic evidence than was possible for the TB in modeling the similarity.

Fig. 4.

Comparison between text-based (TB) and unit-based (UB) techniques.

7.3.3 Semantics in AWN compared to AWP

In addition to the bilingual measure, the proposed method integrated the synset-based with the article-based measures that depend on the semantics in AWN and AWP. To investigate the effects of integrating the semantics for two different sources, the mapping is evaluated by exploiting only one monolingual similarity measureFootnote ⁴ that reflects the efficiency of the corresponding source (AWN or AWP). Figure 5 shows the best accuracy of the mapping using the semantic similarity of monolingual features based on the semantics in the following sources: only AWN, only AWP, and a combination of them. The evaluation shows that the performance of the mapping based on the combined similarity of the two sources is better than using an individual measure for either the unit-based or TB. For a single monolingual source, the similarity based on the semantics of AWP outperformed the semantics of AWN. This is due to the limitations of AWN in terms of its coverage of words and the lack of semantic and lexical relations.

Fig. 5.

Comparison of AWN, AWP, and WordNet–Wikipedia (combined) for semantic measures.

7.3.4 Effects of bilingual features

To determine the effect of bilingual features on the mapping, the monolingual method is assessed for the same data set. The monolingual method is a mapping method using only Arabic features for extracting candidates and measuring semantic similarity. In the matching classification step of the monolingual method, a candidate is classified as mapped or not-mapped based on only two monolingual measures (synset-based and article-based). Figure 6 shows a comparison between the monolingual method (M) and a method that combines Arabic features with English features (M∪B). From this figure, it can be observed that the proposed method that exploited the monolingual and bilingual features had better accuracy than the monolingual method.

Fig. 6.

Monolingual (M) features versus cross-lingual (CL) features for mapping.

Bilingual features in candidate extraction To examine the matching for the mapping data set, the monolingual method is compared to the same method but using bilingual features for candidate extraction only, as shown in Figure 7. The performance of the method is evaluated using the recall (R), which reflects the ratio of correctly mapped articles compared to the mapped class in the data set. For each synset in AWN, the ambitious goal of this step is that for a synset, the identical concept in AWP should be included in the extracted candidate set. The aim of using the bilingual features in this step is to handle the incompatibility of the concept's identifiers in AWN and AWP. The evaluation results in Figure 7 show that the chance of including the correct concept in the candidate set for some synsets increased by using the bilingual features. In other words, the cross-lingual matching was better than monolingual matching for mapping.

Fig. 7.

Monolingual (M) features versus cross-lingual (CL) features for only candidate extraction.

To determine the effect of the bilingual features in candidate extraction, we evaluated this matching step for the mapping data set in terms of the capability of extracting the corresponding concepts of the synsets. Thus, only the synset–article pairs that have been annotated by the experts as a mapped class are selected (129, 30, and 193 pairs for three samples of the data set). For the selected pairs from the data set, let M and B be the sets of identical articles that have been extracted using the monolingual and bilingual features, respectively. Then B – (M∩B) is the set of identical articles that have been retrieved by matching the bilingual features and could not be extracted using the monolingual feature. This set reflects the contribution of the bilingual features for handling the incompatibility of terming the concepts in the two sources for the manually mapped pairs. For the mapped synset–article pairs, Table 9 presents the statistics of correct articles that have been retrieved by matching the monolingual and bilingual features.

Table 9.

Monolingual versus bilingual matching for352 mapped pairs

Bilingual features in semantic similarity In measuring the semantic similarity, the important semantic relations (synonyms) were used to model the similarity based on the bilingual features. Figure 8 shows the accuracy of the monolingual measure and the integrated cross-lingual similarity measure (monolingual and bilingual) for the data set.

Fig. 8.

Monolingual (M) features versus cross-lingual (CL) features for measuring only similarity.

In summary, the evaluation of the monolingual method is considered to be the baseline. The effect of bilingual features on the performance of the mapping can be determined as the improvement ratio of the baseline results against the results when incorporating bilingual features in candidate extraction (Cand), measuring semantic similarity (Sim), and both (Cand& Sim). Figure 9 shows the improvement ratio for the performance method when using bilingual features against the baseline results.

Fig. 9.

Improvement ratio of the accuracy with bilingual features.

7.3.5 Comparison with previous works

From the literature, we adapt five methodsFootnote ⁵ for Arabic sources for mapping Arabic synsets to their corresponding articles in Wikipedia using only Arabic features in the matching (candidate extraction) and measuring the similarity. The differences among these methods can be summarized according to the following characteristics (as in Table 10): type of similarity method (text-based or structure-based), semantic representation technique, similarity measure, representation features, and the mapping criteria (maximum similarity or threshold). Since AWN does not have glosses, we use all semantic information (S, NH, and G) in AWN to represent the synset in all methods.

Table 10.

Characteristics of the state-of-the-art mapping methods adapted for Arabic

For evaluation, these monolingual-based methods are applied to the synsets in the Arabic data set to compare the mappings by different methods against the manually labeled ground-truth mappings. In all methods, only the Arabic identifiers of the synsets and articles are used to retrieve the candidates (articles) for each synset based on the matching of the lemmas. It must be pointed out that the candidates in the data set that were extracted using only bilingual features have been excluded when measuring the similarity and the threshold classification. In other words, a synset with a bilingual candidate pair is classified directly as not-mapped if the Arabic identifiers of the synset are different from the identifiers of the article. In the NG11 and FS12 methods, a threshold-based classification is used to determine the parameter (similarity threshold) that achieves the highest F-measure based on the manual mappings. For the matching classification, the candidate with the highest similarity value (and which exceeds the threshold in NG11 and FS12) is classified as mapped, and other candidates are classified as not-mapped. The details of the implementation of measuring the similarity in each method are presented below.

In RC05, the similarity between a synset and an article is computed based on the vector space model, which represents the synset and article as weighted vectors. The semantic information of all nominal synsets in AWN is transformed as the collection of documents, where each document contains the sequence of words from the S, NH, and G featuresFootnote ⁶ of the corresponding nominal synset. Based on this collection of texts for all nominal synsets in AWN, each synset is represented by the vector that contains the TFIDF weights of each word in its representation features. In AWP, the representation vector of an article contains words paired with their frequency in the free text (after stemming and removal of stop-words). Then, the similarity between the synset and article is computed using the cosine measureFootnote ⁷ between their representation vectors. For the given synset and its candidates, the candidate with highest similarity is classified as mapped, and other candidates are classified as not-mapped.

In PN10 and FS12, a synset and article are each represented as a set of words (disambiguation context) from the features in AWN and AWP. Then, the gloss-based technique is used to measure the similarity between the synset and article-based on their disambiguation context sets. Let DC(s) and DC(a) be disambiguation context sets of the synsets and article a. The similarity in the PN10 method is measured as the intersection between two sets $( {| {DC( s)\mathop \cap \nolimits DC( a )} | + 1} )$ . It is normalized using the joint probability p(a, s). In the FS12 method, the similarity is computed as

(14) $$\begin{equation} Sim_{FS12} \left( {s,a} \right) = \frac{{\left| {DC\left( s \right)\mathop \cap \nolimits DC\left( a \right)} \right|}}{{\min \left( {\left| {DC\left( s \right)} \right|,\left| {DC\left( a \right)} \right|} \right)}} \end{equation}$$

For the methods that depend on the structure-based similarity, the semantic graph of AWN is constructed, in which the nodes represent the synsets and the edges represents all relations among synsets. To measure the semantic relatedness for the NG11 method, the semantic graph of AWN is represented by the row-stochastic transition matrix $M \in \mathbb {R}^{n \times n}$ , where the entry M _ij corresponds to the probability of moving from node i to j: this is (1/degree(i)) if there exists an edge from i to j and 0 otherwise. The PPR algorithm is used to generate the semantic representation vector of a node or set of nodes by solving the following linear system:

(15) $$\begin{equation} P = \alpha MP + \left( {1 - \alpha } \right)v \end{equation}$$

where α is the damping factor, which is set to 0.85, and v is an n-dimensional personalization vector whose initialization differentiates between direct PPR and basic PPR. Direct PPR can be used only to generate the representation vectors of synsets; for a given synset, the personalization vector is initialized by assigning the weight of 1 to the element that corresponds to this synset, and the weight of 0 to the others. The personalization vector in basic PPR is initialized depending on the textual representation of the synsets or articles. For a given bag-of-words, let C = {c ₁, c ₂, . . ., c_k } be the set of synsets (candidates) of the words. The vector is initialized by assigning the weight of 1/k to the elements that correspond to the synsets in C, and the weight of 0 to the others. Since the output of the PPR algorithm is the representation vector over the synsets in AWN, for a given synset–article pair, the semantic relatedness is measured as the similarity between their representation vectors using the x ² measure.

To measure the similarity between a synset and an article in the NP12 method, the semantic graph of AWN is used to build the subgraph (DG) of the disambiguation context of the article. For a given disambiguation context (as a set of words) of article a and synsets, the synsets that can be candidates for each word in this set are pooled together with s to represent the nodes of the subgraph. Then, for each node v in DG, the depth-first search algorithm is used for the AWN graph to find the paths of maximal length L between v and v ∈ V(DG) − v, and add the nodes and edges in each path to DG.Footnote ⁸ Finally, the semantic similaritySim _NP12(s, a) is measured based on the lengths of paths between the synsets and each node in the subgraph of the article a as

(16) $$\begin{equation} Sim_{NP12} \left( {s,a} \right) = \mathop \sum \limits _{s^{\prime } \in DG\left( a \right)} \mathop \sum \limits _{p \in Paths\left( {s,s^{\prime }} \right)} e^{ - \left( {{\rm Length}\left( p \right) - 1} \right)} \end{equation}$$

where the Length(p) returns the number of edges in the path p. This similarity is normalized using the joint probability as

(17) $$\begin{equation} p\left( {a,s} \right) = \frac{{Sim_{NP12} \left( {s,a} \right)}}{{\mathop \sum \nolimits _{\begin{array}{c}s^{\prime } \in senseWN\left( s \right) \\ {a^{\prime } \in senseWP\left( s \right)} \\ \end{array}} Sim_{NP12} \left( {s^{\prime }, a^{\prime}} \right)}} \end{equation}$$

For comparison, Table 11 presents the best evaluation results that have been obtained by applying the mapping methods to the Arabic data set. For theNG11 method, we report the evaluation results for the two versions of the PPR algorithm (basic and direct) for building the semantic vectors to measure the similarity. The difference between these versions is the initialization of the distribution vector, which depends either on the text representation of the synset (TB) or on the synset only as the unit (unit-based technique) for the basic and direct versions, respectively. The evaluation results of the proposed method for two cases (monolingual and cross-lingual features) are shown in the last two lines of Table 11.

Table 11.

Evaluation of mapping methods for the Arabic data set

For the ambiguity issue, the performance of the mapping methods depends heavily on the technique for measuring the similarity, which is used to determine the degree of the semantic association between the synset and the article. To ensure a fair comparison, we evaluate the similarity measures of the state-of-the-art methods for the Arabic mapping data set using the same experimental settings (representation features and the matching classification). For each synset in the data set, the similarity is measured between this synset and each article that was retrieved based on the monolingual and bilingual features. Since the performance of the similarity measures may be affected by the representation features employed for the two sources, we use the same features (R73) from AWN and AWP to represent the synset and article for each measure. Using the same threshold-based classification technique, the threshold is determined for each similarity measure based on the similarity scores of the corresponding measure. The evaluation results of the similarity measures are presented in Table 12.

Table 12.

Comparison of the similarity measures for the Arabic data set

8.1 Adapting the mapping method

The mapping method is adapted to classify each concept in the not-mapped articles as inserted or not-inserted (instead of mapped or not-mapped as in the main method) as follows. For an article, the bilingual features are used to retrieve the set of English synsets in PWN as the candidates using the matching technique. Then, the semantic hierarchy of Arabic is used in parallel with the PWN semantic hierarchy to check whether Arabic features can be extracted for each English synset in the candidate set. For each English synset, Arabic features can be extracted when the parents of this synset have identical Arabic synsets. Consequently, the mapping can be established if all English synsets of an article have at least one parent in the Arabic semantic hierarchy, otherwise the article is ignored in the mapping and extension without further processing. In addition to using Arabic features in measuring semantic similarity, this condition for having parents is also intended to ensure that a potential inserted synset is not isolated in the Arabic hierarchy.

The final output of candidate extraction for an article is the positions in the Arabic semantic hierarchy that align to the positions of candidates in the English hierarchy. To illustrate this step, suppose the given pair [rectifier, ] includes the titles of the article in Arabic and English (Figure 10). The English lemma rectifier is used only to extract two candidates with their parents (person and electrical device) in the semantic taxonomy of PWN. Because these two parents have identical nodes ( and ) in the Arabic semantic hierarchy, there are two positions in AWN (Arabic candidates) for the article, as seen in Figure 10. The semantic similarity between the article and each position candidate is computed by integrating the bilingual and only article-based (monolingual) measures. In the article-based measure, the position candidate is represented as the bag-of-words set from the extracted gloss of this position (hierarchal relations) and the non-hierarchal relations of its parents and siblings. Finally, the article is classified based on the similarity scores as inserted or not-inserted.

Fig. 10.

Example of detecting the positions of an article in AWN.

8.2 Pruning rules

For each article that has been classified as inserted, one node is inserted temporarily in the Arabic semantic hierarchy according to the detected position. To improve the reliability, we introduced two rules to evaluate automatically the inserted nodes based on the hierarchical relations. The rules are applied to the inserted nodes to decide whether the article can be added to AWN as a synset. The first rule depends on the inheritance terming relation between the parent and its child in the hierarchy. The terming relation is when the child concept is labeled by extending the term of its parent concept. For the inserted node, the node of the article has the terming relation if and only if the title of this article is a multiword expression starting with the synonym word of the parent of the inserted position. The following examples are concepts that have the terming relation:

AWP can be constructed in a hierarchical way as an acyclic directed graph using the category features if the relation between a category and its subcategories is identical to the hyponymy semantic relation in WordNet. In this graph, the category ‘Contents’ is the superordinate of all other categories (the root of the hierarchy). The AWP category graph consists of 38,018 nodes and 95,118 edges after removing the cycles using the colored depth-first-search technique (Zesch and Gurevych Reference Zesch and Gurevych2010). Articles are added to this graph as leaf nodes based on the article–category links (links from an article to its categories). This graph is considered to be the semantic hierarchy of AWP.

The second rule depends on the compatibility hierarchal relation that can be captured from the semantic hierarchies of AWN and AWP based on the full mapping results. For an article with its target position node, the related synsets are retrieved from the semantic hierarchy of AWN with their corresponding AWP articles that have been linked using the mapping method. Formally, let y be the position node for the article x, the related synsets (RS) of y with the corresponding articles (RA) are defined as the semantic features of the node in the two sources as: RS(y) = {y ₁, y ₂, . . ., y_m }andRA(y) = {x ₁, x ₂, . . ., x_m }, where x_i is the identical article of the synset y_i on the basis of the full mapping. The related synsets of a node are extracted from the hierarchical relations with respect to the generality and specify notions (as in the extraction of glosses). Based on the semantic hierarchies of AWN and AWP, the relation between x_i and y_i is defined as the distance between the length of the shortest path from x to x_i in the AWP graph and the length of the shortest path from y to y_i in the AWN graph. Therefore, the compatibility between x and y can be determined by the distance vector of the relations among the related synsets with their identical articles as

$$\begin{equation*} DV\left( {x,y} \right) = \left| {Lx_1 - Ly_1 } \right|,\left| {Lx_2 - Ly_2 } \right|, \ldots .,\left| {Lx_m - Ly_m } \right| \end{equation*}$$

where Lx_i /Ly_i is the length of the shortest path between x/y and x_i /y_i in the AWP/AWN semantic graph. Based on this vector, an article is considered compatible to the inserted node if all elements in the distance vector range between 0 and 3.