2.1 Logic Form identification approaches

The usual way of obtaining the LF of the text is by applying rules derived from the syntactical structure of the phrase. From this point of view, such rules are based on syntax. These rules have two parts, the head of the rule and the tail:

• • The head of the rule is a syntactical pattern. For example, the head [adjective name] matches with a specific rule in order to translate such syntactical construction to an equivalent LF.

• • The tail of the rule is the transformation from the recognized pattern to a (maybe incomplete) equivalent logical form. For example, if [white_N snow_J] matches with [adjective name] rule then the system recognizes such a matching and it generates the LF [white_J(x ₁ ) snow_N(x ₁ )].

Some rules are shown in Table 1. Other rules and their descriptions will be shown later in Section 4.

Table 1.

Some logic form transformations

There are two common approaches to obtaining such rules: those based on constituency trees and those based on dependency trees. The two approaches are based on rules, but systems using dependency trees obtain rules more easily (Wenner Reference Wenner2007; Agerri and Peñas Reference Agerri, Peñas and Gelbukh2010) than those based on constituency trees (Rus Reference Rus2002). Both approaches require that each syntactic form has a rule developed by a human expert. The number of necessary rules does not only depend on the desired recall but on the complexity of the language. For instance, a syntactically relatively simple language like English needs only ten rules to cover 90% of the WordNet glosses (Moldovan and Rus Reference Moldovan and Rus2001).

In addition, a system based exclusively on rules requires a new rule for every exception. For example, the elliptical subject is usually labeled by the parser, but if the syntactic analyzer does not tag it, it must be considered as an exception. In our method, the elliptical subject is detected by applying some simple heuristics, like the absence of a noun phrase to the left of the verb (in active phrases). Note that this heuristic is not infallible, but it works because it requires, first, to mark a false positive that, first, the syntactic parser should wrongly label a subject and, second, that the subject should be elliptic. This is possible, but rare.

Note that these systems require precise syntactic analyzers, and they are not always available for every language. For such cases, shallow parsing is applied instead of syntactic parsing. If grammatical roles are not available then it increases the ambiguity of the rules, especially for languages with a more complex syntax than English. Whereas, in English the different parts of a phrase always have a fixed position that depends on its type, in Spanish this is not true. Identification in Spanish of the arguments of the verb is not a suitable task to be resolved by using rules, because word order is not fixed and it can be influenced by syntactic and pragmatic factors, including verb type, animacy, weight, and information structure (Alfaraz Reference Alfaraz2012). Basic word order in Spanish is Subject-Verb-Object SVO (Suñer Reference Suñer1982) but other word orders are possible. For example, the phrase ‘Mary wants a dog’ can be translated to Spanish as any of the following active sentences:

• • Mary quiere un perro (SVO order).

• • Un perro Mary quiere (OSV order).

• • Un perro quiere Mary (OVS order).

• • Quiere Mary un perro (VSO order).

• • Quiere un perro Mary (VOS orderFootnote ¹ ).

• • Quiere un perro (VO order, the subject is omitted if the context of the phrase gives information about who wants the dog).

• • Un perro quiere (OV order, the subject is omitted).

In order to detect the subject constituent Rus (Reference Rus2002) proposes the rule ‘S → NP VP NP’ but this rule only works correctly in the first translation. Thus, LFi for the classification of Spanish phrase constituents requires not only syntactic information encoded into rules but also morphological and semantic information. Note that state-of-the-art syntactic parsers give grammatical functions of the constituents, which eases the derivation of the LF of the sentence.

The most usual way to obtain the LF of a text is by using a set of manuscript rules. The alternative are systems based on dependency trees. Ahn et al. (Reference Ahn, Fissaha, Jijkoun, De Rijke, Mihalcea and Edmonds2004) traverses the dependency structure of the phrase, turning part-of-speech tagged lexical items into LF predicates, creating referential variables for nouns and verbs, and using dependency labels to order the arguments at each node. A similar approach is proposed in Anthony and Patrick (Reference Anthony and Patrick2004).

Systems based on dependency trees (Figure 1) use a representation more similar to the LFi structure than those using constituency trees. It is relatively straightforward to turn dependency structures into LFs (Agerri and Peñas Reference Agerri, Peñas and Gelbukh2010) but even in such cases it is required to encode additional knowledge by using rules.

Fig. 1.

An example of a dependency tree (source: mate-tools using the approach described in Johansson and Nugues (Reference Johansson and Nugues2008)).

3.1 Logic forms identification applications

The trace of the FOL proofs provides a basis for generating humanly comprehensible interpretation justifications. Thus, LFi is applied in several NLP tasks where the understanding of the text is relevant. A few examples are listed below.

Moldovan et al. (Reference Moldovan, Clark, Harabagiu and Hodges2007) implements a question answering system which integrates several sources of knowledge such as WordNet, Suggested Upper Merged Ontology (SUMO) and handmade rules about world knowledge. The whole of the knowledge is managed by using COGEX, a logic prover based on LF. Todorova (Reference Todorova2009) proposes a QA system based on LF and the codification of common sense knowledge applied to simple domains.

COGEX has been used for other NLP tasks such as textual entailment (Fowler et al. Reference Fowler, Hauser, Hodges, Niles, Novischi and Stephan2005) and automatic answer validation (Tatu et al. Reference Tatu, Iles and Moldovan2007). In a similar way, Ferrández et al. (Reference Ferrández, Terol, Muñoz, Martínez-Barco, Palomar, Peters, Clough, Gey, Karlgren, Magnini, Oard, Rijke and Stempfhuber2007) applies LF to both tasks by inferring the associated LF of a sentence through applying NLP rules to the dependency tree. However, the LF used for COGEX are obtained mainly by using a rule-based approach.

Muñoz-Terol, Martínez-Barco and Palomar (Reference Muñoz-Terol, Martínez-Barco and Palomar2007) uses LFi in order to modify the weight of the topics terms in a cross-lingual speech retrieval system. The document collections are transcriptions of spoken news and the system aims to find news about several topics. Each topic term in the LF can modify its term weight according to its predicate type and the relationships between all the predicates in the LF of the topic. For example, LFi helps to identify grammatical utterances that have a circumstantial behavior in the sentence, so the nouns in the utterance have a greater weight.

All these previous systems use logical forms of the same type as our proposal.

LG-Soar (Tustison Reference Tustison2004) is a system capable of parsing eligibility criteria in clinical trials and outputting a semantic representation using LFi. This software aims to help information extraction tasks.

Watson (Ferrucci et al. Reference Ferrucci, Brown, Chu-Carroll, Fan, Gondek, Kalyanpur, Lally, Murdock, Nyberg, Prager, Schlaefer and Welty2010; McCord, Murdock and Boguraev Reference McCord, Murdock and Boguraev2012) is a computer system that competed at the human champion level in real time on the American TV Quiz show, Jeopardy! The extent of the challenge included fielding a real-time automatic contestant on the show, not merely a laboratory exercise. Watson uses logical form generation for knowledge representation and reasoning related tasks.

Nguyen and Shimazu (Reference Nguyen and Shimazu2014) and Nakamura et al. (Reference Nakamura, Kimura, Pham, Nguyen and Shimazu2010) have developed a system for translating legal sentences written in Japanese, English and Vietnamese into logical formulae. The system basically converts words and phrases in a target sentence into predicates in a logical formula, which is used as a kind of interlingua.

Finally, there are several initiatives used in order to codify ontologies such as WordNet to LF (Harabagiu, Miller and Moldovan Reference Harabagiu, Miller and Moldovan1999; Agerri and Peñas Reference Agerri, Peñas and Gelbukh2010). Thus, these resources are available for reasoners based on LF such as COGEX, whenever the LF formalism is the same for the ontology and the reasoner.

3.2 Semantic role labeling

The aim of the SRL task is the detection and classification of the semantic relation between the verb and the other constituents of the phrase, with relations such as agent, patient, instrument, . . .Footnote ² (see Table 2).

Table 2.

An example. SRL for the verb ‘to provide’ in the sentence ‘the Earth provides the food we eat every day’

Gildea and Jurafsky (Reference Gildea and Jurafsky2002) distinguish between role detection and classification, since a real system that tries to identify the semantic role of a verb must disambiguate it, because the sense of the verb determines these roles. In a simple model, it is given the role detection, so the system has only to define the role type. They propose a model based on learning with six properties per role, and these properties are used to relate the syntax and the semantic of each role.

SRL has received much attention (Màrquez et al. Reference Màrquez, Carreras, Litkowski and Stevenson2008; Palmer et al. Reference Palmer, Gildea and Xue2010), partly because of its potential to improve applications that require broad coverage semantic processing. Thus, the interest in SRL has been invigorated for several years by the Conference on Natural Language Learning (CoNLL) and the SemEval competitions.

4.1 Use case

This section shows an example of the global running of the proposed mixed method, using the resources described previously. The Spanish example is: ‘Carlos lee los titulares deportivos de la Estrella Digital’ (‘Carlos reads the Estrella Digital sports headers’). The expected LF is:

Estrella_Digital is marked as noun because the dependency parser detects this expression as a named entity. Thus, the algorithm is applied as follows:

1. (1) The first step requires us to obtain the syntactic tree of the sentence (Figure 2).

2. (2) Then, each head wordFootnote ³ of the sentence and its POS tag has an associated predicate. For example, Carlos_N(x ₁ ) or titular_N(x2).

3. (3) For each noun phrase, intra-phrase transformation rules are applied at the same level. In this example, it is detected that deportivo is an adjective of titular, so they share the same argument (Table 3 Footnote ⁴ ).

4. (4) The LF obtained are spread backwardly, and the intra-phrase transformation rules are applied recursively (Table 4).

5. (5) A feature vector is extracted for each noun phrase depending directly of the verb. An example of such a feature vector is depicted in Table 5.

6. (6) Finally, the classifier is applied for each feature vector obtained in the previous step in order to provide the arguments of the verb predicate.

Fig. 2.

Dependency tree of the phrase ‘Carlos lee los titulares deportivos de la Estrella Digital’ (‘Carlos reads the Estrella Digital sports headers’) generated by FreeLing 3.1.

Table 3.

Example of a rule applied at the same level

Table 4.

Example of a rule applied vertically

Table 5.

Example of a vector of features for the noun phrase ‘titulares deportivos de la Estrella Digital’ (‘Estrella Digital sports headers’)

At the end of steps 3 and 4, we have obtained the LF of non-verbal constituents and an incomplete LF of the verb predicate (it has no known arguments). Step 5 and 6 provide the arguments of the verb predicate in order to complete the LF of the sentence. A global view of the process is depicted in Figure 3.

Fig. 3.

Global view of the pipeline followed by the use case

4.2 LFi based on rules applied to non-verbal constituents

First, a set of hand-coded transformation rules are recursively used to derive the LF of each non-verbal constituent according to the type. For verbs, the LF is partially derived since the arguments of the corresponding predicate are unknown. Since, we are using transformation rules for non-verbal constituents we have only written what (Rus Reference Rus2002) calls intra-phrase transformation rules. They generate predicates for every noun, verb, adjective or adverb. They also assign the variables that describe dependencies local to the phrase. The way that these intra-phrase transformation rules are obtained is depicted below. A set of candidates for verb arguments are obtained at the end of this recursive process. We use an approach based on rules because the syntax of the Spanish constituents is roughly similar to that of the English ones, and systems based on rules for English exceed 90% precision (Rus Reference Rus2004b). In addition, the application of these rules is usually not an ambiguous task, and these constituents of phrases are relatively simple. Examples of such transformation rules are depicted in Tables 3 and 4.

4.3 Derivation of the arguments of the verb predicate using a SRL classifier

Rus (Reference Rus2002) proposes the so-called inter-phrase transformation rules in order to provide the arguments of the verb predicates. Given the ambiguity of Spanish syntax we propose the replacement of such rules by using an approach similar to SRL: each constituent is classified as a semantic role, but instead of finding out the role, the system infers which is the argument with the logical form of the verb. The way that the system makes this inference is similar to the solution proposed by Gildea and Jurafsky (Reference Gildea and Jurafsky2002) for semantic role detection by means of machine learning algorithms. We apply this solution adapted to Spanish with the available resources, which are depicted in Section 5. Seven features are extracted from the constituent by using a dependency tree parser and they are used to train a classifier:

1. (1) Phrase type could be a noun phrase (NP), a prepositional phrase (PP), an adjective phrase(ADJP) or an adverbial phrase(ADVP).

2. (2) Main category has only two values, subjects (S), and verb phrases(VP), corresponding to subjects and objects of verbs, respectively.

3. (3) Syntactic tree path: determines the syntactic tree path from the constituent to the verb.

4. (4) Position of the constituent related to the verb (left or right).

5. (5) Verb voice: active or passive.

6. (6) Main word of the constituent. This word is marked by the parser according with some heuristic rules described in (Collins Reference Collins1999). In Section 5 we describe the parser used.

7. (7) The joint of the Part Of Speech (POS) of each constituent word. For example, the phrase ‘el año pasado’ takes the value {DNJ}.

The first six features are those used by Gildea and Jurafsky (Reference Gildea and Jurafsky2002). The POS joint is used in some SRL systems (Punyakanok et al. Reference Punyakanok, Roth, Yih and Zimak2004; Carreras, Màrquez and Chrupała Reference Carreras, Màrquez and Chrupała2004b; Palmer et al. Reference Palmer, Gildea and Xue2010).

4.4 Comparing the mixed method with similar studies

5.1 Experimentation framework

In our experiments, we have used the linguistic analysis software FreeLing 3.1 (Carreras et al. Reference Carreras, Chao, Padró and Padró2004a; Padró and Stanilovsky Reference Padró and Stanilovsky2013), the semantic annotated corpus AnCora-ES (Taulé et al. 2008) and the machine learning software TimBL (Daelemans et al. Reference Daelemans, Zavrel, van der Sloot and van den Bosch2004). The set of phrases and their LF, used to evaluate the complete system, were translated from the LFi task of Senseval-3 (Rus Reference Rus2004b):

• • FreeLing is a freely available linguistic analysis software package. Supported languages are Spanish, Catalan, French, Galician, Italian, English, Russian, Portuguese, Welsh and Asturian. It implements a POS tagger, a morphological analyzer and a rule-based dependency parser. The syntactic rules and the features extracted from each constituent are derived from the syntactic tree that FreeLing returns. Syntactic analyzers such as Collins (Reference Collins1999) provide the head (the nucleus of the phrase, not a complement). Since FreeLing provides superficial syntactic analysis, syntactic trees are flat, so the number of necessary syntactic rules is higher than in the case of deep analysis. FreeLing has already been used in several industrial projects (Padró and Stanilovsky Reference Padró and Stanilovsky2013).

• • AnCora-ES is a Spanish corpus with syntactic, semantic and pragmatic annotations. As well as the syntactic annotation, it carried out a semantic annotation by means of the synsets of the different wordnets built for this language, as well as an annotation of anaphoric and elliptic elements just as the co-reference. The corpus extension is 100,000 words. The AnCora corpus also is available for Catalan, AnCora-CA is a corpus with similar coverage to AnCora-ES for Catalan. The annotated data is necessary in order for a machine learning system to learn which argument of the logical form of the verb is mapped to the given constituent. For this purpose, each syntactic function has an argument of the logical form of the associated verb, as we can see in Table 6. On the other hand,this syntactic information is provided by AnCora-ES but shallow parsers does not provide such kind of information. Note that our proposal is well suited to be applied by using just shallow parsing, if dependency or constituency parsing is no available.

• • TiMBL is an open source software package implementing several learning algorithms, among them: IB1-IG, an implementation of k-nearest neighbor classification with feature weighting suitable for symbolic feature spaces, and IGTree, a decision-tree approximation of IB1-IG. TiMBL can work with alphanumeric features and performs multi-class classification, so it is useful for NLP tasks. Memory-based language processing is based on the idea that NLP problems can be solved by storing annotated examples of the problem in their literal form in memory, and applying similarity-based reasoning to these examples in order to solve new ones (Daelemans and van den Bosch Reference Daelemans and van den Bosch2005). For the experimentation we use the TiMBL 6.4 version. For the past decade, TiMBL has mostly been used in NLP as a machine learning classifier component including SRL for Spanish and Catalan (Morante and Busser Reference Morante and Busser2007).

  We parameterized the algorithm empirically using the training testset: we found that IB1-IG obtains the best results. Information gain weighting was used to compute relevance. The distance metric is the cosine. We set the confidence threshold to 0.86 because we found that 90% of training examples were correctly classified and the coverage was 94.2% (F ₁ = 0.92).

Table 6.

Mapping between syntactic functions annotated in the AnCora-ES corpus and the corresponding argument position in the LF of the verb

5.2 Experiments and results

In order to evaluate the experiments we considered four performance measures (Rus Reference Rus2004b):

• • Precision at argument level is the number of correctly identified arguments divided by the number of all identified arguments. Recall at argument level is the number of correctly identified arguments divided by the number of arguments that were supposed to be identified.

• • Precision at predicate level is the number of correctly and fully identified predicates (with ALL arguments correctly identified) divided by the number of all attempted predicates. Recall at predicate level is the number of correctly and fully identified predicates (with ALL arguments correctly identified) divided by the number of all predicates that were supposed to be identified.

We evaluated both the classifier and the mixed method in several ways:

• • The performance of identification of verb arguments using SRL, as depicted in the following section.

• • The global performance at predicate and argument level.

• • The performance of the identification of verb arguments using a rule-based approach instead of the proposed mixed method.

  As is described above, these rules are based on previous studies (Rus Reference Rus2002, Reference Rus2004b). Richer rules could be developed, for instance to improve the treatment of ambiguity, but this would be more complex work that we plan to perform in the near future.