2.1 Generation input

In principle, an abstract meaning representation is provided as input to a sentence realiser; in practice, the actual input varies widely from one system to the next because of the variability of representations and linguistic formalisms that a generation system is designed for. On the one hand, the input can be drawn from different linguistic levels of representation, accordingly varying in the degree of ‘abstraction’. Certain systems, e.g. RealPro (Lavoie and Rambow Reference Lavoie and Rambow1997), adopt very low level of abstraction, including syntactic categories, lexical specifications, definiteness and tense decisions, etc. in the input specification; some systems, e.g. Penman/KPML (Bateman Reference Bateman1997), adopt high level of abstraction, including pure semantic roles, such as actor, temporal-locating, etc.; some, for example Nitrogen/HALogen (Langkilde Reference Langkilde2002), take a wide variety of input ranging from rather abstract semantic representation to more fully specified syntactic representation. On the other hand, the expression of the semantic content may take different forms, such as Sentence Plan Language (SPL) specifications adopted by Penman and Nitrogen/HALogen, functional descriptions as inputs to FUF/SURGE (Elhadad and Robin Reference Elhadad and Robin1996), typed feature structures representing logical forms for LinGO/LKB (Carroll et al. Reference Carroll, Copestake, Flickinger and Poznanski1999) and the cognate treebank-based HPSG generator (Nakanishi et al. Reference Nakanishi, Nakanishi and Tsujii2005), non-hierarchical conceptual graphs as semantic input specifications used by the Protector system (Nicolov and Mellish Reference Nicolov and Mellish2000), semantic dependency graphs as used in Amalgam (Corston-Oliver et al. Reference Corston-Oliver, Gamon, Ringger and Moore2002) and OpenCCG (White Reference White2004) and many others. Even though different input representations are specific with respect to their underlying linguistic theories, from an engineering perspective these can be characterised by a common representation in terms of attribute value matrices (AVMs), viz. a recursive structure consisting of attribute names and their values encoding linguistic information at various levels of abstraction. These AVMs, in turn, can be converted to dependency trees or graphs in computational implementations.

In this paper, we present a general-purpose sentence realiser for generic dependency-based input representations. We instantiate the framework using two types of dependency representations popular in the CL/NLP and linguistics literature: on the one hand we use the slightly more abstract Lexical-Functional Grammar (LFG)-based f-structure representations (Kaplan and Bresnan Reference Kaplan and Bresnan1982; Dalrymple et al. Reference Dalrymple, Kaplan, Maxwell and Zaenen1995); on the other hand we use the more ‘surfacy’ dependency representations familiar from the CoNLL (2006–2009) shared tasks on dependency parsing. Below, we first concentrate on the LFG-based representations and turn to the CoNLL-style dependencies in Section 4.2.2.

LFG is a constraint-based grammar formalism, which postulates (minimally) two levels of representation: c(onstituent)-structure and f(unctional)-structure. C-structures are conventional phrase structure trees encoding linear order, hierarchical groupings and syntactic categories of constituents, representing surface syntactic configurations for a particular language. F-structures encode more abstract functional relations, like subj(ect), obj(ect) and adjunct etc., providing deep syntactic analyses across languages. An f-structure is represented in the form of an AVM or a labelled dependency graph, as illustrated in Figure 1, and consists of the following types of attributes:

• • pred is the predicate/head of a local f-structure. The value of pred is a lemma associated with a subcategorisation frame reflecting the unique semantic contribution of each word in the sentence.

• • The atomic-valued features describe linguistic properties of the predicates, such as tense, mood, aspect, pers, num, etc.

• • Grammatical Functions (GFs) indicate the functional relationship between a predicate and the sub-f-structures/dependents. GFs come in the following two different types:

  • – Arguments are subcategorised for by the local predicate, such as subj(ect), obj(ect), obl(ique) etc., and thus only occur once in each local f-structure.

  • – Modifiers, like adjunct, coord(inate), etc., are not subcategorised for by the predicate, and can occur any number of times in a local f-structure.

An LFG-based generator takes as input an f-structure and determines the surface strings that correspond to the given f-structure (Kaplan and Wedekind Reference Kaplan and Wedekind2000).

Fig. 1.

An example of the f-structure and the equivalent dependency tree for the sentence Newspaper reports said the young Polish athlete ran fast.

2.2 Generation from F-structures

A number of sentence realisers have been built based on LFG using either symbolic or statistical approaches. The ParGram/XLE system (Crouch et al. Reference Crouch, Dalrymple, Kaplan, King, Maxwell and Newman2007) comes with a fully fledged unification-based generator and multilingual wide-coverage LFG grammars consisting of handcrafted syntactic rules and lexical entries. Based on the XLE resources, Cahill et al. (Reference Cahill, Forst and Rohrer2007) describe a two-stage surface realiser for German, which first produces a generation space with the symbolic XLE generator, and then selects the best realisation using a log-linear ranker. Cahill and van Genabith (Reference Cahill and van Genabith2006) and Hogan et al. (Reference Hogan, Cafferkey, Cahill and van Genabith2007) present conditional PCFG generation models implemented in chart-style generators for English. Both systems are based on wide-coverage, probabilistic LFG approximations automatically acquired from the Penn-II treebank. A common characteristic of previous LFG-based generators is the use of bi-directional (i.e. reversible) grammar rules (either handcrafted or treebank-based), where the generation process can be viewed as the reverse of parsing. In a sense, generation is driven by an input f-structure representation, but proceeds through functionally annotated context-free grammar (CFG or PCFG) rules and a chart generator to construct the corresponding c-structure, from which final surface strings can be read. By contrast, this paper presents a simple dependency- and n-gram-based, probabilistic generation model, which cuts out the middle-man (i.e. the CFG-component) from previous approaches.

In accordance with a generic dependency formalism, an LFG f-structure is not linearly ordered, i.e. it does not carry information on to the relative surface order of the predicates in an f-structure. Furthermore, predicates in f-structure are lemmas (rather than surface forms), with morphological and semantic information relevant to surface realisation encoded as atomic-valued features. In order to generate a well-formed string from an f-structure, first and foremost we need to put in correct order the set of words corresponding to the predicate lemmas in the f-structure. This is typically performed by a word-based n-gram model operating on the yields of the trees compatible with the input f-structure. As mentioned above, the traditional language model in the (over-)generate-and-select approach is prohibitively expensive to apply, because vast numbers of alternative realisations are produced and word n-gram-based language model probabilities are calculated for each of them. For example, Langkilde (Reference Langkilde2000) reports that Nitrogen deals with trillions of alternatives in the course of generation. To reduce the number of possible realisations, we break down the generation space, as defined by the given f-structure, into each layer of embedding within the f-structure: that is to say, the realiser selects the most likely substring for each sub-f-structure, and concatenates the substrings into a complete sentence by recursively traversing and linearising each level of embedding in the f-structure, respecting linearisations of the already established sub-f-structures, until it reaches the outermost f-structure. This reduction in complexity depends on the simplifying assumption that the input f-structure consists of projective dependencies onlyFootnote ⁴ (as in the example in Figure 1), such that each sub-f-structure can be realised independently of other sub-f-structures (other than, of course, the sub-f-structures that are part of the local sub-f-structure), as all dependencies are with respect to local heads and are single-headed. A similar scenario is obtained in dependency parsing, where most practical dependency parsers (McDonald et al. Reference McDonald, Crammer and Pereira2005; Nivre Reference Nivre2006) are restricted to projective structures. This is partially due to the fact that a majority of sentences in most languages can be represented by projective dependencies (Mel'čuk Reference Mel'čuk1988). In the case of English, Surdeanu et al. (Reference Surdeanu, Johansson, Meyers, Màrquez and Nivre2008) report that in the complete corpus used in the CoNLL-2008 shared task on dependency parsing, only 0.4 per cent of dependencies are non-projective.

While our dependency-based realiser assumes projectivity, most theoretical formulations of dependency grammar (and in particular LFG) recognise the need for non-projective representations of certain linguistic constructions, e.g. long-distance dependencies (LDDs) or (more generally) non-local dependencies (NLDs). Figure 2 exemplifies an f-structure analysis of a sentence discussed in Carroll et al. (Reference Carroll, Copestake, Flickinger and Poznanski1999), where the non-local dependency is represented as a re-entrancy or coindexation in the f-structure and the dependency graph equivalent. The re-entrancy in the f-structure indicates that the word how is associated with two grammatical functions as well as two heads: it bears the discourse function focus of the matrix clause, and at the same time also functions as an adjunct modifying the predicate run in the embedded clause. Alternatively, an account of NLDs in earlier transformational grammar involves movement, viz. how is moved from the position at which it should be interpreted semantically, often known as a ‘gap’ or ‘trace’, to the location where it is realised, which is called a ‘filler’ or ‘antecedent’. Although varying in analysis and terminology, a common property of these accounts is that the interrogative word how is interpreted to relate to two distinct sites – a semantic site (the ‘trace’) and a surface site (the ‘antecedent’). The grammatical function adjunct associated with the semantic site is non-projective because it is governed/dominated by the head verb run in the embedded dependency structure, but in the surface string how is realised at the initial position of sentence outside the substring of the embedded clause the athlete ran. By contrast, the discourse function focus associated with the surface site is locally dependent on the matrix verb and forms a projective dependency. In our approach, we convert non-projective dependencies into projective dependencies: for each sentence realisation we need to maintain the surface site (i.e. the antecedent discourse function) of the NLD while removing the semantic site (i.e. the trace), effectively transforming dependency graphs into dependency trees. Therefore, conversion from non-projective to projective representations involves (i) identifying the non-local dependencies within the f-structure; (ii) removing the ‘trace’ (and associated GF) and keeping the ‘antecedent’, or, in other words, moving the trace up towards the antecedent site (as indicated by the dashed curves in Figure 2). In our LFG implementation for English, three grammaticalised discourse functions, focus, topic and topic_rel, characterise NLDs arising in different types of questions, relative clauses and topicalisations, respectively. As a consequence, the conversion is directly triggered by the three discourse functions. The identification of non-local dependencies in other semantic representations or for other languages may require more elaborate linguistically motivated analyses, for example, Gamon et al. (Reference Gamon, Ringger, Zhang, Moore and Corston-Oliver2002) address extraposition of relative clauses in German.

Fig. 2.

LDD representations in f-structure and dependency tree.

3.1 Dependency-based n-gram models

Given an f-structure, the primary task for surface realisation is to determine the linear order of words or phrases that the f-structure generates. The n-gram models are easy to build and use in practice, hence have become the most popular tool for word ordering in a variety of NLP tasks. However, a traditional language model relying exclusively on lexical information between adjacent words is unable to capture relationships between words spanning a longer distance, and usually performs worse on a clause level than on a phrase level (Filippova and Strube Reference Filippova and Strube2009). Furthermore, from a linguistic point of view, grammatical relations or semantic roles to a large extent govern the word order of sentences in languages such as English and Chinese. Based on these observations, in our model we perform linearisation at each local f-structure level by means of the DN-gram models, i.e. n-gram models over dependency (D) labels or grammatical functions (GFs), which describe languages in terms of functions equally on the phrase level and the clause level. For the example in Figure 1, the GF sequence det ≺ adjunct ≺ pred realises the noun phrase the young polish athlete for the embedded f-structure f ₅; in the same manner but at the embedding f-structure f ₃, the GF sequence subj ≺ pred ≺ adjunct produces the clause the young polish athlete ran fast, reusing and leaving intact the linearisation produced for the embedded f ₅.

Formally, in our most basic model, for a local f-structure f_i containing m GFs (including pred), generating a surface string S ₁^m = s ₁. . .s_m expressed by f_i is equivalent to linearising all the GFs present at f_i. The DN-gram model calculates probabilities for all permutations GF ₁^m = GF ₁. . .GF_m, and searches for the best surface sequence that maximises the probability P(S ₁^m) in terms of maximising P(GF ₁^m). Applying the chain rule and the Markov assumption, the probability of the surface realisation is computed according to (1).

(1)

The basic DN-gram model over bare GFs is not a good probability estimator as it only makes use of a few dozen grammatical function roles. For example, there is no way to capture the difference between two adjunct modifiers according to the labels of the two GFs. In order to facilitate better decisions, we extend the basic model to a number of more complex DN-gram models incorporating contextual and fine-grained lexical information. Equation (2) describes a model that additionally conditions GF sequences on the parent GF Parent_i of the local f-structure f_i being linearised and (3) conditions the n-gram probability on the headword (or predicate) Head_i governing f_i,

(2)

(3)

Alternatively, we can also integrate additional features with the GFs being ordered. Equation (4) stands for a featured model, where Feat_k stands for the function label GF_k augmented with the set of atomic-valued features that specify the sub-f-structure assigned to the attribute GF_k. In contrast with common atomic features included in the feature model, e.g. tense, mood and num, which have a closed (i.e. finite) set of values, the value set of pred is given by a (theoretically) unlimited number of lemmas corresponding to lexical entries. We separate out pred from other atomic-valued features and develop a fully lexicalised DN-gram model in (5), where Lex_k stands for GF_k augmented with its predicate lemma,

(4)

(5)

Table 1 exemplifies DN-gram models in (1) to (5) for realising the string the young polish athlete ran fast represented by f-structure f₃ in Figure 1. Other features from a larger context or at different granularity have the potential to be included in the DN-gram models. In this paper, however, we focus on the general methodology without a full exploration of all possible feature options.

Table 1.

Examples of different DN-gram models for f₃ in Figure 1

In building the DN-gram models, we apply Good-Turing smoothing when calculating probabilities for unlexicalised (gf, parent and feat) DN-grams and interpolated Kneser–Ney smoothing for lexicalised (head and lex) DN-grams.

The individual models described from (1) to (5) can, in turn, be combined in different ways. As lexicalised models are likely to suffer from severe data sparseness, we create combined models by interpolating the individual DN-gram models incrementally (rather than by using chain rules). Klakow (Reference Klakow1998) shows that, in general, log-linear interpolation (LLI) performs better than simple linear interpolation when combining n-gram models of different kinds. Accordingly, we implement LLIs to combine the estimates from different DN-gram models as follows:Footnote ⁵

(6)

Here λ_i is the interpolation weight that adjusts the contribution of the component models P_i. In contrast to simple linear interpolation, no explicit constraints appear on the LLI weights. To estimate optimal sets of interpolation weights, we employ the minimum error rate training (MERT) algorithm (Och Reference Och2003), which has been widely adopted in statistical machine translation systems.

3.2 The realisation algorithm

In addition to word ordering, surface realisation also involves inserting function words and punctuation marks to produce a grammatical sentence.

In our LFG f-structure representations, function words are encoded in atomic-valued features at the local head level. From a linguistic perspective, the number of function words is limited and most of them occupy specific positions in sentences. For instance, the verbal particle to precedes the base form of a verb to form an infinitive in English, and the interrogative particle appears at the end of a question in Chinese etc. In our generation system, a small rule-based component captures heuristics about the use of function words: function words are triggered by a limited set of atomic-valued features and inserted in one of the four possible positions with respect to a local f-structure f_i: (i) before the local predicate pred_i governing f_i; (ii) after the local predicate pred_i; (iii) before the complete string s_i that is yield of f_i; and (iv) after the string s_i. For example, the f-structure attribute to_inf indicates that the local predicate verb is infinitive and hence a particle to should be placed before it. As illustrated by sentence (2) and the f-structure that produces (2), to is generated immediately before the string position in (2) that realises the local predicate feel from f₃. Similarly, the subordinating conjunction that is used to introduce a subordinate clause (the stock market . . .), as f ₄ contains the feature-value pair that = +.

As a language-specific syntactic element, unlike in the CoNLL dependency representations (discussed in Section 4.2.2), punctuation is not conventionally represented at the level of LFG f-structures. We present a maximum entropy model to recover punctuation marks for unpunctuated realisation strings in a separate post-processing component (Guo, Wang and van Genabith Reference Guo, Wang and van Genabith2010). However, this paper concentrates more on the overall dependency-based realisation scheme, and, for simplicity (and following the CoNLL dependency representations), we record all punctuation marks in the f-structure representations as the value of a punc attribute and generate them in the just same way as any other GF.

In order to generate orthographically and semantically correct sentences, morphological alternation has to be determined for inflecting languages, such as English.Footnote ⁶ Inflection indicates grammatical properties, such as number, person, case, or tense, which are encoded as atomic-valued features in f-structures. We call a lexeme together with its morphological features a lexical macro. A lexical macro, as exemplified in Table 2, maps a lexeme to the appropriate word form in the surface realisation. We learn the set of lexical macros automatically from our training corpora consisting of f-structures and the corresponding surface strings.

Table 2.

Examples of lexical macros

In summary, given an input f-structure F consisting of projective dependencies, the generation algorithm recursively traverses F in a bottom-up fashion, and at each level of embedding with (sub-) f-structure f_i:

1. (1) Instantiates the local predicate pred_i at f_i and performs morphological inflections if necessary.

2. (2) Calculates DN-gram probabilities of possible GF permutations licensed by f_i.

3. (3) Finds the most probable GF sequence among all possibilities by Viterbi search.

4. (4) Generates the surface string s_i according to the best GF sequence as a realisation of f_i.

5. (5) Inserts function words in appropriate positions of s_i by heuristic rules and propagates s_i up to the next outer embedding f-structure.

4.1 Experiment design

Similar to many others (Corston-Oliver et al. Reference Corston-Oliver, Gamon, Ringger and Moore2002; Langkilde Reference Langkilde2002; Callaway Reference Callaway2003), we use a pre-processing procedure to convert surface syntax trees, viz. constituent-structure trees hand coded in a treebank or automatically produced by a parser, to the semantic dependency representations expected by the surface realiser. In our experiments, the English Penn Treebank II (PTB-II) (Marcus, Santorini and Marcinkiewicz Reference Marcus, Santorini and Marcinkiewicz1993) and the Chinese Penn Treebank (CTB) (Xue et al. Reference Xue, Xia, Chiou and Palmer2005) are used as source data for testing and evaluation. We follow the methodologies developed in Cahill et al. (Reference Cahill, Burke, O'Donovan, van Genabith and Way2004) and Guo, van Genabith and Wang (Reference Guo, van Genabith and Wang2007) to automatically convert the phrase structure treebank (or parser output) trees into f-structure/dependency banks for English and Chinese, respectively. Grammatical functions in f-structures are unordered. For the purpose of training DN-gram models, we automatically associate surface string positions (numerical word offset from start of the sentence) with the corresponding f-structure predicates to linearise the unordered set of GFs, and GF sequences are collected from each level of embedding in local f-structures for training. Unlike the training data, the f-structures input to the surface realiser do not contain any string position information. But, due to the particulars of the automatic f-structure annotation algorithm, the order of sub-f-structures in set-valued GFs, such as adjunct and coord, happens to correspond to their surface order. To avoid unfairly inflating evaluation results, we lexically reorder the GFs in each local f-structure before the generation process. This resembles the ‘permute, no dir'-type experiment in Langkilde (Reference Langkilde2002), where the modifiers in the input feature-value structures are not ordered.

Experiments for English are carried out on the Wall Street Journal (WSJ) portion of the PTB-II, using standard training/test/development splits. That is, 39,832 sentences from sections 02–21 are used for training, 2,416 sentences from section 23 for testing and 1,700 sentences from section 22 are held out for development. The latest version of the Penn Chinese Treebank 6.0 (CTB6), excluding the portion of ACE broadcast news, is used for experiments on Chinese data. Following the recommended splits (in the list-of-file of the CTB6), we divide the data into 756 files with a total of 15,663 sentences as a training set, 84 files with 1,900 sentences as a test set and 50 files with 1,245 sentences as a development set. Table 3 gives an overview of all the data sets used in the experiments.Footnote ⁷

Table 3.

English and Chinese data sets

^a Total number of sentences.

^b Mean number of tokens per sentence, including punctuation marks.

^c Total number of sub-f-structures.

^d Mean number of sub-f-structures per sentence.

^e Mean number of GFs per sub-f-structure, including punctuation GFs.

In the experiments, we create DN-gram models with the SRILM toolkit (Stolcke Reference Stolcke2002). It has been shown that more sophisticated n-gram models (e.g. with higher n, complex smoothing techniques, clustering or caching) are often not worth the effort of implementing them due to data sparseness and other issues (Goodman Reference Goodman2001); therefore in the experiments reported below we implement trigram models throughout.

4.2 Experimental results

Following Langkilde (Reference Langkilde2002) and much other research on wide-coverage surface realisation, we evaluate the sentence generator by comparing the automatically produced strings to the original treebank sentences, and assess the generation performance in term of three accuracy metrics.

• Exact match is the percentage of generated sentences that exactly match a corresponding reference sentence.

• BLEU score (Papineni et al. Reference Papineni, Roukos, Ward and Zhu2002) is computed as a geometric average of n-gram precisions p_n of the output string with respect to the reference, using n-grams up to length N and uniform weights w_n summing to one. Equation (7) gives the formula, where BP is a sentence brevity penalty factor that adjusts the average n-gram precision, c is the length of the generated string and r is the length of the reference string. In our experiments, we set N = 4 as default:

  (7)

  

• SSA stands for average NIST simple string accuracy, which reflects the average number of insertion (I), deletion (D) and substitution (S) errors between the reference strings (R) and the strings produced by the realiser. SSA is calculated as follows:

  (8)

  

In contrast to many other systems, the dependency-based realisation algorithm guarantees that all input f-structures can produce a complete sentence, hence coverage-dependent evaluation (as adopted in most grammar-based generation systems) is not necessary in the following experiments.

4.2.1 Different languages

We carry out experiments on two typologically different languages: English and Chinese. Tables 4 provides the generation results for the standard EnglishFootnote ⁸ and Chinese test sets. To set up a baseline, we first apply a conventional language model (lang) where only surface word forms are taken into account. To avoid prohibitive overgeneration occurring in previous language model-based approaches, generation proceeds under the projectivity assumption, and at each local f-structure only the realisation with the highest probability is propagated to the next higher level for further concatenation. As shown in the first line of Table 4, a word-based language model achieves only very modest accuracy for both English and Chinese. The DN-gram models starting with the basic model (gf) over bare dependency labels and moving on to models incorporating larger context or more fine-grained lexical and atomic-valued features are reported in lines 2–6. To avoid data sparseness, all extended DN-gram models are smoothed by interpolating them with the basic gf model using the log-linear interpolation described in (6) in Section 3. The first observation is that even the basic gf model makes a substantial difference compared to the lang model with respect to BLEU and SSA scores. And in turn, the models conditioned on parent GF (parent) and head word (head), and the models combined with syntactic features (feat) and lexical features (lex) clearly perform better than the basic gf model. In line with similar observations in parsing (where a lexicalised parser often outperforms a non-lexicalised one), the lex model stands out among all the generation models, boosting the BLEU score of the gf model by nearly 24 percentage points (absolute) for English and 14 percentage points (absolute) for Chinese. In the following, we adopt the lex model as a new basis and combine it gradually with some of the other feature models. This results in six combination models (lines 7–12) with increasing complexity. As can be expected, combining more models to include different types of features shows a steady increase in all evaluation metrics, although the magnitude of the improvement becomes smaller due to some overlap between the features. All the DN-gram models capture some kind of relationship between the nodes of the dependency structure, whereas the language model captures relationships between adjoining words in a sentence. We finally combine these two different types of n-gram models to see if further improvement can be achieved. We find that the role of the language model in English and Chinese is somewhat different. The language model appears to have a strong effect on Chinese, showing a significant improvement across the board, especially with respect to the SSA score. Although the addition of the language model also increases exact match for English by more than 1 percentage point, the improvements in BLEU and SSA scores for English are not that prominent. This difference can probably be explained by the fact that English is much richer than Chinese with respect to word forms. Morphological inflection, abbreviation (e.g. Calif. versus California) and different orthographic conventions (e.g. realise versus realize) are common in English but almost absent in Chinese. We conjecture that using a larger corpus to train the language model and performing more elaborate preprocessing to capture word variations should increase the effect of the language model on the English data.

Table 4.

Results for LFG dependencies for English Penn-II WSJ section 23 and Chinese CTB6 test sets

Apart from the effects due to the language model, in general the DN-gram models perform similarly on the English and Chinese data. Nevertheless, comparing the most complex all model with the simplest gf model, we see that the improvement achieved by mixing more dependency features is evidently greater for English (with an increase of 23 exact match points, 26 BLEU points and 20 SSA points absolute)Footnote ⁹ than for Chinese (with an increase of 17 exact match points, 18 BLEU points and 14 SSA points absolute). This is likely to be the case as the English training set is more than twice the size of the Chinese training set. Apart from that, it probably reflects important differences between the two languages and data sets. Examining the development sets, we find that there are a total of 5,103 coordinated elements (4.10 per sentence) in Chinese, in contrast to 2,104 (1.24 per sentence) in English. Many coordinated elements are tied in the DN-gram probabilities and hence can not be correctly ordered during realisation, which to some extent accounts for the lower SSA scores for Chinese. To be specific, we distinguish the following three types of coordinations in Chinese:

• • Coordinated on the syntactic level, but not on the semantic level, for example:

  1. (2)

     invest million build this construction

     ‘invest million dollars to build the construction’

• • Coordinated on syntactic and semantic levels, but the phrase is usually expressed in a fixed order, for example:

  1. (3)

     reform opening-up

     ‘reform and opening up’

• • Coordinated on syntactic and semantic levels, and the order of the two conjuncts can be swopped freely, for example:

  1. (4)

     1. (a)

        plentiful energy and quick thinking

        ‘energetic and agile’

     2. (b)

        ‘agile and energetic’

Due to lack of inflection and case markers, many sequences of Chinese verb phrases are treated as conjuncts in the f-structure analyses, which give rise to the first type of coordinations, while, in many cases, their English counterparts can be distinguished into different grammatical functions. For example, the infinitive to in sentence (2) indicates that the second verb phrase functions as a complement, as illustrated in Figure 3(b). In principle, this type of syntactic coordination should be solved by a more precise semantic analysis on the f-structure level.

Fig. 3.

Different f-structure analyses for the sentence /invest million dollars to build the construction.

The other two types, as exemplified by (3) and (4), are both genuine semantic coordinations, but differ in the flexibility of their word order. In the former case, it is adequate to capture the word order of a fixed expression by a conventional language model, which has been partially reflected in the scores of the complex all+lang model for Chinese as listed in Table 4. In the latter case where the meaning can be expressed more freely, (4-a) & (4-b) are almost equally acceptable, and choosing any one of them would have the same effect on the actual generation quality (although it may not be reflected in the string-matching-based automatic evaluation results, if only one reference is given).

4.2.2 Different representations

Even though the DN-gram models are presented with LFG f-structures, these are general-purpose models and should be suitable for any labelled bilexical dependency or typed predicate-argument representations. To test this claim, we carry out a set of experiments on the dependency structures used in the CoNLL shared tasks (2006–2009) on dependency parsing.Footnote ¹⁰ We convert the Penn-II treebank trees into an English dependency bank, using the pennconverter toolkitFootnote ¹¹ (Johansson and Nugues Reference Johansson and Nugues2007) that produces dependency trees following the linguistic conventions of the CoNLL-2008 shared task. In a more realistic scenario, such as transfer-based machine translation, the input representations to a realisation component are likely to contain some noise and will not be as perfect as the dependency structures converted from the gold treebank trees. To approximate noisy data, we parse the PTB development and test sentences by a state-of-the-art dependency parser – the MSTParser (McDonald et al. Reference McDonald, Crammer and Pereira2005), and take the parsing output dependency trees as alternative realisation inputs. Table 5 shows the results for the various n-gram generation models given the representations obtained via treebank conversion and automatic parsing, respectively. To give a general picture of the sentences realised by the DN-gram models, Table 6 lists a set of example outputs (and their BLEU scores) for a sentence extracted from the Penn-II WSJ section 22 development set under the various generation models in Table 5, given the converted CoNLL dependencies as inputs.

Table 5.

Results for CoNLL dependencies for English Penn-II WSJ section 23

Table 6.

Example outputs given by CoNLL dependencies

The results in Table 5 follow pretty much the same pattern as in Table 4: all the DN-gram models capturing some type of dependency relations outperform the simple language model, the combination of individual DN-gram models in turn makes further substantial improvements, and again, addition of the language model to the complex feature model gives the best performance with respect to generation accuracy. More importantly, the results show that the DN-gram models do not overfit the training data acquired from gold treebank trees. Even though the dependency trees automatically produced by the MSTParser are noisy (McDonald et al. (Reference McDonald, Crammer and Pereira2005) reported unlabelled accuracy of 90.9 per cent discounting punctuation marks testing on Penn-II WSJ23), there is only a slight decrease (approximately 2 points BLEU and SSA absolute) in the realisation quality compared to dependencies derived from gold trees as generation input.

Another obvious observation is that all the evaluation scores for the CoNLL-style dependency structures (including parser output) are substantially higher than those for the LFG f-structures. On the one hand, the results indicate that the dependency-based n-gram generation models generalise well to different realisation inputs, on the other hand they also reaffirm the statement by many researchers that generation results are heavily dependent on how specific the generation inputs are. We note that CoNLL dependency structures and LFG f-structures exhibit different levels of linguistic abstraction. The CoNLL dependencies tend to be more ‘surfacy’ and fine-grained than the more abstract LFG f-structures:

• • Labels of the CoNLL dependencies are more fine-grained than f-structure grammatical functions, and manifest some surface syntactic features, for example CoNLL-style modifiers are classified into modifiers of adjective or adverbial (AMOD) heads, modifiers of nominal (NMOD) heads and modifiers of prepositions (PMOD).

• • Part-of-speech tags are fully specified in the CoNLL dependency representations and used as a syntactic feature by the feat model, which has a strong positive effect on the realisation quality. By contrast, the LFG models do not use POS information.

• • Nodes of the CoNLL dependency representations are simply the word forms occurring in surface sentences rather than lemmas as encoded in the predicates of the LFG f-structures.

• • Function words are explicitly represented as nodes in the CoNLL dependency representations, and only indirectly as atomic-valued features in the LFG f-structures.

It might be argued that the CoNLL dependency representations do not provide ‘semantic’ specifications, and therefore they are not fully theoretically adequate representations for sentence realisation. However, as outputs of many off-the-shelf and state-of-the-art dependency parsers, the CoNLL-like dependency structures are certainly important representations available to practical NLG systems (including transfer-based machine translation), they have become a de-facto standard in many applications.