3.1 Translation span alignment model

The goal of a typical word alignment model is to identify word-to-word alignments between each parallel sentence pair, whereas the goal of our TSA model is to identify span-to-span alignments between each parallel sentence pair. For each translation span pair, its source (or target) span is a sequence of source (or target) words. Given a source sentence s = s ₁, . . ., s_n, a target sentence t = t ₁, . . ., t_m, and its word alignment A, a translation span pair τ is a pair of source span (s_i, . . ., s_j) and target span (t_p, . . ., t_q):

\begin{equation} \tau = (s_i^j \leftrightarrow t_p^q) \end{equation}

where τ indicates that the source span (s_i, . . ., s_j) and the target span (t_p, . . ., t_q) are translational equivalent.

In the TSA model, we do not require that τ is consistent with the word alignment A. In other words, τ is possibly not an extractable phrase pair from the viewpoints of phrase rule extraction algorithms. Due to this reason, it is impossible for us to directly apply typical phrase rule extraction techniques to identify translation span pairs based on word alignment. Figure 2 depicts a TSA of the example sentence pair in Figure 1. In this example TSA, we can see that the translation span pair {[]↔[the imports]} is not an extractable phrase pair because the English word ‘the’ inside the target span is originally aligned to the Chinese word ‘’ outside the source span. In such a case, it cannot be extracted successfully by a phrase-based system. Next, we will discuss how to automatically build TSAs from parallel sentences.

Fig. 2.

The TSA of the sentence pair in Figure 1.

Following the formulation adopted in statistical word alignment (Brown et al., Reference Brown, Della Pietra, Della Pietra and Mercer1993; Vogel et al., Reference Vogel, Ney and Tillmann1996), the TSA between a source sentence s and a target sentence t is modeled as follows:

(1) \begin{equation} \Pr (t|s) = \sum \limits _{d \in D(s,t)} {\Pr _\theta (t,d|s)} \end{equation}

where D(s, t) is a set of derivations that each of them can align s with t, and Pr_θ(t, d|s) is the alignment model that explains the alignment confidence of d; and θ is the associated set of parameters that is learned from the parsed parallel corpus.

In the TSA model, a derivation d can be defined as a sequence of translation span pair applications, i.e., d = τ₁⊕. . .⊕τ_k, in which τ_i indicates the ith translation span pair of (s, t). Afterwards, the best derivation for a given sentence pair (s, t) is obtained by

(2) \begin{equation} \mathord {\buildrel{\lower3pt\hbox{$\scriptscriptstyle \frown $}} \over d} = \mathop {\arg \max }\limits _{d \in D(s,t)} \Pr _\theta (t,d|s) \end{equation}

In principle, the best derivation obtained from (2) implies the best TSA between the given sentence pair (s, t). Since directly calculating Pr_θ(t, d|s) is infeasible due to the extremely large number of derivations, we need to decompose this model to obtain a feasible solution. Assume that translation span pairs are conditionally independent given s, then Pr_θ(t, d|s) can be decomposed as follows. For conciseness, we will drop the subscript θ from now on:

(3) \begin{equation} \Pr _\theta (t,d|s) \approx \prod \limits _{\tau \in d} {\Pr (\tau |s)} \end{equation}

Notice that the derivation d yields the given target sentence t. Hereafter, our goal is to find the best derivation that implies the best TSA, referred to as TSA generation.

3.2 Sub-tree alignment method

The first intuition for TSA generation is to inject knowledge about syntactic constraints into a word alignment model. A straightforward solution is to adopt some sub-tree alignment techniques (Groves et al. Reference Groves, Hearne and Way2004; Tinsley et al. Reference Tinsley, Zhechev, Hearne and Way2007) to build the TSA of each sentence pair with the help of both-side parse trees. Along this line, we can re-model TSA between a given sentence pair (s, t) as a new task of sub-tree alignment between its associated parse tree pair (S, T). In sub-tree alignment methods, the best derivation should yield the given target parse tree T.

The goal of a sub-tree alignment algorithm is to pair up the sub-trees across both-side parse trees, whose lexical leaf nodes covered are semantically translational equivalent. To achieve this goal, a large amount of research has been done to build sub-tree alignments across both-side parse trees by using various methods, such as heuristic rules (Groves et al. Reference Groves, Hearne and Way2004), score functions based on lexical similarity and co-occurrence (Tinsley et al. Reference Tinsley, Zhechev, Hearne and Way2007), tree kernel based score functions and supervised classification techniques (Sun et al. Reference Sun, Zhang and Tan2010a, Reference Sun, Zhang and Tan2010b).

In this work, we utilized an effective approach proposed by Tinsley et al. (Reference Tinsley, Zhechev, Hearne and Way2007) to implement sub-tree alignment over its both-side constituent-based parse trees. This approach does not rely on some heuristic rules and any supervised classification techniques. This sub-tree alignment approach defines three well-formed criteria (Tinsley et al. Reference Tinsley, Zhechev, Hearne and Way2007) with respect to a tree pair:

1. (a) a node can only be linked once;

2. (b) descendants of a source-linked node may only link to descendants of its target-linked counterpart;

3. (c) ancestors of a source-linked node may only link to ancestors of its target-linked counterpart.

These criteria can forbid alignments between constituents from crossing each other. Given a tree pair < S, T >, the sub-tree alignment approach (Tinsley et al. Reference Tinsley, Zhechev, Hearne and Way2007) first takes all the links between the sub-tree pairs as alignment hypotheses, i.e., the Cartesian product of the two sub-tree sets:

\begin{equation} \psi (S) \times \psi (T) = \lbrace s_1, ..., s_I \rbrace \times \lbrace t_1, ..., t_J \rbrace \end{equation}

where Ψ(S) = s ₁, _. . ., s_i, _. . ., s_I and Ψ(T) = t ₁, _. . ., t_j, _. . ., t_J are defined as the sub-tree spaces of the source and target parse trees, respectively. A sub-tree space contains all possible sub-trees of the given parse tree.

Tinsley's algorithm iteratively selects the best link with maximum score, in which a score functionFootnote ⁴ based on the lexical translation probabilities dumped by GIZA + + is used to compute the confidence score of each sub-tree pair. All zero-scored hypotheses are blocked before the algorithm proceeds. The selection procedure iteratively selects the highest scoring link, blocking all hypotheses that contradict the chosen links, until no non-blocked hypotheses remain.

Figure 3 depicts the sub-tree alignment of the example tree pair shown in Figure 1. It is easy to produce TSA based on sub-tree alignment. For each aligned sub-tree pair (s_i, t_j) ∈ Ψ(S) × Ψ(T) selected by a sub-tree alignment algorithm, the yieldsFootnote ⁵ of s_i and t_j can form a translation span pair. Table 2 depicts TSAs generated from sub-tree alignment shown in Figure 3. For example, we can obtain a translation span pair {[]↔[the imports]} from an aligned sub-tree pair { < NN() >, < NP(DT(the), NNS(imports)) > }.

Fig. 3.

Sub-tree alignment of the example tree pair in Figure 1.

Table 2.

TSA generated from sub-tree alignment shown in Figure 3

3.3 Phrase-based forced decoding method

In practice, however, there are some crucial issues with directly utilizing sub-tree alignment techniques for TSA generation. First, sub-tree alignment models still remain limited by word alignment models that underlie it. Second, sub-tree alignment models are essentially inferred from knowledge about constituent boundary-based syntactic constraints and word alignments of the given sentence/tree pair. Third, since sub-tree alignment models are restricted to respect both the source and target parse trees, many desirable node alignments could be ruled out when the syntax of the two languages diverges and parse errors occur, and node alignment quality thus drops.

To address these challenges, we present another effective TSA generation solution in which a phrase-based forced decoding technique is adopted to search the best derivation defined in (2). As mentioned in Section 3.1, this best derivation implies the TSA of a given sentence pair (s, t). Forced decoding techniques aim to inject global knowledge learned from the whole training data into a TSA model, and construct a TSA preferred by the translation model. It can alleviate the disconnection problemFootnote ⁶ between the TSA model and the translation model.

In this work, we apply a phrase-based forced decoding technique instead of syntax-based forced decoding techniques because phrase-based models can achieve the state-of-the-art translation quality with a large amount of training data, and are not limited by any constituent boundary-based constraints for decoding. Our preliminary experiments showed that more than 60% of the training sentence pairs cannot be decoded successfully by using a tree-to-string forced decoding technique. Next, we will introduce some detailed implementations of phrase-based forced decoding techniques.

Given a source sentence s, the goal of typical phrase-based decoding (Koehn et al. Reference Koehn, Och and Marcu2003) typically is to find a target string t* by the following equation

(4) \begin{equation} t^* = \mathop {\arg \max }\limits _t \Pr _\theta (t|s) \end{equation}

where Pr(t | s) is the probability that t is the translation of the given source sentence s. To model the posterior probability Pr(t | s), most of the state-of-the-art SMT systems utilize the log–linear model proposed by Och and Ney (Reference Och and Ney2002):

(5) \begin{equation} \Pr (t|s) = \frac{{\exp (\sum \nolimits _{i = 1}^{|H|} {\lambda _i \bullet h_i (s,t)})}}{{\sum \nolimits _{t^{\prime }} {\exp (\sum \nolimits _{i = 1}^{|H|} {\lambda _i \bullet h_i (s,t^{\prime })})} }} \end{equation}

where H = {h_i(s, t)|i = 1, . . ., |H|} is a set of features and λ_i is the feature weight corresponding to the ith feature. Note that h_i(s, t) can be regarded as a function that maps each pair of source string s and target string t into a real number, and λ_i can be regarded as the contribution of h_i(s, t) to Pr(t | s). Ideally, λ_i indicates the pairwise correspondence between the feature h_i(s, t) and the overall score Pr(t | s). A positive value of λ_i indicates a correlation between h_i(s, t) and Pr(t | s), while a negative value indicates an inversion correlation.

Different from typical phrase-based decoding techniques, we require that each candidate derivation d must yield the same target sentence t during the phrase-based forced decoding process. Only such a candidate derivation can survive during the forced decoding process. Formally, given a sentence pair (s, t), the phrase-based forced decoding technique aims to search for the best derivationFootnote ⁷d* among all consistent derivations that convert the given source sentence s into the given target sentence t with respect to the current translation model induced from the training data. This can be expressed by

(6) \begin{equation} d^* = \mathop {\arg \max }\limits _{d \in D(s,t) \wedge {\rm TGT}(d) = t} \Pr _\theta ({\rm TGT}(d)|s) \end{equation}

where D(s,t) is the set of candidate derivations that transform s to t, and TGT(d) is a function that outputs the yield of a derivation d. θ indicates parameters of the translation model learned from the parallel corpus.

Here, we describe how to generate the best derivation d* for each sentence pair in training corpus C based on the phrase-based forced decoding technique, which performs via the following four steps:

1. 1. Extract translation rules (i.e., grammar induction) needed for a specific phrase-based SMT paradigm M from bilingual training corpus C;

2. 2. Perform minimum error rate training (MERT) on a development data set to obtain a set of optimized feature weights;

3. 3. For each {s,t} ∈ C, translate s into accurate t based on M with translation rules learned in step 1 and feature weights optimized in step 2;

4. 4. For each {s,t} ∈ C, output the derivation with the maximum model score (defined by (5)) produced in step 3 as d*.

Notice that most standard features such as language model, bidirectional lexical and phrase-based translation probabilities and reordering models are used to perform MERT in step 2, which is similarly done in typical phrase-based models (Koehn et al. Reference Koehn, Och and Marcu2003). Two reordering models we used include the maximum entropy (ME)-based lexicalized reordering model (Xiong et al., Reference Xiong, Liu and Lin2006) and the monotone, swap and discontinuous (MSD) reordering model (Tillman Reference Tillman2004; Galley and Manning Reference Galley and Manning2008). To speed up phrase-based forced decoding in step 3, two optimization techniques can be used as follows.

First, since forced decoding techniques only focus on derivations that yield the same target sentence t, the language model score can be regarded as a constant value when calculating the overall model score of each candidate derivation. In such a case, we can drop the language model feature to speed up forced decoding without adverse effects. In the following comparison experiments, our forced decoding method will consider all translation features except language model feature used by the baseline system.

Second, another simple and effective speed optimization technique we adopted is to decode with some phrase translation rules whose target span matches a span (substring) of the given target sentence t, because as described in step 3, all partial hypotheses that do not match any sequence in t will be discarded during the phrase-based forced decoding process.

The best derivation d* produced by phrase-based forced decoding can be viewed as a sequence of translation steps (i.e., phrase translation rules), expressed by

\begin{equation} d* = r_1 \oplus r_2 \oplus \cdots \oplus r_k , \end{equation}

where r_i indicates a phrase rule used to form d*. ⊕ is a composition operation that combines rules {r ₁, _. . ., r_k} together to produce the target translation.

As mentioned above, the best derivation d* respects the input sentence pair (s, t). It means that for each phrase translation rule r_i used by d*, its source (or target) side exactly matches a span of the given source (or target) sentence. The source side src(r_i) and the target side tgt(r_i) of each phrase translation rule r_i in d* form a translation span pair src(r_i)↔tgt(r_i) of (s, t). In other words, the TSA of (s, t) is a set of translation span pairs generated from phrase translation rules used by the best derivation d*. For better understanding, we summarize in Figure 4 the forced decoding-based TSA generation algorithm proposed in this paper. The forced decoding-based TSA generation on the example sentence pair in Figure 1 is shown in Table 3.

Fig. 4.

Forced decoding-based TSA generation. src(r) and tgt(r) indicate the source and target sides of rule r.

Table 3.

Forced decoding-based TSA generation on the example sentence pair in Figure 1

The phrase-based forced decoding technique can implement a fast search to construct TSAs on the training data. We have empirically shown that our proposed approach requires only about three CPU hours to re-align 1.06M Chinese–English sentence pairsFootnote ⁸ used in our evaluation, and less than thirty minutes for spurious link deletion and new link insertion on this corpus. In practice, there is a risk of forced decoding failure on some training sentence pairs due to beam search-based pruning techniques used for decoding. We observed in our experiments that 10.5% of the training sentence pairs failed in the phrase-based forced decoding, when the beam size is set to 30 in our experimental settings. In such a case, we used their original word alignments for translation rule extraction. Actually increasing the beam size can reduce the decoding failure rate, but would slow down the forced decoding speed.

5.1 Baseline systems

We utilized the NiuTrans open-source SMT toolkit (Xiao et al. Reference Xiao, Zhu, Zhang and Li2012) to build all baseline systems in our comparison experiments. Syntax-based baseline systems were constructed based on tree-to-string and tree-to-tree models, respectively. The feature design is mainly based on a state-of-the-art MT system described in Marcu et al. (Reference Marcu, Wang, Echihabi and Knight2006). For example, we use the bidirectional lexical and phrase-based translation probabilities (four features), a syntactic feature – root normalized conditional probability, an n-gram language model, target word penalty, rule penalty, and three binary features – lexicalized rule, low-frequency rule and composed rule. In both baseline systems, for syntactic (tree-to-tree and tree-to-string) translation rule extraction, minimal GHKM (Galley et al., Reference Galley, Hopkins, Knight and Marcu2004) rules are first extracted from the bilingual corpus whose source and/or target sides are parsed using the Berkeley parser (Petrov et al. Reference Petrov, Barrett, Thibaux and Klein2006). To improve the tree-to-tree rule coverage, the parse trees on both language sides are binarized in a head-out manner (Wang et al., Reference Wang, Knight and Marcu2007). The composed rules are then generated by composing two or three minimal rules. To avoid the overfitting problem and to speed up the system, we restrict the rule composing to the rules of at most depth three, and having at most five terminals and four non-terminals.

The third baseline system is a phrase-based system with two reordering models such as the ME-based lexicalized reordering model (Xiong et al., Reference Xiong, Liu and Lin2006) and the MSD reordering model (Tillman Reference Tillman2004; Galley and Manning Reference Galley and Manning2008). In addition to the reordering features, we use most standard features adopted in current state-of-the-art phrase-based systems (Koehn et al. Reference Koehn, Och and Marcu2003), including bidirectional phrase translation probabilities, bidirectional lexical weights, an n-gram language model, target word penalty and phrase penalty. In this baseline system, all phrase pairs are limited to having source length of at most five, and the reordering limit is set to eight by default.Footnote ⁹ The phrase-based baseline system is also used to implement the phrase-based forced decoding method with optimization techniques mentioned in Section 3.3.

5.2 Experimental setup

In Chinese-to-English translation tasks, our bilingual data consist of 1.06M sentence pairs selected from the NIST portion of the bilingual data provided within NIST MT 2008 evaluation. The GIZA + + toolFootnote ¹⁰ was used to perform the bidirectional word alignment between the source and target sentences, and the final alignment was generated using the heuristic ‘grow-diag-final-and’ method, referred to as the baseline method in the following experiments. A 5-gram language model was trained on the Xinhua portion of English Gigaword corpus in addition to the target side of the bilingual data. We used the NIST 2003 MT evaluation set as our development set (919 sentences) and a large-scale test set (4,529 sentences) of the combination of three NIST evaluation sets of MT04, MT05 and MT06. The translation quality was evaluated in terms of case-insensitive IBM version BLEU metric.

In English-to-Chinese translation tasks, we begin with a training bilingual corpus of English–Chinese bitexts that consists of 1.0M sentence pairs provided by CWMT2011 MT evaluation.Footnote ¹¹ Similarly, the baseline word alignment is implemented by using the GIZA + + tool, also referred to as the baseline method in the following experiments. A 5-gram language model was trained on the Xinhua portion of Chinese Gigaword corpus in addition to the target side of the bilingual data. We used the CWMT2011 MT evaluation development set (500 sentences) for weight tuning and test set (1359 sentences) for translation performance evaluation. The translation quality was also evaluated in terms of case-insensitive IBM version BLEU metric.

All three baseline systems described above used a CKY-style decoder with beam search and cube pruning (Huang and Chiang Reference Huang and Chiang2007) to decode new source sentences. The beam size for beam search was set to 30. All features were linearly combined and their weights were optimized by performing MERT (Och Reference Och2003). In the following performance comparison, statistical significance test is conducted using the bootstrap resampling method proposed in Koehn (Reference Koehn2004).

5.3 Syntax-based translation

We first investigate the effect of both proposed TSA methods on the translation quality of tree-to-tree models, which consider both-side parse trees for syntactic translation rule extraction. Tables 5 and 6 show that the sub-tree alignment and the forced decoding methods can yield significant improvements on translation quality on both Chinese-to-English and English-to-Chinese test sets. The sub-tree alignment method is to improve syntactic rule extraction with the help of building node alignments across both-side parse trees that are useful indicators for tree-to-tree translation rule extraction (Tinsley et al. Reference Tinsley, Zhechev, Hearne and Way2007; Sun et al. Reference Sun, Zhang and Tan2010a, Reference Sun, Zhang and Tan2010b). In the forced decoding-based method, the resulting TSAs are essentially optimized by the translation model, which is used to detect spurious alignment links. Experimental results show that spurious link deletion based on such TSAs can benefit translation quality of tree-to-tree models.

Table 5.

Rule sizes and IBM-BLEU4 (%) scores of various methods in tree-to-tree and tree-to-string Chinese-to-English translation on the NIST development set and test set. Here, * indicates significantly better on test performance at the p = 0.05 level

Table 6.

Rule sizes and IBM-BLEU4 (%) scores of various methods in tree-to-tree and tree-to-string English-to-Chinese translation on the CWMT development set and test set. Here, * indicates significantly better on test performance at the p = 0.05 level

However, when compared with the tree-to-string baseline method, as shown in Tables 5 and 6, the forced decoding method achieves significantly better improvements on translation quality, while the sub-tree alignment method yields unsatisfactory performance in both Chinese-to-English and English-to-Chinese translation tasks. Since the sub-tree alignment method utilizes both-side constituent boundary-based syntactic constraints of each sentence pair, it would result in negative effects on tree-to-string translation models which do not consider target-side parse trees for syntactic translation rule extraction.

To analyze why the sub-tree alignment method yields unsatisfactory performance in tree-to-string translation, we further explore the impact of each TSA method upon rule quality in tree-to-string translation. To achieve this goal, we compare the rule precision, recall and F1-measure of the rule set extracted from our hypothesized word alignments against the rule set extracted from gold word alignments. Two hundreds randomly chosen and manually aligned Chinese–English sentence pairs were used to assess tree-to-string rule quality after incorporating TSA into word alignment.

Table 7 shows the total non-unique number of tree-to-string translation rules extracted from each alignment, the corresponding rule precision, recall and F1-measure. Both TSA methods increase the size of the extracted rule set compared with the baseline method. The forced decoding method achieves better performance on rule recall and F1-measure, and drops a little on rule precision. However, the sub-tree alignment method results in negative effects on the quality of tree-to-string translation rules. As mentioned in Section 3.2, the sub-tree alignment method utilizes bilingual constituent boundary-based constraints to produce the TSA of each sentence pair. It seems that directly injecting bilingual syntactic constraints into word alignment can possibly result in negative effects on tree-to-string translation rule extraction. A similar conclusion was also drawn by Koehn et al. (Reference Koehn, Och and Marcu2003) in phrase-based translation tasks. Interestingly, Table 7 shows that the rule precision, recall and F1-score of each automatic method are still low. About two-thirds of desirable tree-to-string translation rules are missed by various methods due to word alignment errors. It means that there is still much room for our future work to improve translation rule extraction for SMT systems.

Table 7.

Rule precision, recall and F1-score of the tree-to-string rule set extracted from 200 sentence pairs of Chinese–English data

5.4 Other translation comparisons

In this section, we first investigate the effectiveness of both proposed TSA methods on the translation quality of a typical phrase-based model in English-to-Chinese and Chinese-to-English translation tasks, as shown in Tables 8 and 9. Both TSA methods can increase the size of the phrase translation table. For both translation tasks, the forced decoding method yields significant improvements on translation performance, while the sub-tree alignment method results in slightly lower performance than the baseline method. It seems that directly injecting bilingual syntactic constraints into word alignment can result in negative effects on phrase-based translation.

Table 8.

Rule sizes and IBM-BLEU4 (%) scores of various methods in English-to-Chinese and Chinese-to-English phrase-based translation on development set and test set. Here, * indicates significantly better on test performance at the p = 0.05 level

Table 9.

The size of training/development/test sets of each minority language–Chinese pair

We also further investigate the effect of the proposed TSA method on the translation between minority languages and Chinese. The training/development/test sets come from CWMT2011 MT evaluation as shown in Table 9, involving five minority languages, such as Kazakh, Kirgiz, Mongolian, Tibetan and Uyghur. Because there are not any publicly-available syntactic parsers for these minority languages, we choose to demonstrate the effectiveness of the forced decoding method for phrase-based translation, compared with the baseline method. As shown in Table 10, the forced decoding method not only increases the size of the extracted phrase translation table but also improves the translation performance for each language pair, compared with the baseline method. Accordingly, our experimental results have shown that the forced decoding-based TSA model can benefit rule quality by removing spurious links, which thus in turn improves translation performance.

Table 10.

Rule sizes and IBM-BLEU4 (%) scores of various methods in minority language-to-Chinese phrase-based translation on development set and test set. Here, * indicates significantly better on test performance at the p = 0.05 level

Finally, we estimate the effectiveness of both proposed TSA methods on the translation quality of a hierarchical phrase-based model (Chiang Reference Chiang2007) in English-to-Chinese and Chinese-to-English translation tasks, as shown in Table 11. For the hierarchical phrase-based model, we follow the general framework of SCFG where a grammar rule has three parts – a source-side, a target-side and alignments between source and target non-terminals. To learn SCFG rules from word-aligned sentences, we choose the algorithm proposed in Chiang (Reference Chiang2007) and estimate the associated feature values as in the phrase-based system. This hierarchical phrase-based baseline system is built using the NiuTrans open-source SMT toolkit (Xiao et al. Reference Xiao, Zhu, Zhang and Li2012). Similar to performance comparisons of phrase-based translation models shown in Table 8, for both Chinese-to-English and English-to-Chinese translation tasks, the forced decoding method yields significant better improvements on translation performance, while the sub-tree alignment method results in slightly lower translation performance than the baseline hierarchical phrase-based method.

Table 11.

Rule sizes and IBM-BLEU4 (%) scores of various methods in English-to-Chinese and Chinese-to-English hierarchical phrase-based translation on development set and test set. Here, * indicates significantly better on test performance at the p = 0.05 level

5.5 Effects on word alignment

Our goal is to detect some spurious word alignment links for improving the translation rule extraction, instead of directly improving the overall quality of word alignments. However, it is also interesting to investigate the effect of the proposed TSA method on word alignments. Here, we examine the effectiveness of the phrase-based forced decoding method for TSA generation because this method results in significantly better improvements on the translation quality of all systems in previously described experiments. We designed some experiments to evaluate our alignment quality against gold-standard annotations. To achieve this goal, there are 200 randomly chosen and manually aligned Chinese–English sentence pairs used to assessing the word alignment quality after incorporating TSA into word alignment. For word alignment evaluation, we calculated precision, recall and F1-score over word alignment.

Table 12 depicts the word alignment performance of both proposed methods and the number of spurious links and new links detected. Notice that the baseline is the GIZA + + alignment method. We apply the TSAs produced by the sub-tree alignment-based and the forced decoding-based methods to refine the baseline word alignments, respectively, involving spurious link deletion and new link insertion operations.

Table 12.

Word alignment precision, recall and F1-score of various methods on 200 sentence pairs of Chinese–English data

Table 12 shows that the sub-tree alignment-based method causes negative effects on word alignment precision, recall and F1-score performances, while the forced decoding-based method can yield improvements on precision and F1-score, only causing a little negative effect on recall performance. Table 12 shows that there are average 1.5 spurious links deleted and 1.1 new valuable links added by the forced decoding method, and average 6.6 spurious links deleted and 7.1 new links added by the sub-tree alignment method.

To get more intuition on how to apply TSA methods to revise original word alignments, we present some illustrations of word alignments conducted by the baseline method and the proposed forced decoding method in Table 13. In case 1, the forced decoding method removed two original links {17-20 19-21} based on two generated translation span pairs [,]↔[,] | {17-31} and []↔[during the summer vacation] | {19-39 19-40 19-41 19-42}, because the two deleted links violated the two translation span pairs. The one-to-one generated translation span pair [,]↔[,] | {17-31} indicates the new link {17-31}. Notice that the proposed TSA model only considers one-to-one translation span pairs for new link generation.

Table 13.

Five examples of word alignments conducted by the baseline method and the proposed forced decoding-based TSA method. The differences between word alignment results of both methods are marked in bold

Similarly, in case 2, the forced decoding method deleted two baseline links {2-7 17-12} and added a new link {17-24} according to two generated translation span pairs []↔[conducted] | {1-5 2-5} and []↔[received] | {17-24}. In case 3, the forced decoding method deleted a baseline link {10-23} and added a new link {10-25} based on a one-to-one translation span pair []↔[gather] | {10-25}. In case 4, a baseline link {13-2} was removed and a new link {13-16} was inserted based on a translation span pair [,]↔[and] | {13-26}. In case 5, our method deleted a baseline link {13-5} based on a one-to-many translation span pair []↔[traveled by vehicles] | {13-19 13-20 13-21}.