Hostname: page-component-74656f9884-rngvg Total loading time: 0 Render date: 2026-11-06T01:46:22.507Z Has data issue: false hasContentIssue false
  • English
  • Français

Dropped personal pronoun recovery in Chinese SMS*

Published online by Cambridge University Press:  30 May 2017

CHRIS GIANNELLA ,
RANSOM WINDER  and
STACY PETERSEN
CHRIS GIANNELLA
Affiliation:
Department of Human Language Technology, The MITRE Corporation, 7515 Colshire Drive, McLean, VA, 22102, USA e-mails: cgiannella@mitre.org, rwinder@mitre.org, spetersen@mitre.org
RANSOM WINDER
Affiliation:
Department of Human Language Technology, The MITRE Corporation, 7515 Colshire Drive, McLean, VA, 22102, USA e-mails: cgiannella@mitre.org, rwinder@mitre.org, spetersen@mitre.org
STACY PETERSEN
Affiliation:
Department of Human Language Technology, The MITRE Corporation, 7515 Colshire Drive, McLean, VA, 22102, USA e-mails: cgiannella@mitre.org, rwinder@mitre.org, spetersen@mitre.org
Rights & Permissions [Opens in a new window]

Abstract

In written Chinese, personal pronouns are commonly dropped when they can be inferred from context. This practice is particularly common in informal genres like Short Message Service messages sent via cell phones. Restoring dropped personal pronouns can be a useful preprocessing step for information extraction. Dropped personal pronoun recovery can be divided into two subtasks: (1) detecting dropped personal pronoun slots and (2) determining the identity of the pronoun for each slot. We address a simpler version of restoring dropped personal pronouns wherein only the person numbers are identified. After applying a word segmenter, we used a linear-chain conditional random field to predict which words were at the start of an independent clause. Then, using the independent clause start information, as well as lexical and syntactic information, we applied a conditional random field or a maximum-entropy classifier to predict whether a dropped personal pronoun immediately preceded each word and, if so, the person number of the dropped pronoun. We conducted a series of experiments using a manually annotated corpus of Chinese Short Message Service. Our approaches substantially outperformed a rule-based approach based partially on rules developed by Chung and Gildea (2010, Effects of Empty Categories on Machine Translation. Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics. pp. 636–45). Our approaches also outperformed (though by a considerably smaller margin) a machine-learning approach based closely on work by Yang, Liu, and Xue in (2015, Recovering Dropped Pronouns from Chinese Text Messages. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics (ACL). Association for Computational Linguistics. pp. 309–13). Features derived from parsing largely did not help our approaches. We conclude that, given independent clause start information, the parse information we used was largely superfluous for identifying dropped personal pronouns.

Information

Type
Articles
Information
Natural Language Engineering , Volume 23 , Issue 6 , November 2017 , pp. 905 - 927
Copyright
Copyright © Cambridge University Press 2017 

1 Introduction

Chinese is commonly characterized as a ‘pro-drop’ language (Huang Reference Huang, Jaeggli and Safir1989; Baran, Yang and Xue Reference Baran, Yang and Xue2012) since pronouns are commonly dropped when they can be inferred from context. This practice is particularly common in informal genres like Short Message Service (SMS) messages sent via cell phones (Yang, Liu and Xue Reference Yang, Liu and Xue2015). Dropped personal pronouns (I, we, you, etc.) complicate information extraction since entities are commonly assumed to be explicitly mentioned. Hence, extraction opportunities can be missed when pronouns are dropped. As an example, consider Table 1 and the task of relation detection. The message expresses a contact relation between John and the author of the message. A common approach to relation detection is to identify relations within sentences and establish co-reference chains allowing relations to be detected across sentences. However, such an approach will fail to detect the contact relationship in Table 1 without first recovering ‘I.’

Table 1.

The top row contains an example Chinese SMS along with its English translation. The bottom row contains the messages with ‘I’ recovered

Little research has been dedicated to automated dropped-pronoun recovery in Chinese SMS. Only in the last year have studies been published that directly address the problem.

1.1 Problem statement and our contributions

Dropped personal pronoun recovery can be divided into two subtasks: (1) detecting dropped personal pronoun slots and (2) determining the identity of the pronoun for each slot. We believe the complications to information extraction caused by dropped pronouns can largely be mitigated by recovering only person number. Hence, we address a simplified version of the problem herein: only the person numbers are identified. Specifically, given the rawFootnote 2 text of a Chinese SMS, identify all characters that are immediately preceded by a dropped personal pronoun and determine whether the dropped pronoun is first or second person.Footnote 3 We ignore third person since the vast majority (98.3 per cent) of dropped personal pronouns in our data were not third person.

After we applied a word segmenter to the message, we employed a two-stage strategy. In the first stage, a linear-chain conditional random field (CRF) was used to predict which words were at the start of an independent clause.Footnote 4 In the second stage, a label 0, 1, or 2 was assigned to each word: 0 if no dropped personal pronoun were predicted to immediately precede the word; 1 or 2 if a dropped first or second person pronoun were predicted to immediately precede the word. We developed two different algorithms for accomplishing the second stage. The first applied a linear-chain CRF to the sequence of all words in the message to assign labels. The independent clause start (ICS) information and lexical and syntactic information were used to create features. The second algorithm applied a maximum entropy classifier to words close to ICSs – all other words were assigned label 0.

Through experiments on a manually annotated corpus of Chinese SMS, we examined the accuracy of our two approaches (one for each algorithm used in the second stage). We examined the impact of two different word segmenters and the impact of using parse-based features. We compared our approaches to a rule-based baseline approach based partially on rules described in Chung and Gildea (Reference Chung and Gildea2010). Our approaches achieved F scores more than twice as large as this baseline. We also compared our approaches to a machine-learning baseline approach based closely on work described in Yang et al. (Reference Yang, Liu and Xue2015). Our approaches achieved larger F scores than this baseline. For recovering dropped 1st person pronouns, our CRF-based approach achieved an F score difference against the baseline statistically significant at the p < 0.01 level.

Our work is novel with respect to that of Yang et al. in several ways including, but not limited to, the following.Footnote 5 We examined a CRF-based approach for dropped pronoun recovery in addition to a word classifier-based approach. We observed that dropped pronouns tend to occur close to ICSs, thus, we built a system for predicting ICSs and utilized ICSs in both our CRF and word classifier approaches. For example, our classifier is not applied to all words (as done in Yang et al.) rather is applied only to words close to an ICS.

Among other sources of error, the brevity of SMS sometimes made the determination of dropped pronoun person number difficult. Rao, Ettinger, Daume III and Resnik (Reference Rao, Ettinger, Daume and Resnik2015) address this problem by considering dialog threads of SMS. Since our dataset did not contain dialog threads, we focused only on information that could be derived from messages treated individually. But, extending our approach to utilize ideas from Rao et al. is a good direction of future work.

2 Data

On February 3, 2012, we downloaded 29,393 Chinese SMS from the National University of Singapore (Chen and Kan Reference Chen and Kan2013). In October and November 2014, our MITRE colleague Dr Sichu Li annotated the first 3,495 of these messages. In each message, she added all dropped personal pronouns and marked all the characters that were at the start of an independent clause; the beginning of each sentence (and message) was always marked as an ICS. In the example in Table 1, there are two ICSs and one dropped personal pronoun.

As seen in Table 2, dropped personal pronouns were common in our data set. 47.7 per cent of the messages contained one or more dropped personal pronouns, while 14.1 per cent contained two or more. Moreover, mid-message ICSs were also common. Some 35.3 per cent of the messages contained two or more ICSs, while 20.7 per cent contained three or more. Interestingly, some messages had a large number of ICSs and dropped personal pronouns. Despite the 160 character limit for SMS, some messages included many independent thoughts, expressed succinctly. The following is an example message with seven ICSs and two dropped personal pronouns: , ! , , (I have indeed done a thirty-three day budget, but found it not enough after taking into account your journal. One-day meals will cost about twenty yuan, a total of 800 yuan! It’s a pity but I will not be so hungry that my empty stomach will make my chest and back stick together, I am not good enough, but I can still mentor you.)

Table 2.

The second and third columns show the number of messages with N dropped personal pronouns and independent clause starts, respectively. Since the beginning of each message was marked as an independent clause start, no messages contained zero starts

As seen in Table 3, the dropped personal pronouns identified by our annotator were similar to the ten non-abstract pronouns identified by annotators and described in Baran et al. (Reference Baran, Yang and Xue2012) and Yang et al. (Reference Yang, Liu and Xue2015).Footnote 6

Table 3.

The dropped personal pronouns identified by our annotator along with their counts in our data

Of the 2,221 dropped personal pronouns our annotator identified, only thirty-eight (1.7 per cent) were third person. Hence, we ignored all third-person dropped pronouns.

3 Our approach: outline

Our approach involves a pipeline consisting of several steps. Some of these steps require statistical models to be trained from a set of Chinese SMS annotated with ICSs and dropped personal pronouns. We split our annotated message corpus randomly into training (eighty per cent) and testing (twenty per cent) parts.

Training procedure:

  1. (a) Apply a word segmenter to each message in the training part, ignoring those whose resulting segmentation conflicts with the annotations. Specifically, ignore a message if a dropped personal pronoun or ICS annotation occurs between two characters in the same word.

  2. (b) Build a sequence labeler which, given a word segmented message, labels each word with true or false to indicate whether the word is predicted to be the start of an independent clause. Ignore dropped personal pronoun annotations.

  3. (c) Build a sequence labeler which, given a word-segmented message labeled with ICSs, labels each word 0, 1, or 2: 0 if no dropped personal pronoun is predicted to immediately precede the word; 1 or 2 if a dropped first or second person pronoun is predicted to immediately precede the word.

Messages are ignored in step (a) to avoid training on messages known to contain segmentation errors. When the Stanford (UMAC) segmenter was used, 20 (333) out of 2,796 training messages were ignored.

Application procedure: Given a new Chinese SMS, do the following.

  1. (a) Apply a word segmenter to the message.

  2. (b) Apply the ICS sequence labeler to add the ICS labels.

  3. (c) Apply the dropped pronoun person number sequence labeler to add the dropped pronoun labels.

4 Our approach: details

In this section, we describe the details of each step of the pipeline.

4.1 Chinese word segmentation

In our experiments, we compared two Chinese word segmenters. The first was the Stanford Chinese word segmenterFootnote 7 version 1.6.7 using the ‘ctb’ model (Manning et al. Reference Manning, Surdeanu, Bauer, Finkel, Bethard and McClosky2014). This segmenter was not specifically designed for SMS. The second was a segmenter specifically designed for Chinese SMS (Wang, Wong, Chao and Xing Reference Wang, Wong, Chao and Xing2012) which we refer to as the ‘UMAC’ segmenter. The segmenters produced substantially different results: eighty-four per cent of the messages were segmented differently.

4.2 Independent clause start prediction

We built a linear-chain CRF (Lafferty, McCallum and Pereira Reference Lafferty, McCallum and Pereira2001) model from the training part, using a mean zero Gaussian prior (ignoring the dropped personal pronoun annotations). The result was a sequence labeler which, given a new word segmented message, labeled words that were predicted to be the start of an independent clause. We used the following features:

  • • All word unigrams from two words to the left to two words to the right of the current word. The word bigrams consisting of the current word and the word to the left and the word to the right.

  • • All character unigrams and bigrams that are a prefix or suffix of the current word.

In training, the Gaussian prior was set using a random pure training (eighty per cent) and development (twenty per cent) split of the training part. The prior variance was ranged over 0.3, 0.5, 0.7, 0.9, 1, 5, 10, 50, 100. For each value, a CRF model was built using the pure-training part and applied to the development, producing an F score. Let γ denote the prior value that produced the largest F score. The final CRF model was built using the full-training part with the Gaussian prior variance set to γ.

It should be noted that for both training and application, we used the jcarafe Java library which is freely available from Sourceforge.Footnote 8 The problem of Chinese independent clause prediction is similar to that of Chinese comma classification as addressed in Xue and Yang (Reference Xue and Yang2011). However, significant differences between the two problems exist, for example, independent clauses can be separated without punctuation.

4.3 Dropped pronoun person number prediction

We considered two different approaches to building a dropped pronoun person number sequence labeler from the training part.

4.3.1 CRF approach

This approach utilizes a linear-chain CRF to label each word with a 0, 1, or 2. We used all the features described in Section 4.2, plus the following.

  • • [ICS feature] A Boolean feature set to ‘true’ if an ICS was three or fewer words to the left of the current word. Three was chosen because, as seen in Table 4, the vast majority of dropped personal pronouns have an ICS within three words to the left: 95.2 per cent when the Stanford word segmenter was used; 97.8 per cent when UMAC was used.

  • • [POS features] All part-of-speech (POS) tags assigned to words two to the left through two to the right of the current word. We used the Stanford POS taggerFootnote 10 version 3.5.2, using the ‘chinese-distsim’ model (Manning et al. Reference Manning, Surdeanu, Bauer, Finkel, Bethard and McClosky2014) in our experiments. The POS bigram consisting of the POS assigned to the current word and to the word to the left. The POS bigram consisting of the POS assigned to the current word and to the word to the right.

  • • [Parse-based features] All the constituency parse patterns that are satisfied at the current word; details can be found in the Appendix.

Table 4.

The second (third) column contains the numbers of dropped personal pronouns in the training part with an independent clause start N words to the left when the Stanford (UMAC) word segmenter was usedFootnote 9

In training, the Gaussian prior was set as described in Section 4.2.

4.3.2 Maximum-entropy classifier approach

A classifier is applied to some of the words in messages to assign dropped personal pronoun labels. Choosing the words to which the classifier is applied is the key to this approach. Motivated by Table 4, the approach classifies only words with an ICS within three words to the left, all other words are assigned label 0.Footnote 11 Specifically, the approach assigns dropped personal pronoun labels to a new, word-segmented, Chinese SMS as follows. Let <W[1], W[2], . . ., W[m]> denote the sequence of words in the message, ordered from left to right.

  1. (a) Assign all words label 0.

  2. (b) Apply the ICS sequence labeler to <W[1], W[2], . . ., W[m]> and let <W[i 1],W[i 2]..,W[i s]> denote the subsequence of words predicted to have an ICS immediately preceding.

  3. (c) For j = 1 to s, do

    1. (a) For k = 0 to 3, apply the maximum-entropy classifier to W[i j + k], if it returns 1 or 2, then change the label on W[i j + k] accordingly and terminate the ‘For k = 0 to 3. . .’ loop.

To build the classifier, we first construct a set of labeled, Boolean training vectors from the training part (after a word segmenter is applied). For each word that our annotator marks as being the start of an independent clause, we consider that word and the three words to the right. If none of these words is annotated as having a preceding dropped personal pronoun, then a training vector is built from each of these words and the label on each vector is set to 0. Otherwise, a training vector is built from the left-most word annotated as having a preceding dropped personal pronoun and its label is set to 1 (2) if the dropped pronoun was first (second) person. Each training vector has entries that indicate the presence or absence of the following features.

  • • [Word features] All word unigrams, bigrams, and trigrams drawn from the words two to the left through two to the right of the current word.

  • • [POS features] All POS unigrams, bigrams, and trigrams drawn from the POS tags assigned to words three to the left through three to the right of the current word.

  • • [Parse-based features] All the constituency parse patterns that are satisfied at the current word; details can be found in the Appendix.

A mean zero, Laplace prior is used in building the maximum-entropy classifier. The prior variance was set in the similar fashion as described in Section 4.2. We used the MALLET version 2.0.7 Java library (McCallum Reference McCallum2002) in our experiments.

5 Experiments

We compared our approaches DP-CRF and DP-Classifier to two baseline approaches. DP-CRF refers to our pipeline using the CRF-based dropped pronoun person number prediction step. DP-Classifier refers to our pipeline using the maximum entropy classifier-based dropped pronoun person number prediction step. The next two subsections describe details of the two baseline approaches; the subsequent subsection describes the methodology we used to make the comparison.

5.1 Baseline approach: DP-baseline-rules

DP-Baseline-Rules is a rule-based approach utilizing constituency parse information similar to an approach described in Chung and Gildea (Reference Chung and Gildea2010). Their rules were not developed for informal Chinese, and we found them to work poorly by themselves on our messages. As such, we added more rules. Our experiments showed that the addition of our rules resulted in an improvement in accuracy over the rules of Chung and Gildea alone. Our rules all target the subject position in clauses with different syntactic structures. Syntactically, a subject occurs in the specifier position of an intonational phrase (IP), but occasionally additional syntactic information will be present to the left of the subject, such as additional modifiers. Hence, the left-most linear position cannot be relied upon as the position where a subject pronoun is dropped.

DP-Baseline-Rules first applies the Stanford constituency parserFootnote 12 version 3.5.2 using the ‘chinesePCFG.ser’ model (Manning et al. Reference Manning, Surdeanu, Bauer, Finkel, Bethard and McClosky2014) to the word-segmented message. Then, for each word, assigns label 0 if none of the following rules hold. If at least one of the rules holds, then the word is assigned a label 1 or 2, randomly.

  1. (a) Any rule listed in the top right of Table 5 in Chung and Gildea (Reference Chung and Gildea2010).

  2. (b) The word is left-most in a verb phrase (VP) that (1) has an IP parent; (2) does not have an immediate left noun phrase (NP sister); (3) does not have an only child that is an adjective, punctuation mark, quantifier phrase, NP, or verb; and (4) does not have a verb descendent that has a right sister NP which has a pronoun, proper noun, or complementizer phrase descendent. This rule was implemented using the TregexFootnote 13 (Levy and Galen Reference Levy and Galen2006) regular expression.

  • • (VP > IP&(!$ - NP)&(! <: VA|PU|QP|NP|VV)&(! < (VV + $(NP < (PN|NR|CP)))))

Table 5.

The table displays an accuracy comparison between DP-Baseline-Rules, DP-Baseline-Yang, DP-CRF, and DP-Classifier. The second column indicates whether parse-based features were used by DP-CRF or DP-Classifier. The third column indicates the word segmenter usedFootnote 17

This rule captures dropped pronouns (as part of a full IP) at the start of a verb phrase, that does not have an NP subject (2) and there are not any intervening elements of the type: adjective, punctuation mark, quantifier phrase, NP, or verb (3). Finally, part (4) prevents the occasional error in which a pronoun, proper noun, or complementizer phrase is mislabeled by the tagger as having a VV parent, and therefore a VP parent, but is not in fact a VP, and therefore cannot have a subject.

  1. (c) The word is left-most in a VP phrase that (1) has a VP parent; (2) does not have an immediate left NP sister; (3) has a punctuation mark as a left sister; and (4) does not have an only child that is an adjective, punctuation mark, quantifier phrase, NP, or verb.

  • • (VP > VP&(!$ - NP)&$ - PU&(! <: VA|PU|QP|NP|VV))

This rule captures dropped pronouns at the start of a clause that is not necessarily a full IP.

  1. (d) The word is left-most in a VP phrase that (1) has a VP parent; (2) does not have an immediate left NP sister; and (3) has a BA descendent with a right sister IP.

  • • (VP > VP&(!$ - NP)& < (BA$ + IP)

This rule captures dropped pronouns in a VP that is nested within another VP that does not already have an NP subject (2) and is a ba construction. This construction is indicated by the word used to focus on the result or influence of an action. A dropped pronoun may occur before the ba, which is what this rule captures. Chinese ba sentences occur with certain action verbs and the normal Subject Verb Object (SVO) word order is changed to Subject+‘ba’+Object+Verb phrase.

  1. (e) The word is left-most in an IP phrase that has a prepositional phrase descendent with an adverb phrase (ADVP) left sister.

  • • (IP < (PP$ - ADVP))

This rule captures dropped pronouns before an adverb phrase in order detect a subjectless VP further right in the tree, and not predicted by the above rules.

5.2 Baseline approach: DP-Baseline-Yang

DP-Baseline-Yang is a machine-learning approach based closely on an approach to dropped pronoun recovery in Chinese SMS/chat described in Yang et al. (Reference Yang, Liu and Xue2015). In Yang’s approach, after word segmentation, a seventeen-class maximum entropy classifier was trained to predict whether a pronoun was dropped immediately before a given word and, if so, which of sixteen types of dropped pronouns was dropped. Given a corpus of test messages, the classifier was applied separately to every word in the corpus. In DP-Baseline-Yang, after word segmentation, a three-class maximum entropy classifierFootnote 14 was trained to predict whether a first or second person pronoun was dropped immediately before a given token and, if so, which person. DP-Baseline-Yang uses the same features as Yang’s approach except for two: the previous pronoun used by the same (other) participant in an earlier message in the dialog (our data set did not contain SMS dialog threads). Finally, DP-Baseline-Yang used the Stanford parser (version 3.5.2) while Yang’s approach used the Berkeley parser.

Yang’s approach used several types of features: lexical, e.g. previous and following word/POS bigrams; contextual, e.g. if the current word is inside a question; parse-based, e.g. path from the current word to the root of the parse tree. The reader is referred to Section 3 of Yang et al. (Reference Yang, Liu and Xue2015) for a detailed list. Our approaches used many of the same features, as well as, some not used by Yang: ICS feature (CRF approach), all parse-based features described in the appendix except YX and OURS15.

5.3 Methodology

DP-Baseline-Rules, DP-Baseline-Yang, DP-CRF, and DP-Classifier were all applied to the messages in the test partFootnote 15 and precision, recall, and F scores computed. All approaches were carried out using two different word segmenters: Stanford, UMAC.

For all approaches, scores were computed at the character level since there were no ground truth word segmentations in the test data. Specifically, let M 1, M 2, . . ., M n denote the messages in the test part; let c i denote the number of characters in M i; and let M i [j] denote the jth character in M i . All approaches operate at the word level of granularity. Hence, each approach assigns 0 to M i [j] if M i [j] is not the first character in a word. Otherwise, the label assigned to M i [j] is the label assigned to the word containing M i [j]. We carried out the following procedure to compute the precision, recall, and F score of labels 1 and 2 for each approach.

  1. (a) Set TP[1], FP[1], FN[1], TP[2], FP[2], and FN[2] to 0.

  2. (b) For i = 1 to n, do

    1. (a) For j = 1 to c i , do

  3. (1) For L = 1 to 2, do

  4. (i.) If the approach assigned label L to M i [j] and our annotator added an Lth person dropped pronoun immediately before M i [j], then TP[L] is incremented.

  5. (ii.) If the approach assigned label L to M i [j] and our annotator did not add an Lth person dropped pronoun immediately before M i [j], then FP[L] is incremented.

  6. (iii.) If the approach did not assign label L to M i [j] and our annotator added an Lth person dropped pronoun immediately before M i [j], then FN[L] is incremented.

  7. (c) For L = 1 to 2, do

    1. (a) Compute precision = TP[L]/(TP[L]+FP[L]), recall = TP[L]/(TP[L]+ FN[L[), and F = 2*precision*recall/(precision+recall).Footnote 16

To evaluate the statistical significance between F scores, we used permutation tests as discussed in Edington and Onghena (Reference Edington and Onghena2007) and Yeh (Reference Yeh2000). Specifically, consider two approaches and let λ1 i [j] and λ2 i [j] denote the label assigned to M i [j] by approach one and two, respectively. Let F L λ1 1, . . ., λ1 n and F L λ2 1, . . ., λ2 n denote the F score, with respect to person number L, of approach one and two, respectively. The procedure below computes, Sig, the significance level at which the following null hypothesis can be rejected: the F score of approach one is less than or equal to approach two.

  1. (a) Let numF = 0 and do the following 100,000 times.

  2. (b) Return Sig = numF/100,000.

6 Results

Several observations can be made from Tables 5 and 6.

  • • The Stanford segmenter typically resulted in more accurate dropped pronoun recovery than the UMAC segmenter. In five out of eight cases, the Stanford segmenter resulted in larger F scores significant at the 0.95 level. This is a surprising result given that the UMAC segmenter was specifically designed for Chinese SMS while the Stanford segmenter was not.

  • • DP-CRF and DP-Classifier had very similar F scores. In only one case was the difference in F scores significant at a level greater than 0.85.

  • • The parse-based features typically did not help DP-Classifier. In only one case was the difference in F scores significant at a level greater than 0.8. We conclude that the parse information was used is largely superfluous for identifying dropped personal pronouns, once ICS predictions have been made.

  • • Our approaches achieved larger F scores, in all cases, than the baseline approach DP-Yang-Baseline. When the Stanford segmenter was used and our approaches used parse-based features, the F score differences were statistically significant at a level greater than 0.8 in three out of four cases, and greater than 0.99 in one case: 1st person pronouns, DP-CRF versus DP-Yang-Baseline.

  • • Our approaches achieved F scores more than twice as large as the baseline approach DP-Rules-Baseline in all cases.

Table 6.

The table displays statistical significances of the difference between some of the F scores in Table 5. The second column indicates whether parse-based features were used. The third column indicates which word segmenter was used. For example, the first row concerns the F score differences between DP-CRF without parse-based features using the stanford versus the UMAC word segmenters

6.1 The effect of ICS features

Since dropped Chinese pronouns tend to occur near the start of independent clauses, we believe that the ICS features are important to both DP-CRF and DP-Classifier. We examined the impact of modifying DP-CRF and DP-Classifier to use an oracle ICS predictor and the impact of modifying DP-CRF to not use ICS features at all. Two conclusions can be drawn from Tables 7 and 8.

  • • Our use of ICS features made a substantial difference in the recovery of 1st person pronouns but not in the recovery of 2nd person pronouns. Indeed, the second and third rows of Table 7 show that the F scores of DP-CRF with predicted ICS (and using the Stanford word segmenter) were larger than no ICS features. But, the first row of Table 8 shows that the F score advantages were only significant at a high level for first person recovery.

  • • Devoting more effort to improve the ICS predictor could be helpful in improving first person pronoun recovery. As seen in Table 7, Oracle ICS features result in larger F scores for both DP-CRF and DP-Classifier. And, as seen in the second and fourth rows of Table 8, the F score advantages were significant at a 0.95 level.

Table 7.

The table displays accuracies of our approaches using ICS features assigned by the annotator (Oracle), ICS features assigned using an ICS predictor (Predicted), and not using ICS features at all (None). In all cases, parse-based features were not used

Table 8.

The table displays statistical significances of the difference between some of the F scores in Table 7. In all cases, parse-based features were not used and the Stanford word segmenter was used

6.2 Dropped pronoun slot prediction

To further understand the overall error, we examined the accuracy of our approaches when distinguishing between pronoun numbers is not required. We modified our methods to detect only the dropped pronoun slots, without recovering the person number of the dropped pronouns. We simply collapsed the first and second person labels in Sections 4.3 and 5.2 to a single label, both in the training and testing parts. We denote the modified approaches DPslot-Baseline-Yang, DPslot-CRF, and DPslot-Classifier. As seen in Table 9, the F scores increase substantially, (more than thirty-five per cent). However, even detecting dropped pronoun slots is challenging with F scores not exceeding 0.6. As with dropped pronoun recovery, our approaches achieved larger F scores, in all cases, than the baseline approach DPslot-Yang-Baseline.

Table 9.

The table displays an accuracy comparison between DPslot-Baseline-Yang, DPslot-CRF, and DPslot-Classifier. These approaches predict only dropped personal pronoun slots – no person number prediction is made

6.3 Other error analysis

To further understand the overall error, we examined three questions.

  1. (i) How effective are our approaches at recovering dropped pronouns not close to an ICS?

  2. (ii) How prevalent is each type of prediction error, e.g. predicted 2nd when the annotator assigned 1st?

  3. (iii) What properties of messages are associated with errors?

To address the first question, we compared the performance of DP-CRF when recovering dropped pronouns close to an ICS against its performance when recovering pronouns not close to an ICS. As seen in Table 10, dropped pronouns close to an ICS are substantially easier to recover: F scores greater than 0.395 (bottom two rows) versus less than 0.13 (top two rows). Moreover, the benefits of parse-based features were no less when recovering dropped pronouns close to an ICS versus not.

Table 10.

The table displays the scores of DP-CRF, with and without parse-based features, with predicted ICS features, and using the Stanford word segmenter. In the top two rows, the scores were computed only for slots at least four characters to the right of an ICS assigned by the annotator (not close to an ICS). In the bottom two rows, the scores were computed only for slots less than four characters to the right of an ICS (close to an ICS)

To address the second question, we computed the prevalence of each type of prediction error as seen in Table 11. As can be seen, the preponderance of prediction errors were false negatives, particularly for 1st person dropped pronoun recovery.

Table 11.

The table refers to DP-CRF with no-parse features and using the Stanford word segmenter. Displayed are the fraction of times a label assigned by the annotator received a predicted label (null means that no predictor label was received). For example, the first row indicates of the total number of times the annotator assigned label ‘1st’, nine per cent were predicted as ‘2nd’

To address the third question, we examined errors made by DP-CRF with no parse-based features and using the Stanford word segmenter. We tabulated the characters that most frequently occurred immediately to the right of slots where an error was made (the top six were ). We selected twenty-three of the SMS on which errors were made with one of these six characters immediately to the right. We had our annotator translate these messages to English. Based on these translations and our discussions with our annotator, we observed three properties of messages associated with dropped pronoun recovery errors.

  1. 1. Messages contain intervening words between dropped pronoun slots and verbs. For example,

    just first come to stay with me.

The annotator indicated a dropped second person pronoun at the start ‘[you] just first come . . .,’ but DP-CRF indicated none. The first two characters (just first) intervene between the dropped pronoun and the verb (come), the third character.

  1. 2. Messages contain subjects that appear after their verbs. For example,

    • •

    • • The cell phone is not manageable yet. Today have a date you?

DP-CRF indicated a dropped pronoun at the start of the second sentence (between, and ) but the annotator indicated none. The subject (you) is at the end, just before the question mark.

  1. 3. Messages contain insufficient context making the determination of dropped pronoun person number a subtle decision. For example,

    • •

    • • still busy. I have no appetite, how to deal with it.

DP-CRF indicated a dropped first person pronoun before the first character (still) but the annotator indicated a dropped second person pronoun. While a dropped ‘I’ is plausible, the annotator felt a dropped ‘you’ was more plausible. This kind of decision is difficult to automate.

7 Related work

Our focus is limited to literature discussing computational approaches to the automatic resolution of dropped pronouns (and closely related problems) in Chinese. The reader is referred to Huang (Reference Huang, Jaeggli and Safir1989) for a detailed discussion of Chinese pronoun dropping from a theoretical linguistics perspective and to Seki, Fujii and Ishikawa (Reference Seki, Fujii and Ishikawa2002), Kawahara and Kurohashi (Reference Kawahara, Kurohashi, Su, Tsujii, Lee and Kwong2005), Sasano and Kurohashi (Reference Sasano and Kurohashi2011) for computational approaches to dropped pronoun resolution (and related problems) in another language, Japanese.

We discuss literature in two groups: (1) Chinese empty category detection and (2) Chinese dropped (zero) anaphora resolution and closely related efforts.

7.1 Chinese empty category detection

Dropped pronouns are but one type of empty categories. Other types include null elements in control constructions (*PRO*), traces of A movement, and the like (Xue, Xia, Huang and Kroch Reference Xue, Xia, Huang and Kroch2000). The problem of automatically detecting empty categories involves identifying all the words in a document that are immediately preceded by one or two empty categories (unlike pronouns, multiple empty categories can immediately precede a word). Several studies have addressed this problem (or a simpler variant) on formally written Chinese text such as newswire (assumed to be word segmented and sentence split).

Yang and Xue (Reference Yang and Xue2010) addressed a somewhat simpler problem: detect whether an empty category immediately precedes each word but not which category. They trained a maximum entropy classifier and applied it independently to each word. They utilized lexical and parse-based features and found the parse-based features to substantially improve accuracy for empty categories in a subject position.

Chung and Gildea (Reference Chung and Gildea2010) addressed a somewhat simpler problem: detect whether a dropped pronoun or *PRO* (and which of the two) immediately precedes each word. They developed and compared a rule-based approach, a CRF approach, and an approach based on training a parser from manually produced parses including empty categories. They found the rule-based approach to be most effective at detecting dropped *PRO*s and the CRF at detecting dropped pronouns.

Cai, Chiang and Goldberg (Reference Cai, Chiang and Goldberg2011) trained the Berkeley constituency parser on manually produced parse trees with empty categories included. They applied the parser on word lattices for each sentence. Since zero, one, or two empty categories may appear before any word, the lattice allows zero, one, or two empty category markers to appear immediately before any word. The resulting tree has words and empty category markers as terminals, and the empty category markers have a single parent specifying the specific empty category that was missing.

Kong and Zhou (Reference Kong and Zhou2013) developed a method that recursively applies a ‘linear tagger’ approach: each word is tagged with a single empty category or none, to various parts of the sentence. The recursion is based on the authors’ definition of clause and sub-clause structure as defined from a constituency parse tree.

Xue and Yang (Reference Xue and Yang2013), Yang (Reference Yang2014) observed that classifying individual words with a single empty category (or none) will miss cases where two empty categories immediately precede a word. To mitigate this shortcoming, these authors utilized dependency parses and classified all pairs of words and heads in each sentence (multiple empty categories that immediately precede the same word will have different heads).

7.2 Chinese dropped anaphora resolution and closely related efforts

Dropped (zero) anaphora resolution involves identifying all dropped NP slots in a document and, for each dropped phrase, identifying its antecedent or determining that one does not exist. Pronoun resolution is a special case where the dropped NP is restricted to being a pronoun. Several studies have addressed dropped anaphora and dropped pronoun resolution in formally written Chinese text such as newswire. All the studies we discuss assume the text is word segmented.

Yeh and Chen (Reference Yeh and Chen2007) developed a rule-based procedure (using shallow parses of each sentence) to address dropped anaphora resolution. These authors used centering theory (Grosz, Joshi and Weinstein Reference Grosz, Joshi and Weinstein1995) in the design of the rules component that identifies antecedents. Kong and Zhou (Reference Kong and Zhou2010) divided dropped anaphora resolution into three subtasks: (1) dropped anaphora slot detection, (2) anaphorisity determination (determine whether a dropped anaphora has an antecedent), and (3) antecedent identification. They designed approaches for all three subtasks on the basis of tree-kernel support vector machines.

Zhao and Ng (Reference Zhao and Ng2007) addressed dropped pronoun resolution. They used a simple rule-based procedure (based on full constituency parses) to identify dropped pronoun slots and to identify candidate antecedents. They developed a method, using a decision-tree classifier, to assign dropped pronouns to antecedents. Yang, Dai and Cui (Reference Yang, Dai and Cui2008) employed a similar approach, except they used a more sophisticated rule-based approach (based on verbal logic valence theory) to identify dropped pronoun slots. Chen and Ng (Reference Chen and Ng2013) extend Zhao and Ng’s approach by utilizing more complex features and co-reference links between dropped pronouns.

Chen and Ng (Reference Chen and Ng2014) developed an approach to Chinese dropped pronoun resolution that does not require a training set with manually added dropped pronouns and antecedents identified. They utilize explicit pronouns to train an approach for resolving dropped pronouns.

Recently, a different version of dropped pronoun resolution has been addressed in Chinese SMS/chat messages. In this version of the problem (which we call dropped pronoun recovery), the dropped pronouns are to be automatically restored without necessarily linking to an antecedent, if one exists. Yang et al. (Reference Yang, Liu and Xue2015) trained a seventeen-class maximum entropy classifier to assign words to one of sixteen types of dropped pronouns or ‘none.’ The class indicated which, if any, dropped pronoun is predicted to immediately precede the word. Their classifier used lexical, POS-based, and parse-based features. They applied their classifier separately to each word in each testing message. Rao et al. (Reference Rao, Ettinger, Daume and Resnik2015) addressed a simpler version of dropped pronoun recovery in which only the person number (first, second, or third) of the dropped pronouns need be restored. They used the approach in Cai et al. (Reference Cai, Chiang and Goldberg2011) to identify dropped pronoun slots. Motivated by centering theory, Rao et al. trained a sequence labeler which, given an SMS dialog between multiple communicants, jointly assigns a focus labelFootnote 18 to each message and a person number to each dropped pronoun. Instead of training based on manually annotated Chinese SMS dialogs, Rao et al. utilize a parallel Chinese/English SMS corpus whose English side has been manually annotated with recovered dropped pronouns. As such, they avoid the difficulty of manually annotating Chinese SMS. Utilizing a parallel corpus in this fashion is similar in spirit to a large number of studies that have attempted to build foreign language natural language processing tools by projecting information across a parallel corpus. In particular, the work of Rahman and Ng (Reference Rahman and Ng2012) addresses co-reference resolution in this fashion.

7.3 Differences between our research and the literature

Most of the literature addresses problems related to Chinese dropped pronoun detection and recovery in formally written text. Given the substantial difference between formally written text and SMS, further study is warranted on SMS. Very recently, two studies Rao et al. (Reference Rao, Ettinger, Daume and Resnik2015) and Yang et al. (Reference Yang, Liu and Xue2015) have addressed Chinese dropped personal pronoun recovery in SMS. These studies were conducted simultaneously to, and independently of, our research.

Rao et al. used dialog-derived information to address a problem very similar to the one we addressed (Rao et al. included the third person label). Given that our data set did not contain SMS dialog threads, we focused only on information that could be derived from messages treated individually.

The primary differences between our research and that of Yang et al. are as follows:

  1. 1. We addressed a simpler problem where only the person number of dropped pronouns need to be recovered. Yang et al. address the full dropped pronoun recovery problem – the specific pronouns that were dropped need be recovered.

  2. 2. We considered a CRF in addition to a word classifier. Yang et al. considered only a word classifier.

  3. 3. We predicted independent clause breaks (without requiring a parser) and used them as features in our CRF. Yang et al. essentially used a full parser to extract this kind of information to use as features in their classifier.

  4. 4. Our word classifier is not applied to all words (as in Yang et al.), just those close to independent clause breaks.

  5. 5. We did not use dialog-driven information. The classifier developed by Yang et al. utilized two features based on dialog-driven information: the previous pronoun used by the same (other) participant in an earlier message.

  6. 6. We examined the impact of different word segmenters on dropped pronoun recovery accuracy.

8 Conclusion

In summary, we addressed a simplified version of the problem of dropped pronoun recovery detection in Chinese SMS. After applying a word segmenter, we used a CRF to predict which words are at the start of an independent clause. Then, using the ICS information and lexical and syntactic information, we applied a CRF or a maximum-entropy classifier to predict whether a dropped personal pronoun immediately preceded each word and, if so, the person number of the dropped pronoun. We conducted a series of experiments using a manually annotated corpus of Chinese SMS. Our approaches substantially outperformed a rule-based approach based partially on rules developed by Chung and Gildea (Reference Chung and Gildea2010). Our approaches also outperformed (though by a considerably smaller margin) a machine-learning approach based closely on work by Yang et al. (Reference Yang, Liu and Xue2015). Features derived from parsing largely did not help our approaches. We conclude that the parse information we used is largely superfluous for identifying dropped pronouns once ICS prediction has been made. This is potentially connected to the mismatch between the genre of the text the parser is trained upon (Chinese newswire) and the genre of the text to which we applied the parser (Chinese SMS). This mismatch potentially degrades the quality of the parse-based features used for dropped pronoun recovery. Finally, ICS features were only significantly helpful at recovering 1st person pronouns and devoting more effort to improve the ICS predictor could be helpful in that case.

One avenue of future work is to use our dropped pronoun slot detection approach followed by the approach in Rao et al. (Reference Rao, Ettinger, Daume and Resnik2015) to assign pronoun identity information to the slots. To do this, we would need a different data set that contains SMS dialog threads. Another avenue of future work is to distinguish anaphoric dropped personal pronouns from non-anaphoric ones. Such a distinction may be useful given that anaphoric pronouns are typically more useful in downstream applications like information extraction.

Appendix: Parse pattern details

Here, we describe the details of the parsing-based features used by DP classifier (the last bullet in Section 4.3.2). The current word is assigned a subset of the following parse pattern IDs (based on the Stanford constituency parser). The pattern IDs indicate their source ‘YX’ and ‘YLX’ indicate that the pattern was drawn from Yang and Xue (Reference Yang and Xue2010), Yang et al. (Reference Yang, Liu and Xue2015), respectively; ‘OURS’ indicates that the pattern is our own.

YX1: the current word is the first word in the lowest IP dominating this word.

YX2: the current word starts an IP with no subject. Subject is detected heuristically by looking at left sisters of a VP node.

YX3-[POS]: the current word has POS label [POS] and starts an IP with no subject.

YX4: the current word is the first terminal child of a VP following a punctuation mark.

YX5: the POS of current word is NT, and it heads an NP that does not have a subject NP as its right sister.

YX6: the current word is a verb in an NP/VP.

YX7-[PL]: the phrasal label of the parent of the current word is [PL].

YX8: the previous word is a transitive verb, and this verb does not take an object.

YLX1: the current word is ‘ ’ and has no subject.

OURS1: the current word is leftmost (first word) in lowest IP dominating this word.

OURS2: the current word starts an IP with no subject; that is, the VP node has no NP left sisters.

OURS3: the current word is a copula and starts an IP with no subject.

OURS4: the current word is the verb ‘have’ as a main verb and starts an IP with no subject.

OURS5: the current word is a verb (categorized as ‘other’) and starts an IP with no subject.

OURS6: the current word is a predicative adjective and starts an IP with no subject.

OURS7: the current word is bei in short passive construction and starts an IP with no subject.

OURS8: the current word starts an adverb phrase that starts an IP with no subject to the right.

OURS9: the current word is a preposition and starts an IP with no subject to the right.

OURS10: the current word is the first terminal child of a VP following a punctuation mark.

OURS11: the current word is the first terminal child of a VP following a punctuation mark, is immediately dominated by a VP, and has no left sister NP.

OURS12: the POS of current word is NT, and it heads an NP that does not have a subject NP as its right sister.

OURS13: the current word is leftmost word in a VP with a left NP sister.

OURS14: the current word is leftmost word in a VP with a right NP sister.

OURS15: the current word is the verb ‘ ’ and has no subject.

Footnotes

1

Also affiliated with The Dept. of Linguistics, Georgetown University, 3700 O Street NW, Washington DC USA.

*

We are thankful for the assistance provided by our MITRE colleagues. Dr Sichu Li annotated, in efficient and professional fashion, a large subset of the SMS we downloaded from the National University of Singapore. Dr John Prange, Mr Rob Case, and Mr Rod Holland provided valuable feedback on a presentation we gave describing our preliminary research findings. We are also thankful for the assistance provided by our colleagues at other institutions. Professor Nianwen (Bert) Xue at Brandeis University, Boston, USA shared his thoughts and expertise on Chinese dropped pronoun detection, at an early stage of our research. Professor Derek F. Wong and Mr Junwen Xing at the University of Macau, Macau, SAR PRC applied their word segmenter to the National University of Singapore corpus.

2 We do not assume the message is word or sentence segmented.

3 As stated, the problem excludes dropped personal pronoun at the end of the message. We found this a rare occurrence and, for simplicity, ignore this case.

4 An independent clause is a collection of words containing a subject (perhaps a dropped pronoun) and a predicate that modifies the subject and the clause can stand alone as a complete sentence. For example, ‘. . ., ’ (‘. . ., I can still mentor you. ’).

5 The reader is referred to Section 7.3 for more details.

6 Baran et al. and Yang et al. used different data sets than we did.

7 http://nlp.stanford.edu/software/segmenter.shtml

8 http://sourceforge.net/projects/carafe/files/jcarafe/

9 More dropped pronouns have an ICS one word to the left when UMAC is used versus Stanford (85.5 per cent versus 84.6 per cent). However, this difference does not necessarily indicate that UMAC is more accurate or better for correlating dropped pronouns with ICSs. The difference might be due to UMAC producing a coarser, but less accurate segmentation than Stanford.

10 http://nlp.stanford.edu/software/tagger.shtml

11 Dropped pronouns not the subject of the independent clause they inhabit will tend to be missed by this approach. However, only 1.8 per cent of the dropped pronouns in the training part were not subjects, so these types of misses seem acceptable.

12 http://nlp.stanford.edu/software/lex-parser.shtml

13 http://nlp.stanford.edu/software/tregex.shtml

14 We assume that Yang et al. used MALLET with the default settings, thus, we implemented DP-Baseline-Yang using the default settings. As with DP-Classifier, we used MALLET version 2.0.7.

15 In the test part, our annotator added back 408 dropped personal pronouns.

16 If TP[L]+FP[L] is zero, then precision is defined to be one; if TP[L[+FN[L] is zero, then recall is defined to be zero; if precision+recall is zero, then F is defined to be zero.

17 We did not use overall accuracy when reporting results because it was dominated by false negatives. The trivial algorithm labeling all characters with a 0 achieved an overall accuracy of ninety-six per cent. DP-CRF with parse-based features and using the Stanford word segmenter achieved an accuracy of 96.9 per cent.

18 1 when the focus is on the writer of the message. 2 when the focus is on another person in dialog with the writer.

References

Baran, E., Yang, Y., and Xue, N. 2012. Annotating dropped pronouns in Chinese newswire text. In Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC). European Language Resources Association (ELRA). pp. 2795–9.Google Scholar
Cai, S., Chiang, D., and Goldberg, Y. 2011. Language-independent parsing with empty elements. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics (ACL), Stroudsburg, PA USA, Association for Computational Linguistics. pp. 212–6.Google Scholar
Chen, C., and Ng, V. 2013. Chinese zero pronoun resolution: some recent advances. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), Stroudsburg, PA USA, Association for Computational Linguistics. pp. 1360–5.Google Scholar
Chen, T., and Kan, M.-Y. 2013. Creating a live, public short message service corpus: the NUS SMS corpus. Language Resources and Evaluation 47 (2): 299–335. doi: 10.1007/s10579-012-9197-9.Google Scholar
Chen, C., and Ng, V. 2014. Chinese zero pronoun resolution: an unsupervised approach combining ranking and integer linear programming. In Proceedings of the 28th AAAI Conference on Artificial Intelligence, Palo Alto, CA USA, Association for the Advancement of Artificial Intelligence Press. pp. 1622–8.Google Scholar
Chung, T., and Gildea, D. 2010. Effects of empty categories on machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), Stroudsburg, PA USA, Association for Computational Linguistics. pp. 636–45.Google Scholar
Edington, E., and Onghena, P. 2007. Randomization Tests, 4th ed. Boca Raton, FL, USA: CRC Press, Taylor & Francis Group. ISBN: 978-1-58488-589-4.CrossRefGoogle Scholar
Grosz, B., Joshi, A., and Weinstein, S., 1995. Centering: a framework for modeling the local coherence of discourse. Computational Linguistics 21 (2): 203–25.Google Scholar
Huang, C. T. J. 1989. Pro-drop in chinese: a generalized control theory. In Jaeggli, O. and Safir, K. (eds.), Studies in Natural Language and Linguistic Theory: The Null Subject Parameter, vol. 15, pp. 185–214. Netherlands: Springer. doi: 10.1007/978-94-009-2540-3_6.Google Scholar
Kawahara, D., and Kurohashi, S. 2005. Zero pronoun resolution based on automatically constructed case frames and structural preference of antecedents. In Su, K.-Y., Tsujii, J., Lee, J.-L., and Kwong, O. Y. (eds.), Lecture Notes in Computer Science, vol. 3248, pp. 12–21. Berlin Heidelberg: Springer. doi: 10.1007/978-3-540-30211-7_2.Google Scholar
Kong, F., and Zhou, G. 2010. A tree kernel-based unified framework for Chinese zero anaphora resolution. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), Stroudsburg, PA USA, Association for Computational Linguistics. pp. 882–91.Google Scholar
Kong, F., and Zhou, G. 2013. A clause-level hybrid approach to Chinese empty element recovery. In Proceedings of the 23rd International Joint Conference on Artificial Intelligence (IJCAI), Palo Alto, CA USA, Association for the Advancement of Artificial Intelligence Press. pp. 2113–9.Google Scholar
Lafferty, J., McCallum, A., and Pereira, F. 2001. Conditional random fields: probabilistic models for segmenting and labeling sequence data. In Proceedings of the 18th International Conference on Machine Learning (ICML), Burlington, MA USA, Morgan Kaufmann. pp. 282–9.Google Scholar
Levy, R., and Galen, A. 2006. Tregex and tsurgeon: tools for querying and manipulating tree data structures. In Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC). European Language Resources Association (ELRA). pp. 2231–4.Google Scholar
Manning, C., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S. J., and McClosky, D. 2014. The stanford CoreNLP natural language processing toolkit. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL), Stroudsburg, PA USA, Association for Computational Linguistics. pp. 55–60.Google Scholar
McCallum, A. 2002. Accessed July 16, 2013. http://mallet.cs.umass.edu.Google Scholar
Rahman, A., and Ng, V. 2012. Translation-based projection for multilingual coreference resolution. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), Palo Alto, CA USA, Association for Computational Linguistics. pp. 1051–60.Google Scholar
Rao, S., Ettinger, A., Daume, H. III, and Resnik, P. 2015. Dialogue focus tracking for zero pronoun resolution. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), Palo Alto, CA USA, Association for Computational Linguistics. pp. 494–502.Google Scholar
Sasano, R., and Kurohashi, S. 2011. A discriminative approach to japanese zero anaphora resolution with large-scale lexicalized case frames. In Proceedings of the 5th International Joint Conference on Natural Language Processing (IJCNLP), Stroudsburg, PA USA, Association for Computational Linguistics. pp. 758–66.Google Scholar
Seki, K., Fujii, A., and Ishikawa, T. 2002. A probabilistic method for analtyzing japanese anaphora integrating zero pronoun detection and resolution. In Proceedings of the 19th International Conference on Computational Linguistics (COLING), Stroudsburg, PA USA, Association for Computational Linguistics. pp. 1–7.Google Scholar
Wang, L., Wong, D., Chao, L., and Xing, J. 2012. CRFs-based Chinese word segmentation for micro-blog with small-scale data. In Proceedings of the Second CIPS-SIGHAN Joint Conference on Chinese Language Processing, Stroudsburg, PA USA, Association for Computational Linguistics. pp. 51–7.Google Scholar
Xue, N., and Yang, Y. 2011. Chinese sentence segmentation as comma classification. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics (ACL), Stroudsburg, PA USA, Association for Computational Linguistics. pp. 631–5.Google Scholar
Xue, N., and Yang, Y. 2013. Dependency-based empty category detection via phrase structure trees. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), Stroudsburg, PA USA, Association for Computational Linguistics. pp. 1051–60.Google Scholar
Xue, N., Xia, F., Huang, S., and Kroch, A. 2000. The bracketing guidelines for the penn Chinese treebank (3.0). Technical Report No. IRCS-00-08, University of Pennsylvania Institute for Research in Cognitive Science. http://repository.upenn.edu/ircs_reports/39/.Google Scholar
Yang, W., Dai, R., and Cui, X. 2008. Zero pronoun resolution in Chinese using machine learning plus shallow parsing. In Proceedings of the IEEE International Conference on Information and Automation, New York, NY USA, Institute of Electrical and Electronics Engineers. pp. 905–10.Google Scholar
Yang, Y. 2014. Reading between the lines: recovering implicit information from Chinese texts. Ph.D. Stroudsburg, PA USA: Dissertation, Department of Computer Science, Brandeis University.Google Scholar
Yang, Y., Liu, Y., and Xue, N. 2015. Recovering dropped pronouns from Chinese text messages. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics (ACL), Stroudsburg, PA USA, Association for Computational Linguistics. pp. 309–13.Google Scholar
Yang, Y., and Xue, N. 2010. Chasing the ghost: recovering empty categories in the Chinese treebank . In Proceedings of the 23rd International Conference on Computational Linguistics (COLING), Beijing, P.R. CHINA, Tsinghua University Press. pp. 1382–90.Google Scholar
Yeh, A. 2000. More accurate tests for the statistical significance of result differences. In Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics (ACL), Stroudsburg, PA USA, Association for Computational Linguistics. pp. 947–53Google Scholar
Yeh, C.-L., and Chen, Y.-C., 2007. Zero anaphora resolution in Chinese with shallow parsing. Journal of Chinese Language and Computing 17 (1): 41–56.Google Scholar
Zhao, S., and Ng, H.T. 2007. Identification and resolution of Chinese zero pronouns: a machine learning approach. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), Stroudsburg, PA USA, Association for Computational Linguistics. pp. 541–50.Google Scholar
Figure 0

Table 1. The top row contains an example Chinese SMS along with its English translation. The bottom row contains the messages with ‘I’ recovered

Figure 1

Table 2. The second and third columns show the number of messages with N dropped personal pronouns and independent clause starts, respectively. Since the beginning of each message was marked as an independent clause start, no messages contained zero starts

Figure 2

Table 3. The dropped personal pronouns identified by our annotator along with their counts in our data

Figure 3

Table 4. The second (third) column contains the numbers of dropped personal pronouns in the training part with an independent clause start N words to the left when the Stanford (UMAC) word segmenter was used9

Figure 4

Table 5. The table displays an accuracy comparison between DP-Baseline-Rules, DP-Baseline-Yang, DP-CRF, and DP-Classifier. The second column indicates whether parse-based features were used by DP-CRF or DP-Classifier. The third column indicates the word segmenter used17

Figure 5

Table 6. The table displays statistical significances of the difference between some of the F scores in Table 5. The second column indicates whether parse-based features were used. The third column indicates which word segmenter was used. For example, the first row concerns the F score differences between DP-CRF without parse-based features using the stanford versus the UMAC word segmenters

Figure 6

Table 7. The table displays accuracies of our approaches using ICS features assigned by the annotator (Oracle), ICS features assigned using an ICS predictor (Predicted), and not using ICS features at all (None). In all cases, parse-based features were not used

Figure 7

Table 8. The table displays statistical significances of the difference between some of the F scores in Table 7. In all cases, parse-based features were not used and the Stanford word segmenter was used

Figure 8

Table 9. The table displays an accuracy comparison between DPslot-Baseline-Yang, DPslot-CRF, and DPslot-Classifier. These approaches predict only dropped personal pronoun slots – no person number prediction is made

Figure 9

Table 10. The table displays the scores of DP-CRF, with and without parse-based features, with predicted ICS features, and using the Stanford word segmenter. In the top two rows, the scores were computed only for slots at least four characters to the right of an ICS assigned by the annotator (not close to an ICS). In the bottom two rows, the scores were computed only for slots less than four characters to the right of an ICS (close to an ICS)

Figure 10

Table 11. The table refers to DP-CRF with no-parse features and using the Stanford word segmenter. Displayed are the fraction of times a label assigned by the annotator received a predicted label (null means that no predictor label was received). For example, the first row indicates of the total number of times the annotator assigned label ‘1st’, nine per cent were predicted as ‘2nd’