Elamur Research⌕
☰
home / papers / 2016 / Chinese Zero Pronoun Resolution with Deep Neural Networks

Chinese Zero Pronoun Resolution with Deep Neural Networks

Summary

Chinese text often omits pronouns, leaving gaps (zero pronouns) that refer back to earlier mentions. Resolving these anaphoric zero pronouns (AZPs) is harder than resolving overt pronouns because the gaps carry no number or gender cues. The strongest earlier systems, the unsupervised CN14 and CN15 models, depend on heavy feature engineering, use no lexical information, and optimize data likelihood rather than resolution accuracy. Chen and Ng instead train a supervised, ranking-based deep network that they call an embedding matching approach. The AZP and each candidate antecedent are encoded from pretrained word embeddings combined with hand-crafted features. Each encoding passes through several tanh layers to produce a compact representation. The AZP's representation is then scored against each candidate's, and softmax converts the scores into probabilities. On the Chinese part of OntoNotes 5.0, the network surpasses CN15 and sets a new state of the art. The authors present it as the first neural approach to zero pronoun resolution. Its significance is that embeddings let the model generalize over lexical contexts that unsupervised resolvers could not exploit.

Key points

  • Data and evaluation: experiments use the Chinese portion of OntoNotes 5.0 under three settings, each scored by recall, precision and F-score:
    • gold parses with gold AZPs;
    • gold parses with system-detected AZPs;
    • system parses with system AZPs.
  • Training instances: each AZP is paired with four candidates. One is the closest correct antecedent, and the other three are the most salient non-coreferent entities. Salience weights a mention's grammatical role (subject 4, object 2, other 1) and decays by 0.5 per sentence of distance. Only entities mentioned within two sentences of the AZP are considered.
  • Full-system F-scores: 52.2 in Setting 1, 32.1 in Setting 2 and 18.4 in Setting 3.
  • Ablation: using only embedding features drops Setting 1 to 40.0 F, and using only hand-crafted features gives 48.5 F. The two feature types therefore complement each other. During training, F-score levels off near 52% around epoch 80, which the authors take as evidence that the ranking objective tracks the evaluation metric.
  • Error analysis: the lexical context is narrow, covering only the word before the AZP, its governing verb and the candidate's head word. Most errors stem from this limited context. The authors suggest matching sentence embeddings of the surrounding clauses as future work.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

18 papers of our library cite it (of the 140673 whose bibliographies we read); 8 of them have a page here, 10 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p16-1074

@inproceedings{chen2016chinese,
  title = {{Chinese Zero Pronoun Resolution with Deep Neural Networks}},
  author = {Chen, Chen and Ng, Vincent},
  booktitle = {Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2016},
  doi = {10.18653/v1/p16-1074},
  url = {https://doi.org/10.18653/v1/p16-1074}
}

References

33 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Antonio Ferrández and Jesús Peral. 2000. A computational approach to zero-pronouns in Spanish. In Proceedings of the 38th Annual Meeting on Association for Computational Linguistics, pages 166--172. find
  2. Kai-Wei Chang, Rajhans Samdani, and Dan Roth. 2013. A constrained latent variable model for coreference resolution. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 601--612. find our card
  3. Barbara J. Grosz, Aravind K. Joshi, and Scott Weinstein. 1995. Centering: A framework for modeling the local coherence of discourse. Computational Linguistics, 21(2):203--226. find our card
  4. Na-Rae Han. 2006. Korean zero pronouns: Analysis and resolution. Ph.D. thesis, University of Pennsylvania. find
  5. Chen Chen and Vincent Ng. 2013. Chinese zero pronoun resolution: Some recent advances. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1360--1365. find our card
  6. Masatsugu Hangyo, Daisuke Kawahara, and Sadao Kurohashi. 2013. Japanese zero reference resolution considering exophora and author/reader mentions. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 924--934. find
  7. Chen Chen and Vincent Ng. 2014a. Chinese zero pronoun resolution: An unsupervised approach combining ranking and integer linear programming. In Proceedings of the 28th AAAI Conference on Artificial Intelligence, pages 1622--1628. 2012 Joint Conference on Empirical Methods in Natural Language Process… find
  8. Yuta Hayashibe, Mamoru Komachi, and Yuji Matsumoto. 2011. Japanese predicate argument structure analysis exploiting argument position and type. In Proceedings of 5th International Joint Conference on Natural Language Processing, pages 201--209. find
  9. Sudha Rao, Allyson Ettinger, Hal Daumé III, and Philip Resnik. 2015. Dialogue focus tracking for zero pronoun resolution. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 494--503. find
  10. Jerry Hobbs. 1978. Resolving pronoun references. Lingua, 44:311--338. find our card
  11. Ryu Iida and Massimo Poesio. 2011. A cross-lingual ILP solution to zero anaphora resolution. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1, pages 804--813. find our card
  12. Ruslan Salakhutdinov and Geoffrey Hinton. 2007. Semantic hashing. In Proceedings of the SIGIR Workshop on Information Retrieval and Applications of Graphical Models. find
  13. Ryu Iida, Kentaro Inui, and Yuji Matsumoto. 2006. Ex-ploting syntactic patterns as clues in zero-anaphora resolution. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 625--632. find
  14. Ryohei Sasano and Sadao Kurohashi. 2011. A discriminative approach to Japanese zero anaphora resolution with large-scale lexicalized case frames. In Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 758-- 766. find our card
  15. Ryu Iida, Kentaro Inui, and Yuji Matsumoto. 2007. Zero-anaphora resolution by learning rich syntactic pattern features. ACM Transactions on Asian Language Information Processing, 6(4). find
  16. Ryohei Sasano, Daisuke Kawahara, and Sadao Kurohashi. 2008. A fully-lexicalized probabilistic model for Japanese zero anaphora resolution. In Proceedings of the 22nd International Conference on Computational Linguistics, pages 769--776. find our card
  17. Ryu Iida, Kentaro Torisawa, Chikara Hashimoto, Jong-Hoon Oh, and Julien Kloetzer. 2015. Intra-sentential zero anaphora resolution using subject sharing recognition. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2179--2189. find our card
  18. Ryohei Sasano, Daisuke Kawahara, and Sadao Kurohashi. 2009. The effect of corpus size on case frame acquisition for discourse analysis. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 521-… find
  19. Kenji Imamura, Kuniko Saito, and Tomoko Izumi. 2009. Discriminative approach to predicate-argument structure analysis with zero-anaphora resolution. In Proceedings of the ACL-IJCNLP 2009 Conference Short Papers, pages 85--88. find our card
  20. Hideki Isozaki and Tsutomu Hirao. 2003. Japanese zero pronoun resolution based on ranking rules and machine learning. In Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing, pages 184--191. find our card
  21. Kazuhiro Seki, Atsushi Fujii, and Tetsuya Ishikawa. 2002. A probabilistic method for analyzing Japanese anaphora integrating zero pronoun detection and resolution. In Proceedings of the 19th International Conference on Computational Linguistics - Volume 1. find our card
  22. Thorsten Joachims. 1999. Making large-scale SVM learning practical. In Bernhard Scholkopf and Alexander Smola, editors, Advances in Kernel Methods - Support Vector Learning, pages 44--56. MIT Press. find
  23. Hirotoshi Taira, Sanae Fujita, and Masaaki Nagata. 2008. A Japanese predicate argument structure analysis using decision lists. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 523--532. find
  24. Fang Kong and GuoDong Zhou. 2010. A tree kernel-based unified framework for Chinese zero anaphora resolution. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 882--891. find our card
  25. Yotaro Watanabe, Masayuki Asahara, and Yuji Matsumoto. 2010. A structured model for joint learning of argument roles and predicate senses. In Proceedings of the ACL 2010 Conference Short Papers, pages 98--102. find
  26. Sebastian Martschat and Michael Strube. 2015. Latent structures for coreference resolution. In Transactions of the Association for Computational Linguistics, 3:405--418. find our card
  27. Sam Wiseman, Alexander M. Rush, Stuart Shieber, and Jason Weston. 2015. Learning anaphoricity and antecedent ranking features for coreference resolution. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural L… find our card
  28. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. CoNLL-2012 shared task: Modeling multilingual unrestricted coreference in OntoNotes. In Proceedings of Syntactic features (13) find our card
  29. Yaqin Yang and Nianwen Xue. 2010. Chasing the ghost: recovering empty categories in the Chinese Treebank. In Proceedings of the 23rd International Conference on Computational Linguistics: Posters, pages 1382--1390. find
  30. Ching-Long Yeh and Yi-Chun Chen. 2007. Zero anaphora resolution in Chinese with shallow parsing. Journal of Chinese Language and Computing, 17(1):41--56. find
  31. Katsumasa Yoshikawa, Masayuki Asahara, and Yuji Matsumoto. 2011. Jointly extracting Japanese predicate-argument relation with Markov Logic. In Proceedings of 5th International Joint Conference on Natural Language Processing, pages 1125--1133. find
  32. Koichiro Yoshino, Shinsuke Mori, and Tatsuya Kawahara. 2013. Predicate argument structure analysis using partially annotated corpora. In Proceedings of the Sixth International Joint Conference on Natural Language Processing, pages 957--961. find
  33. Shanheng Zhao and Hwee Tou Ng. 2007. Identification and resolution of Chinese zero pronouns: A machine learning approach. In Proceedings of the 2007 Joint Conference on Empirical Methods on Natural Language Processing and Computational Natural Language Learning, pages 541--550. find our card

↑ back to top