Elamur Research⌕
☰
home / papers / 2020 / Anaphoric Zero Pronoun Identification: A Multilingual Approach

Anaphoric Zero Pronoun Identification: A Multilingual Approach

Summary

Pro-drop languages such as Arabic and Chinese routinely omit subject pronouns that still refer back to earlier entities. These are called anaphoric zero pronouns (AZPs). Most prior AZP work assumes their positions are already known and concentrates on resolution. Existing detectors also depend on hand-crafted, language-specific features, and Arabic had no detector at all. The authors address the identification step with a single multilingual pipeline. Candidate gap positions are first extracted from parse trees using syntactic heuristics: before VP nodes in Chinese, after VP heads in Arabic. A binary classifier then labels each candidate using frozen BERT-base Multilingual representations in feature-extraction mode. Experiments use the Arabic and Chinese parts of OntoNotes 5.0, with both gold trees and automatic Berkeley Parser trees. The model is the first neural AZP identifier for Arabic, where it clearly beats a reimplemented Bi-LSTM baseline. On Chinese it surpasses all earlier systems. The findings indicate that contextual BERT embeddings implicitly capture cues about omitted pronouns, which makes end-to-end AZP resolution more realistic.

Key points

  • Data and setup: The paper uses OntoNotes 5.0 (Arabic and Chinese), where AZPs are marked * (Arabic) and *pro* (Chinese). Classes are heavily imbalanced, with Chinese negatives outnumbering positives about 16.2 to 1. To handle this, the authors tune a sampling ratio r, choosing r=2 for Arabic and r=10 for Chinese.
  • Arabic results: F1 reaches 68.2 with gold parses and 47.0 with system parses. The baseline, Chang et al. (2017), a Bi-LSTM with Fasttext embeddings, scores 54.2 and 31.1.
  • Chinese results: F1 reaches 69.1 with gold parses and 68.7 with system parses. This beats the best earlier system by 4.7 and 11.3 points respectively. The comparison covers Chen and Ng (2013, 2014, 2016), Chang et al. (2017) and Kong et al. (2019).
  • Gap between gold and system parses for Arabic: The Berkeley Parser (Kitaev et al., 2018) misses many Arabic VP nodes, so many true AZP positions are never extracted as candidates. For Chinese, the parser works well and the difference is only 0.4 F1.
  • Error analysis:
    • Arabic: the model struggles with AZPs after first-person verbs, which are rare in the training data.
    • Chinese: it misses sample-initial AZPs, which depend on preceding sentences that the input does not include.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

4 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 3 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{aloraini2020anaphoric,
  title = {{Anaphoric Zero Pronoun Identification: A Multilingual Approach}},
  author = {Aloraini, Abdulrahman and Poesio, Massimo},
  booktitle = {Proceedings of the Third Workshop on Computational Models of Reference, Anaphora and Coreference},
  year = {2020},
  url = {https://aclanthology.org/2020.crac-1.3/}
}

References

73 references, 26 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Bashir M. Alnajadat. 2017. Pro-drop in standard arabic. In International Journal of English Linguistics 7.1. find
  2. Abdulrahman Aloraini and Massimo Poesio. 2020. Cross-lingual zero pronoun resolution. In Proceedings of The 12th Language Resources and Evaluation Conference, pages 90–98. find our card
  3. Chinatsu Aone and Scott William Bennett. 1995. Evaluating automated and manual acquisition of anaphora resolution strategies. In ACL ’95 Proceedings of the 33rd annual meeting on Association for Computational Linguistics, pages 122–129. find our card
  4. Hitham M Abo Bakr, Khaled Shaalan, and Ibrahim Ziedan. 2009. A statistical method for detecting the arabic empty category. In Proceedings of the Second International Conference on Arabic Language Resources and Tools, Cairo, Egypt. find
  5. Thomas G Bever and Montserrat Sanz. 1997. Empty categories access their antecedents during comprehension: Unaccusatives in spanish. Linguistic Inquiry, pages 69–91. find
  6. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5:135–146. find
  7. Donna K. Byron, Whitney Gegg-Harrison, and Sun-Hee Lee. 2006. Resolving zero anaphors and pronouns in korean. In Traitement Automatique des Langues 46.1, pages 91–114. find
  8. Tao Chang, Shaohe Lv, Xiaodong Wang, and Dong Wang. 2017. Zero pronoun identification in chinese language with deep neural networks. In 2017 2nd International Conference on Control, Automation and Artificial Intelligence (CAAI 2017). Atlantis Press. find
  9. Chen Chen and Vincent Ng. 2013. Chinese zero pronoun resolution: Some recent advances. In Proceedings of the 2013 conference on empirical methods in natural language processing, pages 1360–1365. find our card
  10. Chen Chen and Vincent Ng. 2014. Chinese zero pronoun resolution: An unsupervised probabilistic model rivaling supervised resolvers. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. find our card
  11. Chen Chen and Vincent Ng. 2015. Chinese zero pronoun resolution: A joint unsupervised discourse-aware model rivaling state-of-the-art resolvers. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language P… find our card
  12. Chen Chen and Vincent Ng. 2016. Chinese zero pronoun resolution with deep neural network. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 778–788. find
  13. Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D Manning. 2019. What does bert look at? an analysis of bert’s attention. arXiv preprint arXiv:1906.04341. find
  14. Susan Converse. 2006. Pronominal anaphora resolution in chinese. In PhD Thesis, University of Pennsylvania. find
  15. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. In arXiv preprint arXiv:1810.04805. find
  16. B. Di Eugenio. 1990. Centering theory and the italian pronominal system. In Proc. of the 13th COLING, Helsinki, Finland. find
  17. Mushira Eid. 1983. On the communicative function of subject pronouns in arabic. In Journal of Linguistics 19.2, pages 287–303. find
  18. Antonio Ferrández and Jesús Peral. 2000. A computational approach to zero-pronouns in spanish. In Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics, pages 166–172. find
  19. Ryan Gabbard. 2010. Null element restoration. In Ph.D Thesis, University of Pennsylvania. find
  20. Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics. find
  21. Yoav Goldberg. 2019. Assessing bert’s syntactic abilities. In arXiv preprint arXiv:1901.05287. find our card
  22. Madhav Gopal and Girish Nath Jha. 2017. Zero pronouns and their resolution in sanskrit texts. In The International Symposium on Intelligent Systems Technologies and Application, pages 255–267. find
  23. Spence Green, Conal Sathi, and Christopher Manning. 2009. Np subject detection in verb-initial arabic clauses. In Proceedings of the Third Workshop on Computational Approaches to Arabic Script-based Languages (CAASL3). Vol. 112. find
  24. Diana Grigorova. 2013. An algorithm for zero pronoun resolution in bulgarian. In Proceedings of the 14th International Conference on Computer Systems and Technologies. find
  25. Diana Grigorova. 2016. Hybrid approach to zero pronoun resolution in bulgarian. In Proceedings of the 17th International Conference on Computer Systems and Technologies 2016, pages 331–338. find
  26. Na-Rae Han. 2004. A korean null pronouns: Classification and annotation. In Proceedings of the 2004 ACL Workshop on Discourse Annotation. Association for Computational Linguistics, 2004., pages 33–40. find
  27. Masatsugu Hangyo, Daisuke Kawahara, and Sadao Kurohashi. 2013. Japanese zero reference resolution considering exophora and author/reader mentions. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 924–934. find
  28. Jerry Hobbs. 1978. Resolving pronoun references. In Lingua, pages 311–338. find our card
  29. C.-T. James Huang. 1984. On the distribution and reference of empty pronouns. In Linguistic Inquiry, Vol. 15, No. 4, pages 531–574. find
  30. Ryu Iida and Massimo Poesio. 2011. A cross-lingual ilp solution to zero anaphora resolution. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, pages 804–813. find our card
  31. Ryu Iida, Kentaro Inui, and Yuji Matsumoto. 2006. Exploiting syntactic patterns as clues in zero-anaphora resolution. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistic, pages 625–632. find our card
  32. Ryu Iida, Kentaro Inui, and Yuji Matsumoto. 2007. Zero-anaphora resolution by learning rich syntactic pattern features. In ACM Transactions on Asian Language Information Processing, 6(4). find
  33. Ryu Iida, Kentaro Torisawa, Chikara Hashimoto, Jong-Hoon Oh, and Julien Kloetzer. 2015. Intra-sentential zero anaphora resolution using subject sharing recognition. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2179–2189. find our card
  34. Hideki Isozaki and Tsutomu Hirao. 2003. Japanese zero pronoun resolution based on ranking rules and machine learning. In Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing,, pages 184–191. find our card
  35. Ganesh Jawahar, Benoît Sagot, and Djamé Seddah. 2019. What does bert learn about the structure of language? In 57th Annual Meeting of the Association for Computational Linguistics (ACL), Florence, Italy. find our card
  36. Sangkeun Jung and Changki Lee. 2018. Deep neural architecture for recovering dropped pronouns in korean. ETRI Journal, 40(2):257–265. find
  37. Megumi Kameyama. 1985. Zero Anaphora: The case of Japanese. Ph.D. thesis, Stanford University, Stanford, CA. find
  38. Yeun-Bae Kim and Terumasa Ehara. 1995. Zero-subject resolution method based on probabilistic inference with evaluation function. In Proceedings of the 3rd Natural Language Processing Pacific- Rim Symposium, pages 721––727. find
  39. YOUNG-JOO Kim. 2000. Subject/object drop in the acquisition of korean: A cross-linguistic comparison. In Journal of East Asian Linguistics 9.4, pages 325–351. find
  40. Nikita Kitaev, Steven Cao, and Dan Klein. 2018. Multilingual constituency parsing with self-attention and pre-training. arXiv preprint arXiv:1812.11760. find our card
  41. Fang Kong and Guodong Zhou. 2010. A tree kernel-based unified framework for chinese zero anaphora resolution. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, pages 882–891. find our card
  42. Fang Kong, Min Zhang, and Guodong Zhou. 2019. Chinese zero pronoun resolution: A chain-to-chain approach. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), 19(1):1–21. find
  43. Olga Kovaleva, Alexey Romanov, Anna Rogers, and Anna Rumshisky. 2019. Revealing the dark secrets of bert. In arXiv preprint arXiv:1908.08593. find our card
  44. Charles N. Li and Sandra A. Thompson. 1979. Third person pronouns and zero anaphora in chinese discourse. In Syntax and Semantics, volume 12: Discourse and Syntax, pages 311–335. Academic Press. find
  45. Ting Liu, Yiming Cui, Qingyu Yin, Weinan Zhang, Shijin Wang, and Guoping Hu. 2017. Generating and exploiting large-scale pseudo training data for zero pronoun resolution. In arXiv preprint arXiv:1606.01603. find our card
  46. Mohamed Maamouri, Ann Bies, Tim Buckwalter, and Wigdan Mekki. 2004. The penn arabic treebank: Building a large-scale annotated arabic corpus. In NEMLAR conference on Arabic language resources and tools, volume 27, pages 466–467. Cairo. find
  47. Mitchell Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of english: The penn treebank. find our card
  48. Claudiu Mihăilă, Iustina Ilisei, , and Diana Inkpen. 2011. Zero pronominal anaphora resolution for the romanian language. In Research Journal on Computer Science and Computer Engineering with Applications, POLIBITS, 42. find
  49. Ruslan Mitkov and Paul Schmidt. 1998. On the complexity of pronominal anaphora resolution in machine translation. STUDIES IN FUNCTIONAL AND STRUCTURAL LINGUISTICS, pages 207–222. find
  50. Ruslan Mitkov, Richard Evans, Constantin Orasan, Catalina Barbu, Lisa Jones, and Violeta Sotirova. 2000. Coreference and anaphora: developing annotating tools, annotated resources and annotation strategies. In Proceedings of the Discourse, Anaphora and Reference Resolution Conference (DAARC2000), pa… find
  51. Vinod Nair and Geoffrey Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th international conference on machine learning (ICML-10), pages 807–814. find
  52. Matthew Peters, Sebastian Ruder, and Noah Smith. 2019. To tune or not to tune? adapting pretrained representations to diverse tasks. In arXiv preprint arXiv:1903.05987. find
  53. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. Conll-2012 shared task: Modeling multilingual unrestricted coreference in ontonotes. In Joint Conference on EMNLP and CoNLL-Shared Task. Association for Computational Linguistics, Association for Computational… find our card
  54. Luz Rello and Iustina Ilisei. 2009. A rule-based approach to the identification of spanish zero pronouns. In Proceedings of the Student Research Workshop, pages 60–65. find
  55. Luz Rello, Gabriela Ferraro, and Iria Gayo. 2012. A first approach to the automatic detection of zero subjects and impersonal constructions in portuguese. Procesamiento del lenguaje natural, 49:163–170. find
  56. Ryohei Sasano and Sadao Kurohashi. 2011. discriminative approach to japanese zero anaphora resolution with large-scale lexicalized case frames. In Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 758–766. find our card
  57. Ryohei Sasano, Daisuke Kawahara, and Sadao Kurohashi. 2008. A fully-lexicalized probabilistic model for japanese zero anaphora resolution. In Proceedings of the 22nd International Conference on Computational Linguistics, pages 769–776. find our card
  58. Ryohei Sasano, Daisuke Kawahara, and Sadao Kurohashi. 2009. The effect of corpus size on case frame acquisition for discourse analysis. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 521–… find
  59. Kazuhiro Seki, Atsushi Fujii, and Tetsuya Ishikawa. 2002. A probabilistic method for analyzing japanese anaphora integrating zero pronoun detection and resolution. In Proceedings of the 19th International Conference on Computational Linguistics - Volume 1, pages 1–7. find our card
  60. Sho Shimazu, Sho Takase, Toshiaki Nakazawa, and Naoaki Okazaki. 2020. Evaluation dataset for zero pronoun in japanese to english translation. In Proceedings of The 12th Language Resources and Evaluation Conference, pages 3630–3634. find
  61. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. find
  62. The “pro-drop” parameter in adult second language acquisition. Language learning, Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Łukasz Kaiser, Stepha… find
  63. Naiwen Xue, Fei Xia, Fu-Dong Chiou, and Marta Palmer. 2005. The penn chinese treebank: Phrase structure annotation of a large corpus. Natural language engineering, 11(2):207. find
  64. Souta Yamashiro, Hitoshi Nishikawa, and Takenobu Tokunaga. 2018. Neural japanese zero anaphora resolution using smoothed large-scale case frames with word embedding. In Proceedings of the 32nd Pacific Asia Conference on Language, Information and Computation. find our card
  65. Ching-Long Yeh and Yi-Chun Chen. 2006. Zero anaphora resolution in chinese with shallow parsing. In Journal of Chinese Language and Computing 17 (1), pages 41–56. find
  66. Qingyu Yin, Yu Zhang, Weinan Zhang, and Ting Liu. 2016. A deep neural network for chinese zero pronoun resolution. In arXiv preprint arXiv:1604.05800. — [arXiv:1604.05800](https://arxiv.org/abs/1604.05800) find
  67. Qingyu Yin, Yu Zhang, Weinan Zhang, and Ting Liu. 2017. Chinese zero pronoun resolution with deep memory network. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1309–1318. find our card
  68. Qingyu Yin, Yu Zhang, Weinan Zhang, Ting Liu, and William Yang Wang. 2018. Zero pronoun resolution with attention-based neural network. In Proceedings of the 27th International Conference on Computational Linguistics, pages 13–23. find our card
  69. Qingyu Yin, Weinan Zhang, Yu Zhang, and Ting Liu. 2019. Chinese zero pronoun resolution: A collaborative filtering-based approach. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), 19(1):1–20. find
  70. Katsumasa Yoshikawa, Masayuki Asahara, and Yuji Matsumoto. 2011. Jointly extracting japanese predicate-argument relation with markov logic. In Proceedings of 5th International Joint Conference on Natural Language Processing, pages 1125–1133. find
  71. Kei Yoshimoto. 1988. Identifying zero pronouns in japanese dialogue. In Coling Budapest 1988 Volume 2: International Conference on Computational Linguistics. find our card
  72. Koichiro Yoshino, Shinsuke Mori, and Tatsuya Kawahara. 2013. Predicate argument structure analysis using partially annotated corpora. In Proceedings of the Sixth International Joint Conference on Natural Language Processing, pages 957–961. find
  73. Shanheng Zhao and Hwee Tou Ng. 2007. Identification and resolution of chinese zero pronouns: A machine learning approach. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 541–550. find our card

↑ back to top