Elamur Research⌕
☰
home / papers / 2021 / Tackling Zero Pronoun Resolution and Non-Zero Coreference Resolution Jointly

Tackling Zero Pronoun Resolution and Non-Zero Coreference Resolution Jointly

Summary

Chinese text often omits pronouns, so a system must recover these hidden "zero pronouns" and link them to their antecedents. It must also cluster explicit mentions of the same entity, which is ordinary (non-zero) coreference. Earlier work handled these tasks separately, often assumed that zero pronoun positions were known, or depended on hand-crafted syntactic features.

Chen et al. propose one end-to-end neural model that solves both tasks together without gold zero pronoun positions. Every gap between tokens is a candidate zero pronoun, and every span is a candidate mention. A gap-masked self-attention encoder places gaps and tokens in the same embedding space. Gaps draw context from neighbouring tokens, while token representations stay unaffected. A two-stage interaction mechanism uses the fact that a gap inside a mention cannot be a zero pronoun, and vice versa. It applies this constraint at both the representation level and the scoring level.

Tested on OntoNotes 5.0 Chinese, the joint model beats strong baselines on both tasks. This shows that the two problems benefit from shared learning.

Key points

  • The model builds on the BERT-based coreference system of Joshi et al. (2019), using the official Chinese BERT-base. Maximum span width is reduced from 30 to 20 tokens to save memory and computation.
  • Training jointly improves non-zero coreference by at least 0.8% over a mention-only baseline. Adding the coreference objective raises zero pronoun resolution by more than 0.46%, compared with Song et al. (2020) as the prior state of the art.
  • In the encoder comparison, gap-masked self-attention performed best, with NZ-F1 of 67.40 and Z-F1 of 33.77. Inserting gaps directly into BERT performed worst, with 58.96 and 27.96.
  • The score-level interaction with max aggregation produced the best scores, NZ-F1 67.71 and Z-F1 34.13. Max aggregation generally did better than mean. Not allowing mentions to resolve to zero pronouns as antecedents ("Not to ZP") worked better overall.
  • Code is available at github.com/cheniison/e2e-joint-coref.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

6 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 4 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.conll-1.40

@inproceedings{chen2021tackling,
  title = {{Tackling Zero Pronoun Resolution and Non-Zero Coreference Resolution Jointly}},
  author = {Chen, Shisong and Gu, Binbin and Qu, Jianfeng and Li, Zhixu and Liu, An and Zhao, Lei and Chen, Zhigang},
  booktitle = {Proceedings of the 25th Conference on Computational Natural Language Learning},
  year = {2021},
  doi = {10.18653/v1/2021.conll-1.40},
  url = {https://doi.org/10.18653/v1/2021.conll-1.40}
}

References

29 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abdulrahman Aloraini and Massimo Poesio. 2020. Cross-lingual zero pronoun resolution. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 90–98, Marseille, France. European Language Resources Association. find our card
  2. Fang Kong and Jian Fu. 2019. Incorporating structural information for better coreference resolution. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019, pages 5039–5045. ijcai.org. find
  3. Saoussen Mathlouthi Bouzid and Chiraz Ben Othmane Zribi. 2020. A generic approach for pronominal anaphora and zero anaphora resolution in arabic language. Procedia Computer Science, 176:642–652. find
  4. Fang Kong and Hwee Tou Ng. 2013. Exploiting zero pronouns to improve Chinese coreference resolution. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 278–288, Seattle, Washington, USA. Association for Computational Linguistics. find our card
  5. Chen Chen and Vincent Ng. 2013. Chinese zero pronoun resolution: Some recent advances. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1360–1365, Seattle, Washington, USA. Association for Computational Linguistics. find our card
  6. Fang Kong and Guodong Zhou. 2010. A tree kernel-based unified framework for Chinese zero anaphora resolution. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 882–891, Cambridge, MA. Association for Computational Linguistics. find our card
  7. Chen Chen and Vincent Ng. 2016. Chinese zero pronoun resolution with deep neural networks. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 778–788, Berlin, Germany. Association for Computational Linguistics. find our card
  8. Kevin Clark and Christopher D. Manning. 2016a. Deep reinforcement learning for mention-ranking coreference models. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2256–2262, Austin, Texas. Association for Computational Linguistics. find our card
  9. Kevin Clark and Christopher D. Manning. 2016b. Improving coreference resolution by learning entity-level distributed representations. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 643–653, Berlin, Germany. Association for Co… find our card
  10. Ryu Iida and Massimo Poesio. 2011. A cross-lingual ilp solution to zero anaphora resolution. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 804–813. find our card
  11. Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end neural coreference resolution. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 188–197, Copenhagen, Denmark. Association for Computational Linguistics. find our card
  12. Kenton Lee, Luheng He, and Luke Zettlemoyer. 2018. Higher-order coreference resolution with coarse-to-fine inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 687–… find our card
  13. Peiqin Lin and Meng Yang. 2020. Hierarchical attention network with pairwise loss for chinese zero pronoun resolution. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAA… find
  14. Ting Liu, Yiming Cui, Qingyu Yin, Wei-Nan Zhang, Shijin Wang, and Guoping Hu. 2017. Generating and exploiting large-scale pseudo training data for zero pronoun resolution. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 102–11… find our card
  15. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net. find
  16. Ruslan Mitkov et al. 1995. Anaphora resolution in machine translation. In Proceedings of the Sixth International conference on Theoretical and Methodological issues in Machine Translation. Citeseer. find our card
  17. Tomohide Shibata and Sadao Kurohashi. 2018. Entity-centric joint modeling of japanese coreference resolution and predicate argument structure analysis. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 579–589. find our card
  18. Linfeng Song, Kun Xu, Yue Zhang, Jianshu Chen, and Dong Yu. 2020. ZPR2: Joint zero pronoun recovery and resolution using multi-task learning and BERT. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5429–5434, Online. Association for Computational Li… find our card
  19. Josef Steinberger, Massimo Poesio, Mijail A Kabadjov, and Karel Ježek. 2007. Two uses of anaphora resolution in summarization. Information Processing & Management, 43(6):1663–1680. find
  20. Qingyu Yin, Weinan Zhang, Yu Zhang, and Ting Liu. 2017. A deep neural network for chinese zero pronoun resolution. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, August 19-25, 2017, pages 3322–3328. ijcai.org. find
  21. Qingyu Yin, Yu Zhang, Wei-Nan Zhang, Ting Liu, and William Yang Wang. 2018a. Deep reinforcement learning for Chinese zero pronoun resolution. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 569–578, Melbourne, Australia. Assoc… find our card
  22. Qingyu Yin, Yu Zhang, Weinan Zhang, Ting Liu, and William Yang Wang. 2018b. Zero pronoun resolution with attention-based neural network. In Proceedings of the 27th International Conference on Computational Linguistics, pages 13–23, Santa Fe, New Mexico, USA. Association for Computational Linguistics… find our card
  23. Dmitry Zelenko, Chinatsu Aone, and Jason Tibbetts. 2004. Coreference resolution for information extraction. In Proceedings of the Conference on Reference Resolution and Its Applications, pages 24–31, Barcelona, Spain. Association for Computational Linguistics. find our card
  24. Rui Zhang, Cícero Nogueira dos Santos, Michihiro Yasunaga, Bing Xiang, and Dragomir Radev. 2018. Neural coreference resolution with deep biaffine attention by joint mention detection and mention clustering. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (V… find our card
  25. Shanheng Zhao and Hwee Tou Ng. 2007. Identification and resolution of chinese zero pronouns: A machine learning approach. In Proceedings of the 2007 joint conference on empirical methods in natural language processing and computational natural language learning (EMNLP-CoNLL), pages 541–550. find our card
  26. Rhea Sukthanker, Soujanya Poria, Erik Cambria, and Ramkumar Thirunavukarasu. 2020. Anaphora and coreference resolution: A review. Information Fusion, 59:139–162. find
  27. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017,… find
  28. Wei Wu, Fei Wang, Arianna Yuan, Fei Wu, and Jiwei Li. 2019. Coreference resolution as query-based span prediction. arXiv preprint arXiv:1911.01746. — [arXiv:1911.01746](https://arxiv.org/abs/1911.01746) find
  29. Wei Yang, Rui Qiao, Haocheng Qin, Amy Sun, Luchen Tan, Kun Xiong, and Ming Li. 2019. End-to-end neural context reconstruction in Chinese dialogue. In Proceedings of the First Workshop on NLP for Conversational AI, pages 68–76, Florence, Italy. Association for Computational Linguistics. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/chen-2021-tackling-zero-pronoun-resolution-non/