Elamur Research⌕
☰
home / papers / 2020 / Multitask Learning-Based Neural Bridging Reference Resolution

Multitask Learning-Based Neural Bridging Reference Resolution

Summary

Bridging reference resolution connects an anaphor to an antecedent that is associated with it but not identical to it, such as "the customers" referring back to a previously mentioned supermarket. Two obstacles have limited this task. Annotated corpora are small, and they define bridging differently, so the best earlier systems were rule-based and tied to a single corpus. Yu and Poesio address both problems with a neural model adapted from the coreference architecture of Lee et al. (2018) and Kantor and Globerson (2019), using BERT and GloVe embeddings. A coreference classifier is trained alongside the bridging classifier and shares part of the network with it, so abundant coreference annotations make up for scarce bridging data. Undersampling corrects the resulting class imbalance, and only a few corpus-independent features are used. The authors tested the same architecture on ARRAU (RST, TRAINS, PEAR), ISNOTES, BASHI and SCICORP, and it set a new state of the art on full bridging resolution for all of them. The results suggest that coreference is an effective auxiliary task and that a single learned model can handle different annotation schemes and domains.

Key points

  • Multi-task gains on ARRAU RST: Joint training with coreference raised scores by 6.5–7.3 p.p. over the single-task baseline, both for the complete bridging task and for its two component steps (anaphor recognition, antecedent selection). Undersampling added a further 1.7 p.p. on full resolution and 6.6 p.p. on anaphor recognition.
  • ARRAU RST versus prior best: The final system beat Roesiger (2018) by 4.5 p.p. on full resolution, 6 p.p. on anaphor recognition and 9.5 p.p. on antecedent selection. Across corpora, the largest gain over earlier reported results reached 36.3 p.p.
  • Data scarcity: ARRAU, the largest bridging corpus, has only about 5.5k bridging pairs, and ISNOTES has 663. On ISNOTES, the system led on full resolution and anaphor recognition. It trailed only systems pre-trained on large synthetic bridging corpora (Hou 2018a; Hou 2020) for antecedent selection.
  • SCICORP (scientific text): Using predicted NP mentions, the system reached F1 scores of 21.8% on full resolution and 46.3% on anaphor recognition, compared with 1.5% and 8.1% for the earlier rule-based system. It also reported the first antecedent selection result on this corpus (33.4% accuracy).
  • BASHI and code: On BASHI, antecedent selection accuracy was 36%, 8.6 p.p. above Hou (2018a). Code is available at github.com/juntaoy/dali-bridging.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

16 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 10 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.coling-main.315

@inproceedings{yu2020multitask,
  title = {{Multitask Learning-Based Neural Bridging Reference Resolution}},
  author = {Yu, Juntao and Poesio, Massimo},
  booktitle = {Proceedings of the 28th International Conference on Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.coling-main.315},
  url = {https://doi.org/10.18653/v1/2020.coling-main.315}
}

References

50 references, 24 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Stefan Baumann and Arndt Riester. 2012. Referential and lexical givenness: semantic, prosodic and cognitive aspects. In Prosody and Meaning. find
  2. Kevin Clark and Christopher D. Manning. 2016. Improving coreference resolution by learning entity-level distributed representations. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 643–653, Berlin, Germany, August. Association… find our card
  3. Kevin Clark, Minh-Thang Luong, Urvashi Khandelwal, Christopher D. Manning, and Quoc V. Le. 2019. BAM! born-again multi-task networks for natural language understanding. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5931–5937, Florence, Italy, July.… find
  4. Herbert H. Clark. 1975. Bridging. In Theoretical Issues in Natural Language Processing. find
  5. Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In ICML. find
  6. Ryan Cotterell and Kevin Duh. 2017. Low-resource named entity recognition with cross-lingual, character-level neural conditional random fields. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 91–96, Taipei, Taiwan, November.… find
  7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find
  8. Alan Garnham. 2001. Mental models and the interpretation of anaphora. Psychology Press. find
  9. Jerry R. Hobbs, Mark Stickel, Doug Appelt, and Paul Martin. 1993. Interpretation as abduction. Artificial Intelligence Journal, 63:69–142. find
  10. Yufang Hou, Katja Markert, and Michael Strube. 2013. Global inference for bridging anaphora resolution. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 907–917, Atlanta, Georgia, June. Associatio… find our card
  11. Yufang Hou, Katja Markert, and Michael Strube. 2014. A rule-based system for unrestricted bridging resolution: Recognizing bridging anaphora and finding links to antecedents. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2082–2093, Doha, Qat… find our card
  12. Yufang Hou, Katja Markert, and Michael Strube. 2018. Unrestricted bridging resolution. Computational Linguistics, 44(2):237–284, June. find our card
  13. Yufang Hou. 2016. Incremental fine-grained information status classification using attention-based LSTMs. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1880–1890, Osaka, Japan, December. find
  14. The COLING 2016 Organizing Committee. find
  15. Yufang Hou. 2018a. A deterministic algorithm for bridging anaphora resolution. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1938–1948, Brussels, Belgium, October-November. Association for Computational Linguistics. find our card
  16. Yufang Hou. 2018b. Enhanced word representations for bridging anaphora resolution. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 1–7, New Orleans, Louisiana, June. Asso… find our card
  17. Yufang Hou. 2020. Bridging anaphora resolution as question answering. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1428–1438, Online, July. Association for Computational Linguistics. find our card
  18. Hans Kamp and Uwe Reyle. 1993. From Discourse to Logic. D. Reidel, Dordrecht. find
  19. Ben Kantor and Amir Globerson. 2019. Coreference resolution with entity equalization. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 673–677, Florence, Italy, July. Association for Computational Linguistics. find our card
  20. Lauri Karttunen. 1976. Discourse referents. In Syntax and Semantics 7 - Notes from the Linguistic Underground, pages 363–385. Academic Press, New York. find
  21. Eliyahu Kiperwasser and Miguel Ballesteros. 2018. Scheduled multi-task learning: From syntax to translation. Transactions of the Association for Computational Linguistics, 6:225–240. find our card
  22. Varada Kolhatkar, Adam Roussel, Stefanie Dipper, and Heike Zinsmeister. 2018. Anaphora with non-nominal antecedents in computational linguistics: a Survey. Computational Linguistics, 44(3):547–612. find our card
  23. Emmanuel Lassalle and Pascal Denis. 2011. Leveraging different meronym discovery methods for bridging resolution in french. In Proc. of 8th DAARC, pages 35–46, Faro, Portugal. find
  24. Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end neural coreference resolution. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 188–197, Copenhagen, Denmark, September. Association for Computational Linguistics. find our card
  25. Kenton Lee, Luheng He, and Luke Zettlemoyer. 2018. Higher-order coreference resolution with coarse-to-fine inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 687–… find our card
  26. Minh-Thang Luong, Quoc V Le, Ilya Sutskever, Oriol Vinyals, and Lukasz Kaiser. 2016. Multi-task sequence to sequence learning. ICLR. find our card
  27. Katja Markert, Yufang Hou, and Michael Strube. 2012. Collective classification for fine-grained information status. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 795–804, Jeju Island, Korea, July. Association for Computation… find our card
  28. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Doha, Qatar, October. Association for Computational Linguistics. find
  29. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolog… find
  30. Massimo Poesio and Ron Artstein. 2008. Anaphoric annotation in the ARRAU corpus. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08), Marrakech, Morocco, May. European Language Resources Association (ELRA). find our card
  31. Massimo Poesio, Rahul Mehta, Axel Maroudas, and Janet Hitzeman. 2004. Learning to resolve bridging references. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04), pages 143–150, Barcelona, Spain, July. find our card
  32. Massimo Poesio, Roland Stuckardt, and Yannick Versley. 2016. Anaphora Resolution: Algorithms, Resources and Applications. Springer, Berlin. find
  33. Massimo Poesio, Yulia Grishina, Varada Kolhatkar, Nafise Moosavi, Ina Roesiger, Adam Roussel, Fabian Simonjetz, Alexandra Uma, Olga Uryupina, Juntao Yu, and Heike Zinsmeister. 2018. Anaphora resolution with the ARRAU corpus. In Proceedings of the First Workshop on Computational Models of Reference,… find our card
  34. Massimo Poesio. 2004. Discourse annotation and semantic annotation in the GNOME corpus. In Proc. of the ACL Workshop on Discourse Annotation. find
  35. Massimo Poesio. 2016. Linguistic and cognitive evidence about anaphora. In M. Poesio, R. Stuckardt, and Y. Versley, editors, Anaphora Resolution: Algorithms, Resources and Applications, chapter 2. Springer. find
  36. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. CoNLL-2012 shared task: Modeling multilingual unrestricted coreference in OntoNotes. In Proceedings of the Sixteenth Conference on Computational Natural Language Learning (CoNLL 2012), Jeju, Korea. find our card
  37. Ellen F. Prince. 1981. Toward a taxonomy of given-new information. In P. Cole, editor, Radical Pragmatics, pages 223–256. Academic Press, New York. find
  38. Ina Roesiger, Arndt Riester, and Jonas Kuhn. 2018. Bridging resolution: Task definition, corpus resources and rule-based experiments. In Proceedings of the 27th International Conference on Computational Linguistics, pages 3516–3528, Santa Fe, New Mexico, USA, August. Association for Computational Li… find our card
  39. Ina Roesiger. 2018. Rule- and learning-based methods for bridging resolution in the ARRAU corpus. In Proceedings of the First Workshop on Computational Models of Reference, Anaphora and Coreference, pages 23–33, New Orleans, Louisiana, June. Association for Computational Linguistics. find our card
  40. Ina Rösiger. 2016. SciCorp: A corpus of English scientific articles annotated for information status analysis. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 1743–1749, Portorož, Slovenia, May. European Language Resources Association (ELR… find our card
  41. Ina Rösiger. 2018. BASHI: A corpus of Wall Street Journal articles annotated with bridging links. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan, May. European Language Resources Association (ELRA). find our card
  42. Candace L. Sidner. 1979. Towards a computational theory of definite anaphora comprehension in English discourse. Ph.D. thesis, MIT. find
  43. Olga Uryupina, Ron Artstein, Antonella Bristot, Federica Cavicchio, Francesca Delogu, Kepa J. Rodriguez, and Massimo Poesio. 2019. Annotating a broad range of anaphoric phenomena, in a variety of genres: the ARRAU corpus. Journal of Natural Language Engineering. find
  44. Renata Vieira and Massimo Poesio. 2000. An empirically-based system for processing definite descriptions. Computational Linguistics, 26(4):539–593, December. find
  45. Bonnie L. Webber. 1979. A Formal Approach to Discourse Anaphora. Garland, New York. find
  46. Bonnie L. Webber. 1991. Structure and ostension in the interpretation of discourse deixis. Language and Cognitive Processes, 6(2):107–135. find
  47. Sam Wiseman, Alexander M. Rush, Stuart Shieber, and Jason Weston. 2015. Learning anaphoricity and antecedent ranking features for coreference resolution. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural L… find our card
  48. Sam Wiseman, Alexander M. Rush, and Stuart M. Shieber. 2016. Learning global features for coreference resolution. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 994–1004, San Diego, California,… find our card
  49. Zhilin Yang, Ruslan Salakhutdinov, and William W Cohen. 2017. Transfer learning for sequence tagging with hierarchical recurrent networks. ICLR. find
  50. Joey Tianyi Zhou, Hao Zhang, Di Jin, Hongyuan Zhu, Meng Fang, Rick Siow Mong Goh, and Kenneth Kwok. 2019. Dual adversarial neural transfer for low-resource named entity recognition. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3461–3471, Florence,… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/yu-2020-multitask-learning-based-neural-bridging/