Elamur Research⌕
☰
home / papers / 2018 / Enhanced Word Representations for Bridging Anaphora Resolution

Enhanced Word Representations for Bridging Anaphora Resolution

Summary

Bridging anaphora resolution links an anaphor to an earlier antecedent through a non-identity relation, such as part-of, employment, or an encyclopedic association. Standard embeddings like GloVe mostly encode similarity between synonyms or words of the same type, while this task depends on relatedness between associated nouns. Hou builds a GloVe-based "PP context model" from Gigaword. Head–modifier noun pairs are taken from prepositional NPs (X of Y) and possessive NPs (Y's X), and each head is tagged with a "_PP" suffix so the vectors keep the asymmetry of the relation.

The resulting vectors, embeddings PP, are evaluated on the ISNotes corpus. A simple rule picks the candidate with the highest cosine similarity to the anaphor, and this alone reaches about 30% accuracy, well above standard GloVe vectors. Adding a single embedding-based constraint to the Markov logic system MLN model II (Hou et al., 2013b) raises its accuracy significantly.

The work matters because it produces a reusable, efficiently computable lexical resource for associative relations. Such a resource is hard to obtain from vanilla distributional embeddings.

Key points

  • Training data and setup: about 197 million noun pairs from automatically parsed Gigaword were treated as two-word "sentences."
    • Vocabulary: roughly 276k entries, including 188k distinct nouns without the suffix.
    • Settings: 100 dimensions, context window of 2, and words seen fewer than 10 times removed.
  • Evaluation data: ISNotes covers 50 WSJ texts from OntoNotes, with about 11,000 NPs and 663 bridging anaphors.
    • Relation types: 14% part-of/attribute-of, 7% set, and 79% "other."
  • Candidate selection: candidates come from the same sentence, the two previous sentences, and the text's first sentence, giving about 19 per anaphor on average.
  • Accuracy with embeddings alone:
    • GloVe GigaWiki14: 18.10%
    • GloVe Giga: 19.00%
    • Same noun pairs without the "_PP" suffix: 22.17%
    • embeddings PP: 30.32%
    • The suffix is therefore the decisive factor.
  • Accuracy within MLN model II: the baseline scores 41.32%, rising to 45.85% with embeddings PP (significant, p < 0.01).
    • For comparison, adding GloVe constraints gives 39.52%, and adding the suffix-free vectors gives 40.42%.
    • In this setup, the NP vector averages the head with the common nouns that precede it.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

14 papers of our library cite it (of the 140673 whose bibliographies we read); 7 of them have a page here, 7 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/n18-2001 · arXiv 1803.04790

@inproceedings{hou2018enhanced,
  title = {{Enhanced Word Representations for Bridging Anaphora Resolution}},
  author = {Hou, Yufang},
  booktitle = {Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)},
  year = {2018},
  doi = {10.18653/v1/n18-2001},
  eprint = {1803.04790},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/n18-2001}
}

References

25 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Danqi Chen, Jason Bolton, and Christopher D. Manning. 2016. A thorough examination of the CNN/Daily mail reading comprehension task. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Germany, 7–12 August 2016. pages 2358–2367. find our card
  2. Herbert H. Clark. 1975. Bridging. In Proceedings of the Conference on Theoretical Issues in Natural Language Processing, Cambridge, Mass., June 1975. pages 169–174. 3 embeddings PP can be downloaded from https://doi.org/10.5281/zenodo.1211616 — [doi:10.5281/zenodo.1211616](https://doi.org/10.5281/zenodo.1211616) ↗
  3. Emmanuel Lassalle and Pascal Denis. 2011. Leveraging different meronym discovery methods for bridging resolution in French. In Proceedings of the 8th Discourse Anaphora and Anaphor Resolution Colloquium (DAARC 2011), Faro, Algarve, Portugal, 6– 7 October 2011. pages 35–46. find
  4. Pedro Domingos and Daniel Lowd. 2009. Markov Logic: An Interface Layer for Artificial Intelligence. Morgan Claypool Publishers. find
  5. Jeanette K. Gundel, Nancy Hedberg, and Ron Zacharski. 1993. Cognitive status and the form of referring expressions in discourse. Language 69:274–307. find
  6. Omer Levy and Yoav Goldberg. 2014. Dependency-based word embeddings. In Proceedings of the 52st Annual Meeting of the Association for Computational Linguistics, Baltimore, USA, 22–27 June 2014. find
  7. Udo Hahn, Michael Strube, and Katja Markert. 1996. Bridging textual ellipses. In Proceedings of the 16th International Conference on Computational Linguistics, Copenhagen, Denmark, 5–9 August 1996. volume 1, pages 496–501. http://www.aclweb.org/anthology/C96-1084.pdf. ↗ our card
  8. Katja Markert, Yufang Hou, and Michael Strube. 2012. Collective classification for fine-grained information status. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, Jeju Island, Korea, 8–14 July 2012. pages 795–804. http://www.aclweb.org/ anthology/P12-1084… ↗ our card
  9. Zellig S. Harris. 1954. Distributional structure. Word 10:146–162. find
  10. Benjamin Heinzerling, Nafise Sadat Moosavi, and Michael Strube. 2017. Revisiting selectional preferences for coreference resolution. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark, 7–11 November 2017. pages 1332–1339. find our card
  11. Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems 26 (NIPS 2013), pages 3111–3119. find
  12. Yufang Hou. 2016. Incremental fine-grained information status classification using attention-based LSTMs. In Proceedings of the 26th International Conference on Computational Linguistics, Osaka, Japan, 11–16 December 2016. pages 1880–1890. find
  13. Courtney Napoles, Matthew Gormley, and Benjamin Van Durme. 2012. Annotated Gigaword. In Proceedings of the Joint Workshop on Automatic Knowledge Base Construction & Web-scale Knowledge Extraction (AKBC-WEKEX) Montréal, Québec, Canada, 7-8 June 2012. pages 95–100. find
  14. Yufang Hou, Katja Markert, and Michael Strube. 2013a. Cascading collective classification for bridging anaphora recognition using a rich linguistic feature set. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, Wash., 18–21 October 2013. pages 814–82… find our card
  15. Robert Parker, David Graff, Junbo Kong, Ke Chen, and Kazuaki Maeda. 2011. English Gigaword Fifth Edition. LDC2011T07. find
  16. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Qatar, 25–29 October 2014. pages 1532–1543. find
  17. Yufang Hou, Katja Markert, and Michael Strube. 2013b. Global inference for bridging anaphora resolution. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Atlanta, Georgia, 9–14 June 2013. pages 907–917.… find our card
  18. Massimo Poesio, Rahul Mehta, Axel Maroudas, and Janet Hitzeman. 2004. Learning to resolve bridging references. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics, Barcelona, Spain, 21–26 July 2004. pages 143–150. find our card
  19. Yufang Hou, Katja Markert, and Michael Strube. 2014. A rule-based system for unrestricted bridging resolution: Recognizing bridging anaphora and finding links to antecedents. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Qatar, 25–29 October 2014. p… find our card
  20. Massimo Poesio, Renata Vieira, and Simone Teufel. 1997. Resolving bridging references in unrestricted text. In Proceedings of the ACL Workshop on Operational Factors in Practical, Robust Anaphora Resolution for Unrestricted Text, Madrid, Spain, July 1997. pages 1–6. find our card
  21. Douwe Kiela, Felix Hill, and Stephen Clark. 2015. Specializing word embeddings for similarity or relatedness. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal, 17–21 September 2015. pages 2044–2048. find
  22. Ellen F. Prince. 1981. Towards a taxonomy of given-new information. In P. Cole, editor, Radical Pragmatics, Academic Press, New York, N.Y., pages 223–255. find
  23. Tim Rocktäschel, Edward Grefenstette, Karl Moritz Hermann, Tomas Kocisky, and Phil Blunsom. 2016. Reasoning about entailment with neural attention. In Proceedings of the 4th International Conference on Learning Representations, San Juan, Puerto Rico, 2-4 May 2016. find our card
  24. Roy Schwartz, Roi Reichart, and Ari Rappoport. 2016. Symmetric patterns and coordinations: Fast and enhanced representations of Verbs and Adjectives. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San… find
  25. Ralph Weischedel, Martha Palmer, Mitchell Marcus, Eduard Hovy, Sameer Pradhan, Lance Ramshaw, Nianwen Xue, Ann Taylor, Jeff Kaufman, Michelle Franchini, Mohammed El-Bachouti, Robert Belvin, and Ann Houston. 2011. OntoNotes release 4.0. LDC2011T03, Philadelphia, Penn.: Linguistic Data Consortium. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/hou-2018-enhanced-word-representations-bridging-anaphora/