Elamur Research⌕
☰
home / papers / 2019 / Where's My Head? Definition, Data Set, and Models for Numeric Fused-Head Identification and Resolution

Where's My Head? Definition, Data Set, and Models for Numeric Fused-Head Identification and Resolution

Paper · 2019 · Yanai Elazar and Yoav Goldberg

Summary

Numbers often appear without the noun they quantify. In "I'm 42" the noun is an age, and in "three at the most" it is a noun mentioned earlier. Humans recover these missing heads easily, but NLP systems mostly ignore them, which causes errors downstream, for example in machine translation. Elazar and Goldberg give the first computational treatment of these fused-head constructions, focusing on numeric fused heads (NFHs). They split the problem into two steps:

  1. Identification: decide whether a number span lacks its head.
  2. Resolution: recover that head. It is either a token elsewhere in the text (Reference cases) or an unstated concept inferred from context or world knowledge (Implicit cases).

The definition is deliberately broad. Anything that behaves like a headless number counts, whether linguists would call it ellipsis, pronominal "one", or something else. The authors build an accurate identification method and a crowd-sourced resolution corpus. They also propose a neural baseline adapted from the Lee et al. (2017) coreference model to handle implicit heads. NFHs turn out to be frequent across genres, and resolving them remains difficult, which makes this a useful open benchmark for implicit-meaning understanding.

Key points

  • NFHs cover 41.2% of numbers in a large English movie-dialog corpus. Their share in other corpora is:
    • TED talks: 35.5%
    • WikiText-2: 33.2%
    • WikiText-103: 32.9%
    • PTB: 22.2%
    • Amazon reviews: 37.5%
  • The resolution data set has about 10K annotated NFH examples. The paper gives its size as roughly 900K tokens in one place and 1M tokens in another.
  • Implicit heads are mostly assigned to one of five classes (YEAR, AGE, CURRENCY, TIME, PEOPLE) or to OTHER. These classes partly align with concepts in Abstract Meaning Representation.
  • The word "one" is the most frequent anchor, at 61% of occurrences. Its pronominal uses are treated as NFHs with the implicit head PEOPLE.
  • Code and data are released at github.com/yanaiela/num_fh.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Allen Institute for Artificial IntelligenceSeattle · United States275 works · 97,634 citations🎓Bar-Ilan UniversityRamat Gan · Israel141 works · 23,225 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Yanai Elazar and Yoav Goldberg. 2019. Where's My Head? Definition, Data Set, and Models for Numeric Fused-Head Identification and Resolution. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/tacl_a_00280

@article{elazar2019where,
  title = {{Where's My Head? Definition, Data Set, and Models for Numeric Fused-Head Identification and Resolution}},
  author = {Yanai Elazar and Yoav Goldberg},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.1162/tacl\_a\_00280},
  url = {https://doi.org/10.1162/tacl_a_00280}
}

References

49 references, 25 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mary Gardiner. 2003. Identifying and resolving oneanaphora. Unpublished Honours thesis, Macquarie University, November. find
  2. Omri Abend and Ari Rappoport. 2013. Universal conceptual cognitive annotation (UCCA). In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 228–238. Sofia.
  3. Matt Gardner, Joel Grus, Mark Neumann, Oyvind Tafjord, Pradeep Dasigi, Nelson F. Liu, Matthew Peters, Michael Schmitz, and Luke S. Zettlemoyer. 2017. Allennlp: A deep semantic natural language processing platform. arXiv preprint arXiv:1803.07640.
  4. Mauro Cettolo, Christian Girardi, and Marcello Federico. 2012, May. Wit 3 : Web inventory of transcribed and translated talks. In Proceedings of the 16 th Conference of the European Association for Machine Translation (EAMT), pages 261–268. Trento. find
  5. Matthew Gerber and Joyce Y. Chai. 2012. Semantic role labeling of implicit arguments for nominal predicates. Computational Linguistics, 38(4):755–798.
  6. Jiatao Gu, Zhengdong Lu, Hang Li, and Victor O. K. Li. 2016. Incorporating copying mechanism in sequence-to-sequence learning. arXiv preprint arXiv:1603.06393.
  7. Chen Chen and Vincent Ng. 2016. Chinese zero pronoun resolution with deep neural networks. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 778–788.
  8. Na-Rae Han. 2004. Korean null pronouns: Classification and annotation. In Proceedings of the 2004 ACL Workshop on Discourse Annotation, pages 33–40. find
  9. Danqi Chen and Christopher Manning. 2014. A fast and accurate dependency parser using neural networks. In Proceedings of the 2014
  10. Masatsugu Hangyo, Daisuke Kawahara, and Sadao Kurohashi. 2013. Japanese zero reference resolution considering exophora and author/reader mentions. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 924–934. find
  11. Mateusz Kopeć. 2014. Zero subject detection for Polish. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, volume 2: Short Papers, pages 221–225. find
  12. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation, 9(8):1735–1780. find
  13. George Lakoff and John Robert Ross. 1970. Gapping and the order of constituents. Progress in Linguistics: A Collection of Papers, 43:249. find
  14. Matthew Honnibal and Mark Johnson. 2015. An improved non-monotonic transition system for dependency parsing. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1373–1378, Lisbon.
  15. Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end neural coreference resolution. arXiv preprint arXiv: 1707.07045.
  16. Rodney Huddleston and Geoffrey K. Pullum. 2002. The Cambridge Grammar of English. Language. Cambridge: Cambridge University Press, pages 1–23. find
  17. Kenton Lee, Luheng He, and Luke S. Zettlemoyer. 2018. Higher-order coreference resolution with coarse-to-fine inference. In Proceedings of the 2018 Annual Conference of the North American Chapter of the Association for Computational Linguistics.
  18. Ryu Iida, Kentaro Inui, and Yuji Matsumoto. 2006. Exploiting syntactic patterns as clues in zero-anaphora resolution. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 625–632.
  19. Iddo Lev, Bill MacCartney, Christopher D Manning, and Roger Levy. 2004. Solving logic puzzles: From robust processing to precise semantics. In Proceedings of the 2nd Workshop on Text Meaning and Interpretation, pages 9–16. find
  20. Ryu Iida, Kentaro Torisawa, Jong-Hoon Oh, Canasai Kruengkrai, and Julien Kloetzer. 2016. Intra-sentential subject zero anaphora resolution using multi-column convolutional neural network. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1244–1254.
  21. Zhengzhong Liu, Edgar Gonzàlez Pellicer, and Daniel Gillick. 2016. Exploring the steps of verb phrase ellipsis. In CORBON@ HLT-NAACL, pages 32–40.
  22. Ross John. 1969. Guess who. In Proceedings of the 5th Chicago Linguistic Society, pages 252–286. find
  23. Anne C. Lobeck. 1995. Ellipsis: Functional Heads, Licensing, and Identification, Oxford University Press on Demand. find
  24. Kian Kenyon-Dean, Jackie Chi Kit Cheung, and Doina Precup. 2016. Verb phrase ellipsis resolution using discriminative and margin-infused algorithms. In Proceedings of EMNLP, pages 1734–1743.
  25. Mitchell P. Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. 1993. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19(2):313–330.
  26. Diederik P. Kingma and Lei Ba. 2015. J. Adam: A method for stochastic optimization. In International Conference on Learning Representations.
  27. Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2016. Pointer sentinel mixture models. arXiv preprint arXiv: 1609.07843.
  28. Fang Kong and Guodong Zhou. 2010. A tree kernel-based unified framework for Chinese zero anaphora resolution. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 882–891.
  29. Claudiu Mihăilă, Iustina Ilisei, and Diana Inkpen. 2010. To be or not to be a zero pronoun: A machine learning approach for romanian. Multilinguality and Interoperability in Language Processing with Emphasis on Romanian, 303–316. find
  30. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems, pages 3111–3119.
  31. Lee, and Luke S. Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Annual Conference of the North American Chapter of the Association for Computational Linguistics.
  32. Marta Recasens, Zhichao Hu, and Olivia Rhinehart. 2016. Sense anaphoric pronouns: Am i one? In CORBON@ HLT-NAACL, pages 1–6.
  33. Tatjana Moor, Michael Roth, and Anette Frank. 2013. Predicate-specific annotations for implicit role binding: Corpus annotation, data analysis and evaluation experiments. In Proceedings of the 10th International Conference on Computational Semantics (IWCS), pages 369–375, Potsdam. find
  34. Subhro Roy and Dan Roth. 2015. Solving general arithmetic word problems. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1743–1752.
  35. Hwee Tou Ng, Yu Zhou, Robert Dale, and Mary Gardiner. 2005. A machine learning approach to identification and resolution of one-anaphora. In International Joint Conference on Artificial Intelligence, volume 19, page 1105. find
  36. Subhro Roy, Tim Vieira, and Dan Roth. 2015. Reasoning about quantities in natural language. Transactions of the Association for Computational Linguistics, 3:1–13. find
  37. Vincent Ng. 2010. Supervised noun phrase coreference research: The first fifteen years. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 1396–1411.
  38. Georgios P. Spithourakis and Sebastian Riedel. 2018. Numeracy for language models: Evaluating and improving their ability to predict numbers. arXiv preprint arXiv:1805.08154. find
  39. Rhea Sukthanker, Soujanya Poria, Erik Cambria, and Ramkumar Thirunavukarasu. 2018. Anaphora and coreference resolution: A review. arXiv preprint arXiv:1805.11824. find
  40. Tadashi Nomoto and Yoshihiko Nitta. 1993. Resolving zero anaphora in japanese. In Proceedings of the Sixth Conference on European Chapter of the Association for Computational Linguistics, pages 315–321. find
  41. Andrew Trask, Felix Hill, Scott Reed, Jack Rae, Chris Dyer, and Phil Blunsom. 2018. Neural arithmetic logic units. arXiv preprint arXiv:1808.00508. find
  42. Sun-Young Oh. 2005. English zero anaphora as an interactional resource. Research on Language and Social Interaction, 38(3):267–302. find
  43. Ralph Weischedel, Sameer Pradhan, Lance Ramshaw, Martha Palmer, Nianwen Xue, Mitchell Marcus, Ann Taylor, Craig Greenberg, Eduard Hovy, and Robert Belvin. 2011. Ontonotes release 4.0. LDC2011T03, Philadelphia, PA: Linguistic Data Consortium. find
  44. Antonio Pareja-Lora, Maria Liakata, and Stefanie Dipper. 2013. Proceedings of the 7th linguistic annotation workshop and interoperability with discourse. In Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discourse. find
  45. Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel,Peter Prettenhofer, Ron Weiss, and Vincent Dubourg. 2011. Scikit-learn: Machine learning in python. Journal of Machine Learning Research, 12(Oct):2825–2830. find
  46. Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Ching-Long Yeh and Yi-Jun Chen. 2001. An empirical study of zero anaphora resolution in chinese based on centering model. In Proceedings of Research on Computational Linguistics Conference XIV, pages 237–251. find
  47. Qingyu Yin, Yu Zhang, Wei-Nan Zhang, Ting Liu, and William Yang Wang. 2018a. Deep reinforcement learning for Chinese zero pronoun resolution. In Proceedings of the 56th Annual Meeting of the Association for Type Computational Linguistics (Volume 1: Long Papers), pages 569–578.
  48. Qingyu Yin, Yu Zhang, Weinan Zhang, Ting Liu, and William Yang Wang. 2018b. Zero pronoun resolution with attention-based neural network. In Proceedings of the 27th International Conference on Computational Linguistics, pages 13–23.
  49. Labels Structure Rui Zhang, Cicero Nogueira dos Santos, Michihiro Yasunaga, Bing Xiang, and Dragomir Radev. 2018. Neural coreference resolution with deep biaffine attention by joint mention detection and mention clustering. arXiv preprint arXiv:1805.04893.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/elazar-2019-where-s-my-head-definition/