Elamur Research⌕
☰
home / papers / 2022 / What does it take to bake a cake? The RecipeRef corpus and anaphora resolution in procedural text

What does it take to bake a cake? The RecipeRef corpus and anaphora resolution in procedural text

Summary

Most anaphora research uses declarative genres such as news or dialogue. Procedural text, where readers must track which entity each step acts on and how it changes, has received little attention. The authors address this for cooking recipes. They first compare the textual properties of recipes and chemical patents. They then adapt an anaphora annotation scheme originally designed for chemical patents so that it covers recipe instructions sampled from RecipeDB. The resulting RecipeRef corpus marks both coreference and bridging links between ingredient-related mentions, using four relation types that separate links with and without a change of state. Two annotators familiar with the domain reached high agreement. For modelling, an end-to-end span-based system is trained jointly on coreference and bridging. A transfer setting pretrains this model on the chemical ChEMU-ref corpus before fine-tuning on recipes. Joint training helps, and the chemical pretraining adds consistent further gains. This suggests that procedural knowledge carries over between domains. The work matters because it supplies a public resource and a baseline for entity tracking in instructional text.

Key points

  • RecipeRef draws on the instruction sections of RecipeDB (118,171 English recipes). Only ingredient terms and referring expressions that take part in anaphoric links are annotated, so singletons are excluded. Premodified phrases such as "chopped chicken" and bare numbers that refer to ingredients count as mentions.
  • The relation types are COREFERENCE, TRANSFORMED, INGREDIENT(WITH-STATE-CHANGE)-ASSOCIATED and INGREDIENT(WITHOUT-STATE-CHANGE)-ASSOCIATED. They are modelled in two task variants, one that considers state change and one that ignores it.
  • The transfer setup pretrains on ChEMU-ref (1,500 chemical patent excerpts) and then fine-tunes on recipes. Under joint training this reaches 27.9% F1 for relation prediction, an absolute gain of 0.8 points. The component models improve by 0.5 F1 (surface coreference) and 0.7 F1 (bridging).
  • In error analysis, the model often labels mixture-denoting mentions as plain COREFERENCE and misses state changes. It also over-predicts the dominant with-state-change bridging class, while coreference is rare in the data.
  • The dataset and code are public (GitHub: biaoyanf/RecipeRef; Mendeley Data DOI 10.17632/rcyskfvdv7.1).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

21 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 17 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2022.findings-acl.275

@inproceedings{fang2022what,
  title = {{What does it take to bake a cake? The RecipeRef corpus and anaphora resolution in procedural text}},
  author = {Fang, Biaoyan and Baldwin, Timothy and Verspoor, Karin},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2022},
  year = {2022},
  doi = {10.18653/v1/2022.findings-acl.275},
  url = {https://doi.org/10.18653/v1/2022.findings-acl.275}
}

References

66 references, 31 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Rahul Agarwal and Kevin Miller. 2011. Information extraction from recipes. Department of Computer Science, Stanford University-2008. find
  2. Nicholas Asher and Alex Lascarides. 1998. Bridging. Journal of Semantics, 15(1):83–113. find
  3. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In 3rd International Conference on Learning Representations (ICLR 2015), San Diego, USA. find our card
  4. Devansh Batra, Nirav Diwan, Utkarsh Upadhyay, Jushaan Singh Kalra, Tript Sharma, Aman Kumar Sharma, Dheeraj Khanna, Jaspreet Singh Marwah, Srilakshmi Kalathil, Navjot Singh, Rudraksh Tuwani, and Ganesh Bagler. 2020. RecipeDB: A resource for exploring recipes. find
  5. Arthur Brack, Daniel Uwe Müller, Anett Hoppe, and Ralph Ewerth. 2021. Coreference resolution in research papers from multiple domains. In Proc. of the 43rd European Conference on Information Retrieval, online. find
  6. Kevin Clark and Christopher D Manning. 2015. Entity-centric coreference resolution with model stacking. find our card
  7. In Proc. of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1405– 1415, Beijing, China. find
  8. Kevin Clark and Christopher D. Manning. 2016a. Deep reinforcement learning for mention-ranking coreference models. In Proc. of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2256–2262, Austin, USA. find our card
  9. Kevin Clark and Christopher D. Manning. 2016b. Improving coreference resolution by learning entity-level distributed representations. In Proc. of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 643–653, Berlin, Germany. find our card
  10. K Bretonnel Cohen, Arrick Lanfranchi, Miji Joo-young Choi, Michael Bada, William A Baumgartner, Natalya Panteleyeva, Karin Verspoor, Martha Palmer, and Lawrence E Hunter. 2017. Coreference annotation and resolution in the Colorado Richly Annotated Full Text (CRAFT) corpus of biomedical journal artic… find
  11. Zeyu Dai, Hongliang Fei, and Ping Li. 2019. Coreference aware representation learning for neural named entity recognition. In IJCAI, pages 4946–4953. find
  12. Bhavana Dalvi, Lifu Huang, Niket Tandon, Wen tau Yih, and Peter Clark. 2018. Tracking state changes in procedural text: A challenge dataset and models for process paragraph comprehension. In NAACL. find
  13. Biaoyan Fang, Christian Druckenbrodt, Saber A Akhondi, Jiayuan He, Timothy Baldwin, and Karin Verspoor. 2021a. ChEMU-ref: A corpus for modeling anaphora resolution in the chemical domain. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics:… find our card
  14. Biaoyan Fang, Christian Druckenbrodt, Saber A. Akhondi, Camilo Thorne, Timothy Baldwin, and Karin Verspoor. 2022. RecipeRef corpus for modeling anaphora resolution from the procedural text of recipes. Mendeley Data. find
  15. Biaoyan Fang, Christian Druckenbrodt, Colleen Yeow Hui Shiuan, Sacha Novakovic, Ralph Hössel, Saber A. Akhondi, Jiayuan He, Meladel Mistica, Timothy Baldwin, and Karin Verspoor. 2021b. ChEMU-Ref dataset for modeling anaphora resolution in the chemical domain. Mendeley Data. find
  16. Annemarie Friedrich, Heike Adel, Federico Tomazic, Johannes Hingerl, Renou Benteau, Anika Marusczyk, and Lukas Lange. 2020. The SOFC-exp corpus and neural approaches to information extraction in the materials science domain. In Proceedings of the 58th Annual Meeting of the Association for Computatio… find
  17. Abbas Ghaddar and Phillippe Langlais. 2016. Wiki-Coref: An English coreference-annotated corpus of Wikipedia articles. In Proc. of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 136–142, Portorož, Slovenia. find our card
  18. Loïc Grobol. 2019. Neural coreference resolution with limited lexical context and explicit mention detection for oral French. In Proc. of the Second Workshop on Computational Models of Reference, Anaphora and Coreference, pages 8–14, Minneapolis, USA. find our card
  19. Jun Harashima, Michiaki Ariga, Kenta Murata, and Masayuki Ioki. 2016. A large-scale recipe and meal data collection as infrastructure for food research. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 2455–2459, Portorož, Slovenia. European… find
  20. Jun Harashima and Makoto Hiramatsu. 2020. Cookpad parsed corpus: Linguistic annotations of Japanese recipes. In Proceedings of the 14th Linguistic Annotation Workshop, pages 87–92, Barcelona, Spain. Association for Computational Linguistics. find
  21. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation, 9(8):1735– 1780. find
  22. Yufang Hou. 2018a. A deterministic algorithm for bridging anaphora resolution. In Proc. of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1938–1948, Brussels, Belgium. find our card
  23. Yufang Hou. 2018b. Enhanced word representations for bridging anaphora resolution. In Proc. of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 1–7, New Orleans, USA. find our card
  24. Yufang Hou. 2020. Bridging anaphora resolution as question answering. In Proc. of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1428–1438, Online. find our card
  25. Yufang Hou, Katja Markert, and Michael Strube. 2014. A rule-based system for unrestricted bridging resolution: Recognizing bridging anaphora and finding links to antecedents. In Proc. of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP 2014), pages 2082–2093, Doha, Qata… find our card
  26. Yufang Hou, Katja Markert, and Michael Strube. 2018. Unrestricted bridging resolution. Computational Linguistics, 44(2):237–284. find our card
  27. De-An Huang, Joseph J Lim, Li Fei-Fei, and Juan Carlos Niebles. 2017. Unsupervised visual-linguistic reference resolution in instructional videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2183–2192. find
  28. Yiwei Jiang, Klim Zaporojets, Johannes Deleu, Thomas Demeester, and Chris Develder. 2020. Recipe instruction semantics corpus (RISeC): Resolving semantic structure and zero anaphora in recipes. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Ling… find
  29. Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, and Omer Levy. 2020. Span-BERT: Improving pre-training by representing and predicting spans. Transactions of the Association for Computational Linguistics, 8:64–77. find our card
  30. Ben Kantor and Amir Globerson. 2019. Coreference resolution with entity equalization. In Proc. of the 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019), pages 673–677, Florence, Italy. find our card
  31. Chloé Kiddon, Ganesa Thandavam Ponnuraj, Luke Zettlemoyer, and Yejin Choi. 2015. Mise en place: Unsupervised interpretation of instructional recipes. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 982–992, Lisbon, Portugal. Association for Computatio… find
  32. Javier Marin, Aritro Biswas, Ferda Ofli, Nicholas Hynes, Amaia Salvador, Yusuf Aytar, Ingmar Weber, and Antonio Torralba. 2019. Recipe1m+: A dataset for learning cross-modal embeddings for cooking recipes and food images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(1):187–203. find
  33. Ashutosh Modi, Tatjana Anikina, Simon Ostermann, and Manfred Pinkal. 2016. InScript: Narrative texts annotated with script information. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 3485– 3493, Portorož, Slovenia. European Language Resourc… find
  34. Nafise Sadat Moosavi and Michael Strube. 2016. Which coreference evaluation metric do you trust? a proposal for a link-based entity aware metric. In Proc. of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 632–642, Berlin, Germany. find our card
  35. Shinsuke Mori, Hirokuni Maeta, Yoko Yamakata, and Tetsuro Sasada. 2014. Flow graph corpus from recipe texts. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 2370–2377, Reykjavik, Iceland. European Language Resources Association (ELRA). find
  36. Jin-Dong Kim, Ngan Nguyen, Yue Wang, Jun’ichi Tsujii, Toshihisa Takagi, and Akinori Yonezawa. 2012. The Genia event and protein coreference tasks of the BioNLP shared task 2011. BMC Bioinformatics, 13(11):S1. find
  37. Sheshera Mysore, Zachary Jensen, Edward Kim, Kevin Huang, Haw-Shiuan Chang, Emma Strubell, Jeffrey Flanigan, Andrew McCallum, and Elsa Olivetti. 2019. The materials science procedural text corpus: Annotating materials synthesis procedures with shallow semantic structures. In Proceedings of the 13th… find
  38. Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end neural coreference resolution. In Proc. of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 188– 197, Copenhagen, Denmark. find our card
  39. Vinod Nair and Geoffrey E Hinton. 2010. Rectified linear units improve restricted Boltzmann machines. In Proc. of the 33rd International Conference on Machine Learning (ICML 2016), New York, USA. find
  40. Kenton Lee, Luheng He, and Luke Zettlemoyer. 2018. Higher-order coreference resolution with coarse-to-fine inference. In Proc. of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 687–692, N… find our card
  41. Yi Luan, Luheng He, Mari Ostendorf, and Hannaneh Hajishirzi. 2018. Multi-task identification of entities, relations, and coreference for scientific knowledge graph construction. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3219–3232, Brussels, Belg… find our card
  42. Xiaoqiang Luo. 2005. On coreference resolution performance metrics. In Proc. of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing (EMNLP 2005), pages 25–32, Vancouver, Canada. find our card
  43. Vincent Ng. 2017. Machine learning for entity coreference resolution: A retrospective look at two decades of research. In Proc. of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI’17), pages 4877–4884, San Francisco, USA. find
  44. Ngan Nguyen, Jin-Dong Kim, and Jun’ichi Tsujii. 2011. Overview of BioNLP 2011 protein coreference shared task. In Proc. of BioNLP Shared Task 2011 Workshop, pages 74–82, Portland, USA. find our card
  45. Taichi Nishimura, Suzushi Tomori, Hayato Hashimoto, Atsushi Hashimoto, Yoko Yamakata, Jun Harashima, Yoshitaka Ushiku, and Shinsuke Mori. 2020. Visual grounding annotation of recipe flow graph. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 4275–4284, Marseille, Franc… find
  46. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In Proc. of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Doha, Qatar. find
  47. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proc. of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, V… find
  48. Massimo Poesio, Ron Artstein, et al. 2008. Anaphoric annotation in the ARRAU corpus. In Proc. of the Sixth International Conference on Language Resources and Evaluation (LREC 2008), Marrakech, Morocco. find our card
  49. Massimo Poesio, Roland Stuckardt, and Yannick Versley. 2016. Anaphora Resolution. Springer. find
  50. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. CoNLL-2012 shared task: Modeling multilingual unrestricted coreference in OntoNotes. In Proc. of EMNLP-CoNLL 2012: Conference on Empirical Methods in Natural Language Processing and Computational Natural Langua… find our card
  51. Marta Recasens and Eduard Hovy. 2011. BLANC: Implementing the Rand index for coreference evaluation. Natural Language Engineering, 17(4):485–510. find
  52. Ina Rösiger. 2016. Scicorp: A corpus of English scientific articles annotated for information status analysis. In Proc. of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), pages 1743–1749, Portorož, Slovenia. find our card
  53. Ina Rösiger. 2018a. BASHI: A corpus of Wall Street Journal articles annotated with bridging links. In Proc. of the Eleventh International Conference on Language Resources and Evaluation (LREC-2018). find our card
  54. Ina Rösiger. 2018b. Rule- and learning-based methods for bridging resolution in the ARRAU corpus. In Proc. of the First Workshop on Computational Models of Reference, Anaphora and Coreference, pages 23–33, New Orleans, USA. find our card
  55. Ina Rösiger. 2019. Computational modelling of coreference and bridging resolution. Ph.D. thesis, Stuttgart University. find
  56. Ina Rösiger, Arndt Riester, and Jonas Kuhn. 2018. Bridging resolution: Task definition, corpus resources and rule-based experiments. In Proc. of the 27th International Conference on Computational Linguistics, pages 3516–3528, Santa Fe, USA. find our card
  57. Gabriel Stanovsky, Noah A. Smith, and Luke Zettlemoyer. 2019. Evaluating gender bias in machine translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1679–1684, Florence, Italy. Association for Computational Linguistics. find our card
  58. Niket Tandon, Bhavana Dalvi Mishra, Joel Grus, Wen tau Yih, Antoine Bosselut, and Peter Clark. 2018. Reasoning about actions and state changes by injecting commonsense knowledge. In EMNLP. find
  59. Ralph Weischedel, Martha Palmer, Mitchell Marcus, Eduard Hovy, Sameer Pradhan, Lance Ramshaw, Nianwen Xue, Ann Taylor, Jeff Kaufman, Michelle Franchini, Mohammed El-Bachouti, Robert Belvin, and Ann Houston. 2013. OntoNotes release 5.0. Linguistic Data Consortium Catalog No. LDC2013T19. find
  60. Sam Wiseman, Alexander M. Rush, Stuart Shieber, and Jason Weston. 2015. Learning anaphoricity and antecedent ranking features for coreference resolution. In Proc. of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Languag… find our card
  61. Sam Wiseman, Alexander M. Rush, and Stuart M. Shieber. 2016. Learning global features for coreference resolution. In Proc. of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 994–1004, San Diego, USA. find our card
  62. Patrick Xia and Benjamin Van Durme. 2021. Moving on from OntoNotes: Coreference resolution model transfer. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5241–5256, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. find
  63. Yoko Yamakata, Shinsuke Mori, and John Carroll. 2020. English recipe flow graph corpus. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 5187–5194, Marseille, France. European Language Resources Association. find
  64. Juntao Yu and Massimo Poesio. 2020. Multitask learning-based neural bridging reference resolution. In Proceedings of the 28th International Conference on Computational Linguistics, pages 3534–3546, Barcelona, Spain (Online). International Committee on Computational Linguistics. find our card
  65. Amir Zeldes. 2017. The GUM corpus: Creating multilayer resources in the classroom. Language Resources and Evaluation, 51(3):581–612. find
  66. Rui Zhang, Cícero Nogueira dos Santos, Michihiro Yasunaga, Bing Xiang, and Dragomir Radev. 2018. Neural coreference resolution with deep biaffine attention by joint mention detection and mention clustering. In Proc. of the 56th Annual Meeting of the Association for Computational Linguistics (Volume… find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/fang-2022-what-does-it-take-bake/