Elamur Research⌕
☰
home / papers / 2023 / Understanding the Cooking Process with English Recipe Text

Understanding the Cooking Process with English Recipe Text

Paper · 2023 · Yi Fan and Anthony Hunter

Summary

Recipes are procedural texts that can be turned into flow graphs. In these graphs, nodes are foods, tools, actions, durations and similar entities, and labelled arcs show how cooking steps connect. Earlier systems for English recipes reached weak end-to-end scores (F1 43.3 for Yamakata et al. 2020 and 72.3 for Donatelli et al. 2021), and neither could assign labels to edges.

Fan and Hunter build a two-stage pipeline. First, an entity tagger stacks BERT, a BiLSTM and a CRF to find the graph nodes. Second, a BERT classifier takes every pair of entities, together with the sentences containing them, and predicts a directed edge label or a "no relation" class. This replaces the spanning-tree and dependency-parser decoders used in earlier work.

On the English recipe flow graph corpus (300-r), the pipeline produces labelled graphs and reports an overall F1 of 92.2. Earlier frameworks scored at most 72.3, and only on unlabelled edges.

This matters because complete labelled graphs are machine-readable. They could support cooking robots, allergen detection or recipe rewriting.

Key points

  • The data is the 300-r English recipe flow graph corpus: 300 recipes, built from Yamakata et al.'s 100 and extended by 200.
    • It has 10 entity tags (F, T, D, Q, Ac, Ac2, Af, At, Sf, St).
    • It has 13 edge labels (Agent, Targ, Dest, T-comp, F-comp, F-eq, F-part-of, F-set, T-eq, T-part-of, A-eq, V-tm, other-mod).
  • NER uses BERT, then a BiLSTM, then a CRF. The CRF enforces consistent BIO spans; for example, an -I tag cannot follow O.
  • Relation extraction is pairwise BERT classification.
    • Entity pairs can sit in the same sentence or in different sentences.
    • The label set adds not_relate in both directions, with e1 always the entity that appears first.
  • Edge-prediction results with the system's own tagger:
    • F1 is 94.5 without edge labels and 90.3 with labels.
    • The baselines score 43.3 (Y'20) and 72.3 ± 0.8 (D'21), both unlabelled.
    • With gold tags, the system reaches 98.1 unlabelled and 92.9 labelled.
    • The abstract cites 92.2 as the overall labelled F1.
  • The authors name several weaknesses:
    • The system has no root-selection strategy.
    • It does not guarantee a connected or complete graph.
    • Its output is sensitive to paraphrase: the same pancake instruction gets a wrong order of actions in one wording and a correct one in another.

Interesting ideas

  • I1. Treating each edge as an independent pairwise decision, instead of one global tree decode, keeps a single wrong node or arc from corrupting the rest of the graph.
  • I2. Adding explicit "no relation" classes in both directions turns graph construction into ordinary classification over all entity pairs, including pairs in different sentences.
  • I3. A CRF layer over BIO tags enforces span-level consistency that per-token classifiers ignore, which matters because NER is scored by whole spans.
  • I4. Feeding a model two wordings of the same instruction exposes inconsistent outputs; the authors suggest paraphrase augmentation that keeps the same entities as a fix.
  • I5. The common heuristic of picking the last action as the root fails under flexible word order, which shows that procedural graphs need an explicit choice of goal or root.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (world): Event-order and state links between entities could be scored pair by pair, so one bad link does not cascade through the whole text-world timeline.
  • I2 (frames): Cross-sentence role or coreference linking could include a directed null label, giving the system a way to state and explain why two mentions are not connected.
  • I3 (tokens): Multiword lemma or chunk tagging could add a transition-constraint decode, so the tagger never emits illegal sequences like O followed by I.
  • I4 (learn): Meaning-preserving rewrites could serve as a self-consistency check, where the frames and world built from each paraphrase should match, and mismatches are flagged as training targets.
  • I5 (world): When building event graphs for procedures, Mova could pick the final goal event from purpose and temporal clues rather than surface position, and record that choice as an explainable step.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University College LondonLondon · United Kingdom88 works · 19,432 citations🔬Heidelberg Institute for Theoretical StudiesHeidelberg · Germany30 works · 1,845 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 1Research subjects and languages the work is about.English · 2,263 works

Cite

Cite as Yi Fan and Anthony Hunter. 2023. Understanding the Cooking Process with English Recipe Text. Findings of the Association for Computational Linguistics: ACL 2023.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.findings-acl.261

@inproceedings{fan2023understanding,
  title = {{Understanding the Cooking Process with English Recipe Text}},
  author = {Yi Fan and Anthony Hunter},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2023},
  year = {2023},
  doi = {10.18653/v1/2023.findings-acl.261},
  url = {https://doi.org/10.18653/v1/2023.findings-acl.261}
}

References

23 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. José Alemany, Stella Heras, Javier Palanca, and Vicente Julián. 2016. Bargaining agents based system for automatic classification of potential allergens in recipes. ADCAIJ: Advances in Distributed Computing and Artificial Intelligence Journal, 5(2):43–51. ↗
  2. Mario Bollini, Stefanie Tellex, Tyler Thompson, Nicholas Roy, and Daniela Rus. 2013. Interpreting and executing recipes with a cooking robot. In Experimental Robotics, pages 481–495. Springer. ↗
  3. Hirokuni Maeta, Tetsuro Sasada, and Shinsuke Mori. 2015. A framework for procedural text understanding. In Proceedings of the 14th International Conference on Parsing Technologies, pages 50–60, Bilbao, Spain. Association for Computational Linguistics. find
  4. Yoshio Momouchi. 1980. Control structures for actions in procedural texts and pt-chart. In Proceedings of the 8th Conference on Computational Linguistics, COLING ’80, page 108–114, USA. Association for Computational Linguistics. find
  5. Francesco Branca, Anna Lartey, Stineke Oenema, Victor Aguayo, Gunhild A Stordalen, Ruth Richardson, Mario Arvelo, and Ashkan Afshin. 2019. Transforming the food system to fight non-communicable diseases. Bmj, 364. ↗
  6. Shinsuke Mori, Hirokuni Maeta, Yoko Yamakata, and Tetsuro Sasada. 2014. Flow graph corpus from recipe texts. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 2370–2377, Reykjavik, Iceland. European Language Resources Association (ELRA). find
  7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. ↗
  8. Shinsuke Mori, Tetsuro Sasada, Yoko Yamakata, and Koichiro Yoshino. 2012. A machine learning approach to recipe text processing. find
  9. Lucia Donatelli, Theresa Schmidt, Debanjali Biswas, Arne Köhn, Fangzhou Zhai, and Alexander Koller. 2021. Aligning actions across recipe graphs. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6930– 6942, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. find
  10. Timothy Dozat and Christopher D Manning. 2016. Deep biaffine attention for neural dependency parsing. arXiv preprint arXiv:1611.01734. ↗
  11. Matt Gardner, Joel Grus, Mark Neumann, Oyvind Tafjord, Pradeep Dasigi, Nelson Liu, Matthew Peters, Michael Schmitz, and Luke Zettlemoyer. 2018. Allennlp: A deep semantic natural language processing platform. arXiv preprint arXiv:1803.07640. ↗
  12. Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirectional lstm-crf models for sequence tagging. arXiv preprint arXiv:1508.01991. ↗
  13. Anupam Jain, Rakhi NK, and Ganesh Bagler. 2015. Analysis of food pairing in regional cuisines of india. PloS one, 10(10):e0139539. ↗
  14. Dan Jurafsky. 2014. The Language of Food: A Linguist Reads the Menu. W. W. Norton. find
  15. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. ↗
  16. John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the Eighteenth International Conference on Machine Learning, ICML ’01, page 282–289, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc. find
  17. Dim P. Papadopoulos, Enrique Mora, Nadiia Chepurko, Kuan Wei Huang, Ferda Ofli, and Antonio Torralba. 2022. Learning program representations for food images and cooking recipes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16559–16569. find
  18. Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 2227–2237, New Orleans, Louisiana. Association for Computational Linguistics. ↗
  19. Matthias B Schulze, Miguel A Martínez-González, Teresa T Fung, Alice H Lichtenstein, and Nita G Forouhi. 2018. Food based dietary patterns and chronic disease prevention. bmj, 361. find
  20. Marieke van Erp, Christian Reynolds, Diana Maynard, Alain Starke, Rebeca Ibáñez Martín, Frederic Andres, Maria C. A. Leite, Damien Alvarez de Toledo, Ximena Schmidt Rivera, Christoph Trattner, Steven Brewer, Carla Adriano Martins, Alana Kluczkovski, Angelina Frankowska, Sarah Bridle, Renata Bertazzi Levy, Fernanda Rauber, Jacqueline Tereza da Silva, and Ulbe Bosma. 2021. Using natural language processing and artificial intelligence to explore the nutrition and sustainability of recipes and food. Frontiers in Artificial Intelligence, 3. ↗
  21. Walter Willett, Johan Rockström, Brent Loken, Marco Springmann, Tim Lang, Sonja Vermeulen, Tara Garnett, David Tilman, Fabrice DeClerck, Amanda Wood, et al. 2019. Food in the anthropocene: the eat– lancet commission on healthy diets from sustainable food systems. The Lancet, 393(10170):447–492. ↗
  22. Yoko Yamakata, John Carroll, and Shinsuke Mori. 2017. A comparison of cooking recipe named entities between japanese and english. In Proceedings of the 9th Workshop on Multimedia for Cooking and Eating Activities in Conjunction with The 2017 International Joint Conference on Artificial Intelligence, CEA2017, page 7–12, New York, NY, USA. Association for Computing Machinery. find
  23. Yoko Yamakata, Shinsuke Mori, and John Carroll. 2020. English recipe flow graph corpus. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 5187–5194, Marseille, France. European Language Resources Association.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/fan-2023-understanding-cooking-process-english-recipe/