Elamur Research⌕
☰
home / papers / 2020 / English Recipe Flow Graph Corpus

English Recipe Flow Graph Corpus

Paper · 2020 · Yoko Yamakata, Shinsuke Mori and John Carroll

Summary

Instructions such as cooking recipes describe how to carry out a task, but software can only search, reason over, or act on them once they are turned into a structured form. The authors build an English resource for this purpose. It contains 300 user-written recipes from Allrecipes UK/Ireland (38,062 tokens), annotated in two layers:

  • Recipe named entities (r-NEs): ten domain tags covering foods, tools, durations, quantities, chef/food/tool actions and states.
  • Flow graphs: these link the entities and record which steps must precede which.

The scheme adapts earlier Japanese work by Mori et al. (2014) and adds tags for phenomena specific to English. Human agreement falls between 82.3 and 90.5 F1 across the layers. For automatic annotation, BERT-NER tags r-NEs with roughly 87.5 F1. A maximum-spanning-tree, dependency-style parser builds flow graphs with 71.1 F1 for edges. It treats the whole recipe's entity sequence as one input, so links across sentences need no separate coreference step. The results show the scheme can be learned, and the corpus is freely available.

Key points

  • The corpus has two subsets:
    • 100-r: popular recipes sampled per dish category, annotated by a native English speaker and verified by the first author.
    • 200-r: randomly sampled recipes, annotated by two Japanese annotators experienced with the Japanese scheme.
  • The tag set consists of ten r-NE labels: Sf, St, F, T, Q, D, Ac, Ac2, At and Af. Ac2 marks the second, non-contiguous part of a discontinuous chef action. At covers actions by automatic tools such as food processors. Spans use IOB2 encoding.
  • Manual r-NE annotation took about 19 minutes per recipe. The second half of 200-r was pre-tagged by PWNER trained on 100-r, and annotators then corrected the output.
  • Agreement on r-NEs reached 89.9% precision, 92.2% recall and 90.5 F1. It was measured as F1, with one annotator treated as gold, because IOB2 tag dependencies make Cohen's kappa unsuitable.
  • Automatic results: BERT-NER scored about 87.5 F1 for r-NEs under 10-fold cross-validation (80/10/10 split). The spanning-tree parser scored 71.1 F1 on flow graph edges.

Interesting ideas

  • I1. The parser handles a whole document's entity sequence as one string, so edges between sentences are found exactly like edges within a sentence, without a coreference stage.
  • I2. A flow graph deliberately ignores surface wording, so "add diced carrots" and "dice carrots and add" map to the same graph because the actions and their order are identical.
  • I3. Intermediate products are modelled as entities that later actions consume, which gives an explicit chain of state changes and ordering constraints.
  • I4. When tag constraints break the independence that kappa assumes, agreement can be reported as precision/recall/F1 against one annotator used as gold.
  • I5. The authors plan to add extra structure, such as edge labels and links where branches merge, monotonically on top of a base tree, so existing correct arcs are never disturbed.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): Linking "it"/"the mixture" in step 3 to a product from step 1 could be scored as a document-level arc rather than through a separate coreference module, and each link would still be explainable as an arc.
  • I2 (back): This equivalence can serve as a test for the text → graph → text compressor: paraphrases that should yield identical graphs, and regenerated text that should reproduce the same graph.
  • I3 (world): The world of a text could track derived objects (dough, sauce, a mixture) as new entities created by events, with precedence edges, which helps answer "what was used to make X" questions.
  • I4 (learn): Agreement between witness parsers on silver UD trees, or between span annotations, can be reported the same way, with LAS-style or span F1 instead of kappa.
  • I5 (tree): Enhanced-UD-style extra edges or repair rules could be layered on a parsed tree as add-only passes, so the basic tree stays intact and each addition remains auditable.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓The University of TokyoTokyo · Japan45 works · 8,692 citations🎓Kyoto UniversityKyoto · Japan27 works · 8,049 citations🎓Pennsylvania State UniversityState College · United States22 works · 4,878 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 1Research subjects and languages the work is about.English · 1,373 works

Cite

Cite as Yoko Yamakata, Shinsuke Mori and John Carroll. 2020. English Recipe Flow Graph Corpus. Proceedings of the Twelfth Language Resources and Evaluation Conference.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{yamakata2020english,
  title = {{English Recipe Flow Graph Corpus}},
  author = {Yoko Yamakata, Shinsuke Mori and John Carroll},
  booktitle = {Proceedings of the Twelfth Language Resources and Evaluation Conference},
  year = {2020},
  url = {https://aclanthology.org/2020.lrec-1.638/}
}

References

23 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Banarescu, L., Bonial, C., Cai, S., Georgescu, M., Griffitt, K., Hermjakob, U., Knight, K., Koehn, P., Palmer, M., and Schneider, N. (2013). Abstract meaning representation for sembanking. In the 7th Linguistic Annotation Workshop and Interoperability with Discourse, pages 178–186.
  2. Ben Abacha, A. and Zweigenbaum, P. (2011). Medical entity recognition: A comparison of semantic and statistical methods. In Proceedings of BioNLP 2011 Workshop, pages 56–64, Portland, Oregon, USA, June. Association for Computational Linguistics. find
  3. Berger, A. L., Pietra, S. A. D., and Pietra, V. J. D. (1996). A maximum entropy approach to natural language processing. Computational Linguistics, 22(1). ↗
  4. Bollini, M., Tellex, S., Thompson, T., Roy, N., and Rus, D. (2013). Interpreting and executing recipes with a cooking robot. In Experimental Robotics, pages 481–495. ↗
  5. Borthwick, A. (1999). A Maximum Entropy Approach to Named Entity Recognition. Ph.D. thesis, New York University. find
  6. Briscoe, T., Carroll, J., and Watson, R. (2006). The second release of the RASP system. In Proceedings of the COL-ING/ACL 2006 Interactive Presentation Sessions, pages 77–80. Association for Computational Linguistics.
  7. Constant, M., Eryigit, G., Monti, J., van der Plas, L., Rosner, C. R. M., and Todirascu, A. (2017). Multiword expression processing: A survey. Computational Linguistics, 43(4):837–892. ↗
  8. Devlin, J., Chang, M. W., Lee, K., and Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4171–4186.
  9. Flanigan, J., Thomson, S., Carbonell, J., Dyer, C., and Smith, N. A. (2014). A discriminative graph-based parser for the abstract meaning representation. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 1426–1436.
  10. Hamada, R., Ide, I., Sakai, S., and Tanaka, H. (2000). Structural analysis of cooking preparation steps in Japanese. In Proceedings of the fifth International Workshop on Information Retrieval with Asian Languages, pages 157–164. find
  11. Hashimoto, A., Mori, N., Funatomi, T., Yamakata, Y., Kakusho, K., and Minoh, M. (2008). Smart kitchen: A user centric cooking support system. In Proceedings of the 12th Information Processing and Management of Uncertainty in Knowledge-Based Systems, pages 848–854. find
  12. Jermsurawong, J. and Habash, N. (2014). Predicting the structure of cooking recipes. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2370–2377. find
  13. Kiddon, C., Ponnuraj, G. T., Zettlemoyer, L., and Choi, Y. (2015). Mise en place: Unsupervised interpretation of instructional recipes. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 982–992. find
  14. Maeta, H., Sasada, T., and Mori, S. (2015). A framework for procedural text understanding. In Proceedings of the 14th International Conference on Parsing Technologies, pages 50–60. find
  15. Momouchi, Y. (1980). Control structures for actions in procedural texts and PT-Chart. In Proceedings of the Eighth International Conference on Computational Linguistics, pages 108–114. find
  16. Mori, S., Maeta, H., Yamakata, Y., and Sasada, T. (2014). Flow graph corpus from recipe texts. In Proceedings of the Nineth International Conference on Language Resources and Evaluation, pages 2370–2377. find
  17. Ratinov, L. and Roth, D. (2009). Design challenges and misconceptions in named entity recognition. In Proceedings of the Thirteenth Conference on Computational Natural Language Learning (CoNLL-2009), pages 147–155, Boulder, Colorado, June. Association for Computational Linguistics.
  18. Sang, E. F. T. K. and Meulder, F. D. (2003). Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. In Proceedings of the Seventh Conference on Computational Natural Language Learning, pages 142–147. ↗
  19. Sang, E. F. T. K. and Veenstra, J. (1999). Representing text chunks. In Proceedings of EACL’99, pages 173–179.
  20. Sasada, T., Mori, S., Kawahara, T., and Yamakata, Y. (2015a). Named entity recognizer trainable from partially annotated data. In Proceedings of the Eleventh International Conference Pacific Association for Computational Linguistics. find
  21. Sasada, T., Mori, S., Yamakata, Y., Maeta, H., and Kawahara, T. (2015b). Definition of recipe terms and corpus annotation for their automatic recognition (in Japanese). Journal of Natural Language Processing, 22(2):107– 131. ↗
  22. Yamakata, Y., Imahori, S., Sugiyama, Y., Mori, S., and Tanaka, K. (2013). Feature extraction and summarization of recipes using flow graph. In Proceedings of the 5th International Conference on Social Informatics, LNCS 8238, pages 241–254. find
  23. Yamakata, Y., Carroll, J., and Mori, S. (2017). A comparison of cooking recipe named entities between Japanese and English. In Proceedings of the 9th Workshop on Multimedia for Cooking and Eating Activities, in Conjunction with IJCAI 2017, pages 7–12. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yamakata-2020-english-recipe-flow-graph-corpus/