Elamur Research⌕
☰
home / papers / 2021 / FantasyCoref: Coreference Resolution on Fantasy Literature Through Omniscient Writer's Point of View

FantasyCoref: Coreference Resolution on Fantasy Literature Through Omniscient Writer's Point of View

Summary

Coreference systems are mostly built on non-fiction such as OntoNotes. They struggle with fantasy stories, where characters disguise themselves, transform, receive prophecies, or are known by shifting names. The authors present FantasyCoref, an English corpus of 211 Grimms' Fairy Tales plus three further fantasy texts (two Arabian Nights tales and Alice's Adventures in Wonderland), totalling about 368K tokens. Three linguists annotated it with CorefAnnotator, adapting the OntoNotes guidelines. Mentions are resolved from the omniscient writer's point of view (OWV): entities are linked according to what the author knows, not what any character or reader believes at a given moment. Four new guideline categories cover these genre-specific cases. A four-stage annotation process reached 87% inter-annotator agreement. Fine-tuning a state-of-the-art neural coreference model on FantasyCoref improved results over the OntoNotes-only model and reduced fiction-specific errors. Its best score of 76.88% still trails OntoNotes-level performance, so the resource points to a hard, underexplored domain relevant to reading comprehension.

Key points

  • Corpus composition: 211 Grimms' stories (GFT) plus a separate test set (AFT) of Aladdin, Ali Baba and the Forty Thieves, and Alice's Adventures in Wonderland, all from Project Gutenberg. The total is 367,891 tokens, and documents are about 2.5× longer than OntoNotes documents. The corpus is available at github.com/emorynlp/FantasyCoref.
  • Guideline categories: The four categories handle lexical variations, wishes or foretelling, alterations to entities, and uneven knowledge between characters.
  • Genre comparison: Knowledge asymmetry appears in 36.4% of the fantasy stories but only 2.0% of 100 sampled OntoNotes documents. For entity changes, the figures are 24.8% vs 1.0%.
  • Agreement: IAA was measured with MUC, B³ and CEAF φ4 rather than Kappa, giving about 87%.
  • Model results: Training on FantasyCoref yielded gains of 2.59 and 3.06 points over the OntoNotes-trained model. A 15-type error analysis showed fewer errors involving long distances, mistaken identities, plot twists and metamorphoses.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

7 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 4 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.crac-1.3

@inproceedings{han2021fantasycoref,
  title = {{FantasyCoref: Coreference Resolution on Fantasy Literature Through Omniscient Writer's Point of View}},
  author = {Han, Sooyoun and Seo, Sumin and Kang, Minji and Kim, Jongin and Choi, Nayoung and Song, Min and Choi, Jinho D.},
  booktitle = {Proceedings of the Fourth Workshop on Computational Models of Reference, Anaphora and Coreference},
  year = {2021},
  doi = {10.18653/v1/2021.crac-1.3},
  url = {https://doi.org/10.18653/v1/2021.crac-1.3}
}

References

20 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Amit Bagga and Breck Baldwin. 1998. Algorithms for scoring coreference chains. In In The First International Conference on Language Resources and Evaluation Workshop on Linguistics Coreference, pages 563–566. find our card
  2. Ben Kantor and Amir Globerson. 2019. Coreference resolution with entity equalization. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 673–677, Florence, Italy. Association for Computational Linguistics. find our card
  3. David Bamman, Olivia Lewke, and Anya Mansoor. 2020. An annotated dataset of coreference in English literature. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 44–54, Marseille, France. European Language Resources Association. find our card
  4. Kenton Lee, Luheng He, and Luke Zettlemoyer. 2018. Higher-order coreference resolution with coarse-to-fine inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 687–… find our card
  5. David Bamman, Sejal Popat, and Sheng Shen. 2019. An annotated dataset of literary entities. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2138–2144, Minneapoli… find
  6. Xuansong Li, Martha Palmer, Nianwen Xue, Lance Ramshaw, Mohamed Maamouri, Ann Bies, Kathryn Conger, Stephen Grimes, and Stephanie Strassel. 2016. Large multi-lingual, multi-level and multi-genre annotation corpus. In Proceedings of the Tenth International Conference on Language Resources and Evaluat… find
  7. Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1):37–46. find
  8. Andreas Cranenburgh, van. 2019. A dutch coreference resolution system with an evaluation on literary fiction. Computational Linguistics in the Netherlands Journal, 9. find
  9. Xiaoqiang Luo. 2005. On coreference resolution performance metrics. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, pages 25–32, Vancouver, British Columbia, Canada. Association for Computational Linguistics. find our card
  10. Tian Ye He. 2007. Coreference resolution on entities and events for hospital discharge summaries. Thesis (M. Eng.)–Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science. find
  11. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. CoNLL-2012 shared task: Modeling multilingual unrestricted coreference in OntoNotes. In Joint Conference on EMNLP and CoNLL - Shared Task, pages 1–40, Jeju Island, Korea. Association for Computational Linguisti… find our card
  12. Felix Hill, Antoine Bordes, Sumit Chopra, and Jason Weston. 2016. The goldilocks principle: Reading children’s books with explicit memory representations. In 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings. find our card
  13. Sameer Pradhan, Lance Ramshaw, Mitchell Marcus, Martha Palmer, Ralph Weischedel, and Nianwen Xue. 2011. CoNLL-2011 shared task: Modeling unrestricted coreference in OntoNotes. In Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task, pages 1–27, Portland, Or… find our card
  14. Eduard H. Hovy, Mitchell P. Marcus, Martha Palmer, Lance A. Ramshaw, and Ralph M. Weischedel. 2006. Ontonotes: The 90% solution. In Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, Proceedings, June 4-9, 2006, New York, New York, Nil… find
  15. Ina Roesiger, Sarah Schulz, and Nils Reiter. 2018. Towards coreference for literary text: Analyzing domain-specific phenomena. In Proceedings of the Second Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature, pages 129–138, Santa Fe, N… find our card
  16. Marc Vilain, John Burger, John Aberdeen, Dennis Connolly, and Lynette Hirschman. 1995. A model-theoretic coreference scoring scheme. In Proceedings of the 6th Conference on Message Understanding, MUC6 ’95, page 45–52, USA. Association for Computational Linguistics. find our card
  17. Wei Wu, Fei Wang, Arianna Yuan, Fei Wu, and Jiwei Li. 2020. CorefQA: Coreference resolution as query-based span prediction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6953–6963, Online. Association for Computational Linguistics. find our card
  18. Liyan Xu and Jinho D. Choi. 2020. Revealing the myth of higher-order inference in coreference resolution. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8527–8533, Online. Association for Computational Linguistics. find our card
  19. Michael Yoder, Sopan Khosla, Qinlan Shen, Aakanksha Naik, Huiming Jin, Hariharan Mu-ralidharan, and Carolyn Rosé. 2021. FanfictionNLP: A text processing pipeline for fanfiction. In Proceedings of the Third Workshop on Narrative Understanding, pages 13–23, Virtual. Association for Computational Lingu… find
  20. Ethan Zhou and Jinho D. Choi. 2018. They exist! introducing plural mentions to coreference resolution and entity linking. In Proceedings of the 27th International Conference on Computational Linguistics, pages 24–34, Santa Fe, New Mexico, USA. Association for Computational Linguistics. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/han-2021-fantasycoref-coreference-resolution-fantasy-literature/