ARRAU: Linguistically-Motivated Annotation of Anaphoric Descriptions
What it is about
The paper presents the second release of ARRAU, a multi-domain corpus for anaphora. Its domains are news (from the RST Discourse Treebank), task-oriented dialogue (TRAINS), fiction (PEAR) and medical leaflets (GNOME). Release 2 has 552 documents, 348,072 tokens and 99,582 markables. Release 1 had 264 documents, 184,748 tokens and 60,884 markables. Each markable has a minimal span (the head or bare name) and a maximal span (the full NP). The annotation also marks referentiality and genericity, with sub-types such as generic, quantifier, expletive, idiom, coordination and predicate. A consistency pass was applied: non-referential markables may not be in a coreference chain; chains are checked for number and gender clashes and for semantic type; brackets may not cross or cross sentence boundaries. Non-referential and singleton markables make up up to a third of all markables. The paper gives no inter-annotator or coreference scores.
Ideas
- I1. Give each Mova mention a minimal span (the head) and a maximal span (the full NP) in
coref/src/mention.rs. Score lenient matches inside the maximal span, and measure how many dev errors become lenient matches. - I2. Turn the consistency rules into code: a chain may not mix non-referential markables, and may not mix genders. Count violations in the coref output before and after each sieve change. A random chain assignment is the negative control, and it must show many violations.
- I3. Use referentiality as a first filter: a non-referential markable never becomes an anaphor. Keep the sub-types (expletive, idiom, quantifier, generic) in a table, so that each filtered mention has a reason.
- I4. Report coreference per domain (news, dialogue, fiction, medical), so that a drop in one domain does not disappear in the average.
For Mova
The code is coref/src/mention.rs, coref/src/sieve.rs and coref/src/eval.rs. The chapters are languages/en/35-reference-and-coreference.md and languages/en/37-register-and-variation.md. The corpus is non-commercial, so it is for evaluation only and is never built into a release. The measure is per-domain coreference F1 on an evaluation split, run twice.
Library summary
Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/L16-1326.pdf
Summary
The authors describe the second release of ARRAU, an English corpus with detailed, linguistically grounded annotation of anaphora. It targets the harder coreference phenomena that mainstream datasets such as OntoNotes and ACE largely leave out. The corpus covers four domains: Wall Street Journal news taken from the RST Discourse Treebank, TRAINS task-oriented dialogues, PEAR fiction and GNOME medical leaflets. Annotation was done in MMAX.
Compared with the 2008 release, the team annotated every available RST and TRAINS document, roughly doubling the corpus. They extended referentiality and genericity labels to all domains, with fine-grained subtypes, and supplied minimal and maximal spans for every markable. They also introduced automatic consistency checks, for example ensuring that non-referential markables never appear in coreference chains and that coreferent mentions agree in gender and semantic type. These checks removed many errors and surfaced genuinely difficult coreference cases.
Because singletons, non-referentials, morphology and bridging are all annotated, ARRAU supports work on mention detection, referentiality, genericity and complex anaphora. The corpus is distributed through LDC.
Key points
- ARRAU 2 contains 552 documents, about 348K tokens and 99,582 markables. ARRAU 1 had 264 documents, about 185K tokens and 60,884 markables.
- TRAINS grew from 35 to 114 dialogues, making it a rare large dialogue collection annotated for anaphora. RST news grew from 204 to 413 documents.
- The revised referentiality annotation added over 10% more non-referring markables to documents already in ARRAU 1. Singletons and non-referentials make up as much as a third of all markables.
- Unlike ACE-05 (220K tokens) and the CoNLL/OntoNotes data (1.5M tokens), ARRAU annotates:
- minimal and maximal spans,
- singletons and non-referentials,
- number, gender and semantic class,
- bridging alongside identity coreference.
- A third release is planned, focusing on extending bridging annotation to all domains and adding consistency constraints for it.
Cite
@inproceedings{uryupina2016arrau,
title = {{ARRAU: Linguistically-Motivated Annotation of Anaphoric Descriptions}},
author = {Uryupina, Olga and Artstein, Ron and Bristot, Antonella and Cavicchio, Federica and Rodriguez, Kepa and Poesio, Massimo},
booktitle = {Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)},
year = {2016},
doi = {10.63317/4vzwtadfojx8},
url = {https://doi.org/10.63317/4vzwtadfojx8}
}References
- Lynn Carlson, Daniel Marcu, and Mary Ellen Okurowski. 2002. Rst discourse treebank ldc2002t07. find
- George Doddington, Alexis Mitchell, Mark Przybocki, Lance Ramshaw, Stephanie Strassell, and Ralph Weischedel. 2004. The automatic content extraction (ACE) program–tasks, data, and evaluation. In Proceedings of the Language Resources and Evaluation Conference. find
- Jonathan K Kummerfeld, Mohit Bansal, David Burkett, and Dan Klein. 2011. Mention detection: Heuristics for the ontonotes annotations. In Conference on Natural Language Learning, pages 102–106. Association for Computational Linguistics. find
- Cristoph Müller and Michael Strube. 2006. Multi-level annotation of linguistic data with MMAX2. In Corpus Technology and Language Pedagogy, pages 197–214. S. Braun and K. Kohn and J. Mukherjee. find
- Haoruo Peng, Kai-Wei Chang, and Dan Roth. 2015. A joint framework for coreference resolution and mention head detection. CoNLL 2015, 51:12. find
- Massimo Poesio and Ron Artstein. 2008. Anaphoric annotation in the arrau corpus. In Proceedings of the Language Resources and Evaluation Conference. find our card
- Sameer Pradhan, Lance Ramshaw, Mitchell Marcus, Martha Palmer, Ralph Weischedel, and Nianwen Xue. 2011. Conll-2011 shared task: Modeling unrestricted coreference in ontonotes. In Proceedings of the Fifteenth Conference on Computational Natural Language Learning (CoNLL 2011), Portland, Oregon, June. find our card
- Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. CoNLL-2012 shared task: Modeling multilingual unrestricted coreference in OntoNotes. In Proceedings of the Sixteenth Conference on Computational Natural Language Learning (CoNLL 2012), Jeju, Korea. find our card
- Marta Recasens, Lluı́s Màrquez, Emili Sapena, M.Antònia Martı́, Mariona Taulé, Véronique Hoste, Massimo Poesio, and Yannick Versley. 2010. SemEval-2010 Task 1: Coreference resolution in multiple languages. In Proceedings of the 5th International Workshop on Semantic Evaluations (SemEval-2010), Uppsala, Sweden. find
- Desislava Zhekova and Sandra Kübler. 2013. Machine learning for mention head detection in multilingual coreference resolution. In RANLP, pages 747–754. find