Elamur Research⌕
☰
home / papers / 2014 / ANCOR_Centre, a large free spoken French coreference corpus: description of the resource and reliability measures

ANCOR_Centre, a large free spoken French coreference corpus: description of the resource and reliability measures

Summary

Data-driven coreference resolution needs large annotated corpora. For French, the biggest freely available resource was DEDE, which has only about 48,000 words and covers only definite descriptions. Muzerelle et al. address this gap with ANCOR_Centre, a free corpus of about 488,000 words of transcribed spoken French. It is built from four existing speech corpora: sociolinguistic interviews from ESLO, plus two highly interactive dialogue sets (OTG and Accueil_UBS).

Coders annotated all noun phrases and pronouns by hand on the Glozz platform. The work ran in four stages that alternated annotation with adjudication by a supervisor. Each mention carries morphosyntactic, definiteness and named-entity features. Each relation is assigned one of five coreference or anaphora types.

The authors measured reliability separately for two tasks:

  • Delimiting relations: agreement between different coders was moderate.
  • Assigning relation types: agreement was high.

Each coder was also very consistent with their own earlier annotations.

The result is one of the largest coreference corpora in any language and, as far as the authors know, the largest dedicated to speech. It is released under a Creative Commons licence and is useful both for machine learning and for linguistic research.

Key points

  • Source corpora: ESLO_ANCOR (417,000 words, 25 h), ESLO_CO2 (35,000 words), OTG (26,000 words, task-oriented dialogue) and Accueil_UBS (10,000 words, phone calls). Conversational speech makes up only about 7% of the total.
  • Size of the annotation: about 116,000 mentions and 51,000 relations. Nominal and pronominal mentions are nearly balanced (51.2% vs 48.8%). Roughly 45,000 named entities are tagged with Ester2 types, PERS being the most frequent.
  • Relation types: direct coreference, indirect coreference, pronominal anaphora, bridging anaphora and pronominal bridging. Near-identity and finer bridging subtypes were deliberately left out.
  • Reliability (κ, π and α were identical): 0.45 inter-coder agreement on delimitation, 0.91 intra-coder agreement on delimitation, and 0.80 inter-coder agreement on type categorization.
  • Availability: released under CC-BY-NC-SA. Version 1.0 uses the discourse-new scheme only; the final release was to add coreference chain and cluster representations.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

18 papers of our library cite it (of the 140673 whose bibliographies we read); 5 of them have a page here, 13 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{muzerelle2014ancor,
  title = {{ANCOR\_Centre, a large free spoken French coreference corpus: description of the resource and reliability measures}},
  author = {Muzerelle, Judith and Lefeuvre, Anaïs and Schang, Emmanuel and Antoine, Jean-Yves and Pelletier, Aurore and Maurel, Denis and Eshkol, Iris and Villaneau, Jeanne},
  booktitle = {Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)},
  year = {2014},
  url = {https://aclanthology.org/L14-1169/}
}

References

25 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Antoine, J.-Y., Villaneau, J., Lefeuvre, A. (2014). Weighted Krippendorff's alpha is a more reliable metrics for multi- coders ordinal annotations: experimental studies on emotion, opinion and coreference annotation. Proc. EACL’2014. (to appear) find
  2. Artstein, R., Poesio, M. (2008) Inter-Coder agreement for Computational Linguistics, Computational Linguistics, 34, pp .555--596. find
  3. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, pp. 37--46. find
  4. Dipper S., Zinmeister H. (2010). Towards a standard for annotating abstract anaphora. Proc. LREC’2010 workshop on Language Resources and Language Technology Standards. Valetta, Malta, pp. 54--59. find
  5. Galliano, S., Gravier, G., Chaubard, L. (2009). The ESTER 2 Evaluation Campaign for the Rich Transcription of French Radio Broadcast. find
  6. Proc. Interspeech 2009, Brighton, UK. find
  7. Gardent, C. and Manuelian, H. (2005). Création d’un corpus annoté de traitement des descriptions définies. Traitement Automatique des Langues, TAL , 46(1). find
  8. Heindrickx, I., Bouma, G., Coppens, F., Daelemans, W., Hoste, V., Kloosterman, G., Mineur, A.-M., Van Der Vloet, J., Verschelde, J.-L. (2008). A coreference corpus and resolution system for Dutch. Proc. LREC’2008. find our card
  9. Hinrichs, E., Kübler, S., Naumann, K., Zinsmeister, H. (2005). Recent developments in linguistic annotations of the TüBa-D/Z Treebank. Proc. 27 th Annual Meeting of the German Linguistic Association, Köln, Germany. find
  10. Iida, R., Mamoru, K., Kentaro, I., Yuji, M. (2007). Annotating a Japanese text corpus with predicate-argument and coreference relations. Proc. Linguistic Annotation Workshop (LAW 2007),Stroudsburg, PA, USA. ACL, pp. 132--139. find our card
  11. Krippendorff, K. (2004). Content Analysis: an Introduction to its Methodology. Chapter 11. Sage: Thousand Oaks, CA. find
  12. Passoneau, R. (2004). Computing reliability for Co-Reference Annotation. Proc. LREC’2004. Lisboa, Portugal. find
  13. Pradhan, S. S., Ramshaw, L., Weischedel, R., MacBride, J., Micciula, L. (2007). Unrestricted coreference: identifying entities and events in OntoNotes. Proc. 1 st IEEE Int. Conf. on Semantic Computing (ICSC’07). Washington, DC. USA. IEEE, pp. 446--453. find
  14. Lefeuvre, A., Antoine, J.-Y., Schang, E. (2014). Le corpus ANCOR\_Centre et son outil de requêtage : application à l’étude de l’accord en genre et nombre dans les coréférences et anaphores en français parlé. Proc. CMLF’2014. (to appear) find
  15. Mathet, Y., Widlöcher, A. (2012). find
  16. The Glozz Platform: A Corpus Annotation and Mining Tool. Proc. 2012 ACM symposium on Document engineering, pp. 171--180. find
  17. Nedoluzzhko, A., Mírovský, J., Ocelák, R., Pergler, J. (2009). Extended coreference relations and bridging anaphora in the Prague Dependancy Treebank. Proc. find
  18. DAARC’2009. Chennai Goa, Indica , pp. 1--16 find
  19. Nicolas, P., Letellier-Zarshenas, S., Schadle, I., Antoine, J.-Y., Caelen, J. (2002). Towards a large corpus of spoken dialogue in French that will be freely available: the “Parole Publique” project and its first realisations. Proc. LREC’2002. Las Palmas de Gran Canaria, Spain. find
  20. Rao, D., MacNamee, P., Dredze, M. (2011). Entity Linking: Finding Extracted Entities in a Knoledge Base. In T. Poibeau, H. Saggion, J. Piskorski, R. Yangarber (Eds.), Multi-source, Multilingual Information Extraction and Summarization. Springer, pp. 93--115. find
  21. Recasens Potau, M., (2010). Coreference: Theory, Annotation, Resolution and Evaluation. PhD Thesis, Universitat de Barcelona, Catalunya, Spain. find
  22. Schang, E., Boyer, A., Muzerelle, J., Antoine, J.-Y., Eskhol, I., Maurel, D. (2011). Coreference and anaphoric annotations for spontaneous speech corpus in French. Proc. DAARC'2011, Discourse Anaphora and Anaphor Resolution Colloquium, Faro, Portugal. find
  23. Ogrodniczuk, M., Kopeć M., Głowinska K., Savary A., Zawisławska, M. (2013). Polish coreference corpus, submitted at LTC’2014. find
  24. Scott, W. (1955). Reliability of content analysis: the case of nominal scale coding. Public Opinions Quaterly. 19, pp. 321--325. van Deemter, K., Kibble, R. (2000). On Coreferring: Coreference in MUC and related annotation schemes. Computational Linguistics, 26(4), pp. 629--637. find
  25. Vieira, R., Salmon-Alt, S., Schang, E. (2002). Multilingual corpora annotation for processing definite descriptions. Proc. Advances in Natural Language Processing 2002, pp. 721--729. . find

↑ back to top