Elamur Research⌕
☰
home / papers / 2012 / Coreference in Spoken vs. Written Texts: a Corpus-based Analysis

Coreference in Spoken vs. Written Texts: a Corpus-based Analysis

Summary

The paper asks why automatic coreference resolution struggles with spoken language. The authors suggest that spoken and written genres organise reference differently, so resolvers built mainly for written input lose accuracy on speech. They compiled an English corpus of 22 texts (about 140,000 tokens): 11 popular-science articles (POPSCI) and 11 manually transcribed and normalised interviews taken from the Backbone Corpus (INTERVIEW). The corpus was annotated with the deterministic, sieve-based Stanford CoreNLP coreference system. The authors then compared the genres on precision, mention length, sentence distance within chains, chain size, syntactic parallelism, grammatical roles, mention types and pronoun morphology. CoreNLP performed noticeably worse on the interviews. The analysis links this drop to speaker-centred first-person pronouns that are split across speaker turns, vague or non-referential it and they, and repeated general nouns that get merged wrongly. Written texts instead show long, heavily embedded mentions. The authors conclude that coreference algorithms need genre-specific parameters: speaker-turn and pronoun-type information for speech, and embedding and lexical-semantic analysis for writing.

Key points

  • Corpus: POPSCI (11 texts) vs. INTERVIEW (11 interviews with American English speakers, from the Backbone Corpus), each text about 6,000 tokens. Spoken features such as fillers and corrections are marked with XML tags.
  • Precision (MUC metric) was estimated by manually correcting 4 of the 22 CoreNLP-annotated texts: 64% for POPSCI vs. 51% for INTERVIEW. The authors report all genre differences as significant (Student's t-test, p < 0.05).
  • Averages for POPSCI vs. INTERVIEW:
    • tokens per sentence: 25.75 vs. 20.73
    • chain size: 3.60 vs. 4.45
    • mention token length (T-length): 3.42 vs. 2.58
  • Pronoun morphology differs strongly between the genres:
    • 1st person singular: 26.9% of pronouns in interviews vs. 5.8% in popular science
    • 3rd person plural: 16.9% in interviews vs. 33.1% in popular science
  • CoreNLP does not handle discourse-deictic reference to clauses or VPs, which adds to errors with vague it in speech.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

6 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 5 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{amoia2012coreference,
  title = {{Coreference in Spoken vs. Written Texts: a Corpus-based Analysis}},
  author = {Amoia, Marilisa and Kunz, Kerstin and Lapshinova-Koltunski, Ekaterina},
  booktitle = {Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12)},
  year = {2012},
  doi = {10.63317/5gb36ny2kine},
  url = {https://doi.org/10.63317/5gb36ny2kine}
}

References

12 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. A. Bagga and B. Baldwin. 1998. Algorithms for scoring coreference chains. In Proceedings of the Linguistic Coreference Workshop at The First International Conference on Language Resources and Evaluation (LRECÕ98), page 563Ð566. find our card
  2. George Doddington, Alexis Mitchell, Mark Przybocki, Lance Ramshaw, Stephanie Strassel, and Ralph Weischedel. 2004. The automatic content extraction (ace) program tasks, data, and evaluation. In NIST. find our card
  3. Miriam Eckert and Michael Strube. 2000. Dialogue acts, synchronising units and anaphora resolution. Journal of Semantics, 17(1):51–89. find
  4. Kurt Kohn, Petra Hoffstaedter, and Johannes Widmann. 2009. Backbone - pedagogic corpora for content and language integrated learning. In Eurocall-2009, Valencia, Spain, September 9-12. find
  5. Heeyoung Lee, Yves Peirsman, Angel Chang, Nathanael Chambers, Mihai Surdeanu, and Dan Jurafsky. 2011. Stanford’s multi-pass sieve coreference resolution system at the CoNLL-2011 Shared Task. In CoNLL-2011 Shared Task. find our card
  6. Xiaoqiang Luo. 2005. On coreference resolution performance metrics. In HLT/EMNLP. find our card
  7. MUC-6. 1995. Coreference task definition. In the Sixth Message Understanding Conference (MUC-6). find
  8. MUC-7. 1997. Coreference task definition. In the Seventh Message Understanding Conference (MUC-7). find
  9. Sameer Pradhan, Lance Ramshaw, Mitchell Marcus, Martha Palmer, Ralph Weischedel, and Nianwen Xue. 2011. Conll-2011 shared task: Modeling unre- stricted coreference in ontonotes. In the Fifteenth Conference on Computational Natural Language Learning (CoNLL 2011). find
  10. Amanda J. Stent and Srinivas Bangalore. 2010. Interaction between dialog structure and coreference resolution. In IEEE SLT 2010, Berkeley, USA. find
  11. Joel Tetreault and James Allen. 2004. Dialogue structure and pronoun resolution. In DAARC’04, Azores, September 23-24. find
  12. M. Vilain, J. Burger, J. Aberdeen, D. Connolly, and L. Hirschman. 1995. A model theoretic coreference scoring scheme. In Proceedings of the Sixth Message Understanding Conference (MUC-6), pages 45–52. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/amoia-2012-coreference-spoken-vs-written-texts/