Elamur Research⌕
☰
home / papers / 2026 / AmbiCoRefVis: A Tool for Visualizing Coreferential Ambiguity

AmbiCoRefVis: A Tool for Visualizing Coreferential Ambiguity

Summary

Coreference annotation often involves genuine ambiguity: a pronoun may plausibly point to several antecedents, and annotators frequently disagree. Established tools such as MMAX2 can display full coreference chains, but they show alternative antecedents only as local links between single mentions. They cannot present competing chains across a whole discourse. The authors, a team of linguists and visualization researchers, built AmbiCoRefVis, an interactive visual analysis tool that presents several global interpretations of referring expressions at once. Following a design-study process with iterative prototyping, they combined two linked views. A text view marks referring expressions in context, and a graph view lays out the tree-like structure of alternative chains. Edge thickness signals which reading most annotators preferred. The tool reads the MAS-XML format and was demonstrated on the crowdsourced Phrase Detectives corpus, where domain experts used it to locate ambiguous cases. The work matters because it treats disagreement as useful data rather than noise. It also makes ambiguity-preserving corpora easier to inspect and analyze.

Key points

  • The tool targets two kinds of ambiguity: whether a markable is discourse-new (DN) or discourse-old (DO), and which antecedent a DO expression takes.
  • It was evaluated on the latest Phrase Detectives release, collected through a game-with-a-purpose with about 25 player judgments per expression on average. The authors describe it as the largest anaphoric corpus that keeps multiple interpretations.
  • The input format is MAS-XML, where each markable is an <ne> element carrying an ID and morphosyntactic features such as category, gender, number and person.
  • Users can filter references by how many players support them, for example showing only links backed by at least 3 annotators. They can also select chains to inspect them across the full context.
  • Development followed the Sedlmair et al. (2012) design study methodology, and matrix-based layouts were tried and rejected along the way. The tool is available at graphics.uni-konstanz.de/AmbiCoRefVis/.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis · Coreference resolution

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/2h3c7hgu4tyd

@inproceedings{paetzold2026ambicorefvis,
  title = {{AmbiCoRefVis: A Tool for Visualizing Coreferential Ambiguity}},
  author = {Paetzold, Patrick and Beiske, Lukas and Zymla, Mark-Matthias and Poesio, Massimo and Butt, Miriam and Weiskopf, Daniel and Deussen, Oliver},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/2h3c7hgu4tyd},
  url = {https://doi.org/10.63317/2h3c7hgu4tyd}
}

References

45 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Giuseppe Di Battista, Peter Eades, Roberto Tamassia, and Ioannis G. Tollis. 1998. Graph Drawing: Algorithms for the Visualization of Graphs, 1st edition. Prentice Hall PTR. find
  2. Christin Beck, Hannah Booth, Mennatallah El-Assady, and Miriam Butt. 2020. Representation problems in linguistic annotations: Ambiguity, variation, uncertainty, error and bias. In Proceedings of the 14th Linguistic Annotation Workshop, pages 60–73. find
  3. Fabian Beck, Michael Burch, Stephan Diehl, and Daniel Weiskopf. 2017. A taxonomy and survey of dynamic graph visualization. Computer Graphics Forum, 36(1):133–159. find
  4. Georges-Pierre Bonneau, Hans-Christian Hege, Chris R. Johnson, Manuel M. Oliveira, Kristin Potter, Penny Rheingans, and Thomas Schultz. 2014. Overview and state-of-the-art of uncertainty visualization. In Charles D. Hansen, Min Chen, Christopher R. Johnson, Arie E. Kaufman, and Hans Hagen, editors,… find
  5. Ari Bornstein, Arie Cattan, and Ido Dagan. 2020. CoRefi: A crowd sourcing suite for coreference annotation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 205–215. Association for Computational Linguistics. find
  6. David Day, Chad McHenry, Robyn Kozierok, and Laurel Riek. 2004. Callisto: A configurable annotation workbench. In Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC). European Language Resources Association. find
  7. Kees Van Deemter and Rodger Kibble. 2000. On Coreferring: Coreference in MUC and Related Annotation Schemes. Computational Linguistics, 26(4):629–637. find our card
  8. Richard Eckart de Castilho, Éva Mújdricza-Maydt, Seid Muhie Yimam, Silvana Hartmann, Iryna Gurevych, Anette Frank, and Chris Biemann. 2016. A Web-based Tool for the Integrated Annotation of Semantic and Syntactic Structures. In Proceedings of the Workshop on Language Technology Resources and Tools f… find
  9. Max Franz, Manfred Cheung, Harry Adel, Alexander Li, YikaJ, Ethan Keller, Sam Ko-rtchmar, Bjørn Fjukstad, Jascha Ephraim, N8th8n8el, and Zachary Blumenfeld. 2022. cytoscape/cytoscape.js-dagre:. find
  10. Max Franz, Christian T. Lopes, Gerardo Huck, Yue Dong, Onur Sumer, and Gary D. Bader. 2015. Cytoscape.js: a graph theory library for visualisation and analysis. Bioinformatics, 32(2):309–311. find
  11. Pantea Haghighatkhah, Mennatallah El-Assady, Jean-Daniel Fekete, Narges Mahyar, Carita Paradis, Vasiliki Simaki, and Bettina Speckmann. 2022. Characterizing uncertainty in the visual text analysis pipeline. In 2022 IEEE 7th Workshop on Visualization for the Digital Humanities, pages 25–30. find
  12. Chris R. Johnson. 2004. Top scientific visualization research problems. IEEE Computer Graphics and Applications, 24(4):13–17. find
  13. Maurice Koch, Nan Cao, Daniel Weiskopf, and Kuno Kurzhals. 2025. Active gaze labeling: Visualization for trust building. IEEE Transactions on Visualization and Computer Graphics, 31(7):3942–3956. find
  14. Olga Krasavina and Christian Chiarcos. 2007. PoCoS - Potsdam Coreference Scheme. In Proceedings of the Linguistic Annotation Workshop, pages 156–163. Association for Computational Linguistics. find our card
  15. Alan M. MacEachren, Robert E. Roth, James O’Brien, Bonan Li, Derek Swingley, and Mark Gahegan. 2012. Visual semiotics & uncertainty visualization: An empirical study. IEEE Transactions on Visualization and Computer Graphics, 18(12):2496–2505. find
  16. Mark-Christoph Müller and Michael Strube. 2006. Multi-level annotation of linguistic data with MMAX2. In Sabine Braun, Kurt Kohn, and Joybrato Mukherjee, editors, Corpus Technology and Language Pedagogy. New Resources, New Tools, New Methods, volume 3 of English Corpus Linguistics, pages 197–214. Pe… find
  17. Christoph Müller and Michael Strube. 2003. Multi-Level Annotation in MMAX. In Proceedings of the Fourth SIGdial Workshop of Discourse and Dialogue, pages 198–207. find
  18. Bruno Oberle. 2018. SACR: A Drag-and-Drop Based Tool for Coreference Annotation. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation. European Language Resources Association. find our card
  19. Maciej Ogrodniczuk, Katarzyna Głowińska, Mateusz Kopeć, Agata Savary, and Magdalena Zawisławska. 2013. The Polish Coreference Corpus. In Proceedings of Language and Technology Conference, pages 215–226. Springer International Publishing. find
  20. Constantin Orăsan. 2003. PALinkA: A highly customisable tool for discourse annotation. In Proceedings of the Fourth SIGdial Workshop of Discourse and Dialogue, pages 39–43. find
  21. Lace Padilla, Matthew Kay, and Jessica Hullman. 2021. Uncertainty Visualization, pages 1–18. John Wiley & Sons, Ltd. find
  22. Alex Pang, Craig M. Wittenbrink, and Suresh K. Lodha. 1997. Approaches to uncertainty visualization. The Visual Computer, 13(8):370–390. find
  23. Massimo Poesio and Ron Artstein. 2005. The Reliability of Anaphoric Annotation, Reconsidered: Taking Ambiguity into Account. In Proceedings of the Workshop on Frontiers in Corpus Annotations II: Pie in the Sky, pages 76–83. Association for Computational Linguistics. find our card
  24. Massimo Poesio, Maris Camilleri, Paloma Carretero Garcia, Juntao Yu, and Mark-Christoph Müller. 2024. The ARRAU 3.0 Corpus. In Proceedings of the 5th Workshop on Computational Approaches to Discourse, pages 127–138. find
  25. Massimo Poesio, Jon Chamberlain, Silviu Paun, Juntao Yu, Alexandra Uma, and Udo Kruschwitz. 2019. A crowdsourced corpus of multiple judgments and disagreement on anaphoric interpretation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguis… find our card
  26. Massimo Poesio, Nils Diewald, Maik Stührenberg, Jon Chamberlain, Daniel Jettka, Daniela Goecke, and Udo Kruschwitz. 2012. Markup infrastructure for the anaphoric bank: Supporting web collaboration. In Alexander Mehler, Kai-Uwe Kühnberger, Henning Lobin, Harald Lüngen, Angelika Storrer, and Andreas W… find
  27. Massimo Poesio and Uwe Reyle. 2001. Underspecification in anaphoric reference. In Proceedings of the Fourth International Workshop on Computational Semantics, pages 286–300. find
  28. Massimo Poesio, Patrick Sturt, Ron Arstein, and Ruth Filik. 2006. Underspecification and anaphora: Theoretical issues and preliminary evidence. Discourse Processes, 42(2):157–175. find
  29. Ellen F. Prince. 1992. The ZPG letter: subjects, definiteness, and information status. In S. Thompson and W. Mann, editors, Discourse description: diverse analyses of a fund-raising text, pages 295–325. John Benjamins. find
  30. Marta Recasens, Eduard Hovy, and M. Antònia Martí. 2011. Identity, non-identity, and near-identity: Addressing the complexity of coreference. Lingua, 121(6):1138–1152. find
  31. Marta Recasens, M. Antonia Martí, and Constantin Orasan. 2012. Annotating near-identity from coreference disagreements. In Proceedings of the Eighth International Conference on Language Resources and Evaluation, pages 165–172. find our card
  32. Nils Reiter. 2018. CorefAnnotator - A New Annotation Tool for Entity References. In Abstracts of EADH: Data in the Digital Humanities. find
  33. Nils Reiter. 2025. nilsreiter/CorefAnnotator. https://github.com/nilsreiter/CorefAnnotator. ↗
  34. Dominik Sacha, Hansi Senaratne, Bum Chul Kwon, Geoffrey P. Ellis, and Daniel A. Keim. 2016. The role of uncertainty, awareness, and trust in visual analytics. IEEE Transactions on Visualization and Computer Graphics, 22(1):240–249. find
  35. Christoph Schulz, Arlind Nocaj, Jochen Görtler, Oliver Deussen, Ulrik Brandes, and Daniel Weiskopf. 2017. Probabilistic graph layout for uncertain network visualization. IEEE Transactions on Visualization and Computer Graphics, 23(1):531–540. find
  36. Michael Sedlmair, Miriah Meyer, and Tamara Munzner. 2012. Design Study Methodology: Reflections from the Trenches and the Stacks. IEEE Transactions on Visualization and Computer Graphics, 18(12):2431–2440. find
  37. Maik Stührenberg, Daniela Goecke, Nils Diewald, Alexander Mehler, and Irene Cramer. 2007. Web-based annotation of anaphoric relations and lexical chains. In Proceedings of the Linguistic Annotation Workshop, pages 140–147. Association for Computational Linguistics. find
  38. Olga Uryupina, Ron Artstein, Antonella Bristot, Federica Cavicchio, Francesca Delogu, Kepa Rodriguez, and Massimo Poesio. 2020. Annotating a broad range of anaphoric phenomena, in a variety of genres: the ARRAU corpus. Journal of Natural Language Engineering, 26(1). find
  39. Yannick Versley. 2008. Vagueness and Referential Ambiguity in a Large-Scale Annotated Corpus. Research on Language and Computation, 6(3-4):333–353. find
  40. Luis von Ahn. 2006. Games with a purpose. Computer, 39(6):92–94. find
  41. Daniel Weiskopf. 2022. Uncertainty visualization: Concepts, methods, and applications in biological data visualization. Frontiers in Bioinformatics, Volume 2 - 2022. find
  42. Antoine Widlöcher and Yann Mathet. 2012. The Glozz platform: a corpus annotation and mining tool. In Proceedings of the 2012 ACM symposium on Document engineering, pages 171–180. ACM. find
  43. Bingyang Ye, Jingxuan Tu, and James Pustejovsky. 2023. Scalar anaphora: Annotating degrees of coreference in text. In Proceedings of the Sixth Workshop on Computational Models of Reference, Anaphora and Coreference, pages 28–38. Association for Computational Linguistics. find
  44. Juntao Yu, Silviu Paun, Maris Camilleri, Paloma Carretero Garcia, Jon Chamberlain, Udo Kruschwitz, and Massimo Poesio. 2022. Aggregating Crowdsourced and Automatic Judgments to Scale Up a Corpus of Anaphoric Reference for Fiction and Wikipedia Texts. find
  45. Amir Zeldes. 2022. Opinion Piece: Can we Fix the Scope for Coreference?: Problems and Solutions for Benchmarks beyond OntoNotes. Dialogue & Discourse, 13(1):41–62. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/paetzold-2026-ambicorefvis-tool-visualizing-coreferential-ambiguity/