Elamur Research⌕
☰
home / papers / 2021 / Event Coreference Data (Almost) for Free: Mining Hyperlinks from Online News

Event Coreference Data (Almost) for Free: Mining Hyperlinks from Online News

Summary

Cross-document event coreference resolution (CDCR) asks which event mentions across many documents describe the same real-world event. Expert annotation is slow and costly, so existing gold corpora are small and narrow in domain. Bugert and Gurevych treat news hyperlinks as a free signal. When journalists mention an event, they often link to an article about it. Anchors in different articles that point to the same URL can therefore be read as coreferring mentions. The authors built an automatic pipeline over Common Crawl that extracts article text, removes duplicates, and discards links unlikely to point at events. Running it on 40 English-language outlets produced HyperCoref, a silver-standard corpus several orders of magnitude larger than hand-annotated resources. They trained a state-of-the-art CDCR model on small slices of this data and tested it on ECB+, FCC-T and GVC. Its scores came close to those of the same model trained on gold data. Almost every pipeline step is language-agnostic, so the method could supply CDCR training data for languages with no annotated corpora, Ukrainian among them.

Key points

  • HyperCoref size: about 2.05M documents, 2.74M hyperlink-based event mentions and 0.81M clusters, drawn from 40 outlets up to the October 2020 Common Crawl snapshot.

  • Pipeline: the authors use newspaper3k to extract text and remove near-duplicates with locality sensitive hashing and TF-IDF clustering. Non-event links are filtered out in four ways:

    • links to other domains,
    • URLs outside the 90% most frequent prefixes,
    • links involving high in-degree or out-degree pages,
    • handwritten rules such as /tag/ or "click here".

    Finally, any anchor-and-URL pair that recurs across pages is removed.

  • Results: the evaluated model follows Cattan et al. (2021). Trained only on silver data from the ABC and BBC subsets (about 2% of all mentions), it scored between 4 pp CoNLL F1 below and 4 pp above gold-trained models. The best setup trained on HyperCoref and tuned hyperparameters on gold data.

  • Comparison with WEC: on ECB+, the ABC-trained model reached 75.04 CoNLL F1. The same RoBERTa architecture trained on the Wikipedia-based WEC corpus reached 69.10.

  • Multilinguality: only the handwritten rules depend on language. Trial runs on Dutch, German and French news gave quality comparable to English. Data and model are released at github.com/UKPLab/emnlp2021-hypercoref-cdcr.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

7 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 6 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.emnlp-main.38

@inproceedings{bugert2021event,
  title = {{Event Coreference Data (Almost) for Free: Mining Hyperlinks from Online News}},
  author = {Bugert, Michael and Gurevych, Iryna},
  booktitle = {Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing},
  year = {2021},
  doi = {10.18653/v1/2021.emnlp-main.38},
  url = {https://doi.org/10.18653/v1/2021.emnlp-main.38}
}

References

47 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Amit Bagga and Breck Baldwin. 1998. Entity-Based Cross-Document Coreferencing Using the Vector Space Model. In 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, Volume 1, pages 79–85, Montreal, Quebec, Canada. Associa… find
  2. Shany Barhom, Vered Shwartz, Alon Eirew, Michael Bugert, Nils Reimers, and Ido Dagan. 2019. Revisiting Joint Modeling of Cross-document Entity and Event Coreference Resolution. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4179–4189, Florence, Ital… find our card
  3. Ari Bornstein, Arie Cattan, and Ido Dagan. 2020. CoRefi: A Crowd Sourcing Suite for Coreference Annotation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 205–215, Online. Association for Computational Linguistics. find
  4. Michael Bugert, Nils Reimers, Shany Barhom, Ido Dagan, and Iryna Gurevych. 2020. Breaking the Subtopic Barrier in Cross-Document Event Coreference Resolution. In Text2Story@ ECIR, pages 23– 29. find
  5. Michael Bugert, Nils Reimers, and Iryna Gurevych. 2021. Generalizing Cross-Document Event Coreference Resolution Across Multiple Corpora. Computational Linguistics, 47(3). find our card
  6. Avi Caciularu, Arman Cohan, Iz Beltagy, Matthew E. Peters, Arie Cattan, and Ido Dagan. 2021. Cross-Document Language Modeling. arXiv preprint. find
  7. Arie Cattan, Alon Eirew, Gabriel Stanovsky, Mandar Joshi, and Ido Dagan. 2021. Cross-document Coreference Resolution over Predicted Mentions. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 5100–5107, Online. Association for Computational Linguistics. find our card
  8. Nathanael Chambers and Dan Jurafsky. 2008. Unsupervised Learning of Narrative Event Chains. In Proceedings of ACL-08: HLT, pages 789–797, Columbus, Ohio. Association for Computational Linguistics. find
  9. Prafulla Kumar Choubey and Ruihong Huang. 2017. Event Coreference Resolution by Iteratively Unfolding Inter-dependencies among Events. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2114–2123, Copenhagen, Denmark. Association for Computational Lingui… find our card
  10. Prafulla Kumar Choubey and Ruihong Huang. 2021. Automatic Data Acquisition for Event Coreference Resolution. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 1185– 1196, Online. Association for Computational Linguistic… find
  11. Agata Cybulska and Piek Vossen. 2014a. Guidelines for ECB+ Annotation of Events and their Coreference. Technical report, VU University Amsterdam. find
  12. Agata Cybulska and Piek Vossen. 2014b. Using a Sledgehammer to Crack a Nut? Lexical Diversity and Event Coreference Resolution. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 4545– 4552, Reykjavik, Iceland. European Language Resources Assoc… find our card
  13. Parag Pravin Dakle, Takshak Desai, and Dan Moldovan. 2020. A Study on Entity Resolution for Email Conversations. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 65–73, Marseille, France. European Language Resources Association. find
  14. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find
  15. Alon Eirew, Arie Cattan, and Ido Dagan. 2021. WEC: Deriving a Large-scale Cross-document Event Coreference dataset from Wikipedia. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2498–2510, Onlin… find our card
  16. Joshua Eisenberg and Michael Sheriff. 2020. Automatic extraction of personal events from dialogue. In Proceedings of the First Joint Workshop on Narrative Understanding, Storylines, and Events, pages 63–71, Online. Association for Computational Linguistics. find
  17. David Graff, Junbo Kong, Ke Chen, and Kazuaki Maeda. 2005. English Gigaword Second Edition. find
  18. Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. 2020. Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8342–8360, Online. A… find
  19. Eduard Hovy, Teruko Mitamura, Felisa Verdejo, Jun Araki, and Andrew Philpot. 2013. Events are Not Simple: Identity, Non-Identity, and Quasi-Identity. In Workshop on Events: Definition, Detection, Coreference, and Representation, pages 21– 28, Atlanta, Georgia. Association for Computational Linguisti… find
  20. Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirectional LSTM-CRF Models for Sequence Tagging. arXiv preprint. find our card
  21. Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end Neural Coreference Resolution. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 188–197, Copenhagen, Denmark. Association for Computational Linguistics. find our card
  22. Jure Leskovec, Anand Rajaraman, and Jeffrey David Ullman. 2020. Mining of Massive Datasets, 3 edition. Cambridge University Press, USA. find
  23. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv preprint. find
  24. Xiaoqiang Luo. 2005. On Coreference Resolution Performance Metrics. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, pages 25–32, Vancouver, British Columbia, Canada. Association for Computational Linguistics. find our card
  25. Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David Mc-Closky. 2014. The Stanford CoreNLP natural language processing toolkit. In Association for Computational Linguistics (ACL) System Demonstrations, pages 55–60. find our card
  26. George A. Miller. 1995. WordNet: A Lexical Database for English. Commun. ACM, 38(11):39–41. find our card
  27. Anne-Lyse Minard, Manuela Speranza, Ruben Urizar, Begoña Altuna, Marieke van Erp, Anneleen Schoen, and Chantal van Son. 2016. MEANTIME, the NewsReader Multilingual Event and Time Corpus. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 4417–4… find
  28. Nafise Sadat Moosavi and Michael Strube. 2016. Which Coreference Evaluation Metric Do You Trust? A Proposal for a Link-based Entity Aware Metric. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 632–642, Berlin, Germany. Associ… find our card
  29. Haoruo Peng, Yangqiu Song, and Dan Roth. 2016. Event Detection and Co-reference with Minimal Supervision. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 392–402, Austin, Texas. Association for Computational Linguistics. find our card
  30. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. CoNLL-2012 Shared Task: Modeling Multilingual Unrestricted Coreference in OntoNotes. In Joint Conference on EMNLP and CoNLL - Shared Task, pages 1–40, Jeju Island, Korea. Association for Computational Linguisti… find our card
  31. Yada Pruksachatkun, Jason Phang, Haokun Liu, Phu Mon Htut, Xiaoyi Zhang, Richard Yuanzhe Pang, Clara Vania, Katharina Kann, and Samuel R. Bowman. 2020. Intermediate-Task Transfer Learning with Pretrained Language Models: When and Why Does It Work? In Proceedings of the 58th Annual Meeting of the Ass… find our card
  32. James Pustejovsky, Patrick Hanks, Roser Sauri, Andrew See, Robert Gaizauskas, Andrea Setzer, Dragomir Radev, Beth Sundheim, David Day, Lisa Tasks. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7882–7926, Online. Association for Computational… find
  33. Ferro, et al. 2003. The TimeBank Corpus. In Corpus linguistics, volume 2003, page 40. Lancaster, UK. find
  34. Nils Reimers. 2018. Universal Machine Learning Methods for Detecting and Temporal Anchoring of Events. Ph.D. thesis, Technische Universität, Darmstadt. find
  35. Dan Roth, Heng Ji, Ming-Wei Chang, and Taylor Cassidy. 2014. Wikification and Beyond: The Challenges of Entity and Concept Grounding. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics: Tutorials, page 7, Baltimore, Maryland, USA. Association for Computational… find
  36. Sebastian Schlag. 2020. High-Quality Hypergraph Partitioning. Ph.D. thesis, Karlsruhe Institute of Technology, Germany. find
  37. Zijie J. Wang, Dongjin Choi, Shenyu Xu, and Diyi Yang. 2021. Putting Humans in the Natural Language Processing Loop: A Survey. arXiv preprint. find
  38. Xiaodong Yu, Wenpeng Yin, and Dan Roth. 2020. Paired Representation Learning for Event and Entity Coreference. arXiv preprint. find
  39. Yutao Zeng, Xiaolong Jin, Saiping Guan, Jiafeng Guo, and Xueqi Cheng. 2020. Event Coreference Resolution with their Paraphrases and Argument-aware Embeddings. In Proceedings of the 28th International Conference on Computational Linguistics, pages 3084–3094, Barcelona, Spain (Online). International C… find our card
  40. Avirup Sil, Fei Huang, and Alexander Yates. 2010. Extracting Action and Event Semantics from Web Text. In AAAI Fall Symposium: Commonsense Knowledge, pages 108–113. find
  41. Matthew Sims, Jong Ho Park, and David Bamman. 2019. Literary Event Detection. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3623–3634, Florence, Italy. Association for Computational Linguistics. find
  42. Sameer Singh, Amarnag Subramanya, Fernando Pereira, and Andrew McCallum. 2012. Wikilinks: A Large-scale Cross-Document Coreference Corpus Labeled via Links to Wikipedia. Technical report, Department of Computer Science, University of Massachusetts, Amherst. find
  43. Shyam Upadhyay, Nitish Gupta, Christos Christodoulopoulos, and Dan Roth. 2016. Revisiting the Evaluation for Cross Document Event Coreference. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1949–1958, Osaka, Japan. find our card
  44. The COLING 2016 Organizing Committee. find
  45. Marc Vilain, John Burger, John Aberdeen, Dennis Connolly, and Lynette Hirschman. 1995. A Model-Theoretic Coreference Scoring Scheme. In Sixth Message Understanding Conference (MUC-6): Proceedings of a Conference Held in Columbia, Maryland, November 6-8, 1995, pages 45–52. find our card
  46. Piek Vossen, Filip Ilievski, Marten Postma, and Rox-ane Segers. 2018. Don’t Annotate, but Validate: a Data-to-Text Method for Capturing Event Data. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), pages 3034–3042, Paris, France. European Langu… find
  47. Tu Vu, Tong Wang, Tsendsuren Munkhdalai, Alessandro Sordoni, Adam Trischler, Andrew Mattarella-Micke, Subhransu Maji, and Mohit Iyyer. 2020. Exploring and Predicting Transferability across NLP find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/bugert-2021-event-coreference-data-almost-free/