📰 Workshop on Evaluation and Comparison of NLP Systems 2021 — accepted papers

Workshop on Evaluation and Comparison of NLP Systems 2021 is the 2021 edition of the Workshop on Evaluation and Comparison of NLP Systems, held in Punta Cana, Dominican Republic, November 2021 (publisher: Association for Computational Linguistics). The ACL Anthology lists 24 papers of it; our library holds 1 with our summaries. Below: the most cited papers, every accepted paper, and ours with their topics, authors and institutions.

Most cited papers

By their OpenAlex citations; a paper with our summary links to its page here, the others to the original.

1. HinGE: A Dataset for Generation and Evaluation of Code-Mixed Hinglish TextVivek Srivastava, Mayank Singh35 citations
2. The Eval4NLP Shared Task on Explainable Quality Estimation: Overview and ResultsMarina Fomicheva, Piyawat Lertvittayakumjorn, Wei Zhao et al.20 citations
3. Statistically Significant Detection of Semantic Shifts using Contextual Word EmbeddingsYang Liu, Alan Medlar, Dorota Glowacka18 citations
4. Developing a Benchmark for Reducing Data Bias in Authorship AttributionBenjamin Murauer, Günther Specht10 citations
5. StoryDB: Broad Multi-language Narrative DatasetAlexey Tikhonov, Igor Samenko, Ivan P. Yamshchikov8 citations
6. SeqScore: Addressing Barriers to Reproducible Named Entity Recognition EvaluationChester Palen-Michel, Nolan Holley, Constantine Lignos7 citations
7. Validating Label Consistency in NER Data AnnotationQingkai Zeng, Mengxia Yu, Wenhao Yu et al.7 citations
8. Error Identification for Machine Translation with Metric Embedding and AttentionRaphael Rubino, Atsushi Fujita, Benjamin Marie6 citations
9. IST-Unbabel 2021 Submission for the Explainable Quality Estimation Shared TaskMarcos Treviso, Nuno M. Guerreiro, Ricardo Rei et al.6 citations
10. How Emotionally Stable is ALBERT? Testing Robustness with Stochastic Weight Averaging on a Sentiment Analysis TaskUrja Khurana, Eric Nalisnick, Antske Fokkens5 citations

All accepted papers (24)

The most cited first; 1 with our summary link to their page here, the others to the original.

1. HinGE: A Dataset for Generation and Evaluation of Code-Mixed Hinglish TextVivek Srivastava, Mayank Singh35 citations
2. The Eval4NLP Shared Task on Explainable Quality Estimation: Overview and ResultsMarina Fomicheva, Piyawat Lertvittayakumjorn, Wei Zhao et al.20 citations
3. Statistically Significant Detection of Semantic Shifts using Contextual Word EmbeddingsYang Liu, Alan Medlar, Dorota Glowacka18 citations
4. Developing a Benchmark for Reducing Data Bias in Authorship AttributionBenjamin Murauer, Günther Specht10 citations
5. StoryDB: Broad Multi-language Narrative DatasetAlexey Tikhonov, Igor Samenko, Ivan P. Yamshchikov8 citations
6. SeqScore: Addressing Barriers to Reproducible Named Entity Recognition EvaluationChester Palen-Michel, Nolan Holley, Constantine Lignos7 citations
7. Validating Label Consistency in NER Data AnnotationQingkai Zeng, Mengxia Yu, Wenhao Yu et al.7 citations
8. Error Identification for Machine Translation with Metric Embedding and AttentionRaphael Rubino, Atsushi Fujita, Benjamin Marie6 citations
9. IST-Unbabel 2021 Submission for the Explainable Quality Estimation Shared TaskMarcos Treviso, Nuno M. Guerreiro, Ricardo Rei et al.6 citations
10. How Emotionally Stable is ALBERT? Testing Robustness with Stochastic Weight Averaging on a Sentiment Analysis TaskUrja Khurana, Eric Nalisnick, Antske Fokkens5 citations
11. Writing Style Author Embedding EvaluationEnzo Terreau, Antoine Gourru, Julien Velcin5 citations
12. Explaining Errors in Machine Translation with Absolute Gradient EnsemblesMelda Eksi, Erik Gelbing, Jonathan Stieber et al.3 citations
13. MIPE: A Metric Independent Pipeline for Effective Code-Mixed NLG EvaluationAyush Garg, Sammed Kagi, Vivek Srivastava et al.3 citations
14. Reference-Free Word- and Sentence-Level Translation Evaluation with Token-Matching MetricsChristoph Wolfgang Leiter3 citations
15. Trainable Ranking Models to Evaluate the Semantic Accuracy of Data-to-Text Neural GeneratorNicolas Garneau, Luc Lamontagne3 citations
16. ESTIME: Estimation of Summary-to-Text Inconsistency by Mismatched EmbeddingsOleg Vasilyev, John Bohannon2 citations
17. Testing Cross-Database Semantic Parsers With Canonical UtterancesHeather Lent, Semih Yavuz, Tao Yu et al.2 citations
18. What is SemEval evaluating? A Systematic Analysis of Evaluation Campaigns in NLPOskar Wysocki, Malina Florea, Dónal Landers et al.2 citations
19. Differential Evaluation: a Qualitative Analysis of Natural Language Processing System Behavior Based Upon Data Resistance to ProcessingLucie Gianola, Hicham El Boukkouri, Cyril Grouin et al.1 citation
20. Error-Sensitive Evaluation for Ordinal Target VariablesDavid Chen, Maury Courtland, Adam Faulkner et al.1 citation
21. Explainable Quality Estimation: CUNI Eval4NLP SubmissionPeter Polák, Muskaan Singh, Ondřej Bojar1 citation
22. Referenceless Parsing-Based Evaluation of AMR-to-English GenerationEmma Manning, Nathan Schneider1 citation · our summary
23. The UMD Submission to the Explainable MT Quality Estimation Shared Task: Combining Explanation Models with Sequence LabelingTasnim Kabir, Marine Carpuat1 citation
24. Evaluation of Unsupervised Automatic Readability Assessors Using Rank CorrelationsYo Ehara

📄 Papers with our summary in our library (1)

The most cited first (OpenAlex citations plus our library's).

1. Referenceless Parsing-Based Evaluation of AMR-to-English GenerationEmma Manning, Nathan Schneider2021 · 9 citations

🧭 Top topics

Topics of its works in our library, with the number of works on each.

📏Evaluation & benchmarks (1)🧾Semantic parsing & meaning representation (1)English (1)

📚 Fields

🎭Arts and Humanities (1)🖥️Computer Science (1)

🧑‍🔬 Top authors

Authors with a page who published here most often: their works here, and the citations of those works.

🏛️ Top institutions

Institutions on the bylines of its works here, by the number of works.

🎓Georgetown UniversityWashington · United States1 work here

Place and dates from the event's site or the ACL Anthology; works and citations from OpenAlex and our library.

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/venues/eval4nlp/2021/