📰 Workshop on Evaluation and Comparison of NLP Systems 2020 — accepted papers

Workshop on Evaluation and Comparison of NLP Systems 2020 is the 2020 edition of the Workshop on Evaluation and Comparison of NLP Systems, held in Online, November 2020 (publisher: Association for Computational Linguistics). The ACL Anthology lists 17 papers of it; our library holds 3 with our summaries. Below: the most cited papers, every accepted paper, and ours with their topics, authors and institutions.

Most cited papers

By their OpenAlex citations; a paper with our summary links to its page here, the others to the original.

1. Probabilistic Extension of Precision, Recall, and F1 Score for More Thorough Evaluation of Classification ModelsReda Yacouby, Dustin Axman778 citations
2. Fill in the BLANC: Human-free quality estimation of document summariesOleg Vasilyev, Vedant Dharnidharka, John Bohannon77 citations · our summary
3. ViLBERTScore: Evaluating Image Caption Using Vision-and-Language BERTHwanhee Lee, Seunghyun Yoon, Franck Dernoncourt et al.43 citations
4. A survey on Recognizing Textual Entailment as an NLP EvaluationAdam Poliak41 citations · our summary
5. Are Some Words Worth More than Others?Shiran Dudy, Steven Bedrick31 citations
6. On Aligning OpenIE Extractions with Knowledge Bases: A Case StudyKiril Gashteovski, Rainer Gemulla, Bhushan Kotnis et al.22 citations
7. Item Response Theory for Efficient Human Evaluation of ChatbotsJoão Sedoc, Lyle Ungar21 citations
8. Truth or Error? Towards systematic analysis of factual errors in abstractive summariesKlaus-Michael Lux, Maya Sappelli, Martha Larson14 citations
9. Best Practices for Crowd-based Evaluation of German Summarization: Comparing Crowd, Expert and Automatic EvaluationNeslihan Iskender, Tim Polzehl, Sebastian Möller9 citations
10. Grammaticality and Language ModellingJingcheng Niu, Gerald Penn5 citations

All accepted papers (17)

The most cited first; 3 with our summary link to their page here, the others to the original.

1. Probabilistic Extension of Precision, Recall, and F1 Score for More Thorough Evaluation of Classification ModelsReda Yacouby, Dustin Axman778 citations
2. Fill in the BLANC: Human-free quality estimation of document summariesOleg Vasilyev, Vedant Dharnidharka, John Bohannon77 citations · our summary
3. ViLBERTScore: Evaluating Image Caption Using Vision-and-Language BERTHwanhee Lee, Seunghyun Yoon, Franck Dernoncourt et al.43 citations
4. A survey on Recognizing Textual Entailment as an NLP EvaluationAdam Poliak41 citations · our summary
5. Are Some Words Worth More than Others?Shiran Dudy, Steven Bedrick31 citations
6. On Aligning OpenIE Extractions with Knowledge Bases: A Case StudyKiril Gashteovski, Rainer Gemulla, Bhushan Kotnis et al.22 citations
7. Item Response Theory for Efficient Human Evaluation of ChatbotsJoão Sedoc, Lyle Ungar21 citations
8. Truth or Error? Towards systematic analysis of factual errors in abstractive summariesKlaus-Michael Lux, Maya Sappelli, Martha Larson14 citations
9. Best Practices for Crowd-based Evaluation of German Summarization: Comparing Crowd, Expert and Automatic EvaluationNeslihan Iskender, Tim Polzehl, Sebastian Möller9 citations
10. Grammaticality and Language ModellingJingcheng Niu, Gerald Penn5 citations
11. BLEU Neighbors: A Reference-less Approach to Automatic EvaluationKawin Ethayarajh, Dorsa Sadigh4 citations
12. Evaluating Word Embeddings on Low-Resource LanguagesNathan Stringham, Mike Izbicki4 citations · our summary
13. Improving Text Generation Evaluation with Batch Centering and Tempered Word Mover DistanceXi Chen, Nan Ding, Tomer Levinboim et al.4 citations
14. ClusterDataSplit: Exploring Challenging Clustering-Based Data Splits for Model Performance EvaluationHanna Wecker, Annemarie Friedrich, Heike Adel3 citations
15. One of these words is not like the other: a reproduction of outlier identification using non-contextual word representationsJesper Brink Andersen, Mikkel Bak Bertelsen, Mikkel Hørby Schou et al.3 citations
16. Artemis: A Novel Annotation Methodology for Indicative Single Document SummarizationRahul Jha, Keping Bi, Yang Li et al.1 citation
17. On the Evaluation of Machine Translation n-best ListsJacob Bremerman, Huda Khayrallah, Douglas Oard et al.

📄 Papers with our summary in our library (3)

The most cited first (OpenAlex citations plus our library's).

1. Fill in the BLANC: Human-free quality estimation of document summariesWorkshop on Evaluation and Comparison of NLP SystemsOleg Vasilyev, Vedant Dharnidharka, John Bohannon2020 · 154 citations
2. A survey on Recognizing Textual Entailment as an NLP EvaluationWorkshop on Evaluation and Comparison of NLP SystemsAdam Poliak2020 · 76 citations
3. Evaluating Word Embeddings on Low-Resource LanguagesNathan Stringham, Mike Izbicki2020 · 8 citations

🧭 Top topics

Topics of its works in our library, with the number of works on each.

📏Evaluation & benchmarks (2)🌱Low-resource languages (1)🧮Reasoning (1)🕎Hebrew & Aramaic (1)Biblical Hebrew (1)

📚 Fields

🎭Arts and Humanities (3)🖥️Computer Science (3)

🧑‍🔬 Top authors

Authors with a page who published here most often: their works here, and the citations of those works.

🏛️ Top institutions

Institutions on the bylines of its works here, by the number of works.

🎓Pomona CollegeClaremont · United States1 work here🎓Columbia UniversityNew York · United States1 work here🎓Barnard CollegeNew York · United States1 work here

Place and dates from the event's site or the ACL Anthology; works and citations from OpenAlex and our library.

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/venues/eval4nlp/2020/