📰 HumEval 2021 — accepted papers

HumEval 2021 is the 2021 edition of the Workshop on Human Evaluation of NLP Systems (HumEval), held in Online, April 2021 (publisher: Association for Computational Linguistics). The ACL Anthology lists 15 papers of it; our library holds 1 with our summaries. Below: the most cited papers, every accepted paper, and ours with their topics, authors and institutions.

📍 Online · 🗓 April 2021

All accepted papers (15)

The most cited first; 1 with our summary link to their page here, the others to the original.

1. A Case Study of Efficacy and Challenges in Practical Human-in-Loop Evaluation of NLP Systems Using Checklist ↗Shaily Bhatt, Rahul Jain, Sandipan Dandapat et al.
2. A Preliminary Study on Evaluating Consultation Notes With Post-Editing ↗Francesco Moramarco, Alex Papadopoulos Korfiatis, Aleksandar Savkov et al.
3. A View From the Crowd: Evaluation Challenges for Time-Offset Interaction Applications ↗Alberto Chierici, Nizar Habash
4. Detecting Post-Edited References and Their Effect on Human Evaluation ↗Věra Kloudová, Ondřej Bojar, Martin Popel
5. Eliciting Explicit Knowledge From Domain Experts in Direct Intrinsic Evaluation of Word Embeddings for Specialized Domains ↗Goya van Boven, Jelke Bloem
6. Estimating Subjective Crowd-Evaluations as an Additional Objective to Improve Natural Language Generation ↗Jakob Nyberg, Maike Paetzel, Ramesh Manuvinakurike
7. Interrater Disagreement Resolution: A Systematic Procedure to Reach Consensus in Annotation Tasks ↗Yvette Oortwijn, Thijs Ossenkoppele, Arianna Betti
8. Is This Translation Error Critical?: Classification-Based Human and Automatic Machine Translation Evaluation Focusing on Critical Errors ↗Katsuhito Sudoh, Kosuke Takahashi, Satoshi Nakamura
9. It's Commonsense, isn't it? Demystifying Human Evaluations in Commonsense-Enhanced NLG Systems ↗Miruna-Adriana Clinciu, Dimitra Gkatzia, Saad Mahamood
10. On User Interfaces for Large-Scale Document-Level Human Evaluation of Machine Translation Outputs ↗Roman Grundkiewicz, Marcin Junczys-Dowmunt, Christian Federmann et al.
11. Reliability of Human Evaluation for Text Summarization: Lessons Learned and Challenges Ahead ↗Neslihan Iskender, Tim Polzehl, Sebastian Möller
12. The Great Misalignment Problem in Human Evaluation of NLP Methods ↗Mika Hämäläinen, Khalid Alnajjar
13. Towards Document-Level Human MT Evaluation: On the Issues of Annotator Agreement, Effort and Misevaluation ↗Sheila Castilho
14. Towards Objectively Evaluating the Quality of Generated Medical Summaries ↗Francesco Moramarco, Damir Juric, Aleksandar Savkov et al.
15. Trading Off Diversity and Quality in Natural Language GenerationHugh Zhang, Daniel Duckworth, Daphne Ippolito et al.our summary

📄 Papers with our summary in our library (1)

The most cited first (OpenAlex citations plus our library's).

1. Trading Off Diversity and Quality in Natural Language GenerationWorkshop on Human Evaluation of NLP Systems (HumEval)Hugh Zhang, Daniel Duckworth, Daphne Ippolito, et al.2021 · 88 citations

🧭 Top topics

Topics of its works in our library, with the number of works on each.

✍️Text generation (1)

📚 Fields

🎭Arts and Humanities (1)🖥️Computer Science (1)

🧑‍🔬 Top authors

Authors with a page who published here most often: their works here, and the citations of those works.

🏛️ Top institutions

Institutions on the bylines of its works here, by the number of works.

🏢GoogleMountain View · United States1 work here🏢OpenAISan Francisco · United States1 work here🎓University of PennsylvaniaPhiladelphia · United States1 work here🎓Stanford UniversityStanford · United States1 work here

Place and dates from the event's site or the ACL Anthology; works and citations from OpenAlex and our library.

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/venues/humeval/2021/