Iryna Gurevych · page 3
Papers
- Sensitivity, Performance, Robustness: Deconstructing the Effect of Sociodemographic Prompting (2024 · 69 citations)
Annotators of subjective NLP tasks such as toxicity labeling often disagree, partly because of their gender, age, education and politics. - IMPLI: Investigating NLI Models' Performance on Figurative Language (2022 · 68 citations)
IMPLI pairs sentences containing an idiom or metaphor with a literal sentence that is or is not entailed by it: 24,029 silver pairs and 1,831 hand-written gold pairs. - NLPeer: A Unified Resource for the Computational Study of Peer Review (2023 · 64 citations)
Computational research on peer review has been held back by three gaps. Most available review data comes from machine learning conferences on OpenReview.net and carries no explicit license. Studies rarely look beyond one research community. Each dataset also uses its own task-specific format. NLPeer addresses all… - Integrating Deep Linguistic Features in Factuality Prediction over Unified Datasets (2017 · 62 citations)
Factuality prediction asks how strongly a source, usually the author, commits to a predicate being true, possible or false. - Revise and Resubmit: An Intertextual Model of Text-based Collaboration in Peer Review (2022 · 54 citations)
Peer review is a clear case of text-based collaboration. Referees comment on a manuscript, authors revise it, and the cycle repeats. Supporting this work computationally requires reasoning across pairs of documents, yet NLP has mostly analyzed texts in isolation. Existing review datasets also cover only… - Concept-Map-Based Multi-Document Summarization using Concept Coreference Resolution and Global Importance Optimization (2017 · 43 citations)
Falke, Meyer and Gurevych study concept-map-based multi-document summarization. The goal is to turn a topic-focused document collection into a small, connected labeled graph that captures the key concepts and their relations. Earlier systems had three weaknesses: they left duplicate concepts unmerged, picked… - AmbiFC: Fact-Checking Ambiguous Claims with Evidence (2024 · 42 citations)
Most fact-checking benchmarks expect one veracity verdict per claim. Real evidence, however, often allows several reasonable readings, and human annotators then disagree legitimately. AmbiFC addresses this gap. It is an English dataset of about 10k claims, built by having Mechanical Turk workers turn real yes/no… - SemEval-2026 Task 13: Detecting Machine-Generated Code with Multiple Programming Languages, Generators, and Application Scenarios (2026 · 41 citations)
This shared task targets the problem of telling whether source code was written by people or produced by LLMs. - Dialogue Coherence Assessment Without Explicit Dialogue Act Labels (2020 · 40 citations)
Automatic dialogue evaluation needs a way to judge coherence. Word-overlap scores such as BLEU and costly human ratings both fall short. Earlier dialogue coherence models reused monologue features such as entity transitions and then added dialogue act (DA) labels on top. This has two costs: utterances are reduced to… - LINSPECTOR: Multilingual Probing Tasks for Word Representations (2020 · 35 citations)
Most probing research tests English representations, often at the sentence level. English is typologically unusual, though, because many languages encode grammatical information inside words through morphology. To fill this gap, the authors present LINSPECTOR, a multilingual suite of word-level probes covering 24… - Generalizing Cross-Document Event Coreference Resolution Across Multiple Corpora (2021 · 33 citations)
Cross-document event coreference resolution (CDCR) requires a system to find event mentions across a document collection and group those that denote the same real-world event. - To Exhibit is not to Loiter: A Multilingual, Sense-Disambiguated Wiktionary for Measuring Verb Similarity (2012 · 29 citations)
The paper turns Wiktionary's semantic relations and translations into a sense-disambiguated resource. Each relation or translation is linked to one sense of its target word by a method that combines features such as a monolingual-only candidate, a source-overlap score, a lexical-expansion score, a relation score and… - FrameNet on the Way to Babel: Creating a Bilingual FrameNet Using Wiktionary as Interlingual Connection (2013 · 26 citations)
FrameNet supports semantic role labeling, but comparable lexicons are scarce outside English. Earlier transfer methods either relied on parallel corpora, which suffer from polysemy errors, or on costly expert resources such as EuroWordNet. Wikipedia-based work covered almost only nouns. Hartmann and Gurevych instead… - Finding Convincing Arguments Using Scalable Bayesian Preference Learning (2018 · 24 citations)
Ranking online arguments by persuasiveness usually requires expensive, carefully curated annotations. Earlier systems trained on crowdsourced pairwise comparisons used a multi-step pipeline: MACE cleaned noisy labels, PageRank converted them into scores, and SVM or BiLSTM models were then trained on the result.… - AICD Bench: A Challenging Benchmark for AI-Generated Code Detection (2026 · 23 citations)
Tools for detecting machine-written source code are usually tested on narrow, custom datasets. These cover few languages and generators and mostly test human-vs-AI separation within the same distribution, which tells little about how detectors behave in real use. The authors address this with AICD Bench, a unified… - BinLin: A Simple Method of Dependency Tree Linearization (2018 · 19 citations)
The paper is a system report for the shallow track of SR'18. The input is a bag of lemmatised UD nodes with their tags and morphology. The system works in two stages: it first orders the lemmas, and then it conjugates and inflects each word. The contribution is a sequential way of ordering lemmas. It is simple and it… - Coreference Reasoning in Machine Reading Comprehension (2021 · 11 citations)
The paper argues that Quoref, a benchmark built to test whether machine reading comprehension (MRC) models resolve coreference, overstates how well they do. - Event Coreference Data (Almost) for Free: Mining Hyperlinks from Online News (2021 · 11 citations)
Cross-document event coreference resolution (CDCR) asks which event mentions across many documents describe the same real-world event. - WordNet—Wikipedia—Wiktionary: Construction of a Three-way Alignment (2014 · 11 citations)
The paper builds a three-way alignment of senses from existing pairwise alignments of WordNet with Wikipedia and with Wiktionary. - Overview of PragTag-2023: Low-Resource Multi-Domain Pragmatic Tagging of Peer Reviews (2023 · 10 citations)
Most NLP work on peer review has used reviews from machine learning conferences. This raises two concerns: models may fail when applied to other research fields, and labeled review data is scarce and costly to produce. The PragTag-2023 shared task, held at the 10th Workshop on Argument Mining, tested both problems… - Problem Solving Through Human–AI Preference-based Cooperation (2025 · 9 citations)
Generative AI systems, including LLMs, vision-language models and code models, still struggle with complex expert-domain tasks. - The Nature of NLP: Analyzing Contributions in NLP Papers (2025 · 8 citations)
What counts as NLP research is contested. The authors argue that the answer can be found in how papers describe their own contributions. They build a taxonomy that splits contributions into two groups. Artifacts are new methods, datasets or tasks. Knowledge covers new insights about methods, datasets, tasks, language… - Automatic Reviewers Fail to Detect Faulty Reasoning in Research Papers: A New Counterfactual Evaluation Framework (2026 · 5 citations)
LLMs are increasingly used as automatic review generators (ARGs), but studies of their quality disagree. One reason is that these evaluations mix many reviewing skills and often ignore publication context. Dycke and Gurevych isolate one skill that depends only on the paper itself: checking whether results… - Can LLMs Automate Fact-Checking Article Writing? (2026 · 5 citations)
Research on automated fact-checking usually ends at spotting claims, retrieving evidence, assigning a verdict and, at most, producing a short justification. - How Quantization Shapes Bias in Large Language Models (2026 · 3 citations)
Quantization is a common way to shrink LLMs for deployment, but its effect on social bias has received little systematic study.
This page in Markdown → https://md.research.elamur.ai/free/en/authors/iryna-gurevych/page/3/