Graham Neubig · page 3
Papers
- SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents (2023 · 138 citations)
Benchmarks for social intelligence in AI are mostly static, and existing interactive setups tend to cover narrow tasks or lack diverse, goal-driven behavior. - Pushing the Limits of Low-Resource Morphological Inflection (2019 · 137 citations)
Many of the world's languages have rich morphology but only tiny annotated datasets. Neural inflection models that excel with plenty of data break down when they get only a few dozen lemma–tag–form examples. The authors address this setting within the cross-lingual track of SIGMORPHON 2019. Their sequence-to-sequence… - Efficient Nearest Neighbor Language Models (2021 · 130 citations)
Non-parametric language models such as the kNN-LM (Khandelwal et al., 2019) improve perplexity by consulting an external datastore of context–token pairs. - Demystifying Long Chain-of-Thought Reasoning in LLMs (2025 · 119 citations)
The paper investigates how large language models acquire long chain-of-thought (CoT) reasoning, meaning extended traces that include branching, backtracking and self-correction, and which training choices make this behaviour appear reliably. - Testing the Ability of Language Models to Interpret Figurative Language (2022 · 114 citations)
Figurative language is everywhere in everyday speech, yet NLP research has mostly focused on detecting metaphors rather than on whether models can actually work out what a metaphor implies. - Evaluating Explanations: How Much Do Explanations from the Teacher Aid Students? (2022 · 113 citations)
Feature-attribution methods that mark certain input tokens as important are widely used to explain NLP models. - Large Language Models Enable Few-Shot Clustering (2024 · 111 citations)
Clustering without supervision is underspecified, because the algorithm cannot know which grouping the user actually wants. - Fine-grained Hallucination Detection and Editing for Language Models (2024 · 100 citations)
Language models frequently state false facts. Most existing checkers give only a yes/no verdict for an entire response, so every error is treated the same way. The authors argue that factual mistakes come in distinct kinds, each requiring a different amount of verification effort, and they organize these kinds into a… - Learning to Filter Context for Retrieval-Augmented Generation (2023 · 88 citations)
Retrieval-augmented generators usually receive the top retrieved passages wholesale. Retrievers are imperfect, so this input often includes irrelevant or distracting sentences, even inside passages that do contain the answer. The result can be hallucination or copying from the wrong text. The authors propose FILCO… - Agent Workflow Memory (2024 · 86 citations)
LM-based web agents often fail at long, multi-step tasks. They usually treat each task in isolation and rely on a fixed set of examples, so they adapt poorly when tasks or websites change. Agent Workflow Memory (AWM) addresses this by letting the agent build reusable routines ("workflows") from earlier trajectories.… - Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generation (2023 · 84 citations)
Large language models trained by maximum likelihood on web-scraped text can produce toxic, wrong or unhelpful output. - In-Context Learning with Long-Context Models: An In-Depth Exploration (2025 · 81 citations)
Long-context language models can now fit thousands of demonstrations in a prompt, sometimes approaching the size of a whole training set. - WikiAsp: A Dataset for Multi-domain Aspect-based Summarization (2021 · 76 citations)
Most aspect-based summarization research targets a single narrow genre, usually product or restaurant reviews, so models rarely generalize across domains. - Expanding Pretrained Models to Thousands More Languages via Lexicon-based Adaptation (2022 · 73 citations)
Multilingual pretrained models such as mBERT work poorly for languages that had little or no text during pretraining. - From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Models (2024 · 51 citations)
Most LLM progress has been credited to spending more compute during training. The gains from spending compute at inference time have been studied by separate communities that use inconsistent terminology. This survey by Welleck and colleagues (CMU, USC, EleutherAI, Meta FAIR, University of Washington) organizes that… - MixEval: Deriving Wisdom of the Crowd from LLM Benchmark Mixtures (2024 · 47 citations)
Evaluating LLMs reliably is difficult. Closed-answer test sets such as MMLU poorly reflect what real users ask. LLM-judged suites like MT-Bench inherit judge biases and contain few queries. Static sets leak into training data over time, and crowd-voting platforms like Chatbot Arena are slow, costly and hard to… - MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scale (2025 · 40 citations)
Open-source multimodal LLMs tend to struggle with complex reasoning. A major reason is that their instruction-tuning data is mostly recycled from academic VQA-style sets such as VQA, AI2D, and ChartQA, which offer short answers and no step-by-step justification. The authors propose a low-cost pipeline built entirely… - Checklists Are Better Than Reward Models For Aligning Language Models (2025 · 37 citations)
Reinforcement learning works well for tasks with checkable answers. It is much harder to apply to open-ended instruction following, where reward models can be gamed and generic criteria such as "helpfulness" miss request-specific needs. The authors propose Reinforcement Learning from Checklist Feedback (RLCF), which… - Neural Lattice Language Models (2018 · 35 citations)
Conventional LSTM language models require each sentence to have a single fixed tokenization, at either the character or the word level. - Handling Syntactic Divergence in Low-resource Machine Translation (2019 · 33 citations)
Back-translation needs a working reverse system, so it fails when parallel data is tiny and the languages differ in word order. - Evaluating Language Models as Synthetic Data Generators (2025 · 32 citations)
Synthetic data now drives much of language model post-training. Earlier data-generation studies, however, each used different generator LMs, data volumes, student models and benchmarks, so their results cannot be compared to show which LM produces the most useful training data. Kim et al. address this with… - Analyzing the Effect of Entrainment on Dialogue Acts (2016 · 23 citations)
Speakers in a conversation tend to converge on each other's behaviour. This convergence is called entrainment. It is well studied for word choice, but little is known about whether it also shapes the structure of a dialogue. The authors ask three questions. Do partners align in the dialogue acts they choose? Does… - Computational Language Acquisition with Theory of Mind (2023 · 21 citations)
Children learn language through social interaction, partly by reasoning about what other people think, an ability called Theory of Mind (ToM). - Crossing the Threshold: Idiomatic Machine Translation through Retrieval Augmentation and Loss Weighting (2023 · 17 citations)
Idioms carry meanings that cannot be assembled from their individual words, and neural MT systems, including commercial ones, still tend to render them literally. - What Goes Into a LM Acceptability Judgment? Rethinking the Impact of Frequency and Length (2025 · 12 citations)
Comparing language model probabilities with human acceptability ratings requires a linking function. Raw sentence probabilities fall for longer sentences and for sentences containing rare words, while human judgments are largely unaffected by these factors. The standard linking function, SLOR, applies the same fixed…
This page in Markdown → https://md.research.elamur.ai/free/en/authors/graham-neubig/page/3/