Danqi Chen · page 2
Papers
- What In-Context Learning “Learns” In-Context: Disentangling Task Recognition and Task Learning (2023 · 102 citations)
The paper asks what language models actually do when they solve tasks from a few in-context examples. Do they simply identify a task they already know from pre-training, or do they acquire new input-label associations from the prompt? The authors separate these into two abilities. Task recognition (TR) means… - How to Train Long-Context Language Models (Effectively) (2025 · 89 citations)
The paper asks how to turn a short-context pre-trained LM into a strong long-context model through continued pre-training followed by SFT. - Reliable, Adaptable, and Attributable Language Models with Retrieval (2024 · 45 citations)
This position paper by Asai et al. contends that retrieval-augmented LMs should succeed purely parametric LMs as the main design for future language models. It contains no new experiments, datasets, or models. Instead, it surveys prior work and lists five weaknesses of parametric models: hallucinated facts, outputs… - Long-Context Language Modeling with Parallel Context Encoding (2024 · 44 citations)
Decoder-only LLMs struggle with long inputs for three reasons. Positional encodings do not extrapolate well past the training length, full attention becomes expensive, and long instruction data is scarce. The authors propose CEPE (Context Expansion with Parallel Encoding), which attaches two trainable parts to a… - LitSearch: A Retrieval Benchmark for Scientific Literature Search (2024 · 39 citations)
Researchers often look for papers that match a precise description, such as a particular method or setting.
This page in Markdown → https://md.research.elamur.ai/free/en/authors/danqi-chen/page/2/