AI Agents That Matter (2024 · 55 citations)
Benchmarks steer the development of LLM-based agents. The Princeton authors argue that current evaluation habits reward topping leaderboards rather than building systems that work in practice. They identify four problems:
CORE-Bench: Fostering the Credibility of Published Research Through a Computational Reproducibility Agent Benchmark (2024 · 39 citations)
Many published studies cannot be rerun to obtain their reported numbers, even when the authors share code and data, and checking this by hand takes expert effort.
Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation (2025 · 11 citations)
Evaluating AI agents is slow, inconsistent and error-prone. Benchmarks can take weeks to run serially, costs are rarely reported, scaffolds are seldom compared, and agents may game tasks or act unsafely without detection. The authors built the Holistic Agent Leaderboard (HAL), an open-source harness. It accepts any…

All works

4 works of theirs in our library, the most cited first; 3 with a page here, the others link their original until we summarise them.

Co-authors here

8 co-authors with a page, the most papers together first.

Papers of our library cite their works 94 times (of the 149,439 papers whose bibliographies we read); 32 citing papers with a page here →.

Updated 3 November 2026