MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering (2024 · 140 citations)
MLE-bench addresses the lack of benchmarks that test whether AI agents can carry out machine learning engineering autonomously from start to finish, rather than solving isolated coding problems.
SWE-Lancer: Can Frontier LLMs Earn $1 Million from Real-World Freelance Software Engineering? (2025 · 26 citations)
Most coding benchmarks, including SWE-Bench, test isolated problems that are easy to unit-test. They show little about how models handle realistic full-stack commercial work, or what that work is worth. The authors build SWE-Lancer from 1,488 real freelance jobs on the Expensify repository. These jobs were posted on…

All works

3 works of theirs in our library, the most cited first; 2 with a page here, the others link their original until we summarise them.

MLE-bench: Evaluating Machine Learning Agents on Machine Learning EngineeringarXiv:2410.07095Jun Shern Chan, Neil Chowdhury, Oliver Jaffe, et al.2024 · 140 citations
GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks ↗arXiv:2510.04374 · no summary yetTejal Patwardhan et al.2025 · 33 citations
SWE-Lancer: Can Frontier LLMs Earn $1 Million from Real-World Freelance Software Engineering?arXiv:2502.12115Samuel Miserendino, Michele Wang, Tejal Patwardhan, et al.2025 · 26 citations

Co-authors here

10 co-authors with a page, the most papers together first.

Where their works here were published, the most works first, with the editions of those that have a page.

Papers of our library cite their works 3,919 times (of the 149,439 papers whose bibliographies we read); 1,080 citing papers with a page here →.

Updated 3 November 2026