🧑‍🔬 Sayash Kapoor

🏛️ Affiliations

🎓Princeton UniversityPrinceton · United States2021–2026 · 37 paperscurrent🎓Indian Institute of Technology KanpurKanpur · India2018–2019 · 5 papers🎓École Polytechnique Fédérale de LausanneLausanne · Switzerland2018 · 2 papers

From the bylines of their papers (OpenAlex, 2026-10-14), ORCID and Wikidata; institutions on papers are matched by machine.

🧭 Fields, topics and concepts

Of its works in our library, with the number of works on each.

📅 Works by year

2020s

📄 Papers

The most cited first (OpenAlex and our library).

1. AI Agents That Matter (2024 · 55 citations)
   Benchmarks steer the development of LLM-based agents. The Princeton authors argue that current evaluation habits reward topping leaderboards rather than building systems that work in practice. They identify four problems:
2. CORE-Bench: Fostering the Credibility of Published Research Through a Computational Reproducibility Agent Benchmark (2024 · 39 citations)
   Many published studies cannot be rerun to obtain their reported numbers, even when the authors share code and data, and checking this by hand takes expert effort.
3. The Leaderboard Illusion (2025 · 22 citations)
   Chatbot Arena has become the main public leaderboard for ranking LLMs through crowdsourced pairwise votes.
4. The Limits of Inference Scaling Through Resampling (2024 · 21 citations)
   Resampling is a popular way to spend more compute at inference time: a model keeps generating candidates until one passes a verifier, such as unit tests.
5. REFORMS: Reporting Standards for Machine Learning Based Science (2023 · 19 citations)
   Machine learning is spreading quickly across scientific disciplines, and so are problems with validity, reproducibility, and generalizability.
6. Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation (2025 · 11 citations)
   Evaluating AI agents is slow, inconsistent and error-prone. Benchmarks can take weeks to run serially, costs are rarely reported, scaffolds are seldom compared, and agents may game tasks or act unsafely without detection. The authors built the Holistic Agent Leaderboard (HAL), an open-source harness. It accepts any…

All works

22 works of theirs in our library, the most cited first; 6 with a page here, the others link their original until we summarise them.

1. The Foundation Model Transparency Index ↗arXiv:2310.12941 · no summary yetRishi Bommasani et al.2023 · 76 citations
2. AI Agents That MatterarXiv:2407.01502Sayash Kapoor, Benedikt Stroebl, Zachary S. Siegel, et al.2024 · 55 citations
3. Foundation Model Transparency Reports ↗Proceedings of the AAAI/ACM Conference on AI Ethics and Society · no summary yetRishi Bommasani et al.2024 · 53 citations
4. International AI Safety Report ↗arXiv (Cornell University) · no summary yetYoshua Bengio et al.2025 · 41 citations
5. Promises and pitfalls of artificial intelligence for legal applications ↗arXiv:2402.01656 · no summary yetSayash Kapoor, Peter Henderson, Arvind Narayanan2024 · 40 citations
6. CORE-Bench: Fostering the Credibility of Published Research Through a Computational Reproducibility Agent BenchmarkarXiv:2409.11363Zachary S. Siegel, Sayash Kapoor, Nitya Nadgir, et al.2024 · 39 citations
7. Considerations for governing open foundation models ↗Science · no summary yetRishi Bommasani et al.2024 · 36 citations
8. On the Societal Impact of Open Foundation Models ↗arXiv (Cornell University) · no summary yetSayash Kapoor et al.2024 · 30 citations
9. The Leaderboard IllusionConference on Neural Information Processing Systems (NeurIPS)Shivalika Singh, Yiyang Nan, Alex Wang, et al.2025 · 22 citations
10. A Safe Harbor for AI Evaluation and Red Teaming ↗arXiv (Cornell University) · no summary yetShayne Longpre et al.2024 · 22 citations
11. The Limits of Inference Scaling Through ResamplingarXiv:2411.17501Benedikt Stroebl, Sayash Kapoor, Arvind Narayanan2024 · 21 citations
12. REFORMS: Reporting Standards for Machine Learning Based SciencearXiv:2308.07832Sayash Kapoor, Emily Cantrell, Kenny Peng, et al.2023 · 19 citations
13. The Foundation Model Transparency Index v1.1: May 2024 ↗arXiv (Cornell University) · no summary yetRishi Bommasani et al.2024 · 14 citations
14. Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent EvaluationarXiv:2510.11977Sayash Kapoor, Benedikt Stroebl, Peter Kirgis, et al.2025 · 11 citations
15. The Responsible Foundation Model Development Cheatsheet: A Review of Tools & Resources ↗arXiv (Cornell University) · no summary yetShayne Longpre et al.2024 · 11 citations
16. The 2025 Foundation Model Transparency Index ↗SuperIntelligence - Robotics - Safety & Alignment · no summary yetAlexander Wan et al.2026 · 9 citations
17. Establishing Best Practices for Building Rigorous Agentic Benchmarks ↗arXiv:2507.02825 · no summary yetYuxuan Zhu et al.2025 · 7 citations
18. Position: On the Societal Impact of Open Foundation Models ↗ANU Open Research (Australian National University) · no summary yetSayash Kapoor et al.2024 · 6 citations
19. Establishing Best Practices in Building Rigorous Agentic Benchmarks ↗neural information processing systems · no summary yetYuxuan Zhu et al.2025 · 4 citations
20. In-House Evaluation Is Not Enough: Towards Robust Third-Party Flaw Disclosure for General-Purpose AI ↗arXiv (Cornell University) · no summary yetShayne Longpre et al.2025 · 2 citations
21. Log analysis is necessary for credible evaluation of AI agents ↗arXiv (Cornell University) · no summary yetPeter Kirgis et al.2026 · 1 citation
22. FLARE-AI: Flaw Reporting for AI ↗arXiv (Cornell University) · no summary yetShayne Longpre et al.2026

🤝 Co-authors here

12 of 19 co-authors with a page, the most papers together first.

• Benedikt Stroebl ×4
• Arvind Narayanan ×3
• Nitya Nadgir ×3
• Zachary S. Siegel ×3
• Ahmet Üstün
• Alex Wang
• Beyza Ermis
• Boyi Wei
• Daniel D'Souza
• Jake M. Hofman
• Marzieh Fadaee
• Noah A. Smith

Venues

Where their works here were published, the most works first, with the editions of those that have a page.

• Conference on Neural Information Processing Systems (NeurIPS): NeurIPS 2025
• ANU Open Research (Australian National University)
• Proceedings of the AAAI/ACM Conference on AI Ethics and Society
• Science
• SuperIntelligence - Robotics - Safety & Alignment
• neural information processing systems

🔗 Cited by

Papers of our library cite their works 291 times (of the 149,439 papers whose bibliographies we read); 66 citing papers with a page here →.

Updated 3 November 2026

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/authors/sayash-kapoor/