🧑‍🔬 Nishant Balepur

🏛️ Affiliations

🎓University of Maryland, College ParkCollege Park · United States2023–2026 · 41 paperscurrent🎓New York UniversityNew York · United States2026 · 4 paperscurrent🎓University of Illinois Urbana-ChampaignUrbana · United States2023 · 4 papers🏢Adobe SystemsSan Jose · United States2025 · 2 papers

From the bylines of their papers (OpenAlex, 2026-10-14), ORCID and Wikidata; institutions on papers are matched by machine. 1 more institution on single bylines are not listed.

🧭 Fields, topics and concepts

Of its works in our library, with the number of works on each.

📅 Works by year

2020s

📄 Papers

The most cited first (OpenAlex and our library).

1. The Prompt Report: A Systematic Survey of Prompt Engineering Techniques (2024 · 189 citations)
   Prompt engineering has grown quickly, but its literature uses inconsistent terminology and lacks a shared picture of which techniques exist and how they relate.
2. Artifacts or Abduction: How Do LLMs Answer Multiple-Choice Questions Without the Question? (2024 · 52 citations)
   Multiple-choice question answering (MCQA) is widely used to rank LLMs, but it is unclear whether high scores reflect the skills these benchmarks are meant to test.
3. Which of These Best Describes Multiple Choice Evaluation with LLMs? A) Forced B) Flawed C) Fixable D) All of the Above (2025 · 28 citations)
   This position paper asks whether multiple choice question answering (MCQA), the default way of benchmarking LLMs, deserves that status.
4. Whose Boat Does it Float? Improving Personalization in Preference Tuning via Inferred User Personas (2025 · 12 citations)
   Standard preference datasets record which of two responses annotators favored but not why. LLMs tuned on them therefore learn one majority notion of quality and adapt poorly when a user states specific needs. The authors treat this as an abductive problem. Given a prompt and a response pair, an LLM writes a short…

All works

13 works of theirs in our library, the most cited first; 4 with a page here, the others link their original until we summarise them.

1. The Prompt Report: A Systematic Survey of Prompt Engineering TechniquesarXiv:2406.06608Sander Schulhoff, Michael Ilie, Nishant Balepur, et al.2024 · 189 citations
2. Artifacts or Abduction: How Do LLMs Answer Multiple-Choice Questions Without the Question?Annual Meeting of the Association for Computational Linguistics (ACL)Nishant Balepur, Abhilasha Ravichander, Rachel Rudinger2024 · 52 citations
3. Which of These Best Describes Multiple Choice Evaluation with LLMs? A) Forced B) Flawed C) Fixable D) All of the AboveAnnual Meeting of the Association for Computational Linguistics (ACL)Nishant Balepur, Rachel Rudinger, Jordan Boyd-Graber2025 · 28 citations
4. Expository Text Generation: Imitate, Retrieve, Paraphrase ↗Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · no summary yetNishant Balepur, Jie Huang, Kevin Chen-Chuan Chang2023 · 23 citations
5. AstaBench: Rigorous Benchmarking of AI Agents with a Scientific Research Suite ↗arXiv:2510.21652 · no summary yetJonathan Bragg et al.2025 · 19 citations
6. Whose Boat Does it Float? Improving Personalization in Preference Tuning via Inferred User PersonasAnnual Meeting of the Association for Computational Linguistics (ACL)Nishant Balepur, Vishakh Padmakumar, Fumeng Yang, et al.2025 · 12 citations
7. DynaMiTE: Discovering Explosive Topic Evolutions with User Guidance ↗Findings of the Association for Computational Linguistics: ACL 2023 · no summary yetNishant Balepur et al.2023 · 7 citations
8. BenchMarker: An Education-Inspired Toolkit for Highlighting Flaws in Multiple-Choice Benchmarks ↗Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · no summary yetNishant Balepur et al.2026 · 2 citations
9. Language Models Don't Know What You Want: Evaluating Personalization in Deep Research Needs Real Users ↗Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · no summary yetNishant Balepur et al.2026 · 2 citations
10. DRACULA: Hunting for the Actions Users Want Deep Research Agents to Execute ↗arXiv (Cornell University) · no summary yetNishant Balepur et al.2026 · 1 citation
11. Check The Scoreboard: An Analysis of Scoring Schemes on Multiple-Choice Evaluation ↗arXiv (Cornell University) · no summary yetNishant Balepur et al.2026
12. (Im)Paired Programming: Coding Agents Improve Productivity but Harm Understanding ↗arXiv (Cornell University) · no summary yetNishant Balepur et al.2026
13. VibeJam: A User Study Platform for Web Development with Agents ↗arXiv (Cornell University) · no summary yetNishant Balepur et al.2026

🤝 Co-authors here

7 co-authors with a page, the most papers together first.

• Rachel Rudinger ×3
• Jordan Boyd-Graber ×2
• Abhilasha Ravichander
• Chenglei Si
• HyoJung Han
• Shi Feng
• Vishakh Padmakumar

Venues

Where their works here were published, the most works first, with the editions of those that have a page.

• Annual Meeting of the Association for Computational Linguistics (ACL) ×5: ACL 2025 ×2, ACL 2024
• Findings of the Association for Computational Linguistics: ACL 2023
• Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

🔗 Cited by

Papers of our library cite their works 209 times (of the 149,439 papers whose bibliographies we read); 36 citing papers with a page here →.

Updated 30 October 2026

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/authors/nishant-balepur/