🧑‍🔬 Sihao Chen

Sihao Chen works on evaluating natural language processing systems. The papers here ask whether models that score well on standard benchmarks actually behave correctly, and how reliable their answers and cited sources are when the questions come from real professional practice. The methods centre on building evaluation resources by hand: careful human edits to existing test items, and questions and judgements gathered from domain experts. This places the work within Evaluation & benchmarks.

The most cited paper here is Evaluating Models' Local Decision Boundaries via Contrast Sets (2020). It argues that when test sets mirror the training distribution, models can succeed through shallow shortcuts. As a remedy, it proposes that dataset authors write small, meaningful edits to test instances, edits that usually flip the correct label. The resulting groups of variants show whether a model's decisions hold up close to real examples, and a consistency measure complements plain accuracy. The second paper, ExpertQA (2024), turns to large language models used by specialists. Trained practitioners across many fields, including medicine and law, wrote hard questions from their own work. They then judged the factuality and attribution of answers from generative, retrieve-and-read and post-hoc attribution systems, claim by claim.

Taken together, the papers move from probing classifier robustness toward checking factual grounding and citation quality in long-form answers. Chen's earlier bylines include the University of Pennsylvania and Google. Co-authors on these papers include Dan Roth, Chaitanya Malaviya, Jonathan Berant and Daniel Khashabi.

Sihao Chen has 2 papers in our library, in 2020–2024, most of them on Evaluation & benchmarks; often with Ally Zhang, Ben Bogin, Chaitanya Malaviya.