Elamur Research⌕
☰
home / papers / 2026 / Teaching People LLM's Errors and Getting it Right

Teaching People LLM's Errors and Getting it Right

Paper · 2026 · Nathan Stringham, Fateme Hashemi Chaleshtori, Xinyuan Yan et al.

Summary

Users often trust LLMs on tasks the models handle poorly. An earlier approach, the AI-integration teaching pipeline of Mozannar et al. (2023), tried to fix this. It finds regions where a model tends to fail, describes them in natural language and teaches users to avoid the model there. However, it gave no gains for an LLM on MMLU. This paper checks each pipeline stage to find out why.

First, the authors define two measures for groups of instances. Coverage is the share of all model errors a group accounts for. Error ratio compares wrong to correct predictions within the group. Grouping MMLU and MathCAMPS items by human-annotated meta-labels revealed several groups that are both large and error-heavy, so teachable failure patterns do exist.

Next, they test whether automatic methods can recover these known patterns. A prompting approach comes fairly close, but the earlier embedding-based method does not.

Finally, a user study shows teaching helps when success is measured by whether people can predict LLM errors, not by assisted task accuracy. The pipeline is therefore promising but needs better pattern discovery and better evaluation.

Key points

  • The pipeline is formalised in four stages: partitioning data by model correctness, generating failure-pattern descriptions, teaching guidelines to users, and evaluating with a user study.
  • Two group-level criteria, coverage and error ratio, are introduced. The thresholds for "high enough" are left to depend on the application or on stakeholders.
  • Meta-labels used: MMLU subjects and MathCAMPS educational standards. Both datasets contained sizable failure-pattern groups for the LLMs studied.
  • The authors evaluate two prompting-based and two embedding-based methods using a recall-style metric. The embedding method of Mozannar et al. (2023) underperforms, which may explain the earlier negative result.
  • The user study on MMLU had 20 participants and cost $390. It found measurable teaching benefits under the proposed error-predictability metric, unlike under human–AI team accuracy.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of UtahSalt Lake City · United States38 works · 5,925 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksSocial Sciences · 1,772 worksSafety Research · 543 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Nathan Stringham, Fateme Hashemi Chaleshtori, Xinyuan Yan, Zhichao Xu, Bei Wang and Ana Marasovic. 2026. Teaching People LLM's Errors and Getting it Right. Proceedings of the 6th Workshop on Trustworthy NLP (TrustNLP 2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.trustnlp-main.11

@inproceedings{stringham2026teaching,
  title = {{Teaching People LLM's Errors and Getting it Right}},
  author = {Nathan Stringham, Fateme Hashemi Chaleshtori, Xinyuan Yan, Zhichao Xu, Bei Wang and Ana Marasovic},
  booktitle = {Proceedings of the 6th Workshop on Trustworthy NLP (TrustNLP 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.trustnlp-main.11},
  url = {https://doi.org/10.18653/v1/2026.trustnlp-main.11}
}

References

42 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. John W Ayers, Adam Poliak, Mark Dredze, Eric C Leas, Zechariah Zhu, Jessica B Kelley, Dennis J Faix, Aaron M Goodman, Christopher A Longhurst, Michael Hogarth, and 1 others. 2023. Comparing physician and artificial intelligence chatbot responses to patient questions posted to a public social media f… find
  2. Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, and Daniel Weld. 2021. Does the whole exceed its parts? the effect of ai explanations on complementary team performance. In Proceedings of the 2021 CHI Conference on Human Factors in Computing System… find
  3. Ángel Alexander Cabrera, Adam Perer, and Jason I Hong. 2023. Improving human-AI collaboration with descriptions of AI behavior. Proceedings of the ACM on Human-Computer Interaction, 7(CSCW1):1– 21. find
  4. Carrie J. Cai, Samantha Winter, David Steiner, Lauren Wilcox, and Michael Terry. 2019. "hello ai": Uncovering the onboarding needs of medical practitioners for human-ai collaborative decision-making. Proc. ACM Hum.-Comput. Interact., 3(CSCW). find
  5. Jacob Cohen. 1968. Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. Psychological Bulletin, 70(4):213–220. find
  6. Jacob Cohen. 1988. Statistical Power Analysis for the Behavioral Sciences, rev. edition. Lawrence Erlbaum Associates, Hillsdale, NJ. find
  7. Greg d’Eon, Jason d’Eon, James R Wright, and Kevin Leyton-Brown. 2022. The spotlight: A general method for discovering systematic errors in deep learning models. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 1962–1981. find
  8. Sabri Eyuboglu, Maya Varma, Khaled Saab, Jean-Benoit Delbrouck, Christopher Lee-Messer, Jared Dunnmon, James Zou, and Christopher Ré. 2022. Domino: Discovering systematic errors with cross-modal embeddings. arXiv preprint arXiv:2203.14960. — [arXiv:2203.14960](https://arxiv.org/abs/2203.14960) find
  9. Ben Green and Yiling Chen. 2019. The principles and limits of algorithm-in-the-loop decision making. Proc. ACM Hum. Comput. Interact., 3(CSCW):50:1– 50:24. find
  10. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR).
  11. Sunnie S. Y. Kim, Q. Vera Liao, Mihaela Vorvoreanu, Stephanie Ballard, and Jennifer Wortman Vaughan. 2024. "i’m not sure, but...": Examining the impact of large language models’ uncertainty expression on user reliance and trust. In Proceedings of the 2024 ACM Conference on Fairness, Accountability,… find
  12. Sunnie S. Y. Kim, Jennifer Wortman Vaughan, Q. Vera Liao, Tania Lombrozo, and Olga Russakovsky. 2025. Fostering appropriate reliance on large language models: The role of explanations, sources, and inconsistencies. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, CHI… find
  13. Vivian Lai, Samuel Carton, Rajat Bhatnagar, Q Vera Liao, Yunfeng Zhang, and Chenhao Tan. 2022. Human-AI collaboration via conditional delegation: A case study of content moderation. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, pages 1–18. find
  14. Vivian Lai, Han Liu, and Chenhao Tan. 2020. “why is’ chicago’deceptive?” towards building model-driven tutorials for humans. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pages 1–13. find
  15. Vivian Lai and Chenhao Tan. 2019. On human predictions with explanations and predictions of machine learning models: A case study on deception detection. In Proceedings of the Conference on Fairness, Accountability, and Transparency, FAT* 2019, Atlanta, GA, USA, January 29-31, 2019, pages 29–38. ACM… find
  16. Pierre Le Jeune, Jiaen Liu, Luca Rossi, and Matteo Dora. 2025. RealHarm: A collection of real-world language model application failures. In Proceedings of the The First Workshop on LLM Security (LLM-SEC), pages 87–100, Vienna, Austria. Association for Computational Linguistics. find
  17. Shubhra Mishra, Gabriel Poesia, Belinda Mo, and Noah Goodman. 2024. Mathcamps: Fine-grained synthesis of mathematical problems from human curricula. In The 4th Workshop on Mathematical Reasoning and AI at NeurIPS’24. find
  18. Hussein Mozannar, Jimin Lee, Dennis Wei, Prasanna Sattigeri, Subhro Das, and David Sontag. 2023. Effective human-ai teams via learned natural language rules and onboarding. In Advances in Neural Information Processing Systems, volume 36, pages 30466– 30498. Curran Associates, Inc. find
  19. Hussein Mozannar, Arvind Satyanarayan, and David Sontag. 2022. Teaching humans when to defer to a classifier via exemplars. In Proceedings of the aaai conference on artificial intelligence, volume 36, pages 5323–5331. find
  20. OpenAI. 2024. Gpt-4o technical report. Accessed July 2025. find
  21. Samir Passi and Mihaela Vorvoreanu. 2022. Overreliance on ai: Literature review. Technical Report MSR-TR-2022-12, Microsoft. find
  22. Emilie Purvine, Davis Brown, Brett Jefferson, Cliff Joslyn, Brenda Praggastis, Archit Rathore, Madelyn Shapiro, Bei Wang, and Youjia Zhou. 2023. Experimental observations of the topology of convolutional neural network activations. Proceedings of the 37th AAAI Conference on Artificial Intelligence (… find
  23. Nazneen Rajani, Weixin Liang, Lingjiao Chen, Meg Mitchell, and James Zou. 2022. Seal: Interactive tool for systematic error analysis and labeling. arXiv preprint arXiv:2210.05839. find
  24. Archit Rathore, Nithin Chalapathi, Sourabh Palande, and Bei Wang. 2021. TopoAct: Visually exploring the shape of activations in deep learning. Computer Graphics Forum (CGF), 40(1):382–397. find
  25. Archit Rathore, Yichu Zhou, Vivek Srikumar, and Bei Wang. 2023. TopoBERT: Exploring the topology of fine-tuned word representations. Information Visualization, 22(3):186–208. find
  26. Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster r-cnn: Towards real-time object detection with region proposal networks. In Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc.
  27. Samuel Sanford Shapiro and Martin B Wilk. 1965. An analysis of variance test for normality (complete samples). Biometrika, 52(3/4):591–611. find
  28. Chenglei Si, Navita Goyal, Tongshuang Wu, Chen Zhao, Shi Feng, Hal Daumé Iii, and Jordan Boyd-Graber. 2024. Large language models help humans verify truthfulness – except when they are convincingly wrong. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Comp… find
  29. Marco Siino, Mariana Falco, Daniele Croce, and Paolo Rosso. 2025. Exploring llms applications in law: A literature review on current legal nlp approaches. IEEE Access. find
  30. Gurjeet Singh, Facundo Mémoli, and Gunnar E Carlsson. 2007. Topological methods for the analysis of high dimensional data sets and 3D object recognition. Eurographics Symposium on Point-Based Graphics, pages 91–100. find
  31. Michelle Vaccaro, Abdullah Almaatouq, and Thomas Malone. 2024. When combinations of humans and ai are useful: A systematic review and meta-analysis. Nature Human Behaviour, pages 1–11. find
  32. Helena Vasconcelos, Matthew Jörke, Madeleine Grunde-McLaughlin, Tobias Gerstenberg, Michael S. Bernstein, and Ranjay Krishna. 2023. Explanations can reduce overreliance on AI systems during decision-making. Proc. ACM Hum. Comput. Interact., 7(CSCW1):1–38. find
  33. Xinyuan Yan, Rita Sevastjanova, Sinie van der Ben, Mennatallah El-Assady, and Bei Wang. 2025a. Explainable Mapper: Charting LLM embedding spaces using perturbation-based explanation and verification agents. arXiv preprint arXiv:2507.18607. — [arXiv:2507.18607](https://arxiv.org/abs/2507.18607) find
  34. Xinyuan Yan, Xiwei Xuan, Jorge Piazentin Ono, Jiajing Guo, Vikram Mohanty, Shekar Arvind Kumar, Liang Gou, Bei Wang, and Liu Ren. 2025b. VISLIX: An XAI framework for validating vision models with slice discovery and analysis. Computer Graphics Forum, 44(3):e70125. find
  35. Yunfeng Zhang, Q. Vera Liao, and Rachel K. E. Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in ai-assisted decision making. CoRR, abs/2001.02114. — [arXiv:2001.02114](https://arxiv.org/abs/2001.02114) find
  36. Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. 2024. Wildchat: 1m chatGPT interaction logs in the wild. In The Twelfth International Conference on Learning Representations.
  37. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. In Proceedings of the 37th International Conference on Neu…
  38. Ruiqi Zhong, Charlie Snell, Dan Klein, and Jacob Steinhardt. 2022. Describing differences between text distributions with natural language. In International Conference on Machine Learning, pages 27099–27116. PMLR. find
  39. Ruiqi Zhong, Peter Zhang, Steve Li, Jinwoo Ahn, Dan Klein, and Jacob Steinhardt. 2023. Goal driven discovery of distributional differences via language descriptions. Advances in Neural Information Processing Systems, 36:40204–40237.
  40. Youjia Zhou, Yi Zhou, Jie Ding, and Bei Wang. 2023. Visualizing and analyzing the topology of neuron activations in deep adversarial training. Topology, Algebra, and Geometry in Machine Learning (TAGML) Workshop at ICML. find
  41. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22, R…
  42. Tongshuang Wu, Marco Tulio Ribeiro, Jeffrey Heer, and Daniel Weld. 2019. Errudite: Scalable, reproducible, and testable error analysis. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 747–763, Florence, Italy. Association for Computational Linguistic… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/stringham-2026-teaching-people-llm-s-errors/