Elamur Research⌕
☰
home / papers / 2026 / DART: Mitigating Harm Drift in Difference-Aware LLMs via Distill-Audit-Repair Training

DART: Mitigating Harm Drift in Difference-Aware LLMs via Distill-Audit-Repair Training

Paper · 2026 · Ziwen Pan, Zihan Liang, Jad Kabbara et al.

Summary

Safety-tuned LLMs tend to ignore demographic differences even when recognizing them is factually or legally appropriate, such as disease prevalence by ancestry or religious hiring exemptions. The authors frame this as difference-awareness classification. Given a prompt about demographic groups, the model gives a short rationale and decides whether a correct answer requires distinguishing between groups (YES) or treating them the same (NO).

They find that fine-tuning for accuracy causes "harm drift": decisions get better, but the explanations become more harmful. To counter this, they propose DART, which has three stages:

  • Distill: a student learns from rationales that a teacher writes when given the gold label.
  • Audit: cases where rationales got worse relative to the baseline are flagged using a toxicity classifier plus LLM-as-Judge.
  • Repair: the flagged cases are retrained on safer alternatives, weighted by severity.

An optional explanation policy applied at inference time adds further safety. DART raises accuracy substantially and sharply reduces harmful rationales. This suggests that correctness and safety can be pursued together when harm is explicitly detected and fixed.

Key points

  • Baseline behaviour. On 1,624 prompts from eight benchmarks (the Wang et al., 2025 suite), Llama-3-8B-Instruct answers YES 88.6% of the time, although only 50.2% of prompts warrant it. Another 26.8% of its outputs are refusals or hedged answers that cannot be parsed.
  • Main results. DART lifts Llama-3-8B-Instruct accuracy from 39.0% to 68.8%. Accuracy on equal-treatment (NO) cases rises from 11.3% to 72.6%, and harm drift cases fall by 72.6%.
  • Transfer to open-ended queries. On 280 real-world questions from medical, legal, policy and education domains, difference-appropriate answers rise from 39.8% to 77.5%. Refusals drop from 34.3% to 3.0%.
  • Value of gold labels. Conditioning the teacher on gold labels matters: when the teacher is given predicted labels instead, Stage I accuracy drops to .641 (down from .682), and drift detection becomes less effective.
  • Validation and cost. The LLM-as-Judge was checked against human annotations on 200 samples (κ = 0.66 and 0.71). Results were replicated on five student models from four families. Safety gains also hold on BOLD, HolisticBias, RealToxicityPrompts and HateCheck. Full training takes about 4 hours on one A100-80GB, and the code is released under the MIT License.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Massachusetts Institute of TechnologyCambridge · United States211 works · 69,000 citations🎓Emory UniversityAtlanta · United States36 works · 39,779 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksSocial Sciences · 1,772 worksSafety Research · 543 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksBias, fairness & safety · 517 works

Cite

Cite as Ziwen Pan, Zihan Liang, Jad Kabbara and Ali Emami. 2026. DART: Mitigating Harm Drift in Difference-Aware LLMs via Distill-Audit-Repair Training. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.244

@inproceedings{pan2026dart,
  title = {{DART: Mitigating Harm Drift in Difference-Aware LLMs via Distill-Audit-Repair Training}},
  author = {Ziwen Pan, Zihan Liang, Jad Kabbara and Ali Emami},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.244},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.244}
}

References

50 references, 25 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, and 1 others. 2022a. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862. find
  2. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, and 32 others. 20… — Constitutional AI: Harmlessness from AI Feedback
  3. Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith. 2020. RealToxicityPrompts: Evaluating neural toxic degeneration in language models. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3356–3369, Online. Association for Computational Linguisti…
  4. Xiusi Chen, Hongzhi Wen, Sreyashi Nag, Chen Luo, Qingyu Yin, Ruirui Li, Zheng Li, and Wei Wang. 2024. IterAlign: Iterative constitutional alignment of large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human La… find
  5. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. The Llama 3 herd of models. arXiv preprint arXiv:2407.21783.
  6. Zheng Chu, Jingchang Chen, Qianglong Chen, Weijiang Yu, Tao He, Haotian Wang, Weihua Peng, Ming Liu, Bing Qin, and Ting Liu. 2024. Navigate through enigmatic labyrinth a survey of chain of thought reasoning: Advances, frontiers and future. In Proceedings of the 62nd Annual Meeting of the Association… find
  7. Jacob Cohen. 1988. Statistical Power Analysis for the Behavioral Sciences, 2nd edition. Lawrence Erlbaum Associates, Hillsdale, NJ. find
  8. Justin Cui, Wei-Lin Chiang, Ion Stoica, and Cho-Jui Hsieh. 2025. OR-Bench: An over-refusal benchmark for large language models. In International Conference on Learning Representations.
  9. Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar. 2022. ToxiGen: A large-scale machine-generated dataset for adversarial and implicit hate speech detection. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1:…
  10. Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531.
  11. Md Imtiaz Hossain, Haipeng Cai, and Marcos Vaz Salles. 2025. Irepair: An intent-aware approach to repair data-driven errors in large language models. Preprint, arXiv:2501.09858. — [arXiv:2501.09858](https://arxiv.org/abs/2501.09858) find
  12. Josef Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji, Xinbo Xu, Mickel Liu, Yizhou Wang, and Yaodong Yang. 2024. Safe RLHF: Safe reinforcement learning from human feedback. In The Twelfth International Conference on Learning Representations.
  13. Cheng-Yu Hsieh, Chun-Liang Li, Chih-Kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alexander Ratner, Ranjay Krishna, Chen-Yu Lee, and Tomas Pfister. 2023. Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes. In Findings of the Association for Comp…
  14. Susmit Das, Rahul Madhavan, Punyajoy Saha, Samyak Jain, Kyuhong Shim, and Animesh Mukherjee. 2025. Tracealign – tracing the drift: Attributing alignment failures to training-time belief sources in llms. Preprint, arXiv:2502.02467. — [arXiv:2502.02467](https://arxiv.org/abs/2502.02467) find
  15. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations.
  16. DeepSeek-AI. 2024. DeepSeek-V2: A strong, economical, and efficient mixture-of-experts language model. arXiv preprint arXiv:2405.04434. find
  17. Evan Hubinger, Carson Denison, Jesse Mu, Mike Lambert, Meg Tong, Monte MacDiarmid, Tamera Lanham, Daniel M. Ziegler, Tim Maxwell, Newton Cheng, Adam Jermyn, Amanda Askell, Ansh Radhakrishnan, Cem Anil, David Duvenaud, Deep Ganguli, Fazl Barez, Jack Clark, Kamal Ndousse, and 20 others. 2024. Sleeper… find
  18. Jwala Dhamala, Tony Sun, Varun Kumar, Satyapriya Krishna, Yada Pruksachatkun, Kai-Wei Chang, and Rahul Gupta. 2021. Bold: Dataset and metrics for measuring biases in open-ended language generation. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21, pa… find
  19. Isabel O. Gallegos, Ryan A. Rossi, Joe Barber, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K. Ahmed. 2024. Bias and fairness in large language models: A survey. Computational Linguistics, 50(3):1097– 1179.
  20. Maurice Jakesch, Advait Bhat, Daniel Buschek, Lior Zalmanson, and Mor Naaman. 2023. Co-writing with opinionated language models affects users’ views. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1–15. ACM. find
  21. Mahammed Kamruzzaman, Md Shovon, and Gene Kim. 2024. Investigating subtler biases in llms: Ageism, beauty, institutional, and nationality bias in generative models. In Findings of the Association for Computational Linguistics: ACL 2024, pages 8940–8965. instructions with human feedback. In Advances… find
  22. J. Richard Landis and Gary G. Koch. 1977. The measurement of observer agreement for categorical data. Biometrics, 33(1):159–174. find
  23. Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. 2022. Red teaming language models with language models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3419–3448, Abu…
  24. Lewis Z. Lewis and Cassidy White. 2023. Mitigating harms of large language models via knowledge distillation for a virtual museum tour guide. In Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP), pages 13–22. Association for Computational Linguistics. find
  25. Xinjie Li, Zhiqi Bu, Mingyuan Ma, and Kaidi Xu. 2025. From judgment to interference: Early stopping llm harmful outputs via streaming content monitoring. Preprint, arXiv:2501.09832. — [arXiv:2501.09832](https://arxiv.org/abs/2501.09832) find
  26. Bo Liu, Xingchao Liu, Xiaojie Jin, Peter Stone, and Qiang Liu. 2021. Conflict-averse gradient descent for multi-task learning. In Advances in Neural Information Processing Systems, volume 34, pages 18878– 18890. Curran Associates, Inc. find
  27. Varvara Logacheva, Daryna Dementieva, Sergey Ustyantsev, Daniil Moskovskiy, David Dale, Irina Krotova, Nikita Semenov, and Alexander Panchenko. 2022. ParaDetox: Detoxification with parallel data. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Lo…
  28. Kaifeng Lyu, Haoyu Zhao, Xinran Gu, Dingli Yu, Anirudh Goyal, and Sanjeev Arora. 2024. Keeping llms aligned after fine-tuning: The crucial role of prompt templates. In Advances in Neural Information Processing Systems (NeurIPS). find
  29. Lucie Charlotte Magister, Jonathan Mallinson, Jakub Adamek, Eric Malmi, and Aliaksei Severyn. 2023. Teaching small language models to reason. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 1773–1781, Toronto, Canada. Associa…
  30. Arindam Mitra, Luciano Del Corro, Shweti Mahajan, Andres Codas, Clarisse Simoes, Sahaj Agarwal, Xuxi Chen, Anastasia Razdaibiedina, Erik Jones, Kriti Aggarwal, Hamid Palangi, Guoqing Zheng, Corby Rosset, Hamed Khanpour, and Ahmed Awadallah. 2023. Orca 2: Teaching small language models how to reason.…
  31. Jonathan Nöther, Adish Singla, and Goran Radanović. 2025. Text-diffusion red-teaming of large language models: Unveiling harmful behaviors with proximity constraints. Preprint, arXiv:2501.01741. — [arXiv:2501.01741](https://arxiv.org/abs/2501.01741) find
  32. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. Training language models to follow Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. 2024. F…
  33. Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. 2023. Direct preference optimization: your language model is secretly a reward model. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Red Ho…
  34. Paul Röttger, Hannah Rose Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy. 2024. XSTest: A test suite for identifying exaggerated safety behaviours in large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computatio…
  35. Paul Röttger, Bertie Vidgen, Dong Nguyen, Zeerak Waseem, Helen Margetts, and Janet Pierrehumbert. 2021. HateCheck: Functional tests for hate speech detection models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference…
  36. Sarthak Roy, Ashish Harshvardhan, Animesh Mukherjee, and Punyajoy Saha. 2023. Probing LLMs for hate speech detection: strengths and vulnerabilities. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 6116–6128, Singapore. Association for Computational Linguistics. find
  37. Eric Michael Smith, Melissa Hall, Melanie Kambadur, Eleonora Presani, and Adina Williams. 2022. “I’m sorry to hear that”: Finding new biases in language models with a holistic descriptor dataset. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9180–92…
  38. Mark Steyvers, Aakriti Kumar, Timothy Balch, Jordan Boyd-Graber, Tianran Cui, Mirta Galesic, Ralph Hertwig, Yuxin Hu, Maryam Karimi-Mamaghan, Michael D Lee, and 1 others. 2025. What large language models know and what people think they know. Nature Machine Intelligence. find
  39. Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, Nathan Sarrazin, Omar Sanseviero, Alexander M. Rush, and Thomas Wolf. 2023. Zephyr: Direct distillation of lm alignment. Preprint, arXiv… find
  40. Angelina Wang, Vinodkumar Prabhakaran, Su Lin Blodgett, and Margaret Mitchell. 2025. Fairness through difference awareness: Measuring desired group discrimination in LLMs. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association… find
  41. Peifeng Wang, Zhengyang Wang, Zheng Li, Yifan Gao, Bing Yin, and Xiang Ren. 2023a. SCOTT: Self-consistent chain-of-thought distillation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5546–5558, Toronto, Canada. Association f…
  42. Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Mengnan Du. 2024. Explainability for large language models: A survey. ACM Trans. Intell. Syst. Technol., 15(2).
  43. Xuhui Zhou, Maarten Sap, Swabha Swayamdipta, Yejin Choi, and Noah Smith. 2021. Challenges in automated debiasing for toxic language detection. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 3143–3155, Online. Associa…
  44. Anna Zink, Ziad Obermeyer, and Emma Pierson. 2024. Race adjustments in clinical algorithms can help correct for racial disparities in data quality. Proceedings of the National Academy of Sciences, 121(34):e2402267121. find
  45. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023b. Self-instruct: Aligning language models with self-generated instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long… find
  46. Zhichao Wang, Bin Bi, Shiva Kumar Pentyala, Kiran Ramnath, Sougata Chaudhuri, Shubham Mehrotra, Zixu, Zhu, Xiang-Bo Mao, Sitaram Asur, Na, and Cheng. 2024. A comprehensive survey of llm alignment techniques: Rlhf, rlaif, ppo, dpo and more. Preprint, arXiv:2407.16216. — [arXiv:2407.16216](https://arxiv.org/abs/2407.16216) find
  47. Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2023. Jailbroken: how does llm safety training fail? In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Red Hook, NY, USA. Curran Associates Inc.
  48. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems, volume 35, pages 24824–24837. Curran Associates, Inc.
  49. Zirui Wu and Yansong Feng. 2025. Beyond templates: Dynamic adaptation of reasoning demonstrations via feasibility-aware exploration. Preprint, arXiv:2501.09798.
  50. Tinghao Xie, Xiangyu Zhao, Zhouxuan Tan, Yichen Gao, Ruoxi Jia, and Peter Henderson. 2025. SORRY-Bench: Systematically evaluating large language find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/pan-2026-dart-mitigating-harm-drift-difference/