Elamur Research⌕
☰
home / papers / 2026 / NEAT: Neuron-Based Early Exit for Large Reasoning Models

NEAT: Neuron-Based Early Exit for Large Reasoning Models

Paper · 2026 · Kang Liu, YongKang Liu, Xiaocui Yang et al.

Summary

Large Reasoning Models frequently keep producing reasoning after they have already reached a correct solution. This "overthinking" wastes compute and can introduce errors late in the output. Earlier early-exit techniques either sample intermediate answers repeatedly, which adds rollouts and latency, or depend on probes trained on labeled hidden-state data. NEAT offers a training-free option that works at the level of individual neurons.

On a small calibration set, it scores FFN neurons by how much each one raises the log probability of the termination token (</think>). It then keeps the neurons whose activation is concentrated late in the reasoning trace, measured by high relative center of mass and low entropy. During decoding, NEAT compares current activations of these neurons with a reference pattern. A strong match ends reasoning, and a weaker match only suppresses reflection tokens such as "Wait."

The authors tested NEAT on four reasoning benchmarks and six models of varying size and architecture. It reduced generated tokens by 22–28% per model with no loss in accuracy, and without extra rollouts or supervision.

Key points

  • Neuron selection has two steps: log-probability-increase attribution toward </think>, then temporal filtering by Center of Mass and Activation Entropy over a calibration set.
  • NEAT uses two interventions, chosen by the strength of the signal: early exit when it is strong, and reflection suppression when it is weaker.
  • Averaged over the four benchmarks, token reduction is 22.0%–28.2% per model. Tested models go up to 14B parameters.
  • On MATH500 with DeepSeek-R1-Distill-Qwen-7B, NEAT kept Vanilla accuracy (92.2) while cutting average tokens from 3743 to 2914. Random neurons dropped accuracy to 77.8, and hidden-state probes reached 82.0, so neither worked as a reliable exit signal.
  • The method requires access to internal activations, so it cannot be used with API-only models. Code: github.com/KpKqwq/Early_Exist_Circuit.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Northeastern UniversityShenyang · China42 works · 1,940 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,426 works

Cite

Cite as Kang Liu, YongKang Liu, Xiaocui Yang, Peidong Wang, Wen Zhang, Shi Feng, Yifei Zhang and Daling Wang. 2026. NEAT: Neuron-Based Early Exit for Large Reasoning Models. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1231

@inproceedings{liu2026neat,
  title = {{NEAT: Neuron-Based Early Exit for Large Reasoning Models}},
  author = {Kang Liu, YongKang Liu, Xiaocui Yang, Peidong Wang, Wen Zhang, Shi Feng, Yifei Zhang and Daling Wang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1231},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1231}
}

References

36 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. AI-MO. 2024. AMC 2023. https://huggingface.co/datasets/AI-MO/aimo-validation-amc. Accessed: 2025-07-26. ↗
  2. Yuheng Chen, Pengfei Cao, Yubo Chen, Kang Liu, and Jun Zhao. 2024. Journey to the center of the knowledge neurons: Discoveries of language-independent knowledge neurons and degenerate knowledge neurons. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 17817–17825. find
  3. Roy Eisenstadt, Itamar Zimerman, and Lior Wolf. 2025. Overclocking llm reasoning: Monitoring and controlling thinking path lengths in llms. arXiv preprint arXiv:2506.07240. — [arXiv:2506.07240](https://arxiv.org/abs/2506.07240) find
  4. Yichao Fu, Junda Chen, Siqi Zhu, Zheyu Fu, Zhongdongming Dai, Aurick Qiao, and Hao Zhang. 2024. find
  5. Efficiently serving llm reasoning programs with certaindex. arXiv e-prints, pages arXiv–2412. find
  6. Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2021. Transformer feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5484–5495. find
  7. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  8. Daniil Gurgurov, Katharina Trinley, Yusser Al Ghussin, Tanja Baeumel, Josef van Genabith, and Simon Ostermann. 2025. Language arithmetics: Towards systematic language neuron identification and manipulation. arXiv preprint arXiv:2507.22608. find
  9. Tingxu Han, Zhenting Wang, Chunrong Fang, Shiyu Zhao, Shiqing Ma, and Zhenyu Chen. 2025. Token-budget-aware llm reasoning. In Findings of the Association for Computational Linguistics: ACL 2025, pages 24842–24855.
  10. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874.
  11. Jiameng Huang, Baijiong Lin, Guhao Feng, Jierun Chen, Di He, and Lu Hou. 2025. Efficient reasoning for large reasoning language models via certainty-guided reflection suppression. arXiv preprint arXiv:2508.05337. — [arXiv:2508.05337](https://arxiv.org/abs/2508.05337) find
  12. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th symposium on operating systems principles, pages 611–626.
  13. Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let’s verify step by step. In The Twelfth International Conference on Learning Representations.
  14. Xin Liu and Lu Wang. 2025. Answer convergence as a signal for early stopping in reasoning. arXiv preprint arXiv:2506.02536.
  15. Chenwei Lou, Zewei Sun, Xinnian Liang, Meng Qu, Wei Shen, Wenqi Wang, Yuntao Li, Qingping Yang, and Shuangzhi Wu. 2025. Adacot: Pareto-optimal adaptive chain-of-thought triggering via reinforcement learning. arXiv preprint arXiv:2505.11896. — [arXiv:2505.11896](https://arxiv.org/abs/2505.11896) find
  16. Wenjie Ma, Jingxuan He, Charlie Snell, Tyler Griggs, Sewon Min, and Matei Zaharia. 2025. Reasoning models can be effective without thinking. arXiv preprint arXiv:2504.09858. find
  17. Yuyang Wu, Yifei Wang, Ziyu Ye, Tianqi Du, Stefanie Jegelka, and Yisen Wang. 2025. When more is less: Understanding chain-of-thought length in llms. arXiv preprint arXiv:2502.07266. find
  18. MAA Committees. 2025. AIME Problems and Solutions. https://artofproblemsolving.com/wiki/index.php/AIME_Problems_and_ Solutions. Accessed: 2025-07-26. ↗
  19. Chenxu Yang, Qingyi Si, Yongjie Duan, Zheliang Zhu, Chenyu Zhu, Qiaowei Li, Minghui Chen, Zheng Lin, and Weiping Wang. 2025. Dynamic early exit in reasoning models. arXiv preprint arXiv:2504.15895. find
  20. OpenAI. 2024. Introducing openai o1. https://openai.com/o1/. Accessed: December 5, 2024. ↗
  21. Zeping Yu and Sophia Ananiadou. 2024a. Interpreting arithmetic mechanism in large language models through comparative neuron analysis. arXiv preprint arXiv:2409.14144. find
  22. Daking Rai and Ziyu Yao. 2024. An investigation of neuron activation as a unified lens to explain chain-of-thought eliciting arithmetic reasoning of llms. arXiv preprint arXiv:2406.12288. find
  23. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. 2024. Gpqa: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling.
  24. Dan Shi, Renren Jin, Tianhao Shen, Weilong Dong, Xinwei Wu, and Deyi Xiong. 2024. Ircan: Mitigating knowledge conflicts in llm generation via identifying and reweighting context-aware neurons. Advances in Neural Information Processing Systems, 37:4997– 5024. find
  25. Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2024. Scaling llm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314.
  26. Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Na Zou, and 1 others. 2025. Stop overthinking: A survey on efficient reasoning for large language models. arXiv preprint arXiv:2503.16419.
  27. Tianyi Tang, Wenyang Luo, Haoyang Huang, Dongdong Zhang, Xiaolei Wang, Wayne Xin Zhao, Furu Wei, and Ji-Rong Wen. 2024. Language-specific neurons: The key to multilingual capabilities in large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics…
  28. Zeping Yu and Sophia Ananiadou. 2024b. Neuron-level knowledge attribution in large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 3267–3280. find
  29. Zeping Yu and Sophia Ananiadou. 2025. Locate-then-merge: Neuron-level parameter fusion for mitigating catastrophic forgetting in multimodal llms. arXiv preprint arXiv:2505.16703. find
  30. Zeping Yu, Yonatan Belinkov, and Sophia Ananiadou. 2025. Back attention: Understanding and enhancing multi-hop reasoning in large language models. arXiv preprint arXiv:2502.10835.
  31. Anqi Zhang, Yulin Chen, Jane Pan, Chen Zhao, Aurojit Panda, Jinyang Li, and He He. 2025a. Reasoning models know when they’re right: Probing hidden states for self-verification. arXiv preprint arXiv:2504.05419. find
  32. Jiajie Zhang, Nianyi Lin, Lei Hou, Ling Feng, and Juanzi Li. 2025b. Adaptthink: Reasoning models can learn when to think. arXiv preprint arXiv:2505.13417.
  33. Haoran Zhao, Yuchen Yan, Yongliang Shen, Haolei Xu, Wenqi Zhang, Kaitao Song, Jian Shao, Weiming Lu, Jun Xiao, and Yueting Zhuang. 2025a. Let llms break free from overthinking via self-braking tuning. arXiv preprint arXiv:2505.14604. — [arXiv:2505.14604](https://arxiv.org/abs/2505.14604) find
  34. Zekai Zhao, Qi Liu, Kun Zhou, Zihan Liu, Yifei Shao, Zhiting Hu, and Biwei Huang. 2025b. Activation control for efficiently eliciting long chain-of-thought ability of language models. arXiv preprint arXiv:2505.17697. — [arXiv:2505.17697](https://arxiv.org/abs/2505.17697) find
  35. Yiru Tang, Kun Zhou, Yingqian Min, Wayne Xin Zhao, Jing Sha, Zhichao Sheng, and Shijin Wang. 2025. Enhancing chain-of-thought reasoning via neuron activation differential analysis. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 16162–16170. find
  36. Chenlong Wang, Yuanning Feng, Dongping Chen, Zhaoyang Chu, Ranjay Krishna, and Tianyi Zhou. 2025. Wait, we don’t need to" wait"! removing thinking tokens improves reasoning efficiency. arXiv preprint arXiv:2506.08343. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liu-2026-neat-neuron-based-early-exit/