Elamur Research⌕
☰
home / papers / 2026 / TELL-TALE: Task Efficient LLMs with Task Aware Layer Elimination

TELL-TALE: Task Efficient LLMs with Task Aware Layer Elimination

Paper · 2026 · Omar Naim, Krish Sharma, Niyar R Barman et al.

Summary

Modern LLMs run every input through all of their layers, yet many layers contribute little or even hurt accuracy on a specific downstream task. The authors propose TALE (Task-Aware Layer Elimination), a greedy, training-free procedure applied at inference time. At each round it tests the removal of every remaining layer, keeps the deletion that gives the best validation accuracy on the target task, and stops once accuracy drops past a tolerance. The motivation comes from projecting intermediate hidden states into the vocabulary space, which showed that earlier layers sometimes answer better than the final one.

TALE was tested on five models across nine reasoning, understanding and commonsense benchmarks, in zero-shot and few-shot settings and under two evaluation protocols. The pruned models match or beat the unpruned baselines while using fewer layers. TALE also outperforms task-agnostic training-free pruning methods on LLaMA 2 and adds gains on top of fine-tuning. This makes it a cheap, post-hoc way to build task-specialized and faster LLMs, for example for multi-agent systems.

Key points

  • Models: LLaMA 3.1 8B, Qwen 2.5 7B, Qwen 2.5 0.5B, Mistral 7B and Lucie 7B. The comparison with prior pruning methods, which rely on perplexity or similarity criteria (e.g. SLEB, BlockPruner), uses LLaMA 2 7B and 13B.
  • Benchmarks: GSM8K-Hard, MATH500, MMLU, BoolQ, Winogrande, CommonsenseQA, BIG-Bench, ARC-Easy and ARC-Challenge.
  • Evaluation: scoring uses both LM-Eval option likelihood and a generation-based "Decoder Eval".
  • Stopping rule and outputs: the search stops at 8% below baseline accuracy. It reports two configurations:
    • BEST, the most accurate configuration;
    • BSBA, the most compressed configuration that still reaches baseline accuracy.
  • Gains by model: Lucie, pretrained on only 3T tokens (versus 15T for LLaMA and 13T for Qwen), gains the most, with double-digit improvements on several tasks. LLaMA gains the least.
  • Analysis: a mutual-information analysis with MINE shows that TALE smooths the drops in MI across layers. Removing every layer where MI decreases, however, badly harms performance.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Centre National de la Recherche ScientifiqueParis · France168 works · 62,369 citations🔬Institut de Recherche en Informatique de ToulouseToulouse · France35 works · 1,872 citations🎓Université de ToulouseToulouse · France16 works · 1,999 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Omar Naim, Krish Sharma, Niyar R Barman and Nicholas Asher. 2026. TELL-TALE: Task Efficient LLMs with Task Aware Layer Elimination. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1136

@inproceedings{naim2026tell,
  title = {{TELL-TALE: Task Efficient LLMs with Task Aware Layer Elimination}},
  author = {Omar Naim, Krish Sharma, Niyar R Barman and Nicholas Asher},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1136},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1136}
}

References

42 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Guillaume Alain and Yoshua Bengio. 2016. Understanding intermediate layers using linear classifier probes. In International Conference on Learning Representations (ICLR) Workshop. find
  2. Alexander A Alemi, Ian Fischer, Joshua V Dillon, and Kevin Murphy. 2016. Deep variational information bottleneck. arXiv preprint arXiv:1612.00410. — [arXiv:1612.00410](https://arxiv.org/abs/1612.00410) find
  3. Yongqi An, Xu Zhao, Tao Yu, Ming Tang, and Jinqiao Wang. 2024. Fluctuation-based adaptive structured pruning for large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 10865–10873. find
  4. Saleh Ashkboos, Maximilian L Croci, Marcelo Gennari do Nascimento, Torsten Hoefler, and James Hensman. 2024. Slicegpt: Compress large language models by deleting rows and columns. arXiv preprint arXiv:2401.15024. find
  5. Hritik Bansal, Karthik Gopalakrishnan, Saket Dingliwal, Sravan Bodapati, Katrin Kirchhoff, and Dan Roth. 2023. Rethinking the role of scale for in-context learning: An interpretability-based case study at 66 billion scale. In Proceedings of the 61st Annual Meeting of the Association for Computationa… find
  6. Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and Devon Hjelm. 2018. Mutual information neural estimation. In International conference on machine learning, pages 531–540. PMLR. find
  7. Yonatan Belinkov. 2022. Probing classifiers: Promises, shortcomings, and advances. Computational Linguistics, 48(1):207–219.
  8. Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019. BoolQ: Exploring the surprising difficulty of natural yes/no questions. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:…
  9. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question answering? try arc, the ai2 reasoning challenge. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP). ArXi…
  10. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems.
  11. Fahim Dalvi, Hassan Sajjad, Nadir Durrani, and Yonatan Belinkov. 2020. Analyzing redundancy in pretrained transformer models. arXiv preprint arXiv:2004.04010. find
  12. Wietse De Vries, Andreas Van Cranenburgh, and Malvina Nissim. 2020. What’s so special about bert’s layers? a closer look at the nlp pipeline in monolingual and multilingual models. arXiv preprint arXiv:2004.06499. find
  13. Robert M. Fano. 1961. Transmission of Information: A Statistical Theory of Communications. M.I.T. Press. find
  14. Elias Frantar and Dan Alistarh. 2023. Sparsegpt: Massive language models can be accurately pruned in one-shot. In International conference on machine learning, pages 10323–10337. PMLR. find
  15. Marylou Gabrié, Andre Manoel, Clément Luneau, Jean Barbier, Nicolas Macris, Florent Krzakala, and Lenka Zdeborová. 2019. Entropy and mutual information in models of deep neural networks. Advances in Neural Information Processing Systems, 32. find
  16. Shuyang Gao, Greg Ver Steeg, and Aram Galstyan. 2015. Efficient estimation of mutual information for strongly dependent variables. arXiv preprint arXiv:1411.2003. — [arXiv:1411.2003](https://arxiv.org/abs/1411.2003) find
  17. Olivier Gouvert, Julie Hunter, Jérôme Louradour, Christophe Cerisara, Evan Dufraisse, Yaya Sy, Laura Rivière, Jean-Pierre Lorré, and 1 others. 2025. The lucie-7b llm and the lucie training dataset: Open resources for multilingual language generation. arXiv preprint arXiv:2503.12294. — [arXiv:2503.12294](https://arxiv.org/abs/2503.12294) find
  18. Yihui He, Xiangyu Zhang, and Jian Sun. 2017. Channel pruning for accelerating very deep neural networks. In Proceedings of the IEEE international conference on computer vision, pages 1389–1397. find
  19. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021a. Measuring massive multi-task language understanding. In International Conference on Learning Representations (ICLR). ArXiv:2009.03300.
  20. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021b. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874.
  21. Lianming Huang, Shangyu Wu, Yufei Cui, Ying Xiong, Haibo Hu, Xue Liu, Tei-Wei Kuo, Nan Guan, and Chun Jason Xue. 2024. Raee: A robust retrieval-augmented early exit framework for efficient inference. Preprint, arXiv:2405.15198. — [arXiv:2405.15198](https://arxiv.org/abs/2405.15198) find
  22. Justin B Kinney and Gurinder S Atwal. 2014. Equitability, mutual information, and the maximal information coefficient. Proceedings of the National Academy of Sciences, 111(9):3354–3359. find
  23. François Lagunas, Ella Charlaix, Victor Sanh, and Alexander M Rush. 2021. Block pruning for faster transformers. arXiv preprint arXiv:2109.04838.
  24. Omar Naim, Jérôme Bolte, and Nicholas Asher. 2025a. Analyzing limits for in-context learning. arXiv preprint arXiv:2502.03503. — [arXiv:2502.03503](https://arxiv.org/abs/2502.03503) find
  25. Omar Naim, Guilhem Fouilhé, and Nicholas Asher. 2025b. Re-examining learning linear functions in context. In German Conference on Artificial Intelligence (Künstliche Intelligenz), pages 104–117. Springer. find
  26. David Peer, Sebastian Stabinger, Stefan Engl, and Antonio Rodríguez-Sánchez. 2022. Greedy-layer pruning: Speeding up transformer models for natural language processing. Pattern Recognition Letters, 157:76–82. find
  27. Hassan Sajjad, Fahim Dalvi, Nadir Durrani, and Preslav Nakov. 2023. On the effect of dropping layers of pre-trained transformer models. Computer Speech & Language, 77:101429. find
  28. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99–106. find
  29. Claude E Shannon. 1948. A mathematical theory of communication. The Bell system technical journal, 27(3):379–423. find
  30. Jiwon Song, Kyungseok Oh, Taesu Kim, Hyungjun Kim, Yulhwa Kim, and Jae-Joon Kim. 2024. Sleb: Streamlining llms through redundancy verification and elimination of transformer blocks. arXiv preprint arXiv:2402.09025. — [arXiv:2402.09025](https://arxiv.org/abs/2402.09025) find
  31. Xinyuan Song, Keyu Wang, PengXiang Li, Lu Yin, and Shiwei Liu. 2025. Demystifying the roles of llm layers in retrieval, knowledge, and reasoning. arXiv preprint arXiv:2510.02091. find
  32. Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, Agnieszka Kluska, Aitor Lewkowycz, Akshat Agarwal, Alethea Power, Alex Ray, Alex Warstadt, Alexander W. Kocurek, Ali Safaya, Ali Tazarv, an… — Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
  33. Mingjie Sun, Zhuang Liu, Anna Bair, and J Zico Kolter. 2023. A simple and effective pruning approach for large language models. arXiv preprint arXiv:2306.11695. find
  34. Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. Commonsenseqa: A question answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  35. Shengkun Tang, Oliver Sieberling, Eldar Kurtic, Zhiqiang Shen, and Dan Alistarh. 2025. Darwinlm: Evolutionary structured pruning of large language models. arXiv preprint arXiv:2502.07780. — [arXiv:2502.07780](https://arxiv.org/abs/2502.07780) find
  36. Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019. Bert rediscovers the classical nlp pipeline. arXiv preprint arXiv:1905.05950.
  37. Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov. 2019. Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned. arXiv preprint arXiv:1905.09418.
  38. Jinyong Wen. 2024. Gaussian mutual information maximization for efficient graph self-supervised learning: Bridging contrastive-based to decorrelation-based. MM ’24, New York, NY, USA. Association for Computing Machinery. find
  39. Maciej Woczyk, Bartosz Wójcik, Klaudia Baazy, Igor Podolak, Jacek Tabor, Marek mieja, and Tomasz Trzciski. 2021. Zero time waste: Recycling predictions in early exit neural networks. Preprint, arXiv:2106.05409. — [arXiv:2106.05409](https://arxiv.org/abs/2106.05409) find
  40. Lu Yin, You Wu, Zhenyu Zhang, Cheng-Yu Hsieh, Yaqing Wang, Yiling Jia, Gen Li, Ajay Jaiswal, Mykola Pechenizkiy, Yi Liang, Michael Bendersky, Zhangyang Wang, and Shiwei Liu. 2024. Outlier weighed layerwise sparsity (owl) a missing secret sauce for pruning llms to high sparsity. In Proceedings of the… find
  41. Yuxin Zhang, Lirui Zhao, Mingbao Lin, Sun Yunyun, Yiwu Yao, Xingjia Han, Jared Tanner, Shiwei Liu, and Rongrong Ji. 2024. Dynamic sparse no training: Training-free fine-tuning for sparse llms. In The Twelfth International Conference on Learning Representations. find
  42. Longguang Zhong, Fanqi Wan, Ruijun Chen, Xiaojun Quan, and Liangzhi Li. 2025. Blockpruner: Fine-grained pruning for large language models. Preprint, arXiv:2406.10594. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/naim-2026-tell-tale-task-efficient-llms/