Elamur Research⌕
☰
home / papers / 2026 / PibE-MPP: A Play-it-by-Ear Masking Performance Plug-in for LLMs

PibE-MPP: A Play-it-by-Ear Masking Performance Plug-in for LLMs

Paper · 2026 · Mengwei Wang, Simin Niu, Xun Liang et al.

Summary

Randomly masking parts of an LLM at inference time can sometimes improve task accuracy. It needs no training, no architectural changes, and leaves the model untouched for other tasks. The catch is that the improvement is unreliable: depending on the random draw, it may flip into a loss. The authors present PibE-MPP, a training-free plug-in that keeps these practical benefits while making the gains predictable. They express the goals as seven constraints (C1–C7). They then derive the masking targets from a first-order Markov view of how the last token's hidden state is built, which points to the attention heads of the final layer. To decide which heads to mask for a given task and model, they first map head-level fusion features onto three axes and use a mean-based rule to build a small, interpretable set of candidate combinations. They then pick the most compact candidate through axis-variance minimization. Tested on six Qwen and LLaMA models and 24 datasets, PibE-MPP gives non-negative gains more often and more consistently than random masking. This makes it a cheap add-on that complements fine-tuning.

Key points

  • Masking targets. The method masks only last-layer attention heads. The authors justify this with token-wise decomposition (Oh and Schuler, 2023) and show it is feasible through random-masking experiments.
  • Two-stage strategy. Stage one builds candidates via a three-axis mapping, using 3D t-SNE with default settings. Stage two selects one combination with axis-variance minimization (AVarMin), which is less sensitive to outlier heads.
  • Evaluation and stability. The method is evaluated on models including LLaMA-3.2-3B, LLaMA-3.2-1B, Qwen2.5-1.5B and DeepSeek-R1-Distill-Qwen-1.5B. Stability is measured with a "gain rate," computed per dataset and per model. Two random-masking runs (RMPP-1/2) give scattered, seed-dependent results, while PibE-MPP is more concentrated and higher.
  • Domain-specific models. On Qwen2.5-Math-1.5B, the method strengthens math performance on most tasks. On Qwen2.5-Coder-3B, it leaves coding ability intact and helps on some tasks.
  • Ablation. Replacing AVarMin with random selection across four models lowers accuracy and increases variability, which confirms the final module is needed.
  • Limitations and code. The authors note that sensitivity to hyperparameters and the selection rule is not studied. The code is released on GitHub.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Renmin University of ChinaBeijing · China139 works · 17,752 citations

Also on the byline: China Jiliang University · Qufu Normal University · College of Modern Science and Technology, China Jiliang University.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Mengwei Wang, Simin Niu, Xun Liang, Yuefeng Ma, Sensen Zhang, Jiawei Yang, Shichao Song, Hanyu Wang and Huayi Lai. 2026. PibE-MPP: A Play-it-by-Ear Masking Performance Plug-in for LLMs. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.105

@inproceedings{wang2026pibe,
  title = {{PibE-MPP: A Play-it-by-Ear Masking Performance Plug-in for LLMs}},
  author = {Mengwei Wang, Simin Niu, Xun Liang, Yuefeng Ma, Sensen Zhang, Jiawei Yang, Shichao Song, Hanyu Wang and Huayi Lai},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.105},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.105}
}

References

42 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Saleh Ashkboos, Maximilian L. Croci, Marcelo Gennari Do Nascimento, and et al. 2024. Slicegpt: Compress large language models by deleting rows and columns. In International Conference on Learning Representations (ICLR). find
  2. Jacob Austin, Augustus Odena, Maxwell Nye, and et al. Program synthesis with large language models.
  3. Tom B. Brown, Benjamin Mann, Nick Ryder, and et al. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems (NeurIPS).
  4. Mark Chen, Jerry Tworek, Heewoo Jun, and et al. 2021. Evaluating large language models trained on code.
  5. Alexis Conneau, Ruty Rinott, Guillaume Lample, and et al. 2018. Xnli: Evaluating cross-lingual sentence representations. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. find
  6. DeepSeek-AI. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.
  7. Elias Frantar and Dan Alistarh. 2023. Sparsegpt: Massive language models can be accurately pruned in one-shot. find
  8. Leo Gao, Jonathan Tow, Baber Abbasi, and et al. 2024. The language model evaluation harness. find
  9. Tianteng Gu, Bei Liu, Bo Xiao, and et al. 2025. Denoiserotator: Enhance pruning robustness for llms via importance concentration. arXiv preprint arXiv:2505.23049. — [arXiv:2505.23049](https://arxiv.org/abs/2505.23049) find
  10. Yue Guan, Jingwen Leng, Chao Li, and et al. 2020. How far does bert look at: Distance-based clustering and analysis of bert’s attention. In Proceedings of COLING. find
  11. Jialong Guo, Xinghao Chen, Yehui Tang, and et al. 2025. Slimllm: Accurate structured pruning for large language models. find
  12. Jiujun He and Huazhen Lin. 2025. Olica: Efficient structured pruning of large language models without retraining. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 22580–22594. PMLR. find
  13. Dan Hendrycks, Collin Burns, Steven Basart, and et al. 2021. Measuring massive multitask language understanding. In Proceedings of the International Conference on Learning Representations (ICLR).
  14. Bairu Hou, Qibin Chen, Jianyu Wang, and et al. 2025. Instruction-following pruning for large language models. find
  15. Huang Huang, Fei Yu, Jianqing Zhu, and et al. 2024. Acegpt, localizing large language models in arabic. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 8139–8163. Associat…
  16. Yuzhen Huang, Yuzhuo Bai, Zhihao Zhu, and et al. Ceval: A multi-level multi-discipline chinese evaluation suite for foundation models. arXiv preprint arXiv:2305.08322. — C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Models
  17. Binyuan Hui, Jian Yang, Zeyu Cui, and et al. 2024. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186. — Qwen2.5-Coder Technical Report
  18. Nandan Kumar Jha and Brandon Reagen. 2025. Entropy-guided attention for private llms. find
  19. Seil Kang, Jinyeong Kim, Junhyeok Kim, and et al. 2025. See what you are told: Visual attention sink in large multimodal models. In International Conference on Learning Representations (ICLR). find
  20. Ivan Kobyzev, Abbas Ghaddar, Dingtao Hu, and et al. 2025. Integral transformer: Denoising attention, not too much not too little. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. find
  21. Fajri Koto, Haonan Li, Sara Shatnawi, and et al. 2024. Arabicmmlu: Assessing massive multitask language understanding in arabic. In Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024, pages 5622–5640. Association for Computa…
  22. Qi Le, Enmao Diao, Ziyan Wang, and et al. 2025. Probe pruning: Accelerating llms through dynamic pruning via model-probing. find
  23. Jian Li, Xing Wang, Zhaopeng Tu, and et al. 2021. On the diversity of multi-head attention. Neurocomputing, 454:14–24. find
  24. Zechun Liu, Changsheng Zhao, Igor Fedorov, and et al. 2025. Spinquant: Llm quantization with learned rotations. find
  25. Sewon Min, Xinxi Lyu, Ari Holtzman, and et al. 2022. Rethinking the role of demonstrations: What makes in-context learning work? In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP).
  26. Niklas Muennighoff, Thomas Wang, Lintang Sutawika, and et al. 2023. Crosslingual generalization through multitask finetuning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023, pages 15991– 1…
  27. Byung-Doh Oh and William Schuler. 2023. Token-wise decomposition of autoregressive language model hidden states for analyzing model predictions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10105–10117, Toronto, Canada. Ass… find
  28. Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, and et al. 2023. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models.
  29. Trans. Mach. Learn. Res., 2023. find
  30. Xuehao Wang, Liyuan Wang, Binghuai Lin, and et al. 2025a. Headmap: Locating and enhancing knowledge circuits in llms. In International Conference on Learning Representations (ICLR). find
  31. Yuxin Wang, MingHua Ma, Zekun Wang, and et al. 2025b. CFSP: An efficient structured pruning framework for LLMs with coarse-to-fine activation information. In Proceedings of the 31st International Conference on Computational Linguistics, pages 9311–9328. Association for Computational Linguistics. find
  32. Wenhao Wu, Yizhong Wang, Guangxuan Xiao, and et al. 2025. Retrieval head mechanistically explains long-context factuality. In International Conference on Learning Representations (ICLR).
  33. Mengzhou Xia, Tianyu Gao, Zhiyuan Zeng, and et al. 2024. Sheared llama: Accelerating language model pre-training via structured pruning. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net. find
  34. An Yang, Baosong Yang, Binyuan Hui, and et al. 2024a. Qwen2 technical report. find
  35. An Yang, Beichen Zhang, Binyuan Hui, and et al. 2024b. Qwen2.5-math technical report: Toward mathematical expert model via self-improvement.
  36. Mingzhe Yang, Sihao Lin, Li Changlin, and et al. 2025. Let LLM tell what to prune and how much to prune. In International Conference on Machine Learning (ICML). find
  37. Yunzhi Yao, Ningyu Zhang, Zekun Xi, and et al. 2024. Knowledge circuits in pretrained transformers. In Advances in Neural Information Processing Systems (NeurIPS). find
  38. Arda Yüksel, Abdullatif Köksal, Lütfi Kerem Senel, and et al. 2024. Turkishmmlu: Measuring massive multitask language understanding in turkish. In Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12-16, 2024, pages 7035–7055. Association for Comput… find
  39. Qwen Team. 2024. Qwen2.5: A party of foundation models. find
  40. Jue Zhang, Qingwei Lin, Saravan Rajmohan, and et al. 2025. From reasoning to answer: Empirical, attention-based and mechanistic insights into distilled DeepSeek r1 models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. find
  41. Alexey Tikhonov and Max Ryabinin. 2021. It’s all in the heads: Using attention heads as a baseline for cross-lingual transfer in commonsense reasoning. In Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, Online Event, August 1-6, 2021, Findings of ACL, pages 3534–3546. Ass…
  42. Wanjun Zhong, Ruixiang Cui, Yiduo Guo, and et al. 2024. Agieval: A human-centric benchmark for evaluating foundation models. In Findings of the Association for Computational Linguistics: NAACL 2024, Mexico City, Mexico, June 16-21, 2024, pages 2299–2314. Association for Computational Linguistics.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-pibe-mpp-play-it-ear/