Elamur Research⌕
☰
home / papers / 2026 / Attn-GS: Attention-Guided Context Compression for Efficient Personalized LLMs

Attn-GS: Attention-Guided Context Compression for Efficient Personalized LLMs

Paper · 2026 · Shenglai Zeng, Tianqi Zheng, Chuan Tian et al.

Summary

Personalizing LLMs usually means feeding them long user histories and profiles, which raises inference latency and API costs. Common fixes, such as keeping only recent interactions or asking an LLM to summarize the profile, handle the context as one undifferentiated block. They ignore which parts the model actually relies on.

The authors first studied attention distributions on MovieLens-1M (movie recommendation) and LaMP-5 (personalized paper title generation). They found two things:

  • Attention already concentrates on informative signal types.
  • Fine-tuning sharpens the split between useful and irrelevant signals.

Building on this, they propose Attn-GS. A small white-box marking model scores sentences by the last token's attention, aggregated per sentence. Sentences above a threshold are marked. A compression model then uses these marks to write a short, task-focused user context offline.

Across tasks, token budgets, and both inference-only and fine-tuning setups, Attn-GS beats truncation, plain summarization, random or full marking, prompting, CoT, and self-reflection baselines. It comes close to full-context quality while using about 50× fewer tokens. This makes personalization cheaper to deploy.

Key points

  • Datasets and marking models:
    • MovieLens-1M: 1,000 training and 1,000 test users; the task is to pick the true last item among 4 random negatives. Llama-3.1-1B-Instruct served as the mark model.
    • LaMP-5: 750 training and 750 test users. Llama-3.1-8B-Instruct served as the mark model.
  • Token efficiency: With GPT-4o-mini as the generator, Attn-GS reaches 48% accuracy with 100 tokens (1.1% of full context). Truncation, direct summarization, and Prompt-GS need 700–750 tokens, so Attn-GS uses about 7× fewer.
  • LaMP-5 with fine-tuning: Attn-GS scores Rouge-L 0.396–0.405 across token budgets. Baselines score about 0.380–0.390, and full context scores 0.4147.
  • Fine-tuned marker: A fine-tuned marking model gives consistently higher accuracy than a non-fine-tuned one. The non-fine-tuned marker still beats the baselines.
  • Ablations:
    • A marking threshold α of roughly 0.2–0.4 works best.
    • Middle layers of the marking model give the most useful attention signals; early layers perform poorly.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢AmazonSeattle · United States128 works · 41,697 citations🎓Michigan State UniversityEast Lansing · United States30 works · 1,375 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,894 worksEfficient models & compression · 297 works

Cite

Cite as Shenglai Zeng, Tianqi Zheng, Chuan Tian, Dante Everaert, Yau-Shian Wang, Yupin Huang, Michael J. Morais, Rohit Patki, Jinjin Tian, Xinnan Dai, Kai Guo, Monica Xiao Cheng and Hui Liu. 2026. Attn-GS: Attention-Guided Context Compression for Efficient Personalized LLMs. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1663

@inproceedings{zeng2026attn,
  title = {{Attn-GS: Attention-Guided Context Compression for Efficient Personalized LLMs}},
  author = {Shenglai Zeng, Tianqi Zheng, Chuan Tian, Dante Everaert, Yau-Shian Wang, Yupin Huang, Michael J. Morais, Rohit Patki, Jinjin Tian, Xinnan Dai, Kai Guo, Monica Xiao Cheng and Hui Liu},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1663},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1663}
}

References

37 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mansheel Agarwal, Valliappan Raju, Rajesh Dey, and Ipseeta Nanda. 2022. Descriptive research on ai-based tools to aid personalized customer service: Case of chatgpt. Journal of Reproducible Research, 1(1):140–146. find
  2. Shiqi Chen, Miao Xiong, Junteng Liu, Zhengxuan Wu, Teng Xiao, Siyang Gao, and Junxian He. 2024. In-context sharpness as alerts: An inner representation perspective for hallucination mitigation. arXiv preprint arXiv:2403.01548. — [arXiv:2403.01548](https://arxiv.org/abs/2403.01548) find
  3. Sunhao Dai, Ninglu Shao, Haiyuan Zhao, Weijie Yu, Zihua Si, Chen Xu, Zhongxiang Sun, Xiao Zhang, and Jun Xu. 2023. Uncovering chatgpt’s capabilities in recommender systems. In Proceedings of the 17th ACM Conference on Recommender Systems, pages 1126–1132. find
  4. Dante Everaert, Rohit Patki, Tianqi Zheng, and Christopher Potts. 2024. Amazonqac: A large-scale, naturalistic query autocomplete dataset. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 1046–1055. find
  5. Rhys Gould, Euan Ong, George Ogden, and Arthur Conmy. 2023. Successor heads: Recurring, interpretable attention heads in the wild. arXiv preprint arXiv:2312.09230. find
  6. Danny Halawi, Jean-Stanislas Denain, and Jacob Steinhardt. 2023. Overthinking the truth: Understanding how language models process false demonstrations. arXiv preprint arXiv:2307.09476. find
  7. F. Maxwell Harper and Joseph A. Konstan. 2015. The movielens datasets: History and context. ACM Trans. Interact. Intell. Syst., 5(4). find
  8. Pengfei He, Zhenwei Dai, Xianfeng Tang, Yue Xing, Hui Liu, Jingying Zeng, Qiankun Peng, Shrivats Agrawal, Samarth Varshney, Suhang Wang, et al. 2025. Attention knows whom to trust: Attention-based trust management for llm multi-agent systems. arXiv preprint arXiv:2506.02546. — [arXiv:2506.02546](https://arxiv.org/abs/2506.02546) find
  9. Ziwei Ji, Tiezheng Yu, Yan Xu, Nayeon Lee, Etsuko Ishii, and Pascale Fung. 2023. Towards mitigating llm hallucination via self reflection. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1827–1843.
  10. Guangyuan Jiang, Manjie Xu, Song-Chun Zhu, Wenjuan Han, Chi Zhang, and Yixin Zhu. 2023. Evaluating and inducing personality in pre-trained language models. Advances in Neural Information Processing Systems, 36:10622–10643.
  11. Sheng Li and Handong Zhao. 2021. A survey on representation learning for user modeling. In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, pages 4997–5003. find
  12. Yuhong Li, Yingbing Huang, Bowen Yang, Bharat Venkitesh, Acyr Locatelli, Hanchen Ye, Tianle Cai, Patrick Lewis, and Deming Chen. 2024. Snapkv: Llm knows what you are looking for before generation. Advances in Neural Information Processing Systems, 37:22947–22970.
  13. Junling Liu, Chao Liu, Peilin Zhou, Renjie Lv, Kang Zhou, and Yan Zhang. 2023. Is chatgpt a good recommender? a preliminary study. arXiv preprint arXiv:2304.10149. — [arXiv:2304.10149](https://arxiv.org/abs/2304.10149) find
  14. Qijiong Liu, Nuo Chen, Tetsuya Sakai, and Xiao-Ming Wu. 2024. Once: Boosting content-based recommendation with both open-and closed-source large language models. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining, pages 452–461. find
  15. Sun, Ruihua Song, et al. 2025b. User behavior simulation with large language model-based agents. ACM Transactions on Information Systems, 43(2):1–37.
  16. Zhining Liu, Rana Ali Amjad, Ravinarayana Ad-kathimar, Tianxin Wei, and Hanghang Tong. 2025. Selfelicit: Your language model secretly knows where is the relevant evidence. arXiv preprint arXiv:2502.08767.
  17. Shen Wang, Tianlong Xu, Hang Li, Chaoli Zhang, Joleen Liang, Jiliang Tang, Philip S Yu, and Qingsong Wen. 2024. Large language models for education: A survey and outlook. arXiv preprint arXiv:2403.18105.
  18. Hanjia Lyu, Song Jiang, Hanqing Zeng, Yinglong Xia, Qifan Wang, Si Zhang, Ren Chen, Christopher Leung, Jiajie Tang, and Jiebo Luo. 2023. Llm-rec: Personalized recommendation via prompting large language models. arXiv preprint arXiv:2307.15780. find
  19. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837.
  20. Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, et al. 2022. In-context learning and induction heads. arXiv preprint arXiv:2209.11895.
  21. Fan Yang, Zheng Chen, Ziyan Jiang, Eunah Cho, Xiaojiang Huang, and Yanbin Lu. 2023. Palr: Personalization aware llms for recommendation. arXiv preprint arXiv:2305.07622. — [arXiv:2305.07622](https://arxiv.org/abs/2305.07622) find
  22. Shashank Rajput, Nikhil Mehta, Anima Singh, Raghu-nandan Hulikal Keshavan, Trung Vu, Lukasz Heldt, Lichan Hong, Yi Tay, Vinh Tran, Jonah Samost, et al. 2023. Recommender systems with generative retrieval. Advances in Neural Information Processing Systems, 36:10299–10315. find
  23. Alireza Salemi, Sheshera Mysore, Michael Bendersky, and Hamed Zamani. 2024. Lamp: When large language models meet personalization. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7370–7392.
  24. Gregory Serapio-García, Mustafa Safdari, Clément Crepy, Luning Sun, Stephen Fitz, Marwa Abdulhai, Aleksandra Faust, and Maja Matarić. 2023. Personality traits in large language models.
  25. Yunxiao Shi, Wujiang Xu, Zeqi Zhang, Xing Zi, Qiang Wu, and Min Xu. 2025. Personax: A recommendation agent oriented user modeling framework for long behavior sequence. arXiv preprint arXiv:2503.02398. find
  26. Jiwei Tang, Zhicheng Zhang, Shunlong Wu, Jingheng Ye, Lichen Bai, Zitai Wang, Tingwei Lu, Lin Hai, Yiming Zhao, Hai-Tao Zheng, et al. 2025. Gmsa: Enhancing context compression via group merging and layer semantic alignment. arXiv preprint arXiv:2505.12215.
  27. Fali Wang, Minhua Lin, Yao Ma, Hui Liu, Qi He, Xianfeng Tang, Jiliang Tang, Jian Pei, and Suhang Wang. 2025a. A survey on small language models in the era of large language models: Architecture, capabilities, and trustworthiness. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discover… find
  28. Lei Wang, Jingsen Zhang, Hao Yang, Zhi-Yuan Chen, Jiakai Tang, Zeyu Zhang, Xu Chen, Yankai Lin, Hao Qinkai Yu, Mingyu Jin, Dong Shu, Chong Zhang, Lizhou Fan, Wenyue Hua, Suiyuan Zhu, Yanda Meng, Zhenting Wang, Mengnan Du, et al. 2024. Health-llm: Personalized retrieval-augmented disease prediction s… — [arXiv:2402.00746](https://arxiv.org/abs/2402.00746) find
  29. Shenglai Zeng, Pengfei He, Kai Guo, Tianqi Zheng, Hanqing Lu, Yue Xing, and Hui Liu. 2025a. Towards context-robust llms: A gated representation fine-tuning approach. arXiv preprint arXiv:2502.14100.
  30. Shenglai Zeng, Jiankun Zhang, Pengfei He, Yue Xing, Yiding Liu, Han Xu, Jie Ren, Shuaiqiang Wang, Dawei Yin, Yi Chang, et al. 2024. The good and the bad: Exploring privacy issues in retrieval-augmented generation (rag). arXiv preprint arXiv:2402.16893. find
  31. Shenglai Zeng, Jiankun Zhang, Bingheng Li, Yuping Lin, Tianqi Zheng, Dante Everaert, Hanqing Lu, Hui Liu, Yue Xing, Monica Xiao Cheng, et al. 2025b. Towards knowledge checking in retrieval-augmented generation: A representation perspective. In Proceedings of the 2025 Conference of the Nations of the…
  32. An Zhang, Yuxin Chen, Leheng Sheng, Xiang Wang, and Tat-Seng Chua. 2024a. On generative agents in recommendation. In Proceedings of the 47th international ACM SIGIR conference on research and development in Information Retrieval, pages 1807– 1817. find
  33. Junjie Zhang, Yupeng Hou, Ruobing Xie, Wenqi Sun, Julian McAuley, Wayne Xin Zhao, Leyu Lin, and Ji-Rong Wen. 2024b. Agentcf: Collaborative learning with autonomous language agents for recommender systems. In Proceedings of the ACM Web Conference 2024, pages 3679–3689. find
  34. Zihuai Zhao, Wenqi Fan, Jiatong Li, Yunqing Liu, Xiaowei Mei, Yiqi Wang, Zhen Wen, Fei Wang, Xiangyu Zhao, Jiliang Tang, et al. 2024. Recommender systems in the era of large language models (llms). find
  35. IEEE Transactions on Knowledge and Data Engineering, 36(11):6889–6907. find
  36. Yujia Zhou, Qiannan Zhu, Jiajie Jin, and Zhicheng Dou. 2024. Cognitive personalized search integrating large language models with an efficient memory mechanism. In Proceedings of the ACM Web Conference 2024, pages 1464–1473. find
  37. Xi Zhu, Yu Wang, Hang Gao, Wujiang Xu, Chen Wang, Zhiwei Liu, Kun Wang, Mingyu Jin, Linsey Pang, Qingsong Weng, et al. 2025. Recommender systems meet large language model agents: A survey. Foundations and Trends® in Privacy and Security, 7(4):247–396. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zeng-2026-attn-gs-attention-guided-context/