Elamur Research⌕
☰
home / papers / 2026 / Read As Human: Compressing Context via Parallelizable Close Reading and Skimming

Read As Human: Compressing Context via Parallelizable Close Reading and Skimming

Paper · 2026 · Jiwei Tang, Shilei Liu, Zhicheng Zhang et al.

Summary

Long prompts, such as those produced by retrieval-augmented generation, make LLM inference slow because attention cost grows quadratically. They can also hurt accuracy because much of the text is redundant. Existing query-aware compressors have two weaknesses. They either read the whole context at once or compress it step by step, which is slow. They also either throw away text, risking information loss, or turn everything into opaque vectors, losing readability. RAM (Read As HuMan) borrows from how people read. It splits the context into equal chunks and encodes each chunk alongside the query in parallel with a shared LLM-based encoder. Chunks judged highly relevant are kept word for word ("close reading"). The rest are condensed, under query guidance, into a single summary vector each ("skimming"). The decoder then receives this mix of plain text and vectors. A contrastive loss over positive and negative query–segment pairs sharpens the choice between the two modes. Across QA and summarization benchmarks and two backbones, RAM beats prior compressors and runs much faster on long inputs. This makes it a practical, partly interpretable option for long-context deployment.

Key points

  • Relevance scoring: each segment and the query are represented by mean-pooled last hidden states. A temperature-scaled softmax over their cosine similarities ranks the segments. The compression ratio determines how many segments are kept verbatim.
  • Speed: RAM is about 12× faster end to end than the lightweight baseline Provence on inputs averaging 16K tokens (maximum 32K).
  • Robustness to compression: with Qwen3-4B as backbone, Provence, LongLLMLingua and EXIT lose EM steadily as compression tightens, while RAM stays relatively stable.
  • Ablation (Qwen3-4B, 8× compression), NaturalQuestions EM:
    • Full model: 62.41.
    • Without close reading: 46.40.
    • Without the contrastive loss: 49.27.
    • Without skimming: 58.38.
    • Average-pooling skimming: 59.54.
    • On 2WikiMQA, EM falls from 39.63 to 33.29–37.21 across the same variants.
  • Limitation: the contrastive-learning labels for NarrativeQA were generated by Qwen3-235B-A22B-Instruct, so they may contain errors.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tsinghua UniversityBeijing · China333 works · 87,019 citations🏢Alibaba GroupHangzhou · China168 works · 15,386 citations🎓Northeastern UniversityShenyang · China42 works · 1,940 citations🎓Tsinghua Shenzhen International Graduate SchoolShenzhen · China28 works · 3,035 citations🏢Alibaba Group (United States)San Mateo · United States7 works · 329 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 works

Cite

Cite as Jiwei Tang, Shilei Liu, Zhicheng Zhang, Qingsong Lv, Runsong Zhao, Tingwei Lu, Langming Liu, Haibin Chen, Yujin Yuan, Hai-Tao Zheng, Wenbo Su and Bo Zheng. 2026. Read As Human: Compressing Context via Parallelizable Close Reading and Skimming. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1309

@inproceedings{tang2026read,
  title = {{Read As Human: Compressing Context via Parallelizable Close Reading and Skimming}},
  author = {Jiwei Tang, Shilei Liu, Zhicheng Zhang, Qingsong Lv, Runsong Zhao, Tingwei Lu, Langming Liu, Haibin Chen, Yujin Yuan, Hai-Tao Zheng, Wenbo Su and Bo Zheng},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1309},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1309}
}

References

54 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Zhiwei Cao, Qian Cao, Yu Lu, Ningxin Peng, Luyang Huang, Shanbo Cheng, and Jinsong Su. 2024. Retaining key information under high compression ratios: Query-guided compressor for llms. In ACL (1), pages 12685–12695. Association for Computational Linguistics. find
  2. Shaoshen Chen, Yangning Li, Zishan Xu, Yinghui Li, Xin Su, Zifei Shan, and Hai tao Zheng. 2025. Dast: Context-aware compression in llms via dynamic allocation of soft tokens. Preprint, arXiv:2502.11493. find
  3. Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey E. Hinton. 2020. A simple framework for contrastive learning of visual representations. In ICML, volume 119 of Proceedings of Machine Learning Research, pages 1597–1607. PMLR.
  4. Xin Cheng, Xun Wang, Xingxing Zhang, Tao Ge, Si-Qing Chen, Furu Wei, Huishuai Zhang, and Dongyan Zhao. 2024. xrag: Extreme context compression for retrieval-augmented generation with one token. In NeurIPS. find
  5. Alexis Chevalier, Alexander Wettig, Anirudh Ajith, and Danqi Chen. 2023. Adapting language models to compress contexts. In EMNLP, pages 3829–3846. Association for Computational Linguistics.
  6. Nadezhda Chirkova, Thibault Formal, Vassilina Nikoulina, and Stéphane Clinchant. 2025. Provence: efficient and robust context pruning for retrieval-augmented generation. In ICLR. OpenReview.net. find
  7. Eunseong Choi, June Park, Hyeri Lee, and Jongwuk Lee. 2025. Conflict-aware soft prompting for retrieval-augmented generation. CoRR, abs/2508.15253. find
  8. Yuhong Dai, Jianxun Lian, Yitian Huang, Wei Zhang, Mingyang Zhou, Mingqi Wu, Xing Xie, and Hao Liao. 2025. Pretraining context compressor for large language models with embedding-based memory. In ACL (1), pages 28715–28732. Association for Computational Linguistics. find
  9. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. CoRR, abs/2501.12948.
  10. Chenlong Deng, Zhisong Zhang, Kelong Mao, Shuaiyi Li, Tianqing Fang, Hongming Zhang, Haitao Mi, Dong Yu, and Zhicheng Dou. 2025. Unigist: Towards general and hardware-aligned sequence-level long context compression. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. find
  11. Geoffrey B. Duggan and Stephen J. Payne. 2011. Skim reading by satisficing: evidence from eye tracking. In CHI, pages 1141–1150. ACM. find
  12. Alexander R. Fabbri, Irene Li, Tianwei She, Suyi Li, and Dragomir R. Radev. 2019. Multi-news: A large-scale multi-document summarization dataset and abstractive hierarchical model. In ACL (1), pages 1074– 1084. Association for Computational Linguistics.
  13. Yixiong Fang, Tianran Sun, Yuling Shi, and Xiaodong Gu. 2025. Attentionrag: Attention-guided context pruning in retrieval-augmented generation. CoRR, abs/2503.10720. — [arXiv:2503.10720](https://arxiv.org/abs/2503.10720) find
  14. Tao Ge, Jing Hu, Lei Wang, Xun Wang, Si-Qing Chen, and Furu Wei. 2024. In-context autoencoder for context compression in a large language model. In ICLR. OpenReview.net. find
  15. Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020. Constructing A multi-hop QA dataset for comprehensive evaluation of reasoning steps. In COLING, pages 6609–6625. International Committee on Computational Linguistics.
  16. Taeho Hwang, Sukmin Cho, Soyeong Jeong, Hoyun Song, SeungYoon Han, and Jong C. Park. 2025a. EXIT: context-aware extractive compression for enhancing retrieval-augmented generation. In ACL (Findings), pages 4895–4924. Association for Computational Linguistics. find
  17. Taeho Hwang, Sukmin Cho, Soyeong Jeong, Hoyun Song, SeungYoon Han, and Jong C. Park. 2025b. EXIT: Context-aware extractive compression for enhancing retrieval-augmented generation. In Findings of the Association for Computational Linguistics: ACL 2025, pages 4895–4924, Vienna, Austria. Association f… find
  18. Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. 2023. Llmlingua: Compressing prompts for accelerated inference of large language models. In EMNLP, pages 13358–13376. Association for Computational Linguistics.
  19. Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. 2024. Longllmlingua: Accelerating and enhancing llms in long context scenarios via prompt compression. In ACL (1), pages 1658–1677. Association for Computational Linguistics.
  20. Tomás Kociský, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gábor Melis, and Edward Grefenstette. 2018. The narrativeqa reading comprehension challenge. Trans. Assoc. Comput. Linguistics, 6:317–328.
  21. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. In NeurIPS.
  22. Yangning Li, Shaoshen Chen, Yinghui Li, Yankai Chen, Hai-Tao Zheng, Hui Wang, Wenhao Jiang, and Philip S. Yu. 2025a. Admtree: Compressing lengthy context with adaptive semantic trees. Preprint, arXiv:2512.04550. — [arXiv:2512.04550](https://arxiv.org/abs/2512.04550) find
  23. Yucheng Li, Bo Dong, Frank Guerin, and Chenghua Lin. 2023. Compressing context to enhance inference efficiency of large language models. In EMNLP, pages 6342–6353. Association for Computational Linguistics.
  24. Zongqian Li, Yixuan Su, and Nigel Collier. 2025b. 500xcompressor: Generalized prompt compression for large language models. In ACL (1), pages 25081– 25091. Association for Computational Linguistics.
  25. Huanxuan Liao, Wen Hu, Yao Xu, Shizhu He, Jun Zhao, and Kang Liu. 2025. Beyond hard and soft: Hybrid context compression for balancing local and global information retention. CoRR, abs/2505.15774. — [arXiv:2505.15774](https://arxiv.org/abs/2505.15774) find
  26. Barys Liskavets, Maxim Ushakov, Shuvendu Roy, Mark Klibanov, Ali Etemad, and Shane K. Luke. 2025. Prompt compression with context-aware sentence encoding for fast and improved LLM inference. In AAAI, pages 24595–24604. AAAI Press. find
  27. Langming Liu, Shilei Liu, Yujin Yuan, Yizhen Zhang, Bencheng Yan, Zhiyuan Zeng, Zihao Wang, Jiaqi Liu, Di Wang, Wenbo Su, Pengjie Wang, Jian Xu, and Bo Zheng. 2025a. Uqabench: Evaluating user embedding for prompting llms in personalized question answering. In KDD (2), pages 5652–5661. ACM. find
  28. Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024a. Lost in the middle: How language models use long contexts. Trans. Assoc. Comput. Linguistics, 12:157–173.
  29. Xin Liu, Runsong Zhao, Pengcheng Huang, Xinyu Liu, Junyi Xiao, Chunyang Xiao, Tong Xiao, Shengxiang Gao, Zhengtao Yu, and Jingbo Zhu. 2025b. Autoencoding-free context compression for llms via contextual semantic anchors. Preprint, arXiv:2510.08907. — [arXiv:2510.08907](https://arxiv.org/abs/2510.08907) find
  30. Xinyu Liu, Runsong Zhao, Pengcheng Huang, Chunyang Xiao, Bei Li, Jingang Wang, Tong Xiao, and Jingbo Zhu. 2024b. Forgetting curve: A reliable method for evaluating memorization capability for long-context models. Preprint, arXiv:2410.04727. find
  31. Qingsong Lv, Yangning Li, Zihua Lan, Zishan Xu, Jiwei Tang, Yinghui Li, Wenhao Jiang, Hai-Tao Zheng, and Philip S. Yu. 2025. RAISE: reinforenced adaptive instruction selection for large language models. CoRR, abs/2504.07282. — [arXiv:2504.07282](https://arxiv.org/abs/2504.07282) find
  32. Jesse Mu, Xiang Li, and Noah D. Goodman. 2023. Learning to compress prompts with gist tokens. In NeurIPS.
  33. Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin, Victor Rühle, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, and Dongmei Zhang. 2024. Llmlingua-2: Data distillation for efficient and faithful task-agnostic prompt compression. In ACL (Findings), pages 96…
  34. Aleksandar Petrov, Mark Sandler, Andrey Zhmoginov, Nolan Miller, and Max Vladymyrov. 2025. Long context in-context compression by getting to the gist of gisting. CoRR, abs/2504.08934. — [arXiv:2504.08934](https://arxiv.org/abs/2504.08934) find
  35. David Rau, Shuai Wang, Hervé Déjean, and Stéphane Clinchant. 2024. Context embeddings for efficient answer generation in RAG. CoRR, abs/2407.09252. — [arXiv:2407.09252](https://arxiv.org/abs/2407.09252) find
  36. Claude E. Shannon. 1948. A mathematical theory of communication. Bell Syst. Tech. J., 27(3):379–423. find
  37. Sijun Tan, Xiuyu Li, Shishir G. Patil, Ziyang Wu, Tianjun Zhang, Kurt Keutzer, Joseph Gonzalez, and Raluca A. Popa. 2024. Lloco: Learning long contexts offline. In EMNLP, pages 17605–17621. Association for Computational Linguistics. find
  38. Jiwei Tang, Jin Xu, Tingwei Lu, Zhicheng Zhang, Yiming Zhao, Lin Hai, and Hai-Tao Zheng. 2025a. Perception compressor: A training-free prompt compression framework in long context scenarios. In NAACL (Findings), pages 4093–4108. Association for Computational Linguistics. find
  39. Jiwei Tang, Zhicheng Zhang, Shunlong Wu, Jingheng Ye, Lichen Bai, Zitai Wang, Tingwei Lu, Jiaqi Chen, Lin Hai, Hai-Tao Zheng, and Hong-Gee Kim. 2025b. GMSA: enhancing context compression via group merging and layer semantic alignment. CoRR, abs/2505.12215.
  40. Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, and 1 others. 2025. Kimi k2: Open agentic intelligence. Preprint, arXiv:2507.20534.
  41. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NIPS, pages 5998–6008.
  42. Maryanne Wolf. 2018. Reader, come home: The reading brain in a digital world. Harper, an imprint of HarperCollinsPublishers. find
  43. Fangyuan Xu, Weijia Shi, and Eunsol Choi. 2024. RE-COMP: improving retrieval-augmented lms with context compression and selective augmentation. In ICLR. OpenReview.net. find
  44. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, and 1 others. 2024. Qwen2 technical report. CoRR, abs/2407.10671. find
  45. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In EMNLP, pages 2369–2380. Association for Computational Linguistics.
  46. Xubing Ye, Yukang Gan, Xiaoke Huang, Yixiao Ge, and Yansong Tang. 2025. Voco-llama: Towards vision compression with large language models. In CVPR, pages 29836–29846. Computer Vision Foundation / IEEE. find
  47. Chanwoong Yoon, Taewhoo Lee, Hyeon Hwang, Minbyul Jeong, and Jaewoo Kang. 2024. Compact: Compressing retrieved documents actively for question answering. In EMNLP, pages 21424–21439. Association for Computational Linguistics.
  48. Shaoxiong Zhan, Yanlin Lai, Zheng Liu, Hai Lin, Shen Li, Xiaodong Cai, Zijian Lin, Wen Huang, and Hai-Tao Zheng. 2026a. 3viewsense: Spatial and mental perspective reasoning from orthographic views in vision-language models. CoRR, abs/2603.07751. — [arXiv:2603.07751](https://arxiv.org/abs/2603.07751) find
  49. Shaoxiong Zhan, Yanlin Lai, Ziyu Lu, Dahua Lin, Ziqing Yang, and Fei Tan. 2026b. Mathsmith: Towards extremely hard mathematical reasoning by forging synthetic problems with a reinforced policy. In AAAI, pages 34602–34610. AAAI Press. find
  50. Shaoxiong Zhan, Hai Lin, Hongming Tan, Xiaodong Cai, Hai-Tao Zheng, Xin Su, Zifei Shan, Ruitong Liu, and Hong-Gee Kim. 2025. Lexsembridge: Fine-grained dense representation enhancement through token-aware embedding augmentation. In ECAI, Frontiers in Artificial Intelligence and Applications, pages 2… find
  51. Peitian Zhang, Zheng Liu, Shitao Xiao, Ninglu Shao, Qiwei Ye, and Zhicheng Dou. 2025. Long context compression with activation beacon. In ICLR. Open-Review.net.
  52. Runsong Zhao, Xin Liu, Xinyu Liu, Pengcheng Huang, Chunyang Xiao, Tong Xiao, and Jingbo Zhu. 2025a. Position ids matter: An enhanced position layout for efficient context compression in large language models. Preprint, arXiv:2409.14364. find
  53. Yiming Zhao, Jiwei Tang, Shimin Di, Libin Zheng, Jianxing Yu, and Jian Yin. 2025b. Cos: Towards optimal event scheduling via chain-of-scheduling. CoRR, abs/2511.12913. — [arXiv:2511.12913](https://arxiv.org/abs/2511.12913) find
  54. Yunlong Zhao, Haoran Wu, and Bo Xu. 2025c. Leveraging attention to effectively compress prompts for long-context llms. In AAAI, pages 26048–26056. AAAI Press. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tang-2026-read-as-human-compressing-context/