Elamur Research⌕
☰
home / papers / 2026 / Task-Related In-Context Learning

Task-Related In-Context Learning

Paper · 2026 · Wenqiang Wang, Peng Chen, Yan Xiao et al.

Summary

Conventional in-context learning (ICL) presumes that the retrieval pool and the test set use the same label inventory. This paper examines the opposite extreme: the two label spaces share nothing, but the retrieval data is still connected to the target task. An example would be attribute labels such as colour or gender when the goal is to predict species. The authors call such a pool a Task-Related Retrieval Dataset (TRRD).

They propose TRICL, a three-step pipeline:

  1. A simple Bayesian test, estimated from label frequencies in a small random sample, keeps only the retrieval labels that increase the probability of some target label. These form a "related dataset."
  2. Standard ICL on this related task produces an intermediate prediction.
  3. That prediction narrows the candidate target labels, and a zero-shot prompt makes the final choice within the reduced set.

Experiments cover Emotion, SST5, SNLI and STSB with GLM4 9B, LLaMA 3.1 8B and Qwen2.5 7B. TRICL beats retrieval-based ICL baselines, plain prompting and CoT. The work matters because it shows that useful demonstrations need not share labels with the test set, provided they carry task-relevant signal.

Key points

  • Naive ICL fails here. On Emotion with 5 demonstrations and no label overlap, the best naive ICL method reached 47.4%, below plain prompting at 53.2%.
  • Headline results. TRICL reaches up to 71.0% accuracy (SNLI, Qwen2.5 7B) and an MSE of about 0.8 on STSB. Its first stage averages 92.0% accuracy, which makes the label-space reduction reliable.
  • Scales across model sizes. Tested on models from 1B to 70B parameters plus GPT-4o, TRICL peaks at 67.4% on Emotion (10-shot).
  • Works as a plug-in. Adding TRICL's output-space reduction to the CR and CD baselines raises their average accuracy by 27.6% (GLM4 9B, Emotion, 10-shot).
  • Extends to generation. For summarization, STS-B serves as the related pool for choosing among candidate summaries, giving ROUGE-1 = 0.244. The cost is two queries per input, with time and token overhead the authors describe as modest.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Sun Yat-sen UniversityGuangzhou · China53 works · 9,185 citations🔬Peng Cheng LaboratoryShenzhen · China38 works · 2,504 citations🎓Lanzhou UniversityLanzhou · China5 works · 24 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Wenqiang Wang, Peng Chen, Yan Xiao, Yangshijie Zhang, Xiaoyue Lu, Jianjie Huang and Xiaochun Cao. 2026. Task-Related In-Context Learning. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.2154

@inproceedings{wang2026task,
  title = {{Task-Related In-Context Learning}},
  author = {Wenqiang Wang, Peng Chen, Yan Xiao, Yangshijie Zhang, Xiaoyue Lu, Jianjie Huang and Xiaochun Cao},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.2154},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.2154}
}

References

46 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Samuel R Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632– 642.
  2. Peng Chen, Fangjun Huang, and Chao Huang. 2026. Dyc-clip: Dynamic context-aware multi-modal prompt learning for zero-shot anomaly detection. Pattern Recognition, page 113215. find
  3. Chen Cheng, Xinzhi Yu, Haodong Wen, Jinsong Sun, Guanzhang Yue, Yihao Zhang, and Zeming Wei. 2024. Exploring the robustness of in-context learning with noisy labels. arXiv preprint arXiv:2404.18191. — [arXiv:2404.18191](https://arxiv.org/abs/2404.18191) find
  4. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  5. Luc Devroye, László Györfi, and Gábor Lugosi. 2013. A probabilistic theory of pattern recognition, volume 31. Springer Science & Business Media. find
  6. Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An image is worth 16x16 words: Transformers for image recognition at scale. In 9th In…
  7. Yu Fei, Yifan Hou, Zeming Chen, and Antoine Bosselut. 2023. Mitigating label biases for in-context learning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14014–14031.
  8. Yao Fu, Hao Peng, Ashish Sabharwal, Peter Clark, and Tushar Khot. 2023. Complexity-based prompting for multi-step reasoning. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net.
  9. Karan Gupta, Sumegh Roychowdhury, Siva Rajesh Kasa, Santhosh Kumar Kasa, Anish Bhanushali, Nikhil Pattisapu, and Prasanna Srinivasa Murthy. 2023. How robust are llms to in-context majority label bias? arXiv preprint arXiv:2312.16549. — [arXiv:2312.16549](https://arxiv.org/abs/2312.16549) find
  10. Zheng Yi Ho, Siyuan Liang, Sen Zhang, Yibing Zhan, and Dacheng Tao. 2024. Novo: Norm voting off hallucinations with attention heads in large language models. arXiv preprint arXiv:2410.08970. — [arXiv:2410.08970](https://arxiv.org/abs/2410.08970) find
  11. Fanding Huang, Guanbo Huang, Xiao Fan, Yi He, Xiao Liang, Xiao Chen, Qinting Jiang, Faisal Nadeem Khan, Jingyan Jiang, and Zhi Wang. 2026. Semantic-space exploration and exploitation in rlvr for llm reasoning. Preprint, arXiv:2509.23808. find
  12. Fanding Huang, Jingyan Jiang, Qinting Jiang, Hebei Li, Faisal Nadeem Khan, and Zhi Wang. 2025. Cosmic: Clique-oriented semantic multi-space integration for robust clip test-time adaptation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 9772–9781. find
  13. Xiaonan Li and Xipeng Qiu. 2023a. Finding support examples for in-context learning. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6219–6235.
  14. Xiaonan Li and Xipeng Qiu. 2023b. Mot: Pre-thinking and recalling enable chatgpt to self-improve with memory-of-thoughts. CoRR. find
  15. Xiaoxia Li, Siyuan Liang, Jiyi Zhang, Han Fang, Aishan Liu, and Ee-Chien Chang. 2024a. Semantic mirror jailbreak: Genetic algorithm based jailbreak prompts against open-source llms. arXiv preprint arXiv:2402.14872. — [arXiv:2402.14872](https://arxiv.org/abs/2402.14872) find
  16. Xiping Li, Xiangyu Dong, Xingyi Zhang, Kun Xie, Yuanhao Feng, Bo Wang, Guilin Li, Wuxiong Zeng, Xiujun Shu, and Sibo Wang. 2025. Chi-square wavelet graph neural networks for heterogeneous graph anomaly detection. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining… find
  17. Xiping Li and Jianghong Ma. 2026. Aim-cot: Active information-driven multimodal chain-of-thought for vision-language reasoning. Preprint, arXiv:2509.25699. find
  18. Xiping Li, Jianghong Ma, Kangzhe Liu, Shanshan Feng, Haijun Zhang, and Yutong Wang. 2024b. Category-based and popularity-guided video game recommendation: a balance-oriented framework. In Proceedings of the ACM Web Conference 2024, pages 3734– 3744. find
  19. Xiping Li, Aier Yang, Jianghong Ma, Kangzhe Liu, Shanshan Feng, Haijun Zhang, and Yi Zhao. 2026. Cpgrec+: A balance-oriented framework for personalized video game recommendations. ACM Transactions on Information Systems, 44(3):1–44. find
  20. Ziqian Lin and Kangwook Lee. 2024. Dual operating modes of in-context learning. ICLR. Available at https://openreview.net/forum?id= HkAtRP9cOY. ↗
  21. Xinwei Liu, Xiaojun Jia, Jindong Gu, Yuan Xun, Siyuan Liang, and Xiaochun Cao. 2023. Does few-shot learning suffer from backdoor attacks? arXiv preprint arXiv:2401.01377. — [arXiv:2401.01377](https://arxiv.org/abs/2401.01377) find
  22. Xinwei Liu, Xiaojun Jia, Yuan Xun, Siyuan Liang, and Xiaochun Cao. 2024a. Multimodal unlearnable examples: Protecting data against multimodal contrastive learning. arXiv preprint arXiv:2407.16307. — [arXiv:2407.16307](https://arxiv.org/abs/2407.16307) find
  23. Yinpeng Liu, Jiawei Liu, Xiang Shi, Qikai Cheng, Yong Huang, and Wei Lu. 2024b. Let’s learn step by step: Enhancing in-context learning ability with curriculum learning. arXiv preprint arXiv:2402.10738. — [arXiv:2402.10738](https://arxiv.org/abs/2402.10738) find
  24. Shuang Luo, Linjun Shou, Shuming Ma, Jiawei Liu, and Zhiwei Zhang. 2024. In-context learning with retrieval: A survey. Preprint. arXiv:2401.06247. — [arXiv:2401.06247](https://arxiv.org/abs/2401.06247) find
  25. Xinxi Lyu, Sewon Min, Iz Beltagy, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. Z-ICL: Zero-shot in-context learning with pseudo-demonstrations. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2304–2317. find
  26. Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022. Rethinking the role of demonstrations: What makes in-context learning work? In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11048–11064.
  27. Ranjita Naik, Varun Chandrasekaran, Mert Yuksekgonul, Hamid Palangi, and Besmira Nushi. 2023. Diversity of thought improves reasoning abilities of large language models. find
  28. Jongho Park, Jaeseung Park, Zheyang Xiong, Nayoung Lee, Jaewoong Cho, Samet Oymak, Kangwook Lee, and Dimitris Papailiopoulos. 2024. Can mamba learn how to learn? a comparative study on in-context learning tasks. arXiv preprint arXiv:2402.04124. find
  29. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning transferable visual models from natural language supervision. In Proceedings of the 38th Internatio…
  30. Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. 2022. Emergent abilities of large language models.
  31. Trans. Mach. Learn. Res., 2022. find
  32. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, and 1 others. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. find
  33. Jerry Wei, Le Hou, Andrew Kyle Lampinen, Xiangning Chen, Da Huang, Yi Tay, Xinyun Chen, Yifeng Lu, Denny Zhou, Tengyu Ma, and Quoc V Le. 2023. Symbol tuning improves in-context learning in language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pag… find
  34. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020a. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67.
  35. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020b. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21:140:1–140:67.
  36. Elvis Saravia, Hsien-Chi Toby Liu, Yen-Hao Huang, Junlin Wu, and Yi-Hsuan Chen. 2018. Carer: Contextualized affect representations for emotion recognition. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3687–3697.
  37. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2018. Glue: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP Workshop on BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 35…
  38. Lean Wang, Lei Li, Damai Dai, Deli Chen, Hao Zhou, Fandong Meng, Jie Zhou, and Xu Sun. 2023. Label words are anchors: An information flow perspective for understanding in-context learning. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9840–9855.
  39. Zhiyong Wu, Yaoxiang Wang, Jiacheng Ye, and Lingpeng Kong. 2023. Self-adaptive in-context learning: An information compression perspective for in-context example selection and ordering. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)… find
  40. Jiacheng Ye, Zhiyong Wu, Jiangtao Feng, Tao Yu, and Lingpeng Kong. 2023. Compositional exemplars for in-context learning. In International Conference on Machine Learning, pages 39818–39833. PMLR. find
  41. Zonghao Ying, Aishan Liu, Tianyuan Zhang, Zhengmin Yu, Siyuan Liang, Xianglong Liu, and Dacheng Tao. 2024. Jailbreak vision language models via bi-modal adversarial prompt. arXiv preprint arXiv:2406.04031. — [arXiv:2406.04031](https://arxiv.org/abs/2406.04031) find
  42. Yucheng Zhou, Xiang Li, Qianning Wang, and Jianbing Shen. 2024. Visual in-context learning for large vision-language models. arXiv preprint arXiv:2402.11574. find
  43. Yucheng Zhou, Tao Shen, Xiubo Geng, Chongyang Tao, Can Xu, Guodong Long, Binxing Jiao, and Daxin Jiang. 2023. Towards robust ranker for text retrieval. In Findings of the Association for Computational Linguistics: ACL 2023, Toronto, Canada, July 9-14, 2023, pages 5387–5401. Association for Computati… find
  44. Song Wang, Zihan Chen, Chengshuai Shi, Cong Shen, and Jundong Li. 2024. Mixture of demonstrations for in-context learning. Advances in Neural Information Processing Systems, 37:88091–88116. find
  45. Wenqiang Wang and Yangshijie Zhang. 2025. Incomplete in-context learning. arXiv preprint arXiv:2505.07251.
  46. Albert Webson and Ellie Pavlick. 2022. Do prompt-based models really understand the meaning of their prompts? In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2300–2344, Seattle, United States. As…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-task-related-context-learning/