Elamur Research⌕
☰
home / papers / 2026 / From Atomic to Complex tasks: Cross-Tasking Improves Zero-Shot Argument Generation and Retrieval

From Atomic to Complex tasks: Cross-Tasking Improves Zero-Shot Argument Generation and Retrieval

Paper · 2026 · Lynda Tamine, Ahmed Rayane Kebir, Merveille Dona Codjo et al.

Summary

This paper asks whether LLMs can perform demanding argumentation tasks, argument generation (AG) and argument retrieval (AR), without being trained on them, by reusing simpler skills learned beforehand. The authors treat AG and AR as complex tasks built on three foundational ("atomic") capabilities:

  • Recognizing how an argument is structured (AST)
  • Determining the relations that hold between arguments (ARL)
  • Argument Quality Assessment (AQA)

They introduce ArgLLM, which includes:

  • an instruction dataset of 18.9K examples, created by converting six atomic-task datasets into multiple-choice QA format
  • six cross-tasking strategies, based on either multi-tasking (MT) or merging separately fine-tuned models (MG)
  • a set of 12 resulting models, built on Mistral and Llama backbones

These models are then tested zero-shot on four AG and AR benchmarks. Cross-tasked models generally beat their base models, and the results hold across strategies, tasks and model sizes. Gains are not uniform, however: some smaller configurations lose ground, particularly in retrieval.

The approach matters because it avoids task-specific fine-tuning and heavy prompt engineering, at a moderate compute cost. It also points toward modular, more interpretable argumentation systems.

Key points

  • Atomic tasks: six tasks in three categories (AST, ARL, AQA), cast as multiple-choice QA with one manually designed template per dataset. The transformed data were not fully verified. Datasets are released on Hugging Face (mcodjo/argllm-datasets).
  • Complex-task benchmarks: AEG (BLEU-4) and iDebate (BLEU) for generation; Arguana and Touché (NDCG@10) for retrieval.
  • Llama-3-8B zero-shot baselines: 4.45 BLEU-4 on AEG, 2.70 BLEU on iDebate, 32.14 NDCG@10 on Arguana and 33.35 on Touché. ArgLLM results are reported as relative gains or losses against these figures.
  • Training regimes: both PEFT and full fine-tuning were tested at 1B, 3B and 8B scales. PEFT fits on a single H100, while full fine-tuning may require up to four.
  • Stated limitations: only two small-sized LLM families were evaluated, test-set contamination is possible, and no LLM-as-a-judge evaluation was performed.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Institut de Recherche en Informatique de ToulouseToulouse · France35 works · 1,872 citations🎓Université de ToulouseToulouse · France16 works · 1,999 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPhilosophy · 284 works

🧭 Topics 1Research subjects and languages the work is about.Information retrieval & search · 473 works

Cite

Cite as Lynda Tamine, Ahmed Rayane Kebir, Merveille Dona Codjo, Enzo Pasquies and Jose G Moreno. 2026. From Atomic to Complex tasks: Cross-Tasking Improves Zero-Shot Argument Generation and Retrieval. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1246

@inproceedings{tamine2026atomic,
  title = {{From Atomic to Complex tasks: Cross-Tasking Improves Zero-Shot Argument Generation and Retrieval}},
  author = {Lynda Tamine, Ahmed Rayane Kebir, Merveille Dona Codjo, Enzo Pasquies and Jose G Moreno},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1246},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1246}
}

References

57 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anwoy Chatterjee, Eshaan Tanwar, Subhabrata Dutta, and Tanmoy Chakraborty. 2024. Language models can exploit cross-task in-context learning for data-scarce novel tasks. In Annual Meeting of the Association for Computational Linguistics. find
  2. Mohammad Yeghaneh Abkenar, Weixing Wang, Hendrik Graupner, and Manfred Stede. 2024. Assessing open-source large language models on argumentation mining subtasks. arXiv preprint arXiv:2411.05639. — [arXiv:2411.05639](https://arxiv.org/abs/2411.05639) find
  3. AI@Meta. 2024. Llama 3 model card. find
  4. Milad Alshomary, Shahbaz Syed, Martin Potthast, and Henning Wachsmuth. 2020. Target inference in argument conclusion generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4334–4345, Online. Association for Computational Linguistics. find
  5. Guizhen Chen, Liying Cheng, Anh Tuan Luu, and Lidong Bing. 2024a. Exploring the potential of large language models in computational argumentation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2309–2330, Bangkok, Thailand. A… find
  6. Jiaao Chen, Xiaoman Pan, Dian Yu, Kaiqiang Song, Xiaoyang Wang, Dong Yu, and Jianshu Chen. 2024b. Skills-in-context: Unlocking compositionality in large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 13838–13890, Miami, Florida, USA. Association for… find
  7. Kaustubh Dhole, Kai Shu, and Eugene Agichtein. 2025. ConQRet: A new benchmark for fine-grained automatic evaluation of retrieval augmented computational argumentation. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human… find
  8. Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. 2024. Improving factuality and reasoning in language models through multiagent debate. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org.
  9. Kaveh Eskandari Miandoab and Vasanth Sarathy. 2024. “let’s argue both sides”: Argument generation can force small models to utilize previously inaccessible reasoning capabilities. In Proceedings of the 1st Workshop on Customizable NLP: Progress and Challenges in Customizing NLP for a Domain, Applica… find
  10. Amirhossein Farzam, Shashank Shekhar, Isaac Mehlhaff, and Marco Morucci. 2024. Multi-task learning improves performance in deep argument mining models. In Proceedings of the 11th Workshop on Argument Mining (ArgMining 2024), pages 46–58, Bangkok, Thailand. Association for Computational Linguistics. find
  11. Thibault Formal, Benjamin Piwowarski, and Stéphane Clinchant. 2021. Splade: Sparse lexical and expansion model for first stage ranking. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2288–2292.
  12. Gabriel Freedman, Adam Dejl, Deniz Gorur, Xiang Yin, Antonio Rago, and Francesca Toni. 2024. Argumentative large language models for explainable and contestable decision-making. find
  13. Damián Furman, Pablo Torres, José Rodríguez, Diego Letzen, Maria Martinez, and Laura Alemany. 2023. High-quality argumentative information in low resources approaches improve counter-narrative generation. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2942–2956, Sing… find
  14. Debela Gemechu and Chris Reed. 2024. External knowledge-driven argument mining: Leveraging attention-enhanced multi-network models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 3688– 3709, Miami, Florida, USA. Association for Computational Linguist… find
  15. Deniz Gorur, Antonio Rago, and Francesca Toni. 2024. Can large language models perform relation-based argument mining? arXiv preprint arXiv:2402.11243. find
  16. Ankita Gupta, Ethan Zuckerman, and Brendan O’Connor. 2024. Harnessing toulmin’s theory for zero-shot argument explication. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10259–10276, Bangkok, Thailand. Association for Computa… find
  17. Qianyu He, Jie Zeng, Wenhao Huang, Lina Chen, Jin Xiao, Qianxi He, Xunzhe Zhou, Jiaqing Liang, and Yanghua Xiao. 2024. Can large language models understand real-world complex instructions? In Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on I… find
  18. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. 2022. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3.
  19. Dieuwke Hupkes, Verna Dankers, Mathijs Mul, and Elia Bruni. 2020. Compositionality decomposed: How do neural networks generalise? (extended abstract). In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, pages 5065–5069. International Joint Conferen…
  20. Young Kyun Jang, Dat Huynh, Ashish Shah, Wen-Kai Chen, and Ser-Nam Lim. 2024. Spherical linear interpolation and text-anchoring for zero-shot composed image retrieval. In Computer Vision – ECCV 2024: 18th European Conference, Milan, Italy, September 29–October 4, 2024, Proceedings, Part XIX, page 23… find
  21. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… find
  22. Yohan Jo, Haneul Yoo, Jinyeong Bak, Alice H. Oh, Chris Reed, and Eduard H. Hovy. 2021. Knowledge-enhanced evidence retrieval for counterargument generation. ArXiv, abs/2109.09057. find
  23. Wei-Yu Kao and An-Zi Yen. 2024. MAGIC: Multi-argument generation with self-refinement for domain generalization in automatic fact-checking. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 10891– 10902… find
  24. Tushar Khot, Kyle Richardson, Daniel Khashabi, and Ashish Sabharwal. 2021. Hey ai, can you solve complex tasks by talking to agents? In Association of Computational Linguistics (ACL) Findings. find
  25. Johannes Kiesel, Çağrı Çöltekin, Marcel Gohsen, Sebastian Heineking, Maximilian Heinrich, Maik Fröbe, Tim Hagen, Mohammad Aliannejadi, Sharat Anand, Tomaž Erjavec, Matthias Hagen, Matyáš Kopp, Nikola Ljubešić, Katja Meden, Nailia Mirzakhmedova, Vaidas Morkevičius, Harrisen Scells, Moritz Wolter,… find
  26. Xiaoxi Li, Jiajie Jin, Yujia Zhou, Yuyao Zhang, Peitian Zhang, Yutao Zhu, and Zhicheng Dou. 2025. From matching to generation: A survey on generative information retrieval. 43(3). find
  27. Bill Yuchen Lin, Kangmin Tan, Chris Miller, Beiwen Tian, and Xiang Ren. 2022. Unsupervised cross-task generalization via retrieval augmentation. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22, Red Hook, NY, USA. Curran Associates Inc. find
  28. Jiayu Lin, Rong Ye, Meng Han, Qi Zhang, Ruofei Lai, Xinyu Zhang, Zhao Cao, Xuanjing Huang, and Zhongyu Wei. 2023. Argue with me tersely: Towards sentence-level counter-argument generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 16705–16720, S… find
  29. Zhiyue Liu, Jiahai Wang, and Zhenghong Li. 2021. Topic-to-essay generation with comprehensive knowledge enhancement. ArXiv, abs/2106.15142. — [arXiv:2106.15142](https://arxiv.org/abs/2106.15142) find
  30. Xueguang Ma, Tommaso Teofili, and Jimmy Lin. 2023. Anserini gets dense retrieval: Integration of lucene’s hnsw indexes. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pages 5366–5370. find
  31. Sourab Mangrulkar, Sylvain Gugger, Lysandre Debut, Younes Belkada, Sayak Paul, Benjamin Bossan, and Marian Tietz. 2022. PEFT: State-of-the-art parameter-efficient fine-tuning methods. https://github.com/huggingface/peft. ↗
  32. Luca Mouchel, Debjit Paul, Shaobo Cui, Robert West, Antoine Bosselut, and Boi Faltings. 2024. A logical fallacy-informed framework for argument generation. arXiv preprint arXiv:2408.03618. find
  33. Arianna Muti, Federico Ruggeri, Khalid Al Khatib, Alberto Barrón-Cedeño, and Tommaso Caselli. 2024. Language is scary when over-analyzed: Unpacking implied misogynistic reasoning with argumentation theory-driven prompts. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language… find
  34. Preslav Nakov, Alberto Barrón-Cedeño, Giovanni Da San Martino, Firoj Alam, Julia Maria Struß, Thomas Mandl, Rubén Míguez, Tommaso Caselli, Mucahid Kutlu, Wajdi Zaghouani, et al. 2022. Overview of the clef–2022 checkthat! lab on fighting the covid-19 infodemic and fake news detection. In Internationa… find
  35. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI. Accessed: 2024-11-15. find
  36. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1–67.
  37. Joshua Robinson and David Wingate. 2022. Leveraging large language models for multiple choice question answering. International Conference on Learning Representations.
  38. Christian Stab and Iryna Gurevych. 2014. Annotating argument components and relations in persuasive essays. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 1501–1510, Dublin, Ireland. Dublin City University and Association for Co…
  39. Maja Stahl, Timon Ziegenbein, Joonsuk Park, and Henning Wachsmuth. 2025. ArgInstruct: Specialized instruction fine-tuning for computational argumentation. In Findings of the Association for Computational Linguistics: ACL 2025, pages 11103–11127, Vienna, Austria. Association for Computational Linguis… find
  40. George Stoica, Pratik Ramesh, Boglarka Ecsedi, Leshem Choshen, and Judy Hoffman. 2025. Model merging with SVD to tie the knots. In The Thirteenth International Conference on Learning Representations. find
  41. Nandan Thakur, Luiz Bonifacio, Maik Fröbe, Alexander Bondarenko, Ehsan Kamalloo, Martin Potthast, Matthias Hagen, and Jimmy Lin. 2024. Systematic evaluation of neural retrieval models on the touché 2020 argument retrieval subset of beir. In Proceedings of the 47th International ACM SIGIR Conference… find
  42. Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. Beir: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2). find
  43. Henning Wachsmuth, Gabriella Lapesa, Elena Cabrio, Anne Lauscher, Joonsuk Park, Eva Maria Vecchi, Serena Villata, and Timon Ziegenbein. 2024. Argument quality assessment in the age of instruction-following large language models. In Proceedings of the 2024 Joint International Conference on Computatio… find
  44. Henning Wachsmuth, Nona Naderi, Yufang Hou, Yonatan Bilu, Vinodkumar Prabhakaran, Tim Alberd-ingk Thijm, Graeme Hirst, and Benno Stein. 2017a. Computational argumentation quality assessment in natural language. In Proceedings of the 15th Conference of the European Chapter of the Association for Comp…
  45. Henning Wachsmuth, Martin Potthast, Khalid Al Khatib, Yamen Ajjour, Jana Puschmann, Jiani Qu, Jonas Dorsch, Viorel Morari, Janek Bevendorff, and Benno Stein. 2017b. Building an argument search engine for the web. In Proceedings of the 4th Workshop on Argument Mining, pages 49–59. find
  46. Henning Wachsmuth, Benno Stein, and Yamen Ajjour. 2017c. “pagerank” for argument relevance. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 1117–1127. find
  47. Henning Wachsmuth, Shahbaz Syed, and Benno Stein. 2018. Retrieval of the best counterargument without prior topic knowledge. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 241–251, Melbourne, Australia. Association for Comput…
  48. Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt. 2022. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inferenc…
  49. Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal. 2023a. Ties-merging: resolving interference when merging models. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Red Hook, NY, USA. Curran Associates Inc.
  50. Prateek Yadav, Derek Tam, Leshem Choshen, Colin A Raffel, and Mohit Bansal. 2023b. Ties-merging: Resolving interference when merging models. Advances in Neural Information Processing Systems, 36:7093–7115.
  51. Qinyuan Ye. 2024. Cross-task generalization abilities of large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 4: Student Research Workshop), pages 255–262, Mexico City, Mexico.… find
  52. Kun Zhang, Jiali Zeng, Fandong Meng, Yuanzhuo Wang, Shiqi Sun, Long Bai, Huawei Shen, and Jie Zhou. 2024. Tree-of-reasoning question decomposition for complex question answering with large language models. AAAI’24/IAAI’24/EAAI’24. AAAI Press. find
  53. Yi Zhang, Sujay Kumar Jauhar, Julia Kiseleva, Ryen White, and Dan Roth. 2021. Learning to decompose and organize complex tasks. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2726–2735, Online.… find
  54. Lu Wang and Wang Ling. 2016. Neural network-based abstract generation for opinions and arguments. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 47–57, San Diego, California. Association for Com…
  55. Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang 0002, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc V. Le, and Ed H. Chi. 2023. Least-to-most prompting enables complex reasoning in large language models. In The Eleventh International Conference on Learning Represen…
  56. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22, R…
  57. Shengyao Zhuang, Honglei Zhuang, Bevan Koopman, and Guido Zuccon. 2024. A setwise approach for effective and highly efficient zero-shot ranking with large language models. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 38–47. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tamine-2026-atomic-complex-tasks-cross-tasking/