Elamur Research⌕
☰
home / papers / 2026 / Reassessing Active Learning Adoption in Contemporary NLP: A Community Survey

Reassessing Active Learning Adoption in Contemporary NLP: A Community Survey

Paper · 2026 · Julia Romberg, Christopher Schröder, Julius Gonsior et al.

Summary

Active learning (AL) aims to reduce labeling costs by letting a model choose which instances a human should annotate. AL research in *CL venues has grown with transformer models and LLMs, but it is unclear whether these advances have reached real-world practice. Romberg, Schröder and colleagues address this with an online community survey modelled on the one Tomanek and Olsson ran in 2009.

The questionnaire has 52 questions in seven groups and uses branching logic. It covers annotation needs, alternatives to AL, methodological and task setup, and expected trends. The authors analysed answers from 144 academic and industry NLP practitioners both quantitatively and through qualitative coding.

Respondents expect data annotation to remain a bottleneck and AL to stay relevant. LLMs are now the typical backbone model and are viewed as beneficial to AL. However, three obstacles reported over 15 years ago still persist: setup complexity, uncertain cost savings, and inadequate tooling. The authors suggest ways to mitigate each one. They also release the anonymized responses, an uncommon empirical record of how AL is actually applied.

Key points

  • The survey ran from 15 December 2024 to 26 January 2025. It was distributed through mailing lists (ACL, ELRA corpora, ELRA SIGUL, etc.), social media, annotation tool providers, and direct emails to 601 authors of AL papers published 2009–2024.
  • The sample of 144 respondents leans towards academia (107) and Europe (87), but spans 44 countries.
  • In the 2009 baseline survey, only about 20% of respondents had used AL for annotation, despite wide awareness of the method.
  • The study is structured around five research questions (RQ1–RQ5). These cover annotation needs, AL versus alternative methods, current AL practice, observed and expected developments, and comparison with the 2009 survey.
  • The anonymized dataset is available at doi.org/10.7802/2990 under CC BY-NC-SA 4.0.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Leipzig UniversityLeipzig · Germany26 works · 8,622 citations🎓Technische Universität DresdenDresden · Germany13 works · 7,336 citations🎓Center for Scalable Data Analytics and Artificial IntelligenceDresden · Germany10 works · 77 citations🔬GESIS - Leibniz Institute for the Social SciencesMannheim · Germany10 works · 5 citations🔬Institut für Angewandte Informatik (InfAI) e. V.Leipzig · Germany2 works · 1 citation

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksEducation · 203 works

Cite

Cite as Julia Romberg, Christopher Schröder, Julius Gonsior, Katrin Tomanek and Fredrik Olsson. 2026. Reassessing Active Learning Adoption in Contemporary NLP: A Community Survey. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-long.120

@inproceedings{romberg2026reassessing,
  title = {{Reassessing Active Learning Adoption in Contemporary NLP: A Community Survey}},
  author = {Julia Romberg, Christopher Schröder, Julius Gonsior, Katrin Tomanek and Fredrik Olsson},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.120},
  url = {https://doi.org/10.18653/v1/2026.eacl-long.120}
}

References

63 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Joachim Baumann, Paul Röttger, Aleksandra Urman, Albert Wendsjö, Flor Miriam Plaza del Arco, Johannes B. Gruber, and Dirk Hovy. 2025. Large language model hacking: Quantifying the hidden risks of using llms for text annotation. Preprint, arXiv:2509.08825. — [arXiv:2509.08825](https://arxiv.org/abs/2509.08825) find
  2. Markus Bayer and Christian Reuter. 2024. Activellm: Large language model-based active learning for textual few-shot scenarios. Preprint, arXiv:2405.10808.
  3. Verena Blaschke, Christoph Purschke, Hinrich Schuetze, and Barbara Plank. 2024. What do dialect speakers want? a survey of attitudes towards language technology for German dialects. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), p… find
  4. Ekaterina Borisova, Raia Abu Ahmad, Leyla Garcia-Castro, Ricardo Usbeck, and Georg Rehm. 2024. Surveying the FAIRness of annotation tools: Difficult to find, difficult to reuse. In Proceedings of The 18th Linguistic Annotation Workshop (LAW-XVIII), pages 29–45, St. Julians, Malta. Association for Co… find
  5. Liat Ein-Dor, Alon Halfon, Ariel Gera, Eyal Shnarch, Lena Dankin, Leshem Choshen, Marina Danilevsky, Ranit Aharonov, Yoav Katz, and Noam Slonim. 2020. Active Learning for BERT: An Empirical Study. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), page… find
  6. Daria Galimzianova and Leonid Sanochkin. 2024. Efficient active learning with adapters. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 14374–14383, Miami, Florida, USA. Association for Computational Linguistics. find
  7. Abhishek Ghose and Emma Thuong Nguyen. 2024. On the fragility of active learners for text classification. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 22217–22233, Miami, Florida, USA. Association for Computational Linguistics. find
  8. Bertram Højer, Terne Sasha Thorn Jakobsen, Anna Rogers, and Stefan Heinrich. 2025. Research community perspectives on “intelligence” and large language models. In Findings of the Association for Computational Linguistics: ACL 2025, pages 25796– 25812, Vienna, Austria. Association for Computational L… find
  9. Yufang Hou, Charles Jochim, Martin Gleize, Francesca Bonin, and Debasis Ganguly. 2021. TDMSci: A specialized corpus for scientific literature entity tagging of tasks datasets and metrics. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics:… find
  10. Fran Jelenić, Josip Jukić, Nina Drobac, and Jan Snajder. 2023. On dataset transferability in active learning for transformers. In Findings of the Association for Computational Linguistics: ACL 2023, pages 2282–2295, Toronto, Canada. Association for Computational Linguistics. find
  11. Josip Jukić, Fran Jelenić, Miroslav Bićanić, and Jan Snajder. 2023. ALANNO: An active learning annotation system for mortals. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, pages 228–235, Dubrovnik, Croatia.… find
  12. Nataliia Kholodna, Sahib Julka, Mohammad Khodadadi, Muhammed Nurullah Gumus, and Michael Granitzer. 2024. Llms in the loop: Leveraging large language model annotations for active learning in low-resource languages. In Machine Learning and Knowledge Discovery in Databases. find
  13. Applied Data Science Track: European Conference, ECML PKDD 2024, Vilnius, Lithuania, September 9–13, 2024, Proceedings, Part X, page 397–412, Berlin, Heidelberg. Springer-Verlag. find
  14. Jan-Christoph Klie, Michael Bugert, Beto Boullosa, Richard Eckart de Castilho, and Iryna Gurevych. 2018. The INCEpTION platform: Machine-assisted and knowledge-oriented interactive annotation. In Proceedings of the 27th International Conference on Computational Linguistics: System Demonstrations, pa…
  15. Philipp Kohl, Yoka Krämer, Claudia Fohry, and Bodo Kraft. 2024. Scoping review of active learning strategies and their evaluation environments for entity recognition tasks. In Deep Learning Theory and Applications, pages 84–106, Cham. Springer Nature Switzerland. find
  16. Dongseop Kwon, Sun Kim, Soo Yong Shin, and John Wilbur. 2013. Bioqrator: a web-based interactive biomedical literature curating system. In Proceedings of the fourth biocreative challenge evaluation workshop, volume 1, pages 241–246. find
  17. David D. Lewis and William A. Gale. 1994. A sequential algorithm for training text classifiers. In Proceedings of the 17th Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval, pages 3–12. Springer, ACM/Springer. find
  18. Dongyuan Li, Ying Zhang, Zhen Wang, Shiyin Tan, Satoshi Kosugi, and Manabu Okumura. 2024. Active learning for abstractive text summarization via LLM-determined curriculum and certainty gain maximization. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 8959–8971, Miami… find
  19. Jinggui Liang, Lizi Liao, Hao Fei, Bobo Li, and Jing Jiang. 2024. Actively learn from LLMs with uncertainty propagation for generalized category discovery. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologie… find
  20. Mingnan Liu and Laura Wronski. 2018. Examining completion rates in web surveys via over 25,000 real-world surveys. Social Science Computer Review, 36(1):116–124. find
  21. David Lowell, Zachary C. Lipton, and Byron C. Wallace. 2019. Practical obstacles to deploying active learning. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 21–… find
  22. Haocheng Luo, Wei Tan, Ngoc Nguyen, and Lan Du. 2023. Re-weighting tokens: A simple and effective active learning strategy for named entity recognition. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 12725–12734, Singapore. Association for Computational Linguistics. find
  23. Katerina Margatina and Nikolaos Aletras. 2023. On the limitations of simulating active learning. In Findings of the Association for Computational Linguistics: ACL 2023, pages 4402–4419, Toronto, Canada. Association for Computational Linguistics. find
  24. Katerina Margatina, Loic Barrault, and Nikolaos Aletras. 2022. On the importance of effectively adapting pretrained language models for active learning. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 825–836, Dublin, Ireland… find
  25. Katerina Margatina, Timo Schick, Nikolaos Aletras, and Jane Dwivedi-Yu. 2023. Active learning principles for in-context learning with large language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5011–5034, Singapore. Association for Computational Linguistics… find
  26. Katerina Margatina, Giorgos Vernikos, Loïc Barrault, and Nikolaos Aletras. 2021. Active learning by acquiring contrastive examples. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 650–663, Online and Punta Cana, Dominican Republic. Association for Com… find
  27. Vânia Mendonça, Ricardo Rei, Luísa Coheur, and Alberto Sardinha. 2023. Onception: Active learning with expert advice for real world machine translation. Computational Linguistics, 49(2):325–372. find
  28. Julian Michael, Ari Holtzman, Alicia Parrish, Aaron Mueller, Alex Wang, Angelica Chen, Divyam Madaan, Nikita Nangia, Richard Yuanzhe Pang, Jason Phang, and Samuel R. Bowman. 2023. What do NLP researchers believe? results of the NLP community metasurvey. In Proceedings of the 61st Annual Meeting of t… find
  29. Charlotte Nachtegael, Jacopo De Stefani, and Tom Lenaerts. 2023. ALAMBIC : Active learning automation methods to battle inefficient curation. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, pages 117–127, Dubrovni… find
  30. Fredrik Olsson. 2009. A literature survey of active machine learning in the context of natural language processing. SICS Technical Report T2009:06, Swedish Institute of Computer Science. find
  31. Jiaxin Pei, Aparna Ananthasubramaniam, Xingyao Wang, Naitian Zhou, Apostolos Dedeloudis, Jackson Sargent, and David Jurgens. 2022. POTATO: The portable text annotation tool. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 327–33…
  32. Gabrielle Ras, Ning Xie, Marcel van Gerven, and Derek Doran. 2022. Explainable deep learning: A field guide for the uninitiated. J. Artif. Int. Res., 73. find
  33. Anna Rogers and Sasha Luccioni. 2024. Position: Key claims in LLM research have a long tail of footnotes. In Forty-first International Conference on Machine Learning. find
  34. Julia Romberg and Tobias Escher. 2022. Automated topic categorisation of citizens’ contributions: Reducing manual labelling efforts through active learning. find
  35. In Electronic Government, pages 369–385, Cham. Springer International Publishing. find
  36. Christopher Schröder, Andreas Niekler, and Martin Potthast. 2022. Revisiting uncertainty-based query strategies for active learning with transformers. In Findings of the Association for Computational Linguistics: ACL 2022, pages 2194–2203, Dublin, Ireland. Association for Computational Linguistics. find
  37. Christopher Schröder and Andreas Niekler. 2020. A survey of active learning for text classification using deep neural networks. Preprint, arXiv:2008.07267. find
  38. Artem Shelmanov, Vadim Liventsev, Danil Kireev, Nikita Khromov, Alexander Panchenko, Irina Fedulova, and Dmitry V. Dylov. 2019. Active learning with deep pre-trained models for sequence tagging of clinical and biomedical texts. In 2019 IEEE International Conference on Bioinformatics and Biomedicine… find
  39. Artem Shelmanov, Dmitri Puzyrev, Lyubov Kupriyanova, Denis Belyakov, Daniil Larionov, Nikita Khromov, Olga Kozlova, Ekaterina Artemova, Dmitry V. Dylov, and Alexander Panchenko. 2021. Active learning for sequence tagging with deep pre-trained models and Bayesian uncertainty estimates. In Proceedings… find
  40. Yanyao Shen, Hyokun Yun, Zachary Lipton, Yakov Kronrod, and Animashree Anandkumar. 2017. Deep active learning for named entity recognition. In Proceedings of the 2nd Workshop on Representation Learning for NLP, pages 252–256, Vancouver, Canada. Association for Computational Linguistics. find
  41. Eyal Shnarch, Alon Halfon, Ariel Gera, Marina Danilevsky, Yannis Katsis, Leshem Choshen, Martin Santillan Cooper, Dina Epelboim, Zheng Zhang, and Dakuo Wang. 2022a. Label sleuth: From unlabeled text to a classifier in a few hours. In Proceedings of the 2022 Conference on Empirical Methods in Natural… find
  42. Eyal Shnarch, Alon Halfon, Ariel Gera, Marina Danilevsky, Yannis Katsis, Leshem Choshen, Martin Santillan Cooper, Dina Epelboim, Zheng Zhang, Dakuo Wang, Lucy Yip, Liat Ein-Dor, Lena Dankin, Ilya Shnayderman, Ranit Aharonov, Yunyao Li, Naftali Liberman, Philip Levin Slesarev, Gwilym Newton, and 3 ot… find
  43. Arjun Subramonian, Xingdi Yuan, Hal Daumé III, and Su Lin Blodgett. 2023. It takes two to tango: Navigating conceptualizations of NLP tasks and measurements of performance. In Findings of the Association for Computational Linguistics: ACL 2023, pages 3234–3279, Toronto, Canada. Association for Compu… find
  44. Zhen Tan, Dawei Li, Song Wang, Alimohammad Beigi, Bohan Jiang, Amrita Bhattacharjee, Mansooreh Karami, Jundong Li, Lu Cheng, and Huan Liu. 2024. Large language models for data annotation and synthesis: A survey. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processin…
  45. Katrin Tomanek and Fredrik Olsson. 2009. A web survey on the use of active learning to support annotation of text data. In Proceedings of the NAACL HLT 2009 Workshop on Active Learning for Natural Language Processing, pages 45–48, Boulder, Colorado. Association for Computational Linguistics. find
  46. Manuel Tonneau, Dhaval Adjodah, Joao Palotti, Nir Grinberg, and Samuel Fraiberger. 2022. Multilingual detection of personal employment status on Twitter. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6564–6587, Dublin, Irela… find
  47. Akim Tsvigun, Ivan Lysenko, Danila Sedashov, Ivan Lazichny, Eldar Damirov, Vladimir Karlov, Artemy Belousov, Leonid Sanochkin, Maxim Panov, Alexander Panchenko, Mikhail Burtsev, and Artem Shelmanov. 2022a. Active learning for abstractive text summarization. In Findings of the Association for Computa… find
  48. Akim Tsvigun, Leonid Sanochkin, Daniil Larionov, Gleb Kuzmin, Artem Vazhentsev, Ivan Lazichny, Nikita Khromov, Danil Kireev, Aleksandr Rubashevskii, Olga Shahmatova, Dmitry V. Dylov, Igor Galitskiy, and Artem Shelmanov. 2022b. ALTool-box: A set of tools for active learning annotation of natural lang… find
  49. Akim Tsvigun, Artem Shelmanov, Gleb Kuzmin, Leonid Sanochkin, Daniil Larionov, Gleb Gusev, Manvel Avetisian, and Leonid Zhukov. 2022c. Towards computationally feasible deep active learning. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 1198–1218, Seattle, United Sta… find
  50. Michiel Van Der Meer, Neele Falk, Pradeep K. Murukannaiah, and Enrico Liscio. 2024. Annotator-centric active learning for subjective NLP tasks. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 18537–18555, Miami, Florida, USA. Association for Computati… find
  51. Xinpeng Wang and Barbara Plank. 2023. ACTOR: Active learning with annotator-specific classification heads to embrace human label variation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2046–2052, Singapore. Association for Computational Linguistics… find
  52. Leon Weber and Barbara Plank. 2023. ActiveAED: A human in the loop improves annotation error detection. In Findings of the Association for Computational Linguistics: ACL 2023, pages 8834–8845, Toronto, Canada. Association for Computational Linguistics. find
  53. Max Wiechmann, Seid Muhie Yimam, and Chris Biemann. 2021. ActiveAnno: General-purpose document-level annotation tool with active learning integration. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: De… find
  54. Yu Xia, Xu Liu, Tong Yu, Sungchul Kim, Ryan Rossi, Anup Rao, Tung Mai, and Shuai Li. 2024. Hallucination diversity-aware active learning for text summarization. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Techno… find
  55. Yu Xia, Subhojyoti Mukherjee, Zhouhang Xie, Junda Wu, Xintong Li, Ryan Aponte, Hanjia Lyu, Joe Barrow, Hongjie Chen, Franck Dernoncourt, Branislav Kveton, Tong Yu, Ruiyi Zhang, Jiuxiang Gu, Nesreen K. Ahmed, Yu Wang, Xiang Chen, Hanieh Deilamsalehy, Sungchul Kim, and 15 others. 2025. From selection…
  56. Ruixuan Xiao, Yiwen Dong, Junbo Zhao, Runze Wu, Minmin Lin, Gang Chen, and Haobo Wang. 2023. FreeAL: Towards human-free active learning in the era of large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 14520–14535, Singapore. Associ… find
  57. Michelle Yuan, Hsuan-Tien Lin, and Jordan Boyd-Graber. 2020. Cold-start active learning through self-supervised language modeling. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7935–7948, Online. Association for Computational Linguistics. find
  58. Kamer Yüksel, Ahmet Gunduz, Mohamed Al-badrashiny, and Hassan Sawaf. 2023. EvolveMT: an ensemble MT engine improving itself with usage only. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 5: Industry Track), pages 341–346, Toronto, Canada. Associat… find
  59. Xiangkai Zeng, Sarthak Garg, Rajen Chatterjee, Udhyakumar Nallasamy, and Matthias Paulik. 2019. Empirical evaluation of active learning techniques for neural MT. In Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019), pages 84–93, Hong Kong, China. Associat… find
  60. Ruoyu Zhang, Yanzeng Li, Yongliang Ma, Ming Zhou, and Lei Zou. 2023. LLMaAA: Making large language models as active annotators. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 13088–13103, Singapore. Association for Computational Linguistics. find
  61. Zhisong Zhang, Emma Strubell, and Eduard Hovy. 2022. A survey of active learning for natural language processing. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 6166–6190, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. find
  62. Yuekai Zhao, Haoran Zhang, Shuchang Zhou, and Zhihua Zhang. 2020. Active learning approaches to enhancing neural machine translation. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1796–1806, Online. Association for Computational Linguistics. find
  63. Kaitlyn Zhou, Su Lin Blodgett, Adam Trischler, Hal Daumé III, Kaheer Suleman, and Alexandra Olteanu. 2022. Deconstructing NLG evaluation: Evaluation practices, assumptions, and their implications. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computationa… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/romberg-2026-reassessing-active-learning-adoption-contemporary/