Elamur Research⌕
☰
home / papers / 2026 / How DDAIR you? Disambiguated Data Augmentation for Intent Recognition

How DDAIR you? Disambiguated Data Augmentation for Intent Recognition

Paper · 2026 · Galo Castillo-López, Alexis Lombard, Nasredine Semmar et al.

Summary

LLM-based data augmentation for few-shot intent recognition can produce synthetic utterances that fit a different intent better than the one they were generated for. Such task-level ambiguity can hurt the classifier. DDAIR addresses this by combining sentence encoders with LLMs. For each intent, the method averages the embeddings of the few in-context examples into a centroid. A generated utterance counts as ambiguous when its nearest centroid belongs to another class. Ambiguous items are either discarded or re-generated with a prompt that names both the target intent and the conflicting intent, repeating the process up to three times.

Experiments use Mistral 7B and Llama-3 8B as generators, three encoders, and BERT classifiers on BANKING77, CLINC150 and MPGT. With each disambiguation round, the share of ambiguous synthetic data shrinks, and the classifiers usually benefit. The largest gains appear on MPGT, whose intents are broad. The work shows that an inexpensive embedding check can make synthetic training data cleaner, though it requires extra LLM calls.

Key points

  • Setup: Generation uses the instruct versions of Mistral 7B and Llama-3 8B with 2–5 ICL examples per intent and one LLM call per utterance. Each n-shot setting is sampled five times. Ambiguity detection uses BGE, MPNet and MiniLM-L6 embeddings.

  • Evaluation: The authors measure three things:

    • the ambiguity ratio after each step;
    • Silhouette coefficients computed against the gold intent clusters;
    • macro-F1 of fine-tuned uncased BERT-base, averaged over 3 seeds × 5 ICL sets.

    They also compare against the PVI-based filtering approach of Lin et al. (2023).

  • Ambiguity reduction: Ambiguity ratios fall after every iteration in all configurations. MPGT, an 8-intent multi-class version of a hospital-robot dialogue corpus, is the most ambiguous dataset.

  • Classification results: On MPGT, simply dropping ambiguous examples works best, giving gains of up to ≈6 macro-F1 points with Mistral and ≈3 with Llama. On BANKING77 and CLINC150, dropping is the weakest option, and the best models come after 2–3 re-generation rounds.

  • Resources: Code is available at github.com/gaalocastillo/ddair. Total generation time was 23.6 h for Mistral and 32.3 h for Llama on one A100, with estimated emissions of about 6.03 kgCO₂eq.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Université Paris-SaclayGif-sur-Yvette · France45 works · 6,127 citations🏛️Commissariat à l'Énergie Atomique et aux Énergies AlternativesParis · France11 works · 416 citations🏛️Laboratoire d'Intégration des Systèmes et des TechnologiesPalaiseau · France8 works · 321 citations🔬CEA Paris-Saclay - Etablissement de SaclaySaclay · France4 works · 207 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Galo Castillo-López, Alexis Lombard, Nasredine Semmar and Gaël de Chalendar. 2026. How DDAIR you? Disambiguated Data Augmentation for Intent Recognition. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-short.20

@inproceedings{castillolopez2026how,
  title = {{How DDAIR you? Disambiguated Data Augmentation for Intent Recognition}},
  author = {Galo Castillo-López, Alexis Lombard, Nasredine Semmar and Gaël {de Chalendar}},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-short.20},
  url = {https://doi.org/10.18653/v1/2026.eacl-short.20}
}

References

36 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Angus Addlesee, Weronika Sieińska, Nancie Gunson, Daniel Hernandez Garcia, Christian Dondrup, and Oliver Lemon. 2023. Multi-party goal tracking with LLMs: Comparing pre-training, fine-tuning, and prompt engineering. In Proceedings of the 24th Annual Meeting of the Special Interest Group on Discours… find
  2. Olatz Arbelaitz, Ibai Gurrutxaga, Javier Muguerza, Jesús M Pérez, and Iñigo Perona. 2013. An extensive comparative study of cluster validity indices. Pattern recognition, 46(1):243–256. find
  3. Gaurav Arora, Shreya Jain, and Srujana Merugu. 2024. Intent detection in the age of LLMs. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 1559–1570, Miami, Florida, US. Association for Computational Linguistics. find
  4. Jesse Atuhurra, Hidetaka Kamigaito, Taro Watanabe, and Eric Nichols. 2024. Domain adaptation in intent classification systems: a review. arXiv preprint arXiv:2404.14415. https://mlco2.github.io/impact/ https://chatgpt.com/ — [arXiv:2404.14415](https://arxiv.org/abs/2404.14415) ↗
  5. Sherleen Beatricia, Albert Reinaldo, Muhammad Fikri Hasani, and 1 others. 2024. Text augmentation for indonesian intent classification: Comparative study. In 2024 11th International Conference on Information Technology, Computer, and Electrical Engineering (ICITACEE), pages 265–270. IEEE. find
  6. Alberto Benayas, Sicilia Miguel-Ángel, and Marçal Mora-Cantallops. 2024. Enhancing intent classifier training with large language model-generated data. Applied Artificial Intelligence, 38(1):2414483. find
  7. Iñigo Casanueva, Tadas Temčinas, Daniela Gerz, Matthew Henderson, and Ivan Vulić. 2020. Efficient intent detection with dual sentence encoders. In Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AI, pages 38–45, Online. Association for Computational Linguistics.
  8. Galo Castillo-López, Gael de Chalendar, and Nasredine Semmar. 2025. Intent recognition and out-of-scope detection using LLMs in multi-party conversations. In Proceedings of the 26th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 504–512, Avignon, France. Association fo… find
  9. Maarten De Raedt, Fréderic Godin, Thomas Demeester, and Chris Develder. 2023. IDAS: Intent discovery with abstractive summarization. In Proceedings of the 5th Workshop on NLP for Conversational AI (NLP4ConvAI 2023), pages 71–88. Association for Computational Linguistics. find
  10. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies,…
  11. Kawin Ethayarajh, Yejin Choi, and Swabha Swayamdipta. 2022. Understanding dataset difficulty with V-usable information. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 5988–6008. PMLR. find
  12. Alessandra Ferrera, Giulio Mezzotero, and Domenico Ursino. 2025. A linguistics-based approach to refining automatic intent detection in conversational agent design. Information Sciences, 689:121493. find
  13. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.
  14. Mengze Hong, Wailing Ng, Chen Jason Zhang, Yuanfeng Song, and Di Jiang. 2025. Dial-In LLM: Human-Aligned LLM-in-the-loop Intent Clustering for Customer Service Dialogues. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 5885–5900, Suzhou, China. Associ… find
  15. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… find
  16. Alexandre Lacoste, Alexandra Luccioni, Victor Schmidt, and Thomas Dandres. 2019. Quantifying the carbon emissions of machine learning. arXiv preprint arXiv:1910.09700.
  17. Stefan Larson, Anish Mahendran, Joseph J. Peper, Christopher Clarke, Andrew Lee, Parker Hill, Jonathan K. Kummerfeld, Kevin Leach, Michael A. Laurenzano, Lingjia Tang, and Jason Mars. 2019. An evaluation dataset for intent classification and out-of-scope prediction. In Proceedings of the 2019 Confer… find
  18. I-Fan Lin, Faegheh Hasibi, and Suzan Verberne. 2024. Generate then refine: Data augmentation for zero-shot intent detection. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 13138–13146, Miami, Florida, USA. Association for Computational Linguistics. find
  19. Yen-Ting Lin, Alexandros Papangelis, Seokhwan Kim, Sungjin Lee, Devamanyu Hazarika, Mahdi Namazifar, Di Jin, Yang Liu, and Dilek Hakkani-Tur. 2023. Selective in-context data augmentation for intent detection using pointwise V-information. In Proceedings of the 17th Conference of the European Chapter… find
  20. Alisa Liu, Zhaofeng Wu, Julian Michael, Alane Suhr, Peter West, Alexander Koller, Swabha Swayamdipta, Noah Smith, and Yejin Choi. 2023. We’re afraid language models aren’t modeling ambiguity. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 790–807, Si…
  21. Pengfei Liu, Youzhang Ning, King Keung Wu, Kun Li, and Helen Meng. 2021. Open intent discovery through unsupervised semantic clustering and dependency parsing. IEEE CogInfoCom-2021. find
  22. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
  23. Elyas Meguellati, Assaad Zeghina, Shazia Sadiq, and Gianluca Demartini. 2025. Llm-based semantic augmentation for harmful content detection. In Proceedings of the International AAAI Conference on Web and Social Media, volume 19, pages 1190–1209. find
  24. Gyutae Park, Ingeol Baek, Byeongjeong Kim, Joongbo Shin, and Hwanhee Lee. 2025. Dynamic label name refinement for few-shot dialogue intent classification. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 41–52, Vienna, Austria… find
  25. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics.
  26. Peter J Rousseeuw. 1987. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of computational and applied mathematics, 20:53–65. find
  27. Gaurav Sahu, Pau Rodriguez, Issam Laradji, Parmida Atighehchian, David Vazquez, and Dzmitry Bahdanau. 2022. Data augmentation for intent classification with off-the-shelf large language models. In Proceedings of the 4th Workshop on NLP for Conversational AI, pages 47–57, Dublin, Ireland. Association…
  28. Yusuf Sali and Sıtkı Can Toraman. 2025. Navigating the unknown: Intent classification and out-of-distribution detection using large language models. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 14652–14664, Suzhou, China. Association for Computational Linguistics. find
  29. Lijia Si, Caili Guo, Zheng Li, and Yang Yang. 2025. A unified framework of data augmentation using large language models for text-based cross-modal retrieval. Pattern Recognition, page 111755. find
  30. Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2020. Mpnet: Masked and permuted pre-training for language understanding. Advances in neural information processing systems, 33:16857– 16867.
  31. Lucas Vendramin, Ricardo JGB Campello, and Eduardo R Hruschka. 2010. Relative clustering validity criteria: A comparative overview. Statistical analysis and data mining: the ASA data science journal, 3(4):209–235. find
  32. Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou. 2020. Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers. Advances in neural information processing systems, 33:5776–5788. find
  33. Yusong Wang, Dongyuan Li, Jialun Shen, Yicheng Xu, Mingkun Xu, Kotaro Funakoshi, and Manabu Okumura. 2024. Lambda: large language model-based data augmentation for multi-modal machine translation. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 15240– 15253. find
  34. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, and 1 others. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 conference on empirical methods in natural lan…
  35. Shitao Xiao, Zheng Liu, Peitian Zhang, Niklas Muennighoff, Defu Lian, and Jian-Yun Nie. 2024. C-Pack: Packed Resources For General Chinese Embeddings. In Proceedings of the 47th International ACM SI-GIR Conference on Research and Development in Information Retrieval, SIGIR ’24, pages 641–649, New Yo…
  36. Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, and 1 others. 2022. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/lopez-2026-how-ddair-you-disambiguated-data/