Elamur Research⌕
☰
home / papers / 2026 / Supporting human operators during customer service interactions with agentic-RAG

Supporting human operators during customer service interactions with agentic-RAG

Paper · 2026 · Juan Barrionuevo-Valenzuela, Daniel Calderón-González, Zoraida Callejas et al.

Summary

Call-center operators must find correct answers quickly while still handling customers with empathy, and fully automated bots tend to frustrate users. The authors therefore build a real-time "copilot" that keeps the human operator in charge. It follows the chat, summarizes it, pulls in relevant domain knowledge and proposes candidate replies. They first tried the Italian LUNA corpus and a Graph RAG design, but graphs built from dialogs in Neo4j and Memgraph were inconsistent and costly. They switched to the ABCD corpus, anonymized it and split it into one vector store per topic. An agentic-RAG pipeline in LangChain/LangGraph routes each turn to a topic, retrieves fragments, checks their relevance, rewrites the query when needed, and generates grounded suggestions with Gemini (1.5 Flash, later 2.5 Flash). The system was compared with a naïve RAG baseline using human annotation and an LLM judge. It produced better quality, relevance and accuracy of suggestions, especially in ambiguous or multi-topic conversations. Numeric scores were not in the excerpt provided. The work shows how assistive, non-replacing AI can be engineered for noisy dialog data.

Key points

  • Data: ABCD has over 10,000 acted human–human dialogs covering 10 topics and 55 subtopics. The authors exported it into a topic-based folder hierarchy and built ten separate vector databases so that topics do not mix during retrieval.
  • Anonymization: Emoticons were removed and 12 types of sensitive entities were handled in three ways:
    • regex rules, applied in a fixed order, for cards, passwords, ZIP/PIN codes, emails, phones and order/account IDs;
    • bert-large-NER with a confidence threshold for names and addresses;
    • manual fixes for security answers and some usernames.
  • Agents: The pipeline uses 11 agents, one for each of the 10 ABCD topics plus off_topic. A router picks the agent, a relevance evaluator checks the retrieved fragments, and query rewriting with re-routing is triggered when retrieval fails. Off-topic turns skip RAG, and the copilot instead prompts the operator to gather useful information.
  • Infrastructure: A Flask-SocketIO chat server and a separate RAG worker communicate asynchronously through Redis pub/sub in Docker. Migrating from Gemini 1.5 Flash to Gemini 2.5 Flash, with prompts unchanged, gave faster and more coherent output.
  • Prompt design: The final prompt has the model silently draft 5–10 candidate phrases and filter them with a checklist (grounded in documents, actionable, concise, non-duplicative). It then outputs 3–8 replies, each prefixed with ">", or a fixed "Issue resolved" marker when the problem is solved.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

Cite

Cite as Juan Barrionuevo-Valenzuela, Daniel Calderón-González, Zoraida Callejas and David Griol. 2026. Supporting human operators during customer service interactions with agentic-RAG. Proceedings of the 16th International Workshop on Spoken Dialogue System Technology.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{barrionuevovalenzuela2026supporting,
  title = {{Supporting human operators during customer service interactions with agentic-RAG}},
  author = {Juan Barrionuevo-Valenzuela, Daniel Calderón-González, Zoraida Callejas and David Griol},
  booktitle = {Proceedings of the 16th International Workshop on Spoken Dialogue System Technology},
  year = {2026},
  url = {https://aclanthology.org/2026.iwsds-1.35/}
}

References

14 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Throughout the development process, a complete workflow was designed for collecting, preprocess-References ing, and exploiting call center dialog data. The preprocessing phase proved crucial, as it ensured Filippo Bianchini. 2025. Retrieval-Augmented Generation. In Francesca De Luzi, Flavia Monti, a… find
  2. Derek Chen, Howard Chen, Yi Yang, Alexander Lin, and Zhou Yu. 2021. Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systems. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lang… find
  3. Dorina Chicu, Maria del Mar Pàmies, Gerard Ryan, and Christine Cross. 2019. Exploring the influence of the human factor on customer satisfaction in call centres. BRQ Business Research Quarterly, 22(2):83–95. find
  4. Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang. 2024. Retrieval-Augmented Generation for Large Language Models: A Survey. arXiv preprint. 2312.10997.
  5. Satya Karteek Gudipati. 2025. Transforming the Working Style of Call Center Agents Through Generative AI. International Journal of Advanced Computer Science and Applications, 16(6):9–15. find
  6. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2021. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. arXiv preprint. 2005.11401.
  7. Diego Ordóńez-Camacho, Rafael Melgarejo-Heredia, Mohsen Abbasi, and Lucía González-Solis. 2024. Aurel_ai: Automating an Institutional Help Desk Using an LLM Chatbot. In Proceedings of the 28th World Multi-Conference on Systemics, Cybernetics and Informatics, pages 81–84, Orlando, Florida, USA. find
  8. Massimo Pacella, Paride Vasco, Gabriele Papadia, and Vincenzo Giliberti. 2024. An Assessment of Digitalization Techniques in Contact Centers and Their Impact on Agent Performance and Well-Being. Sustainability, 16(2):1–19. find
  9. Aashraya Sachdeva, Sai Nishanth Padala, Anup Pattnaik, Varun Nathan, Cijo George, and Jithendra Vepa. 2023. Tailored Real-Time Call Summarization System for Contact Centers. In Proceedings of Interspeech, pages 5261–5262, Dublin, Ireland. find
  10. Aditi Singh, Abul Ehtesham, Saket Kumar, and Tala Talaei Khoei. 2025. Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG. arXiv preprint. 2501.09136. find
  11. Lingli Wang, Ni Huang, Yili Hong, Luning Liu, Xun-hua Guo, and Guoqing Chen. 2023. Voice-based AI in call center customer service: A natural field experiment. Production and Operations Management, 32(4):1002–1018. find
  12. Jochen Wirtz, Paul G. Patterson, Werner H. Kunz, Thorsten Gruber, Vinh Nhat Lu, Stefanie Paluch, and Antje Martins. 2018. Brave new world: service robots in the frontline. Journal of Service Management, 29(5):907–931. find
  13. Jochen Wulf and Jürg Meierhofer. 2024. Utilizing Large Language Models for Automating Technical Customer Support. arXiv preprint. 2406.01407. find
  14. Ruby Wenjiao Zhang, Xiaoning Liang, and Szu-Hsin Wu. 2024. When chatbots fail: exploring user coping following a chatbots-induced service failure. Information Technology & People, 37(8):175–195. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/valenzuela-2026-supporting-human-operators-during-customer/