Elamur Research⌕
☰
home / papers / 2026 / TingIS: Real-time Risk Event Discovery from Noisy Customer Incidents at Enterprise Scale

TingIS: Real-time Risk Event Discovery from Noisy Customer Incidents at Enterprise Scale

Paper · 2026 · Jun Wang, Ziyin Zhang, Rui Wang et al.

Summary

Large cloud-native and fintech services sometimes fail in ways that internal monitoring does not catch. Complaints from users can expose these failures early, but the complaints are colloquial, noisy, arrive in heavy streams and span many business lines. TingIS is an end-to-end system from Ant Group that turns this feedback into structured "risk events", aiming to flag a problem from as few as three incidents. The pipeline has five modules:

  • an LLM rewrites each complaint into a short "subject + problem" summary;
  • a cascaded router assigns each incident to a business domain (biz_code);
  • an event linking engine groups incidents and connects them to known events;
  • a layered store tracks event state;
  • a denoising stage filters false alarms using knowledge bases, statistics and behavioral signals.

The linking engine combines LSH pre-clustering, LLM checks of cluster purity, and time-decayed retrieval of historical events. Over a month of production use, TingIS detected 95% of high-priority incidents, and its P90 delay in raising alerts was 3.5 minutes. On benchmarks built from real data, it beat baselines in routing, clustering and SNR. The work shows that LLMs can be used in real-time industrial monitoring when cheap indexing limits how often they are called.

Key points

  • Production scale: the system handled over 300,000 incidents per day, with peaks above 2,000 per minute. One batch takes about 12.4 s end to end, and LLM calls account for roughly 70% of that time. Daily usage is about 8.0M tokens.
  • Models used:
    • Qwen3-8B for summaries;
    • BGE-M3 for embeddings;
    • BGE-Reranker-V2-M3 for routing, applied to a Top-10 candidate pool;
    • Kimi-K2 for cluster auditing and the final merge-or-create decision.
  • Routing results: cascaded routing (keywords first, then semantic retrieval) reached Acc@1 0.669, compared with 0.460 for parallel fusion. The reranker deliberately leaves some incidents unassigned, giving 88.1% coverage. It becomes more useful as the knowledge base gets less complete.
  • Clustering and ablation: TingIS cut the event mismerge rate from 64.3% (DBSCAN) to 21.5%. Removing business-domain partitioning lowers B³-F1 by 15.6%. Removing the LLM summaries costs 7.0% B³-F1.
  • Historical linking: historical matches are scored as semantic similarity multiplied by an exponential decay over days since the event was last active. This stops stale events from absorbing new, unrelated incidents.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Shanghai Jiao Tong UniversityShanghai · China149 works · 16,161 citations

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksInformation Systems · 1,437 worksComputer Networks and Communications · 64 works

Cite

Cite as Jun Wang, Ziyin Zhang, Rui Wang, Hang Yu, Peng Di and Rui Wang. 2026. TingIS: Real-time Risk Event Discovery from Noisy Customer Incidents at Enterprise Scale. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-industry.147

@inproceedings{wang2026tingis,
  title = {{TingIS: Real-time Risk Event Discovery from Noisy Customer Incidents at Enterprise Scale}},
  author = {Jun Wang, Ziyin Zhang, Rui Wang, Hang Yu, Peng Di and Rui Wang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-industry.147},
  url = {https://doi.org/10.18653/v1/2026.acl-industry.147}
}

References

25 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Charu C. Aggarwal, Jiawei Han, Jianyong Wang, and Philip S. Yu. 2003. A framework for clustering evolving data streams. In Proceedings of 29th International Conference on Very Large Data Bases, VLDB 2003, Berlin, Germany, September 9-12, 2003, pages 81–92. Morgan Kaufmann. find
  2. Ann Cao. 2025. Alipay bears cost of system error that applied discounts to user transactions. find
  3. Feng Cao, Martin Ester, Weining Qian, and Aoying Zhou. 2006. Density-based clustering over an evolving data stream with noise. In Proceedings of the Sixth SIAM International Conference on Data Mining, April 20-22, 2006, Bethesda, MD, USA, pages 328–339. SIAM. find
  4. Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. M3-embedding: Multi-linguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. In Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and…
  5. Barry Elad. 2025. Alipay statistics 2025: User adoption, transaction volumes, and technological innovations. find
  6. Mateusz Fedoryszak, Brent Frederick, Vijay Rajaram, and Changtao Zhong. 2019. Real-time event detection on social data streams. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, Anchorage, AK, USA, August 4-8, 2019, pages 2774– 2782. ACM. find
  7. Hansi Hettiarachchi, Mariam Adedoyin-Olowe, Jagdev Bhogal, and Mohamed Medhat Gaber. 2022. Embed2detect: temporally clustered embedded words for event detection in social media. Mach. Learn., 111(1):49–87. find
  8. Chen Huang and Guoxiu He. 2025. Text clustering as classification with llms. In Proceedings of the 2025 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region, SIGIR-AP 2025, Xi’an, China, December 7-10, 2025, pages 374–384. ACM. find
  9. Junegak Joung and Harrison M. Kim. 2021. Automated keyword filtering in latent dirichlet allocation for identifying product attributes from online reviews. Journal of Mechanical Design, 143(8):084501. find
  10. Chapter of the Association for Computational Linguistics: Main Volume, EACL 2021, Online, April 19 - 23, 2021, pages 2330–2340. Association for Computational Linguistics. find
  11. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November…
  12. Kimi Team. 2025. Kimi K2: open agentic intelligence. CoRR, abs/2507.20534.
  13. Zihan Liao, Hang Yu, Jianguo Li, Jun Wang, and Wei Zhang. 2024. D2LLM: Decomposed and distilled large language models for semantic search. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14798–14814, Bangkok, Thailand. Associa… find
  14. Vijay Viswanathan, Kiril Gashteovski, Carolin Lawrence, Tongshuang Wu, and Graham Neubig. 2024. Large language models enable few-shot clustering. Trans. Assoc. Comput. Linguistics, 12:321– 333.
  15. Shichen Liu, Fei Xiao, Wenwu Ou, and Luo Si. 2017. Cascade ranking for operational e-commerce search. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada, August 13 - 17, 2017, pages 1557–1565. ACM. find
  16. Wei Lu and Ali A. Ghorbani. 2009. Network anomaly detection based on wavelet analysis. find
  17. EURASIP J. Adv. Signal Process., 2009. find
  18. Sindhu Tipirneni, Ravinarayana Adkathimar, Nurendra Choudhary, Gaurush Hiranandani, Rana Ali Amjad, Vassilis N. Ioannidis, Changhe Yuan, and Chandan K. Reddy. 2024. Context-aware clustering using large language models. CoRR, abs/2405.00988. — [arXiv:2405.00988](https://arxiv.org/abs/2405.00988) find
  19. Ying Wang, Mengye Ren, and Andrew Gordon Wilson. 2025. In-context clustering with large language models. CoRR, abs/2510.08466. — [arXiv:2510.08466](https://arxiv.org/abs/2510.08466) find
  20. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 40 others. 2025. Qwen3 technical report. CoRR, abs/2505.09388.
  21. Priyaranjan Pattnayak, Amit Agarwal, Hansa Meghwani, Hitesh Laxmichand Patel, and Srikant Panda. 2025. Hybrid AI for responsive multi-turn online conversations with novel dynamic routing and feedback adaptation. CoRR, abs/2506.02097. find
  22. Alina Petukhova, João P. Matos-Carvalho, and Nuno Fachada. 2025. Text clustering with large language model embeddings. International Journal of Cognitive Computing in Engineering, 6:100–108. find
  23. Faraz Rasheed, Peter Peng, Reda Alhajj, and Jon G. Rokne. 2009. Fourier transform based spatial outlier mining. In Intelligent Data Engineering and Automated Learning - IDEAL 2009, 10th International Conference, Burgos, Spain, September 23-26, 2009. Proceedings, volume 5788 of Lecture Notes in Compu… find
  24. Hansheng Ren, Bixiong Xu, Yujing Wang, Chao Yi, Congrui Huang, Xiaoyu Kou, Tony Xing, Mao Yang, Jie Tong, and Qi Zhang. 2019. Time-series anomaly detection service at microsoft. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, Anchorage,… find
  25. Kailash Karthik Saravanakumar, Miguel Ballesteros, Muthu Kumar Chandrasekaran, and Kathleen R. McKeown. 2021. Event-driven news stream clustering using entity-aware contextual embeddings. In Proceedings of the 16th Conference of the European find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-tingis-real-time-risk-event/