Elamur Research⌕
☰
home / papers / 2026 / AppTek Call-Center Dialogues: A Multi-Accent Long-Form Benchmark for English ASR

AppTek Call-Center Dialogues: A Multi-Accent Long-Form Benchmark for English ASR

Paper · 2026 · Eugen Beck, Sarah Beranek, Uma Moothiringote et al.

Summary

Benchmarks for English ASR mostly contain short, pre-cut clips of read or prepared speech, rarely label speaker accent, and may already appear in the web-scale training data of open-weight models. The authors address this with the AppTek Call-Center Dialogues corpus, built only for evaluation and never published before release. Speakers role-played agent–customer phone calls across sixteen service topics. Every speaker was recruited from one of fourteen English accent groups, and the calls were transcribed verbatim by professional annotators who used no automatic pre-transcription. Quality assurance had several rounds, including an automatic consistency check that sent suspicious segments back for re-review. Several open-weight ASR systems were scored under different segmentation strategies. Word error rates varied strongly with both accent and segmentation, and manual segmentation was usually the best. Strong scores on general American English did not carry over reliably to other accents. The resource matters because it gives a fresh, contamination-resistant, accent-labelled test of long-form conversational ASR, which is the setting that call-center deployments actually face.

Key points

  • The corpus has 128.6 hours of speech, 156 speakers, 1,746 recordings and 85 annotators. Each accent has roughly 8–11 hours and 10–16 speakers, and no speaker appears in more than one accent group.

  • The fourteen accent groups cover English as spoken in India, China, Singapore, Mexico, Canada, Australia, South Africa, Ireland, Wales, Scotland and Britain, along with AAVE, General US and Southern US varieties.

  • Sessions lasted 5–15 minutes (10.4 on average). Audio was captured over VoIP at 16 kHz with one channel per speaker, mostly on laptops and phones in quiet homes.

  • The transcription markup flags several phenomena:

    • fillers with #;
    • cut-off words with ~;
    • initialisms with <initial> tags;
    • non-English spans with <lang:X> tags;
    • unintelligible stretches with (()).

    Spelling follows US conventions for every accent.

  • The data is released on Hugging Face under CC BY-SA 4.0. About five hours were translated into Chinese, German, Japanese and Spanish to serve as blind test data for the IWSLT 2026 shared task.

Interesting ideas

  • I1. Transcripts were checked with "guided recognition." A recognizer was biased toward each proposed transcript, and a segment was flagged only when it still disagreed confidently (edit distance ≥4, minimum word confidence ≥0.56). Thresholds were tuned to flag 10% of segments, and about 40% of those turned out to need fixes.
  • I2. Reviewers re-examined flagged segments without seeing the machine's alternative hypothesis, so they were not anchored to the model's guess.
  • I3. To limit contamination, the authors commissioned new material that had never been online and suggest refreshing test sets over time, since models trained on unknown data may already have memorised public sets.
  • I4. Results are broken down by accent, and the authors watch the gap between the best and worst groups rather than only the average. A model can improve overall while staying fragile on some groups.
  • I5. Segmentation is treated as a variable in the experiment, not a fixed preprocessing step. Comparing manual and automatic boundaries showed how much error comes from boundary detection alone.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): a silver UD tree could be re-parsed by a witness parser nudged toward it, and only confident disagreements would be sent to repair or an LLM teacher, which focuses review effort where it pays off.
  • I2 (learn): when an LLM teacher or human adjudicates a flagged parse, hiding the competing parser's output keeps the judgement independent and makes agreement statistics more trustworthy.
  • I3 (answer): fables, word problems and school tests used for evaluation could be newly written in-house and rotated periodically, so scores reflect understanding rather than memorised internet text.
  • I4 (tree): LAS could be reported per genre or register (spoken, social media, legal) with the worst-group score tracked, revealing where gparse or the perceptron parser quietly fails.
  • I5 (tree): parsing could be evaluated on both gold and predicted sentence boundaries to isolate how much tree error comes from the segmenter rather than the parser.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSignal Processing · 452 works

🧭 Topics 4Research subjects and languages the work is about.English · 2,263 worksEvaluation & benchmarks · 1,220 worksDialogue systems · 624 worksSpeech & spoken language · 480 works

Cite

Cite as Eugen Beck, Sarah Beranek, Uma Moothiringote, Daniel Mann, Wilfried Michel, Katie Nguyen and Taylor Tragemann. 2026. AppTek Call-Center Dialogues: A Multi-Accent Long-Form Benchmark for English ASR. arXiv:2604.27543.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2604.27543 · arXiv 2604.27543

@misc{beck2026apptek,
  title = {{AppTek Call-Center Dialogues: A Multi-Accent Long-Form Benchmark for English ASR}},
  author = {Eugen Beck, Sarah Beranek, Uma Moothiringote, Daniel Mann, Wilfried Michel, Katie Nguyen and Taylor Tragemann},
  year = {2026},
  doi = {10.48550/arxiv.2604.27543},
  eprint = {2604.27543},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2604.27543}
}

References

12 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. V. Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: An ASR corpus based on public domain audio books,” in 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2015, pp. 5206–5210. [13] X. Shi, X. Wang, Z. Guo, Y. Wang, P. Zhang, X. Zhang, Z. Guo, H. Hao, Y. Xi, B. Yang, J. Xu, J. Zhou, and J. Lin, “Qwen3-ASR Technical Report,” 2026. [Online]. Available: https://arxiv.org/abs/2601.21337 [14] A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” in Proceedings of the 40th International Conference on Machine Learning, ser. ICML’23. JMLR.org, 2023. [15] Silero Team, “Silero VAD: pre-trained enterprise-grade Voice Activity Detector (VAD), Number Detector and Language Classifier,” https://github.com/snakers4/silero-vad/tree/v5.1.2, 2024. [16] V. Srivastav, S. Zheng, E. Bezzam, E. L. Bihan, A. Moumen, and S. Gandhi, “Open ASR Leaderboard: Towards Reproducible and Transparent Multilingual Speech Recognition Evaluation,” 2025. [Online]. Available: https://arxiv.org/abs/2510.06961 ↗
  2. R. Ardila, M. Branson, K. Davis, M. Kohler, J. Meyer, M. Henretty, R. Morais, L. Saunders, F. Tyers, and G. Weber, “Common Voice: A Massively-Multilingual Speech Corpus,” in Proceedings of The 12th Language Resources and Evaluation Conference. Marseille, France: European Language Resources Association, May 2020, pp. 4218–4222. [Online]. Available: https://www.aclweb.org/anthology/2020.lrec-1.520 [17] A. Singh, A. Fry, A. Perelman, A. Tart, A. Ganesh, A. El-Kishky, A. McLaughlin, A. Low, A. Ostrow, A. Ananthram, A. Nathan, A. Luo, A. Helyar, A. Madry, A. Efremov, A. Spyra, A. Baker-Whitcomb, A. Beutel, A. Karpenko, A. Makelov et al., “OpenAI GPT-5 System Card,” 2025. [Online]. Available: https://arxiv.org/abs/2601.03267 ↗
  3. A. Conneau, M. Ma, S. Khanuja, Y. Zhang, V. Axelrod, S. Dalmia, J. Riesa, C. Rivera, and A. Bapna, “FLEURS: FEW-Shot Learning Evaluation of Universal Representations of Speech,” in 2022 IEEE Spoken Language Technology Workshop (SLT), 2023, pp. 798–805. [18] S. Agarwal, L. Ahmad, J. Ai, S. Altman, A. Applebaum, E. Arbus, R. K. Arora, Y. Bai, B. Baker, H. Bao, B. Barak, A. Bennett, T. Bertao, N. Brett, E. Brevdo, G. Brockman, S. Bubeck, C. Chang, K. Chen, M. Chen et al., “gpt-oss-120b & gpt-oss-20b Model Card,” 2025. [Online]. Available: https://arxiv.org/abs/2508.10925 ↗
  4. L. D. Consortium, “2000 HUB5 English Evaluation Transcripts,” Philadelphia, 2002, lDC2002T43, Web Download. ↗
  5. J. Carletta, S. Ashby, S. Bourban, M. Flynn, M. Guillemot, T. Hain, J. Kadlec, V. Karaiskos, W. Kraaij, M. Kronenthal, G. Lathoud, M. Lincoln, A. Lisowska, I. McCowan, W. Post, D. Reidsma, and P. Wellner, “The AMI Meeting Corpus: A Pre-announcement,” in Machine Learning for Multimodal Interaction, S. Renals and S. Bengio, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2006, pp. 28–39. find
  6. J. Barker, S. Watanabe, E. Vincent, and J. Trmal, “The Fifth ’CHiME’ Speech Separation and Recognition Challenge: Dataset, Task and Baselines,” in Proc. Interspeech 2018, 2018, pp. 1561– 1565. find
  7. M. D. Rio, P. Ha, Q. McNamara, C. Miller, and S. Chandra, “Earnings-22: A Practical Benchmark for Accents in the Wild,” 2022. [Online]. Available: https://arxiv.org/abs/2203.15591 ↗
  8. M. Sekoyan, N. R. Koluguri, N. Tadevosyan, P. Zelasko, T. Bartley, N. Karpov, J. Balam, and B. Ginsburg, “Canary-1B-v2 & Parakeet-TDT-0.6B-v3: Efficient and High-Performance Models for Multilingual ASR and AST,” 2025. [Online]. Available: https://arxiv.org/abs/2509.14128 ↗
  9. NVIDIA NeMo Team, “Canary-Qwen-2.5B: A Speech-Augmented Language Model,” https://huggingface.co/nvidia/ canary-qwen-2.5b, 2025, Accessed: 2025-10-19. ↗
  10. G. Saon, A. Dekel, A. Brooks, T. Nagano, A. Daniels, A. Satt, A. Mittal, B. Kingsbury, D. Haws, E. Morais, G. Kurata, H. Aronowitz, I. Ibrahim, J. Kuo, K. Soule, L. Lastras, M. Suzuki, R. Hoory, S. Thomas et al., “Granite-speech: open-source speech-aware LLMs with strong English ASR capabilities,” 2025. [Online]. Available: https://arxiv.org/abs/2505.08699 ↗
  11. N. Zeghidour, E. Kharitonov, M. Orsini, V. Volhejn, G. de Marmiesse, E. Grave, P. Pérez, L. Mazaré, and A. Défossez, “Streaming Sequence-to-Sequence Learning with Delayed Streams Modeling,” 2025. [Online]. Available: https://arxiv.org/abs/2509.08753 ↗
  12. A. Abouelenin, A. Ashfaq, A. Atkinson, H. Awadalla, N. Bach, J. Bao, A. Benhaim, M. Cai, V. Chaudhary, C. Chen, D. Chen, D. Chen, J. Chen, W. Chen, Y.-C. Chen, Y. ling Chen, Q. Dai, X. Dai, R. Fan, M. Gao et al., “Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs,” 2025. [Online]. Available: https://arxiv.org/abs/2503.01743 ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/beck-2026-apptek-call-center-dialogues-multi/