Elamur Research⌕
☰
home / papers / 2023 / Wiki-En-ASR-Adapt: Large-scale synthetic dataset for English ASR Customization

Wiki-En-ASR-Adapt: Large-scale synthetic dataset for English ASR Customization

Paper · 2023 · Alexandra Antonova

Summary

End-to-end ASR systems often get rare names and terms wrong. One fix is contextual spelling correction (CSC), which post-edits the ASR output using a user's list of phrases, the biasing list. CSC works with any black-box recognizer, but public training data for it barely exists. The only open alternative, UserLibri, is small and covers fiction only.

The author builds a large synthetic resource from English Wikipedia:

  • Corruptions: titles go through G2P, then TTS, then several ASR models, which yields realistic misrecognitions.
  • Sub-phrase pairs: character-level alignment splits each pair into aligned smaller units.
  • Context: the original phrases are found in real Wikipedia paragraphs, and corrupted variants are put back in exactly their place.
  • Biasing lists: simulated lists include two kinds of mined hard negatives, related phrases and phrases resembling common words.

The release consists of aggregate tables plus code, so users can generate training sets with chosen properties. A customization model trained on 10 million examples cut WER by up to 18% relative to the baseline transcription. The hard negatives reduced false alarms. The work matters because it gives open research on ASR personalization a broad, multi-domain training source.

Key points

  • The source is 4.5 million English Wikipedia titles. Pronunciations come from G2P in CMU format, audio is synthesized with FastPitch and HiFi-GAN, and transcription uses greedy decoding with Conformer-CTC, FastConformer-CTC, Conformer-Transducer and Quartznet. Whisper models were evaluated but used only for comparison.
  • On the synthetic audio, WER ranges from roughly 69% to 94%. Predictions from different models overlap only about 20%, or about 10% with Quartznet, so pooling all models multiplies the variety of errors.
  • GIZA++ alignment treats characters, including spaces, as "words". It expands the titles into 26 million original/corrupted pairs.
  • The Keys2Paragraph table holds 34 million paragraphs, each paired with the reference phrases it contains. Matching is plain lowercase text search with IDF filtering; no NER is used.
  • The dataset is published on Hugging Face as bene-ges/wiki-en-asr-adapt.

Interesting ideas

  • I1. GIZA++ is run on characters instead of words, so a whole-phrase corruption pair splits into aligned word-level sub-pairs for free, which enlarges the inventory without extra synthesis.
  • I2. Noise is inserted only at the exact place where the original phrase really occurs in authentic text, never at random positions, so every synthetic example keeps a natural context.
  • I3. Candidate lists are deliberately seeded with hard negatives: look-alikes of the correct phrase and phrases that resemble ordinary words. This teaches the model to discriminate and curbs over-triggering.
  • I4. Systems with low agreement serve as a cheap diversity source: their outputs are pooled rather than voted on, because their errors barely overlap.
  • I5. Phrase matching skips NER and relies on IDF thresholds, which also reject phrases whose edge words are too frequent. Thresholds are relaxed when the text shows capitalization.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): The same character-level alignment could map raw-text spellings to treebank tokenization or lemma forms, mining correspondence tables for the tokenizer and lemmatizer.
  • I2 (learn): When building perturbed or silver training data for the parsers, corruptions could be anchored to genuine occurrences, so the model never learns from implausible sentence contexts.
  • I3 (world): Entity linking against Wikidata/WordNet could be trained and evaluated with confusable decoy candidates, giving a natural negative control against false links.
  • I4 (learn): Disagreements between witness parsers could be kept as a pool of realistic error patterns for repair-rule induction, not just discarded when silver data is agreed.
  • I5 (world): Gazetteer lookup of compiled knowledge entries in text could use the same cheap, explainable IDF-plus-case filter to avoid spurious matches on common words.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Moscow Institute of Physics and TechnologyDolgoprudny · Russia8 works · 2,761 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.English · 2,263 worksSpeech & spoken language · 480 works

Cite

Cite as Alexandra Antonova. 2023. Wiki-En-ASR-Adapt: Large-scale synthetic dataset for English ASR Customization. arXiv:2309.17267.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2309.17267 · arXiv 2309.17267

@misc{antonova2023wiki,
  title = {{Wiki-En-ASR-Adapt: Large-scale synthetic dataset for English ASR Customization}},
  author = {Alexandra Antonova},
  year = {2023},
  doi = {10.48550/arxiv.2309.17267},
  eprint = {2309.17267},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2309.17267}
}

References

28 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ding Zhao, Tara N. Sainath, David Rybach, Pat Rondon, Deepti Bhatia, Bo Li, and Ruoming Pang, “Shallow-fusion end-to-end contextual biasing,” in Interspeech, 2019. find
  2. Aditya Gourav, Linda Liu, Ankur Gandhe, Yile Gu, Guitang Lan, Xiangyang Huang, Shashank Kalmane, Gautam Tiwari, Denis Filimonov, Ariya Rastrow, et al., “Personalization strategies for end-to-end speech recognition systems,” in ICASSP, 2021. find
  3. Jennifer Drexler Fox and Natalie Delworth, “Improving contextual recognition of rare words with an alternate spelling prediction model,” in Interspeech, 2022. find
  4. Golan Pundak, Tara N. Sainath, Rohit Prabhavalkar, Anjuli Kannan, and Ding Zhao, “Deep context: End-to-end contextual speech recognition,” 2018 IEEE Spoken Language Technology Workshop (SLT), pp. 418–425, 2018. find
  5. Mahaveer Jain, Gil Keren, Jay Mahadeokar, Geoffrey Zweig, Florian Metze, and Yatharth Saraf, “Contextual RNN-T for open domain ASR,” in Interspeech, 2020. find
  6. Feng-Ju (Claire) Chang, Jing Liu, Martin Radfar, Athanasios Mouchtaris, Maurizio Omologo, Ariya Rastrow, and Siegfried Kunzmann, “Context-aware transformer transducer for speech recognition,” in ASRU, 2021. ↗
  7. Kanthashree Mysore Sathyendra, Thejaswi Muniyappa, Feng-Ju (Claire) Chang, Jing Liu, Jinru Su, Grant Strimel, Athanasios Mouchtaris, and Siegfried Kunzmann, “Contextual adapters for personalized speech recognition in neural transducers,” in ICASSP 2022, 2022. ↗
  8. Khe Chai Sim, Françoise Beaufays, Arnaud Benard, Dhruv Guliani, Andreas Kabel, Nikhil Khare, Tamar Lucassen, Petr Zadrazil, Harry Zhang, Leif Johnson, et al., “Personalization of end-to-end speech recognition on mobile devices for named entities,” in Automatic Speech Recognition and Understanding Workshop (ASRU), 2019. find
  9. Vladimir Bataev, Roman Korostik, Evgeny Shabalin, Vitaly Lavrukhin, and Boris Ginsburg, “Text-only domain adaptation for end-to-end asr using integrated text-to-mel-spectrogram generator,” in Interspeech, 2023. find
  10. Xiaoqiang Wang, Yanqing Liu, Jinyu Li, Veljko Mil-janic, Sheng Zhao, and Hosam Khalil, “Towards contextual spelling correction for customization of end-to-end speech recognition systems,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 30, pp. 3089–3097, 2022. ↗
  11. Alexandra Antonova, Evelina Bakhturina, and Boris Ginsburg, “Spellmapper: A non-autoregressive neural spellchecker for asr customization with candidate retrieval based on n-gram mappings,” in Interspeech, 2023. find
  12. Rao Ma, Mengjie Qian, Mark J. F. Gales, and Kate M. Knill, “Adapting an unadaptable asr system,” in Interspeech, 2023. ↗
  13. Philip Harding, Sibo Tong, and Simon Wiesler, “Selective biasing with trie-based contextual adapters for personalised speech recognition using neural transducers,” in Interspeech, 2023. find
  14. Felix Stahlberg and Shankar Kumar, “Synthetic data generation for grammatical error correction with tagged corruption models,” in Proceedings of the 16th Workshop on Innovative Use of NLP for Building Educational Applications, Online, Apr. 2021, pp. 37–47, Association for Computational Linguistics. ↗
  15. Hao Zhang, Richard Sproat, Axel H Ng, Felix Stahlberg, Xiaochang Peng, Kyle Gorman, and Brian Roark, “Neural models of text normalization for speech applications,” Computational Linguistics, vol. 45, no. 2, pp. 293–337, 2019. ↗
  16. Theresa Breiner, Swaroop Ramaswamy, Ehsan Variani, Shefali Garg, Rajiv Mathews, Khe Chai Sim, Kilol Gupta, Mingqing Chen, and Lara McConnaughey, “UserLibri: A dataset for ASR personalization using only text,” in Interspeech, 2022. find
  17. Uri Alon, Golan Pundak, and Tara N. Sainath, “Contextual speech recognition with difficult negative training examples,” ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6440–6444, 2018. ↗
  18. Maurits Bleeker, Pawel Swietojanski, Stefan Braun, and Xiaodan Zhuang, “Approximate nearest neighbour phrase mining for contextual speech recognition,” 2023. find
  19. Adrian Łańcucki, “FastPitch: Parallel text-to-speech with pitch prediction,” in ICASSP, 2021. ↗
  20. Jungil Kong, Jaehyeon Kim, and Jaekyoung Bae, “Hifi-GAN: Generative adversarial networks for efficient and high fidelity speech synthesis,” in NeurIPS, 2020. find
  21. Dima Rekesh, Samuel Kriman, Somshubra Majumdar, Vahid Noroozi, He Huang, Oleksii Hrinchuk, Ankur Kumar, and Boris Ginsburg, “Fast conformer with linearly scalable attention for efficient speech recognition,” 2023. find
  22. Anmol Gulati, James Qin, Chung-Cheng Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han, Shibo Wang, Zhengdong Zhang, Yonghui Wu, and Ruoming Pang, “Conformer: Convolution-augmented transformer for speech recognition,” in Interspeech. 2020, ISCA.
  23. Samuel Kriman, Stanislav Beliaev, Boris Ginsburg, Jocelyn Huang, Oleksii Kuchaiev, Vitaly Lavrukhin, Ryan Leary, Jason Li, and Yang Zhang, “Quartznet: Deep automatic speech recognition with 1d time-channel separable convolutions,” ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6124–6128, 2019. ↗
  24. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever, “Robust speech recognition via large-scale weak supervision,” in International Conference on Machine Learning, 2022. ↗
  25. Franz Josef Och and Hermann Ney, “A systematic comparison of various statistical alignment models,” Computational linguistics, vol. 29, no. 1, pp. 19–51, 2003. ↗
  26. O. Kuchaiev, J. Li, H. Nguyen, O. Hrinchuk, R. Leary, B. Ginsburg, S. Kriman, S. Beliaev, V. Lavrukhin, J. Cook, et al., “NeMo: a toolkit for building AI applications using neural modules,” in Systems for ML Workshop, NeurIPS, 2019. ↗
  27. Timo Baumann, Arne Köhn, and Felix Hennig, “The Spoken Wikipedia corpus collection: Harvesting, alignment and an application to hyperlistening,” Language Resources and Evaluation, vol. 53, pp. 303–329, 2019. ↗
  28. Patrick K. O’Neill, Vitaly Lavrukhin, Somshubra Majumdar, Vahid Noroozi, Yuekai Zhang, Oleksii Kuchaiev, Jagadeesh Balam, Yuliya Dovzhenko, Keenan Freyberg, Michael D. Shulman, Boris Ginsburg, Shinji Watanabe, and Georg Kucsko, “SPGISpeech: 5, 000 hours of transcribed financial audio for fully formatted end-to-end speech recognition,” in Interspeech, 2021. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/antonova-2023-wiki-en-asr-adapt-large/