Elamur Research⌕
☰
home / papers / 2026 / The Limits of Data Scaling: Sub-token Utilization and Acoustic Saturation in Multilingual ASR

The Limits of Data Scaling: Sub-token Utilization and Acoustic Saturation in Multilingual ASR

Paper · 2026 · Siyu Liang, Nicolas Ballier, Gina-Anne Levow et al.

Summary

The paper asks how much speech is needed before a multilingual ASR model has shown the full set of sub-tokens it actually uses for a language. It also asks whether unequal pre-training data changes that usage. Instead of measuring error rates, the authors probe Whisper large-v2 while it decodes Common Voice 17.0 audio in 49 languages. At every decoding step they record the Top-K candidate sub-tokens and count how many distinct ones have appeared as more audio is fed in.

The number of distinct sub-tokens found shows no meaningful link to how many hours of a language Whisper saw in training. For every language, discovery grows quickly and then levels off along an exponential saturation curve. The authors call the level-off point acoustic saturation time (AST), and it usually falls near two hours of audio. Script and typology turn out to matter more than data scale: Latin-script languages look better on the metrics than Cyrillic, CJK or Semitic ones. The authors argue this gives a model-based way to judge whether a corpus is large enough and to compare languages more fairly.

Key points

  • Setup: Whisper large-v2 (680k hours of training audio, shared BPE vocabulary of about 50k) is run on Common Voice 17.0.
    • Of 65 candidate languages, 49 remain after error-rate filtering. English is left out because its training share is so large.
  • Decoding protocol: The language ID token is forced, with beam size 5, temperature 0.2 and K = 50.
    • Audio is processed in nested cumulative windows from 10 to 120 minutes, in 10-minute steps.
  • Saturation model: Discovery counts are fitted per language with an exponential saturation curve.
    • AST is the time needed to reach 90% of the fitted asymptote, computed as −ln(0.1)/k. It is typically around two hours.
  • Distribution shape: At 120 minutes, rank–frequency distributions look Zipf-like.
    • A Zipf–Mandelbrot law, chosen by AIC, fits better than plain Zipf, and its parameters differ by script.
  • Token length: Frequency-weighted mean sub-token length rises with resource level.
    • Within the Latin-script subset this correlation is modest but significant.

Interesting ideas

  • I1. Recording the whole Top-K candidate list at each step, not just the winning output, shows how much of a model's label inventory is really in play.
  • I2. Fitting a saturation curve to "new types discovered vs. data seen" and stopping at 90% of the asymptote turns "is this enough data?" into a measurable threshold.
  • I3. Building evaluation windows so each larger one fully contains the smaller ones keeps growth curves monotone and comparable across conditions.
  • I4. Comparing plain Zipf with Zipf–Mandelbrot via AIC uses the fitted head and tail parameters as compact descriptors of how concentrated usage is.
  • I5. Repeating a correlation inside a single-script subset separates the effect of the writing system from the effect of data size.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tree): Log the top-K arc labels considered by the perceptron parser and gparse to find UD relations the parser never seriously considers in a given genre.
  • I2 (learn): Track how many new frame or relation types appear as silver data grows, and stop collecting for a domain once the curve flattens.
  • I3 (answer): Grow the fable and school-test evaluation sets as nested slices, so that changes in reader accuracy reflect added material rather than resampling noise.
  • I4 (world): Fit both laws to entity and predicate frequencies across text collections to spot genres where a few concepts dominate the world model.
  • I5 (learn): When linking parser quality to training size, compare within one treebank family or genre so dialect differences do not masquerade as scale effects.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksSignal Processing · 213 works

🧭 Topics 2Research subjects and languages the work is about.Multilingual & cross-lingual NLP · 467 worksSpeech & spoken language · 233 works

Cite

Cite as Siyu Liang, Nicolas Ballier, Gina-Anne Levow and Richard Wright. 2026. The Limits of Data Scaling: Sub-token Utilization and Acoustic Saturation in Multilingual ASR. Proceedings of the Fifteenth Language Resources and Evaluation Conference.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/5pht8xx8yosb

@inproceedings{liang2026limits,
  title = {{The Limits of Data Scaling: Sub-token Utilization and Acoustic Saturation in Multilingual ASR}},
  author = {Siyu Liang, Nicolas Ballier, Gina-Anne Levow and Richard Wright},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/5pht8xx8yosb},
  url = {https://doi.org/10.63317/5pht8xx8yosb}
}

References

22 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Vaibhav Aggarwal, Shabari S Nair, Yash Verma, and Yash Jogi. 2025. Adopting Whisper for Confidence Estimation. In ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. ISSN: 2379-190X. find
  2. Orevaoghene Ahia, Sachin Kumar, Hila Gonen, Valentin Hofmann, Tomasz Limisiewicz, Yulia Tsvetkov, and Noah A. Smith. 2024. MAGNET: Improving the Multilingual Fairness of Language Models with Adaptive Gradient-Based Tokenization. Advances in Neural Information Processing Systems, 37:47790–47814. find
  3. Rosana Ardila, Megan Branson, Kelly Davis, Michael Kohler, Josh Meyer, Michael Henretty, Reuben Morais, Lindsay Saunders, Francis Tyers, and Gregor Weber. 2020. Common Voice: A Massively-Multilingual Speech Corpus. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 4218–4222, Marseille, France. European Language Resources Association. ↗
  4. Nicolas Ballier, Taylor Arnold, Adrien Méli, Tori Thurston, and Jean-Baptiste Yunès. 2024. Whisper for L2 speech scoring. International Journal of Speech Technology, 27(4):923–934. ↗
  5. Mingyu Cui, Daxin Tan, Yifan Yang, Dingdong Wang, Huimeng Wang, Xiao Chen, Xie Chen, and Xunying Liu. 2024. Exploring SSL Discrete Tokens for Multilingual ASR. ArXiv:2409.08805 [cs]. ↗
  6. Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. 2017. On Calibration of Modern Neural Networks. In Proceedings of the 34th International Conference on Machine Learning, pages 1321–1330. PMLR. ISSN: 2640-3498. ↗
  7. Yiwei Guo, Zhihan Li, Hankun Wang, Bohan Li, Chongtian Shao, Hanglei Zhang, Chenpeng Du, Xie Chen, Shujie Liu, and Kai Yu. 2025. Recent Advances in Discrete Speech Tokens: A Review. ArXiv:2502.06490 [eess]. ↗
  8. Ximena Gutierrez-Vasques, Christian Bentz, and Tanja Samardžić. 2023. Languages Through 1 https://altae.u-pariscite.fr/project/gifra/the Looking Glass of BPE Compression. Computational Linguistics, 49(4):943–1001. Place: Cambridge, MA Publisher: MIT Press. ↗
  9. Tahir Javed, Sumanth Doddapaneni, Abhigyan Raman, Kaushal Santosh Bhogale, Gowtham Ramesh, Anoop Kunchukuttan, Pratyush Kumar, and Mitesh M. Khapra. 2022. Towards Building ASR Systems for the Next Billion Users. Proceedings of the AAAI Conference on Artificial Intelligence, 36(10):10813–10821. Number: 10. ↗
  10. Allison Koenecke, Anna Seo Gyeong Choi, Katelyn X. Mei, Hilke Schellmann, and Mona Sloane. 2024. Careless Whisper: Speech-to-Text Hallucination Harms. In The 2024 ACM Conference on Fairness, Accountability, and Transparency, pages 1672–1681. ArXiv:2402.08021 [cs]. ↗
  11. Anna Langedijk, Hosein Mohebbi, Gabriele Sarti, Willem Zuidema, and Jaap Jumelet. 2024. DecoderLens: Layerwise Interpretation of Encoder-Decoder Transformers. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 4764–4780, Mexico City, Mexico. Association for Computational Linguistics. find
  12. Benjamin Muller, Antonios Anastasopoulos, Benoît Sagot, and Djamé Seddah. 2021. When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Models. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 448–462, Online. Association for Computational Linguistics.
  13. Aleksandar Petrov, Emanuele La Malfa, Philip Torr, and Adel Bibi. 2023. Language Model Tokenizers Introduce Unfairness Between Languages. Advances in Neural Information Processing Systems, 36:36963–36990.
  14. Vineel Pratap, Andros Tjandra, Bowen Shi, Paden Tomasello, Arun Babu, Sayani Kundu, Ali Elkahky, Zhaoheng Ni, Apoorv Vyas, Maryam Fazel-Zarandi, Alexei Baevski, Yossi Adi, Xiaohui Zhang, Wei-Ning Hsu, Alexis Conneau, and Michael Auli. 2024. Scaling Speech Technology to 1,000+ Languages. Journal of Machine Learning Research, 25(97):1–52. ↗
  15. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine Mcleavey, and Ilya Sutskever. 2023. Robust Speech Recognition via Large-Scale Weak Supervision. In Proceedings of the 40th International Conference on Machine Learning, pages 28492–28518. PMLR. ISSN: 2640-3498. ↗
  16. Najm Ul Sehar, Ayesha Khalid, Farah Adeeba, and Sarmad Hussain. 2025. Benchmarking Whisper for Low-Resource Speech Recognition: An N-Shot Evaluation on Pashto, Punjabi, and Urdu. In Proceedings of the First Workshop on Challenges in Processing South Asian Languages (CHiPSAL 2025), pages 202–207, Abu Dhabi, UAE. International Committee on Computational Linguistics. find
  17. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural Machine Translation of Rare Words with Subword Units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715–1725, Berlin, Germany. Association for Computational Linguistics. ↗
  18. Zheshu Song, Jianheng Zhuo, Yifan Yang, Ziyang Ma, Shixiong Zhang, and Xie Chen. 2024. LoRA-Whisper: Parameter-Efficient and Extensible Multilingual ASR. In Interspeech 2024, pages 3934– 3938. ISCA. find
  19. Monorama Swain, Anna Katrine Van Zee, and Anders Søgaard. 2024. On Mitigating Performance Disparities in Multilingual Speech Recognition. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 5647–5655, Miami, Florida, USA. Association for Computational Linguistics. find
  20. Andros Tjandra, Nayan Singhal, David Zhang, Ozlem Kalinli, Abdelrahman Mohamed, Duc Le, and Michael L. Seltzer. 2023. Massively Multilingual ASR on 70 Languages: Tokenization, Architecture, and Generalization Capabilities. In ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. ISSN: 2379-190X. find
  21. Anna Zee, Marc Zee, and Anders Søgaard. 2024. Group Fairness in Multilingual Speech Recognition Models. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 2213–2226, Mexico City, Mexico. Association for Computational Linguistics. 8. find
  22. Language Resource References Ardila, Rosana and Branson, Megan and Davis, Kelly and Henretty, Michael and Kohler, Michael and Meyer, Josh and Morais, Reuben and Saunders, Lindsay and Tyers, Francis M. and Weber, Gregor. 2024. Common Voice Corpus 17.0. Mozilla. Mozilla Foundation. Available at https://commonvoice.mozilla.org/. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liang-2026-limits-data-scaling-sub-token/