Elamur Research⌕
☰
home / papers / 2026 / Constrained CTC Decoding for Efficient Diacritic Restoration

Constrained CTC Decoding for Efficient Diacritic Restoration

Paper · 2026 · Rufael Marew, Amr Keleg and Hanan Aldarmaki

Summary

The paper addresses speech-to-text diacritization of Arabic. Given a recording and its undiacritized transcript, the goal is a fully vowelled transcript that agrees with both. Earlier systems pair an ASR model with a separate text decoder and train them jointly. The authors replace that multi-modal pipeline with a decoding-time restriction on a plain CTC acoustic model.

A wav2vec-style ASR model is fine-tuned with CTC on diacritized speech from ClArTTS (Classical Arabic) and ArVoice (MSA), about 18h in total. At inference, the undiacritized transcript becomes a linear-chain lattice. Each base letter is emitted as fixed, and after every letter comes a slot that may hold only a diacritic label or the blank. Decoding therefore behaves like partial forced alignment: the letters are locked, and only the vowel marks are chosen.

Against a heavier multi-modal baseline, this method lowers diacritic error rate on both test sets, and the gains are statistically significant. It is also simpler and faster, because it works as a drop-in constraint with no extra module and no retraining.

Key points

  • Data: ClArTTS supplies 12h train / 0.3h test of CA, and ArVoice supplies 6h train / 0.9h test of MSA. Baselines additionally use the text-only Tashkeela corpus. The code is released as DiaCTC on GitHub.
  • Labels follow a one-label-per-letter scheme. Stacked diacritics are fused into one composite token, and the CTC blank stands for "no diacritic".
  • The constraint lattice turns transcript c1c2c3 into the pattern c1.c2.c3.. Each wildcard slot gets parallel arcs for every diacritic label plus blank. It can be built by WFST composition or by pruning beam expansions to the arcs allowed from the current state.
  • Reference diacritic coverage of the test sets is 73.19 for ArVoice and 79.13 for ClArTTS. WER/CER are reported both with and without diacritics, which separates letter errors from diacritic errors.
  • The appendix simulates mixed supervision. Coverage for two training speakers is cut from about 74% to about 37%. When data are mixed naively, coverage for one unseen speaker falls from 75% to 35%. Adding <no diac> and <unk diac> tokens fixes this, with <unk diac> banned at inference, and DER does not get worse than in the naive setup.

Interesting ideas

  • I1. Lock the part of the output that is already known and let the model choose only in designated open slots, using a lattice applied at decoding time; this enforces structure without retraining.
  • I2. Keep "unknown label" separate from "no label" during training, then remove the unknown option at inference so the model must commit to a real answer; this lets partially annotated data be used without teaching the model to abstain.
  • I3. Report error both with and without the null/empty predictions to tell precision apart from coverage, and check that forcing extra coverage does not reduce quality.
  • I4. Score the output on the full form and on a stripped form so errors in the fixed skeleton are not mixed up with errors in the added layer.
  • I5. Hold out test speakers but replace their training data with parallel synthetic voices, so the content stays covered while the speaker remains unseen.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): A per-token lattice of lexicon-licensed tags/lemmas could constrain the tagger so it never outputs a morphological analysis the lexicon forbids, and each choice stays explainable as "best among allowed".
  • I2 (learn): Silver data where witness parsers disagree on an arc or role could be tagged as unknown rather than dropped, and that tag forbidden at prediction, so noisy corpora add coverage without biasing the parser toward omissions.
  • I3 (learn): Frame and role evaluation could score assigned roles separately from how often roles are assigned at all, which acts as a negative control against a system that looks accurate only by leaving slots empty.
  • I4 (tree): Parser evaluation could report attachment on tokens with gold versus predicted tokenization/POS, isolating how much LAS loss comes from the tokens stage versus parsing itself.
  • I5 (learn): A source or genre could be held out while its content enters training only through regenerated paraphrases, testing whether the model generalizes beyond surface style.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Mohamed bin Zayed University of Artificial IntelligenceAbu Dhabi · United Arab Emirates47 works · 4,456 citations

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 9,085 worksArtificial Intelligence · 8,888 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksSignal Processing · 213 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 1,485 worksSpeech & spoken language · 233 works

Cite

Cite as Rufael Marew, Amr Keleg and Hanan Aldarmaki. 2026. Constrained CTC Decoding for Efficient Diacritic Restoration. arXiv:2607.18946.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2607.18946 · arXiv 2607.18946

@misc{marew2026constrained,
  title = {{Constrained CTC Decoding for Efficient Diacritic Restoration}},
  author = {Rufael Marew, Amr Keleg and Hanan Aldarmaki},
  year = {2026},
  doi = {10.48550/arxiv.2607.18946},
  eprint = {2607.18946},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2607.18946}
}

References

22 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. N. Y. Habash, Introduction to Arabic natural language processing. Morgan & Claypool Publishers, 2010. find
  2. N. Habash, A. Soudi, and T. Buckwalter, On Arabic Transliteration. Dordrecht: Springer Netherlands, 2007, pp. 15–22. [Online]. Available: https://doi.org/10.1007/978-1-4020-6046-5 2 ↗
  3. A. Mohamed and H. Mubarak, “Advancing Arabic diacritization: Improved datasets, benchmarking, and state-of-the-art models,” in Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Nov. 2025, pp. 16 707–16 719. [Online]. Available: https://aclanthology.org/2025.emnlp-main.846/ ↗
  4. S. Shatnawi, S. Alqahtani, and H. Aldarmaki, “Automatic restoration of diacritics for speech data sets,” in Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). Association for Computational Linguistics, 2024, pp. 4166–4176. [Online]. Available: https://aclanthology.org/2024.naacl-long.233 ↗
  5. S. Shatnawi, S. Alqahtani, S. Shehata, and H. Aldarmaki, “Data augmentation for speech-based diacritic restoration,” in Proceedings of the Second Arabic Natural Language Processing Conference. Association for Computational Linguistics, Aug. 2024, pp. 160–169. [Online]. Available: https://aclanthology.org/ 2024.arabicnlp-1.15/ ↗
  6. A. Ghannam, N. Alharthi, F. Alasmary, K. Al Tabash, S. Sadah, and L. Ghouti, “Abjad AI at NADI 2025: CATT-Whisper: Multimodal diacritic restoration using text and speech representations,” in Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks, 2025, pp. 757–761. ↗
  7. B. Talafha, H. O. Toyin, P. Sullivan, A. A. Elmadany, A. Juma, A. Djanibekov, C. Zhang, H. Alshehhi, H. Aldarmaki, M. Jarrar, N. Habash, and M. Abdul-Mageed, “NADI 2025: The first multidialectal Arabic speech processing shared task,” in Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks. Association for Computational Linguistics, Nov. 2025, pp. 720–733. [Online]. Available: https://aclanthology.org/2025.arabicnlp-sharedtasks.99/ ↗
  8. N. Habash and O. Rambow, “Arabic diacritization through full morphological tagging,” in Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Companion Volume, Short Papers on XX - NAACL ’07. Association for Computational Linguistics, 2007, pp. 53–56. [Online]. Available: http://portal.acm.org/citation.cfm?doid=1614108.1614122 ↗
  9. Y. Belinkov and J. Glass, “Arabic diacritization with recurrent neural networks,” in Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2015, pp. 2281–2285. [Online]. Available: http://aclweb.org/anthology/D15-1274 ↗
  10. H. Aldarmaki and A. Ghannam, “Diacritic recognition performance in Arabic ASR,” in INTERSPEECH 2023. ISCA, 2023, pp. 361–365. [Online]. Available: https://www.isca-archive.org/ interspeech 2023/aldarmaki23 interspeech.html ↗
  11. H. Alaqel and K. El Hindi, “Improving diacritical Arabic speech recognition: Transformer-based models with transfer learning and hybrid data augmentation,” Information, vol. 16, no. 3, p. 161, 2025, publisher: Multidisciplinary Digital Publishing Institute. [Online]. Available: https://www.mdpi.com/2078-2489/16/3/161 ↗
  12. M. Mohri, F. Pereira, and M. Riley, “Weighted finite-state transducers in speech recognition,” Computer Speech & Language, vol. 16, no. 1, pp. 69–88, 2002. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0885230801901846 ↗
  13. C. Hokamp and Q. Liu, “Lexically constrained decoding for sequence generation using grid beam search,” in Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 2017, pp. 1535–1546. [Online]. Available: https://aclanthology.org/P17-1141/ ↗
  14. V. Pratap, A. Hannun, G. Synnaeve, and R. Collobert, “Star temporal classification: Sequence modeling with partially labeled data,” in Advances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, Eds., vol. 35. Curran Associates, Inc., 2022, pp. 13 392–13 403. [Online]. Available: https://proceedings.neurips.cc/paper files/paper/2022/file/ 57587d8d6a7ede0e5302fc22d0878c53-Paper-Conference.pdf ↗
  15. D. Gao, M. Wiesner, H. Xu, L. P. Garcia, D. Povey, and S. Khudanpur, “Bypass Temporal Classification: Weakly Supervised Automatic Speech Recognition with Imperfect Transcripts,” in Interspeech 2023, 2023, pp. 924–928. find
  16. D. Gao, H. Xu, D. Raj, L. P. G. Perera, D. Povey, and S. Khudanpur, “Learning from flawed data: Weakly supervised automatic speech recognition,” in 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 2023, pp. 1–8. find
  17. Y. Nakagome and M. Hentschel, “WCTC-biasing: Retraining-free contextual biasing ASR with wildcard CTC-based keyword spotting and inter-layer biasing,” in Interspeech 2025. ISCA, 2025, pp. 5178–5182. [Online]. Available: https://www.isca-archive.org/interspeech 2025/nakagome25 interspeech.html ↗
  18. Y. Higuchi, S. Watanabe, N. Chen, T. Ogawa, and T. Kobayashi, “Mask CTC: Non-autoregressive end-to-end ASR with CTC and mask predict,” in Interspeech 2020, 2020, pp. 3655–3659. find
  19. T. Zerrouki and A. Balla, “Tashkeela: Novel corpus of Arabic vocalized texts, data for auto-diacritization systems,” Data in Brief, vol. 11, pp. 147–151, 2017. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S2352340917300112 ↗
  20. A. Kulkarni, A. Kulkarni, S. A. M. Shatnawi, and H. Aldarmaki, “ClArTTS: An Open-Source Classical Arabic Text-to-Speech Corpus,” in Interspeech 2023, 2023, pp. 5511–5515.
  21. H. Toyin, R. Marew, H. Alblooshi, S. M. Magdy, and H. Aldarmaki, “ArVoice: A Multi-Speaker Dataset for Arabic Speech Synthesis,” in Interspeech 2025, 2025, pp. 4808–4812.
  22. J. Grosman, “Fine-tuned XLSR-53 large model for speech recognition in Arabic,” https://huggingface.co/jonatasgrosman/ wav2vec2-large-xlsr-53-arabic, 2021. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/marew-2026-constrained-ctc-decoding-efficient-diacritic/