📄 Thesis Proposal: Self-Adaptive and Epistemic Uncertainty-Guided ASR of Dense Intra-Sentential Code-Switched Speech for African Low-Resource Languages

Summary

Large multilingual speech models such as Whisper and MMS still struggle when African speakers alternate languages several times within one sentence. Errors cluster at language transitions, adaptation favors the dominant language, and global WER hides these failures. This Student Research Workshop paper is a thesis proposal, so it reports no experimental results. It outlines a framework for Hausa–English (Engausa) and Hausa–Yorùbá built around three research questions:

• Corpus design: a new corpus with token-level language tags, switch regions modeled as time windows around transitions, and a distinction between borrowings and true switches. It will have labeled and unlabeled partitions and will be expanded with LLM-generated text constrained by Equivalence Constraint Theory and the Matrix Language Frame Model.
• Uncertainty at switch points: testing whether epistemic uncertainty, estimated via Monte Carlo Dropout, rises at switch points and can filter pseudo-labels for self-training.
• Switch-aware adaptation: fine-tuning with auxiliary language identification and boundary supervision while limiting catastrophic forgetting.

The work matters because it treats code-switching as structured linguistic behavior rather than noise. It also introduces switch-sensitive evaluation for under-served African languages.

Key points

• Planned data per language pair: about 10 hours of manually transcribed speech plus 20+ hours of unlabeled spontaneous recordings. Splits will be speaker-separated and balanced by switch density, and a double-annotated subset will measure inter-annotator agreement.
• A switch region is defined as all frames within a window of width δ around aligned language-transition times, which accommodates pauses and co-articulation.
• Four baselines are planned: a zero-adaptation multilingual model, standard fine-tuning, naive self-training, and joint ASR + LID without boundary weighting. Proposed components will be added incrementally, ending in a full combined model.
• Evaluation will use overall WER, switch-region WER, and per-language WER, with Δlang (the absolute WER gap between languages) as an imbalance measure. Expected Calibration Error (ECE) will assess confidence calibration.
• The authors argue that existing African resources (OkwuGbé, NaijaVoices, WAXAL) are mostly monolingual and lack dense switch annotation. They cite SEAME as the reference corpus for Mandarin–English code-switched ASR.