📄 Mixat: A Data Set of Bilingual Emirati-English Speech

Summary

Emirati Arabic is poorly covered by speech technology. Many young Emiratis also move between their dialect and English within a single sentence, which current ASR handles badly. The authors built Mixat, a corpus of about 15 hours of speech taken, with permission, from two podcasts by bilingual Emirati hosts. One podcast is a host–guest conversation and the other is a scripted solo monologue, so the corpus covers both spontaneous and structured speech. The audio was cut near utterance boundaries and transcribed with each language in its own script, Arabic letters for Arabic and Latin spelling for English, so every switch point is visible in the text. A first annotation pass by non-Emirati Arabic speakers was then checked by an Emirati validator for local spelling conventions. The authors also measured mixing intensity with an adapted Code-Mixing Index. Off-the-shelf Whisper, MMS and ArTST all failed badly, with word error rates above 100% on the combined data. This shows that dialectal and code-switched Arabic still needs dedicated resources.

Key points

• Mixat has 14.9 hours and 5,316 utterances from 28 episodes, 14 per podcast. Part 1 comes from "The Direction" (conversational) and Part 2 from "Think With Hessa" (monologue).
• 1,947 utterances (36%) contain code-switching, 3,266 are monolingual Arabic and only 103 are English-only. Average CMI of the code-switched utterances is 0.12 in Part 1, 0.09 in Part 2 and 0.11 overall.
• The CMI gives equal weight (0.5 each) to two parts: the minority-language word share and the number of switch points per word. It is modified from Chowdhury et al. (2020).
• Overall WER was 168.52% for Whisper (medium), 147.2% for MMS (MMS-1ball) and 112.2% for ArTST. On the dialect alone, Whisper reached about 196% WER, but it scored 12.06% on English segments. ArTST, which is monolingual, reached 341.6% on English.
• The authors plan to release the data for research at github.com/mbzuai-nlp/mixat.

Interesting ideas

• I1. Writing each language in its own script turns the transcription itself into a language label, so switch points can be found without a separate tagging layer.
• I2. The mixing metric combines how much of the minority language appears with how often the speaker switches, so it separates one long borrowed stretch from constant alternation.
• I3. The annotation pipeline uses general-language annotators first and a native dialect speaker as the final validator, which keeps expert effort small.
• I4. Results are reported separately by language, mixed segments and register (conversation vs monologue), and WER above 100% exposes insertion and hallucination rather than mere misrecognition.
• I5. The conversational part contains many one-word turns such as "ok" or "right", while the monologue has a roughly normal length distribution, so the two registers stress systems differently.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tokens): The tokenizer could mark non-English or foreign-script spans as a distinct token class, so the tagger and lemmatizer skip them explicitly instead of guessing.
• I2 (learn): A similar two-part score could measure how far an input text drifts from clean English. Evaluation sets could then be split by that score, showing where parser accuracy collapses.
• I3 (learn): This fits the LLM teacher tiers: a cheap model or parser proposes silver labels, and only a stronger tier or human reviews them for dialect-specific conventions.
• I4 (learn): Mova's evaluation should break scores down by genre and input type. It should also track over-generation errors separately, since aggregate accuracy can hide them.
• I5 (tree): The parsers could be tested on a dedicated set of fragmentary conversational utterances, checking that they produce sensible UD trees for discourse markers and one-word turns.