📄 QVoice: Arabic Speech Pronunciation Learning Application

Summary

QVoice is a mobile app for practising Arabic pronunciation. It targets two groups: non-native learners, and native speakers whose regional dialect affects their Modern Standard Arabic (MSA). The authors present it as a system demo.

The app has a Flutter front-end and a Flask back-end. The back-end runs an end-to-end mispronunciation detection model. This model scores each character of a recording and gives a separate score for the whole utterance, which lets it pinpoint insertions, deletions and substitutions. An extra attention component looks for errors caused by dialect.

The practice content comes from more than 400 words and phrases drawn from educational sources. Each item carries:

• a transliteration and an English translation,
• reference audio,
• an example sentence,
• extra explanatory material.

Items are vowelized with Farasa and voiced by a TTS engine.

The paper reports response times but no detection accuracy figures. Its value is in showing how fine-grained diagnostic output can be combined with visual, audio and L1-linked cues to support self-paced pronunciation learning.

Key points

• The pipeline has three steps: the front-end fetches a practice item, the user's recording goes to a Flask API, and an end-to-end model returns character-level scores plus one utterance-level score.
• The model detects insertions, deletions, substitutions and mispronounced characters. A dedicated attention mechanism targets errors driven by dialect.
• The database holds more than 400 words/phrases from sources such as Aljazeera Learning and BTEC (Basic Travel Expression Corpus).
  • Text is vowelized with Farasa, and audio comes from QCRI's TTS (Natiq).
  • Supplementary material was produced by an internal generative model and verified for each item.
• The interface has three views:
  • Practice View: the word, an image, a transliteration and a translation.
  • Feedback View: a star rating, color-coded characters on a red-to-green scale, the model's predicted sequence, and reference audio at normal and slowed speed.
  • Assistant View: an example sentence with its translation and TTS audio, plus a graphophonic explanation that uses English examples.
• Reported latency is under 1 second to fetch an item and under 1.5 seconds to return feedback. No accuracy numbers are given in the paper.

Interesting ideas

• I1. The model outputs what it believes was actually pronounced and aligns that against the reference, which turns a bare score into typed edits (inserted, dropped or swapped units). — For Mova (learn): Aligning a witness parser's output against agreed silver trees could label each disagreement by edit type (attachment, label, missing arc), giving structured input for self-induced repair rules.
• I2. Scores are reported at two granularities: fine per-unit scores shown in color, plus one coarse overall rating, so users can see both where it went wrong and how bad it was overall. — For Mova (tree): Each token's arc could carry a confidence shown as a color in the explanation view, alongside a sentence-level parse score, making weak spots in a tree easy to locate.
• I3. A separate model component is devoted to one known source of systematic error (dialect interference) instead of hoping the main model absorbs it. — For Mova (tree): During UD dialect conversion between treebanks, a dedicated detector for convention-specific constructions could flag arcs whose labels depend on the source dialect rather than on the sentence itself.
• I4. Each lexical item is stored as a rich bundle (translation, transliteration, usage sentence, audio), and the parts produced by a generative model are checked by humans for every item. — For Mova (world): Compiled lexicon entries could likewise pair each sense with an LLM-proposed example sentence that is verified before use, giving concrete contexts that ground frames and verb classes.
• I5. Showing a target word inside a full example sentence helps the learner see how it behaves in context, not just in isolation. — For Mova (back): The generator could be tested by producing a contextual sentence for a given lemma/frame and parsing it back, checking that the intended usage survives the round trip.