📄 FOOCTTS: Generating Arabic Speech with Acoustic Environment for Football Commentator

Summary

FOOCTTS is a demo system that turns typed Arabic text into speech in the voice of a Tunisian-dialect football commentator. The output keeps the stadium crowd noise in the background rather than producing clean studio audio. The authors had only about 15 minutes of usable commentator speech, so they built an almost fully automatic data pipeline:

• YouTube match videos were cut into segments.
• A large code-switching Arabic ASR model transcribed the speech.
• French words and names were transliterated into Arabic script.
• Farasa added vowel marks (diacritics).
• CTC segmentation aligned audio chunks with transcript pieces.

A small manual step labeled segments by excitement level. For synthesis, a VITS end-to-end model was first trained on one hour of a QASR broadcast-news anchor and then fine-tuned on the commentator data. The evaluation is informal: re-voicing commentary for a past game gave speech that the authors found hard to distinguish from a real commentator. The work matters because it shows that a pretrain-then-adapt recipe with ASR-generated labels can give a dialectal, noisy-domain voice from minutes of found audio.

Key points

• Data and labeling:
  • Audio was extracted from YouTube at 22 kHz.
  • inaSpeechSegmenter tagged segments as speech, noise, music or silence, and the non-speech parts were dropped.
  • A pretrained code-switching Arabic ASR model produced the transcripts.
• Model and training:
  • The model is VITS (a CVAE with normalizing flows and adversarial training), which needs no separate vocoder stage.
  • It was trained with an ESPnet recipe (qasr_tts).
  • Pre-training used 1 hour of one male anchor from the QASR corpus, split into 1150 training and 25 validation utterances.
  • Fine-tuning used the commentator corpus, with 25 utterances set aside as a development set and another 25 reserved as a test set.
• Emotion annotation: about 15 minutes of dialectal speech were hand-labeled as neutral, excited or very excited, with pitch as the guiding cue.
• Known limitation: Farasa diacritizes by standard grammar rules, which can differ from how the commentator actually pronounced the words.
• Deployment: the web front end (JS/CSS) talks to a Flask/Nginx backend, which calls the Farasa server and then runs TTS inference. Choosing the emotion at inference time is left for future work.

Interesting ideas

• I1. Train first on a larger, clean source domain (broadcast news), then adapt with a very small amount of noisy target-domain data, so that minutes of data can be enough. — For Mova (tree): gparse could be pre-trained on a standard English UD treebank and then lightly fine-tuned on a small in-domain set (fables, school texts) instead of being trained from scratch.
• I2. Use a strong pretrained model (here ASR) as an automatic labeler for raw found data, with humans adding only a thin extra layer (excitement labels). — For Mova (learn): this matches the silver-data idea, where witness parsers produce labels on unlabeled text and humans or the LLM teacher add only the scarce, high-value annotations.
• I3. Normalization should follow the actual surface form, not the textbook norm: grammar-based diacritics mismatched the real dialectal pronunciation. — For Mova (tokens): lemmatizer and normalizer rules for informal or dialectal English should keep evidence of what was actually written, rather than silently forcing it into standard forms.
• I4. Foreign-language insertions (French words, player names) were transliterated into the target script so the model sees a single consistent symbol inventory. — For Mova (tokens): code-switched words and foreign names could be mapped to a dedicated, consistent token class, so the tagger does not fragment them into noise.
• I5. Re-segment long chunks so that each training sample carries exactly one label (one emotion), making the supervision signal cleaner. — For Mova (frames): when training speech-act or irony detectors, split text so that each unit carries a single act or tone label, which avoids mixed signals from multi-clause passages.