📄 Network of Data Centres (NetDC): BNSC - An Arabic Broadcast News Speech Corpus

Summary

This paper documents how ELDA built BNSC, a transcribed speech corpus of Modern Standard Arabic radio news. The corpus was the first concrete product of NetDC, a European-funded project (December 2000 to December 2002) that set up co-operation between ELRA/ELDA in Europe and the LDC in the United States. The partners wanted to test their shared rules for producing and distributing language resources on a real collection. So they wrote a reusable specification template, drawing on SpeechDat experience, which fixed file formats, directory layout, naming, annotation levels and transcription conventions.

Daily news from Radio Orient in Paris was recorded and then transcribed by native speakers in Transcriber. The transcripts mark sections, topics and speaker turns. A phonetic lexicon in Arabic SAMPA was produced alongside them. SPEX checked the final corpus against the specification.

The result is more than 20 hours of transcribed broadcast news, distributed by both ELDA (catalogue reference S0157) and LDC. It is useful for ASR, topic tracking and retrieval work on Arabic.

Key points

• The corpus has 37 recordings made between 1 November 2001 and 6 January 2002, taken from Radio Orient (FM 94.3 MHz, Paris). The 18h bulletin gave about 25 minutes of news. Some 23h bulletins with chronicles gave about 35 minutes each.
• Each recording has three files:
  • a .wav file with raw 16 kHz, 16-bit mono PCM audio;
  • a .sam file with the SAM label;
  • a .trs file with the XML transcription.
• Files are named DD_YYMMDD_SSS_LL. Supporting tables include SESSION.TBL, SPEAKER.TBL, SAMPSTAT.TXT (sample range, clipping rate, duration) and the SAMPA/DARPA lexicons.
• Transcription was done in Transcriber 1.4.2, in two passes: first sections and turns, then the text itself. One 35-minute broadcast took about 5 days to transcribe.
• Quality control combined several steps:
  • a transcription manual given to all transcribers;
  • a check of each transcript by a second transcriber;
  • weekly transcriber meetings;
  • an external Quality Check report written by SPEX.

Interesting ideas

• I1. The specification was written as a uniform template that can be filled in for any new database, so formats, naming and annotation levels are settled before any data is collected.
• I2. Annotation happens in two passes: first the coarse structure (sections, topics, speaker turns), and only then the detailed content inside each unit.
• I3. A second annotator cross-checks the work, and regular meetings feed decided changes back into the guidelines, so the conventions evolve in a controlled way.
• I4. The pronunciation lexicon was generated automatically from the transcripts and then checked by hand on samples, with a frequency count stored for each entry.
• I5. Inline comments in the transcripts attach the familiar Latin-script name to transliterated proper names (for example Colin Powell), keeping a link between surface form and identity.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tree): Each treebank we import or convert between UD dialects could get a filled-in spec of this kind, so conversion rules are declared up front and not discovered ad hoc.
• I2 (frames): Texts such as tales could first be split into narrator and character turns, so that frames and speech acts are attributed to the right speaker before roles are assigned.
• I3 (learn): Disagreements between witness parsers or LLM teacher tiers could be logged and reviewed in batches, with every decision recorded as a versioned guideline or repair rule.
• I4 (tokens): Lemma and morphology lexicons could be induced automatically and audited on frequency-weighted samples, so human checking goes where errors cost most.
• I5 (world): Entity mentions could carry a similar canonical-identity annotation, such as a Wikidata ID, next to their surface form, which makes entity tracking across a text explainable.