📄 The American English SALA-II Data Collection

Summary

This paper reports how OHSU's Center for Spoken Language Understanding built the American English part of SALA-II. SALA-II is a consortium-owned corpus of read and spontaneous speech over cellular phones, intended for training speech recognizers and modelled on SpeechDat. The specifications demanded 4000 speakers, balanced by gender, age, nine accent regions and five noise environments. Each speaker read 44 prompted items plus phonetically rich sentences and words.

The author covers three areas:

• Prompt sheets. 20,000 distinct sheets were generated. Mid-collection, sheets not yet handed out were regenerated to fill coverage gaps.
• Tooling. Recording, transcription and status tracking ran through an SQL database.
• Recruitment. This proved the hardest part.

Fund-raising through non-profits underdelivered, so staff carried cellphones on recruiting trips across the country. The final corpus has 4122 sessions from 4090 speakers. An analysis of phone coverage shows that the phonetically rich sentences substantially raised the number of distinct phones each session contained. The paper is a practical guide for anyone planning a large multi-speaker collection.

Key points

• The corpus belongs to four companies, Loquendo, Microsoft, Natural Speech Communication and Siemens, and 1000 speakers are allotted to each. Distribution goes via ELDA.
• The phonetically rich sentence pool had 4412 items, drawn from Harvard/Timit sentences, children's stories and purpose-written sentences with rare phones. Each speaker read 9 sentences, with each sentence capped at 10 occurrences in the corpus.
• After 3200 speakers, 800 undistributed sheets were regenerated to favour the least-used items. Prefilling transcriptions with expected text increased transcriber throughput by 30%.
• Non-profits recruited 1363 speakers and direct recruiting (paid $5 each) brought in 2760. The rarest phone, [Z], appeared 3,092 times across 2074 sessions.
• With the sentences, 4060 sessions contained at least 36 distinct phones, versus 3009 without them. For at least 39 phones, the count fell from 2937 to 151 without the sentences.

Interesting ideas

• I1. Making every prompt sheet unique, instead of printing copies of a few thousand, keeps uneven uptake from skewing item coverage.
• I2. A feedback loop counts which items are under-represented partway through collection and regenerates the not-yet-used material, ordered so that the rarest items are consumed first.
• I3. Transcribers began from an automatically generated expected reading, with numbers, money amounts and company names verbalized, which made correction much faster than typing from scratch.
• I4. Coverage was measured on what speakers actually said, with flagged mispronunciations removed, and was compared with and without the targeted material to show its specific contribution.
• I5. The authors note that limiting each session to at most one rare-phone sentence would have spread rare phones over more distinct speakers.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): Generate each self-play or evaluation probe as a distinct instance so that no small subset of items dominates the measured results.
• I2 (learn): Periodically count which UD relations or frame types are scarce in silver data and steer the next batch of sentences sent to witness parsers toward them.
• I3 (back): A small deterministic verbalizer of numbers and amounts could also prefill human or LLM-teacher corrections of generated text.
• I4 (learn): Report evaluation on actually parsed output and add an ablation that removes the targeted training subset, which acts as a negative control for its claimed effect.
• I5 (answer): When building school-test or fable QA sets, distribute rare constructions across many separate test items so that one hard item cannot hide or inflate the measured skill.