📄 ProPOSEC: A Prosody and PoS Annotated Spoken English Corpus

Summary

The authors present ProPOSEC, a corpus resource that fills a gap in Aix-MARSEC. That dataset had rich prosodic tiers but no syntactic tags. ProPOSEC covers Section A (Commentary) of the Spoken English Corpus (SEC). For each word token it lines up three sources: LOB tags from SEC, prosodic and phonetic data from Aix-MARSEC, and canonical entries from the authors' ProPOSEL lexicon, which works with NLTK. The result is a pipe-delimited text file with twelve parallel fields that Perl or Python can query easily.

Building it required several merge steps:

• reconciling spellings by hand,
• re-joining enclitics,
• aligning sequences that differ in length because punctuation and pauses are placed differently,
• bridging the fine LOB tagset and the coarse C5 tagset through a one-to-many mapping,
• looking up lexicon values by word plus C5 tag.

The authors frame ProPOSEC as a testbed for finding correlations between tiers, such as their earlier finding linking phrase breaks to words with complex vowels. Such correlations could become features for phrase-break prediction in TTS, ASR and punctuation restoration.

Key points

• The twelve fields are: file number, word, LOB tag, C5 tag, two SAM-PA transcriptions (Aix and ProPOSEL), syllable count, lexical stress pattern, content/function flag, two DISC forms, and phoneme-level tonic stress marks from Aix-MARSEC.
• Lexical stress is coded as digit strings, with 1 for primary, 2 for secondary and 0 for unstressed stress (for example, 201 for disappear).
• The build has six steps:
  • hand-fixing orthography (e.g. TWA becomes tee double u ay);
  • rebuilding enclitics (will_MD + not_XNOT becomes won't_MD+XNOT);
  • aligning tags with Aix despite length mismatches;
  • a one-to-many C5/LOB mapping, with a patch to remove redundant tags;
  • lookup in a Python dictionary keyed by word + C5.
• After automatic tagging, only 15 of 629 tokens in Section A08 were left untagged, and these were fixed by hand.
• An earlier study found that gold phrase breaks correlate significantly with diphthong- or triphthong-bearing words. The authors also show that a naive per-token WEKA arff file ignores context and is therefore inadequate for break prediction.

Interesting ideas

• I1. Look up lexicon entries by a composite key of surface word plus coarse PoS tag, so homographs with different pronunciations or properties get the right entry.
• I2. Bridge tagsets of different granularity with an explicit many-to-one table plus a separate fix-up for cases where one coarse tag splits into several fine ones (LOB IN maps to C5 PRF or PRP).
• I3. Keep the corpus-specific (connected-speech) form and the canonical dictionary form in adjacent fields, so the divergence between them becomes data that can be studied.
• I4. Automate the bulk of the alignment, then measure and hand-fix the small residue, reporting its size as a quality indicator.
• I5. A flat per-token feature dump hides the context that actually drives the target label, so sliding windows or derived summary features are needed.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tokens): key en's lemma and morphology tables on (form, UPOS) so the tagger's decision selects the entry, e.g. noun versus verb readings of houses.
• I2 (tree): UD dialect and treebank conversion could store such asymmetric mappings explicitly and log every ambiguous direction instead of silently picking one.
• I3 (learn): silver data could keep each witness parser's analysis in parallel tiers, so disagreements are analysable features rather than discarded noise.
• I4 (learn): self-induced repair rules could report the count of unresolved tokens per document as an explicit metric and route only that residue to an LLM teacher tier.
• I5 (learn): evaluation with negative controls could include a context-free per-token baseline to show that a model's gains really come from structure and not from token identity.