📄 Automatically Detecting Reduced-formed English Pronunciations by Using Deep Learning

Summary

The paper addresses automatic detection of reduced-form pronunciations in the speech of Chinese learners of English. It targets two phenomena: reduction, where a single word is simplified, and liaison, where adjacent words are linked. The authors argue that most CALL tools train only canonical pronunciations, and that a detector is needed before a production-training tool can give feedback. The data come from practice recordings in a commercial learning app. Professional annotators labelled the recordings in Praat.

The baseline is a Kaldi ASR that force-aligns each target word and then decodes it through a small network with two paths, one for the formal pronunciation and one for the reduced one. The proposed alternative is a 1D CNN classifier over MFCC frames. It covers reductions that leave no clear phonetic variant and show up only as lower energy or shorter duration. A variant of the CNN adds attention weights computed from the identity of the target word.

The CNN clearly beats the ASR baseline on reduction (F-1 0.690 → 0.757), and attention adds a little more (0.763). Liaison shows the same pattern with smaller gains.

Key points

• Data: about 8,570 reduction practice clips, rated by seven annotators, and 4,027 liaison clips, rated by three annotators. Table 1 lists 11,563 reduction labels (3,610 yes) and 4,027 liaison labels (1,334 yes).
• Annotation quality:
  • For reduction, a label was kept only if at least four raters agreed.
  • Kappa rose from 0.63 to 0.79 after raters were retrained between the two rating stages.
  • Liaison agreement was higher, at 0.74–0.82.
• ASR baseline: Kaldi 9-layer TDNN trained on roughly 2,500 hours of in-house read speech, with 9% WER on learner speech.
• CNN:
  • Input is 40-dimensional MFCCs extracted with librosa, padded to a fixed length (0.5 s for reduction).
  • Two 1D convolution blocks (128 and 256 filters) with batch normalization feed global max pooling, dropout and a sigmoid output.
  • The model was built in Keras/TensorFlow.
• Results:
  • The reduction model is evaluated by cross-validation repeated 5 times.
  • Reduction: F-1 0.690 / 0.757 / 0.763 and accuracy roughly 70.4% / 76.2% / 77.0% for ASR / CNN / CNN+ATTN.
  • Liaison: F-1 0.692 / 0.719 / 0.720.

Interesting ideas

• I1. An enumerated-variant decoder can only find alternatives it was told about, while a direct classifier also catches subtle cases that have no distinct variant, so "list the alternatives" approaches should be checked for this blind spot.
• I2. A one-hot item identity can be mapped through a learned linear layer and a softmax into per-dimension weights that rescale shared features, giving each item its own feature emphasis at very low parameter cost.
• I3. For pairwise decisions, the two sounds at the boundary (end of word one, start of word two) are fused into a single token that conditions the classifier, rather than modelling each word separately.
• I4. Items on which annotators fail to reach a quorum are dropped as too hard to label, rather than forced into a label, and annotators are retrained once measured agreement is low.
• I5. Repeating cross-validation several times and averaging makes small gains measurable despite random training noise; here that is how the attention gain of only +0.006 F-1 is reported.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tokens): Contraction and reduced-spelling handling ("gonna", "gotcha", "'re") in the tokenizer could keep the explicit variant table but add a fallback detector for nonstandard forms the table misses.
• I2 (tree): The gparse parser could rescale its shared encoder features with lemma- or relation-conditioned weights for troublesome function words, while keeping one model and inspectable per-word weight vectors.
• I3 (tree): Arc-scoring features could include a fused boundary token built from the two words' tags or lemmas, which helps attachment decisions for tight pairs such as phrasal verbs or compounds.
• I4 (learn): Silver data from witness parsers could keep only arcs that a quorum of parsers agree on, and the LLM-teacher prompts could be recalibrated whenever agreement on a probe set falls.
• I5 (learn): Small parser or frame-extractor improvements should be reported as averages over several seeds before they count as real.