📄 Curvature aware gradient projection prototype meta learning framework improves low resource English dialect recognition

Summary

Recognising non-mainstream English dialects from speech is hard when only a handful of labelled recordings exist: deep models overfit, and the acoustic cues separating dialects are small compared with speaker-to-speaker variation. Chen and Ai propose a few-shot meta-learning framework that changes both the metric and the optimiser of a prototypical network. Speech is encoded by a convolutional front end followed by a BiLSTM into fixed-length embeddings. These embeddings are placed in a Riemannian (Poincaré) space, where class prototypes are compared with queries by geodesic distance rather than Euclidean distance. During training, the method estimates eigenvalues of the local Hessian of the loss and projects each gradient onto the low-curvature ("flat") directions. This suppresses noise-driven parameter drift on tiny support sets. Adaptation is learned entirely from source-domain episodes, so no target-dialect data, labelled or unlabelled, is needed. The paper reports 78.4% accuracy in 5-way 1-shot episodes and 87.6% average accuracy on unseen dialect variants. It presents this as a data-augmentation-free path for fine-grained speech classification.

Key points

• Results: 78.4% accuracy in 5-way 1-shot episodes and 87.6% mean accuracy on English dialect variants never seen during meta-training.
• Encoder: pre-emphasis, then four 1-D convolution layers, each with batch normalization and ReLU. Kernel size is stated as 3 × 3, stride is 1, and channel count grows layer by layer. A BiLSTM follows, and its forward and backward states are concatenated.
• Metric: geodesic distance on a Poincaré manifold replaces Euclidean distance between prototypes and queries. The motivation is that dialect feature manifolds are curved and that Euclidean prototypes become biased under that curvature.
• Optimiser: Hessian eigen-analysis is computed on the fly, and gradients are orthogonally projected into the flat-curvature subspace of the loss.
• Positioning against other approaches:
  • Unlike adapters and LoRA, the method needs no target-domain labels.
  • Unlike active learning, it needs no unlabelled target pool.
  • It is framed as more stable than MAML-style accent adaptation.
  • Unlike hyperbolic few-shot methods, which only change the distance function, it also shapes the optimisation.

Interesting ideas

• I1. Flat-subspace gradient projection. Each update is restricted to the directions where the loss surface is flat, so a few noisy examples cannot push the weights along sharp, overfitting directions. This is a general guard against small-data updates damaging a pretrained model.
• I2. Geodesic distance in a curved space. Prototypes are compared by distance in a curved (hyperbolic) space instead of a flat one, which suits classes whose structure is nested or hierarchical rather than evenly spread.
• I3. Class prototypes from one or a few labelled examples. Each class is represented by the mean embedding of its few examples, and the model learns how to adapt from simulated episodes rather than from target data. This makes it cheap to add new categories without retraining.
• I4. Separate evaluation on held-out variants. Accuracy on dialect variants never seen in training is reported apart from the standard episode accuracy, which tests real generalisation rather than memorisation of the training dialects.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): when gparse or the perceptron parser is refreshed on small silver batches agreed by witness parsers, projecting away high-curvature components could keep rare-construction fixes from eroding general parsing accuracy.
• I2 (world): concept embeddings derived from the WordNet/FrameNet ontology are naturally tree-shaped, so Poincaré-distance prototypes could give more faithful similarity judgments when placing new entities under ontology nodes.
• I3 (frames): rare VerbNet classes or speech-act types with only one or two annotated examples could be assigned by nearest prototype over tree-based features, and the nearest prototype itself provides the explanation for the decision.
• I4 (tree): parser evaluation could hold out whole English UD treebanks or converted dialect varieties and report them separately, exposing how well dialect conversion and parsing transfer.