📄 Measuring Syntactic Difference in British English

Summary

The paper asks whether regional syntactic variation inside British English can be detected automatically from an annotated corpus. It extends the corpus comparison approach of Nerbonne and Wiersma (2006). That approach used POS trigrams as a stand-in for syntax and checked differences with a permutation test. Sanders replaces the trigrams with Sampson's (2000) leaf-ancestor paths, where each leaf records its chain of nodes up to the root, so higher-level tree structure becomes visible. The data is the parsed speech section of ICE-GB. Speakers are split by birthplace into 11 regions: the nine English Government Office Regions plus Scotland and Wales. A Monte Carlo permutation test with Kessler's Recurrence metric compares the regions, after a two-step normalization for sentence length and path frequency. Differences appear only between the larger regions. The summed size of the two compared corpora seems to need about 250,000 words for paths and about 100,000 for trigrams. This shows that dialect syntax can be measured from treebanks, and it gives a rough sense of how much data such a measurement requires.

Key points

• Corpus: ICE-GB parsed speech, with 20 POS tags at the leaves and internal nodes labelled with a category and a function. Regions average 4,682 sentences and 44,726 words, with a mean sentence length of 9.55 words. London is by far the largest region, at 244,341 words.
• Each sentence of N words yields N leaf-ancestor paths, compared with N−2 trigrams. Because the counts are so close, the same normalization works for both representations.
• Significance (p < 0.05) was found mainly in comparisons involving London, Southeast England and Scotland. Trigrams flagged more region pairs than paths did.
• A control run compared two random samples from the same corpus and found them not significantly different, as expected.
• The Norwegian L2 corpora used in the earlier study were 221k and 84k words. The British regions were up to ten times smaller than these, yet some differences were still detectable.

Interesting ideas

• I1. Turning a tree into a bag of root-to-leaf label chains gives feature types that a frequency test can treat as atomic symbols, while still carrying structure above the word level.
• I2. Sampson's bracket marks record whether a leaf is the leftmost or rightmost child of its highest qualifying node. This keeps different trees from collapsing into the same set of paths.
• I3. A permutation test compares the observed difference score with scores from random reshufflings of the pooled data. It reports a significance verdict instead of a raw distance, which makes "these two sets differ" a testable claim.
• I4. Running the same test on two halves of one corpus serves as a negative control: the method must report no difference there.
• I5. Corpora could be grouped by the point at which a subdivision stops producing a significant difference, letting the data define the boundaries rather than prior labels.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tree): Root-to-leaf deprel/UPOS chains from UD trees could be compared across our treebanks, or between perceptron and gparse output, to spot systematic structural drift that LAS alone hides.
• I2 (tree): When path strings serve as keys for parser features or for checking UD dialect conversion, adding boundary marks would stop distinct attachments from looking identical.
• I3 (learn): Silver data from a new domain could be tested this way against the training treebank before mixing, flagging distribution shift with a p-value.
• I4 (learn): Any shift or difference detector in our evaluation should include a same-source split as a sanity check, so false alarms are caught early.
• I5 (learn): Silver-data sources or text genres could be merged or kept apart according to whether their parses are statistically distinguishable, rather than by assumed categories.