📄 The Syntactic Regularity of English Noun Phrases

Summary

Sampson (1987a) had argued that NPs in the LOB treebank show no clean split between frequent well-formed types and rare "deviant" ones. Type frequencies seemed to fall smoothly along a long tail, which he took as evidence that generative grammars cannot succeed. Taylor, Grover and Briscoe re-examine this using a superset of the same data. They take about 10,000 NPs from the LOB treebank, including coordinated NPs that Sampson had left out. They group the NPs into types and then semi-automatically merge types that differ only in ways the grammar treats identically. For one token per type, they check by hand whether the feature-based, unification-style Alvey ANLT grammar yields the semantically appropriate parse, and they weight each rule by how many tokens it covers. With small changes, the grammar handles nearly 97% of the NP tokens. A few dozen generalisations cover roughly 700 types, and most failures turn out to be gaps in the grammar rather than mysterious deviance. The main lesson is that the long tail reflects Sampson's flat, atomic-category analysis, not the data itself.

Key points

• Source data: the LOB treebank, a 50,000-word hand-parsed subset of the 1-million-word LOB corpus. The sample is the same 38,212-word portion Sampson used, without genres G and P.
• From 10,150 extracted NPs, 17 were dropped as mis-analysed and 119 as foreign phrases or formulae. That left 10,014 NPs, which fell into 2358 literal tag-sequence types.
• Merging types along ANLT-relevant generalisations (for example, ignoring the number of the head noun) reduced these to 707 types. Of those, 622 were parsed successfully.
• Token coverage reached 96.88%, leaving 312 NPs (3.12%) unanalysed. The authors describe about 700 types as reducible to between 36 and 54 grammatical generalisations.
• Failures included:
  • non-AP premodifiers such as measure phrases;
  • right-node raising of nouns;
  • unlike-category coordination inside AP modifiers;
  • premodifiers that select a particular postmodifier;
  • order effects tied to heaviness;
  • interjected material, which the authors consider largely a pragmatic matter.

Interesting ideas

• I1. The shape of a type/token distribution depends on the analysis scheme: flat trees with atomic labels inflate the long tail, while recursive, feature-based structure collapses it.
• I2. Rule generality is measured by counting each rule's uses in a representative parse and multiplying by that type's token frequency, which gives a corpus-weighted importance score.
• I3. Only one representative token per merged type is checked by hand, which makes exhaustive manual evaluation of a large sample affordable.
• I4. Each failure is classified as either a grammar oversight with a specific missing rule or a genuinely extra-grammatical phenomenon such as interjections.
• I5. Punctuation is treated as non-syntactic, except where it marks a real distinction such as restrictive versus non-restrictive postmodification.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tree): Before treating rare UD configurations in a treebank as noise, re-count them after normalising features the parser ignores, so that real gaps can be told apart from labelling artefacts.
• I2 (learn): Self-induced repair rules could be ranked by tokens covered rather than by types matched, so the rare idiosyncratic rules that need review stand out.
• I3 (learn): Evaluation and LLM-teacher checks could sample one sentence per structural signature instead of random sentences, covering the construction space at lower cost.
• I4 (tree): Parser error reports could carry the same two-way tag, so fixable coverage gaps are not lumped together with material that belongs to pragmatics.
• I5 (frames): Comma-marked non-restrictive modifiers could be flagged as background information rather than as identifying conditions on the referent.