📄 The Frequency Distribution of English Syntagms

Summary

Beebe asks whether the frequency of syntactic constructions, rather than vocabulary, can tell apart subsets of English such as genres. The practical motive was remedial English teaching: courses for a given genre should stress the structures that genre uses most. Sentences are hand-parsed with a simple dependency grammar that non-programmers can apply. Each tree is written as a linear string using a notation adapted from Wiswesser's chemical line formulas. A program turns each string into node level numbers, cuts out every subtree (called a "syntagm"), and stores them in a matrix grouped by head category, with counts, depth, length and type/token ratios. Chi-square tests then check whether batches within a genre are homogeneous and whether genres differ. Batches of newspaper editorials were mostly homogeneous. Higher School Certificate essays were also homogeneous, but they differed clearly from editorials. Per-sentence counts of prepositional-phrase syntagms fit a Poisson distribution well. The author presents this as a pilot study that shows syntactic profiles can serve as a genre marker.

Key points

• Notation: the "Beebe line notation" walks each sentence group (subject, verb, object) along its leftmost path first. It writes a dash for each step back to a branching point and puts boundary markers between groups.
• Category codes: letters stand for categories (e.g., H = prepositional phrase, P = preposition, N = noun, T = determiner, Q = qualifier). Numeric subcodes refine them, such as L1–L5 for clause types and Q1–Q3 for adjective degree.
• Homogeneity within editorials: ten editorial batches gave chi-square 101.88 at df 90 (not significant). Batches 1–10 against 11–20 gave 9.76 at df 10 (not significant).
• Genre contrast: four HSC essay batches gave 27.33 at df 21 (not significant). Essays against an equal number of editorial sentences gave 26.31 at df 10, significant at the .01 level.
• Poisson fit: the corpus had 415 sentences from twenty editorials. HP-N had a mean of about 0.636 per sentence, and its chi-square against Poisson was 1.959 at df 3. All five tested prepositional syntagms fit Poisson at the 10% level, and no syntagm appeared more than about ten times in one sentence.

Interesting ideas

• I1. Encoding a tree as a bracket-free string plus a parallel array of depth levels lets every subtree be extracted with one linear scan: start at a node and stop when the level falls below the start node's level.
• I2. A genre is characterised by the relative frequencies of whole subtree shapes, and chi-square homogeneity is checked across batches before any genre comparison is trusted.
• I3. Each sentence's count of a construction is compared with a Poisson expectation, so deviations reveal clustering or avoidance beyond chance.
• I4. Optional arrowheads record where word order departs from a stated default order, and counting them tests how valid that default actually is.
• I5. Syntagm type/token ratios are tracked as more sentences are added, which shows when the inventory of constructions saturates.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tree): The same depth-array trick could quickly enumerate all UD subtrees for indexing, counting or pattern matching without recursive traversal.
• I2 (learn): Subtree-shape profiles of silver-parsed text could flag domain shift or a misbehaving parser, using within-genre batch homogeneity as the negative control.
• I3 (learn): Applying this to parser outputs could show when a model over- or under-produces constructions such as stacked PPs, compared with gold treebank rates.
• I4 (back): Generation could use such measured departure rates to decide when non-canonical order, such as fronted adverbials, is natural for a given genre.
• I5 (world): Saturation curves could estimate how much text is needed before frame or construction inventories for a domain stop growing.