📄 Computer-Aided Grammatical Tagging of Spoken English

Summary

The paper outlines a planned system for grammatically tagging the London-Lund Corpus of spoken English, about 450,000 words. The corpus is transcribed orthographically and annotated with prosody: tone units, nuclei, stresses and pauses. Svartvik notes that people agreed tagged corpora were useful, but there was no consensus on the tag scheme or on how to produce it. His design works from surface signals that listeners plausibly rely on, namely written word forms, prosody and dictionary word classes.

The proposed pipeline has three steps:

• Word types get candidate classes from the Longman Dictionary of Contemporary English.
• Ambiguous tokens are then disambiguated.
• Phrases are built by ordered, cyclically applied parsing rules inside each tone unit.

Two design choices stand out. The tone unit serves as the unit of analysis, and the phrase search runs right to left, because English phrase heads tend to sit at the right edge. Since fully automatic tagging was not yet feasible, the process is partly interactive. This is a design proposal: it reports no accuracy figures. Its value lies in linking prosodic segmentation to syntactic chunking.

Key points

• The target resource is the London-Lund Corpus of spoken English, about 450,000 words. It is prosodically transcribed and partly available on magnetic tape and in print. The work belongs to the Survey of Spoken English (SSE).
• Word class tags for types come from a general-purpose dictionary, the Longman Dictionary of Contemporary English. Token disambiguation and phrase tagging follow.
• The tone unit is the basic analysis unit. Earlier Survey work (Quirk et al 1964) found tone units average 5.3 words and largely coincide with group, phrase and clause units.
• Phrase rules are applied in a fixed order and cyclically: verb phrases (VPH), adverb phrases (APH), adjective phrases (JPH), noun phrases (NPH), prepositional phrases (PPH).
• Unlike discourse models built for toy domains such as blocks worlds (e.g. Davey 1978), the system targets unrestricted speech. Svartvik cites Winograd's use of Halliday's systemic grammar as relevant.

Interesting ideas

• I1. Scanning for phrases from right to left fits head-final English phrases, and the author claims this avoids left-recursion trouble and a flood of predicted alternatives.
• I2. Using intonation-based units as the segmentation for grammar works because these units mostly match phrases or clauses, which gives natural chunk boundaries.
• I3. The prosodic nucleus tends to fall on the phrase head, so a non-syntactic signal can serve as evidence for where the head lies.
• I4. Candidate tags come from a general dictionary listing all possible word classes, and context then narrows them down. This keeps the lexicon explainable and separate from disambiguation.
• I5. A semi-manual, interactive mode handles cases the automatic rules cannot. This openly accepts partial automation while the rules mature.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tree): A right-to-left pre-chunking pass could propose nominal and verbal heads first, giving the perceptron parser strong head anchors before it attaches dependents.
• I2 (tokens): Punctuation, line breaks or dialogue turns could play the role of tone units, letting the tagger and chunker limit ambiguity resolution to short spans.
• I3 (learn): Independent cues such as emphasis markup or capitalised stress in transcripts could act as an extra witness when voting on silver head choices.
• I4 (tokens): Mova's lexicon could store all dictionary classes per type and log which contextual rule removed each one, making each tagging decision traceable.
• I5 (learn): Unresolved spans could be routed to an LLM teacher tier or a human, and those decisions harvested to induce new ordered repair rules.