🌐 punct: punctuation
punct is the Universal Dependencies relation that attaches a punctuation token, such as a comma, full stop, bracket, quotation mark or dash, to the word heading the clause or phrase that the mark belongs to. Punctuation is always the dependent and almost never governs anything.
What it is
In Universal Dependencies (UD), every token of a sentence, including every punctuation mark, must hang somewhere in the dependency tree. The relation punct is how marks of punctuation get their place. UD builds on lexicalism: the syntactic word is the basic unit, and typed relations link words into a tree headed by content words (Marneffe et al. 2021; Nivre et al. 2016). Punctuation is split into separate tokens during tokenization and tagged PUNCT. It receives a dependency like any other token, although it contributes no predicate–argument content.
The guidelines give three attachment rules:
- A mark attaches to the head of the unit it delimits or ends. Sentence-final punctuation goes to the root.
- Paired marks such as brackets and quotes both attach to the head of the material they enclose.
- A comma separating conjuncts attaches to the conjunct that follows it.
- Where several heads would be possible, the highest one that keeps the tree non-crossing (projective) is chosen.
The dependent of punct is the punctuation token. Its head is usually a VERB or NOUN, sometimes an ADJ, PROPN or ADV.
punct is told apart from several neighbouring relations:
- A symbol acting as a word gets that word's function, not punct. The ampersand in Smith & Jones is cc, and a slash meaning "or" can also be cc.
- Emoticons are discourse.
- Material set off by colons or dashes as a separate clause is linked by parataxis, while the dash itself remains punct.
The universal inventory of 37 relation types lets languages add subtypes (Nivre et al. 2020). For punctuation, however, no widely shared subtype (punct:subtype) has become standard.
Examples
Arrows point from head to dependent.
- She left early. The full stop attaches to the root: left → punct → .
- We bought apples, pears and plums. The comma attaches to the following conjunct: pears → punct → ,
- The talk (on Tuesday) was long. Both brackets attach to the head of the enclosed phrase: Tuesday → punct → ( and Tuesday → punct → )
| Sentence | Head | Dependent |
|---|---|---|
| She left early. | left (VERB) | . |
| apples, pears | pears (NOUN) | , |
| (on Tuesday) | Tuesday (PROPN) | ( and ) |
Across languages
UD aims at parallel annotation across typologically different languages, and recoverability means the original tokens are preserved even when words are split (Nivre et al. 2016; Marneffe et al. 2021). punct is therefore defined the same way everywhere, but writing systems differ in what they supply:
- Spanish opens questions with ¿.
- Chinese and Japanese use full-width commas and enumeration marks.
- Greek uses a semicolon-like sign as its question mark.
- Texts in many historical languages were written without punctuation. Their treebanks contain only what modern editors added.
In the UD 2.18 treebanks of English and French, punct accounts for roughly one token in eight in each language. In both, attachment to a verb is by far the most frequent pattern, and attachment to a noun comes a distant second. The two languages then diverge: English attaches punctuation to adjectives more often than to proper nouns, and French shows the reverse order. Adverbs come last in both.
In treebanks and research
UD descends from the universal Stanford dependencies (Marneffe et al. 2014). In v2, a basic tree is mandatory for every treebank and an enhanced graph is optional (Nivre et al. 2020). The enhanced representation keeps the basic tree in full and adds edges between content words (Schuster & Manning 2016), so punct arcs carry over unchanged. Tutorials on UD principles and tools cover the validators that check such conventions (Zeman 2021). Studies of annotation consistency across UD corpora examine whether relations are applied uniformly (Marneffe et al. 2017). Punctuation attachment is a typical place for differences between treebanks.
In dependency parsing, punct is easy to label but its head is often ambiguous. Older evaluations frequently left punctuation out of the score. The CoNLL 2017 and 2018 shared tasks parsed raw text for many languages (Marneffe et al. 2021; Nivre et al. 2020) and scored all tokens. The standard metric is the labeled attachment score:
LAS = C / N
Here C is the number of tokens that receive both the correct head and the correct relation label, and N is the number of tokens scored. Including or excluding punct tokens in N changes the figure, so papers should state which convention they follow when comparing parsers on treebanks.
What punct joins: the most frequent head tag → dependent tag
The five most frequent in each language's treebanks, by tokens.
The numbers
| Pair | Tokens |
|---|---|
| English: VERB → PUNCT | 72,716 |
| English: NOUN → PUNCT | 31,384 |
| English: ADJ → PUNCT | 10,801 |
| English: PROPN → PUNCT | 8,499 |
| English: ADV → PUNCT | 5,333 |
| French: VERB → PUNCT | 71,082 |
| French: NOUN → PUNCT | 29,254 |
| French: PROPN → PUNCT | 7,291 |
| French: ADJ → PUNCT | 6,275 |
| French: ADV → PUNCT | 3,776 |
Key papers
Related concepts
- Universal Dependencies — Universal Dependencies is a framework and an open collection of treebanks that annotates the grammar of many languages with one shared…
- Dependency parsing — Dependency parsing is the task of recovering a sentence's syntactic structure as directed, labelled links between its words. Each word is…
More papers
Topics: 🌐 Universal Dependencies · 🌳 Dependency parsing
Shelves: 🗣️ Language and Linguistics
This page in Markdown → https://md.research.elamur.ai/free/en/concepts/ud-punct/