🏷️ PUNCT: punctuation
PUNCT is the universal part-of-speech tag for punctuation in Universal Dependencies. It covers non-alphabetic characters, or short groups of them, such as full stops, commas, quotation marks and brackets, that mark off sentences, clauses and phrases in written text.
What it is
Punctuation marks sit between words but are not words in the usual grammatical sense. In Universal Dependencies (UD), tokenization splits them off as separate tokens, and each one receives the tag PUNCT. The tag belongs to the closed list of seventeen universal part-of-speech tags in UD version 2 (Nivre et al. 2020). That list was already present in the first release, which extended the earlier Google set (Nivre et al. 2016).
The guidelines describe punctuation by what it does. A punctuation mark delimits linguistic units in writing: it closes a sentence, sets off an apposition, or opens and closes a quotation. It has no lexical meaning of its own.
Borderline cases
The nearest neighbouring tag is SYM. Characters that stand for a word or a concept, such as $, %, §, +, = or an emoticon, are symbols. Characters that structure the text are punctuation.
The same character can fall on either side:
- A hyphen inside the range "3–5" reads as "to" and is usually tagged SYM.
- A dash setting off an aside is PUNCT.
- An ampersand that replaces "and" may be tagged as a conjunction.
The decision rests on function, not on the character's shape.
A full stop inside an abbreviation such as "etc." or "Dr." normally stays attached to the word and is not a separate PUNCT token.
Features and relations
Universal features rarely apply to PUNCT, since it does not inflect. Some treebanks add language-specific features:
- PunctType gives the kind of mark, with values such as Peri, Comm, Quot, Brck or Dash.
- PunctSide gives the position of a paired mark, Ini for opening and Fin for closing.
In the syntax, almost every PUNCT token attaches with the relation punct. Its head is usually the head of the clause or phrase the mark delimits, which in a simple sentence is the main predicate. The relation root occurs only rarely, in "sentences" made of punctuation alone, such as a lone "…" or a line of dashes. Because UD makes content words the heads and attaches function elements to them (Marneffe et al. 2021), punctuation is always a leaf and never has dependents.
Examples
In "Well, it rained." the comma and the full stop are both PUNCT, and both attach as punct to "rained".
In "She said: «Non.»" the colon and both guillemets are PUNCT. They attach to the verb or to the quoted word, depending on which unit they delimit.
The tokens split by UD rules for the Spanish "¿Vienes?" and the German "„Ja", sagte er." are shown below.
| Text | Tokens tagged PUNCT |
|---|---|
| ¿Vienes? | ¿ ? |
| „Ja", sagte er. | „ " , . |
Across languages
Writing systems differ in the inventory of marks, not in the category itself:
- Spanish opens questions and exclamations with inverted marks.
- Greek uses ";" as its question mark.
- Arabic and Persian write a reversed comma "،".
- Hindi closes sentences with the danda "।".
- Chinese and Japanese use the ideographic full stop "。" and the enumeration comma "、".
- French typography puts a space before ";", ":", "!" and "?", which matters for tokenizers but not for tagging.
Historical texts raise a different problem. In Latin and Biblical Hebrew corpora, punctuation is often added by modern editors or taken from cantillation and verse marks, so treebanks must decide what counts as PUNCT at all.
The share of tokens that are punctuation also varies with genre and with the conventions of each language. For example, the frequency of commas is far from constant across languages.
In treebanks and research
Petrov, Das and McDonald (2012) proposed a compact tagset of twelve coarse categories, and punctuation was one of them under the label "." (Petrov et al. 2012). They mapped the native tags of many treebanks onto this set to make part-of-speech tagging comparable across languages, and the UPOS inventory grew out of it (Petrov et al. 2012).
Tagset harmonization of this kind was also made modular by Interset, a universal feature inventory with decode and encode drivers for each tagset (Zeman 2008). Interset later fed the morphological layer of UD (Nivre et al. 2016).
In the English and French UD treebanks, the punct relation accounts for nearly all PUNCT tokens, and root appears only as a rare exception.
Punctuation matters for quality control. Consistency can be checked by comparing distributions of UPOS trigrams between treebanks of the same language (Aggarwal & Zeman 2020). Because PUNCT is so frequent, differences in how treebanks split or attach it affect such comparisons. Broader audits of annotation consistency in UD (Marneffe et al. 2017) and tutorials on the scheme's principles (Zeman 2021) treat punctuation attachment as part of the general guidelines.
In dependency parsing evaluation, punctuation is sometimes excluded from attachment scores. This convention dates from pre-UD shared tasks, so scores computed with and without punctuation are not directly comparable.
The relations PUNCT words take most often
The five most frequent in each language's treebanks, by tokens.
The numbers
| Pair | Tokens |
|---|---|
| English: punct | 144,343 |
| English: root | 118 |
| French: punct | 130,636 |
| French: root | 3 |
Key papers
Related concepts
- Universal Dependencies — Universal Dependencies is a framework and an open collection of treebanks that annotates the grammar of many languages with one shared…
- Part-of-speech tagging — Part-of-speech tagging is the task of assigning each token in a text a grammatical category, such as noun, verb, adjective or determiner…
- Morphology — Morphology is the branch of linguistics that studies the internal structure of words. It covers how words are built from meaningful units…
More papers
Topics: 🔤 Part-of-speech tagging · 🌐 Universal Dependencies
Shelves: 🗣️ Language and Linguistics
This page in Markdown → https://md.research.elamur.ai/free/en/concepts/upos-punct/