🌐 Universal Dependencies
Universal Dependencies is a framework and an open collection of treebanks that annotates the grammar of many languages with one shared inventory of part-of-speech tags, morphological features and dependency relations. The shared inventory makes syntactic analyses comparable across languages.
What it is
Universal Dependencies (UD) has two parts. It is a set of annotation guidelines, and it is a growing body of treebanks (Nivre et al. 2016; Nivre et al. 2020). A treebank here is a corpus in which every sentence carries a syntactic analysis following those guidelines. How much of that analysis was annotated or checked by hand varies from treebank to treebank. A full account of the framework's design and theory appears in the 2021 Computational Linguistics article by de Marneffe, Manning, Nivre and Zeman (Marneffe et al. 2021).
The core layers of annotation for each word include:
- a lemma, the word's base form, such as "sit" for "sat";
- a universal part-of-speech tag (UPOS), such as NOUN, VERB, ADP or DET;
- a set of morphological features written as attribute–value pairs, such as Number=Sing or Tense=Past;
- a single head word plus a typed dependency relation linking the word to that head, such as nsubj, obj, obl or case.
Language-specific tags and subtypes of relations may be added. The core inventory itself stays the same for every language.
The key idea
The central mechanism is the dependency tree. Every word is attached to exactly one other word, or to an artificial root, and each attachment carries a label from the shared relation set:
T = { (h(i), r(i), i) : i = 1 … n }, h(i) ∈ {0, 1, …, n}, r(i) ∈ R
LAS = |{ i : ĥ(i) = h(i) and r̂(i) = r(i) }| / n
The symbols are as follows:
- n is the number of syntactic words in the sentence, and i indexes them. A syntactic word can differ from a surface token. Spanish "del", for example, is analysed as two words, "de" and "el". In CoNLL-U the original token is kept on an extra line whose ID is a range, such as 3-4.
- h(i) is the position of the head of word i. The value 0 stands for the root.
- r(i) is the relation label, drawn from the universal set R.
- T is the whole tree. It must be connected and acyclic, with exactly one word having h(i) = 0.
- In the second line, ĥ and r̂ are the head and label predicted by a parser.
- LAS, the labeled attachment score, is the share of words for which both are correct.
LAS was the main metric of the CoNLL shared tasks on UD parsing. Those tasks started from raw text, so predicted words need not match the gold words. LAS was therefore computed as an F1 score over aligned words (Zeman et al. 2017). The 2018 edition added two further metrics, both computed over content words only (Zeman et al. 2018):
- MLAS also checks the tag, a subset of the morphological features, and the attachment of function words.
- BLEX also checks lemmas.
A distinctive design choice is that content words act as heads. Function words such as prepositions, auxiliaries and articles attach to the content word they modify. This keeps the core predicate–argument skeleton similar across languages that express the same grammar with cases, particles or word order.
Treebanks are stored in the CoNLL-U format. Each word occupies one line with tab-separated fields: ID, FORM, LEMMA, UPOS, XPOS (a language-specific tag), FEATS, HEAD, DEPREL, DEPS (enhanced graph) and MISC. Comment lines carry sentence metadata. In the example below, the columns are aligned with spaces for readability; real files separate them with tabs.
# sent_id = ex-1
# text = The cat sat on the mat.
1 The the DET DT Definite=Def|PronType=Art 2 det _ _
2 cat cat NOUN NN Number=Sing 3 nsubj _ _
3 sat sit VERB VBD Mood=Ind|Tense=Past|VerbForm=Fin 0 root _ _
4 on on ADP IN _ 6 case _ _
5 the the DET DT Definite=Def|PronType=Art 6 det _ _
6 mat mat NOUN NN Number=Sing 3 obl _ SpaceAfter=No
7 . . PUNCT . _ 3 punct _ _
In this example "on" depends on "mat" as case, rather than heading the phrase. "mat" in turn attaches to the verb as an oblique dependent. The attribute SpaceAfter=No in the MISC field of "mat" states that no space follows that word. This allows the original text to be rebuilt from the annotation.
History and founding papers
Much of modern English treebanking grew from the Penn Treebank. It provided a large phrase-structure annotation of English text (Marcus et al. 1993) and later added predicate–argument information (Marcus et al. 1994). Many resources were layered on or extracted from it:
- PropBank added semantic roles (Kingsbury & Palmer 2002).
- CCGbank derived categorial-grammar derivations and dependency structures (Hockenmaier & Steedman 2007).
- The Penn Discourse Treebank added connective-based discourse relations over the same Wall Street Journal texts (Miltsakaki et al. 2004; Prasad et al. 2008).
Treebanks for other languages followed, each with its own conventions. Examples include:
- the Portuguese Floresta Sintá(c)tica, built from Constraint Grammar parser output corrected by hand (Afonso et al. 2002);
- the Arabic Treebank (Maamouri & Bies 2004);
- the Swedish Talbanken05, with both phrase-structure and dependency layers (Nivre et al. 2006);
- a French treebank converted to dependencies for statistical parsing (Candito et al. 2010);
- the Chinese Treebank, on which constituent parsers were also evaluated after their output was converted to dependencies (Zhang & Clark 2009).
This diversity made cross-lingual comparison and transfer difficult. A parser's apparent error could simply reflect a different annotation convention.
Several earlier efforts attempted harmonization, and UD drew on three of them (Nivre et al. 2016):
- the Stanford dependencies for English supplied the relation-style analysis;
- a universal part-of-speech tagset offered a coarse shared tag layer;
- the Interset approach mapped morphological tagsets onto a common feature space.
McDonald and colleagues combined such ideas into a jointly annotated set of treebanks for several languages, aimed at multilingual parsing (McDonald et al. 2013). The first official UD release merged these strands into one set of guidelines and one collection. The paper describing it appeared in 2016, after the release itself (Nivre et al. 2016). Version 2 revised the guidelines and expanded the collection considerably (Nivre et al. 2020).
How it is used today
UD treebanks are widely used to evaluate dependency parsers across many languages, notably in the CoNLL shared tasks (Zeman et al. 2017; Zeman et al. 2018).
The 2017 and 2018 shared tasks required systems to start from raw text. Systems had to segment sentences and words, tag, lemmatize and parse, all without gold preprocessing (Zeman et al. 2017; Zeman et al. 2018). For testing, the 2018 task used 82 treebanks covering 57 languages, taken from UD release 2.2. The 2017 edition had used 81 treebanks in 49 languages (Zeman et al. 2018). The 2018 submissions showed several trends (Zeman et al. 2018):
- graph-based parsers became more common than transition-based ones;
- pre-trained word embeddings were used widely;
- multilingual transfer was frequent for languages with little data.
Trainable toolkits operate directly on CoNLL-U files. UDPipe, for example, covers tokenization, morphological analysis, tagging and parsing (Straka et al. 2016), and it served as the 2018 baseline (Zeman et al. 2018). The shared label set makes it possible to train one multilingual network on many treebanks at once. Kondratyuk and Straka, building on multilingual BERT, parsed 75 languages with a single model (Kondratyuk & Straka 2019).
Beyond parsing, UD serves as a substrate for further layers of meaning:
- Enhanced dependencies add extra edges, for example for shared arguments and controlled subjects, which turns trees into graphs. Schuster and Manning proposed such a representation for English and argued that it suits language understanding tasks better (Schuster & Manning 2016). Enhanced graphs are now part of the version 2 guidelines, stored in the DEPS field (Nivre et al. 2020).
- Universal Decompositional Semantics attaches semantic property judgments to UD structures (White et al. 2016).
Linguists also use the collection for corpus-based typology. They compare word order, morphology and construction frequencies across languages annotated in the same way.
Open questions
The content-head principle is debated. Surface-Syntactic UD (SUD) proposes a scheme in which function words are heads, while remaining nearly convertible to and from standard UD (Gerdes et al. 2018). Which choice serves parsing, typology or downstream tasks best remains disputed.
Consistency is a second concern. Many treebanks were converted from older native schemes, such as those surveyed above. As a result, the same construction can still be annotated differently from one language, or one treebank, to another.
Low-resource languages are a persistent difficulty. In the 2018 shared task, nine treebanks counted as low-resource: some offered just a few dozen training sentences and others offered none (Zeman et al. 2018). Raw-text tokenization also limits end-to-end scores, especially for languages without spaces or with rich clitic systems.
Finally, UD remains a sentence-level syntactic scheme, even with its enhanced graphs (Schuster & Manning 2016). How it should connect to deeper semantic representations and to discourse-level annotation of the kind found in the Penn Discourse Treebank (Prasad et al. 2008) is still an active area of design.
Key papers
Related concepts
- Dependency parsing — Dependency parsing is the task of recovering a sentence's syntactic structure as directed, labelled links between its words. Each word is…
- Part-of-speech tagging — Part-of-speech tagging is the task of assigning each token in a text a grammatical category, such as noun, verb, adjective or determiner…
- Corpus linguistics — Corpus linguistics is the empirical study of language through large, principled collections of naturally occurring text or transcribed…
- Context-free grammar — A context-free grammar is a formal system of rewriting rules in which each rule replaces a single nonterminal symbol with a string of…
More papers
Topics: 🌐 Universal Dependencies · 🌳 Dependency parsing · 🗂️ Treebanks · 🌍 Multilingual & cross-lingual NLP
Shelves: 🗣️ Linguistics & languages
This page in Markdown → https://md.research.elamur.ai/free/en/concepts/universal-dependencies/