🗣️ Language and Linguistics

A survey · about 17 minutes to read · every paper named here has its own page in this library

Linguistics is the scientific study of human language: how sounds and signs combine into words, how words combine into sentences, how sentences carry meaning, and how people use all of this to inform, persuade, ask and apologise. This shelf gathers the parts of that science that meet the written record and the computer. It holds grammars and treebanks, parsers and morphological analysers, lexicons of senses and frames, theories of discourse and politeness, and a large body of work on particular languages, from English and French to Arabic, Hebrew, Aramaic, Latin and Ancient Greek.

The shelf is deliberately broad. Some works here are classics of theoretical linguistics and the philosophy of language, written as books for human readers. Others are resource papers that describe an annotated corpus, a shared task or a toolkit, and many are experimental papers from computational linguistics that measure how well a program can recover structure that linguists describe. What ties them together is the object of study: language itself, its structure and its use, rather than a particular application.

The papers here span two centuries, from nineteenth-century grammars and editions to the latest work on large language models, but the bulk of the collection is recent. Read as a whole, the shelf tells a story that runs from hand-written grammars through annotated corpora and statistical models to neural networks, while the basic questions — what is a word, what is a sentence, what does it mean, and what is the speaker doing with it — have stayed remarkably stable.

What it is

Linguistics divides its subject into levels. Phonetics and phonology study sounds and sound systems; morphology studies the internal structure of words; syntax studies how words form phrases and sentences; semantics studies literal meaning; pragmatics studies meaning in context; and discourse studies how sentences connect into texts and conversations. Alongside these levels run fields defined by perspective: historical linguistics, typology, sociolinguistics, psycholinguistics, and the description of individual languages.

This shelf is strongest on the levels from morphology to discourse, and on their computational treatment. It covers syntax and grammar formalisms, treebanks and dependency parsing, morphology and tokenization and segmentation, lexical semantics and word sense disambiguation, semantic role labeling and semantic parsing, discourse analysis, coreference resolution, speech act theory and implicature. It also covers language-specific work, above all on Semitic languages and Hebrew and Aramaic, with smaller collections on Latin, Ancient Greek and Akkadian and cuneiform.

The boundary with natural language processing is porous, and many papers stand on both shelves. A rough rule: a paper belongs here when it is about a linguistic structure or a language (a treebank, a theory of coherence, a morphological analyser for Arabic), and on the NLP shelf when it is mainly about an application or a learning method. Machine translation is a second close neighbour, because translation is where two languages meet and where many evaluation methods for linguistic output were first developed.

History and milestones

Grammars, philology and structure

The oldest items on the shelf are grammars, dictionaries and text editions from the nineteenth and early twentieth centuries. They belong to the tradition of philology, which studied languages through their written texts, especially classical and biblical languages, and which produced comparative grammar and the historical study of language change. Twentieth-century structuralism treated a language as a system of contrasts, and from the late 1950s generative grammar recast syntax as the study of the rules that generate the sentences of a language.

Meaning and use

In the 1960s and 1970s philosophers and linguists turned to how language is used. Austin's idea that utterances are acts was systematised in Searle's Speech Acts (1969). Grice's theory of conversational implicature, Lakoff's study of hedges (1973) and Brown and Levinson's theory of politeness (1987) followed. Discourse theory took shape in the same years: Hobbs's algorithm for pronoun resolution (1978), Rhetorical Structure Theory (1988), and centering theory, circulated in the 1980s and published in its final form in 1995.

The corpus turn

Around 1990 the field changed direction. Large machine-readable corpora became available, and researchers began to measure language rather than only reason about it. Church and Hanks (1990) used mutual information to find word associations in newswire; Dunning (1993) gave a better statistic for rare events; Gale and Church (1993) aligned the sentences of parallel texts. The decisive resource was the Penn Treebank, released in the early 1990s: a large body of English text with part-of-speech tags and phrase-structure trees. In the lexicon, WordNet and FrameNet organised word meaning into networks of senses and frames.

Statistical parsing and annotation layers

The late 1990s and 2000s were the era of statistical models trained on annotated corpora. Collins's head-driven models and Charniak's parser set the standard for English constituency parsing; Klein and Manning showed how far an unlexicalised grammar could go. New annotation layers were stacked on the same newswire text: predicate-argument structure in PropBank, discourse relations in the Penn Discourse TreeBank, and noun predicates in NomBank. Dependency grammar returned to the centre with the CoNLL-X shared task of 2006, which compared parsers for many languages on a common format.

Neural networks and universal annotation

From around 2011, neural networks replaced hand-built features. Collobert and colleagues learned word vectors from raw text and reused them for tagging; Chen and Manning built a fast neural dependency parser; attention, introduced for translation by Bahdanau, Cho and Bengio, spread to every task. In parallel, Universal Dependencies set one annotation scheme for treebanks in dozens of languages. Since the late 2010s, pretrained language models have become the default tool, and much recent work on this shelf asks what such models know about grammar, meaning and use, and how to build them for languages beyond English.

Main problems and tasks

The shelf organises naturally around a set of recurring problems, each with its own data and evaluation.

• Segmentation and tagging. Splitting text into sentences, words and morphemes, then assigning parts of speech and morphological features. In English this is close to solved; in Arabic, Hebrew or Turkish, where a written word may contain a preposition, an article, a stem and a pronoun, it remains a central problem.
• Syntactic parsing. Assigning a phrase-structure tree or a dependency tree to a sentence. Parsing is evaluated against treebanks, by bracket F-score for constituents and by attachment score for dependencies.
• Semantic analysis. Identifying who did what to whom (semantic role labeling), which sense of a word is meant (word sense disambiguation), and how a whole sentence maps to a meaning representation such as Abstract Meaning Representation.
• Lexical knowledge. Building and evaluating dictionaries, sense inventories, collocation lists and multiword expressions, and measuring similarity between words.
• Reference and coherence. Resolving pronouns and other mentions to the entities they denote, identifying discourse relations between clauses, and modelling what makes a text coherent.
• Pragmatics and dialogue. Recognising speech acts and dialogue acts, politeness, hedging, irony and sarcasm, and the inferences listeners draw beyond what is said.
• Language description and documentation. Building resources for individual languages, especially historical and low-resource languages, and tools that let linguists annotate text, audio and video.
• Evaluation. Designing metrics and test sets that measure linguistic output fairly, a problem that runs through every other task and that has its own topic, evaluation and benchmarks.

Methods and schools

Rule-based and formal approaches

The oldest computational tradition writes grammars by hand. Formalisms such as Head-driven Phrase Structure Grammar (HPSG), Lexical-Functional Grammar, Tree-Adjoining Grammar and Combinatory Categorial Grammar (CCG) give precise accounts of syntax and its link to meaning. Large hand-built grammars, such as the English Resource Grammar, are discussed in work like Flickinger's paper on efficient typed grammars. Formal approaches remain strong where precision and explanation matter, and they underlie many treebanks: CCGbank, for instance, re-expressed the Penn Treebank in CCG, which made wide-coverage CCG parsing possible.

Statistical and feature-based learning

The statistical school treats linguistic analysis as prediction from annotated examples. Generative models (probabilistic grammars, hidden Markov models) gave way in the 2000s to discriminative ones: maximum-entropy models, conditional random fields, and the structured perceptron of Collins. Dependency parsing split into two families: transition-based parsers that build a tree word by word (Nivre's algorithm, MaltParser) and graph-based parsers that search for the best-scoring tree (McDonald's spanning-tree parser). Reranking (Charniak and Johnson) and self-training (McClosky and colleagues) squeezed further accuracy from the same data.

Distributional semantics

A separate line holds that a word's meaning can be read from the company it keeps. Count-based models built vectors from co-occurrence statistics; prediction-based models such as word2vec learned them. Baroni, Dinu and Kruszewski compared the two systematically and found the predictive vectors better on most benchmarks, while Levy and Goldberg showed that much of their apparent magic could be reproduced with sparse explicit vectors. SimLex-999 sharpened the debate by separating similarity from mere association.

Neural models

Neural methods replaced manual features with learned representations: recurrent networks (Kiperwasser and Goldberg's BiLSTM parser, stack LSTMs), biaffine attention, self-attentive encoders, and finally pretrained transformers such as SpanBERT. Toolkits like Stanza and UDPipe brought full neural pipelines to many languages.

The main debates

Several debates recur across the shelf. One is about lexicalisation: how much parsing accuracy comes from word-specific statistics and how much from structure. Another is about representation: phrase structure or dependencies, content words or function words as heads, one universal scheme or language-specific ones. A third concerns word senses: whether a fixed inventory of discrete senses is a useful fiction, as argued in Kilgarriff's provocatively titled «I don't believe in word senses». The most recent asks whether neural models learn linguistic structure or exploit shallow cues. McCoy, Pavlick and Linzen and Jia and Liang built test sets that expose such shortcuts, and Manning's essay on tagging argued that the last errors require linguistics, not just more data.

The languages on this shelf

Much of the shelf is organised by language rather than by method, and the languages differ in what makes them hard.

English and the other major European languages

English is the reference point of the field: its first large treebank, its sense network and most of its benchmarks were built for it, and reference works such as The Cambridge Grammar of the English Language still anchor descriptive claims. Work on English here ranges from converting constituent trees into dependencies to lexicons of discourse connectives and irony in social media. French appears through resources such as CamemBERT, a pretrained model built for French, and comparative studies such as concession in English and German. Mova's paper cards on English stand on their own English topic.

Semitic languages

The Semitic family is the largest language-specific collection here. Its words are built from consonantal roots placed into vowel patterns, and its writing systems usually leave out short vowels, so one spelling may correspond to several different words. Arabic work concentrates on segmentation, tagging and disambiguation, diacritisation and lemmatisation and the gap between Modern Standard Arabic and the spoken dialects. Hebrew work spans Biblical, Rabbinic and Modern Hebrew, from joint morphological and syntactic disambiguation to lexical semantics of the Bible, as in a study of two verbs for boiling in Biblical Hebrew and the Septuagint. Aramaic, Ugaritic and Akkadian bring the oldest written records of the family, with questions of genealogical subgrouping and the vocabulary of Ugaritic texts. The topics Semitic languages and Hebrew and Aramaic survey them in depth, and Mova's cards have shelves for Biblical Hebrew, Hebrew, Aramaic and the Semitic family.

Classical and ancient languages

Latin, Ancient Greek and the cuneiform languages share a set of problems: a closed and often damaged corpus, rich inflection, and no native speakers to consult. Work here includes finite-state analysis of Ancient Greek and Ancient Hebrew stress, sentence alignment between Latin and Greek, restoring Latin legal manuscripts, an Akkadian treebank of Neo-Assyrian royal inscriptions, filling gaps in broken Akkadian tablets and the EvaCun shared task on Akkadian and Sumerian lemmatisation. The topics Latin, Ancient Greek and Akkadian and cuneiform collect them.

Datasets, resources and benchmarks

The field's progress has been driven by shared resources. The most important ones represented here are:

Shared tasks have played a similar role: CoNLL-X and the CoNLL 2017 task for multilingual parsing, CoNLL-2005 and CoNLL-2009 for semantic roles, the SPMRL 2013 task for morphologically rich languages, the DISRPT tasks for discourse segmentation, and the yearly WMT findings for translation. Annotation tools such as ELAN and rstWeb support the human work behind all of these.

Landmark papers in this library

BLEU (2002)

Papineni and colleagues proposed an automatic translation score built from clipped n-gram precision against human references, with a penalty for output that is too short. It is the most cited work on the shelf because it made rapid, repeatable evaluation possible. Its weaknesses at sentence level led to METEOR, TER, chrF and the learned metric COMET.

Speech Acts (1969)

Searle's book treated speaking as rule-governed action and analysed the conditions under which a promise, a request or an assertion succeeds. Its notion of devices that signal illocutionary force became the basis for later work on dialogue acts, such as the Switchboard dialogue act model of Stolcke and colleagues.

Politeness (1987)

Brown and Levinson explained polite language as the management of "face", a speaker's wish to be approved of and to be left unimpeded. Comparing English, Tzeltal and Tamil, they argued that the strategies are universal. The book remains the standard reference in pragmatics and in computational studies of politeness, including control of politeness in neural translation.

Rhetorical Structure Theory (1988)

Mann and Thompson analysed texts as trees of spans joined by relations such as Evidence, Concession and Elaboration, most of them pairing a central nucleus with a supporting satellite. RST became one of the main frameworks for discourse parsing and led to the RST Discourse Treebank and to sentence-level discourse parsers.

The Penn Treebank (1993)

Marcus, Santorini and Marcinkiewicz described how millions of words of English were tagged and bracketed by correcting automatic output rather than annotating from scratch. Almost every later parser for English was trained and tested on this resource, and it was extended with predicate-argument structure, PropBank and the Penn Discourse TreeBank.

WordNet (1994) and FrameNet (1998)

WordNet grouped English words into synonym sets connected by hyponymy, meronymy and other relations, and became the default sense inventory for English. FrameNet, led by Charles Fillmore, described words by the situations they evoke and the roles in them. FrameNet made possible the first statistical semantic role labeler of Gildea and Jurafsky, and WordNet became the base for affective resources such as WordNet-Affect and SentiWordNet.

Centering (1995)

Grosz, Joshi and Weinstein modelled local coherence through the entities each utterance looks back to and looks forward to, and through rules linking pronouns to the most salient entity. Centering shaped pronoun resolution and coherence modelling for two decades and was tested on many languages.

Head-driven and unlexicalised parsing (2000–2003)

Collins generated trees outward from their heads with probabilities conditioned on head words, distance and subcategorisation; Charniak pushed accuracy higher with a smoothing scheme inspired by log-linear models. Klein and Manning then showed that careful structural splits of grammar categories, with no word statistics at all, could rival early lexicalised parsers — a result later automated by Petrov and colleagues.

PropBank (2005)

Palmer, Gildea and Kingsbury labelled the arguments of every verb in the Penn Treebank newswire with numbered roles defined per verb sense. Semantic role labelers have been trained on this corpus ever since, and its role sets were later reused as the predicates of AMR.

Stanford typed dependencies (2006)

De Marneffe, MacCartney and Manning converted phrase-structure trees into labelled grammatical relations that favour content words as heads. The scheme, refined in a 2008 paper and generalised in Universal Stanford Dependencies, became the direct ancestor of Universal Dependencies.

Neural NLP from scratch (2011)

Collobert and colleagues trained one neural architecture for tagging, chunking, named entities and semantic roles, using word vectors learned from unlabelled text instead of engineered features. With Turian, Ratinov and Bengio's comparison of word representations, it set the stage for embeddings throughout the field.

Abstract Meaning Representation (2013)

Banarescu and colleagues proposed a single graph for the meaning of a whole sentence, abstracting away from syntax so that paraphrases receive the same graph. AMR created a new task, semantic parsing into graphs, and a new kind of semantic treebank.

Stanford CoreNLP (2014)

Manning and colleagues described a deliberately simple pipeline that chains tokenisation, tagging, parsing, named entities and coreference. Its wide adoption made linguistic analysis an ordinary tool for researchers outside linguistics, a role later taken up by Stanza and AllenNLP.

Universal Dependencies (2016)

Nivre and colleagues defined one inventory of part-of-speech tags, morphological features and dependency relations for all languages, with function words attached to the content words they serve. UD grew into the largest multilingual collection of treebanks; its later state is described in Universal Dependencies v2 and in the 2021 overview, and it has its own topic, Universal Dependencies.

People

• Nizar Habash (New York University Abu Dhabi) is the most prolific author on this shelf. His work on Arabic runs from joint tokenisation, tagging and disambiguation through the MADAMIRA analyser to the open CAMeL Tools, and includes a conventional orthography for dialectal Arabic.
• Owen Rambow (Stony Brook University) co-authored much of the same Arabic morphology work, including morphological tagging, diacritisation and lemmatisation.
• Massimo Poesio (Queen Mary University of London) has shaped anaphora annotation and resolution, from the ARRAU corpus to bridging references and the BART toolkit.
• Christopher Manning (Stanford University) is among the most cited authors here, with work on unlexicalised parsing, typed dependencies, neural dependency parsing and the CoreNLP and Stanza toolkits.
• Joakim Nivre (Uppsala University) introduced efficient transition-based dependency parsing, co-organised the CoNLL 2008 shared task, and led Universal Dependencies.
• Marie-Catherine de Marneffe (UCLouvain) designed the Stanford dependency scheme and co-founded Universal Dependencies.
• Daniel Zeman (Charles University) co-maintains Universal Dependencies and organised the CoNLL 2017 parsing task.
• Reut Tsarfaty (Bar-Ilan University) works on parsing morphologically rich languages, especially Hebrew, from joint morphological and syntactic disambiguation to the SPMRL shared task.
• Avi Shmidman (Bar-Ilan University) builds tools for Hebrew, including the Nakdan diacritiser, DictaBERT and DictaLM 2.0.
• Martha Palmer led PropBank and its precursor From TreeBank to PropBank, and took part in AMR and the OntoNotes coreference tasks.
• Bonnie Webber (University of Edinburgh) and Aravind Joshi developed the Penn Discourse Treebank and its theory of anaphora and discourse structure; Joshi is also a co-author of centering.
• Michael Strube (Heidelberg Institute for Theoretical Studies) works on coreference, from using WordNet and Wikipedia as knowledge to scoring coreference partitions.
• Vincent Ng improved learning-based coreference, surveyed fifteen years of the task and studied hard Winograd-style pronouns.
• Amir Zeldes and Nathan Schneider (both Georgetown University) work on discourse annotation and lexical semantics respectively; Zeldes co-created CorefUD, and Schneider co-authored AMR and frame-semantic parsing.
• Dan Klein (University of California, Berkeley) contributed unlexicalised and latent-annotation parsing and self-attentive constituency parsing.
• Manfred Stede (University of Potsdam) and Vera Demberg (Saarland University) study discourse connectives and relations, from the DiMLex lexicon to implicit relations.

Institutions and places

The authors on this shelf cluster in a few kinds of places. Large American research universities — Carnegie Mellon, Johns Hopkins, Stanford, New York University, the University of Washington and Georgetown — are prominent, alongside industrial laboratories such as Google, Amazon and the Allen Institute for Artificial Intelligence. In Europe, the University of Edinburgh, Charles University in Prague (home of the Prague Dependency Treebank tradition and much of the Universal Dependencies infrastructure), Saarland University, Utrecht University and the French CNRS stand out.

A distinctive feature of this shelf is the weight of the Middle East and the Gulf, which reflects its strength in Semitic languages. New York University Abu Dhabi and the Mohamed bin Zayed University of Artificial Intelligence in the United Arab Emirates, Hamad bin Khalifa University in Qatar, and Bar-Ilan University in Israel are among the leading homes of authors here, working respectively on Arabic and on Hebrew.

The venues show the field's dependence on shared resources. The Language Resources and Evaluation Conference (LREC) carries more papers on this shelf than any other venue, followed by the journal Computational Linguistics, Glossa, Transactions of the ACL and the annual ACL meetings.

Open questions

• What do language models know about grammar? Pretrained models parse and tag well, but it is still unclear how far they represent syntax, and whether they generalise as humans do or rely on shallow heuristics. This connects to interpretability and psycholinguistics.
• Morphologically rich and low-resource languages. Most tools still work best for English. Arabic dialects, Hebrew, Amharic and many others have rich morphology, little annotated data and often non-standard spelling, and historical languages add damaged and undeciphered texts.
• Meaning beyond the sentence. Coreference, discourse relations and implicature still lag behind syntax. Implicit discourse relations, bridging anaphora and inherent disagreement among annotators all resist simple solutions.
• Representations and theory. Debates continue over the right unit of meaning (senses, frames, graphs), the right syntactic representation, and whether one universal scheme can serve all languages fairly.
• Evaluation. Automatic metrics often reward surface overlap. Designing tests that track human judgement and expose bias remains a central concern.
• Pragmatics in machines. Politeness, irony and indirect requests depend on social context; modelling them, as in work on pragmatic reasoning in language games, remains an open frontier.

Neighbouring topics

Linguistics shares most of its papers with natural language processing, which takes linguistic analysis as a component of downstream tasks like question answering, information extraction and summarization. The second-closest shelf is machine translation, together with its topic of the same name, machine translation: alignment, parallel corpora and translation metrics are linguistic problems as much as engineering ones. Smaller bridges lead to Artificial Intelligence, to history and humanities through philology and ancient languages, to law through legal language and legal AI, and to education through grammatical error correction and learner language.

Within the shelf, the largest topics are the Semitic languages and coreference, followed by Hebrew and Aramaic, translation, sentiment and emotion and discourse. Other topics connect linguistics to multilingual NLP, dialogue systems, speech, reasoning, named entity recognition and vision and language.

Key works in this library

• Speech Acts (Searle) — the classic theory of illocutionary acts.
• Hedges (Lakoff) — fuzzy category membership and the words that adjust it.
• Politeness: Some Universals in Language Usage — face and politeness strategies across cultures.
• Rhetorical Structure Theory — texts as trees of nucleus–satellite relations.
• Word association norms (Church and Hanks) — corpus statistics for word associations.
• The Penn Treebank (Marcus et al.) — the founding English treebank.
• Sentence alignment (Gale and Church) — length-based alignment of parallel texts.
• WordNet: A Lexical Database for English — the standard English sense network.
• Centering theory (Grosz, Joshi and Weinstein) — attention and pronouns in local coherence.
• The Berkeley FrameNet Project — frame semantics as a lexical resource.
• A Maximum-Entropy-Inspired Parser — a landmark statistical constituency parser.
• The Cambridge Grammar of the English Language — a modern reference grammar of English.
• Multiword expressions (Sag et al.) — a classification of idioms and fixed phrases.
• Automatic Labeling of Semantic Roles — the first statistical semantic role labeler.
• BLEU — the most widely used translation metric.
• Head-driven statistical parsing (Collins) — lexicalised generative parsing.
• Accurate Unlexicalized Parsing — structure over word statistics.
• The Proposition Bank — verb-specific semantic roles over the treebank.
• CoNLL-X shared task — the first large multilingual parsing comparison.
• The Penn Discourse TreeBank 2.0 — connective-based discourse relations.
• Information structure in discourse — the Question Under Discussion model of pragmatics.
• MADAMIRA — morphological analysis and disambiguation for Arabic.
• Farasa — fast Arabic word segmentation.
• Universal Dependencies v1 — one annotation scheme for many languages.
• Deep biaffine attention (Dozat and Manning) — a simple, strong neural graph-based parser.