Concepts
30 concepts of linguistics and language technology, each explained on one page: what it is, the key idea, where it came from, how it is used and what is still open. Every claim links to the papers behind it.
BLEU score
BLEU score (Bilingual Evaluation Understudy) is an automatic metric for machine translation. It rates a candidate text by how many of its word n-grams also occur in one or more human reference translations, and it penalises candidates that are too short. The page sets out the key idea, where it came from, how it is used today and which questions are still open. Every claim links to the papers behind it, and the key papers follow, the most cited first. It leads on to Language model and Transformer and attention.
26 key papers🧸Child-directed speech
Child-directed speech is the speech register adults and older children use when talking to young children. It typically has higher and more variable pitch, slower tempo, shorter and simpler utterances, frequent repetition and a restricted, concrete vocabulary. In many communities it makes up a large share of the linguistic input to early language acquisition. The page sets out the key idea, where it came from, how it is used today and which questions are still open. Every claim links to the papers behind it, and the key papers follow, the most cited first.
27 key papers💡Cognitive linguistics
Cognitive linguistics is an approach to language that treats grammar and meaning as products of general human cognition, including categorisation, perception, bodily experience and imagination. It does not treat language as an autonomous formal module, and it identifies meaning closely with conceptualisation. The page sets out the key idea, where it came from, how it is used today and which questions are still open. Every claim links to the papers behind it, and the key papers follow, the most cited first. It leads on to Construction grammar and Psycholinguistics.
27 key papers🏗️Construction grammar
Construction grammar is a family of linguistic theories holding that knowledge of a language consists of constructions. Constructions are conventional pairings of form and meaning at every size and level of abstraction, from words and idioms to abstract clause patterns, and they are linked in a structured network. The page sets out the key idea, where it came from, how it is used today and which questions are still open. Every claim links to the papers behind it, and the key papers follow, the most cited first. It leads on to Cognitive linguistics and Context-free grammar.
30 key papers🌲Context-free grammar
A context-free grammar is a formal system of rewriting rules in which each rule replaces a single nonterminal symbol with a string of symbols, regardless of surrounding context. It defines a language as the set of terminal strings derivable from a designated start symbol. The page sets out the key idea, where it came from, how it is used today and which questions are still open. Every claim links to the papers behind it, and the key papers follow, the most cited first. It leads on to Dependency parsing and Construction grammar.
25 key papers🔗Coreference resolution
Coreference resolution is the task of finding the expressions in a text that refer to the same entity as one another, such as names, descriptive noun phrases and pronouns. It groups those expressions into clusters, one cluster per entity, so that the document's mentions form a partition. Event coreference and cross-document coreference are related but separate tasks. The page sets out the key idea, where it came from, how it is used today and which questions are still open. Every claim links to the papers behind it, and the key papers follow, the most cited first.
29 key papers📚Corpus linguistics
Corpus linguistics is the empirical study of language through large, principled collections of naturally occurring text or transcribed speech (corpora). These are searched and counted with software to describe frequency, co-occurrence and variation, and to test claims about usage against real data. The page sets out the key idea, where it came from, how it is used today and which questions are still open. Every claim links to the papers behind it, and the key papers follow, the most cited first. It leads on to Zipf's law and Distributional semantics.
23 key papers🌳Dependency parsing
Dependency parsing is the task of recovering a sentence's syntactic structure as directed, labelled links between its words. Each word is attached to exactly one governing word, its head, or to an artificial root, and the links contain no cycles, so that together they form a tree. The page sets out the key idea, where it came from, how it is used today and which questions are still open. Every claim links to the papers behind it, and the key papers follow, the most cited first. It leads on to Universal Dependencies and Context-free grammar.
29 key papers🧵Discourse analysis
Discourse analysis is the study of how stretches of language beyond the single sentence form coherent wholes. It covers the relations that link clauses and sentences (cause, contrast, elaboration), the structures those relations build, and the computational methods that recover them from text. The page sets out the key idea, where it came from, how it is used today and which questions are still open. Every claim links to the papers behind it, and the key papers follow, the most cited first. It leads on to Coreference resolution and Implicature.
30 key papers🌫️Distributional semantics
Distributional semantics is the study and modelling of meaning through the statistics of linguistic context. Words, phrases or sentences are represented by the contexts they occur in, on the assumption that expressions sharing comparable contexts are likely to be close in meaning. The page sets out the key idea, where it came from, how it is used today and which questions are still open. Every claim links to the papers behind it, and the key papers follow, the most cited first. It leads on to Word embeddings and Semantic similarity.
28 key papers😉Implicature
Implicature is the part of what a speaker communicates that goes beyond the literal content of the words. Hearers infer it by assuming the speaker is cooperative, as when "some students passed" is taken to mean that not all did. Unlike entailment, conversational implicature can be cancelled. The page sets out the key idea, where it came from, how it is used today and which questions are still open. Every claim links to the papers behind it, and the key papers follow, the most cited first.
27 key papers🧠Knowledge representation and reasoning
Knowledge representation and reasoning is the area of artificial intelligence that studies how facts, concepts and rules about a domain can be encoded in a form a machine can store, and how new conclusions can be derived from that encoding by sound or plausible inference. The page sets out the key idea, where it came from, how it is used today and which questions are still open. Every claim links to the papers behind it, and the key papers follow, the most cited first. It leads on to Ontology engineering and Semantic parsing.
30 key papers🗣️Language model
A language model is a probability distribution over sequences of tokens (words, subwords or characters). It is usually factored into next-token predictions, so it can score how likely a text is and generate new text one token at a time. The page sets out the key idea, where it came from, how it is used today and which questions are still open. Every claim links to the papers behind it, and the key papers follow, the most cited first. It leads on to Transformer and attention and Scaling laws.
30 key papers📖Lexical semantics
Lexical semantics is the study of word meaning. It asks what words and multiword expressions denote, how a word's senses are organized, how words relate to one another through synonymy, hypernymy, antonymy or meronymy, and how meaning shifts with context and over time. The page sets out the key idea, where it came from, how it is used today and which questions are still open. Every claim links to the papers behind it, and the key papers follow, the most cited first. It leads on to Word sense disambiguation and Distributional semantics.
30 key papers📒Lexicography
Lexicography is the discipline of compiling, describing and designing dictionaries and related lexical resources. It covers both the practical craft of writing entries and the theoretical study of how word knowledge should be selected, structured and presented to human users or to machines. The page sets out the key idea, where it came from, how it is used today and which questions are still open. Every claim links to the papers behind it, and the key papers follow, the most cited first. It leads on to Lexical semantics and Corpus linguistics.
30 key papers🗜️Minimum description length
Minimum description length (MDL) is a model-selection principle rooted in information theory. It says the best explanation of a dataset is the hypothesis that, together with the data encoded with its help, yields the shortest total code. This balances goodness of fit against model complexity. The page sets out the key idea, where it came from, how it is used today and which questions are still open. Every claim links to the papers behind it, and the key papers follow, the most cited first.
9 key papers🏷️Named entity recognition
Named entity recognition locates stretches of text that refer to specific real-world things, such as people, organizations, places, products or dates, and gives each one a category drawn from a fixed or open inventory of entity types. The page sets out the key idea, where it came from, how it is used today and which questions are still open. Every claim links to the papers behind it, and the key papers follow, the most cited first. It leads on to Coreference resolution and Part-of-speech tagging.
30 key papers🕸️Ontology engineering
Ontology engineering is the discipline that studies how to design, build, evaluate, maintain and reuse ontologies. An ontology here is an explicit, formal model of a domain's concepts, relations and constraints that both people and software can interpret and reason over. The page sets out the key idea, where it came from, how it is used today and which questions are still open. Every claim links to the papers behind it, and the key papers follow, the most cited first. It leads on to Knowledge representation and reasoning and Lexicography.
25 key papers🔤Part-of-speech tagging
Part-of-speech tagging is the task of assigning each token in a text a grammatical category, such as noun, verb, adjective or determiner, from a fixed tagset. The tag is chosen from the word itself and its surrounding context, which resolves words that can belong to several categories. The page sets out the key idea, where it came from, how it is used today and which questions are still open. Every claim links to the papers behind it, and the key papers follow, the most cited first. It leads on to Universal Dependencies and Dependency parsing.
30 key papers🧩Psycholinguistics
Psycholinguistics is the study of the mental processes by which people produce, comprehend and acquire language. It combines experimental methods, such as reading times, eye tracking, priming and brain recordings, with formal models that predict where and why language processing becomes difficult. The page sets out the key idea, where it came from, how it is used today and which questions are still open. Every claim links to the papers behind it, and the key papers follow, the most cited first. It leads on to Child-directed speech and Cognitive linguistics.
18 key papers📈Scaling laws
Scaling laws are empirical relationships, usually power laws, that describe how a neural network's performance, often its test loss, changes predictably as model size, training data or compute grows. They let researchers forecast large-model behaviour from small experiments and plan how to split a budget. The page sets out the key idea, where it came from, how it is used today and which questions are still open. Every claim links to the papers behind it, and the key papers follow, the most cited first. It leads on to Language model and Transformer and attention.
30 key papers🗺️Semantic parsing
Semantic parsing is the task of mapping a natural-language utterance to a formal, machine-interpretable representation of its meaning, such as a logical form, a database query, a meaning graph or a frame structure, which a computer can execute, query or reason over. The page sets out the key idea, where it came from, how it is used today and which questions are still open. Every claim links to the papers behind it, and the key papers follow, the most cited first. It leads on to Semantic role labeling and Context-free grammar.
30 key papers🎭Semantic role labeling
Semantic role labeling is the task of finding, for each predicate in a sentence, the phrases that act as its participants and labeling each with a role such as agent, theme, time or place. The result is a shallow picture of who did what to whom, plus when and where. The page sets out the key idea, where it came from, how it is used today and which questions are still open. Every claim links to the papers behind it, and the key papers follow, the most cited first. It leads on to Semantic parsing and Dependency parsing.
30 key papers≈Semantic similarity
Semantic similarity is a graded measure of how close two linguistic units, such as words, concepts, sentences or documents, are in meaning. It is computed either from structured knowledge such as taxonomies, or from vector representations, and is evaluated against human judgments. The page sets out the key idea, where it came from, how it is used today and which questions are still open. Every claim links to the papers behind it, and the key papers follow, the most cited first. It leads on to Distributional semantics and Word embeddings.
29 key papers✂️Subword tokenization
Subword tokenization is a way of splitting text into units smaller than words but usually larger than single characters. A vocabulary of fixed size is learned from a corpus, so frequent words stay whole and rare or unseen words break into known pieces. The page sets out the key idea, where it came from, how it is used today and which questions are still open. Every claim links to the papers behind it, and the key papers follow, the most cited first. It leads on to Language model and Minimum description length.
30 key papers⚡Transformer and attention
The Transformer is a neural network architecture that represents a sequence by letting each element gather information from all others through learned, weighted comparisons called attention. It replaces recurrence and convolution with stacked attention and feed-forward layers. The page sets out the key idea, where it came from, how it is used today and which questions are still open. Every claim links to the papers behind it, and the key papers follow, the most cited first. It leads on to Language model and Scaling laws.
30 key papers🌐Universal Dependencies
Universal Dependencies is a framework and an open collection of treebanks that annotates the grammar of many languages with one shared inventory of part-of-speech tags, morphological features and dependency relations. The shared inventory makes syntactic analyses comparable across languages. The page sets out the key idea, where it came from, how it is used today and which questions are still open. Every claim links to the papers behind it, and the key papers follow, the most cited first. It leads on to Dependency parsing and Part-of-speech tagging.
22 key papers🧮Word embeddings
Word embeddings are dense, real-valued vectors that represent the words of a vocabulary as points in a shared geometric space. They are usually learned from co-occurrence patterns in large unlabeled corpora, so that words used in similar contexts end up close to one another. The page sets out the key idea, where it came from, how it is used today and which questions are still open. Every claim links to the papers behind it, and the key papers follow, the most cited first. It leads on to Distributional semantics and Semantic similarity.
30 key papers🔀Word sense disambiguation
Word sense disambiguation (WSD) identifies which meaning of an ambiguous word is intended in a given context. It is usually framed as picking one entry from a fixed sense inventory, such as the synsets of WordNet. The page sets out the key idea, where it came from, how it is used today and which questions are still open. Every claim links to the papers behind it, and the key papers follow, the most cited first. It leads on to Lexical semantics and Semantic similarity.
30 key papers📉Zipf's law
Zipf's law is the empirical regularity that, when the items of a collection such as the words of a text are ranked by frequency, each item's frequency is roughly inversely proportional to a power of its rank. A few items are therefore extremely common and most are rare. The page sets out the key idea, where it came from, how it is used today and which questions are still open. Every claim links to the papers behind it, and the key papers follow, the most cited first. It leads on to Corpus linguistics and Minimum description length.
12 key papersThis page in Markdown → https://md.research.elamur.ai/free/en/concepts/