📊 Machine learning

About 16 minutes to read · every paper named here has its own page in this library

What it is

Suppose you want a program that sorts news stories into topics, or flags whether a film review is positive or negative. One option is to write the rules yourself: if the story mentions "election", file it under Politics. Rules like these are brittle and expensive to maintain, and they break when the vocabulary shifts. Machine learning (ML) takes a different route. You collect examples that are already sorted and choose a family of candidate functions. A general procedure then searches that family for the member that best reproduces the examples. The program is learned from data, not programmed.

Text categorisation shows the shift clearly. During the 1990s the field moved from hand-written expert rules, which it called knowledge engineering, to classifiers induced from labelled documents. By the early 2000s these learned systems matched trained human indexers at far lower labour cost (Sebastiani 2002). The same move happened across natural language processing (NLP), speech, vision and game playing. Today most artificial intelligence systems are built by learning from data, not by hand-coding their behaviour.

Learning problems are usually grouped by the kind of signal the data provides.

The lines between these settings blur. Semi-supervised methods mix a few labelled documents with many unlabelled ones (Joachims 1999). Weak supervision replaces labels with indirect signals. For example, a semantic parsing system can learn to follow navigation instructions by checking whether executing them reaches the right place, with no annotated logical forms (Artzi & Zettlemoyer 2013). Self-supervised pre-training is typically followed by supervised fine-tuning (Devlin et al. 2019). Reinforcement learning (RL) is now applied on top of pre-trained large language models (DeepSeek-AI et al. 2025).

How it works

Model, loss and optimisation

Almost every ML method has three parts. The first is a model: a parametrised family of functions, such as linear separators, decision trees or deep neural networks. The second is a loss, which scores how wrong a prediction is. The third is an optimiser, which adjusts the parameters to reduce the average loss on the training data. That average is called the empirical risk:

R̂(θ) = (1/n) · Σᵢ₌₁ⁿ L( f_θ(xᵢ), yᵢ )

Here f_θ is the model with parameters θ, and (xᵢ, yᵢ) is the i-th of n training pairs, xᵢ an input and yᵢ its correct output. L is the loss function, for example squared error for numbers or cross-entropy for class labels. R̂(θ) is the empirical risk, the mean loss over the sample. The true quantity of interest is the expected loss on new data drawn from the same source, which can only be estimated.

For differentiable models the standard optimiser is gradient descent. Each step moves the parameters a little against the gradient of the risk:

θ ← θ − η · ∇θ R̂(θ)

Here η is the learning rate, a small positive step size, and ∇θ R̂(θ) is the vector of partial derivatives of the risk with respect to each parameter. In practice the gradient is estimated on a small random minibatch, which gives stochastic gradient descent. Adam keeps running averages of each parameter's gradient and squared gradient, so every parameter gets its own step size. A bias correction compensates for the averages starting at zero. The suggested defaults include a step size of 0.001 and decay rates of 0.9 and 0.999 for the two averages (Kingma & Ba 2014). Not every method minimises the risk by gradient steps. Support vector machines (SVMs) solve a convex quadratic program (Cortes & Vapnik 1995), and decision trees are grown greedily, split by split.

Generalisation and overfitting

A model that only memorised its training data would be useless, so the real goal is generalisation: low error on examples it has not seen. When a model fits quirks of the sample rather than the underlying pattern, it is said to overfit. Training error then keeps falling while error on new data rises.

The standard defence is to split the data three ways. A training set fits the parameters. A validation set chooses hyperparameters and compares models. A test set, touched once, estimates final performance. Results are more trustworthy when averaged over many random splits; Breiman, for example, reported means over 100 learning/test partitions with standard errors (Breiman 1996). Validation scores can even drive automated design. In neural architecture search, a controller network proposes architectures and is rewarded by their validation accuracy (Zoph & Le 2016).

Theory gives partial guarantees. For a maximum-margin separator, the expected test error is bounded by the expected fraction of training points that become support vectors. The bound does not depend explicitly on the dimension of the feature space (Cortes & Vapnik 1995). This helps explain why SVMs could use every word of a vocabulary without overfitting text (Joachims 1998).

Overfitting also has costs beyond accuracy. An attacker's ability to tell whether a record was in the training set grows with the generalisation gap. Avoiding overfitting is still not enough: a training algorithm that has been subverted can stay stable and still leak membership (Yeom et al. 2017).

The bias–variance trade-off

For squared-error regression, the expected error of a learned predictor at an input splits into three parts:

E[(y − f̂(x))²] = Bias[f̂(x)]² + Var[f̂(x)] + σ²

Here f̂ is the predictor learned from a random training sample and y is the observed target at input x. Bias is the gap between the average prediction (over possible training samples) and the truth. Variance measures how much the prediction changes from one sample to another. σ² is irreducible noise in the targets.

Simple models tend to have high bias and low variance; flexible ones the reverse. Regularisation, early stopping and model selection move a learner along this trade-off. Ensembles attack the variance term directly. Bagging averages models trained on bootstrap resamples. It helps most when the base learner is unstable, meaning small data changes yield very different models, and it helps little for stable learners (Breiman 1996). Data augmentation is another lever when data are scarce. Simple random edits such as synonym swaps and deletions reduced overfitting of small text classifiers, though the benefit faded with large datasets (Wei & Zou 2019).

History

The idea of learning machines is older than the name. Rosenblatt's perceptron (1958) learned a linear separator from examples. Arthur Samuel's checkers program (1959) is usually credited with popularising the phrase "machine learning". Backpropagation, popularised by Rumelhart, Hinton and Williams (1986), made multi-layer networks trainable. Quinlan's decision-tree learners (ID3, 1986) and Vapnik's statistical learning theory supplied other foundations.

The 1990s brought a wave of principled, often convex methods. The soft-margin support-vector network let maximum-margin classifiers tolerate training errors and use kernels to reach very high-dimensional feature spaces (Cortes & Vapnik 1995). Bagging showed how to improve the accuracy of an unstable learner (Breiman 1996). Language processing absorbed these tools quickly. SVMs became a default for text classification (Joachims 1998), and a survey consolidated learned text categorisation as a discipline (Sebastiani 2002). In what is often called the statistical turn, tasks that had relied on hand-built grammars and rules were recast as classification problems over annotated corpora.

Around 2010 learned representations began to replace hand-engineered features. Word clusters and embeddings induced from unlabelled text improved supervised taggers when added as features (Turian et al. 2010). In 2012 deep convolutional networks trained on ImageNet changed computer vision (Krizhevsky et al. 2012). In 2013 a deep network learned to play Atari games from raw pixels through reinforcement learning (Mnih et al. 2013). Adam then gave practitioners a robust default optimiser (Kingma & Ba 2014). In 2015 a generative method was proposed that learns to reverse a gradual noising process, laying the groundwork for later diffusion models (Sohl-Dickstein et al. 2015). Open-source frameworks such as TensorFlow, Google Brain's successor to DistBelief, made large models routine to build (Abadi et al. 2016).

The next phase was pre-training at scale. BERT, a Transformer encoder pre-trained on masked-word prediction, set new results on eleven NLP tasks. It also showed that one general model, lightly fine-tuned, could replace many task-specific architectures (Devlin et al. 2019). The same self-supervised recipe spread to speech (Baevski et al. 2020).

Reinforcement learning from human preferences dates from 2017 (Christiano et al. 2017). Around 2022 it moved to the centre of language-model training, with assistants tuned on human comparisons (Bai et al. 2022). Since then, rewards that can be checked automatically have been used to train reasoning (DeepSeek-AI et al. 2025; Yu et al. 2025).

Methods

Linear models and naive Bayes

Linear regression predicts a number as a weighted sum of input features, fitted by least squares. Logistic regression passes the same weighted sum through a sigmoid to give a class probability and is trained with cross-entropy loss. Both are fast and easy to inspect, and they often perform well. One study of COMPAS, a commercial recidivism tool that relies on 137 features, found that a plain logistic regression predicted just as accurately (Mehrabi et al. 2019).

Naive Bayes assumes that features are independent once the class is known, and multiplies per-feature probabilities. It is crude but quick, and it became the standard baseline in text work. In Joachims' experiments, even a naive Bayes classifier given nothing but the lowest-ranked words beat chance. This supports the claim that few text features are truly irrelevant (Joachims 1998). Against SVMs and maximum-entropy models on sentiment, naive Bayes tended to come last, though by modest margins (Pang et al. 2002).

Support vector machines and kernels

An SVM looks for the separating hyperplane with the widest margin. Slack variables allow some training points to be misclassified. The solution depends only on the support vectors, the points nearest the boundary (Cortes & Vapnik 1995). A kernel function computes inner products in a vast implicit feature space without ever building it. The original experiments used polynomial surfaces of degree up to 7 on 256-dimensional character images (Cortes & Vapnik 1995).

In text categorisation over the Reuters-21578 and Ohsumed collections, SVMs beat naive Bayes, Rocchio, k-nearest neighbours and C4.5 for every parameter setting tested (Joachims 1998). A transductive variant also uses the unlabelled test documents when placing the margin. It gave the largest gains when labelled data were tiny (Joachims 1999).

Through the 2000s SVMs drove much of NLP:

• chunking, a form of shallow parsing, with polynomial kernels and voting across chunk representations (Kudo & Matsumoto 2001);
• deterministic dependency parsing by choosing Shift, Left or Right actions (Yamada & Matsumoto 2003);
• part-of-speech tagging with a language-independent tagger generator (Giménez & Màrquez 2004);
• semantic role labeling over PropBank (Pradhan et al. 2004);
• reading-level assessment from n-gram language-model features (Schwarm & Ostendorf 2005).

Decision trees and ensembles

A decision tree splits the data on one feature at a time until the leaves are nearly pure. Trees are interpretable and handle mixed feature types. They are also unstable, which makes them ideal for ensembles. Bagged trees cut test misclassification by between 6% and 77% across Breiman's classification datasets (Breiman 1996). Boosting trains models in sequence, each one reweighting the examples its predecessors got wrong. It is credited, together with SVMs, for much of the progress in text categorisation (Sebastiani 2002).

In language work, trees made coreference resolution learnable. A tree classifier over 12 cheap features, deciding whether two noun phrases corefer, matched the best rule-based systems on the MUC data (Soon et al. 2001). Later work changed how antecedents are clustered and training instances selected, and expanded the features to 53 with careful selection (Ng & Cardie 2002).

Clustering and learned representations

Unsupervised methods find structure without labels. k-means, usually traced to MacQueen (1967), assigns each point to the nearest of k centroids and then moves each centroid to the mean of its points, repeating until stable. Topic modeling and latent semantic indexing find latent themes in document collections.

In NLP, clustering and distributional semantics led to word embeddings. A controlled comparison found that Brown clusters and neural embeddings induced from unlabelled text each improved named entity recognition and chunking. Brown clusters helped most on rare words (Turian et al. 2010). Sentence-level vectors followed. One approach trained an encoder on natural-language inference pairs (Conneau et al. 2017). Another extended a word-prediction objective to whole sentences with n-gram features (Pagliardini et al. 2018).

Deep learning

Deep learning stacks many layers of learned transformations and trains them end to end with backpropagation and stochastic gradients. Convolutional networks dominated vision after 2012 (Krizhevsky et al. 2012). Recurrent encoder-decoders, and later the Transformer, took over sequence tasks.

A language model such as BERT is pre-trained on unlabelled text. Its masked-language-model objective lets every layer use both left and right context. A second objective, next-sentence prediction, teaches relations between pairs of sentences (Devlin et al. 2019). Parameter-sharing variants made such models lighter (Lan et al. 2019). Inputs to these models are broken into pieces by subword tokenization; BERT used a WordPiece vocabulary (Devlin et al. 2019).

In speech, wav2vec 2.0 masks spans of latent audio frames and learns to identify the true ones among distractors. After pre-training it needs as little as 10 minutes of transcribed speech. With that much it reached word error rates of 4.8 on the clean and 8.2 on the other Librispeech test set (Baevski et al. 2020). Large unlabelled multilingual corpora such as VoxPopuli supply raw material for this kind of speech pre-training and self-training (Wang et al. 2021).

Reinforcement learning

In reinforcement learning an agent acts, receives rewards and adjusts a policy to maximise long-run return. Policy-gradient methods descended from Williams' REINFORCE (1992) do not need the reward to be differentiable. That is why they were used to search architectures (Zoph & Le 2016) and to optimise non-differentiable text objectives. Examples include:

• coreference metrics (Clark & Manning 2016);
• simplicity rewards for sentence simplification (Zhang & Lapata 2017);
• conversational rewards for dialogue systems (Li et al. 2016);
• query correctness for text-to-SQL (Zhong et al. 2017).

AlphaZero, from David Silver and colleagues, combined self-play with tree search and reached top strength in chess, shogi and Go from the rules alone (al. 2017). Recurrent policies trained across many tasks can learn to learn quickly within a new one (Duan et al. 2016).

For language models, rewards first came from human comparisons. A preference model was trained on crowdworker choices, and its scores were optimised with proximal policy optimisation (PPO) (Bai et al. 2022). More recently the rewards are rule-checked answers in mathematics and code. DeepSeek-R1 was trained with group relative policy optimisation (GRPO). This method scores each answer against the rewards of the other answers drawn for the same question, so no separate critic network is needed (DeepSeek-AI et al. 2025). Open reproductions added changes such as decoupled clipping and dynamic sampling to keep long chain-of-thought training stable (Yu et al. 2025). A gradient-free variant has agents write verbal critiques of their failures and reread them on the next attempt (Shinn et al. 2023).

Software and hardware

Modern ML depends on its tooling. TensorFlow represents a computation and its mutable state as one dataflow graph (Abadi et al. 2016). It can be deployed anywhere from a phone to a cluster of thousands of GPUs (Abadi et al. 2016). MXNet mixes declarative graphs with imperative array code (Chen et al. 2015). PyTorch 2 keeps eager Python programming and adds graph capture and compilation (Ansel et al. 2024). On the hardware side, Google's TPU v4 links thousands of chips through reconfigurable optical switches and adds dedicated support for embeddings (Zhou; & Patterson 2023). The engineering side is covered under GPU and parallel computing.

Datasets and benchmarks

How performance is measured

Evaluation compares predictions with held-out gold answers, and the metric must suit the task.

• Classification reports accuracy, precision, recall and F1. In multi-category text work, scores are averaged across categories; Joachims used the microaveraged precision/recall breakeven point (Joachims 1998).
• Coreference reports the average of the MUC, B³ and CEAF F1 scores, often called the CoNLL score (Clark & Manning 2016). MUC counts recovered coreference links, B³ scores each mention's cluster, and CEAF aligns predicted entities with gold ones.
• Speech recognition reports word error rate (Baevski et al. 2020).
• Generation uses overlap metrics such as BLEU and simplification-specific scores such as SARI, together with human ratings (Zhang & Lapata 2017).
• Code and mathematics use pass@1, the share of problems solved on the first sample (DeepSeek-AI et al. 2025; Shinn et al. 2023).

Human judgement remains indispensable where automatic scores fall short. Dialogue systems, for example, have been compared through paired judgements of single replies and of simulated multi-turn conversations (Li et al. 2016). The broader questions are collected under evaluation and benchmarks.

Shared datasets

Progress has been organised around shared data. A few that recur in the work above:

Fairness research keeps its own set, including UCI Adult, German Credit and COMPAS records, and WinoBias for gender bias in coreference (Mehrabi et al. 2019).

When benchmark numbers mislead

Benchmark numbers can be fragile. A role labeller trained on Wall Street Journal text was tested on the standard test section. There it reached an F1 of 87 for combined argument identification and classification on hand-corrected parses, and 79 on automatic parses. On a small AQUAINT set of other newswire it fell to 63.3 (Pradhan et al. 2004).

Self-written unit tests can be wrong too. On one code benchmark, 16.3% of solutions that passed an agent's own tests were in fact incorrect (Shinn et al. 2023). In large-scale RL, training reward was a poor guide to validation accuracy (Yu et al. 2025).

People and institutions

The founding methods carry their authors' names. Corinna Cortes and Vladimir Vapnik introduced the soft-margin support-vector network (Cortes & Vapnik 1995). Leo Breiman proposed bagging (Breiman 1996). Thorsten Joachims established SVMs for text and their transductive extension (Joachims 1998; Joachims 1999). Diederik Kingma and Jimmy Ba designed Adam (Kingma & Ba 2014).

In language processing, Vincent Ng and Claire Cardie reworked learning-based coreference resolution (Ng & Cardie 2002). Cardie is at Cornell University. Wee Meng Soon, Hwee Tou Ng and Daniel Lim built the decision-tree resolver that their work started from (Soon et al. 2001). Yoshua Bengio co-authored the controlled comparison of word representations for semi-supervised learning (Turian et al. 2010). Jacob Devlin and colleagues at Google introduced BERT (Devlin et al. 2019). Google Brain also built TensorFlow (Abadi et al. 2016).

In reinforcement learning, Quoc V. Le and Barret Zoph turned architecture design into a learning problem (Zoph & Le 2016). Jianfeng Gao, now at Microsoft, co-authored early RL training of neural dialogue agents (Li et al. 2016). Karthik Narasimhan, now at Princeton University, and Shunyu Yao co-authored Reflexion (Shinn et al. 2023). Junxiao Song, now at DeepSeek, is among the authors of DeepSeek-R1 (DeepSeek-AI et al. 2025). On the human side of learning systems, Cynthia Rudin, now at Duke University, has argued for interpretable models in high-stakes use (Rudin 2018; Semenova; & Zhong 2021).

Much of the field's work comes from a few dozen universities and industrial labs, among them Stanford University, Tsinghua University, Carnegie Mellon University, the Massachusetts Institute of Technology, Google DeepMind and Meta.

Open questions

Interpretability and trust

Interpretability is still unsettled. Rudin argues that post-hoc explanations of black boxes are imperfect copies of the original models and can mislead. On structured data with meaningful features, she adds, simple interpretable models usually perform about as well as complex ones (Rudin 2018). A later survey by Rudin and colleagues lists ten technical challenges for interpretable learning, from sparse logical models to interpretable reinforcement learning (Semenova; & Zhong 2021).

Doshi-Velez and Kim note that the field lacks an agreed definition of interpretability. They propose evaluating it at three levels: real tasks with real users, simplified tasks with real users, and proxy tasks with no humans at all (Doshi-Velez & Kim 2017). How to explain what deep networks have learned inside is the concern of interpretability research.

Fairness, privacy and security

Fairness criteria conflict. Equalized odds requires equal true- and false-positive rates across groups, and it can be met by adjusting thresholds after training. Demographic parity, by contrast, can rule out even a perfect predictor (Hardt et al. 2016). Bias enters through a loop that runs from data to algorithm to users and back into future data (Mehrabi et al. 2019). Work on this is gathered under bias, fairness and safety.

Security and privacy remain open. A network trained on poisoned data can hide a backdoor that a small trigger activates, while accuracy on clean inputs barely changes (Gu et al. 2017). Privacy is tied to overfitting, but, as described under generalisation above, a model that generalises well can still expose its training records (Yeom et al. 2017).

What reinforcement learning teaches language models

Reinforcement learning on language models raises its own questions. Rule-based rewards can teach long, self-checking reasoning without human-written demonstrations. The resulting models, though, can be hard to read and can mix languages (DeepSeek-AI et al. 2025). Neural reward models are open to reward hacking at scale (DeepSeek-AI et al. 2025). Alignment training appeared nearly free in capability for large models but costly for small ones (Bai et al. 2022). It is not yet known whether such gains extend from mathematics and code and programming to other kinds of reasoning.

Learning from little data

Data efficiency is a persistent theme. Self-supervision lets speech recognisers work from a tiny transcribed sample, as described under deep learning above (Baevski et al. 2020). Augmentation and transduction help small text tasks (Wei & Zou 2019; Joachims 1999). Even so, most of the world's languages remain low-resource languages. How scaling laws, few-shot learning and principles such as minimum description length fit together into a single account of why learned models generalise is still an open problem.

The library in figures