🏷️ Named entity recognition
Named entity recognition locates stretches of text that refer to specific real-world things, such as people, organizations, places, products or dates, and gives each one a category drawn from a fixed or open inventory of entity types.
What it is
Named entity recognition (NER) turns unstructured text into a list of typed mentions. In "Andrew Little met Microsoft executives in Bonn", a system should mark "Andrew Little" as a person, "Microsoft" as an organization and "Bonn" as a location. The task has two parts. Segmentation finds where a name begins and ends. Classification decides what kind of thing the name denotes.
Inventories vary by benchmark. The CoNLL-2003 news data uses four labels: person, location, organization and miscellaneous (Sang & Meulder 2003; McCallum & Li 2003). The MUC evaluations also covered times, dates, percentages and monetary amounts (Zhou & Su 2002). Biomedical NER targets proteins, DNA, RNA, cell types and cell lines (Collier et al. 2004).
A further distinction separates two settings. In flat NER, mentions never overlap. In nested NER, one mention sits inside another, as with a country inside an organization name (Finkel & Manning 2009; Li et al. 2020).
NER is often the first stage of a larger pipeline. Entity linking maps each recognized mention to a knowledge-base entry (Bunescu & Paşca 2006; Yamada et al. 2016; Wu et al. 2020). Relation discovery then looks for links between pairs of tagged entities (Hasegawa et al. 2004).
The key idea
The dominant formulation treats NER as sequence labeling. Each token receives a tag that encodes both its position inside an entity and the entity type. The resulting tag sequence can then be decoded back into spans.
| Token | Andrew | Little | visited | New | York |
|---|---|---|---|---|---|
| BIO | B-PER | I-PER | O | B-LOC | I-LOC |
| BILOU | B-PER | L-PER | O | B-LOC | L-LOC |
In BIO, B marks the beginning of a span, I marks a token inside it, and O marks a token outside any entity. BILOU (also called BIOES) adds L for the last token of a span and U for a single-token entity. On CoNLL-2003 test data, BILOU outperformed BIO, at 90.57 versus 89.15 F1 (Ratinov & Roth 2009).
A classic model for scoring tag sequences is the linear-chain conditional random field:
P(y | x) = (1 / Z(x)) · exp( Σₜ Σₖ λₖ · fₖ(yₜ₋₁, yₜ, x, t) )
Here x is the input sentence and y = y₁ … yₙ is a candidate tag sequence. The index t runs over token positions, and y₀ is a fixed start symbol. Each fₖ is a feature function. It looks at the previous tag yₜ₋₁, the current tag yₜ and any part of the sentence around position t. Typical features include capitalization, affixes or a gazetteer match. Each λₖ is a learned weight. Z(x) is a normalizing sum over all possible tag sequences, so the probabilities add to one. The best sequence is found with Viterbi dynamic programming.
McCallum and Li used this model, and so did Settles, each with a different feature set (McCallum & Li 2003; Settles 2004). In neural systems, the hand-built fₖ are replaced by scores computed from learned token representations. One example is a character CNN combined with a bidirectional LSTM (Chiu & Nichols 2015). Another is a bidirectional LSTM topped by a CRF output layer (Lample et al. 2016).
Exact structured decoding is not always essential. For a model trained on CoNLL-2003 with baseline features, greedy left-to-right decoding scored nearly the same as Viterbi (83.29 vs 83.71 F1) and ran over 100 times faster (Ratinov & Roth 2009).
Several alternatives to per-token tagging now exist:
- Span classification scores every start–end pair directly, as in a biaffine parser-style model (Yu et al. 2020).
- Question answering phrases each entity type as a query and extracts the answer spans (Li et al. 2020).
- Generation produces marked-up text or filled templates. It uses either a sequence-to-sequence (seq2seq) model, which maps an input text to an output text, or a large language model, a very large pretrained text generator steered by prompts (Cui et al. 2021; Wang et al. 2025).
History and founding papers
The task took its modern shape in the named entity evaluations of the Message Understanding Conferences, notably MUC-6 and MUC-7. Systems in those evaluations often relied on hand-written rules and name lists (Zhou & Su 2002; Mikheev et al. 1999). Mikheev, Moens and Grover questioned how much those lists mattered. On MUC-7 data, their system recognized persons and organizations in the high-80s F-range with no gazetteers at all. A small list of well-known place names recovered most of the location performance (Mikheev et al. 1999).
In the same year, Collins and Singer showed that names could be classified from just seven seed rules plus unlabeled text. Their method exploited two redundant views of each example, spelling and context. It exceeded 91% accuracy on a hand-labelled test set drawn from 1,000 random examples (Collins & Singer 1999).
Statistical learning soon matched rule-based systems. Zhou and Su's HMM chunk tagger reported F-measures of 96.6 on MUC-6 and 94.1 on MUC-7 (Zhou & Su 2002).
The CoNLL-2003 shared task gave the field a lasting multilingual benchmark in English and German (Sang & Meulder 2003). One entry was a CRF with automatic feature induction and web-harvested lexicons. It reached 84.04 F1 on English test data with about 6.4 thousand features (McCallum & Li 2003). Another entry combined several diverse classifiers and reached 88.76 F1 on English test data (Florian et al. 2003).
Biomedical NER gained its reference benchmark with JNLPBA 2004. There the best total F-score was 72.6 (Collier et al. 2004), and a CRF with mainly orthographic features scored 69.8 (Settles 2004).
Ratinov and Roth then examined the main design choices in a systematic study: chunk encoding, decoding, non-local features and external knowledge. Their system reached 90.8 F1 on CoNLL-2003 (Ratinov & Roth 2009). Finkel and Manning recast nested NER as constituency parsing (Finkel & Manning 2009).
The neural turn came around 2015–2016. Chiu and Nichols combined a character-level CNN with a stacked bidirectional LSTM. They reported 91.62 F1 on CoNLL-2003 and 86.28 on OntoNotes 5.0 (Chiu & Nichols 2015). Lample and colleagues proposed two taggers: a bidirectional LSTM with a CRF output layer and a transition-based stack-LSTM model (Lample et al. 2016). A survey of this period built on the first of these by adding affix features. It found that neural models learning their own features outperformed feature-engineered systems on CoNLL-2002 and CoNLL-2003 news data in all four languages: Spanish, Dutch, English and German (Yadav & Bethard 2018).
How it is used today
Pretrained transformer encoders now underlie most high-scoring systems. A biaffine span model using BERT reached 93.5 F1 on English CoNLL-2003. At its publication in 2020, it also reported the best results on the nested corpora ACE 2004, ACE 2005 and GENIA (Yu et al. 2020).
Nested labels can also be linearized into per-token label sequences instead of explicit trees, hypergraphs or stacked recognizers. In one study using this encoding, a multilabel LSTM-CRF suited flat or lightly nested data. A seq2seq decoder, even without contextual embeddings, beat the previous best results on ACE-2004, ACE-2005, GENIA and the Czech CNEC corpus (Straková et al. 2019).
Efficiency remains a practical concern for large-scale tagging. Iterated dilated convolutions came close to Bi-LSTM-CRF accuracy while being 14–20 times faster at test time (Strubell et al. 2017).
Language-specific issues shape model design. Chinese has no explicit word boundaries. This has motivated lattice models that inject lexicon words without committing to one segmentation (Zhang & Yang 2018). A later transformer variant, FLAT, encodes the lattice through relative positions. It beats lattice LSTM in average F1 and runs faster at inference. Batching speeds FLAT up 4.97 times, against 2.1 times for lattice LSTM (Li et al. 2020).
Social-media text required separate work:
- English tweets: tweet-adapted tagging pipelines with distant supervision from Freebase (Ritter et al. 2011), and KNN-plus-CRF bootstrapping (Liu et al. 2011).
- Chinese Weibo posts: embeddings trained jointly with the tagger. Test F1 still stayed around 44 for name mentions and around 40 for name plus nominal mentions (Peng & Dredze 2015).
Coverage of more languages is expanding through community datasets such as MasakhaNER for ten African languages (Adelani et al. 2021).
In low-resource settings, prompting and generation are increasingly common. One approach uses BART, a pretrained seq2seq model, to score filled natural-language templates for each candidate span. It beat fine-tuned BERT by 10.88 to 15.34 F1 in low-resource transfer to the MIT Movie, MIT Restaurant and ATIS (flight) datasets (Cui et al. 2021). GPT-NER reported roughly 60 F1 with just 8 training sentences, where supervised models scored near zero (Wang et al. 2025).
Open questions
Generalization beyond memorized names is the central unresolved issue. The WNUT 2017 task removed test texts whose entity surface forms already appeared in training. The best system then reached only about 42 entity F1 (Derczynski et al. 2017). Similar weaknesses appear elsewhere. In MasakhaNER, entities unseen in training lost about 5 F1 points, and entities of four or more words lost about 20 (Adelani et al. 2021).
Domain shift remains severe. Stanford NER, trained on CoNLL-2003 news, fell from 90.8 F1 on news to 45.8 F1 on tweets (Liu et al. 2011). This is a different tagger from LBJ-NER, despite the identical news figure. Transfer from Wikipedia-based data to African-language news stayed below 40 F1 (Adelani et al. 2021). How best to transfer across domains, languages and label sets is still open. Query-based and template-based models are promising candidates (Li et al. 2020; Cui et al. 2021).
Other questions concern resources and evaluation:
- The value of external knowledge such as gazetteers is inconsistent across settings (Mikheev et al. 1999; Settles 2004).
- Annotation quality limits measured progress. In one Weibo error analysis, about 43% of sampled errors were annotation mistakes (Peng & Dredze 2015).
- It is unclear when large language models can reliably replace supervised taggers, given their tendency to overpredict entities (Wang et al. 2025).
- Whether nested structure should be kept in corpora and modeled by default is still debated (Finkel & Manning 2009).
Key papers
Related concepts
- Coreference resolution — Coreference resolution is the task of finding the expressions in a text that refer to the same entity as one another, such as names…
- Part-of-speech tagging — Part-of-speech tagging is the task of assigning each token in a text a grammatical category, such as noun, verb, adjective or determiner…
- Word embeddings — Word embeddings are dense, real-valued vectors that represent the words of a vocabulary as points in a shared geometric space. They are…
- Knowledge representation and reasoning — Knowledge representation and reasoning is the area of artificial intelligence that studies how facts, concepts and rules about a domain can…
More papers
Topics: 🏷️ Named entity recognition · 🔎 Information extraction
Shelves: 💬 Natural language processing
This page in Markdown → https://md.research.elamur.ai/free/en/concepts/named-entity-recognition/