📄 A Hidden Markov Model Based System for Entity Extraction from Social Media English Text at FIRE 2015

Summary

The paper describes a named entity recognizer for English tweets, built for ESM-IL, a FIRE 2015 shared task on identifying entities in social-media posts written in Indian languages. The task covers a fine-grained inventory of entity types, including person, location, organization, artifact, facilities, money, date, period, and living thing. The author converts the organizers' offset-based annotation files into IOB format and trains a trigram Hidden Markov Model.

The main twist is the observation unit. Each token is not observed as a bare word but as a composite symbol with three parts:

• the word itself;
• either its POS tag from MontyTagger or, if the word appears in a gazetteer, a gazetteer tag;
• a rule-derived "meta tag" for surface shape, such as capitalization, hashtag, @-mention, colon, or hyphen.

Unseen symbols are scored through suffix statistics of rare training symbols. Decoding uses Viterbi. A small post-processing rule then removes malformed entity spans.

The system ranked first among English submissions with P/R/F of 61.96/39.46/48.21. This shows that a cheap generative model enriched with lexical resources and shape features can compete on noisy social-media text.

Key points

• The model is a trigram HMM. Transition probabilities are smoothed by deleted interpolation over trigram, bigram, and unigram tag counts, and the best sequence is found with Viterbi.
• Each observation symbol is a triplet <word, X-tag, meta-tag>, where the X-tag is a MontyTagger POS tag or a gazetteer tag. Gazetteer lists include first names (Bperson), later name parts (Iperson), facilities, months, days, periods, counts, and money.
• Meta tags come from ordered hand rules. Examples:
  • ICAP for an initial capital;
  • ABBR for abbreviations;
  • CHAS/HASH for hashtags;
  • ATSY for @-mentions;
  • CCOL/COLN for a trailing colon;
  • CHYP/HYPH for hyphens;
  • ALDT for all-digit tokens.
• Unknown triplets are handled by appending the X-tag and meta-tag to the word to form a pseudo word. Its probability is then estimated by suffix analysis over rare pseudo words (frequency ≤ 2) in training.
• The English data lists the figures 11,003 and 9641 in the training/test columns, with 21 NE types. The official result was F = 48.21, ahead of the next-best English run at 42.64.

Interesting ideas

• I1. Packing extra evidence (a POS or lexicon tag plus a shape class) into one composite observed symbol lets a plain HMM use rich features without changing its equations or decoder.
• I2. When a token hits a gazetteer, the lexicon tag replaces the POS tag. This gives the resource priority over a statistical guess, and the switch stays visible in the output.
• I3. Appending the feature tags to the word before running suffix analysis makes the unknown-word model sensitive to both morphology and context class at once.
• I4. A deterministic post-check removes any I-XXX run that starts directly after O. The author treats such runs as model errors, not as broken entities to be repaired.
• I5. Shape features specific to social media (hashtags, @-mentions, colons after names) are encoded as a small ordered rule cascade, which is trivial to audit.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tokens): The en tagger could use composite keys like word+shape+lexicon class as a cheap, fully inspectable backoff signal for noisy or out-of-domain input.
• I2 (world): Entity typing in the text world could record "typed by lexicon hit" versus "typed by model" as separate provenance, which keeps each entity's class explainable.
• I3 (tokens): Unknown-word handling in the lemmatizer/tagger could condition suffix statistics on shape and lexicon class, not on the raw string alone.
• I4 (learn): This works as a self-induced repair rule. Its effect can be checked against a negative control: compare the rule's deletions with gold spans to confirm it removes more false entities than true ones.
• I5 (tokens): The tokenizer could emit an explicit shape tag for each token, so later stages can tell a Twitter handle from an ordinary proper noun.