📄 Simple Effective Microblog Named Entity Recognition: Arabic as an Example

Summary

The paper addresses named entity recognition (NER) on Arabic tweets. Newswire-trained systems perform poorly on tweets because the language is informal and dialectal, spelling varies, entity names are often shortened, and Arabic has no capitalization cue. The authors build a CRF baseline with simple character-affix and word-window features. They then test three language-independent improvements: a large gazetteer mined from Wikipedia, domain adaptation that mixes news and tweet training data, and a semi-supervised two-pass scheme. In the two-pass scheme, entities found automatically in a large unlabeled tweet collection become a confidence-weighted gazetteer feature. They also annotate a new Arabic tweet NER dataset, with training and test tweets taken from different time periods. Combining all three techniques raises overall F-measure to 65.2. That is 35.3 points above the news-trained baseline and 19.9 points above the tweet-trained baseline. The study shows which cheap techniques help most for noisy social-media text when linguistic tools are weak.

Key points

• The baseline uses CRF++ with the leading and trailing 1–4 letters of each word plus the raw current, previous and next words. It is trained either on ANERCORP (about 150k news tokens) or on 3,646 newly annotated tweets from May 2012.
• The test set has 1,423 tweets (about 26k tokens) from November 2011. Only 25% of its entities appear in the tweet training data. Overall F rises from 29.9 with news training to 45.3 with tweet training, so in-domain data had the largest effect of any single factor.
• The Wikipedia gazetteer holds 70,908 locations, 26,391 organizations and 81,880 persons. Titles were typed via category trigger words, and redirects were added. It covers 60% of test entities and lifts F to 37.2 with news training and 55.0 with tweet training. It mainly boosts precision.
• Domain adaptation replicates the tweet set alongside ANERCORP. A factor of 7 was chosen on an 80/20 split, testing values from 1 to 9. This reaches F 54.6 alone and 59.5 with the gazetteer, and it mainly boosts recall.
• The two-pass method tags 65M unlabeled tweets. The resulting automatic gazetteer covers 74% of test entities. It adds 3.2 F over the tweet baseline and 5.3 F over the baseline with the Wikipedia gazetteer, so the method works better when the first-pass system is stronger.

Interesting ideas

• I1. Run a tagger over a huge unlabeled corpus, drop items found only once, and feed the survivors back as a lexicon feature weighted by how often each was given each type. This turns noisy self-labels into a soft prior rather than hard training labels.
• I2. Continuous confidence values are rounded to one significant figure and fused into the label string (e.g. a B-ORG tag with a .8 suffix), so a learner that only handles symbolic features can still use graded evidence.
• I3. Domain adaptation is done simply by repeating the small in-domain set several times next to a large out-of-domain set, with the repeat count tuned on held-out data.
• I4. The test tweets come from a different time window than the training tweets, which checks generalization to new entities rather than memorization of recurring ones.
• I5. Wikipedia titles are typed as person, location or organization by checking whether their categories contain trigger words such as "cities" or "births", which gives a large typed lexicon with almost no manual work.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): Silver entity or lemma lists harvested from Mova's own runs could become weighted features for the next training round, instead of being trusted outright.
• I2 (tokens): The perceptron tagger could take binned lexicon-confidence features the same way, and each feature would stay readable when explaining a decision.
• I3 (tree): When mixing a small target-dialect treebank with large UD treebanks for the parser, a tuned replication factor is a cheap and transparent way to weight the domains.
• I4 (learn): Evaluation splits for Mova could be separated by source or period, giving a harder control that exposes overfitting to frequent names.
• I5 (world): Similar category-keyword rules could cheaply assign coarse ontology types to Wikidata or Wikipedia entities in Mova's knowledge layer.