Elamur Research⌕
☰
home / papers / 2019 / Arabic Named Entity Recognition: What Works and What's Next

Arabic Named Entity Recognition: What Works and What's Next

Paper · 2019 · Liyuan Liu, Jingbo Shang and Jiawei Han

Summary

The paper describes the system that won a Topcoder competition on Arabic named entity recognition. The competition was built on the AQMAR benchmark, which consists of Arabic Wikipedia articles annotated with four entity types, and only training-set labels were allowed. The authors first repaired inconsistent label sequences in the data and released the cleaned version. Their model is a layered pipeline:

  • token representations from three sources: frozen pre-trained word embeddings, a contextual signal from a character-level language model, and a very simple word-shape feature;
  • several LSTM-CRF taggers trained on these representations, with their outputs merged by a sequence-aware voting scheme;
  • a dictionary built from the training data, whose matches are added only where they do not clash with the taggers' predictions.

The full system reaches 75.82% test F1 and beat all other entrants. An ablation removes components one at a time. The largest gains come from the language model and from preprocessing its training text. Hand-made shape features still help despite the contextual model, which the authors attribute to scarce training data. The dictionary contributes only a small gain. Code and models are public.

Key points

  • Data: AQMAR has 28 hand-annotated Arabic Wikipedia articles, split 14/7/7 into train/dev/test, with types PER, LOC, ORG and MISC.

    • The authors fixed malformed tag sequences, such as an O tag inside an entity or an I- tag whose type differs from the preceding B- tag.

  • Competition: 137 registrants made 220 submissions from 30 participants.

    • The final model scores P 81.06 / R 71.22 / F1 75.82.

  • Cumulative ablation (F1):

    ConfigurationF1
    Full system75.82
    Without dictionary75.70
    Without ensemble73.80
    Without word shape71.47
    Without LM preprocessing66.07
    Without language model (plain LSTM-CRF with word embeddings)54.50

  • Dictionary on its own is precise but rarely fires: P 64.35, R 8.83, F1 15.53.

  • Word-shape feature: it has just three classes (number, contains Latin letters, other/Arabic).

    • Most of its benefit shows up on entities that mix Arabic and English.

Interesting ideas

  • I1. Before training or scoring, automatically repair structurally invalid tag sequences, such as broken spans or type switches inside an entity. Noisy encodings silently distort both what the model learns and how it is compared.
  • I2. Ensemble sequence taggers by counting how often each model emits each adjacent label pair and each per-token label, then decode the best path with dynamic programming. The merged output stays coherent, which per-token majority voting does not guarantee.
  • I3. Let a high-precision, low-recall lookup only fill gaps: dictionary matches are kept solely when they do not overlap the learned model's spans. Report the lookup's standalone scores so its real contribution is visible.
  • I4. A tiny, explainable shape class (digits, Latin script, other script) gave a clear gain even on top of a character language model, because small training sets leave rare patterns under-learned.
  • I5. Run ablations cumulatively, stripping one component after another down to a plain baseline. This shows how much each layer adds on top of the rest rather than in isolation.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Silver UD trees agreed by witness parsers could pass a structural validator (single root, no cycles, legal span or label combinations) that repairs or rejects them before they are used as training data.
  • I2 (learn): Votes from several witness taggers could be fused this way so the combined silver tag sequence never contains transitions that no witness produced together.
  • I3 (world): Entity mentions matched against Wikidata or WordNet names could be added to the text's world only where the parser-driven entity detection left nothing, keeping the learned analysis primary and each addition traceable.
  • I4 (tokens): The tagger could get explicit shape or script features for numbers, mixed-script tokens and foreign words, which are cheap, transparent and helpful for rare tokens.
  • I5 (learn): Evaluation reports could peel off repair rules, teacher-tier data and ensembles step by step, with the bare pipeline serving as a built-in control.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Illinois Urbana-ChampaignUrbana · United States184 works · 69,372 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksNamed entity recognition · 200 works

Cite

Cite as Liyuan Liu, Jingbo Shang and Jiawei Han. 2019. Arabic Named Entity Recognition: What Works and What's Next. Proceedings of the Fourth Arabic Natural Language Processing Workshop.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w19-4607

@inproceedings{liu2019arabic,
  title = {{Arabic Named Entity Recognition: What Works and What's Next}},
  author = {Liyuan Liu, Jingbo Shang and Jiawei Han},
  booktitle = {Proceedings of the Fourth Arabic Natural Language Processing Workshop},
  year = {2019},
  doi = {10.18653/v1/w19-4607},
  url = {https://doi.org/10.18653/v1/w19-4607}
}

References

28 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. John Lafferty, Andrew McCallum, and Fernando CN Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. ICML. find
  2. Ahmed Abdelali, Kareem Darwish, Nadir Durrani, and Hamdy Mubarak. 2016. Farasa: A fast and furious segmenter for arabic. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, pages 11–16.
  3. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. In Proceedings of NAACL-HLT, pages 260–270. ↗
  4. Alan Akbik, Duncan Blythe, and Roland Vollgraf. 2018. Contextual string embeddings for sequence labeling. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1638–1649.
  5. Liyuan Liu, Xiang Ren, Jingbo Shang, Xiaotao Gu, Jian Peng, and Jiawei Han. 2018a. Efficient contextualized representation: Language model pruning for sequence labeling. In EMNLP. find
  6. Mikel Artetxe, Gorka Labaka, Inigo Lopez-Gazpio, and Eneko Agirre. 2018. Uncovering divergent linguistic information in word embeddings with lessons for intrinsic and extrinsic evaluation. In Proceedings of the 22nd Conference on Computational Natural Language Learning, pages 282–291. find
  7. Liyuan Liu, Jingbo Shang, Xiang Ren, Frank Fangzheng Xu, Huan Gui, Jian Peng, and Jiawei Han. 2018b. Empower sequence labeling with task-aware neural language model. In Thirty-Second AAAI Conference on Artificial Intelligence.
  8. Xuezhe Ma and Eduard Hovy. 2016. End-to-end sequence labeling via bi-directional lstm-cnns-crf. arXiv preprint arXiv:1603.01354. ↗
  9. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5:135–146. ↗
  10. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. ↗
  11. Hai Leong Chieu and Hwee Tou Ng. 2002. Named entity recognition: a maximum entropy approach using global information. In COLING. find
  12. Behrang Mohit, Nathan Schneider, Rishav Bhowmick, Kemal Oflazer, and Noah A Smith. 2012. Recall-oriented learning of named entities in arabic wikipedia. In EACL.
  13. Kareem Darwish. 2013. Named entity recognition using cross-lingual resources: Arabic as an example. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1558–1567.
  14. Nam Nguyen and Yunsong Guo. 2007. Comparisons of sequence labeling algorithms and extensions. In ICML, pages 681–688. ACM. find
  15. Timothy Dozat. 2016. Incorporating nesterov momentum into adam. find
  16. Arfath Pasha, Mohamed Al-Badrashiny, Mona T Diab, Ahmed El Kholy, Ramy Eskander, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan Roth. 2014. Madamira: A fast, comprehensive tool for morphological analysis and disambiguation of arabic. In LREC, volume 14, pages 1094–1101.
  17. Radu Florian, Abe Ittycheriah, Hongyan Jing, and Tong Zhang. 2003. Named entity recognition through classifier combination. In NAACL.
  18. Mourad Gridach. 2016. Character-aware neural networks for arabic named entity recognition for social media. In Proceedings of the 6th workshop on South and Southeast Asian natural language processing (WSSANLP2016), pages 23–32.
  19. Matthew Peters, Waleed Ammar, Chandra Bhagavatula, and Russell Power. 2017. Semi-supervised sequence tagging with bidirectional language models. In Proceedings of the ACL, volume 1, pages 1756– 1765.
  20. Nizar Y Habash. 2010. Introduction to arabic natural language processing. Synthesis Lectures on Human Language Technologies. ↗
  21. Zellig S Harris. 1954. Distributional structure. Word, 10(2-3):146–162. find
  22. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of NAACL-HLT, volume 1, pages 2227–2237.
  23. Chadi Helwe and Shady Elbassuoni. 2017. Arabic named entity recognition via deep co-learning. Artificial Intelligence Review, pages 1–19. find
  24. Lev Ratinov and Dan Roth. 2009. Design challenges and misconceptions in named entity recognition. In Proceedings of the ACL.
  25. Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirectional lstm-crf models for sequence tagging. arXiv preprint arXiv:1508.01991. ↗
  26. Nils Reimers and Iryna Gurevych. 2017. Optimal hyperparameters for deep lstm-networks for sequence labeling tasks. arXiv preprint arXiv:1707.06799. ↗
  27. Khaled Shaalan. 2014. A survey of arabic named entity recognition and classification. Computational Linguistics. ↗
  28. Jingbo Shang, Liyuan Liu, Xiaotao Gu, Xiang Ren, Teng Ren, and Jiawei Han. 2018. Learning named entity tagger using domain-specific dictionary. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2054–2064.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liu-2019-arabic-named-entity-recognition-what/