Elamur Research⌕
☰
home / papers / 2015 / Combining Minimally-supervised Methods for Arabic Named Entity Recognition

Combining Minimally-supervised Methods for Arabic Named Entity Recognition

Paper · 2015 · Maha Althobaiti, Udo Kruschwitz and Massimo Poesio

Summary

Supervised Arabic NER performs well, but every new domain or new entity class needs fresh expert annotation. The authors start from an observation about two cheaper alternatives. Bootstrapped semi-supervised learning (SSL) is precise but misses many entities. Wikipedia-based distant learning (DL) finds more entities but makes more mistakes.

They built one strong classifier of each kind. The SSL system grows its seed lists by learning patterns. The DL system trains a supervised model on Arabic Wikipedia text that was annotated automatically from anchor texts, redirects and inter-language links. The two outputs were then merged with equal voting, weighted voting, and the Independent Bayesian Classifier Combination (IBCC) scheme, which had earlier been proposed for sentiment analysis. The authors also introduce "restrictions" that decide when base predictions may be combined at all.

On the ANERcorp test set, IBCC beats the best single classifier by about 8 F-score points, while voting gains only 4–6. The authors present this as the first combination of minimally supervised methods for NER, and the approach transfers to new types and domains without manual labelling.

Key points

  • Base classifiers on ANERcorp:
    • SSL: P 86.03, R 51.29, F 64.27.
    • DL: P 76.44, R 56.42, F 64.92.
  • With restriction, overall IBCC reaches P 77.66, R 69.04, F 73.10. Weighted voting gets 71.76 and equal voting 69.99.
  • The SSL ranking counts the number of distinct patterns that extract a candidate, not its raw frequency. Each iteration keeps only the top m candidates, where m is the previous iteration's count plus one (for example 20 → 21 → 22).
  • Restricting the combination raised recall for every scheme without lowering precision. IBCC gave the best recall overall.
  • Significance was tested at α = 0.01 with a sign test and with bootstrap resampling over documents (10,000 replicates). All pairs differ significantly except equal vs. weighted voting without restriction (sign test p = 0.3394, bootstrap p = 0.508).

Interesting ideas

  • I1. Pair a high-precision, low-recall learner with a higher-recall, noisier learner built from different evidence, and let a combiner exploit how their errors complement each other.
  • I2. When bootstrapping, score a candidate by how many different contexts or patterns produce it, not by how often it appears, because a bad item tends to repeat in one narrow context.
  • I3. Grow the accepted set by just one item per iteration, so a single bad addition cannot quickly contaminate the next round.
  • I4. Model each base classifier's reliability probabilistically, as IBCC does, rather than giving it a fixed vote weight; this gave clearly better recall than voting.
  • I5. Treat the combination step as gated by explicit restriction rules, for example output non-entity whenever any base system says non-entity, to tune the output toward precision or recall as needed.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Silver data could deliberately join a strict rule-based witness with a broad statistical parser, so the agreement step gains coverage without losing the strict witness's precision.
  • I2 (world): Lexicon and ontology entries induced from text, such as new members of a verb class or entity type, could be admitted only when several distinct syntactic frames support them.
  • I3 (learn): Self-induced repair rules could be promoted one at a time per round, which keeps every addition traceable and limits drift.
  • I4 (learn): Witness parsers could be weighted per label, such as per dependency relation, using learned confusion behaviour instead of a flat majority vote.
  • I5 (tree): An explicit gate stating when witness arcs may be merged would let precision-critical downstream stages ask for conservative trees, and the reason for each merge decision could be logged.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of EssexColchester · United Kingdom34 works · 3,537 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksNamed entity recognition · 200 works

Cite

Cite as Maha Althobaiti, Udo Kruschwitz and Massimo Poesio. 2015. Combining Minimally-supervised Methods for Arabic Named Entity Recognition. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/tacl_a_00136

@article{althobaiti2015combining,
  title = {{Combining Minimally-supervised Methods for Arabic Named Entity Recognition}},
  author = {Maha Althobaiti, Udo Kruschwitz and Massimo Poesio},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2015},
  doi = {10.1162/tacl\_a\_00136},
  url = {https://doi.org/10.1162/tacl_a_00136}
}

References

52 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Sherief Abdallah, Khaled Shaalan, and Muhammad Shoaib. 2012. Integrating rule-based system with classification for arabic named entity recognition. In Computational Linguistics and Intelligent Text Processing, pages 311–322. Springer. find
  2. Samir AbdelRahman, Mohamed Elarnaoty, Marwa Magdy, and Aly Fahmy. 2010. Integrated machine learning techniques for arabic named entity recognition. IJCSI, 7:27–36. find
  3. Ahmed Abdul-Hamid and Kareem Darwish. 2010. Simplified feature set for Arabic named entity recognition. In Proceedings of the 2010 Named Entities Workshop, pages 110–115. Association for Computational Linguistics.
  4. Steven Abney. 2010. Semisupervised learning for computational linguistics. CRC Press. find
  5. Fahd Alotaibi and Mark Lee. 2012. Mapping Arabic Wikipedia into the named entities taxonomy. In Proceedings of COLING 2012: Posters, pages 43–52, Mumbai, India, December.
  6. The COLING 2012 Organizing Committee. find
  7. Fahd Alotaibi and Mark Lee. 2013. Automatically Developing a Fine-grained Arabic Named Entity Corpus and Gazetteer by utilizing Wikipedia. In IJCNLP.
  8. Maha Althobaiti, Udo Kruschwitz, and Massimo Poesio. 2013. A semi-supervised learning approach to arabic named entity recognition. In Proceedings of the International Conference Recent Advances in Natural Language Processing RANLP 2013, pages 32–40, Hissar, Bulgaria, September. INCOMA Ltd. Shoumen, BUL-GARIA.
  9. Maha Althobaiti, Udo Kruschwitz, and Massimo Poesio. 2014. Automatic Creation of Arabic Named Entity Annotated Corpus Using Wikipedia. In Proceedings of the Student Research Workshop at the 14th Conference of the European Chapter of the Association for Computational Linguistics (EACL), pages 106–115, Gothenburg.
  10. Marco Baroni, Brian Murphy, Eduard Barbu, and Massimo Poesio. 2010. Strudel: A Corpus-Based Semantic Model Based on Properties and Types. Cognitive Science, 34(2):222–254. ↗
  11. Eric Bauer and Ron Kohavi. 1999. An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine learning, 36(1-2):105–139. ↗
  12. Yassine Benajiba and Paolo Rosso. 2008. Arabic named entity recognition using conditional random fields. In Proc. of Workshop on HLT & NLP within the Arabic World, LREC, volume 8, pages 143–153. find
  13. Yassine Benajiba, Paolo Rosso, and José Miguel Benedı́ruiz. 2007a. Anersys: An Arabic Named Entity Recognition System based on Maximum Entropy. In Computational Linguistics and Intelligent Text Processing, pages 143–153. Springer. find
  14. Yassine Benajiba, Paolo Rosso, and José Miguel Benedı́ruiz. 2007b. Anersys: An arabic named entity recognition system based on maximum entropy. In Computational Linguistics and Intelligent Text Processing, pages 143–153. Springer. find
  15. Yassine Benajiba, Mona Diab, Paolo Rosso, et al. 2008. Arabic named entity recognition: An svm-based approach. In Proceedings of 2008 Arab International Conference on Information Technology (ACIT), pages 16–18. ↗
  16. Bob Carpenter. 2008. Multilevel bayesian models of categorical data annotation. Unpublished manuscript. Available online at http://lingpipe-blog.com/lingpipe-white-papers/, last accessed 15-March-2015. ↗
  17. Nancy Chinchor, Erica Brown, Lisa Ferro, and Patty Robinson. 1999. 1999 Named Entity Recognition Task Definition. MITRE and SAIC. find
  18. Trevor Cohn and Lucia Specia. 2013. Modelling annotator bias with multi-task gaussian processes: An application to machine translation quality estimation. In ACL, pages 32–42. find
  19. Kareem Darwish. 2013. Named Entity Recognition using Cross-lingual Resources: Arabic as an Example. In ACL, pages 1558–1567.
  20. Alexander Philip Dawid and Allan M Skene. 1979. Maximum likelihood estimation of observer error-rates using the em algorithm. Applied statistics, pages 20–28. ↗
  21. Thomas G. Dietterich. 2000a. Ensemble methods in machine learning. In Multiple Classifier Systems, volume 1857 of Lecture Notes in Computer Science, pages 1– 15. Springer Berlin Heidelberg. find
  22. Thomas G Dietterich. 2000b. An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization. Machine learning, 40(2):139–157. ↗
  23. Bradley Efron and Robert J Tibshirani. 1994. An introduction to the bootstrap. CRC press. ↗
  24. Ali Elsebai, Farid Meziane, and Fatma Zohra Belkredim. 2009. A rule based persons names arabic extraction system. Communications of the IBIMA, 11(6):53–59. find
  25. Radu Florian, Abe Ittycheriah, Hongyan Jing, and Tong Zhang. 2003. Named entity recognition through classifier combination. In Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003-Volume 4, pages 168–171. Association for Computational Linguistics.
  26. Zoubin Ghahramani and Hyun-Chul Kim. 2003. Bayesian classifier combination. Technical report, University College London. find
  27. Y Haitovsky, A Smith, and Y Liu. 2002. Modelling disagreements among and within raters assessments from the bayesian point of view. In Draft. Presented at the Valencia meeting 2002. find
  28. Hyun-Chul Kim and Zoubin Ghahramani. 2012. Bayesian classifier combination. In International conference on artificial intelligence and statistics, pages 619–627. find
  29. Abby Levenberg, Stephen Pulman, Karo Moilanen, Edwin Simpson, and Stephen Roberts. 2014. Predicting economic indicators from web text using sentiment composition. International Journal of Computer and Communication Engineering, 3(2):109–115. ↗
  30. Richard Maclin and David Opitz. 1997. An empirical evaluation of bagging and boosting. AAAI/IAAI, 1997:546–551. find
  31. Slim Mesfar. 2007. Named entity recognition for arabic using syntactic grammars. In Natural Language Processing and Information Systems, pages 305–316. Springer. find
  32. Peter Mika, Massimiliano Ciaramita, Hugo Zaragoza, and Jordi Atserias. 2008. Learning to Tag and Tagging to Learn: A Case Study on Wikipedia. volume 23, pages 26–33. find
  33. Mike Mintz, Steven Bills, Rion Snow, and Dan Jurafsky. 2009. Distant supervision for relation extraction without labeled data. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2 - Volume 2, ACL ’09, pages 1003–1011, Stroudsburg, PA, USA. Association for Computational Linguistics.
  34. David Nadeau and Satoshi Sekine. 2007. A survey of named entity recognition and classification. Lingvisticae Investigationes, 30(1):3–26. ↗
  35. David Nadeau. 2007. Semi-supervised named entity recognition: learning to recognize 100 entity types with little supervision. ↗
  36. Truc-Vien T. Nguyen and Alessandro Moschitti. 2011. End-to-end relation extraction using distant supervision from external semantic repositories. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Short Papers - Volume 2, HLT ’11, pages 277–282, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  37. Joel Nothman, Nicky Ringland, Will Radford, Tara Murphy, and James R Curran. 2013. Learning multilingual Named Entity Recognition from Wikipedia. Artificial Intelligence, 194:151–175. ↗
  38. Mai Oudah and Khaled F Shaalan. 2012. A pipeline arabic named entity recognition using a hybrid approach. In COLING, pages 2159–2176.
  39. Marius Pasca, Dekang Lin, Jeffrey Bigham, Andrei Lifchits, and Alpa Jain. 2006. Organizing and searching the world wide web of facts-step one: the one-million fact extraction challenge. In AAAI, volume 6, pages 1400–1405. find
  40. Alexander E Richman and Patrick Schone. 2008. Mining Wiki Resources for Multilingual Named Entity Recognition. In ACL, pages 1–9. find
  41. Ellen Riloff and Rosie Jones. 1999. Learning dictionaries for information extraction by multi-level bootstrapping. In AAAI, pages 474–479. find
  42. Sriparna Saha and Asif Ekbal. 2013. Combining multiple classifiers using vote based classifier ensemble technique for named entity recognition. Data & Knowledge Engineering, 85:15–39. ↗
  43. Satoshi Sekine et al. 1998. NYU: Description of the Japanese NE system used for MET-2. In Proceedings of the Seventh Message Understanding Conference (MUC-7), volume 17. find
  44. Khaled Shaalan and Hafsa Raza. 2009. Nera: Named entity recognition for arabic. Journal of the American Society for Information Science and Technology, 60(8):1652–1663. ↗
  45. Edwin Simpson, Stephen Roberts, Ioannis Psorakis, and Arfon Smith. 2013. Dynamic bayesian combination of multiple imperfect classifiers. In Decision Making and Imperfection, pages 1–35. Springer. ↗
  46. Anders Søgaard, Anders Johannsen, Barbara Plank, Dirk Hovy, and Hector Martinez. 2014. Whats in a p-value in nlp? In Proceedings of the eighteenth conference on computational natural language learning (CONLL14), pages 1–10. find
  47. Sam Tardif, James R. Curran, and Tara Murphy. 2009. Improved Text Categorisation for Wikipedia Named Entities. In Proceedings of the Australasian Language Technology Association Workshop, pages 104–108. find
  48. Sergey Tulyakov, Stefan Jaeger, Venu Govindaraju, and David Doermann. 2008. Review of classifier combination methods. In Machine Learning in Document Analysis and Recognition, pages 361–386. Springer. ↗
  49. Merijn Van Erp, Louis Vuurpijl, and Lambert Schomaker. 2002. An overview and comparison of voting methods for pattern recognition. In Eighth International Workshop on Frontiers in Handwriting Recognition, pages 195–200. IEEE. find
  50. Hans Van Halteren, Walter Daelemans, and Jakub Zavrel. 2001. Improving accuracy in word class tagging through the combination of machine learning systems. Computational linguistics, 27(2):199–229. ↗
  51. Wajdi Zaghouani. 2014. Critical Survey of the Freely Available Arabic Corpora. In Workshop on Free/Open-Source Arabic Corpora and Corpora Processing Tools, pages 1–8, Reykjavik, Iceland. ↗
  52. Tong Zhang, Fred Damerau, and David Johnson. 2002. Text chunking based on a generalization of winnow. The Journal of Machine Learning Research, 2:615– 637. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/althobaiti-2015-combining-minimally-supervised-methods-arabic/