Elamur Research⌕
☰
home / papers / 2013 / Traitement automatique des entités nommées en arabe: détection et traduction [Automatic processing of Arabic named entities: detection and translation]

Traitement automatique des entités nommées en arabe: détection et traduction [Automatic processing of Arabic named entities: detection and translation]

Paper · 2013 · Souhir Gahbiche-Braham, Hélène Bonneau-Maynard and François Yvon

Summary

This paper examines how named entities (person, location, organization) in Arabic news text can be detected and then handled better by a statistical Arabic→French translation system. Arabic makes detection hard. It has no capital letters, many given names are also ordinary nouns or adjectives, articles and prepositions can attach to names, and spellings and transliterations vary from region to region. The work was done within the SAMAR project on AFP Arabic newswire. The authors built a statistical detector, NERAr, and coupled it with a morphological segmenter. They improved it by adding more features and by unsupervised adaptation. They then tried several ways of passing the detected entities to the translation system, using bilingual proper-name dictionaries compiled during SAMAR. In the best configuration, automatic metrics and manual checks agree on two gains: entity translation improves slightly, and fewer errors come from unknown words. This matters because news keeps introducing new names. The approach avoids having to retrain the translation system every time new entities appear.

Key points

  • The baseline translation system is trained on 265,000 Arabic–French sentence pairs from AFP (December 2009 to July 2012). December 2010 was held out to build the dev and test sets.
  • About 25% of unknown forms in this application are named entities. Earlier work by Habash reported roughly 40% proper names among unknown words for Arabic–English.
  • The detector found 37,446 distinct entities up to March 2012. After that, around 100 new ones appeared each week, totaling 1,599 over 16 weeks: 1,122 persons, 366 locations and 111 organizations.
  • Arcade II is a parallel corpus with entity annotation on both the Arabic and French sides. On it, the baseline translated only 55.41% of entities correctly: LOC 70.38%, PERS 51.24%, ORG 44.63%.
  • NERAr was developed in three stages: a baseline, a version with a larger feature set, and a version with unsupervised adaptation. Integration with translation was tested on two different test corpora.

Interesting ideas

  • I1. The authors score entity translation cheaply: for each sentence, they search the system output for every entity in the reference, which yields per-type accuracy without any human judging.
  • I2. They chart how many previously unseen entities arrive each week, which shows that names keep coming and justifies external dictionaries over retraining.
  • I3. Unknown words are first sorted into categories, then each category is handled differently: segment the decomposable ones, and route names to transliteration or dictionary lookup instead of copying them blindly.
  • I4. Names that are also common words (Nour, which also means "light"; Farhan, which also means "glad") are a key source of ambiguity when surface cues like capitals are missing.
  • I5. One name can have several regional spellings (Jamila, Djamila, Gamila), which makes plain name lists less useful.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (back): After text → graph → text regeneration, check whether each entity of the source world reappears in the output, reporting results per entity type as a fast fidelity metric.
  • I2 (world): Track how often unseen entities appear across incoming texts, to decide when the Wikidata-backed ontology and knowledge graph need updating rather than the models themselves.
  • I3 (tokens): Classify each out-of-vocabulary token (morphological variant, proper name, typo) before lemmatizing, so names are not forced through a guessed lemma.
  • I4 (frames): When capitalization is unreliable (headlines, all-caps text, social media), let the semantic role a word fills in its frame help decide whether it is a person name or an ordinary word.
  • I5 (world): Store spelling variants as aliases of a single entity node, so coreference and holdings attach to one entity rather than splitting across spellings.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 works

🧭 Topics 2Research subjects and languages the work is about.Machine translation · 1,105 worksSemitic languages · 3,066 works

Cite

Cite as Souhir Gahbiche-Braham, Hélène Bonneau-Maynard and François Yvon. 2013. Traitement automatique des entités nommées en arabe: détection et traduction [Automatic processing of Arabic named entities: detection and translation]. Traitement Automatique des Langues.

The paper itself, not this page. BibTeX from the library's record of the work.

@misc{gahbichebraham2013traitement,
  title = {{Traitement automatique des entités nommées en arabe: détection et traduction [Automatic processing of Arabic named entities: detection and translation]}},
  author = {Souhir Gahbiche-Braham, Hélène Bonneau-Maynard and François Yvon},
  year = {2013},
  url = {https://aclanthology.org/2013.tal-2.5/}
}

References

62 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abdul Hamid A., Darwish K., « Simplified Feature Set for Arabic Named Entity Recognition », Proc. of the 2010 Named Entities Workshop, Uppsala, p. 110-115, July, 2010. find
  2. Abdul Rauf S., Efficient corpus selection for Statistical Machine Translation, PhD thesis, Université du Maine, Le Mans, 2012. find
  3. Al-Jumaily H. T., Martínez P., Martínez-Fernández J. L., der Goot E. V., « A real time Named Entity Recognition system for Arabic text mining », Language Resources and Evaluation, vol. 46, no 4, p. 543-563, 2012. find
  4. Al-Onaizan Y., Knight K., « Machine transliteration of names in Arabic text », Proc. of the ACL-02 workshop on Computational approaches to semitic languages, p. 1-13, 2002a. find
  5. Al-Onaizan Y., Knight K., « Translating named entities using monolingual and bilingual resources », Proc. of the 40th Annual Meeting on ACL, ACL ’02, Philadelphia, PA, USA, p. 400-408, 2002b. 24. Nous remercions un des relecteurs pour ses remarques sur l’interaction entre ces deux composants de notre système. find
  6. Allauzen A., Modélisation linguistique pour l’indexation automatique de documents audiovisuels, PhD thesis, Université Paris Sud, Orsay, 2003. find
  7. Banerjee S., Lavie A., « METEOR : An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments », Proc. of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation, Ann Arbor, Michigan, p. 65-72, 2005. — METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments
  8. Béchet F., Sagot B., Stern R., « Coopération de méthodes statistiques et symboliques pour l’adaptation non supervisée d’un système d’étiquetage en entités nommées », actes de la conférence TALN, Montpellier, France, 2011. find
  9. Bellagarda J. R., « An overview of statistical language model adaptation », Proc. of the ISCA Workshop on Adaptation Methods for Speech Recognition, Sophia Antipolis, p. 165-174, 2001. find
  10. Benajiba Y., Diab M., Rosso P., « Arabic named entity recognition using optimized feature sets », Proc. of EMNLP, EMNLP, Stroudsburg, PA, USA, p. 284-293, 2008.
  11. Benajiba Y., Rosso P., « Conquering the NER task for the Arabic language by combining the Maximum Entropy with POS-tag information », Proceedings of Workshop on Natural Language-Independant Engineering, IJCAI, 2007. find
  12. Benajiba Y., Rosso P., « Arabic named entity recognition using Conditional Random Fields », Proceedings of the Conference on Language Resources and Evaluation, 2008. find
  13. Benajiba Y., Rosso P., Benedí J.-M., « ANERsys : An Arabic Named Entity Recognition System Based on Maximum Entropy », CICLing, p. 143-153, 2007. find
  14. Blitzer J., Domain adaptation of natural language processing systems, PhD thesis, University Philadelphia, PA, USA, 2008. find
  15. Buckwalter T., Buckwalter Arabic Morphological Analyzer Version 1.0, Catalog No. LDC2002L49, Linguistic Data Consortium, University of Pennsylvania, 2002. find
  16. Che W., Wang M., Manning C. D., Liu T., « Named Entity Recognition with Bilingual Constraints », NAACL-HLT, 2013. find
  17. Chiao Y.-c., Kraif O., Laurent D., Minh T., Nguyen H., Semmar N., Stuck F., Véronis J., Zaghouani W., « Evaluation of multilingual text alignment systems : the ARCADE II project », In Proc. of LREC, 2006. find
  18. Chinchor N., Robinson P., « MUC-7 Named Entity Task Definition », Proceedings of the seventh Message Understanding Conference (MUC 7), 1997. find
  19. Daumé III H., « Frustratingly Easy Domain Adaptation », Proc. of the 45th Annual Meeting of the ACL, Prague, Czech Republic, June, 2007. find
  20. Daumé III H., Deoskar T., McClosky D., Plank B., Tiedemann J. (eds), Proc. of the 2010 Workshop on Domain Adaptation for NLP, Uppsala, Sweden, July, 2010. find
  21. Daumé III H., Marcu D., « Domain Adaptation for Statistical Classifiers », Journal of Artificial Intelligence Research, vol. 26, p. 101-126, 2006. find
  22. Diab M., « Second Generation Tools (AMIRA 2.0) : Fast and Robust Tokenization, POS tagging, and Base Phrase Chunking », in K. Choukri, B. Maegaard (eds), Proc. of the Second International Conference on Arabic Language Resources and Tools, The MEDAR Consortium, Cairo, Egypt, April, 2009. find
  23. Fehri H., Haddar K., Hamadou A. B., « Recognition and Translation of Arabic Named Entities with NooJ Using a New Representation Model », in M. Constant, A. Maletti, A. Savary (eds), FSMNLP, 9th International Workshop, ACL, Bois, France, p. 134-142, 2011.
  24. Traitement automatique des EN en arabe TAL Volume 54 – n ◦ 2/2013 find
  25. Fung P., McKeown K., « Finding Terminology Translations from Non-parallel Corpora », Proceedings of the 5th Annual Workshop on Very Large Corpora, Hong-Kong, p. 192-202, 1997. find
  26. Gahbiche-Braham S., Bonneau-Maynard H., Yvon F., « Two Ways to Use a Noisy Parallel News Corpus for Improving Statistical Machine Translation », Proc. of Workshop on Building and Using Comparable Corpora, Portland, OR, p. 44-51, 2011. find
  27. Gahbiche-Braham S., Maynard H., Lavergne T., Yvon F., « Joint Segmentation and POS Tagging for Arabic Using a CRF-based Classifier », in N. Calzolari, K. Choukri, T. Declerck, M. U. Dogan, B. Maegaard, J. Mariani, J. Odijk, S. Piperidis (eds), Proc. of the 8th LREC, ELRA, Istanbul, Turkey, 2012. find
  28. Gao Q., Vogel S., « Parallel implementations of word alignment tool », In Software Engineering, Testing, and Quality Assurance for Natural Language Processing, p. 49-57, 2008.
  29. Habash N., « Four techniques for online handling of out-of-vocabulary words in Arabic-English statistical machine translation », Proc. of the 46th Annual Meeting of the Association for Computational Linguistics on Human Language Technologies, p. 57-60, 2008. find
  30. Habash N., Arabic Natural Language Processing, Synthesis Lectures on Human Language Technologies, organ & Claypool Publishers, 2011. find
  31. Habash N., Rambow O., Roth R., « MADA+TOKAN : A Toolkit for Arabic Tokenization, Diacritization, Morphological Disambiguation, POS Tagging, Stemming and Lemmatization », in K. Choukri, B. Maegaard (eds), Proc. of the 2nd International Conference on Arabic Language Resources and Tools, The MEDAR Consortium, Cairo, Egypt, April, 2009. find
  32. Hal D., Jagarlamudi J., « Domain adaptation for machine translation by mining unseen words », Proceedings of the 49th Annual Meeting of the ACL :HLT : short papers - Volume 2, HLT ’11, ACL, Stroudsburg, PA, USA, p. 407-412, 2011. find
  33. Hálek O., Rosa R., Tamchyna A., Bojar O., « Named Entities from Wikipedia for Machine Translation », in M. Lopatková (ed.), ITAT 2011 Information Technologies – Applications and Theory, vol. 788, p. 23-30, September, 2011. find
  34. Hassan A., Fahmy H., Hassan H., « Improving Named Entity Translation by Exploiting Comparable and Parallel Corpora », Proc. of the Conference on Recent Advances in Natural Language Processing (RANLP ’07), Borovets, Bulgaria, 2007. find
  35. Hassan H., Sorensen J., « An integrated approach for Arabic-English named entity translation », Proceedings of the ACL Workshop on Computational Approaches to Semitic Languages, Semitic ’05, ACL, Stroudsburg, PA, USA, p. 87-93, 2005. find
  36. Heintz I., « Arabic Language Modeling with Finite State Transducers », Proc. of the ACL-08 : HLT Student Research Workshop, ACL, Columbus, Ohio, p. 37-42, June, 2008. find
  37. Hermjakob U., Knight K., Daumé III H., « Name Translation in Statistical Machine Translation - Learning When to Transliterate », Proc. of ACL-08 : HLT, Ohio, p. 389-397, June, 2008. find
  38. Jiang J., Zhai C., « Instance Weighting for Domain Adaptation in NLP », Proc. of the 45th Annual Meeting of the ACL, Prague, Czech Republic, p. 264-271, June, 2007.
  39. Jiang L., Zhou M., Chien L.-F., Niu C., « Named Entity Translation with Web Mining and Transliteration », Proc. of the 20th International Joint Conference on Artificial Intelligence, p. 1629-1634, 2007. find
  40. Kashani M. M., Joanis E., Kuhn R., Foster G., Popowich F., « Integration of an Arabic transliteration module into a statistical machine translation system », Proc. of the 2nd Workshop on Statistical Machine Translation, StatMT ’07, ACL, Stroudsburg, PA, USA, p. 17-24, 2007.
  41. Koehn P., Hoang H., Birch A., Callison-Burch C., Federico M., Bertoldi N., Cowan B., Shen W., Moran C., Zens R., Dyer C., Bojar O., Constantin A., Herbst E., « Moses : open source toolkit for statistical machine translation », In Proceedings of the 45th Annual Meeting of the ACL, p. 177-180, 2007. find
  42. Lafferty J., McCallum A., Pereira F., « Conditional Random Fields : Probabilistic Models for Segmenting and Labeling Sequence Data », Proceedings of the 18th International Conference on Machine Learning (ICML), Morgan Kaufmann, San Francisco, CA, p. 282-289, 2001. find
  43. Lavergne T., Cappé O., Yvon F., « Practical Very Large Scale CRFs », Proceedings of the 48th Annual Meeting of the ACL, p. 504-513, July, 2010.
  44. Lavie A., Agarwal A., « METEOR : An Automatic Metric for MT Evaluation with High Levels of Correlation with Human Judgments », Proc. of the ACL Workshop on Statistical Machine Translation, Prague, Czech Republic, p. 338-231, 2007. — METEOR: An Automatic Metric for MT Evaluation with High Levels of Correlation with Human Judgments
  45. Ling W., Calado P., Martins B., Trancoso I., Black A., Coheu L., « Named Entity Translation using Anchor Texts », Proc. of the IWSLT, San Francisco, USA, 2011. find
  46. Maamouri M., Bies A., Buckwalter T., Jin H., Mekki W., Arabic Treebank : Part 3 (full corpus) v 2.0 (MPG + Syntactic Analysis), no LDC2005T20, Linguistic Data Consortium, 2005a. find
  47. Maamouri M., Bies A., Buckwalter T., Jin H., Mekki W., Arabic Treebank : Part 4 v 1.0 (MPG Annotation), no LDC2005T30, Linguistic Data Consortium, 2005b. find
  48. Maloney J., Niv M., « TAGARAB : a fast, accurate Arabic name recognizer using high-precision morphological analysis », Proc. of the Workshop on Computational Approaches to Semitic Languages, Semitic ’98, Stroudsburg, PA, USA, p. 8-15, 1998. — TAGARAB: A Fast, Accurate Arabic Name Recognizer Using High-Precision Morphological Analysis
  49. Mansour S., « MorphTagger : HMM-Based Arabic Segmentation for Statistical Machine Translation », IWSLT, Paris, France, p. 321-327, December, 2010. find
  50. Mihalcea R., « Co-training and Self-training for Word Sense Disambiguation », in H. T. Ng, E. Riloff (eds), HLT-NAACL Workshop : CoNLL-2004, Boston, p. 33-40, May, 2004. find
  51. Moore R. C., « Learning translations of named-entity phrases from parallel corpora », Proceedings of the tenth conference on European chapter of the ACL - Volume 1, EACL ’03, ACL, Stroudsburg, PA, USA, p. 259-266, 2003. find
  52. Papineni K., Roukos S., Ward T., Zhu W.-J., « BLEU : a method for automatic evaluation of machine translation », Proc. of the 40th Annual Meeting on ACL, Stroudsburg, PA, USA, p. 311-318, 2002. — Bleu: a Method for Automatic Evaluation of Machine Translation
  53. Rapp R., « Identifying word translations in non-parallel texts », Proceedings of the 33rd annual meeting on ACL, ACL ’95, ACL, Stroudsburg, PA, USA, p. 320-322, 1995. find
  54. Samy D., Moreno A., Ma Guirao J., « A Proposal For An Arabic Named Entity Tagger Leveraging a Parallel Corpus », Proceedings of RANLP’05, 2005. find
  55. Santanu P., Kumar Naskar S., Pecina P., Bandyopadhyay S., Way A., « Handling Named Entities and Compound Verbs in Phrase-Based Statistical Machine Translation », Proc. of the COLING 2010 Workshop on Multiword Expressions : from Theory to Applications (MWE 2010), Beijing, China, p. 46-54, August, 2010. find
  56. Shaalan K., Raza H., « NERA : Named Entity Recognition for Arabic », Journal of the American Society for Information Science and Technology, vol. 60, no 9, p. 1652-1663, 2009. find
  57. Sokolovska N., Cappé O., Yvon F., « Sélection de caractéristiques pour les champs aléatoires conditionnels par pénalisation L 1 », TAL, vol. 50, no 3, p. 139-171, 2009. find
  58. Traitement automatique des EN en arabe TAL Volume 54 – n ◦ 2/2013 find
  59. Zaghouani W., Pouliquen B., Ebrahim M., Steinberger R., « Adapting a resource-light highly multilingual Named Entity Recognition system to Arabic », Proc. of the Seventh conference on International Language Resources and Evaluation (LREC’10), p. 563-567, 2010.
  60. Zhang M., Li H., Kumaran A., Liu M., « Report of NEWS2011 Machine Transliteration Shared Task », Proc. of the 2011 Named Entities Workshop, 2011. find
  61. Zhang Y., The Application of Source Language Information in Statistical Machine Translation, PhD thesis, RWTH Aachen University, 2012. find
  62. Zitouni I., Sorensen J., Luo X., Florian R., « The Impact of Morphological Stemming on Arabic Mention Detection and Coreference Resolution », Proc. of Workshop on Computational Approaches to Semitic Languages, Ann Arbor, Michigan, p. 63-70, June, 2005.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/braham-2013-traitement-automatique-des-entites-nommees/