Elamur Research⌕
☰
home / papers / 2012 / Joint Segmentation and POS Tagging for Arabic Using a CRF-based Classifier

Joint Segmentation and POS Tagging for Arabic Using a CRF-based Classifier

Paper · 2012 · Souhir Gahbiche-Braham, Hélène Bonneau-Maynard, Thomas Lavergne et al.

Summary

Arabic words often fuse several units into one surface form: conjunctions, prepositions, the future marker, the article and a stem. Statistical MT therefore needs a morphological split before training. The usual tools (MADA, MorphTagger) depend on the BAMA database and SRILM, and MADA also runs many SVMTool classifiers, so they are slow on large bitexts. The authors train a single CRF with the Wapiti toolkit that predicts a composite tag per word: the main POS plus three prefix slots (conjunction; preposition, subordinator or future marker; definite article). A handful of deterministic rules then normalize and split the word. The features are only lexical windows, character prefixes and suffixes, and punctuation or digit tests, with no external analyzer. Predicting POS and segmentation jointly beat running them in sequence: POS error fell from 4.2% to 3.72%. In Arabic-to-French SMT, the joint model matched MADA and MorphTagger on BLEU, METEOR and TER, while running about 30 times faster than MADA. A light, self-contained preprocessor can therefore replace a heavy pipeline without hurting downstream quality.

Key points

  • The output label has the form POS+pr1+pr2+pr3:
    • pr1 is CONJ for w+ or f+;
    • pr2 is PREP for b+, l+ or k+, SUB for l+, or FUT for s+;
    • pr3 is DET for Al+;
    • each slot can also be none.
  • Wapiti uses L1 regularization to obtain sparse models. This keeps training tractable even though CRF cost grows quadratically with label-set size and the label set has several hundred labels.
  • The features are:
    • word unigrams in a 7-word window, bigrams in a 5-word window and trigrams in a 3-word window;
    • prefixes and suffixes of up to 5 characters in a 3-word window;
    • punctuation and digit tests in a 5-word window.
  • MT results:
    • BLEU was 32.8 for MADA-D2, 32.9 for MADA-TB, 33.2 for MorphTagger, and 32.8 / 33.1 / 33.3 for the authors' SEG, POS-then-SEG and POS+SEG setups.
    • Speed was 90 w/s for MADA, 2020 for MorphTagger and 2960 for POS+SEG.
  • Error analysis:
    • pr3 errors are mostly over-segmentation, because Al can also belong to proper nouns.
    • 6.70% of segmentation errors come from named entities being split wrongly, for example blqADy read as b+lqADy.

Interesting ideas

  • I1. Instead of a cascade of classifiers, fold closely related decisions into one composite label so that only valid combinations can be produced and the decisions inform each other.
  • I2. Detecting the article first changes how the preceding preposition is restored (llX becomes l+Al+X). In other words, one predicted slot decides how a deterministic rewrite rule applies afterwards.
  • I3. Measure a preprocessing component by its effect on the final application and by its speed, not only by intrinsic accuracy. A cheaper module that leaves downstream scores unchanged is a real win.
  • I4. Large composite label sets suffer from sparsity, and the paper counters this with features that look only at parts of the input and with L1 sparsity.
  • I5. Error analysis showed that proper nouns beginning with prefix-like letters are a systematic source of over-segmentation.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): The en tagger could jointly predict POS plus contraction or clitic splits ("don't", "'s" as possessive vs. "is") in one pass, avoiding inconsistent tag/split pairs.
  • I2 (tokens): Keep tokenization rules explainable by having the model emit a small tag that a hand-written rule then uses to choose the split, so every split traces back to a tag and a rule.
  • I3 (learn): When comparing tokenizer or parser variants, also report downstream answer accuracy and throughput, so improvements that only show up on intrinsic metrics are exposed.
  • I4 (tree): The perceptron parser's combined relation-plus-attachment labels could share weights through features defined on label components, helping rare UD relation subtypes.
  • I5 (tokens): Add a negative-control check so that capitalized or known-entity tokens are not split or lemmatized by affix rules (e.g. names ending in "-s" or "-ed").

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Centre National de la Recherche ScientifiqueParis · France169 works · 62,488 citations🔬Laboratoire d'Informatique pour la Mécanique et les Sciences de l'IngénieurOrsay · France25 works · 7,462 citations🎓Université Paris-SudOrsay · France10 works · 5,119 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 3,066 worksPart-of-speech tagging · 126 works

Cite

Cite as Souhir Gahbiche-Braham, Hélène Bonneau-Maynard, Thomas Lavergne and François Yvon. 2012. Joint Segmentation and POS Tagging for Arabic Using a CRF-based Classifier. Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{gahbichebraham2012joint,
  title = {{Joint Segmentation and POS Tagging for Arabic Using a CRF-based Classifier}},
  author = {Souhir Gahbiche-Braham, Hélène Bonneau-Maynard, Thomas Lavergne and François Yvon},
  booktitle = {Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12)},
  year = {2012},
  doi = {10.63317/4z2gophdike2},
  url = {https://doi.org/10.63317/4z2gophdike2}
}

References

30 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation, pages 65–72, Ann Arbor, Michigan.
  2. Tim Buckwalter. 2004. Buckwalter Arabic Morphological Analyzer Version 2.0. LDC2004L02. ↗
  3. Mona Diab. 2009. Second generation tools (amira 2.0): Fast and robust tokenization, POS tagging, and base phrase chunking. In Khalid Choukri and Bente Maegaard, editors, Proc. of the Second International Conference on Arabic Language Resources and Tools, Cairo, Egypt, April. The MEDAR Consortium. find
  4. Anas El Isbihani, Shahram Khadivi, Oliver Bender, and Hermann Ney. 2006. Morpho-syntactic Arabic preprocessing for Arabic to English statistical machine translation. In Human Language Technology Conf. / North American Chapter of the ACL Annual Meeting (HLT-NAACL), Workshop on SMT, pages 15–22, New York. Association for Computational Linguistics.
  5. Souhir Gahbiche-Braham, Hélène Bonneau-Maynard, and François Yvon. 2011. Two ways to use a noisy parallel news corpus for improving statistical machine translation. In Proceedings of the 4th Workshop on Building and Using Comparable Corpora: Comparable Corpora and the Web, pages 44–51, Portland, Oregon, June. Association for Computational Linguistics. find
  6. Qin Gao and Stephan Vogel. 2008. Parallel implementations of word alignment tool. In Software Engineering, Testing, and Quality Assurance for Natural Language Processing, SETQA-NLP ’08, pages 49–57.
  7. Jianfeng Gao, Galen Andrew, Mark Johnson, and Kristina Toutanova. 2007. A comparative study of parameter estimation methods for statistical natural language processing. In Proc. of the 45th Annual Meeting of the Association of Computational Linguistics, pages 824–831, Prague, Czech Republic, June. Association for Computational Linguistics. find
  8. Jesús Giménez and Lluı́s Màrquez. 2004. SVMtool: A general POS tagger generator based on support vector machines. In In Proc. of the 4th International Conference on Language Resources and Evaluation, pages 43– 46.
  9. Nizar Habash and Owen Rambow. 2005. Arabic tokenization, part-of-speech tagging and morphological disambiguation in one fell swoop. In Proc. of the 43rd Annual Meeting on ACL, ACL ’05, pages 573–580, Stroudsburg, PA, USA. Association for Computational Linguistics.
  10. Nizar Habash and Fatiha Sadat. 2006. Arabic preprocessing schemes for statistical machine translation. In Proc. of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers, NAACL-Short ’06, pages 49–52, USA. ACL.
  11. Nizar Habash, Owen Rambow, and Ryan Roth. 2010. Mada+tokan manual. Technical Report CCLS-07-01, Columbia University. ↗
  12. Nizar Habash. 2010. Introduction to Arabic Natural Language Processing. Synthesis Lectures on Human Language Technologies. Morgan & Claypool Publishers. ↗
  13. Ahmed El Kholy and Nizar Habash. 2012. Orthographic and morphological processing for english-arabic statistical machine translation. Machine Translation, 26(1-2):25–45. ↗
  14. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondřej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: open source toolkit for statistical machine translation. In Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions, ACL ’07, pages 177–180, Morristown, NJ, USA. Association for Computational Linguistics. find
  15. Seth Kulick. 2010. Simultaneous tokenization and part-of-speech tagging for Arabic without a morphological analyzer. In Proc. of the ACL 2010 Conference Short Papers, ACLShort ’10, pages 342–347, Stroudsburg, PA, USA. Association for Computational Linguistics.
  16. Seth Kulick. 2011. Exploiting separation of closed-class categories for arabic tokenization and part-of-speech tagging. ACM Transactions on Asian Language Information Processing (TALIP), 10:4. find
  17. Thomas Lavergne, Olivier Cappé, and François Yvon. 2010. Practical very large scale CRFs. In Proc. the 48th ACL, pages 504–513.
  18. Mohamed Maamouri, Ann Bies, Tim Buckwalter, Hubert Jin, and Wigdan Mekki. 2005a. Arabic Treebank: Part 3 (full corpus) v 2.0 (MPG + Syntactic Analysis). Number LDC2005T20. Linguistic Data Consortium. find
  19. Mohamed Maamouri, Ann Bies, Tim Buckwalter, Hubert Jin, and Wigdan Mekki. 2005b. Arabic Treebank: Part 4 v 1.0 (MPG Annotation). Number LDC2005T30. Linguistic Data Consortium. find
  20. Mohamed Maamouri, Dave Graff, Basma Bouziri, Sondos Krouna, Ann Bies, and Seth Kulick. 2010. LDC Standard Arabic Morphological Analyzer (SAMA) Version 3.1. Number LDC2010L01. Linguistic Data Consortium. ↗
  21. Saab Mansour, Khalil Sima’an, and Yoad Winter. 2007. find
  22. Smoothing a lexicon-based pos tagger for arabic and hebrew. In Proceedings of the 2007 Workshop on Computational Approaches to Semitic Languages: Common Issues and Resources, Semitic ’07, pages 97–103, Stroudsburg, PA, USA. Association for Computational Linguistics.
  23. Saab Mansour. 2010. Morphtagger: HMM-based Arabic segmentation for statistical machine translation. In International Workshop on Spoken Language Translation, pages 321–327, Paris, France, December.
  24. Erwin Marsi, Antal van den Bosch, and Abdelhadi Soudi. 2005. Memory-based morphological analysis generation and part-of-speech tagging of Arabic. In Proc. of the ACL 2005 workshop on computational approaches to Semitic languages, pages 1–8, Ann Arbor, USA.
  25. ThuyLinh Nguyen and Stephan Vogel. 2008. Context-based Arabic morphological analysis for machine translation. In Proc. of the 12th Conference on Computational Natural Language Learning, CoNLL ’08, pages 135– 142, Stroudsburg, PA, USA. ACL.
  26. Owen Rambow Nizar Habash and Ryan Roth. 2009. Mada+tokan: A toolkit for Arabic tokenization, diacritization, morphological disambiguation, POS tagging, stemming and lemmatization. In Khalid Choukri and Bente Maegaard, editors, Proc. of the 2nd International Conference on Arabic Language Resources and Tools, Cairo, Egypt, April. The MEDAR Consortium. find
  27. Franz Josef Och. 2003. Minimum error rate training in statistical machine translation. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, pages 160–167, Sapporo, Japan. find
  28. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pages 311–318, Stroudsburg, PA, USA. Association for Computational Linguistics.
  29. Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of the conference of the Association for Machine Translation in the America (AMTA), pages 223– 231.
  30. Andreas Stolcke. 2002. SRILM - An Extensible Language Modeling Toolkit. In Proceedings International Conference on Spoken Language Processing, pages 901–904. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/braham-2012-joint-segmentation-pos-tagging-arabic/