Elamur Research⌕
☰
home / papers / 2014 / Combining strategies for tagging and parsing Arabic

Combining strategies for tagging and parsing Arabic

Paper · 2014 · Maytham Alabbas and Allan Ramsay

Summary

The paper asks how to merge the outputs of several Arabic POS taggers and dependency parsers so that the ensemble beats every member. The data is Penn Arabic Treebank (PATB) Part 1 v3.0, about 165K words. Three taggers are combined: AMIRA, MADA and MXL, an affix-based maximum-likelihood tagger. Each tagger's output is first cleaned with transformation-based retagging (TBR). The authors then build per-tag reliability tables: for every tag, how often is this tagger right when it predicts it? For each word, the prediction of the tagger with the highest reliability wins. On a coarse 39-tag set this gives 0.995 accuracy, against 0.967 for the best single tagger. On the fine 305-tag set it gives 0.96, against 0.936. It also beats several majority-voting variants. Applied to parsing (MSTParser plus two MALTParser variants), picking the most reliable parser per arc fails. What works is trusting a pair of similar parsers where they agree and deferring to a structurally different third parser where they disagree. The general lesson is that an ensemble's design should depend on whether its outputs interact.

Key points

  • Tagging data and setup: PATB Part 1 v3.0 (~165K words), 10-fold cross-validation, about 150K words for training and 15K for testing per fold. Two tagsets were used: a coarse 39-tag set that extends the Bies tagset, and the fine Buckwalter-based tagset with 305 tags.
  • Single-tagger accuracy after TBR: AMIRA 0.953 / 0.888, MXL 0.956 / 0.912, MADA 0.967 / 0.936 (coarse / fine). TBR helped AMIRA most, largely by repairing Penn-style labels such as JJ into PATB labels such as ADJ.
  • Combination results, coarse / fine:
    • "Just most confident" reached 0.995 / 0.96.
    • Majority voting with a most-confident backoff reached 0.992 / 0.945.
    • Majority voting with a fixed backoff tagger reached 0.975–0.982 / 0.906–0.918.
    • Backing off to MXL beat backing off to MADA, even though MADA is the stronger tagger alone.
  • Parsing setup and single parsers: 5000 PATB sentences, 5-fold cross-validation. Labelled accuracy was MSTParser 0.816, MALTParser 1 (Nivre arc-eager) 0.797, MALTParser 2 (stack-eager) 0.796.
  • Parsing combinations:
    • The best scheme used the two MALTParsers when they agreed and MSTParser otherwise, reaching 0.848.
    • Choosing the most confident parser per arc gave 0.789, below every individual parser.

Interesting ideas

  • I1. Instead of fixed per-system weights, measure each system's precision per predicted label on training data and let the most reliable prediction win token by token
  • I2. Use a learned rule-based retagging pass (TBR) to find systematic label renamings between a tool's tagset and the target tagset, such as JJ versus ADJ, before combining tools
  • I3. In an ensemble, the voters' agreement matters more than their individual strength: two similar systems agreeing is weak evidence, so the fallback should be the most dissimilar system, not the best one
  • I4. Choosing each local decision independently can produce outputs that are plausible piece by piece but globally incoherent, which is why per-arc selection hurt parsing
  • I5. Disagreement between near-identical models can serve as a signal that the input itself is anomalous at that point

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): Running several taggers or lemmatizers side by side, a per-tag precision table could decide each token, and the winning system and its precision would serve as a human-readable explanation.
  • I2 (tree): When bringing in treebanks or parsers that use another UD dialect, induced transformation rules could find and fix consistent label mismatches automatically and keep them as inspectable rules.
  • I3 (learn): When building silver data from several witness parsers, agreement between close relatives (e.g. two perceptron variants) should be discounted, and disagreements should go to the architecturally different gparse.
  • I4 (tree): Any arc-level merge of parser outputs should be followed by a global well-formedness check (single root, tree shape, projectivity where expected) rather than trusting the best arcs in isolation.
  • I5 (learn): Sentences where sibling parsers diverge could be routed to an LLM teacher tier or to repair-rule induction, rather than being added to silver data.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of ManchesterManchester · United Kingdom87 works · 44,938 citations🎓University of BasrahBasra · Iraq3 works · 42 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Maytham Alabbas and Allan Ramsay. 2014. Combining strategies for tagging and parsing Arabic. Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/w14-3609

@inproceedings{alabbas2014combining,
  title = {{Combining strategies for tagging and parsing Arabic}},
  author = {Maytham Alabbas and Allan Ramsay},
  booktitle = {Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP)},
  year = {2014},
  doi = {10.3115/v1/w14-3609},
  url = {https://doi.org/10.3115/v1/w14-3609}
}

References

17 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. E Brill. 1995. Transformation-based error-driven learning and natural language processing: a case study in part of speech tagging. Computational Linguistics, 23(4):543–565.
  2. T Buckwalter. 2004. Buckwalter Arabic morphological analyzer version 2.0. Linguistic Data Consortium. find
  3. T Buckwalter. 2007. Issues in Arabic morphological analysis. ARabic computational morphology, pages 23–41. ↗
  4. M. Diab. 2009. Second Generation Tools (AMIRA 2.0): Fast and Robust Tokenization, POS Tagging, and Base Phrase Chunking. In Proceedings of the Second International Conference on Arabic Language Resources and Tools, pages 285–288, Cairo, Eygpt, April. The MEDAR Consortium. find
  5. S Green and C D Manning. 2010. Better arabic parsing: Baselines, evaluations, and analysis. In Proceedings of the 23rd International Conference on Computational Linguistics, COLING ’10, pages 394–402, Stroudsburg, PA, USA. Association for Computational Linguistics. N. Habash, O. Rambow, and R. Roth. MADA+TOKAN: A Toolkit for Arabic Tokenization, Diacritization, Morphological Disambiguation, POS Tagging, Stemming and Lemmatization. In Proceedings of the Second International Conference on Arabic Language Resources and Tools, Cairo. The MEDAR Consortium.
  6. J C Henderson and E Brill. 2000. Exploiting diversity in natural language processing: Combining parsers. CoRR, cs.CL/0006003. ↗
  7. T Lager. 1999. µ-tbl lite: a small, extendible transformation-based learner. In Proceedings of the 9th European Conference on Computational Linguistics (EACL-99), pages 279–280, Bergen. Association for Computational Linguistics. find
  8. M Maamouri and A Bies. 2004. Developing an Arabic treebank: methods, guidelines, procedures, and tools. In Proceedings of the Workshop on Computational Approaches to Arabic Script-based Languages, pages 2–9, Geneva.
  9. R McDonald, K Lerman, and F Pereira. 2006a. Multilingual dependency parsing with a two-stage discriminative parser. In Tenth Conference on Computational Natural Language Learning (CoNLL-X), New York. find
  10. R McDonald, K Lerman, and F Pereira. 2006b. Multilingual dependency parsing with a two-stage discriminative parser. In Tenth Conference on Computational Natural Language Learning (CoNLL-X), New York. find
  11. J. Nivre, J. Hall, and J. Nilsson. 2006. MaltParser: A data-driven parser-generator for dependency parsing. In Proceedings of the International Conference on Language Resources and Evaluation (LREC), volume 6, pages 2216–2219.
  12. J. Nivre, J. Hall, J. Nilsson, A. Chanev, G. Eryigit, S. Kübler, S. Marinov, and E. Marsi. 2007. Malt-Parser: A language-independent system for data-driven dependency parsing. Natural Language Engineering, 13(02):95–135. find
  13. J Nivre, L Rimell, R McDonald, and C Gómez-Rodrı́guez. 2010. Evaluation of dependency parsers on unbounded dependencies. In Proceedings of the 23rd International Conference on Computational Linguistics, COLING ’10, pages 833–841, Beijing.
  14. A. Ramsay and Y. Sabtan. 2009. Bootstrapping a lexicon-free tagger for Arabic. In Proceedings of the 9th Conference on Language Engineering, pages 202–215, Cairo, Egypt, December. find
  15. Claudio De Stefano, Antonio Della Cioppa, and Angelo Marcelli. 2002. An adaptive weighted majority vote rule for combining multiple classifiers. In ICPR (2), pages 192–195. find
  16. Kevin Woods, W. Philip Kegelmeyer, Jr., and Kevin Bowyer. 1997. Combination of multiple classifiers using local accuracy estimates. IEEE Trans. Pattern 2009. Anal. Mach. Intell., 19(4):405–410, April. ↗
  17. D. Zeman and Z. Žabokrtskỳ. 2005. Improving parsing accuracy by combining diverse dependency parsers. In Proceedings of the Ninth International Workshop on Parsing Technology, pages 171–178. Association for Computational Linguistics. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/alabbas-2014-combining-strategies-tagging-parsing-arabic/