Choosing an Optimal Architecture for Segmentation and POS-Tagging of Modern Hebrew

What it is about

Bar-Haim et al. build an HMM segmenter and POS tagger for Modern Hebrew from a morphological analyser and a very small tagged corpus (the Hebrew Treebank, from Ha'aretz, with under 30K words per training fold), plus about 340K untagged words. They compare two tokenisations: a word-level model, where each tag is a whole segmentation pattern such as IN-H-NN, and a morpheme-level model, where the tags are plain POS and the input is the segments. The morpheme model tags better (90.21 against 89.42 word-level, 5-fold cross-validation) but segments no better. Error analysis shows that the covert definite article h causes many errors: b-bit ("in a house") against b-h-bit ("in the house"). Their best model, M+h, makes the definiteness marker a feature of the stem's tag and reaches 90.51 tagging and 96.74 segmentation accuracy. A lexical model interpolated with an unsupervised estimate (weight 0.85 on the annotated counts) adds 1.5 points to tagging. The analyser misses 3.78% of test words, which caps accuracy at 96.22%.

Ideas

• I1. Definiteness as a stem feature: when b, k or l precedes a noun, decide between the bare and the h-marked reading from lexical counts of definite and indefinite uses. The paper's example lmmflh (25 occurrences, only 2 indefinite) shows the signal. This is a count table for languages/he/06-prepositions-particles.md, with the spelling side in languages/he/02-script-and-spelling.md.
• I2. Interpolate the lexical model: relative frequency from the annotated data with weight 0.85, plus an unsupervised estimate from a large untagged corpus. The same interpolation can be measured on the lex table of tools/hbo, on held-out text.
• I3. A minimal-pair test: bbit hgdwl ("in the big house") against bbit gdwl ("in a big house"). The adjective's definiteness decides the segmentation. Use as a test case for Modern Hebrew segmentation in languages/he/11-ud-analysis.md.

For Mova

I1 goes into languages/he/06-prepositions-particles.md and languages/he/02-script-and-spelling.md. I2 is an idea for the lex table in tools/hbo and for the Modern Hebrew notes in languages/he/11-ud-analysis.md, to be measured. I3 is a test case for languages/he/11-ud-analysis.md. The paper's tagger is an HMM; Mova would take the tokenisation choice and the lexical smoothing, not the tagger.

Topics: Part-of-speech tagging

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.3115/1621787.1621796

@inproceedings{barhaim2005choosing,
  title = {{Choosing an Optimal Architecture for Segmentation and POS-Tagging of Modern Hebrew}},
  author = {Bar-Haim, Roy and Sima'an, Khalil and Winter, Yoad},
  booktitle = {Proceedings of the ACL Workshop on Computational Approaches to Semitic Languages},
  year = {2005},
  doi = {10.3115/1621787.1621796},
  url = {https://doi.org/10.3115/1621787.1621796}
}

References

14 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Meni Adler. 2001. Hidden Markov Model for Hebrew part-of-speech tagging. Master’s thesis, Ben Gurion University, Israel. In Hebrew. find
2. Leonard Baum. 1972. An inequality and associated maximization technique in statistical estimation for probabilistic functions of a Markov process. In Inequalities III:Proceedings of the Third Symposium on Inequalities, University of California, Los Angeles, pp.1-8. find
3. Eric Brill. 1995. Transformation-based error-driven learning and natural language processing: A case study in part-of-speech tagging. Computational Linguistic, 21:784–789. find our card
4. David Carmel and Yoelle Maarek. 1999. Morphological disambiguation for Hebrew search systems. In Proceedings of the 4th international workshop,NGITS-99. find
5. Eugene Charniak, Curtis Hendrickson, Neil Jacobson, and Mike Perkowitz. 1993. Equations for part-of-speech tagging. In National Conference on Artificial Intelligence, pages 784–789. find
6. K. W. Church. 1988. A stochastic parts program and noun phrase parser for unrestricted text. In Proc. of the Second Conference on Applied Natural Language Processing, pages 136–143, Austin, TX. find
7. Mona Diab, Kadri Hacioglu, and Daniel Jurafsky. 2004. Automatic tagging of Arabic text: From raw text to base phrase chunks. In HLT-NAACL 2004: Short Papers, pages 149–152. find
8. S.M. Katz. 1987. Estimation of probabilities from sparse data from the language model component of a speech recognizer. IEEE Transactions of Acoustics, Speech and Signal Processing, 35(3):400–401. find
9. Young-Suk Lee, Kishore Papineni, Salim Roukos, Ossama Emam, and Hany Hassan. 2003. Language model based Arabic word segmentation. In ACL, pages 399–406. find
10. M. Levinger, U. Ornan, and A. Itai. 1995. Morphological disambiguation in Hebrew using a priori probabilities. Computational Linguistics, 21:383–404. find
11. Moshe Levinger. 1992. Morphological disambiguation in Hebrew. Master’s thesis, Computer Science Department, Technion, Haifa, Israel. In Hebrew. find
12. Erel Segal. 2000. Hebrew morphological analyzer for Hebrew undotted texts. Master’s thesis, Computer Science Department, Technion, Haifa, Israel. http://www.cs.technion.ac.il/-∼erelsgl/bxi/hmntx/teud.html. ↗
13. K. Sima’an, A. Itai, Y. Winter, A. Altman, and N. Nativ. 2001. Building a tree-bank of Modern Hebrew text. Traitment Automatique des Langues, 42:347–380. find
14. Andreas Stolcke. 2002. SRILM - an extensible language modeling toolkit. In ICSLP, pages 901–904, Denver, Colorado, September. find