Elamur Research
☰
home / papers / 2004 / Applying Conditional Random Fields to Japanese Morphological Analysis

Applying Conditional Random Fields to Japanese Morphological Analysis

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" does not allow redistributing the text itself. Original — aclanthology.org/W04-3230.pdf

Summary

Japanese is written without spaces, so a morphological analyzer has to decide word boundaries and part-of-speech tags together. Earlier CRF work assumed the token sequence was already fixed, so CRFs could not be applied directly. Kudo, Yamamoto and Matsumoto reformulate CRFs over a lexicon-derived word lattice rather than character-level B/I tagging. In their version, candidate paths may contain different numbers of tokens, and one globally normalized model scores each complete path. This lets them combine many overlapping features: levels of the hierarchical POS tagset, conjugation form and type, affixes and character types. Generative HMMs struggle to use such features. The model also avoids the label and length biases that push MEMMs toward low-entropy or short paths. On two standard annotated corpora, KC and RWCP, using the same data as earlier HMM and MEMM studies, CRFs achieve the best segmentation and tagging scores. The approach shows that discriminative sequence models can exploit a lexicon, and it carries over to other unsegmented languages such as Chinese or Thai.

Key points

  • On KC, L2-CRFs score F=98.96 for segmentation, 98.31 for top-level POS and 96.75 for full tags. MEMMs (Uchimoto 01) reach 96.44/95.81/94.27, rule-based JUMAN 98.70/98.09/93.73, and a bigram HMM 96.22/94.96/91.85.
  • On RWCP, L2-CRFs score 99.11/98.73/97.66, ahead of the extended HMMs of Asahara 00 (98.87/98.33/96.95).
  • Length bias in MEMMs shows up in the error counts. MEMMs output too-long tokens 416 times, which is 70% of their errors. CRFs do so 79 times and HMMs 306 times.
  • L1-regularized CRFs are slightly less accurate but use about 1/8–1/6 as many active features: 90,163 vs 791,798 on KC and 101,757 vs 580,032 on RWCP.
  • The IPA tagset used by ChaSen has roughly 500 distinct labels when hierarchy branches are counted. CRF feature templates cover these levels without the hand-tuned smoothing that HMMs require.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Computing & GPUs

Topics: Part-of-speech tagging

Cited by

199 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 195 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{kudo2004applying,
  title = {{Applying Conditional Random Fields to Japanese Morphological Analysis}},
  author = {Kudo, Taku and Yamamoto, Kaoru and Matsumoto, Yuji},
  booktitle = {Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing},
  year = {2004},
  url = {https://aclanthology.org/W04-3230/}
}

References

18 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Masayuki Asahara and Yuji Matsumoto. 2000. Extended models and tools for high-performance part-of-speech tagger. In Proc of COLING, pages 21–27. find
  2. Richard H. Byrd, Peihuang Lu, Jorge Nocedal, and Ci You Zhu. 1995. A limited memory algorithm for bound constrained optimization. SIAM Journal on Scientific Computing, 16(6):1190–1208. find
  3. Stanley F. Chen and Ronald. Rosenfeld. 1999. A gaussian prior for smoothing maximum entropy models. Technical report, Carnegie Mellon University. find
  4. Joshua Goodman. 2004. Exponential priors for maximum entropy models. In Proc. of HLT/NAACL. find
  5. John Lafferty, Andrew McCallum, and Fernando Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proc. of ICML, pages 282–289. find
  6. Dong C. Liu and Jorge Nocedal. 1989. On the limited memory BFGS method for large scale optimization. Math. Programming, 45(3, (Ser. B)):503–528. find
  7. Andrew McCallum and Wei Li. 2003. Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons. In In Proc. of CoNLL. find
  8. Andrew McCallum, Dayne Freitag, and Fernando Pereira. 2000. Maximum entropy markov models for information and segmentation. In Proc. of ICML, pages 591–598. find
  9. Andrew McCallum. 2003. Efficiently inducing features of conditional random fields. In Nineteenth Conference on Uncertainty in Artificial Intelligence (UAI03). find
  10. Fuchun Peng and Andrew McCallum. 2004. Accurate information extraction from research papers. In Proc. of HLT/NAACL. find
  11. Fuchun Peng, Fangfang Feng, and Andrew McCallum. 2004. Chinese segmentation and new word detection using conditional random fields (to appear). In Proc. of COLING. find
  12. Simon Perkins, Kevin Lacker, and James Thiler. 2003. Grafting: Fast, incremental feature selection by gradient descent in function space. JMLR, 3:1333–1356. find
  13. Della Pietra, Stephen, Vincent J. Della Pietra, and John D. Lafferty. 1997. Inducing features of random fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(4):380–393. find
  14. David Pinto, Andrew McCallum, Xing Wei, and W. Bruce Croft. 2003. Table extraction using conditional random fields. In In Proc. of SIGIR, pages 235–242. find
  15. Fei Sha and Fernando Pereira. 2003. Shallow parsing with conditional random fields. In Proc. of HLT-NAACL, pages 213–220. find
  16. Kiyotaka Uchimoto, Satoshi Sekine, and Hitoshi Isahara. 2001. The unknown word problem: a morphological analysis of Japanese using maximum entropy aided by a dictionary. In Proc. of EMNLP, pages 91–99. find
  17. Kiyotaka Uchimoto, Chikashi Nobata, Atsushi Yamada, Satoshi Sekine, and Hitoshi Isahara. 2002. Morphological analysis of the spontaneous speech corpus. In Proc of COLING, pages 1298– 1302. find
  18. Kiyotaka Uchimoto, Chikashi Nobata, Atsushi Yamada, and Hitoshi Isahara Satoshi Sekine. 2003. Morphological analysis of a large spontaneous speech corpus in Japanese. In Proc. of ACL, pages 479–488. find