Elamur Research⌕
☰
home / papers / 2018 / Chinese NER Using Lattice LSTM

Chinese NER Using Lattice LSTM

Summary

Chinese named entity recognition depends heavily on word boundaries. A segment-then-tag pipeline inherits segmentation mistakes, which are common for unseen entity names and for text from new domains. Character-level taggers avoid this problem but ignore helpful word-level evidence. Zhang and Yang propose a lattice LSTM that keeps characters as the backbone of an LSTM-CRF tagger. It also links in every substring that matches an entry of a large lexicon, built from automatically segmented raw text with pretrained word embeddings. Extra word cells and gates let each character weigh information from the matched word paths, so the model uses lexical knowledge without committing to a single segmentation. The authors compare it with strong baselines: character-based models with bigram and soft segmentation features, and word-based models with character CNN/LSTM. On OntoNotes, MSRA, Weibo and a resume dataset, the lattice model wins significantly every time (p < 0.01) and achieves state-of-the-art results. The work offers a segmentation-free way to bring word information into Chinese sequence labeling.

Key points

  • The architecture is a character-centric LSTM-CRF with BIOES tagging. Lexicon words get recurrent cells but no hidden vectors of their own, and gating decides how much each word contributes.
  • Gains track lexicon coverage of test entities, measured as error reduction over the best character baseline:
    • OntoNotes: 78.72% coverage, 7.34% error reduction.
    • MSRA: 73.76% coverage, 16.11% error reduction.
    • Weibo: 62.56% coverage, 4.72% error reduction.
    • resume: 29.63% coverage, 0.89% error reduction.
  • On OntoNotes, the word-based baseline is strong on short sentences but degrades on long ones. The lattice model is more robust to sentence length than the hybrid baselines.
  • A case study ("东莞台协", Taiwan Association in Dongguan) shows that the lattice recovers an organization name that both segmentation-dependent baselines get wrong, even though the lexicon contains noisy words.
  • Code and data are released at github.com/jiesutd/LatticeLSTM.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Named entity recognition

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p18-1144

@inproceedings{zhang2018chinese,
  title = {{Chinese NER Using Lattice LSTM}},
  author = {Zhang, Yue and Yang, Jie},
  booktitle = {Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2018},
  doi = {10.18653/v1/p18-1144},
  url = {https://doi.org/10.18653/v1/p18-1144}
}

References

52 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Wanxiang Che, Mengqiu Wang, Christopher D Manning, and Ting Liu. 2013. Named entity recognition with bilingual constraints. In HLT-NAACL. pages 52–62. find
  2. Aitao Chen, Fuchun Peng, Roy Shan, and Gordon Sun. 2006a. Chinese named entity recognition with conditional probabilistic models. In Proceedings of the Fifth SIGHAN Workshop on Chinese Language Processing. pages 173–176. find
  3. SIGHAN Workshop on Chinese Language Processing. find
  4. Wenliang Chen, Yujie Zhang, and Hitoshi Isahara. 2006b. Chinese named entity recognition with conditional random fields. In Proceedings of the Fifth SIGHAN Workshop on Chinese Language Processing. pages 118–121. find
  5. Shen Huang, Xu Sun, and Houfeng Wang. 2017. Addressing domain adaptation for chinese word segmentation with global recurrent structure. In IJC-NLP. volume 1, pages 184–193. find
  6. Xinchi Chen, Xipeng Qiu, Chenxi Zhu, Pengfei Liu, and Xuanjing Huang. 2015. Long short-term memory neural networks for chinese word segmentation. In EMNLP. Lisbon, Portugal, pages 1197– 1206. http://aclweb.org/anthology/D15-1141. ↗
  7. Xinchi Chen, Zhan Shi, Xipeng Qiu, and Xuanjing Huang. 2017. Adversarial multi-criteria learning for Chinese word segmentation. In ACL. volume 1, pages 1193–1203. find
  8. Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint arXiv:1508.01991 . find our card
  9. Wenbin Jiang, Meng Sun, Yajuan Lü, Yating Yang, and Qun Liu. 2013. Discriminative learning with natural annotations: Word segmentation as a case study. In ACL. volume 1, pages 761–769. find
  10. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. In NAACL-HLT. pages 260–270. find
  11. Jason Chiu and Eric Nichols. 2016. Named entity recognition with bidirec-Yann LeCun, Bernhard Boser, John S Denker, Don-tional LSTM-CNNs. TACL 4:357–370. nie Henderson, Richard E Howard, Wayne Hubbard, https://transacl.org/ojs/index.php/tacl/article/view/792. and Lawrence D Jackel. 1989. Backpropaga… ↗
  12. Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from Gina-Anne Levow. 2006. The third international Chi-scratch. Journal of Machine Learning Research nese language processing bakeoff: Word segmen-12(Aug):2493–2537. tation and named entity recognition. In Procee… find
  13. Cıcero dos Santos, Victor Guimaraes, RJ Niterói, and Rio de Janeiro. 2015. Boosting named entity recognition with neural character embeddings. In Proceedings of NEWS 2015 The Fifth Named Entities Workshop. page 25. find
  14. Jianfeng Gao, Mu Li, Chang-Ning Huang, and Andi Wu. 2005. Chinese word segmentation and named entity recognition: A pragmatic approach. Computational Linguistics 31(4):531–574. find
  15. James Hammerton. 2003. Named entity recognition with long short-term memory. In HLT-NAACL 2003-Volume 4. pages 172–175. find
  16. Hangfeng He and Xu Sun. 2017a. F-score driven max margin neural network for named entity recognition in Chinese social media. In EACL. volume 2, pages 713–718. find
  17. Hangfeng He and Xu Sun. 2017b. A unified model for cross-domain and semi-supervised named entity recognition in Chinese social media. In AAAI. pages 3216–3222. find
  18. Jingzhou He and Houfeng Wang. 2008. Chinese named entity recognition and word segmentation based on character. In Proceedings of the Sixth ings of the Fifth SIGHAN Workshop on Chinese Language Processing. pages 108–117. find
  19. Haibo Li, Masato Hagiwara, Qi Li, and Heng Ji. 2014. Comparison of the impact of word segmentation on name tagging for Chinese and japanese. In LREC. pages 2532–2536. find
  20. Liyuan Liu, Jingbo Shang, Frank Xu, Xiang Ren, Huan Gui, Jian Peng, and Jiawei Han. 2018. Empower sequence labeling with task-aware neural language model. AAAI . find our card
  21. Yang Liu and Yue Zhang. 2012. Unsupervised domain adaptation for joint segmentation and pos-tagging. Proceedings of COLING 2012: Posters pages 745– 754. find
  22. Yijia Liu, Yue Zhang, Wanxiang Che, Ting Liu, and Fan Wu. 2014. Domain adaptation for crf-based Chinese word segmentation using free annotations. In EMNLP. pages 864–874. find
  23. Zhangxun Liu, Conghui Zhu, and Tiejun Zhao. 2010. Chinese named entity recognition with a sequence labeling approach: based on characters, or based on words? In Advanced intelligent computing theories and applications. With aspects of artificial intelligence, Springer, pages 634–640. find
  24. Yanan Lu, Yue Zhang, and Dong-Hong Ji. 2016. Multi-prototype Chinese character embedding. In LREC. find
  25. Gang Luo, Xiaojiang Huang, Chin-Yew Lin, and Zaiqing Nie. 2015. Joint entity recognition and disambiguation. In EMNLP. pages 879–888. find
  26. Xuezhe Ma and Eduard Hovy. 2016. End-to-end sequence labeling via Bi-directional LSTM-CNNs-CRF. In ACL. volume 1, pages 1064–1074. find our card
  27. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems. pages 3111–3119. find
  28. Alexandre Passos, Vineet Kumar, and Andrew McCallum. 2014. Lexicon infused phrase embeddings for named entity resolution. In CoNLL. pages 78–86. find our card
  29. Nanyun Peng and Mark Dredze. 2015. Named entity recognition for Chinese social media with jointly trained embeddings. In EMNLP. pages 548–554. find our card
  30. Nanyun Peng and Mark Dredze. 2016. Improving named entity recognition for Chinese social media with word segmentation representation learning. In ACL. volume 2, pages 149–155. find
  31. Nanyun Peng and Mark Dredze. 2017. Supplementary results for named entity recognition on chinese social media with an updated dataset . find
  32. Nanyun Peng, Hoifung Poon, Chris Quirk, Kristina Toutanova, and Wen-tau Yih. 2017. Cross-sentence n-ary relation extraction with graph lstms. TACL 5:101–115. find our card
  33. Matthew Peters, Waleed Ammar, Chandra Bhagavatula, and Russell Power. 2017. Semi-supervised sequence tagging with bidirectional language models. In ACL. volume 1, pages 1756–1765. find our card
  34. Likun Qiu and Yue Zhang. 2015. Word segmentation for chinese novels. In AAAI. pages 2440–2446. find
  35. Lev Ratinov and Dan Roth. 2009. Design challenges and misconceptions in named entity recognition. In CoNLL. pages 147–155. find our card
  36. Marek Rei. 2017. Semi-supervised multitask learning for sequence labeling. In ACL. volume 1, pages 2121–2130. find
  37. Matthias Sperber, Graham Neubig, Jan Niehues, and Alex Waibel. 2017. Neural lattice-to-sequence models for uncertain inputs. In EMNLP. pages 1380– 1389. https://www.aclweb.org/anthology/D17-1145. ↗
  38. Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. JMLR 15(1):1929–1958. find
  39. Jinsong Su, Zhixing Tan, Deyi Xiong, Rongrong Ji, Xiaodong Shi, and Yang Liu. 2017. Lattice-based recurrent neural network encoders for neural machine translation. In AAAI. pages 3302–3308. find
  40. Lin Sun, Kui Jia, Kevin Chen, Dit-Yan Yeung, Bertram E. Shi, and Silvio Savarese. 2017. Lattice long short-term memory for human action recognition. In ICCV. find
  41. Kai Sheng Tai, Richard Socher, and Christopher D. Manning. 2015. Improved semantic representations from tree-structured long short-term memory networks. In ACL-IJCNLP. Beijing, China, pages 1556–1566. http://www.aclweb.org/anthology/P15-1150. ↗
  42. Mengqiu Wang, Wanxiang Che, and Christopher D Manning. 2013. Effective bilingual constraints for semi-supervised learning of named entity recognizers. In AAAI. find
  43. Ralph Weischedel, Sameer Pradhan, Lance Ramshaw, Martha Palmer, Nianwen Xue, Mitchell Marcus, Ann Taylor, Craig Greenberg, Eduard Hovy, Robert Belvin, et al. 2011. Ontonotes release 4.0. LDC2011T03, Philadelphia, Penn.: Linguistic Data Consortium . find
  44. Y Xu, Y Wang, T Liu, J Liu, Y Fan, Y Qian, J Tsujii, and EI Chang. 2014. Joint segmentation and named entity recognition using dual decomposition in Chinese discharge summaries. JAMIA 21(e1):e84–92. find
  45. Naiwen Xue, Fei Xia, Fu-Dong Chiou, and Marta Palmer. 2005. The penn Chinese treebank: Phrase structure annotation of a large corpus. Natural language engineering 11(02):207–238. find
  46. Nianwen Xue. 2003. Chinese word segmentation as character tagging. In International Journal of Computational Linguistics and Chinese Language Processing. find our card
  47. Jie Yang, Zhiyang Teng, Meishan Zhang, and Yue Zhang. 2016. Combining discrete and neural features for sequence labeling. In International Conference on Computational Linguistics and Intelligent Text Processing. find
  48. Jie Yang, Yue Zhang, and Fei Dong. 2017a. Neural word segmentation with rich pretraining. In ACL. Vancouver, Canada, pages 839–849. http://aclweb.org/anthology/P17-1078. ↗
  49. Zhilin Yang, Ruslan Salakhutdinov, and William W Cohen. 2017b. Transfer learning for sequence tagging with hierarchical recurrent networks. In ICLR. find
  50. Suxiang Zhang, Ying Qin, Juan Wen, and Xiaojie Wang. 2006. Word segmentation and named entity recognition for sighan bakeoff3. In Proceedings of the Fifth SIGHAN Workshop on Chinese Language Processing. pages 158–161. find
  51. Hai Zhao and Chunyu Kit. 2008. Unsupervised segmentation helps supervised learning of character tagging for word segmentation and named entity recognition. In Proceedings of the Sixth SIGHAN Workshop on Chinese Language Processing. find
  52. Junsheng Zhou, Weiguang Qu, and Fen Zhang. 2013. Chinese named entity recognition via joint identification and categorization. Chinese Journal of Electronics 22(2):225–230. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2018-chinese-ner-lattice-lstm/