Elamur Research⌕
☰
home / papers / 2015 / Cross-lingual Dependency Parsing Based on Distributed Representations

Cross-lingual Dependency Parsing Based on Distributed Representations

Summary

The paper tackles parser induction for languages with little or no annotated syntax. A parser is trained only on an English treebank and then applied to other languages. Earlier model-transfer work relied on delexicalized parsers that discard word identity, which especially hurts labeled parsing. The authors restore lexical information by placing words from both languages in one shared embedding space.

They build on the Chen and Manning (2014) greedy arc-standard neural parser with its cube activation, and add distance, valency and cross-lingual word cluster features. Two methods produce the bilingual word vectors:

  • Robust projection (PROJ)
  • CCA-based alignment

Neither method needs parallel text; each requires only a bilingual lexicon.

Training uses the English universal multilingual treebanks (McDonald et al., 2013), with German, Spanish and French as targets. The full system lowers LAS error by 10.9% on average relative to the delexicalized baseline. It also beats McDonald et al. (2013) with projected cluster features. This matters because many low-resource languages lack the bitexts that other transfer approaches depend on.

Key points

  • Data: McDonald et al.'s (2013) universal multilingual treebank collection, transferring from English (EN) to German (DE), Spanish (ES) and French (FR). Metrics are UAS and LAS.
  • The combined system (embeddings plus clusters) gives an average 10.9% relative LAS error reduction over the delexicalized parser.
  • In the comparison of bilingual embeddings, PROJ achieves the best LAS: 49.94 (DE), 61.76 (ES) and 61.55 (FR). CCA scores 49.32, 59.65 and 59.50, while MTL, BIAE, BICVM and BILBOWA score lower.
  • Some competing embeddings cover only about 31% of words in the DE test treebank, compared with over 70% for CCA and PROJ.
  • The code is released at github.com/jiangfeng1124/acl15-clnndep.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Dependency parsing · Treebanks

Cited by

89 papers of our library cite it (of the 140884 whose bibliographies we read); 9 of them have a page here, 80 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/p15-1119

@inproceedings{guo2015cross,
  title = {{Cross-lingual Dependency Parsing Based on Distributed Representations}},
  author = {Guo, Jiang and Che, Wanxiang and Yarowsky, David and Wang, Haifeng and Liu, Ting},
  booktitle = {Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)},
  year = {2015},
  doi = {10.3115/v1/p15-1119},
  url = {https://doi.org/10.3115/v1/p15-1119}
}

References

56 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Peter F Brown, Peter V Desouza, Robert L Mercer, Vincent J Della Pietra, and Jenifer C Lai. 1992. Class-based n-gram models of natural language. Computational linguistics, 18(4):467–479. find
  2. Sabine Buchholz and Erwin Marsi. 2006. Conll-x shared task on multilingual dependency parsing. In CoNLL, pages 149–164. find our card
  3. Yuan Cao and Sanjeev Khudanpur. 2014. Online learning in tensor space. In ACL, pages 666–675. find
  4. Sarath Chandar, Stanislas Lauly, Hugo Larochelle, Mitesh Khapra, Balaraman Ravindran, Vikas Raykar, and Amrita Saha. 2014. An autoencoder approach to learning bilingual word representations. In NIPS, pages 1853–1861. find
  5. Wanxiang Che, Mengqiu Wang, Christopher D. Manning, and Ting Liu. 2013. Named entity recognition with bilingual constraints. In NAACL, pages 52–62. find
  6. Danqi Chen and Christopher D Manning. 2014. A fast and accurate dependency parser using neural networks. In EMNLP, pages 740–750. find our card
  7. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. JMLR, 12:2493–2537. find our card
  8. Dipanjan Das and Slav Petrov. 2011. Unsupervised part-of-speech tagging with bilingual graph-based projections. In ACL, pages 600–609. find our card
  9. Marie-Catherine De Marneffe and Christopher D Manning. 2008. The stanford typed dependencies representation. In Coling 2008: Proceedings of the workshop on Cross-Framework and Cross-Domain Parser Evaluation, pages 1–8. find our card
  10. Marie-Catherine De Marneffe, Bill MacCartney, Christopher D Manning, et al. 2006. Generating typed dependency parses from phrase structure parses. In LREC, volume 6, pages 449–454. find our card
  11. John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. JMLR, 12:2121–2159. find
  12. Greg Durrett, Adam Pauls, and Dan Klein. 2012. Syntactic transfer using a bilingual lexicon. In EMNLP, pages 1–11, July. find
  13. Chris Dyer, Adam Lopez, Juri Ganitkevitch, Johnathan Weese, Ferhan Ture, Phil Blunsom, Hendra Setiawan, Vladimir Eidelman, and Philip Resnik. 2010. cdec: A decoder, alignment, and learning framework for finite-state and context-free translation models. In ACL. find our card
  14. Manaal Faruqui and Chris Dyer. 2014. Improving vector space word representations using multilingual correlation. In EACL, pages 462–471. find our card
  15. Kuzman Ganchev, Jennifer Gillenwater, and Ben Taskar. 2009. Dependency grammar induction via bitext projection constraints. In ACL-IJCNLP, pages 369–377. find
  16. Stephan Gouws, Yoshua Bengio, and Greg Corrado. 2014. Bilbowa: Fast bilingual distributed representations without word alignments. arXiv preprint arXiv:1410.2455. — [arXiv:1410.2455](https://arxiv.org/abs/1410.2455) find
  17. Weiwei Guo and Mona Diab. 2010. Combining orthogonal monolingual and multilingual sources of evidence for all words wsd. In ACL, pages 1542– 1551, July. find
  18. Jiang Guo, Wanxiang Che, Haifeng Wang, and Ting Liu. 2014. Revisiting embedding features for simple semi-supervised learning. In EMNLP, pages 110–120. find
  19. Aria Haghighi, Percy Liang, Taylor Berg-Kirkpatrick, and Dan Klein. 2008. Learning bilingual lexicons from monolingual corpora. In ACL, volume 2008, pages 771–779. find our card
  20. Karl Moritz Hermann and Phil Blunsom. 2014. Multilingual models for compositional distributed semantics. In ACL, pages 58–68, June. find our card
  21. Rebecca Hwa, Philip Resnik, Amy Weinberg, Clara Cabezas, and Okan Kolak. 2005. Bootstrapping parsers via syntactic projection across parallel texts. Natural language engineering, 11(03):311–325. find
  22. Wenbin Jiang, Qun Liu, and Yajuan Lv. 2011. Relaxed cross-lingual projection of constituent syntax. In EMNLP, pages 1192–1201. find
  23. Mitesh Khapra, Saurabh Sohoney, Anup Kulkarni, and Pushpak Bhattacharyya. 2010. Value for money: Balancing annotation effort, lexicon building and accuracy for multilingual wsd. In COLING, pages 555–563. find
  24. Seokhwan Kim and Gary Geunbae Lee. 2012. A graph-based cross-lingual projection approach for weakly supervised relation extraction. In ACL, pages 48–53. find
  25. Sungchul Kim, Kristina Toutanova, and Hwanjo Yu. 2012. Multilingual named entity recognition using parallel data and metadata from wikipedia. In ACL, pages 694–702. find
  26. Dan Klein and Christopher D Manning. 2004. Corpus-based induction of syntactic structure: Models of dependency and constituency. In ACL, page 478. find
  27. Alexandre Klementiev, Ivan Titov, and Binod Bhattarai. 2012. Inducing crosslingual distributed representations of words. In COLING, pages 1459–1474. find our card
  28. Philipp Koehn and Kevin Knight. 2002. Learning a translation lexicon from monolingual corpora. In Proceedings of the ACL-02 workshop on Unsupervised lexical acquisition-Volume 9, pages 9–16. find our card
  29. Kai Liu, Yajuan Lü, Wenbin Jiang, and Qun Liu. 2013. Bilingually-guided monolingual dependency grammar induction. In ACL, pages 1063–1072. find
  30. Xuezhe Ma and Fei Xia. 2014. Unsupervised dependency parsing with transferring distribution via parallel guidance and entropy regularization. In ACL, pages 1337–1348. find
  31. Gideon S Mann and David Yarowsky. 2001. Multipath translation lexicon induction via bridge languages. In NAACL, pages 1–8. find
  32. Ryan McDonald and Joakim Nivre. 2007. Characterizing the errors of data-driven dependency parsing models. In EMNLP-CoNLL, pages 122–131. find our card
  33. Ryan McDonald, Slav Petrov, and Keith Hall. 2011. Multi-source transfer of delexicalized dependency parsers. In EMNLP, pages 62–72. find our card
  34. Ryan T McDonald, Joakim Nivre, Yvonne Quirmbach-Brundage, Yoav Goldberg, Dipanjan Das, Kuzman Ganchev, Keith B Hall, Slav Petrov, Hao Zhang, Oscar Täckström, et al. 2013. Universal dependency annotation for multilingual parsing. In ACL, pages 92–97. find
  35. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. find
  36. Tahira Naseem, Harr Chen, Regina Barzilay, and Mark Johnson. 2010. Using universal linguistic knowledge to guide grammar induction. In EMNLP, pages 1234–1244. find
  37. Tahira Naseem, Regina Barzilay, and Amir Globerson. 2012. Selective sharing for multilingual dependency parsing. In ACL, pages 629–637. find our card
  38. Joakim Nivre. 2004. Incrementality in deterministic dependency parsing. In Proceedings of the Workshop on Incremental Parsing: Bringing Engineering and Cognition Together, pages 50–57. find our card
  39. Slav Petrov, Dipanjan Das, and Ryan McDonald. 2011. A universal part-of-speech tagset. arXiv preprint arXiv:1104.2086. find
  40. David A Smith and Jason Eisner. 2009. Parser adaptation and projection with quasi-synchronous grammar features. In EMNLP, pages 822–831. Association for Computational Linguistics. find
  41. Ilya Sutskever, Oriol Vinyals, and Quoc VV Le. 2014. Sequence to sequence learning with neural networks. In NIPS, pages 3104–3112. find
  42. Oscar Täckström, Ryan McDonald, and Jakob Uszkoreit. 2012. Cross-lingual word clusters for direct transfer of linguistic structure. In NAACL, pages 477–487. find our card
  43. Jörg Tiedemann and Joakim Nivre. 2014. Treebank translation for cross-lingual parser induction. CoNLL-2014, page 130. find
  44. Jörg Tiedemann. 2014. Rediscovering annotation projection for cross-lingual parser induction. In Proc. COLING. find
  45. Jörg Tiedemann. 2015. Improving the cross-lingual projection of syntactic dependencies. In Nordic Conference of Computational Linguistics NODAL-IDA 2015, page 191. find
  46. Ivan Titov and Alexandre Klementiev. 2012. Crosslingual induction of semantic roles. In ACL, pages 647–656. find
  47. Joseph Turian, Lev Ratinov, and Yoshua Bengio. 2010. Word representations: a simple and general method for semi-supervised learning. In ACL, pages 384– 394. find our card
  48. Jakob Uszkoreit and Thorsten Brants. 2008. Distributed word clustering for large scale class-based language modeling in machine translation. In ACL, pages 755–762. find
  49. Mengqiu Wang and Christopher D. Manning. 2013. Effect of non-linear deep architecture in sequence labeling. In IJCNLP, pages 1285–1291. find
  50. Mengqiu Wang and Christopher D Manning. 2014. Cross-lingual projected expectation regularization for weakly supervised learning. TACL, 2:55–66. find
  51. Min Xiao and Yuhong Guo. 2014. Distributed word representation learning for cross-lingual dependency parsing. In CoNLL, pages 119–129. find
  52. David Yarowsky, Grace Ngai, and Richard Wicentowski. 2001. Inducing multilingual text analysis tools via robust projection across aligned corpora. In Proceedings of the first international conference on Human language technology research, pages 1– 8. find our card
  53. Yue Zhang and Stephen Clark. 2011. Syntactic processing using the generalized perceptron and beam search. Computational Linguistics, 37(1):105–151. find
  54. Yue Zhang and Joakim Nivre. 2011. Transition-based dependency parsing with rich non-local features. In ACL, pages 188–193. find
  55. Hai Zhao, Yan Song, Chunyu Kit, and Guodong Zhou. 2009. Cross language dependency parsing using a bilingual lexicon. In ACL-IJCNLP, pages 55–63. find
  56. Tao Zhuang and Chengqing Zong. 2010. Joint inference for bilingual semantic role labeling. In EMNLP, pages 304–314. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/guo-2015-cross-lingual-dependency-parsing-based/