Elamur Research⌕
☰
home / papers / 2019 / Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing

Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing

Summary

Most NLP systems still work well only for a small set of well-resourced languages. They depend on labeled data, and their design often contains hidden assumptions tied to particular languages. Ponti et al. examine whether linguistic typology can help close this gap. Typology is the systematic study of how the world's languages differ in structure and meaning. The authors survey earlier and recent neural work along three lines: hand-built typological databases such as WALS, methods that infer typological features automatically from corpora and parallel texts, and techniques for feeding these features into models. They find that typology-aware systems reliably outperform typology-agnostic baselines, but usually by small margins. They give two reasons. Database entries are sparse, coarse, and categorical. Researchers also use only a narrow slice of the available features. The authors propose learning typology directly from raw data as continuous, context-sensitive representations, which would suit modern machine learning better and benefit low-resource languages.

Key points

  • Typology is applied mainly to morphosyntax, especially dependency parsing. Most systems take a few WALS features, mostly word order. Other resources go largely unused: lexical-semantic databases (IDS, ASJP), sound-system databases such as LAPSyD, PHOIBLE, and StressTyp2, and ValPaL, which documents how predicates combine with their arguments. These could help semantic role labeling, word sense disambiguation, and sentiment analysis.
  • Researchers fill gaps in the databases in four main ways: heuristics from morphosyntactic annotation, propagation from similar or clustered languages, supervised predictors, and distributional analysis of multi-parallel texts. None of these is best for every feature type, so the authors suggest combining them in ensembles.
  • The most common integration method is "selective sharing" in cross-lingual transfer. A model learns universal properties from all languages and language-specific properties only from typologically similar ones. Joint multilingual models also take typological vectors as input or condition their hidden layers on them.
  • Typological features outperform genealogical ones. The advantage is clearest on diverse language samples and when fine-grained differences between languages matter.
  • Typology also supports rule-based and Bayesian priors, data selection and synthesis, and explaining why performance differs across languages. Estimates of the number of languages vary: Glottolog counts 7,748 and Ethnologue counts 7,097.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

82 papers of our library cite it (of the 140884 whose bibliographies we read); 7 of them have a page here, 75 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/coli_a_00357

@article{ponti2019modeling,
  title = {{Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing}},
  author = {Ponti, Edoardo Maria and O'Horan, Helen and Berzak, Yevgeni and Vulić, Ivan and Reichart, Roi and Poibeau, Thierry and Shutova, Ekaterina and Korhonen, Anna},
  journal = {Computational Linguistics},
  year = {2019},
  doi = {10.1162/coli\_a\_00357},
  url = {https://doi.org/10.1162/coli_a_00357}
}

References

150 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Adel, Heike, Ngoc Thang Vu, and Tanja Schultz. 2013. Combination of recurrent neural networks and factored language models for code-switching language modeling. In Proceedings of ACL, pages 206–211. Agić, Željko. 2017. Cross-lingual parser selection for low-resource languages. In Proceedings of the… find
  2. Almeida, Mariana SC, Cláudia Pinto, Helena Figueira, Pedro Mendes, and André FT Martins. 2015. Aligning opinions: Cross-lingual opinion mining with dependencies. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on N… find
  3. Ammar, Waleed, George Mulcaire, Miguel Ballesteros, Chris Dyer, and Noah A. Smith. 2016. Many languages, one parser. TACL, 4:431–444. find our card
  4. Artetxe, Mikel, Gorka Labaka, Eneko Agirre, and Kyunghyun Cho. 2018. Unsupervised neural machine translation. In Proceedings of the Sixth International Conference on Learning Representations, pages 1–12. find our card
  5. Asgari, Ehsaneddin and Hinrich Schütze. 2017. Past, present, future: A computational investigation of the typology of tense in 1,000 languages. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 113–124. find
  6. Bakker, Dik. 2010. Language sampling. In J. J. Song, editor, The Oxford Handbook of Linguistic Typology, Oxford University Press, pages 100–127. find
  7. Banea, Carmen, Rada Mihalcea, Janyce Wiebe, and Samer Hassan. 2008. Multilingual subjectivity analysis using machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 127–135. find
  8. Bender, Emily M. 2009. Linguistically naïve != language independent: Why NLP needs linguistic typology. In Proceedings of the EACL 2009 Workshop on the Interaction Between Linguistics and Computational Linguistics: Virtuous, Vicious or Vacuous?, pages 26–32. find
  9. Bender, Emily M. 2011. On achieving and evaluating language-independence in NLP. Linguistic Issues in Language Technology, 3(6):1–26. find
  10. Bender, Emily M. 2014. Language collage: Grammatical description with the lingo grammar matrix. In Proceedings of LREC, pages 2447–2451. find
  11. Bender, Emily M. 2016. Linguistic typology in natural language processing. Linguistic Typology, 20(3):645–660. find
  12. Bender, Emily M., Michael Wayne Goodman, Joshua Crowgey, and Fei Xia. 2013. Towards creating precision grammars from interlinear glossed text: Inferring large-scale typological properties. In Proceedings of LaTeCH 2013, pages 74–83, Sofia. find
  13. Berlin, Brent and Paul Kay. 1969. find
  14. Basic Color Terms: Their Universality and Evolution. California University Press. Berzak, Yevgeni, Roi Reichart, and Boris Katz. 2014. Reconstructing native language typology from foreign language usage. In Proceedings of CoNLL, pages 21–29, Baltimore, MD. find
  15. Berzak, Yevgeni, Roi Reichart, and Boris Katz. 2015. Contrastive analysis with predictive power: Typology driven estimation of grammatical error distributions in ESL. In Proceedings of CoNLL, pages 94–102, Beijing. find
  16. Bickel, Balthasar. 2007. Typology in the 21st century: Major current developments. Linguistic Typology, 11(1):239–251. find
  17. Bickel, Balthasar. 2015. Distributional typology: Statistical inquiries into the dynamics of linguistic diversity. In Bernd Heine and Heiko Narrog, editors, Oxford Handbook of Linguistic Analysis. 901–923. Bickel, Balthasar, Johanna Nichols, Taras Zakharko, Alena Witzlack-Makarevich, Kristine Hildeb… find
  18. Bjerva, Johannes and Isabelle Augenstein. 2018. From phonology to syntax: Unsupervised linguistic typology at different levels with language embeddings. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find
  19. Bowerman, Melissa and Soonja Choi. 2001. Shaping meanings for language: Universal and language-specific in the acquisition of semantic categories, In Melissa Bowerman and Stephen Levinson, editors, Language Acquisition and Conceptual Development, Cambridge University Press, pages 475–511. find
  20. Bybee, Joan and James L. McClelland. 2005. Alternatives to the combinatorial paradigm of linguistic theory based on domain general principles of human cognition. The Linguistic Review, 22(2-4):381–410. find
  21. Bybee, Joan L. 1988. The diachronic dimension in explanation. In J. A. Hawkins, editor, Explaining Language Universals, Basil Blackwell, pages 350–379. find
  22. Chandar, Sarath, Stanislas Lauly, Hugo Larochelle, Mitesh Khapra, Balaraman Ravindran, Vikas C. Raykar, and Amrita Saha. 2014. An autoencoder approach to learning bilingual word representations. In Proceedings of Advances in Neural Information Processing Systems, pages 1853–1861. find
  23. Chang, Ming Wei, Lev Ratinov, and Dan Roth. 2007. Guiding semi-supervision with constraint-driven learning. In Proceedings of ACL, pages 280–287, Prague. find
  24. Chen, Xilun, Yu Sun, Ben Athiwaratkun, Claire Cardie, and Kilian Weinberger. 2018. Adversarial deep averaging networks for cross-lingual sentiment classification. Transactions of the Association for Computational Linguistics, 6:557–570. find our card
  25. Cohen, Shay B. 2016. Bayesian Analysis in Natural Language Processing, Synthesis Lectures on Human Language Technologies. find
  26. Morgan and Claypool. Coke, Reed, Ben King, and Dragomir R. Radev. 2016. Classifying syntactic regularities for hundreds of languages. CoRR, abs/1603.08016. — [arXiv:1603.08016](https://arxiv.org/abs/1603.08016) find
  27. Collins, Chris and Richard Kayne. 2009. Syntactic structures of the world’s languages. http://sswl.railsplayground.net/. ↗
  28. Collins, Michael. 2002. Discriminative training methods for hidden Markov models: Theory and experiments with perceptron algorithms. In Proceedings of EMNLP, pages 1–8, Philadelphia, PA. find our card
  29. Comrie, Bernard. 1989. Language Universals and Linguistic Typology: Syntax and Morphology. University of Chicago Press. find
  30. Conneau, Alexis, Guillaume Lample, Marc’Aurelio Ranzato, Ludovic Denoyer, and Hervé Jégou. 2017. Word translation without parallel data. arXiv preprint arXiv:1710.04087. find
  31. Conneau, Alexis, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. XNLI: Evaluating cross-lingual sentence representations. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2475–2485. find
  32. Copestake, Ann, Dan Flickinger, Carl Pollard, and Ivan A. Sag. 2005. Minimal recursion semantics: An introduction. Research on Language and Computation, 3(2-3):281–332. find
  33. Corbett, Greville G. 2010. Implicational hierarchies. In J. J. Song, editor, The Oxford Handbook of Linguistic Typology, Oxford University Press, pages 190–205. find
  34. Cotterell, Ryan and Jason Eisner. 2017. Probabilistic typology: Deep generative models of vowel inventories. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1182–1192. find
  35. Cotterell, Ryan and Jason Eisner. 2018. A deep generative model of vowel formant typology. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), volume 1, pages 37–46. find
  36. Crammer, Koby and Yoram Singer. 2003. Ultraconservative online algorithms for multiclass problems. Journal of Machine Learning Research, 3:951–991. find
  37. Cristofaro, S. and P. Ramat. 1999. Introduzione alla tipologia linguistica. find
  38. Carocci. Croft, William. 1995. Autonomy and functionalist linguistics. Language, 71(3):490–532. find
  39. Explaining Language Change: An Evolutionary Approach. Pearson Education. Croft, William. 2003. find
  40. Typology and Universals. Cambridge University Press. Croft, William, Dawn Nordquist, Katherine Looney, and Michael Regan. 2017. Linguistic typology meets universal dependencies. In Proceedings of the 15th International Workshop on Treebanks and Linguistic Theories (TLT15), pages 63–75. find
  41. Croft, William and Keith T. Poole. 2008. Inferring universals from grammatical variation: Multidimensional scaling for typological analysis. Theoretical Linguistics, 34(1):1–37. Daiber, Joachim, Miloš Stanojević, and Khalil Sima’an. 2016. Universal reordering via linguistic typology. In Proceedings… find
  42. Cambridge University Press. Das, Dipanjan and Slav Petrov. 2011. Unsupervised part-of-speech tagging with bilingual graph-based projections. In ACL, pages 600–609. find our card
  43. Daumé III, Hal and Lyle Campbell. 2007. A Bayesian model for discovering typological implications. In Proceedings of ACL, pages 65–72, Prague. find
  44. Dempster, Arthur P., Nan M. Laird, and Donald B. Rubin. 1977. Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society. Series B (Methodological). 39(1):1–38. find
  45. Deri, Aliya and Kevin Knight. 2016. Grapheme-to-phoneme models for (almost) any language. In Proceedings of ACL, pages 399–408, Berlin. find
  46. Dixon, Robert M. W. 1977. Where have all the adjectives gone? Studies in Language, 1(1):19–80. find
  47. Ergativity. Cambridge University Press. Dryer, Matthew S. 1998. Why statistical universals are better than absolute universals. In Papers from the 33rd Regional Meeting of the Chicago Linguistic Society, pages 1–23. Dryer, Matthew S. and Martin Haspelmath, editors. 2013. WALS Online. Max Planck Inst… find
  48. Duong, Long, Trevor Cohn, Steven Bird, and Paul Cook. 2015a. Low resource dependency parsing: Cross-lingual parameter sharing in a neural network parser. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural L… find
  49. Duong, Long, Trevor Cohn, Steven Bird, and Paul Cook. 2015b. A neural network model for low-resource universal dependency parsing. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 339–348. find
  50. Duong, Long, Hiroshi Kanayama, Tengfei Ma, Steven Bird, and Trevor Cohn. 2016. Learning crosslingual word embeddings without bilingual corpora. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1285–1295. find
  51. Coevolution: Genes, Culture, and Human Diversity. Stanford University Press. Durrett, Greg, Adam Pauls, and Dan Klein. 2012. Syntactic transfer using a bilingual lexicon. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Languag… find
  52. Evans, Nicholas. 2011. In Jae Jung Song editor, Semantic typology, The Oxford Handbook of Linguistic Typology, Oxford University Press, pages 504–533. find
  53. Evans, Nicholas and Stephen C. Levinson. 2009. The myth of language universals: Language diversity and its importance for cognitive science. Behavioral and Brain sciences, 32(5):429–448. find
  54. Fang, Meng and Trevor Cohn. 2017. Model transfer for tagging low-resource languages using a bilingual dictionary. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), volume 2, pages 587–593. find
  55. Faruqui, Manaal, Jesse Dodge, Sujay Kumar Jauhar, Chris Dyer, Eduard Hovy, and Noah A. Smith. 2015. Retrofitting word vectors to semantic lexicons. In Proceedings of NAACL-HLT, pages 1606–1615, Denver, CO. find
  56. Fernández, Alejandro Moreo, Andrea Esuli, and Fabrizio Sebastiani. 2015. Distributional correspondence indexing for cross-lingual and cross-domain sentiment classification. Journal of Artificial Intelligence Research, 55:131–163. find
  57. Ganchev, Kuzman, Jennifer Gillenwater, Ben Taskar et al. 2010. Posterior regularization for structured latent variable models. Journal of Machine Learning Research, 11:2001–2049. find
  58. Georgi, Ryan, Fei Xia, and William Lewis. 2010. Comparing language similarity across genetic and typologically based groupings. In Proceedings of COLING, pages 385–393, Beijing. Gerz, Daniela, Edoardo Maria Ponti, Jason Naradowsky, Roi Reichart, Anna Korhonen, and Ivan Vulić. 2018a. Language modeli… find
  59. Globerson, Amir and Tommi S. Jaakkola. 2007. Fixing max-product: Convergent message passing algorithms for MAP LP-relaxations. In Proceedings of NIPS, pages 553–560, Vancouver. Goedemans, Rob, Jeffrey Heinz, and Harry Van der Hulst, editors . 2014. Stresstyp2. University of Connecticut, University o… find
  60. National Science Foundation. Gouws, Stephan, Yoshua Bengio, and Greg Corrado. 2015. Bilbowa: Fast bilingual distributed representations without word alignments. In International Conference on Machine Learning, pages 748–756. find
  61. Gouws, Stephan and Anders Søgaard. 2015. Simple task-specific bilingual word embeddings. In Proceedings of NAACL-HLT, pages 1386–1390, Denver, CO. find
  62. Greenberg. 1978. Diachrony, synchrony and language universals. In Joseph H. Greenberg, Charles A. Ferguson, and Edith A. Moravcsik, editors, Universals of Human Language, Vol. 1: Method and Theory, Stanford University Press, pages 61–92. find
  63. Greenberg, Joseph H. 1963. Some universals of grammar with particular reference to the order of meaningful elements. Universals of Language, 2:73–113. find
  64. Greenberg, Joseph H. 1966a. Synchronic and diachronic universals in phonology. Language, 42(2):508–517. find
  65. Greenberg, Joseph H. 1966b. Universals of language. find
  66. MIT Press. Guo, Jiang, Wanxiang Che, Haifeng Wang, and Ting Liu. 2016. A universal framework for inductive transfer parsing across multi-typed treebanks. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 12–22. find
  67. Guo, Jiang, Wanxiang Che, David Yarowsky, Haifeng Wang, and Ting Liu. 2015. Cross-lingual dependency parsing based on distributed representations. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language… find our card
  68. Ha, Thanh Le, Jan Niehues, and Alexander Waibel. 2016. Toward multilingual neural machine translation with universal encoder and decoder. In Proceedings of the 2016 International Workshop on Spoken Language Translation (IWSLT). find our card
  69. Hammarström, Harald, Robert Forkel, Martin Haspelmath, and Sebastian Bank, editors. 2016. Glottolog 2.7. Max Planck Institute for the Science of Human History, Jena. Hartmann, Iren, Martin Haspelmath, and Bradley Taylor, editors . 2013. Valency Patterns Leipzig. Max Planck Institute for Evolutionary… find
  70. Haspelmath, Martin. 1999. Optimality and diachronic adaptation. Zeitschrift für Sprachwissenschaft, 18(2):180–205. find
  71. Haspelmath, Martin. 2007. Pre-established categories don’t exist: Consequences for language description and typology. Linguistic Typology, 11(1):119–132. Haspelmath, Martin and Uri Tadmor, editors. 2009. WOLD. Max Planck Institute for Evolutionary Anthropology, Leipzig. find
  72. Hermann, Karl Moritz and Phil Blunsom. 2014. Multilingual distributed representations without word alignment. In Proceedings of ICLR. find
  73. Hu, Zhiting, Xuezhe Ma, Zhengzhong Liu, Eduard Hovy, and Eric Xing. 2016a. Harnessing deep neural networks with logic rules. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 2410–2420. find our card
  74. Hu, Zhiting, Zichao Yang, Ruslan Salakhutdinov, and Eric Xing. 2016b. Deep neural networks with massive learned knowledge. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1670–1679. find
  75. Hwa, Rebecca, Philip Resnik, Amy Weinberg, Clara I. Cabezas, and Okan Kolak. 2005. Bootstrapping parsers via syntactic projection across parallel texts. Natural Language Engineering, 11(3):311–325. find
  76. Jensen, Finn V. 1996. An Introduction to Bayesian Networks, volume 210. find
  77. University College London Press, London. Johnson, Melvin, Mike Schuster, Quoc V. Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado et al. 2017. Google’s multilingual neural machine translation system: Enabling zero-shot translation. Transacti… find
  78. Khapra, Mitesh M., Salil Joshi, Arindam Chatterjee, and Pushpak Bhattacharyya. 2011. Together we can: Bilingual bootstrapping for WSD. In Proceedings of ACL, pages 561–569, Portland, OR. find
  79. Klementiev, Alexandre, Ivan Titov, and Binod Bhattarai. 2012. Inducing crosslingual distributed representations of words. In Proceedings of COLING, pages 1459–1474, Mumbai. find our card
  80. Komodakis, Nikos, Nikos Paragios, and Georgios Tziritas. 2011. MRF energy minimization and beyond via dual decomposition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(3):531–552. find
  81. Kozhevnikov, Mikhail and Ivan Titov. 2013. Cross-lingual transfer of semantic role labeling models. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 1190–1200. find
  82. Lauly, Stanislas, Alex Boulanger, and Hugo Larochelle. 2013. Learning multilingual word representations using a bag-of-words autoencoder. In Deep Learning Workshop at NIPS. find
  83. Lefever, Els, Véronique Hoste, and Martine De Cock. 2011. Parasense or how to use parallel corpora for word sense disambiguation. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, volume 2, pages 317–322. find
  84. Lewis, M. Paul, Gary F. Simons, and Charles D. Fennig. 2016. Ethnologue: Languages of the World, 19th ed., SIL International. find
  85. Lewis, William D. and Fei Xia. 2008. Automatically identifying computationally relevant typological features. In Proceedings of IJCNLP, pages 685–690, Hyderabad. find
  86. Littel, Patrick, David R. Mortensen, and Lori Levin. 2016. URIEL Typological database. find
  87. Carnegie Mellon University, Pittsburgh: PA. Liu, Haitao. 2010. Dependency direction as a means of word-order typology: A method based on dependency treebanks. Lingua, 120(6):1567–1578. find
  88. Lu, Xia. 2013. Exploring word order universals: A probabilistic graphical model approach. In Proceedings of ACL (Student Research Workshop), pages 150–157. find
  89. Luong, Thang, Hieu Pham, and Christopher D. Manning. 2015. Bilingual word representations with monolingual quality in mind. In Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing, pages 151–159. find our card
  90. Maddieson, Ian, Sébastien Flavier, Egidio Marsico, Christophe Coupé, and François Pellegrino. 2013. LAPSyd: Lyon-Albuquerque phonological systems database. In Proceedings of INTERSPEECH, pages 3022–3026, Lyon. find
  91. Majid, Asifa, Melissa Bowerman, Miriam van Staden, and James S. Boster. 2007. The semantic categories of cutting and breaking events: A crosslinguistic perspective. Cognitive Linguistics, 18(2):133–152. find
  92. Malaviya, Chaitanya, Graham Neubig, and Patrick Littell. 2017. Learning language representations for typology prediction. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2529–2535. find
  93. Mann, Gideon S. and Andrew McCallum. 2008. Generalized expectation criteria for semi-supervised learning of conditional random fields. In Proceedings of ACL, pages 870–878, Columbus, OH. find
  94. McDonald, Ryan, Koby Crammer, and Fernando Pereira. 2005. Online large-margin training of dependency parsers. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, pages 91–98. Michaelis, Susanne Maria, Philippe Maurer, Martin Haspelmath, and Magnus Huber, edito… find our card
  95. Mikolov, Tomas, Quoc V. Le, and Ilya Sutskever. 2013. Exploiting similarities among languages for machine translation. arXiv preprint arXiv:1309.4168. Moran, Steven, Daniel McCloy, and Richard Wright, editors. 2014. PHOIBLE Online. Max Planck Institute for Evolutionary Anthropology, Leipzig. Mrkšić… find
  96. Murawaki, Yugo. 2017. Diachrony-aware induction of binary latent representations from typological features. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), volume 1, pages 451–461. find
  97. Naseem, Tahira, Regina Barzilay, and Amir Globerson. 2012. Selective sharing for multilingual dependency parsing. In Proceedings of ACL, pages 629–637, Jeju Island. find our card
  98. Naseem, Tahira, Harr Chen, Regina Barzilay, and Mark Johnson. 2010. Using universal linguistic knowledge to guide grammar induction. In Proceedings of EMNLP 2010, pages 1234–1244. find
  99. Nichols, Johanna. 1992. Language Diversity in Space and Time. University of Chicago Press. find
  100. Niehues, Jan, Teresa Herrmann, Stephan Vogel, and Alex Waibel. 2011. Wider context by using bilingual language models in machine translation. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 198–206. find
  101. Nivre, Joakim, Marie-Catherine de Marneffe, Filip Ginter, Yoav Goldberg, Jan Hajic, Christopher D. Manning, Ryan McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, Reut Tsarfaty, and Daniel Zeman. 2016. Universal dependencies v1: A multilingual treebank collection. In Proceedings of LREC, pages… find our card
  102. Osborne, D., S. Narayan, and S. B. Cohen. 2016. Encoding prior knowledge with eigenword embeddings. Transactions of the Association for Computational Linguistics, 4:417–430. find
  103. Östling, Robert. 2015. Word order typology through multilingual word alignment. In Proceedings of ACL, pages 205–211, Beijing. find
  104. Östling, Robert and Jörg Tiedemann. 2017. Continuous multilinguality with language vectors. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, volume 2, pages 644–649. find
  105. Padó, Sebastian and Mirella Lapata. 2005. Cross-linguistic projection of role-semantic information. In Proceedings of EMNLP, pages 859–866, Vancouver. find
  106. Padó, Sebastian and Mirella Lapata. 2009. Cross-lingual annotation projection for semantic roles. Journal of Artificial Intelligence Research, 36(1):307–340. find
  107. Pappas, Nikolaos and Andrei Popescu-Belis. 2017. Multilingual hierarchical attention networks for document classification. In 8th International Joint Conference on Natural Language Processing (IJCNLP), pages 1015–1025. find
  108. Plank, Frans and Elena Filiminova. 1996. Universals archive. http://www.ling.unikonstanz.de/pages/proj/ sprachbau.htm. Universität Konstanz. Ponti, Edoardo Maria, Roi Reichart, Anna Korhonen, and Ivan Vulić. 2018a. Isomorphic transfer of syntactic structures in cross-lingual NLP. In Proceedings of… ↗
  109. Reichart, Roi and Regina Barzilay. 2012. Multi event extraction guided by global constraints. In Proceedings of NAACL, pages 70–79, Montreal. find
  110. Rosa, Rudolf and Zdenek Zabokrtsky. 2015. KLcpos3—a language similarity measure for delexicalized parser transfer. In Proceedings of ACL, pages 243–249, Beijing. find
  111. Rothe, Sascha and Hinrich Schütze. 2015. AutoExtend: Extending word embeddings to embeddings for synsets and lexemes. In Proceedings of ACL, pages 1793–1803, Beijing. Rotman, Guy, Ivan Vulić, and Roi Reichart. 2018. Bridging languages through images with deep partial canonical correlation analysis.… find
  112. Roy, Rishiraj Saha, Rahul Katare, Niloy Ganguly, and Monojit Choudhury. 2014. Automatic discovery of adposition typology. In Proceedings of COLING, pages 1037–1046. find
  113. Ruder, Sebastian. 2018. A survey of cross-lingual embedding models. Journal of Artificial Intelligence Research. To appear. find
  114. Rush, Alexander M., Roi Reichart, Michael Collins, and Amir Globerson. 2012. find
  115. Improved parsing and POS tagging using inter-sentence consistency constraints. In Proceedings of EMNLP-CoNLL, pages 1434–1444, Jeju Island. Sapir, Edward. 2014 [1921]. find
  116. Language. Cambridge University Press. Schone, Patrick and Daniel Jurafsky. 2001. Language-independent induction of part of speech class labels using only language universals. In IJCAI-2001 Workshop “Text Learning: Beyond Supervision.” Sennrich, Rico and Barry Haddow. 2016. Linguistic input features… find
  117. Silberer, Carina and Simone Paolo Ponzetto. 2010. UHD: Cross-lingual word sense disambiguation using multilingual co-occurrence graphs. In Proceedings of the 5th International Workshop on Semantic Evaluation, pages 134–137. find
  118. Snyder, Ben. 2010. Unsupervised Multilingual Learning, PhD thesis. Massachussetts Institute of Technology. find
  119. Snyder, Benjamin and Regina Barzilay. 2008. Unsupervised multilingual learning for morphological segmentation. In Proceedings of ACL-08: HLT, pages 737–745. find
  120. Søgaard, Anders. 2011. Data point selection for cross-language adaptation of dependency parsers. In Proceedings of ACL, pages 682–686, Portland, OR. find
  121. Søgaard, Anders and Julie Wulff. 2012. An empirical study of non-lexical extensions to delexicalized transfer. Proceedings of COLING 2012: Posters, pages 1181–1190. find
  122. Sproat, Richard. 2016. Language typology in speech and language technology. Linguistic Typology, 20(3):635–644. find
  123. Täckström, Oscar, Ryan McDonald, and Joakim Nivre. 2013. Target language adaptation of discriminative transfer parsers. In Proceedings of NAACL-HLT, pages 1061–1071, Atlanta, GA. find
  124. Täckström, Oscar, Ryan McDonald, and Jakob Uszkoreit. 2012. Cross-lingual word clusters for direct transfer of linguistic structure. In Proceedings of NAACL-HLT, pages 477–487, Montreal. find our card
  125. Tai, Kai Sheng, Richard Socher, and Christopher D. Manning. 2015. Improved semantic representations from tree-structured long short-term memory networks. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural L… find
  126. Takamura, Hiroya, Ryo Nagata, and Yoshifumi Kawasaki. 2016. Discriminative analysis of linguistic features for typological study. In Proceedings of LREC, pages 69–76, Portorož. find
  127. Talmy, Leonard. 1991. Path to realization: A typology of event conflation. In Proceedings of the Seventeenth Annual Meeting of the Berkeley Linguistics Society: General Session and Parasession on the Grammar of Event Structure, pages 480–519. find
  128. Taskar, Ben, Carlos Guestrin, and Daphne Koller. 2004. Max-margin Markov networks. In Proceedings of NIPS, pages 25–32, Vancouver. find
  129. Teh, Yee Whye, Hal Daumé III, and Daniel M. Roy. 2007. Bayesian agglomerative clustering with coalescents. In Proceedings of NIPS, pages 1473–1480. find
  130. Tiedemann, Jörg. 2015. Cross-lingual dependency parsing with universal dependencies and predicted POS labels. In Proceedings of the Third International Conference on Dependency Linguistics (Depling 2015), pages 340–349. find our card
  131. Titov, Ivan and Alexandre Klementiev. 2012. Crosslingual induction of semantic roles. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1, pages 647–656. find
  132. Tsvetkov, Yulia, Sunayana Sitaram, Manaal Faruqui, Guillaume Lample, Patrick Littell, David Mortensen, Alan W. Black, Lori Levin, and Chris Dyer. 2016. Polyglot neural language models: A case study in cross-lingual phonetic representation learning. In Proceedings of NAACL, pages 1357–1366, San Diego… find
  133. Upadhyay, Shyam, Manaal Faruqui, Chris Dyer, and Dan Roth. 2016. Cross-lingual models of word embeddings: An empirical comparison. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1661–1670. Vulić, Ivan, Wim De Smet,… find
  134. Wälchli, Bernhard and Michael Cysouw. 2012. Lexical typology through similarity semantics: Toward a semantic map of motion verbs. Linguistics, 50(3):671–710. find
  135. Wang, Dingquan and Jason Eisner. 2016. The galactic dependencies treebanks: Getting more data by synthesizing new languages. Transactions of the Association for Computational Linguistics, 4:491–505. find our card
  136. Wang, Dingquan and Jason Eisner. 2017. Fine-grained prediction of syntactic typology: Discovering latent structure with supervised learning. Transactions of the Association for Computational Linguistics, 5:147–162. find
  137. Wang, Hai and Hoifung Poon. 2018. Deep probabilistic logic: A unifying framework for indirect supervision. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 1891–1902. find
  138. Wang, Mengqiu and Christopher D. Manning. 2014. Cross-lingual pseudo-projected expectation regularization for weakly supervised learning. Transactions of the Association for Computational Linguistics, 2:55–66. Wichmann, Søren, Eric W. Holman, and Cecil H. Brown, editors . 2016. The ASJP Database (ve… find
  139. Wisniewski, Guillaume, Nicolas Pécheux, Souhir Gahbiche-Braham, and François Yvon. 2014. Cross-lingual part-of-speech tagging through ambiguous learning. In Proceedings of EMNLP, pages 1779–1785, Doha. find
  140. Xiao, Min and Yuhong Guo. 2014. Distributed word representation learning for cross-lingual dependency parsing. In Proceedings of CoNLL, pages 119–129. find
  141. Yang, Zhilin, Ruslan Salakhutdinov, and William Cohen. 2016. Multi-task cross-lingual sequence tagging from scratch. arXiv preprint arXiv:1603.06270. — [arXiv:1603.06270](https://arxiv.org/abs/1603.06270) find
  142. Yarowsky, David, Grace Ngai, and Richard Wicentowski. 2001. Inducing multilingual text analysis tools via robust projection across aligned corpora. In Proceedings of the First International Conference on Human Language Technology Research, pages 1–8. find our card
  143. Zeman, Daniel and Philip Resnik. 2008. Cross-language parser adaptation between related languages. In Proceedings of IJCNLP, pages 35–42. find our card
  144. Zennaki, Othman, Nasredine Semmar, and Laurent Besacier. 2016. Inducing multilingual text analysis tools using bidirectional recurrent neural networks. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 450–460. find
  145. Zhang, Yuan and Regina Barzilay. 2015. Hierarchical low-rank tensors for multilingual transfer parsing. In Proceedings of EMNLP, pages 1857–1867, Lisbon. find
  146. Zhang, Yuan, David Gaddy, Regina Barzilay, and Tommi Jaakkola. 2016. Ten pairs to tag—multilingual POS tagging via coarse mapping between embeddings. In Proceedings of NAACL, pages 1307–1317, San Diego, CA. find
  147. Zhang, Yuan, Roi Reichart, Regina Barzilay, and Amir Globerson. 2012. Learning to map into a universal POS tagset. In Proceedings of EMNLP, pages 1368–1378, Jeju Island. find
  148. Zhou, Guangyou, Tingting He, Jun Zhao, and Wensheng Wu. 2015. A subspace learning framework for cross-lingual sentiment classification with partial parallel data. In Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence (IJCAI 2015), pages 1426–1432. find
  149. Zhou, Xinjie, Xianjun Wan, and Jianguo Xiao. 2016. Cross-lingual sentiment classification with bilingual document representation learning. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 1403–1412. find
  150. Ziser, Yftah and Roi Reichart. 2018. Deep pivot-based modeling for cross-language cross-domain transfer with minimal guidance. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 238–249. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ponti-2019-modeling-language-variation-universals-survey/