Elamur Research⌕
☰
home / papers / 2020 / Neural Unsupervised Domain Adaptation in NLP–-A Survey

Neural Unsupervised Domain Adaptation in NLP–-A Survey

Summary

Ramponi and Plank survey neural methods for unsupervised domain adaptation (UDA) in NLP. In UDA, a model trained on labeled source data must work on a different target distribution where only unlabeled text exists. They argue this setup is harder than supervised DA but much closer to real deployment, since annotating every new target is expensive and does not scale. The survey sorts the literature into three families:

  • Model-centric: changes to features, losses, architectures or parameters, such as pivot-based and adversarial approaches.
  • Data-centric: pseudo-labeling, data selection and pre-training.
  • Hybrid: combinations of the two.

The authors also question what "domain" means in NLP. They propose the broader term "variety," based on Plank's variety space, in which each corpus samples many latent dimensions such as genre, topic and demographics. Their comparative overview finds that sentiment analysis dominates the field and that methods are rarely tested across several tasks. The paper matters because it maps the field, exposes these evaluation gaps, and links UDA to the wider goal of out-of-distribution generalization.

Key points

  • The taxonomy (model-centric, data-centric, hybrid) adapts the data/model-centric distinction from Chu and Wang's MT survey and adds a hybrid class. Table 1 maps each method to the tasks it was evaluated on.
  • The analysis of Table 1 shows two problems: a column bias toward sentiment benchmarks, and row sparsity, meaning methods are seldom evaluated on multiple tasks.
  • Pre-trained transformers (BERT, SciBERT, BioBERT) help adaptation but have known weaknesses:
    • they lose accuracy out of domain;
    • they are often overconfident;
    • fine-tuning is unstable across random seeds and data order (Dodge et al., 2020).
  • The authors make three recommendations:
    • build multi-task UDA benchmarks that go beyond 1:1 adaptation;
    • release unlabeled data from the broader distribution that labeled sets were sampled from;
    • publish unaggregated annotations.
  • Open problems include X scarcity, where even unlabeled target text is rare (for example, clinical data). A curated reading list accompanies the survey in the awesome-neural-adaptation-in-NLP GitHub repository.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

104 papers of our library cite it (of the 140884 whose bibliographies we read); 2 of them have a page here, 102 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.coling-main.603

@inproceedings{ramponi2020neural,
  title = {{Neural Unsupervised Domain Adaptation in NLP–-A Survey}},
  author = {Ramponi, Alan and Plank, Barbara},
  booktitle = {Proceedings of the 28th International Conference on Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.coling-main.603},
  url = {https://doi.org/10.18653/v1/2020.coling-main.603}
}

References

135 references, 25 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Steven Abney. 2007. Semisupervised learning for computational linguistics. CRC press, 1st edition. find
  2. Roee Aharoni and Yoav Goldberg. 2020. Unsupervised domain clusters in pretrained language models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7747–7763, Online, July. Association for Computational Linguistics. find our card
  3. Firoj Alam, Shafiq Joty, and Muhammad Imran. 2018a. Domain Adaptation with Adversarial Training and Graph Embeddings. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Long Papers), pages 1077–1087. find
  4. Firoj Alam, Shafiq Joty, and Muhammad Imran. 2018b. Domain adaptation with adversarial training and graph embeddings. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1077–1087, Melbourne, Australia, July. Association for Compu… find
  5. Rie Kubota Ando and Tong Zhang. 2005. A framework for learning predictive structures from multiple tasks and unlabeled data. Journal of Machine Learning Research, 6(Nov):1817–1853. find
  6. Martin Arjovsky, Soumith Chintala, and Léon Bottou. 2017. Wasserstein generative adversarial networks. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 214–223, International… find
  7. Mikel Artetxe, Sebastian Ruder, Dani Yogatama, Gorka Labaka, and Eneko Agirre. 2020. A call for more rigor in unsupervised cross-lingual learning. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7375–7388, Online, July. Association for Computational… find
  8. Amittai Axelrod, Xiaodong He, and Jianfeng Gao. 2011. Domain adaptation via pseudo in-domain data selection. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 355–362, Edinburgh, Scotland, UK., July. Association for Computational Linguistics. find our card
  9. Timothy Baldwin, Paul Cook, Marco Lui, Andrew MacKinlay, and Li Wang. 2013. How noisy social media text, how diffrnt social media sources? In Proceedings of the Sixth International Joint Conference on Natural Language Processing, pages 356–364, Nagoya, Japan, October. Asian Federation of Natural Lan… find our card
  10. Yonatan Belinkov and James Glass. 2019. Analysis methods in neural language processing: A survey. Transactions of the Association for Computational Linguistics, 7:49–72. find our card
  11. Iz Beltagy, Kyle Lo, and Arman Cohan. 2019. SciBERT: A pretrained language model for scientific text. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3615–3620, H… find our card
  12. Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. 2010. A theory of learning from different domains. Machine Learning, 79(1-2):151–175. find
  13. Eyal Ben-David, Carmel Rabinovitz, and Roi Reichart. 2020. Perl: Pivot-based domain adaptation for pre-trained deep contextualized embedding models. Transactions of the Association for Computational Linguistics (To Appear). find
  14. Emily M Bender and Batya Friedman. 2018. Data statements for natural language processing: Toward mitigating system bias and enabling better science. Transactions of the Association for Computational Linguistics, 6:587– 604. find
  15. Yoshua Bengio. 2019. Deep learning for ai. Turing Award Lecture, Heidelberg Laureate Forum. http://www. iro.umontreal.ca/˜bengioy/HLF-Turing-23sept2019.pdf. Last accessed on 2020-05-26. find
  16. John Blitzer, Ryan McDonald, and Fernando Pereira. 2006. Domain adaptation with structural correspondence learning. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing (EMNLP 2006), pages 120–128. find our card
  17. Avrim Blum and Tom Mitchell. 1998. Combining labeled and unlabeled data with co-training. In Proceedings of the eleventh annual conference on Computational learning theory, pages 92–100. find our card
  18. Rens Bod. 1999. Context-sensitive spoken dialogue processing with the dop model. Natural Language Engineering, 5(4):309–323. find
  19. Konstantinos Bousmalis, George Trigeorgis, Nathan Silberman, Dilip Krishnan, and Dumitru Erhan. 2016. Domain Separation Networks. In Advances in Neural Information Processing Systems 29 (NIPS 2016), pages 343–351. find
  20. Statistical parsing with a context-free grammar and word statistics. AAAI/IAAI, Minmin Chen, Kilian Q Weinberger, Fei Sha, and Los Angeles. 2012. Marginalized Denoising Autoencoders for Domain Adaptation. In Proceedings of the 29th International Conference on Machine Learning (ICML-12), pages 767–77… find
  21. Chenhui Chu and Rui Wang. 2018. A Survey of Domain Adaptation for Neural Machine Translation. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1304–1319. find our card
  22. Stephane Clinchant, Gabriela Csurka, and Boris Chidlovskii. 2016. A Domain Adaptation Regularization for Denoising Autoencoders. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 26–31. find
  23. Gabriela Csurka. 2017. A comprehensive survey on domain adaptation for visual applications. In Domain Adaptation in Computer Vision Applications, pages 1–35. Springer International Publishing, Cham. find
  24. Xia Cui and Danushka Bollegala. 2019. Self-adaptation for unsupervised domain adaptation. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019), pages 213–222, Varna, Bulgaria, September. INCOMA Ltd. find
  25. Hal Daumé III. 2007. Frustratingly Easy Domain Adaptation. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, Prague, Czech Republic. Association for Computational Linguistics. find our card
  26. Leon Derczynski, Kalina Bontcheva, and Ian Roberts. 2016. Broad twitter corpus: A diverse named entity recognition resource. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1169–1179, Osaka, Japan, December. find
  27. The COLING 2016 Organizing Committee. find
  28. Shrey Desai, Barea Sinno, Alex Rosenfeld, and Junyi Jessy Li. 2019. Adaptive ensembling: Unsupervised domain adaptation for political document analysis. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Lan… find
  29. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find
  30. Jesse Dodge, Gabriel Ilharco, Roy Schwartz, Ali Farhadi, Hannaneh Hajishirzi, and Noah Smith. 2020. Fine-tuning pretrained language models: Weight initializations, data orders, and early stopping. arXiv preprint arXiv:2002.06305. find
  31. Jacob Eisenstein. 2013. What to do about bad language on the internet. In Proceedings of the 2013 conference of the North American Chapter of the association for computational linguistics: Human language technologies, pages 359–369. find our card
  32. Hady Elsahar and Matthias Gallé. 2019. To annotate or not? predicting performance drop under domain shift. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2163–2… find
  33. Jennifer Foster, Ozlem Cetinoglu, Joachim Wagner, Joseph Le Roux, Stephen Hogan, Joakim Nivre, Deirdre Hogan, and Josef Van Genabith. 2011. \# hardtoparse: Pos tagging and parsing the twitterverse. In Workshops at the Twenty-Fifth AAAI Conference on Artificial Intelligence. find
  34. W. N. Francis and H. Kucera. 1979. Brown corpus manual. Technical report, Department of Linguistics, Brown University, Providence, Rhode Island, US. find
  35. Lisheng Fu, Thien Huu Nguyen, Min Bonan, and Ralph Grishman. 2017. Domain Adaptation for Relation Extraction with Domain Adversarial Neural Network. In Proceedings of the The 8th International Joint Conference on Natural Language Processing, pages 425–429. find
  36. Yaroslav Ganin and Victor Lempitsky. 2015. Unsupervised Domain Adaptation by Backpropagation. In Proceedings of the 32nd International Conference on Machine Learning, volume 37, pages 1180–1189. find
  37. Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Francois Laviolette, Mario Marchand, and Victor Lempitsky. 2016. Domain-Adversarial Training of Neural Networks. Journal of Machine Learning Research, 17:1–35. find
  38. Mor Geva, Yoav Goldberg, and Jonathan Berant. 2019. Are we modeling the task or the annotator? an investigation of annotator bias in natural language understanding datasets. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conf… find
  39. Deepanway Ghosal, Devamanyu Hazarika, Abhinaba Roy, Navonil Majumder, Rada Mihalcea, and Soujanya Poria. 2020. KinGDOM: Knowledge-Guided DOMain Adaptation for Sentiment Analysis. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3198–3210, Online, July… find
  40. Kevin Gimpel, Nathan Schneider, Brendan O’Connor, Dipanjan Das, Daniel Mills, Jacob Eisenstein, Michael Heilman, Dani Yogatama, Jeffrey Flanigan, and Noah A Smith. 2011. Part-of-speech tagging for twitter: Annotation, features, and experiments. In Proceedings of the 49th Annual Meeting of the Associ… find our card
  41. Xavier Glorot, Antoine Bordes, and Yoshua Bengio. 2011. Domain adaptation for large-scale sentiment classification: A deep learning approach. In Proceedings of the 28th International Conference on Machine Learning (ICML-11), pages 513–520. find
  42. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative Adversarial Nets. In Advances in Neural Information Processing Systems 27 (NIPS 2014), pages 2672–2680. find
  43. Arthur Gretton, Karsten Borgwardt, Malte Rasch, Bernhard Schölkopf, and Alex J Smola. 2007. A kernel method for the two-sample-problem. In Advances in neural information processing systems, pages 513–520. find
  44. Arthur Gretton, Alex Smola, Jiayuan Huang, Marcel Schmittfull, Karsten Borgwardt, and Bernhard Schölkopf. 2009. Covariate shift by kernel mean matching. Dataset shift in machine learning, 3(4):5. find
  45. Han Guo, Ramakanth Pasunuru, and Mohit Bansal. 2020. Multi-source domain adaptation for text classification via distancenet-bandits. In AAAI, pages 7830–7838. find
  46. Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. 2020. Don’t stop pretraining: Adapt language models to domains and tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8342–8360, Online, J… find
  47. Xiaochuang Han and Jacob Eisenstein. 2019. Unsupervised domain adaptation of contextualized embeddings for sequence labeling. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJC… find our card
  48. Dan Hendrycks, Xiaoyuan Liu, Eric Wallace, Adam Dziedzic, Rishabh Krishnan, and Dawn Song. 2020. Pre-trained transformers improve out-of-distribution robustness. arXiv preprint arXiv:2004.06100. find our card
  49. Dirk Hovy. 2015. Demographic factors improve classification performance. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 752–762, Beijing, China, July.… find our card
  50. Jeremy Howard and Sebastian Ruder. 2018. Universal Language Model Fine-tuning for Text Classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 328–339, Melbourne, Australia, July. Association for Computational Linguist… find
  51. Nancy Ide and Keith Suderman. 2004. The American national corpus first release. In Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC’04), Lisbon, Portugal, May. European Language Resources Association (ELRA). find
  52. Chen Jia, Xiaobo Liang, and Yue Zhang. 2019. Cross-domain NER using cross-domain language modeling. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2464–2474, Florence, Italy, July. Association for Computational Linguistics. find
  53. Jing Jiang and ChengXiang Zhai. 2007. Instance weighting for domain adaptation in nlp. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 264–271. Association for Computational Linguistics. find our card
  54. Jing Jiang. 2008. A literature survey on domain adaptation of statistical classifiers. http://www.mysmu.edu/faculty/jingjiang/papers/da_survey.pdf, Last accessed on 2020-05-26. ↗
  55. Young-Bum Kim, Karl Stratos, and Dongchan Kim. 2017. Adversarial Adaptation of Synthetic or Stale Data. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vancouver, Canada, July. Association for Computational Linguistics. find
  56. David Krueger, Ethan Caballero, Joern-Henrik Jacobsen, Amy Zhang, Jonathan Binas, Remi Le Priol, and Aaron Courville. 2020. Out-of-distribution generalization via risk extrapolation (rex). arXiv preprint arXiv:2003.00688. — [arXiv:2003.00688](https://arxiv.org/abs/2003.00688) find
  57. Samuli Laine and Timo Aila. arXiv:1610.02242. 2016. — [arXiv:1610.02242](https://arxiv.org/abs/1610.02242) find
  58. Temporal ensembling for semi-supervised learning. arXiv preprint Egoitz Laparra, Steven Bethard, and Timothy A Miller. 2020. Rethinking domain adaptation for machine learning over clinical language. JAMIA Open. find
  59. Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. 2020. Biobert: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36(4):1234–1240. find our card
  60. Zheng Li, Yu Zhang, Ying Wei, Yuxiang Wu, and Qiang Yang. 2017. End-to-End Adversarial Memory Network for Cross-domain Sentiment Classification. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, pages 2237–2243, Melbourne, Australia, August. International… find
  61. Yitong Li, Timothy Baldwin, and Trevor Cohn. 2018a. What’s in a Domain? Learning Domain-Robust Text Representations using Adversarial Training. In Proceedings of NAACL-HLT 2018, pages 474–479. find
  62. Zheng Li, Ying Wei, Yu Zhang, and Qiang Yang. 2018b. Hierarchical attention transfer network for cross-domain sentiment classification. In Thirty-Second AAAI Conference on Artificial Intelligence. find
  63. Zhenghua Li, Xue Peng, Min Zhang, Rui Wang, and Luo Si. 2019. Semi-supervised Domain Adaptation for Dependency Parsing. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, July. Association for Computational Linguistics. find
  64. KyungTae Lim, Jay Yoon Lee, Jaime Carbonell, and Thierry Poibeau. 2020. Semi-Supervised Learning on Meta Structure: Multi-Task Tagging and Parsing in Low-Resource Scenarios. In The Thirty-Fourth AAAI Conference on Artificial Intelligence. find
  65. Yu-Hsiang Lin, Chian-Yu Chen, Jean Lee, Zirui Li, Yuyan Zhang, Mengzhou Xia, Shruti Rijhwani, Junxian He, Zhisong Zhang, Xuezhe Ma, Antonios Anastasopoulos, Patrick Littell, and Graham Neubig. 2019. Choosing transfer languages for cross-lingual learning. In Proceedings of the 57th Annual Meeting of… find our card
  66. Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19(2):313–330. find our card
  67. Anna Margolis. 2011. A literature review of domain adaptation with unlabeled data. Tech. Rep., University of Washington (USA). find
  68. SC Martin Arjovsky and Leon Bottou. 2017. Wasserstein generative adversarial networks. In Proceedings of the 34 th International Conference on Machine Learning, Sydney, Australia. find
  69. David McClosky, Eugene Charniak, and Mark Johnson. 2006. Reranking and Self-Training for Parser Adaptation. International Conference on Computational Linguistics (COLING) and Annual Meeting of the Association for Computational Linguistics (ACL), pages 337–344, July. find our card
  70. Timothy Miller. 2019. Simplified Neural Unsupervised Domain Adaptation. In Proceedings of NAACL-HLT 2019, pages 414–419. find
  71. Robert C. Moore and William Lewis. 2010. Intelligent selection of language model training data. In Proceedings of the ACL 2010 Conference Short Papers, pages 220–224, Uppsala, Sweden, July. Association for Computational Linguistics. find our card
  72. Aakanksha Naik and Carolyn Rose. 2020. Towards open domain event trigger identification using adversarial domain adaptation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7618–7624, Online, July. Association for Computational Linguistics. find
  73. Thien Huu Nguyen and Ralph Grishman. 2015. Relation Extraction: Perspective from Convolutional Neural Networks. In Proceedings of NAACL-HLT 2015, pages 39–48, Denver, Colorado. find our card
  74. Dong Nguyen, A Seza Doğruöz, Carolyn P Rosé, and Franciska de Jong. 2016. Computational sociolinguistics: A survey. Computational linguistics, 42(3):537–593. find
  75. Yonatan Oren, Shiori Sagawa, Tatsunori Hashimoto, and Percy Liang. 2019. Distributionally robust language modeling. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), page… find
  76. Sinno Jialin Pan and Qiang Yang. 2009. A survey on transfer learning. IEEE Transactions on knowledge and data engineering, 22(10):1345–1359. find
  77. Sinno Jialin Pan, Xiaochuan Ni, Jian-tao Sun, Qiang Yang, and Zheng Chen. 2010. Cross-Domain Sentiment Classification via Spectral Feature Alignment. In Proceedings of the 19th international conference on World wide web. ACM, pages 751–760. find
  78. Vishal M. Patel, Raghuraman Gopalan, Ruonan Li, and Rama Chellappa. 2015. Visual Domain Adaptation: A survey of recent advances. IEEE Signal Processing Magazine, 32(3):53–69. find
  79. Nanyun Peng and Mark Dredze. 2017. Multi-task domain adaptation for sequence tagging. In Proceedings of the 2nd Workshop on Representation Learning for NLP, pages 91–100, Vancouver, Canada, August. Association for Computational Linguistics. find
  80. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolog… find
  81. Slav Petrov and Ryan McDonald. 2012. Overview of the 2012 shared task on parsing the web. In Notes of the First Workshop on Syntactic Analysis of NonCanonical Language (SANCL). find
  82. Slav Petrov, Pi-Chuan Chang, Michael Ringgaard, and Hiyan Alshawi. 2010. Uptraining for accurate deterministic question parsing. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 705–713, Cambridge, MA, October. Association for Computational Linguistics… find
  83. Jason Phang, Thibault Févry, and Samuel R. Bowman. 2018. Sentence encoders on stilts: Supplementary training on intermediate labeled-data tasks. arXiv preprint arXiv:1811.01088. find our card
  84. Jason Phang, Phu Mon Htut, Yada Pruksachatkun, Haokun Liu, Clara Vania, Katharina Kann, Iacer Calixto, and Samuel R. Bowman. 2020. English intermediate-task training improves zero-shot cross-lingual transfer too. arXiv preprint arXiv:2005.13013. find
  85. Barbara Plank and Alessandro Moschitti. 2013. Embedding Semantic Similarity in Tree Kernels for Domain Adaptation of Relation Extraction. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 1498–1507. find
  86. Barbara Plank and Khalil Sima’an. 2008. Subdomain sensitive statistical parsing using raw corpora. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08), Marrakech, Morocco, May. European Language Resources Association (ELRA). find
  87. Barbara Plank and Gertjan van Noord. 2011. Effective measures of domain similarity for parsing. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 1566–1576, Portland, Oregon, USA, June. Association for Computational Linguis… find
  88. Barbara Plank, Dirk Hovy, and Anders Søgaard. 2014a. Linguistically debatable or just plain wrong? In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 507–511, Baltimore, Maryland, June. Association for Computational Linguistics. find our card
  89. Barbara Plank, Anders Johannsen, and Anders Søgaard. 2014b. Importance weighting and unsupervised domain adaptation of POS taggers: a negative result. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 968–973, Doha, Qatar, October. Association f… find
  90. Barbara Plank. 2011. Domain Adaptation for Parsing. Ph.D. thesis, University of Groningen. find
  91. Barbara Plank. 2016. What to do about non-standard (or non-canonical) language in NLP. In Proceedings of the 13th Conference on Natural Language Processing (KONVENS 2016). find
  92. Sujith Ravi, Kevin Knight, and Radu Soricut. 2008. Automatic prediction of parser accuracy. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 887–896, Honolulu, Hawaii, October. Association for Computational Linguistics. find
  93. Roi Reichart and Ari Rappoport. 2007. Self-training for enhancement and domain adaptation of statistical parsers trained on small datasets. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 616–623. find
  94. Robert Remus. 2012. Domain adaptation using domain similarity- and domain complexity-based instance selection for cross-domain sentiment analysis. In Proceedings of the 2012 IEEE 12th International Conference on Data Mining Workshops, ICDMW ’12, page 717–723, USA. IEEE Computer Society. find
  95. Nils Rethmeier and Barbara Plank. 2019. MoRTy: Unsupervised learning of task-specialized word embeddings by autoencoding. In Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019), pages 49–54, Florence, Italy, August. Association for Computational Linguistics. find
  96. Nils Rethmeier, Vageesh Kumar Saxena, and Isabelle Augenstein. 2020. TX-Ray: Quantifying and explaining model-knowledge transfer in (un-)supervised nlp. In UAI. find
  97. Anthony Rios, Ramakanth Kavuluru, and Zhiyong Lu. 2018. Generalizing biomedical relation classification with neural adversarial domain adaptation. Bioinformatics, 34(17):2973–2981. find
  98. Gil Rocha and Henrique Lopes Cardoso. 2019. A comparative analysis of unsupervised language adaptation methods. In Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019), pages 11–21, Hong Kong, China, November. Association for Computational Linguistics. find
  99. Guy Rotman and Roi Reichart. 2019. Deep contextualized self-training for low resource dependency parsing. Transactions of the Association for Computational Linguistics, 7(0):695–713. find
  100. Sebastian Ruder and Barbara Plank. 2017. Learning to select data for transfer learning with Bayesian optimization. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 372–382, Copenhagen, Denmark, September. Association for Computational Linguistics. find
  101. Sebastian Ruder and Barbara Plank. 2018. Strong baselines for neural semi-supervised learning under domain shift. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1044–1054, Melbourne, Australia, July. Association for Computati… find
  102. Sebastian Ruder, Ivan Vulić, and Anders Søgaard. 2019. A survey of cross-lingual word embedding models. Journal of Artificial Intelligence Research, 65:569–631. find
  103. Sebastian Ruder. 2019. Neural Transfer Learning for Natural Language Processing. Ph.D. thesis, National University of Ireland, Galway. find
  104. Kuniaki Saito, Yoshitaka Ushiku, and Tatsuya Harada. 2017. Asymmetric tri-training for unsupervised domain adaptation. In International Conference on Machine Learning, pages 2988–2997. find
  105. Motoki Sato, Hitoshi Manabe, Hiroshi Noji, and Yuji Matsumoto. 2017. Adversarial training for cross-domain universal dependency parsing. In Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 71–79, Vancouver, Canada, August. Association for… find
  106. Darsh J Shah, Tao Lei, Alessandro Moschitti, Salvatore Romeo, and Preslav Nakov. 2018. Adversarial Domain Adaptation for Duplicate Question Detection. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1056–1063. find
  107. Deven Santosh Shah, H. Andrew Schwartz, and Dirk Hovy. 2020. Predictive biases in natural language processing models: A conceptual framework and overview. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5248–5264, Online, July. Association for Comput… find our card
  108. Jian Shen, Yanru Qu, Weinan Zhang, and Yong Yu. 2018. Wasserstein Distance Guided Representation Learning for Domain Adaptation. In The Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18), pages 4058–4065. find
  109. Ge Shi, Chong Feng, Lifu Huang, Boliang Zhang, Heng Ji, Lejian Liao, and Heyan Huang. 2018. Genre Separation Network with Adversarial Training for Cross-genre Relation Extraction. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1018–1023. find
  110. Anders Søgaard and Christian Rishøj. 2010. Semi-supervised dependency parsing using generalized tri-training. In Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pages 1065– 1073. find
  111. Mark Steedman, Rebecca Hwa, Stephen Clark, Miles Osborne, Anoop Sarkar, Julia Hockenmaier, Paul Ruhlen, Steven Baker, and Jeremiah Crim. 2003. Example selection for bootstrapping statistical parsers. In Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of the… find
  112. Yulia Tsvetkov, Manaal Faruqui, Wang Ling, Brian MacWhinney, and Chris Dyer. 2016. Learning the curriculum with Bayesian optimization for task-specific word representation learning. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pa… find
  113. Ahmet Üstün, Rob van der Goot, Gosse Bouma, and Gertjan van Noord. 2019. Multi-team: A multi-attention, multi-decoder approach to morphological analysis. In Proceedings of the 16th Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 35–49, Florence, Italy, August. Ass… find
  114. Vincent Van Asch and Walter Daelemans. 2010. Using domain similarity for performance estimation. In Proceedings of the 2010 Workshop on Domain Adaptation for Natural Language Processing, pages 31–36, Uppsala, Sweden, July. Association for Computational Linguistics. find
  115. Marlies van der Wees, Arianna Bisazza, and Christof Monz. 2017. Dynamic data selection for neural machine translation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1400–1410, Copenhagen, Denmark, September. Association for Computational Linguistics… find
  116. Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. 2008. Extracting and Composing Robust Features with Denoising. In Proceedings of the 25th International Conference on Machine Learning, pages 1096–1103. find
  117. Riccardo Volpi, Hongseok Namkoong, Ozan Sener, John C Duchi, Vittorio Murino, and Silvio Savarese. 2018. Generalizing to unseen domains via adversarial data augmentation. In Advances in Neural Information Processing Systems, pages 5334–5344. find
  118. Huazheng Wang, Zhe Gan, Xiaodong Liu, Jingjing Liu, Jianfeng Gao, and Hongning Wang. 2019. Adversarial Domain Adaptation for Machine Reading Comprehension. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural… find
  119. Bonnie Webber. 2009. Genre distinctions for discourse in the Penn TreeBank. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pages 674–682, Suntec, Singapore, August. Association for C… find
  120. Karl Weiss, Taghi M. Khoshgoftaar, and DingDing Wang. 2016. A survey of transfer learning. Journal of Big Data, 3(1):9. find
  121. Garrett Wilson and Diane J. Cook. 2020. A Survey of Unsupervised Deep Domain Adaptation. arXiv preprint, arXiv:1812.02849, February. — [arXiv:1812.02849](https://arxiv.org/abs/1812.02849) find
  122. Yi Wu, David Bamman, and Stuart Russell. 2017. Adversarial training for relation extraction. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1778–1783, Copenhagen, Denmark, September. Association for Computational Linguistics. find
  123. Mengzhou Xia, Antonios Anastasopoulos, Ruochen Xu, Yiming Yang, and Graham Neubig. 2020. Predicting performance for natural language processing tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8625–8646. Association for Computational Linguistic… find
  124. Brian Xu, Mitra Mohtarami, and James Glass. 2019. Adversarial Domain Adaptation for Stance Detection. arXiv preprint arXiv:1902.02401, February. — [arXiv:1902.02401](https://arxiv.org/abs/1902.02401) find
  125. Yi Yang and Jacob Eisenstein. 2014. Fast Easy Unsupervised Domain Adaptation with Marginalized Structured Dropout. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 538–544, Baltimore, Maryland, June. Association for Computatio… find
  126. Qiang Yang, Yu Zhang, Wenyuan Dai, and Sinno Jialin Pan. 2020. Transfer Learning. Cambridge University Press. find
  127. Michihiro Yasunaga, Jungo Kasai, and Dragomir Radev. 2018. Robust multilingual part-of-speech tagging via adversarial training. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), p… find
  128. Juntao Yu, Mohab Elkaref, and Bernd Bohnet. 2015. Domain adaptation for dependency parsing via self-training. In Proceedings of the 14th International Conference on Parsing Technologies, pages 1–10, Bilbao, Spain, July. Association for Computational Linguistics. find
  129. Yuan Zhang, Jason Baldridge, and Luheng He. 2019. PAWS: Paraphrase adversaries from word scrambling. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1298–1308, M… find our card
  130. Zhi-Hua Zhou and Ming Li. 2005. Tri-training: Exploiting unlabeled data using three classifiers. IEEE Transactions on knowledge and Data Engineering, 17(11):1529–1541. find
  131. Xiaojin Zhu and Andrew B Goldberg. 2009. Introduction to semi-supervised learning. Synthesis lectures on artificial intelligence and machine learning, 3(1):1–130. find
  132. Yftah Ziser and Roi Reichart. 2017. Neural Structural Correspondence Learning for Domain Adaptation. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 400– 410. find
  133. Yftah Ziser and Roi Reichart. 2018a. Deep Pivot-Based Modeling for Cross-language Cross-domain Transfer with Minimal Guidance. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 238–249. find
  134. Yftah Ziser and Roi Reichart. 2018b. Pivot Based Language Modeling for Improved Neural Domain Adaptation. In Proceedings of NAACL-HLT 2018, pages 1241–1251. find
  135. Yftah Ziser and Roi Reichart. 2019. Task Refinement Learning for Improved Accuracy and Stability of Unsupervised Domain Adaptation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ramponi-2020-neural-unsupervised-domain-adaptation-nlp/