Elamur Research⌕
☰
home / papers / 2021 / Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLP

Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLP

Summary

MaChAmp is a toolkit for fine-tuning pre-trained contextualized embeddings on one or many NLP tasks simultaneously, while keeping the setup simple. It is built on AllenNLP and can load any Hugging Face encoder. A shared encoder feeds separate task-specific decoders. Users describe datasets and tasks in a JSON configuration file. The toolkit covers many task types:

  • tagging, lemmatization, BIO tagging and multi-label tagging,
  • classification,
  • dependency parsing,
  • masked language modeling,
  • seq2seq generation.

The authors chose the default hyperparameters through a grid search over GLUE and EWT using mBERT. They then checked that single-task models are close to published results, which holds for UD but less so for GLUE and machine translation. Finally, they compared several multi-dataset strategies on all UD treebanks and GLUE. Joint training helps when the datasets share the same tasks, as in UD, but not when the tasks differ, as in GLUE. Smoothing how often each dataset is sampled is the most consistently useful option. The work matters because it lowers the effort needed to run multi-task and multilingual experiments and makes them easier to reproduce.

Key points

  • Supported task types: seq, string2string, seq_bio (CRF decoder), multiseq, dependency (deep biaffine parser with Chu-Liu/Edmonds decoding), mlm, classification and seq2seq (RNN decoder with attention).

  • Masked language modeling: follows the BERT setup. 15% of tokens are selected; of these, 80% become [MASK], 10% become a random token and 10% stay unchanged. Next sentence prediction is left out.

  • Hyperparameter defaults: tuned on the GLUE classification tasks and EWT 2.6 with mBERT. Training sets were capped at 20,000 sentences.

  • Average UD LAS by setup:

    SetupUD LAS
    Single72.22
    All72.82
    Smoothed (α = 0.5)73.74
    Separate decoders73.69

  • Average GLUE accuracy by setup:

    SetupGLUE accuracy
    Single82.38
    All80.96
    Smoothed81.87

    Multi-task training only helped RTE among the GLUE tasks.

  • Effect by treebank size: the largest gain is for treebanks under 1k sentences, where LAS rises from 15.1 to 45.4 with smoothing. For treebanks over 10k sentences the change is marginal (80.1 → about 81.0).

Interesting ideas

  • I1. One dataset per batch, with selective loss. Each batch draws from a single dataset, and only the losses of tasks present in that batch are computed. This lets corpora with different annotation layers be mixed without filling in missing labels.
  • I2. Tunable sampling smoothing (α). A parameter controls how much small datasets are upsampled relative to large ones. In these experiments it lifted low-resource treebanks a lot, and it would suit any collection of very unevenly sized corpora.
  • I3. Dataset or language embeddings. Telling the model which dataset an instance comes from can help on top of contextual encoders, especially for medium-sized data. It is a cheap way to handle domain or genre variation among treebanks of one language.
  • I4. Picking a proxy model for languages with no training data. For zero-shot treebanks, the source parser was chosen by word overlap between the first 10 test sentences and each candidate training set. This simple selection rule can be reused wherever no target-language training data exists.
  • I5. Predicting edit rules instead of labels. For lemmatization, the model predicts a token-to-lemma transformation rather than the lemma itself, which shrinks the label vocabulary sharply. This should help morphologically rich languages.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Treebanks

Cited by

104 papers of our library cite it (of the 140884 whose bibliographies we read); 8 of them have a page here, 96 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.eacl-demos.22

@inproceedings{goot2021massive,
  title = {{Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLP}},
  author = {Goot, Rob van der and Üstün, Ahmet and Ramponi, Alan and Sharaf, Ibrahim and Plank, Barbara},
  booktitle = {Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations},
  year = {2021},
  doi = {10.18653/v1/2021.eacl-demos.22},
  url = {https://doi.org/10.18653/v1/2021.eacl-demos.22}
}

References

213 references, 32 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anne Abeillé, Lionel Clément, and Alexandra Kinyon. 2000. Building a treebank for French. In Proceedings of the Second International Conference on Language Resources and Evaluation (LREC’00), Athens, Greece. European Language Resources Association (ELRA). find
  2. Noëmi Aepli and Simon Clematide. 2018. Parsing approaches for Swiss German. In Proceedings of the 3rd Swiss Text Analytics Conference (SwissText), Winterthur, Switzerland. find
  3. Željko Agić, Anders Johannsen, Barbara Plank, Héctor Martı́nez Alonso, Natalie Schluter, and Anders Søgaard. 2016. Multilingual projection for parsing truly low-resource languages. Transactions of the Association for Computational Linguistics, 4:301– 312. find our card
  4. Željko Agić and Nikola Ljubešić. 2015. Universal Dependencies for Croatian (that work for Serbian, too). In The 5th Workshop on Balto-Slavic Natural Language Processing, pages 1–8, Hissar, Bulgaria. IN-COMA Ltd. Shoumen, BULGARIA. find our card
  5. Lars Ahrenberg. 2015. Converting an English-Swedish parallel treebank to Universal Dependencies. In Proceedings of the Third International Conference on Dependency Linguistics (Depling 2015), pages 10– 19, Uppsala, Sweden. Uppsala University, Uppsala, Sweden. find our card
  6. Linda Alfieri and Fabio Tamburini. 2016. (almost) automatic conversion of the Venice Italian Treebank into the merged Italian Dependency Treebank format. In CLiC-it/EVALITA. find
  7. Ika Alfina, Indra Budi, and Heru Suhartanto. 2020. Tree rotations for dependency trees: Converting the head-directionality of noun phrases. Journal of Computer Science, 16(11):1585–1597. find
  8. Héctor Martı́nez Alonso and Daniel Zeman. 2016. Universal Dependencies for the AnCora treebanks. Procesamiento del Lenguaje Natural, 57:91–98. find
  9. Waleed Ammar, George Mulcaire, Miguel Ballesteros, Chris Dyer, and Noah A. Smith. 2016. Many languages, one parser. Transactions of the Association for Computational Linguistics, 4:431–444. find our card
  10. Angelina Aquino, Franz de Leon, and Mary Ann Bacolod. 2020. UD Tagalog-Ugnayan. https://github.com/UniversalDependencies/UD_ Tagalog-Ugnayan . ↗
  11. Carolina Coelho Aragon. 2018. Variações estilı́sticas e sociais no discurso dos falantes akuntsú. Polifonia, 25(38.1):90–103. find
  12. Maria Jesus Aranzabe, Aitziber Atutxa, Kepa Bengoetxea, Arantza Diaz de Ilarraza, Iakes Goenaga, Koldo Gojenola, and Larraitz Uria. 2015. Automatic conversion of the Basque dependency treebank to universal dependencies. In Proceedings of the fourteenth international workshop on treebanks an linguist… find
  13. Masayuki Asahara, Hiroshi Kanayama, Takaaki Tanaka, Yusuke Miyao, Sumire Uematsu, Shinsuke Mori, Yuji Matsumoto, Mai Omura, and Yugo Murawaki. 2018. Universal Dependencies version 2 for Japanese. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)… find our card
  14. Elena Badmaeva and Francis M. Tyers. 2017. Dependency treebank for Buryat. In Proceedings of the 15th International Workshop on Treebanks and Linguistic Theories (TLT15), pages 1–12. find
  15. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In 3rd International Conference on Learning Representations, ICLR 2015, Conference Track Proceedings, San Diego, CA, USA. find our card
  16. David Bamman and Gregory Crane. 2011. The ancient Greek and Latin dependency treebanks. In Language technology for cultural heritage, pages 79–98. Springer. find
  17. Verginica Barbu Mititelu, Radu Ion, Radu Simionescu, Elena Irimia, and Cenel-Augusto Perez. 2016. The Romanian treebank annotated according to Universal Dependencies. In Proceedings of the tenth international conference on natural language processing (hrtal2016). find
  18. Colin Batchelor. 2019. Universal dependencies for Scottish Gaelic: syntax. In Proceedings of the Celtic Language Technology Workshop, pages 7–15, Dublin, Ireland. European Association for Machine Translation. find our card
  19. Shabnam Behzad and Amir Zeldes. 2020. A cross-genre ensemble approach to robust Reddit part of speech tagging. In Proceedings of the 12th Web as Corpus Workshop, pages 50–56, Marseille, France. European Language Resources Association. find
  20. Eduard Bejček, Eva Hajičová, Jan Hajič, Pavlı́na Jı́nová, Václava Kettnerová, Veronika Kolářová, Marie Mikulová, Jiřı́ Mı́rovskỳ, Anna Nedoluzhko, Jarmila Panevová, et al. 2013. Prague dependency treebank 3.0. find
  21. Luisa Bentivogli, Ido Dagan, Hoa Trang Dang, Danilo Giampiccolo, and Bernardo Magnini. 2009. The fifth PASCAL recognizing textual entailment challenge. In Proceedings of the Second Text Analysis Conference, TAC 2009, Gaithersburg, Maryland, USA. find
  22. Yevgeni Berzak, Jessica Kenney, Carolyn Spadine, Jing Xian Wang, Lucia Lam, Keiko Sophie Mori, Sebastian Garza, and Boris Katz. 2016. Universal Dependencies for learner English. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages… find our card
  23. Irshad Bhat, Riyaz A. Bhat, Manish Shrivastava, and Dipti Sharma. 2018. Universal Dependency parsing for Hindi-English code-switching. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Pap… find
  24. Riyaz Ahmad Bhat, Rajesh Bhatt, Annahita Farudi, Prescott Klassen, Bhuvana Narasimhan, Martha Palmer, Owen Rambow, Dipti Misra Sharma, Ashwini Vaidya, Sri Ramagurumurthy Vishnu, et al. 2016. The hindi/urdu treebank project. In Handbook of Linguistic Annotation. Springer Press. find
  25. Agne Bielinskiene, Loic Boizou, and Jolanta Kovalevskaite. 2016. Lithuanian dependency treebank. In Human Language Technologies–The Baltic Perspective: Proceedings of the Seventh International Conference Baltic HLT 2016, volume 289, page 107. IOS Press. find
  26. Su Lin Blodgett, Johnny Wei, and Brendan O’Connor. 2018. Twitter Universal Dependency parsing for African-American and mainstream American English. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1415–1425, Melbourne, Australi… find our card
  27. Ondřej Bojar, Christian Buck, Christian Federmann, Barry Haddow, Philipp Koehn, Johannes Leveling, Christof Monz, Pavel Pecina, Matt Post, Herve Saint-Amand, Radu Soricut, Lucia Specia, and Aleš Tamchyna. 2014. Findings of the 2014 workshop on statistical machine translation. In Proceedings of the… find our card
  28. Guillaume Bonfante, Bruno Guillaume, and Guy Perrier. 2018. Application of Graph Rewriting to Natural Language Processing. Wiley Online Library. find
  29. Emanuel Borges Völker, Maximilian Wendt, Felix Hennig, and Arne Köhn. 2019. HDT-UD: A very large Universal Dependencies treebank for German. In Proceedings of the Third Workshop on Universal Dependencies (UDW, SyntaxFest 2019), pages 46–57, Paris, France. Association for Computational Linguistics. find
  30. Cristina Bosco, Felice Dell’Orletta, Simonetta Montemagni, Manuela Sanguinetti, and Maria Simi. 2014. find
  31. The EVALITA 2014 dependency parsing task. In EVALITA 2014 Evaluation of NLP and Speech Tools for Italian, pages 1–8. Pisa University Press. find
  32. Gosse Bouma and Gertjan van Noord. 2017. Increasing return on annotation investment: The automatic construction of a Universal Dependency treebank for Dutch. In Proceedings of the NoDaLiDa 2017 Workshop on Universal Dependencies (UDW 2017), pages 19–26, Gothenburg, Sweden. Association for Computatio… find
  33. Anouck Braggaar and Rob van der Goot. 2021. Challenges in annotating and parsing spoken, code-switched, Frisian-Dutch data. In Proceedings of the Second Workshop on Domain Adaptation for Natural Language Processing. find
  34. Sabine Brants, Stefanie Dipper, Peter Eisenberg, Silvia Hansen-Schirra, Esther König, Wolfgang Lezius, Christian Rohrer, George Smith, and Hans Uszkoreit. 2004. TIGER: Linguistic interpretation of a german corpus. Research on language and computation, 2(4):597–620. find
  35. Bernard Caron, Marine Courtin, Kim Gerdes, and Sylvain Kahane. 2019. A surface-syntactic UD treebank for Naija. In Proceedings of the 18th International Workshop on Treebanks and Linguistic Theories (TLT, SyntaxFest 2019), pages 13–24, Paris, France. Association for Computational Linguistics. find
  36. Rich Caruana. 1993. Multitask learning: A knowledge-based source of inductive bias. In Proceedings of the Tenth International Conference on Machine Learning, pages 41–48, Amherst, MA, USA. find
  37. Rich Caruana. 1997. Multitask learning. In Learning to learn, pages 95–133. Springer. find
  38. Flavio Massimiliano Cecchini, Marco Passarotti, Paola Marongiu, and Daniel Zeman. 2018. Challenges in converting the index Thomisticus treebank into Universal Dependencies. In Proceedings of the Second Workshop on Universal Dependencies (UDW 2018), pages 27–36, Brussels, Belgium. Association for Com… find
  39. Slavomı́r Čéplö. 2018. Constituent order in Maltese: A quantitative analysis. find
  40. Daniel Cer, Mona Diab, Eneko Agirre, Iñigo Lopez-Gazpio, and Lucia Specia. 2017. SemEval-2017 task 1: Semantic textual similarity multilingual and crosslingual focused evaluation. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 1–14, Vancouver, Canada.… find
  41. Özlem Çetinoğlu and Çağrı Çöltekin. 2019. Challenges of annotating a code-switching treebank. In Proceedings of the 18th International Workshop on Treebanks and Linguistic Theories (TLT, SyntaxFest 2019), pages 82–90, Paris, France. Association for Computational Linguistics. find
  42. Mauro Cettolo, Niehues Jan, Stüker Sebastian, Luisa Bentivogli, Roldano Cattoni, and Marcello Federico. 2014. find
  43. The IWSLT 2014 evaluation campaign. In International Workshop on Spoken Language Translation, Lake Tahoe, CA, USA. find
  44. Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder–decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Langu… find
  45. Grzegorz Chrupała, Ákos Kádár, and Afra Alishahi. 2015. Learning language through pictures. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 112– 118… find
  46. Grzegorz Chrupała. 2006. Simple data-driven context-sensitive lemmatization. SEPLN, 37:121–127. find
  47. Yoeng-Jin Chu. 1965. On the shortest arborescence of a directed graph. Scientia Sinica, 14:1396–1400. find
  48. Jayeol Chun, Na-Rae Han, Jena D. Hwang, and Jinho D. Choi. 2018. Building Universal Dependency treebanks in Korean. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA). find
  49. Alessandra Teresa Cignarella, Cristina Bosco, and Paolo Rosso. 2019. Presenting TWITTIRÒ-UD: An Italian Twitter treebank in Universal Dependencies. In Proceedings of the Fifth International Conference on Dependency Linguistics (Depling, SyntaxFest 2019), pages 190–197, Paris, France. Association fo… find
  50. Kevin Clark, Minh-Thang Luong, Urvashi Khandelwal, Christopher D. Manning, and Quoc V. Le. 2019. BAM! born-again multi-task networks for natural language understanding. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5931–5937, Florence, Italy. Assoc… find our card
  51. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research, 12:2493– 2537. find our card
  52. Cagrı Cöltekin. 2015. A grammar-book treebank of Turkish. In Proceedings of the 14th workshop on Treebanks and Linguistic Theories (TLT 14), pages 35–49. find
  53. Alexis Conneau and Guillaume Lample. 2019. Cross-lingual language model pretraining. In Advances in Neural Information Processing Systems, pages 7059–7069, Vancouver, Canada. find
  54. Sam Davidson, Dian Yu, and Zhou Yu. 2019. Dependency parsing for spoken dialog systems. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1513– 1519, Hong Kong, Chi… find our card
  55. Mehmet Oguz Derin. 2020. UD Old Turkish-Tonqq. https://github.com/ ↗
  56. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find
  57. Cheikh Bamba Dione. 2019. Developing Universal Dependencies for Wolof. In Proceedings of the Third Workshop on Universal Dependencies (UDW, SyntaxFest 2019), pages 12–23, Paris, France. Association for Computational Linguistics. find
  58. Peter Dirix, Liesbeth Augustinus, Daniel van Niekerk, and Frank Van Eynde. 2017. Universal Dependencies for Afrikaans. In Proceedings of the NoDaLiDa 2017 Workshop on Universal Dependencies (UDW 2017), pages 38–47, Gothenburg, Sweden. Association for Computational Linguistics. find
  59. Kaja Dobrovoljc, Tomaž Erjavec, and Simon Krek. 2017. The Universal Dependencies treebank for Slovenian. In Proceedings of the 6th Workshop on Balto-Slavic Natural Language Processing, pages 33–38, Valencia, Spain. Association for Computational Linguistics. find
  60. Kaja Dobrovoljc and Joakim Nivre. 2016. The Universal Dependencies treebank of spoken Slovenian. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 1566–1573, Portorož, Slovenia. European Language Resources Association (ELRA). find
  61. William B. Dolan and Chris Brockett. 2005. Automatically constructing a corpus of sentential paraphrases. In Proceedings of the Third International Workshop on Paraphrasing (IWP2005), pages 9–16, Jeju Island, Korea. find our card
  62. Marcos Garcia. 2016. Universal dependencies guidelines for the Galician-TreeGal treebank. Technical report, Technical Report, LyS Group, Universidade da Coruna. find
  63. Matt Gardner, Joel Grus, Mark Neumann, Oyvind Tafjord, Pradeep Dasigi, Nelson F. Liu, Matthew Peters, Michael Schmitz, and Luke Zettlemoyer. 2018. AllenNLP: A deep semantic natural language processing platform. In Proceedings of Workshop for NLP Open Source Software (NLP-OSS), pages 1– 6, Melbourne,… find our card
  64. Fabrı̀cio Ferraz Gerardi. 2020. UD Tupinamba-TuDeT. https://github.com/ ↗
  65. Timothy Dozat and Christopher D Manning. 2017. Deep biaffine attention for neural dependency parsing. In Proceedings of 5th International Conference on Learning Representations, ICLR 2017, Conference Track Proceedings, Toulon, France. find our card
  66. Fabrı̀cio Ferraz Gerardi. 2021. Mundurukú. find
  67. Kira Droganova, Olga Lyashevskaya, and Daniel Zeman. 2018. Data conversion and consistency of monolingual corpora: Russian UD treebanks. In Proceedings of the 17th international workshop on treebanks and linguistic theories (tlt 2018), 155, pages 53–66. find
  68. Memduh Gökırmak and Francis M. Tyers. 2017. A dependency treebank for Kurmanji Kurdish. In Proceedings of the Fourth International Conference on Dependency Linguistics (Depling 2017), pages 64–72, Pisa,Italy. Linköping University Electronic Press. find
  69. Puneet Dwivedi and Guha Easha. 2017. Universal Dependencies for Sanskrit. International Journal of Advance Research, Ideas and Innovations in Technology, 3(4). find
  70. Xavier Gómez Guinovart. 2017. Recursos integrados da lingua galega para a investigación lingüı́stica. Gallaecia. Estudos de lingüı́stica portuguesa e galega. Santiago de Compostela: Universidade de Santiago, pages 1037–1048. find
  71. Hanne Eckhoff, Kristin Bech, Gerlof Bouma, Kristine Eide, Dag Haug, Odd Einar Haugen, and Marius Jøhndal. 2018. The PROIEL treebank family: a standard for early attestations of Indo-European languages. Language Resources and Evaluation, 52(1):29–65. find
  72. Rob van der Goot and Gertjan van Noord. 2018. Modeling input uncertainty in neural network dependency parsing. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4984–4991, Brussels, Belgium. Association for Computational Linguistics. find
  73. Hanne Martine Eckhoff and Aleksandrs Berdičevskis. 2015. Linguistics vs. digital editions: The Tromsø Old Russian and OCS treebank. Scripta & e-Scripta, 14(15):9–25. find
  74. Rob van der Goot, Ahmet Üstün, Alan Ramponi, and Barbara Plank. 2020. Massive choice, ample tasks (MaChAmp): A toolkit for multi-task learning in NLP. arXiv preprint arXiv:2005.14672v2. — [arXiv:2005.14672](https://arxiv.org/abs/2005.14672) find
  75. Jack Edmonds. 1967. Optimum branchings. Journal of Research of the national Bureau of Standards B, 71(4):233–240. find
  76. Normunds Gruzitis, Lauma Pretkalnina, Baiba Saulite, Laura Rituma, Gunta Nespore-Berzkalne, Arturs Znotins, and Peteris Paikens. 2018. Creation of a balanced state-of-the-art multilayer corpus for NLU. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC… find
  77. Marhaba Eli, Weinila Mushajiang, Tuergen Yibulayin, Kahaerjiang Abiderexiti, and Yan Liu. 2016. Universal dependencies for Uyghur. In Proceedings of the Third International Workshop on Worldwide Language Service Infrastructure and Second Workshop on Open Infrastructures and Analysis Frameworks for H… find
  78. The COLING 2016 Organizing Committee. find
  79. Marı́lia Fernanda Pereira de Freitas. 2017. A posse em apurinã: Descrição de construções atributivas e predicativas em comparação com outras lı́nguas aruák. Belém: Programa de Pós-Graduação em Letras, Universidade Federal do Pará (Tese de Doutorado). find
  80. The structure of Bruno Guillaume, Marie-Catherine de Marneffe, and Guy Perrier. 2019. Conversion et améliorations de corpus du Français annotés en Universal Dependencies. Traitement Automatique des Langues, 60(2):71–95. find
  81. Jan Hajič, Otakar Smrž, Petr Zemánek, Petr Pajas, Jan Šnaidauf, Emanuel Beška, Jakub Krácmar, and Kamila Hassanová. 2009. Prague Arabic dependency treebank 1.0. find
  82. Kazuma Hashimoto, Caiming Xiong, Yoshimasa Tsuruoka, and Richard Socher. 2017. A joint many-task model: Growing a neural network for multiple NLP tasks. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1923–1933, Copenhagen, Denmark. Association for Co… find our card
  83. Hildur Jónsdóttir and Anton Karl Ingason. 2020. Creating a parallel Icelandic dependency treebank from raw text to Universal Dependencies. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 2924–2931, Marseille, France. European Language Resources Association. find
  84. UD Finnish-OOD. https://github.com/UniversalDependencies/UD_ Finnish-OOD . ↗
  85. Dag TT Haug and Marius Jøhndal. 2008. Creating a parallel treebank of the old Indo-European Bible translations. In Proceedings of the second workshop on language technology for cultural heritage data (LaTeCH 2008), pages 27–34. find
  86. Johannes Heinecke and Francis M. Tyers. 2019. Development of a Universal Dependencies treebank for Welsh. In Proceedings of the Celtic Language Technology Workshop, pages 21–31, Dublin, Ireland. European Association for Machine Translation. find
  87. Oliver Hellwig, Salvatore Scarlata, Elia Ackermann, and Paul Widmer. 2020. The treebank of Vedic Sanskrit. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 5137– 5146, Marseille, France. European Language Resources Association. find
  88. Barbora Hladká, Jan Hajič, Jirka Hana, Jaroslava Hlaváčová, Jiřı́ Mı́rovskỳ, and Jan Raab. 2008. The Czech academic corpus 2.0 guide. The Prague Bulletin of Mathematical Linguistics, 89(1):41–96. find
  89. Jeremy Howard and Sebastian Ruder. 2018. Universal language model fine-tuning for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 328–339, Melbourne, Australia. Association for Computational Linguistics. find
  90. Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, and Melvin Johnson. 2020. XTREME: A massively multilingual multi-task benchmark for evaluating cross-lingual generalization. In Proceedings of the 37th International Conference on Machine Learning, volume 119, pages 4411–4421. find our card
  91. Anton Karl Ingason, Eirı́kur Rögnvaldsson, Einar Freyr Sigurosson, and Joel C. Wallenberg. 2020. The Faroese parsed historical corpus. CLARIN-IS, Stofnun Árna Magnússonar. find
  92. Olájı́dé Ishola and Daniel Zeman. 2020. Yorùbá dependency treebank (YTB). In Proceedings of the 12th Language Resources and Evaluation Conference, pages 5178–5186, Marseille, France. European Language Resources Association. find
  93. Tomás Jelı́nek. 2017. FicTree: A manually annotated treebank of Czech fiction. In ITAT, pages 181–185. find
  94. Anders Johannsen, Héctor Martı́nez Alonso, and Barbara Plank. 2015. Universal dependencies for Danish. In International Workshop on Treebanks and Linguistic Theories (TLT14), page 157. find
  95. Dan Kondratyuk and Milan Straka. 2019. 75 languages, 1 model: Parsing universal dependencies universally. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2779–279… find our card
  96. Kamil Kopacewicz. PISANDUB. 2018. find
  97. UD Akkadian-https://github. com/UniversalDependencies/UD_ Akkadian-PISANDUB . find
  98. Natalia Kotsyba, Bohdan Moskalevskyi, Mykhailo Romanenko, Halyna Samoridna, Ivanka Kosovska, Olha Lytvyn, Oksana Orlenko, Hanna Brovko, Bohdana Matushko, Natalia Onyshchuk, Valeriia Pareviazko, Yaroslava Rychyk, Anastasiia Stetsenko, Snizhana Umanets, and Larysa Masenko. 2018. UD Ukrainian-IU. https… find
  99. Vincent Krı́ž, Barbora Hladká, and Zdenka Uresova. 2016. Czech legal text treebank 1.0. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 2387– 2392. find
  100. Anne Lacheret-Dujour, Sylvain Kahane, and Paola Pietrandrea. 2019. Rhapsodie: A prosodic and syntactic treebank for spoken French, volume 89. John Benjamins Publishing Company. find
  101. John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the Eighteenth International Conference on Machine Learning, ICML ’01, page 282–289, San Francisco, CA, USA. find
  102. Angeliki Lazaridou, Nghia The Pham, and Marco Baroni. 2015. Combining language and vision with a multimodal skip-gram model. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 153–163, Denver, Color… find our card
  103. John Lee, Herman Leung, and Keying Li. 2017. Towards Universal Dependencies for learner Chinese. In Proceedings of the NoDaLiDa 2017 Workshop on Universal Dependencies (UDW 2017), pages 67– 71, Gothenburg, Sweden. Association for Computational Linguistics. find
  104. Hector Levesque, Ernest Davis, and Leora Morgenstern. 2012. The Winograd schema challenge. In Thirteenth International Conference on the Principles of Knowledge Representation and Reasoning, Rome, Italy. find
  105. Xiaodong Liu, Kevin Duh, Liyuan Liu, and Jianfeng Gao. 2020. Very deep transformers for neural machine translation. arXiv preprint arXiv:2008.07772v2. — [arXiv:2008.07772](https://arxiv.org/abs/2008.07772) find
  106. Yijia Liu, Yi Zhu, Wanxiang Che, Bing Qin, Nathan Schneider, and Noah A. Smith. 2018. Parsing tweets into Universal Dependencies. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers),… find our card
  107. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692. find
  108. Mikko Luukko, Aleksi Sahala, Sam Hardwick, and Krister Lindén. 2020. Akkadian treebank for early neo-assyrian royal inscriptions. In Proceedings of the 19th International Workshop on Treebanks and Linguistic Theories, pages 124–134, Düsseldorf, Germany. Association for Computational Linguistics. find our card
  109. Cătălina Mărănduc, Cenel-Augusto Perez, and Radu Simionescu. 2016. Social media-processing Romanian chat and discourse analysis. Computación y Sistemas, 20(3):405–414. find
  110. Héctor Martı́nez Alonso and Barbara Plank. 2017. When is multitask learning effective? semantic sequence prediction under varying data conditions. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, Valencia, Spain.… find
  111. Héctor Martı́nez Alonso, Djamé Seddah, and Benoı̂t Sagot. 2016. From noisy questions to Minecraft texts: Annotation challenges in extreme syntax scenario. In Proceedings of the 2nd Workshop on Noisy User-generated Text (WNUT), pages 13–23, Osaka, Japan. find
  112. The COLING 2016 Organizing Committee. find
  113. Ryan McDonald, Joakim Nivre, Yvonne Quirmbach-Brundage, Yoav Goldberg, Dipanjan Das, Kuzman Ganchev, Keith Hall, Slav Petrov, Hao Zhang, Oscar Täckström, Claudia Bedini, Núria Bertomeu Castelló, and Jungmee Lee. 2013. Universal Dependency annotation for multilingual parsing. In Proceedings of th… find
  114. Maria Mitrofan, Verginica Barbu Mititelu, and Grigorina Mitrofan. 2019. MoNERo: a biomedical gold standard corpus for the Romanian language. In Proceedings of the 18th BioNLP Workshop and Shared Task, pages 71–79, Florence, Italy. Association for Computational Linguistics. find
  115. Olga Lyashevskaya. 2019. A reusable tagset for the morphologically rich language in change: A case of Middle Russian. In Proceedings of the International Conference Dialogue 2019, pages 422–434. find
  116. Foroushani Mojiri, Hossein Amir, Hamid Aghaei, and Amir Ahmadi. 2020. UD Soi-AHA. https: find
  117. Olga Lyashevskaya, Angelika Peljak-Łapińska, and Daria Petrova. 2017. UD Belarusian-HSE. https: find
  118. AmirHossein Mojiri Foroushani, Hamid Aghaei, and Amir Ahmadi. 2020a. UD Khunsari-AHA. https://github.com/UniversalDependencies/UD_ Belarusian-HSE . //github.com/UniversalDependencies/UD_ Khunsari-AHA . ↗
  119. Olga Lyashevskaya and Dmitry 2017. UD Lithuanian-HSE. //github.com/UniversalDependencies/UD_ Soi-AHA . find
  120. Sichinava. https://github.com/UniversalDependencies/ UD_Lithuanian-HSE . ↗
  121. Teresa Lynn and Jennifer Foster. 2016. Universal dependencies for irish. In CLTW. find
  122. Aibek Makazhanov, Aitolkyn Sultangazina, Olzhas Makhambetov, and Zhandos Yessenbayev. 2015. Syntactic annotation of Kazakh: Following the Universal Dependencies guidelines. a report. In 3rd International Conference on Turkic Languages Processing, (TurkLang 2015), pages 338–350. find
  123. Christopher D Manning. 2015. Computational linguistics and deep learning. Computational Linguistics, 41(4):701–707. find
  124. AmirHossein Mojiri Foroushani, Hamid Aghaei, and Amir Ahmadi. 2020b. UD Nayini-AHA. https://github.com/UniversalDependencies/UD_ Nayini-AHA . ↗
  125. Kadri Muischnek, Kaili Müürisep, Tiina Puolakainen, Eleri Aedmaa, Riin Kirt, and Dage Särg. 2014. Estonian dependency treebank and its annotation scheme. In Proceedings of 13th workshop on treebanks and linguistic theories (TLT13), pages 285–291. find
  126. Kadri Muischnek, Kaili Müürisep, and Dage Dage Särg. 2019. CG roots of UD treebank of Estonian web language. In Proceedings of the NoDaLiDa 2019 Workshop on Constraint Grammar-Methods, Tools and Applications, 30 September 2019, Turku, Finland, 168, pages 23–26. Linköping University Electronic Pr… find
  127. Robert Munro. 2020. Human-in-the-loop machine learning. Sl: O’REILLY MEDIA. find
  128. Phuong-Thai Nguyen, Xuan-Luong Vu, Thi-Minh-Huyen Nguyen, Van-Hiep Nguyen, and Hong-Phuong Le. 2009. Building a large syntactically-annotated corpus of Vietnamese. In Proceedings of the Third Linguistic Annotation Workshop (LAW III), pages 182–185, Suntec, Singapore. Association for Computational Li… find
  129. Rik van Noord, Antonio Toral, and Johan Bos. 2020. Character-level representations improve DRS-based semantic parsing Even in the age of BERT. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4587–4603, Online. Association for Computational Lin… find
  130. Atul Kr. Ojha and Daniel Zeman. 2020. Universal Dependency treebanks for low-resource Indian languages: The case of Bhojpuri. In Proceedings of the WILDRE5– 5th Workshop on Indian Language Data: Resources and Evaluation, pages 33–38, Marseille, France. European Language Resources Association (ELRA). find
  131. Mai Omura, Yuta Takahashi, and Masayuki Asahara. 2017. Universal dependency for modern Japanese. In Proceedings of the 7th Conference of Japanese Association for Digital Humanities (JADH2017), pages 34–36. find
  132. Robert Östling, Carl Börstell, Moa Gärdenfors, and Mats Wirén. 2017. Universal Dependencies for Swedish Sign Language. In Proceedings of the 21st Nordic Conference on Computational Linguistics, pages 303–308, Gothenburg, Sweden. Association for Computational Linguistics. find
  133. Lilja Øvrelid and Petter Hohle. 2016. Universal Dependencies for Norwegian. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 1579–1585, Portorož, Slovenia. European Language Resources Association (ELRA). find
  134. Lilja Øvrelid, Andre Kåsen, Kristin Hagen, Anders Nøklestad, Per Erik Solberg, and Janne Bondi Johannessen. 2018. The LIA treebank of spoken Norwegian dialects. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Langua… find
  135. Martha Palmer, Rajesh Bhatt, Bhuvana Narasimhan, Owen Rambow, Dipti Misra Sharma, and Fei Xia. 2009. Hindi syntax: Annotating dependency, lexical predicate-argument structure, and phrase structure. In The 7th International Conference on Natural Language Processing, pages 14–17. find
  136. Niko Partanen, Rogier Blokland, KyungTae Lim, Thierry Poibeau, and Michael Rießler. 2018. The first Komi-Zyrian Universal Dependencies treebanks. In Proceedings of the Second Workshop on Universal Dependencies (UDW 2018), pages 126– 132, Brussels, Belgium. Association for Computational Linguistics. find
  137. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Te-jani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and… find our card
  138. Agnieszka Patejuk and Adam Przepiórkowski. 2018. From Lexical Functional Grammar to Enhanced Universal Dependencies: Linguistically informed treebanks of Polish. Institute of Computer Science, Polish Academy of Sciences, Warsaw. find
  139. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolog… find
  140. Jussi Piitulainen and UD Finnish-FTB. Hanna Nurmi. 2017. https://github.com/ UniversalDependencies/UD_Finnish-FTB . ↗
  141. Tommi A Pirinen. 2019. Building minority dependency treebanks, dictionaries and computational grammars at the same time—an experiment in Karelian treebanking. In Proceedings of the Third Workshop on Universal Dependencies (UDW, SyntaxFest 2019), pages 132–136, Paris, France. Association for Computat… find
  142. Barbara Plank, Anders Søgaard, and Yoav Goldberg. 2016. Multilingual part-of-speech tagging with bidirectional long short-term memory models and auxiliary loss. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 412–418, Berlin,… find our card
  143. Prokopis Prokopidis and Haris Papageorgiou. 2017. Universal Dependencies for Greek. In Proceedings of the NoDaLiDa 2017 Workshop on Universal Dependencies (UDW 2017), pages 102–106, Gothenburg, Sweden. Association for Computational Linguistics. find
  144. Yada Pruksachatkun, Phil Yeres, Haokun Liu, Jason Phang, Phu Mon Htut, Alex Wang, Ian Tenney, and Samuel R. Bowman. 2020. jiant: A software toolkit for research on general-purpose text understanding models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: S… find
  145. Sampo Pyysalo, Jenna Kanerva, Anna Missilä, Veronika Laippala, and Filip Ginter. 2015. Universal Dependencies for Finnish. In Proceedings of the 20th Nordic Conference of Computational Linguistics (NODALIDA 2015), pages 163–172, Vilnius, Lithuania. Linköping University Electronic Press, Sweden. find
  146. Peng Qi and Koichi UD Chinese-GSDSimp. Yasuoka. 2019. https://github.com/UniversalDependencies/UD_ Chinese-GSDSimp . ↗
  147. Alexandre Rademaker, Fabricio Chalub, Livy Real, Cláudia Freitas, Eckhard Bick, and Valeria de Paiva. 2017. Universal Dependencies for Portuguese. In Proceedings of the Fourth International Conference on Dependency Linguistics (Depling 2017), pages 197–206, Pisa,Italy. Linköping University Electro… find
  148. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI blog. find
  149. Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. Know what you don’t know: Unanswerable questions for SQuAD. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 784– 789, Melbourne, Australia. Association for Computational Lin… find our card
  150. Taraka Rama and Sowmya Vajjala. 2017. A Telugu treebank based on a grammar book. In Proceedings of the 16th International Workshop on Treebanks and Linguistic Theories, pages 119–128, Prague, Czech Republic. find
  151. Loganathan Ramasamy and Zdeněk Žabokrtský. 2012. Prague dependency style treebank for Tamil. In Proceedings of Eighth International Conference on Language Resources and Evaluation (LREC 2012), pages 1888–1894, İstanbul, Turkey. find
  152. Vinit Ravishankar. 2017. A Universal Dependencies treebank for Marathi. In Proceedings of the 16th International Workshop on Treebanks and Linguistic Theories, pages 190–200, Prague, Czech Republic. find
  153. Ines Rehbein, Josef Ruppenhofer, and Bich-Ngoc Do. 2019. tweeDe – a Universal Dependencies treebank for German tweets. In Proceedings of the 18th International Workshop on Treebanks and Linguistic Theories (TLT, SyntaxFest 2019), pages 100–108, Paris, France. Association for Computational Linguistic… find
  154. Eirı́kur Rögnvaldsson, Anton Karl Ingason, Einar Freyr Sigurosson, and Joel Wallenberg. 2012. The Icelandic parsed historical corpus (IcePaHC). In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), pages 1977–1984, Istanbul, Turkey. European Language… find
  155. Sebastian Ruder. 2017. An overview of multi-task learning in deep neural networks. arXiv preprint arXiv:1706.05098. find our card
  156. Sebastian Ruder and Barbara Plank. 2018. Strong baselines for neural semi-supervised learning under domain shift. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1044–1054, Melbourne, Australia. Association for Computational L… find
  157. Jack Rueter and Niko Partanen. 2019. Survey of Uralic Universal Dependencies development. In Workshop on Universal Dependencies, page 78. The Association for Computational Linguistics. find
  158. Jack Rueter, Niko Partanen, and Larisa Ponomareva. 2020. On the questions in developing computational infrastructure for Komi-permyak. In Proceedings of the Sixth International Workshop on Computational Linguistics of Uralic Languages, pages 15–25, Wien, Austria. Association for Computational Lingui… find
  159. Jack Rueter and Francis Tyers. 2018. Towards an open-source universal-dependency treebank for Erzya. In Proceedings of the Fourth International Workshop on Computational Linguistics of Uralic Languages, pages 106–118. find
  160. Jack Michael Rueter. 2018. Mordva. In The Uralic Languages. Routledge. find
  161. Mohammad Sadegh Rasooli, Pegah Safari, Amirsaeid Moloodi, and Alireza Nourian. 2020. The Persian dependency treebank made universal. arXiv eprints, pages arXiv–2009. find our card
  162. Alessio Salomoni. 2019. UD German-LIT. https://github.com/UniversalDependencies/UD_ German-LIT . ↗
  163. Tanja Samardžić, Mirjana Starović, Željko Agić, and Nikola Ljubešić. 2017. Universal Dependencies for Serbian in comparison with Croatian and other Slavic languages. In Proceedings of the 6th Workshop on Balto-Slavic Natural Language Processing, pages 39–44, Valencia, Spain. Association for C… find
  164. Stephanie Samson and UD Tagalog-TRG. Cagrı Cöltekin. 2020. https://github.com/ UniversalDependencies/UD_Tagalog-TRG . ↗
  165. Manuela Sanguinetti and Cristina Bosco. 2014. Towards a Universal Stanford Dependencies parallel treebank. In Proceedings of the 13th Workshop on Treebanks and Linguistic Theories (TLT-13). Springer. find
  166. Manuela Sanguinetti, Cristina Bosco, Alberto Lavelli, Alessandro Mazzei, Oronzo Antonelli, and Fabio Tamburini. 2018. PoSTWITA-UD: an Italian Twitter treebank in Universal Dependencies. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki… find
  167. Victor Sanh, Thomas Wolf, and Sebastian Ruder. 2019. A hierarchical multi-task approach for learning embeddings from semantic tasks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 6949–6956, Honolulu, Hawaii, USA. find
  168. Kengatharaiyer Sarveswaran and Gihan Dias. 2020. ThamizhiUDp: A dependency parser for Tamil. arXiv preprint arXiv:2012.13436. find
  169. Kevin Scannell. 2020. Universal Dependencies for Manx Gaelic. In Proceedings of the Fourth Workshop on Universal Dependencies (UDW 2020), pages 152–157, Barcelona, Spain (Online). Association for Computational Linguistics. find
  170. Djamé Seddah and Marie Candito. 2016. Hard time parsing questions: Building a QuestionBank for French. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 2366–2370, Portorož, Slovenia. European Language Resources Association (ELRA). find
  171. Mojgan Seraji, Filip Ginter, and Joakim Nivre. 2016. Universal Dependencies for Persian. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 2361–2365, Portorož, Slovenia. European Language Resources Association (ELRA). find
  172. Binyam Ephrem Seyoum, Yusuke Miyao, and Baye Yimam Mekonnen. 2018. Universal Dependencies for Amharic. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA). find our card
  173. Tatiana Shavrina and Olga Shapovalova. 2017. To the methodology of corpus construction for machine learning:“Taiga” syntax tree corpus and parser. In Proceedings of “CORPORA-2017” International Conference, pages 78–84. find
  174. Mo Shen, Ryan McDonald, Daniel Zeman, and Peng Qi. 2016. UD Chinese-GSD. https://github.com/UniversalDependencies/UD_ Chinese-GSD . ↗
  175. Timothy Shopen. 2018. UD Warlpiri-UFAL. https://github.com/UniversalDependencies/UD_ Lithuanian-HSE . ↗
  176. Natalia Silveira, Timothy Dozat, Marie-Catherine de Marneffe, Samuel Bowman, Miriam Connor, John Bauer, and Chris Manning. 2014. A gold standard dependency corpus for English. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 2897– 2904, Reykj… find
  177. Kiril Simov, Petya Osenova, Alexander Simov, and Milen Kouylekov. 2005. Design and implementation of the Bulgarian HPSG-based treebank. Journal of Research on Language and Computation. Special Issue, pages 495–522. find
  178. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1631–… find
  179. Anders Søgaard and Yoav Goldberg. 2016. Deep multi-task learning with low level tasks supervised at lower layers. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 231–235, Berlin, Germany. Association for Computational Linguis… find our card
  180. Achim Stein and Sophie Prévost. 2013. Syntactic annotation of medieval texts. New methods in historical corpora, 3:275. find
  181. Milan Straka. 2018. UDPipe 2.0 prototype at CoNLL 2018 UD shared task. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 197–207, Brussels, Belgium. Association for Computational Linguistics. find our card
  182. Sara Stymne, Miryam de Lhoneux, Aaron Smith, and Joakim Nivre. 2018. Parser training with heterogeneous treebanks. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 619– 625, Melbourne, Australia. Association for Computational… find
  183. Umut Sulubacak, Memduh Gokirmak, Francis Tyers, Çağrı Çöltekin, Joakim Nivre, and Gülşen Eryiğit. 2016. Universal Dependencies for Turkish. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 3444–3454, Osaka, Japan. find
  184. The COLING 2016 Organizing Committee. find
  185. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems, volume 27, pages 3104–3112, Montreal, Canada. find
  186. Guillaume Thomas. 2019. Universal Dependencies for Mbyá Guaranı́. In Proceedings of the Third Workshop on Universal Dependencies (UDW, SyntaxFest 2019), pages 70–77, Paris, France. Association for Computational Linguistics. find
  187. Erik F. Tjong Kim Sang and Fien De Meulder. 2003. Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. In Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, pages 142–147. find
  188. Marsida Toska, Joakim Nivre, and Daniel Zeman. 2020. Universal Dependencies for Albanian. In Proceedings of the Fourth Workshop on Universal Dependencies (UDW 2020), pages 178–188, Barcelona, Spain (Online). Association for Computational Linguistics. find
  189. Utku Türk, Furkan Atmaca, Şaziye Betül Özateş, Gözde Berk, Seyyit Talha Bedir, Abdullatif Köksal, Balkiz Öztürk Başaran, Tunga Güngör, and Arzucan Özgür. 2020. Resources for Turkish dependency parsing: Introducing the BOUN treebank and the BoAT annotation tool. find
  190. Francis Tyers and Karina Mishchenkova. 2020. Dependency annotation of noun incorporation in polysynthetic languages. In Proceedings of the Fourth Workshop on Universal Dependencies (UDW 2020), pages 195–204, Barcelona, Spain (Online). Association for Computational Linguistics. find
  191. Francis Tyers, Mariya Sheyanova, Aleksandra Martynova, Pavel Stepachev, and Konstantin Vinogorodskiy. 2018. Multi-source synthetic treebank creation for improved cross-lingual dependency parsing. In Proceedings of the Second Workshop on Universal Dependencies (UDW 2018), pages 144–150, Brussels, Bel… find
  192. Francis M Tyers and Vinit Ravishankar. 2018. A prototype dependency treebank for Breton. In Actes de la Conférence TALN. Volume 1 - Articles longs, articles courts de TALN, pages 197–204, Rennes, France. ATALA. find
  193. Francis M. Tyers and Mariya Sheyanova. 2017. Annotation schemes in North Sámi dependency parsing. In Proceedings of the Third Workshop on Computational Linguistics for Uralic Languages, pages 66– 75, St. Petersburg, Russia. Association for Computational Linguistics. find
  194. Veronika Vincze, Dóra Szauter, Attila Almási, György Móra, Zoltán Alexin, and János Csirik. 2010. Hungarian dependency treebank. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC’10), Valletta, Malta. European Language Resources Association (ELRA… find
  195. Valentin Vydrin. 2013. Bamana Reference Corpus (BRC). Procedia-Social and Behavioral Sciences, 95:75–80. find
  196. Joachim Wagner, James Barry, and Jennifer Foster. 2020. Treebank embedding vectors for out-of-domain dependency parsing. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8812–8818, Online. Association for Computational Linguistics. find
  197. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 353–355,… find our card
  198. Hongmin Wang, Jie Yang, and Yue Zhang. 2019. From genesis to creole language: Transfer learning for Singlish Universal Dependencies parsing and pos tagging. ACM Transactions on Asian and Low-Resource Language Information Processing (TAL-LIP), 19(1):1–29. find
  199. Alex Warstadt, Amanpreet Singh, and Samuel R Bowman. 2019. Neural network acceptability judgments. Transactions of the Association for Computational Linguistics, 7:625–641. find
  200. Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Pa… find
  201. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lho… find our card
  202. Tak-sum Wong, Kim Gerdes, Herman Leung, and John Lee. 2017. Quantitative comparative syntax on the Cantonese-Mandarin parallel dependency treebank. In Proceedings of the Fourth International Conference on Dependency Linguistics (Depling 2017), pages 266–275, Pisa,Italy. Linköping University Electro… find
  203. Alina Wróblewska. 2018. Extended and enhanced Polish dependency bank in Universal Dependencies format. In Proceedings of the Second Workshop on Universal Dependencies (UDW 2018), pages 173–182, Brussels, Belgium. Association for Computational Linguistics. find
  204. UD Assyrian-AS. https://github.com/UniversalDependencies/UD_ Assyrian-AS . ↗
  205. Koichi Yasuoka. 2019. Universal dependencies treebank of the four books in Classical Chinese. In DADH2019: 10th International Conference of Digital Archives and Digital Humanities, pages 20–28. Digital Archives and Digital Humanities. find
  206. M Yavrumyan, H Khachatrian, A Danielyan, and G Arakelyan. 2017. ArmTDP: Eastern Armenian treebank and dependency parser. In XI International Conference on Armenian Linguistics, Abstracts. Yerevan. find
  207. Manzil Zaheer, Sashank Reddi, Devendra Sachan, Satyen Kale, and Sanjiv Kumar. 2018. Adaptive methods for nonconvex optimization. In Advances in Neural Information Processing Systems, volume 31, pages 9793–9803, Montreal, Canada. find
  208. Shorouq Zahra. 2020. Parsing low-resource Levantine Arabic: Annotation projection versus small-sized annotated data. find
  209. Amir Zeldes. 2017. The GUM corpus: Creating multilayer resources in the classroom. Language Resources and Evaluation, 51(3):581–612. find
  210. Amir Zeldes and Mitchell Abrams. 2018. The Coptic Universal Dependency treebank. In Proceedings of the Second Workshop on Universal Dependencies (UDW 2018), pages 192–201, Brussels, Belgium. Association for Computational Linguistics. find
  211. Dan Zeman, Anna Nedoluzhko, and Martin Majliš. 2017. UD Upper Sorbian-UFAL. https://github.com/UniversalDependencies/UD_ Upper_Sorbian-UFAL . ↗
  212. Daniel Zeman. 2017. Slovak dependency treebank in Universal Dependencies. Journal of Linguistics/Jazykovednỳ casopis, 68(2):385–395. find
  213. Dataset size RTE 2k MRPC 4k CoLa 9k SST-2 67k QNLI 105k QQP 364k MNLI 393k MNLI-mis 393k SNLI 550k Single All Smoothed 67.1 69.3 72.9 85.5 81.6 82.8 74.7 70.2 72.7 88.4 88.2 87.6 85.2 82.3 83.1 90.5 90.1 90.3 80.2 79.2 78.8 80.8 79.7 80.1 88.9 88.1 88.4 *Table 5: The scores (accuracy) per dataset on… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/goot-2021-massive-choice-ample-tasks-machamp/