Elamur Research⌕
☰
home / papers / 2018 / Data Augmentation via Dependency Tree Morphing for Low-Resource Languages

Data Augmentation via Dependency Tree Morphing for Low-Resource Languages

Summary

Neural models perform poorly when labeled data is scarce. Unlike computer vision or speech processing, NLP has almost no simple ways to artificially enlarge the training set. The authors carry over two classic image-processing operations to dependency trees. "Cropping" (crop) keeps the predicate with only one subject or object together with its subtree, which yields a shorter sentence. "Rotation" (rotate) reorders such fragments around the root. This suits languages with free word order and a rich case system. Both methods were applied to small treebanks from UD v2.1. The effect was tested on POS tagging: a character-level bi-LSTM model with the same hyperparameters was used for all languages. Augmentation improved accuracy for most languages. Languages with the smallest training sets gained the most. Baltic, Slavic (including Ukrainian), Uralic and Turkic languages also gain consistently. For Germanic languages, which have few case markers, it barely helps. The work offers a simple, largely language-independent way to expand the data for low-resource languages.

Key points

  • Data: languages from UD v2.1 with fewer than 120K tokens. Languages without a standard split and those smaller than 5K tokens were discarded. The results table has 19 languages from different families, among them Ukrainian, Belarusian, Slovak, Serbian, Lithuanian and Latvian.
  • Fragments for the operations are identified by the labels nsubj, iobj, obj and obl. A multiword root expression is recognized by the labels fixed, flat, cop and compound. Only fragments at the first level of the tree are reordered. Out of the (n+1)! possible permutations, n are chosen at random, and each operation is applied with probability p ∈ {0.3, 0.7, 1}.
  • The largest relative accuracy gains were for Irish (62.75 → 76.35, +21.68%) and Lithuanian (61.51 → 68.27, +10.98%). The Pearson correlation coefficient between treebank size and gain is −0.35.
  • Ukrainian: 94.41 → 94.57 (rotate, p = 0.3), i.e. +0.17%. For Slavic languages, rotation is generally slightly better than cropping. Exceptions with almost no benefit or a drop: Old Church Slavonic (the corpus consists of biblical texts), Coptic (−0.15%), Telugu and the Germanic languages.
  • The code is available on GitHub: gozdesahin/crop-rotate-augment.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Dependency parsing · Treebanks

Cited by

57 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 53 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d18-1545

@inproceedings{sahin2018data,
  title = {{Data Augmentation via Dependency Tree Morphing for Low-Resource Languages}},
  author = {Şahin, Gözde Gül and Steedman, Mark},
  booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
  doi = {10.18653/v1/d18-1545},
  url = {https://doi.org/10.18653/v1/d18-1545}
}

References

13 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Dan C. Ciresan, Ueli Meier, and Jürgen Schmidhuber. 2012. Multi-column deep neural networks for image classification. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA, June 16-21, 2012, pages 3642–3649. find
  2. Marzieh Fadaee, Arianna Bisazza, and Christof Monz. 2017. Data augmentation for low-resource neural machine translation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 2: Short Papers, pages 567–573. find our card
  3. Richard Futrell, Kyle Mahowald, and Edward Gibson. 2015. Quantifying word order freedom in dependency corpora. In Proceedings of the Third International Conference on Dependency Linguistics (Depling 2015), pages 91–100. Uppsala University, Uppsala, Sweden. find
  4. David Vickrey and Daphne Koller. 2008. Sentence simplification for semantic role labeling. In ACL 2008, Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics, June 15-20, 2008, Columbus, Ohio, USA, pages 344–352. find
  5. Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. In Advances in Neural Information Processing Systems 28, pages 649–657. Curran Associates, Inc. find our card
  6. William Hartmann, Tim Ng, Roger Hsiao, Stavros Tsakalidis, and Richard M Schwartz. 2016. Two-stage data augmentation for low-resourced speech recognition. In Interspeech, pages 2378–2382. find
  7. Robin Jia and Percy Liang. 2016. Data recombination for neural semantic parsing. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers. find
  8. Tom Ko, Vijayaditya Peddinti, Daniel Povey, and Sanjeev Khudanpur. 2015. Audio augmentation for speech recognition. In INTERSPEECH 2015, 16th Annual Conference of the International Speech Communication Association, Dresden, Germany, September 6-10, 2015, pages 3586–3589. find
  9. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems 25: 26th Annual Conference on Neural Information Processing Systems 2012. Proceedings of a meeting held December 3-6, 2… find
  10. Wang Ling, Tiago Luis, Luis Marujo, Ramon F Astudillo, Silvio Amir, Chris Dyer, Alan W Black, and Isabel Trancoso. 2015. Finding function in form: Compositional character models for open vocabulary word representation. In EMNLP, pages 1520– 1530. find our card
  11. Joakim Nivre et al. 2017. Universal dependencies 2.1. LINDAT/CLARIN digital library at the Institute of Formal and Applied Linguistics, Faculty of Mathematics and Physics, Charles University. find
  12. Anton Ragni, Kate M. Knill, Shakti P. Rath, and Mark J. F. Gales. 2014. Data augmentation for low resource languages. In INTERSPEECH 2014, 15th Annual Conference of the International Speech Communication Association, Singapore, September 14-18, 2014, pages 810–814. find
  13. Naoya Takahashi, Michael Gygli, Beat Pfister, and Luc Van Gool. 2016. Deep convolutional neural networks and data augmentation for acoustic event detection. In Interspeech, pages 2982–2986. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/ahin-2018-data-augmentation-dependency-tree-morphing/