Elamur Research⌕
☰
home / papers / 2021 / Cascading Adaptors to Leverage English Data to Improve Performance of Question Answering for Low-Resource Languages

Cascading Adaptors to Leverage English Data to Improve Performance of Question Answering for Low-Resource Languages

Paper · 2021 · Hariom Pandya, Bhavik Ardeshna and Brijesh Bhatt

Summary

The paper asks how far multilingual transformers can push extractive question answering for languages with little QA training data, and whether small adapter modules help more than plain fine-tuning. The authors had no training split for the non-English languages, so they built one. They combined the XQuAD data with the MLQA test portion to form training data and reserved the MLQA dev set as a shared test set for seven languages: English, German, Spanish, Arabic, Hindi, Vietnamese and Simplified Chinese.

They compared four setups:

  • A: full fine-tuning of mBERT and XLM-RoBERTa.
  • B: task adapters only (Houlsby vs. Pfeiffer).
  • C: language adapters, alone and stacked under a task adapter.
  • D: MAD-X-style zero-shot transfer. A task adapter trained over the English language adapter is reused after swapping in the target-language adapter.

XLM-RoBERTa large gave the strongest scores. Among base-size variants, language adapters gave small gains for several languages. Zero-shot MAD-X came close to supervised results for Hindi and German. The work matters because it shows how to reuse English QA data cheaply through modular, swappable components.

Key points

  • Data split: train = MLQA test + XQuAD; test = MLQA dev.
    • Sizes for most languages are roughly 5.7k–6.9k train and about 500 test (Hindi 6854/507).
    • English has 12780/1148.
  • Setup A (F1/EM):
    • Hindi: mBERT 56.25/39.45, XLM-R base 64.49/48.32, XLM-R large 73.37/56.02.
    • English with XLM-R large reaches 85.98/74.39.
    • Chinese is weakest for all models (about 42–50 F1).
  • Setup C, language adapter only: beats task adapters for Hindi (66.14/49.11), Spanish (70.25/49.6), German, Chinese and English.
    • Stacking a task adapter on top lowered scores for most languages; Arabic was the exception.
  • Task adapter architecture: the choice between Houlsby and Pfeiffer had negligible effect for most languages.
  • Setup D, zero-shot MAD-X from English:
    • Hindi 65.24/48.91 and German 60.42/43.35.
    • Arabic drops to 50.12/31.33.
    • Jaccard and WER are reported alongside F1/EM for all setups.

Interesting ideas

  • I1. Keeping the task skill and the language knowledge in separate plug-in modules lets one task module be trained once on English and reused by swapping only the language part; this is useful wherever one capability must serve several input varieties.
  • I2. When a benchmark ships only evaluation splits, the authors carve their own training data from them but hold one fixed split as the test set for every language, so comparisons stay consistent.
  • I3. Freezing the backbone and updating only a small adapter makes the trained component cheap to store, share and compare.
  • I4. Scoring answers with several overlap metrics (EM, F1, Jaccard, WER) shows partial-credit behaviour that exact match alone hides.
  • I5. The tables show that stacking more modules does not automatically help (language adapter alone often beat language + task), which argues for always comparing against the simpler configuration.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tree): The same split could keep a parser's core weights fixed while small swappable modules handle different UD dialects or genres, rather than retraining gparse per treebank.
  • I2 (learn): Our evaluation sets for fables and school tests could follow the same discipline: one frozen test portion shared by all experiments, with everything else free for silver training.
  • I3 (learn): Repair rules or teacher-tier corrections could likewise be packaged as small add-ons over a frozen base parser, so each one's effect is measurable and reversible.
  • I4 (answer): QA readers on tales could report exact and token-overlap scores together, separating "wrong span" from "slightly too long span" errors.
  • I5 (learn): Every added component in the pipeline should be evaluated against the version without it, as a built-in negative control.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Dharmsinh Desai UniversityNadiād · India2 works · 92 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 4Research subjects and languages the work is about.English · 2,263 worksSemitic languages · 2,908 worksQuestion answering · 441 worksLow-resource languages · 267 works

Cite

Cite as Hariom Pandya, Bhavik Ardeshna and Brijesh Bhatt. 2021. Cascading Adaptors to Leverage English Data to Improve Performance of Question Answering for Low-Resource Languages. Proceedings of the 18th International Conference on Natural Language Processing (ICON).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{pandya2021cascading,
  title = {{Cascading Adaptors to Leverage English Data to Improve Performance of Question Answering for Low-Resource Languages}},
  author = {Hariom Pandya, Bhavik Ardeshna and Brijesh Bhatt},
  booktitle = {Proceedings of the 18th International Conference on Natural Language Processing (ICON)},
  year = {2021},
  url = {https://aclanthology.org/2021.icon-main.66/}
}

References

19 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. 2020. On the cross-lingual transferability of monolingual representations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4623–4637, Online. Association for Computational Linguistics.
  2. Ankur Bapna and Orhan Firat. 2019. Simple, scalable adaptation for neural machine translation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1538– 1548, Hong Kong, China. Association for Computational Linguistics. ↗
  3. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8440– 8451, Online. Association for Computational Linguistics. ↗
  4. Pieter Delobelle, Thomas Winters, and Bettina Berendt. 2020. RobBERT: a Dutch RoBERTa-based Language Model. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3255–3265, Online. Association for Computational Linguistics.
  5. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. ↗
  6. Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for NLP. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 2790–2799. PMLR. ↗
  7. Divyanshu Kakwani, Anoop Kunchukuttan, Satish Golla, Gokul N.C., Avik Bhattacharyya, Mitesh M. Khapra, and Pratyush Kumar. 2020. IndicNLPSuite: Monolingual corpora, evaluation benchmarks and pre-trained multilingual language models for Indian languages. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4948– 4961, Online. Association for Computational Linguistics.
  8. Patrick Lewis, Barlas Oguz, Ruty Rinott, Sebastian Riedel, and Holger Schwenk. 2020. MLQA: Evaluating cross-lingual extractive question answering. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7315– 7330, Online. Association for Computational Linguistics. ↗
  9. Rudra Murthy, Anoop Kunchukuttan, and Pushpak Bhattacharyya. 2019. Addressing word-order divergence in multilingual neural machine translation for extremely low resource languages. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3868–3873, Minneapolis, Minnesota. Association for Computational Linguistics. find
  10. HA Pandya and BS Bhatt. 2021. Question answering survey: Directions, challenges, datasets, evaluation matrices. Journal of Xidian University, 15(4):152– 168. ↗
  11. Youngja Park, Siddharth Patwardhan, Karthik Visweswariah, and Stephen Gates. 2008. An empirical analysis of word error rate and keyword error rate. pages 2070–2073. find
  12. Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. 2021. AdapterFusion: Non-destructive task composition for transfer learning. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 487–503, Online. Association for Computational Linguistics. ↗
  13. Jonas Pfeiffer, Andreas Rücklé, Clifton Poth, Aishwarya Kamath, Ivan Vulić, Sebastian Ruder, Kyunghyun Cho, and Iryna Gurevych. 2020a. AdapterHub: A framework for adapting transformers. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 46–54, Online. Association for Computational Linguistics.
  14. Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych, and Sebastian Ruder. 2020b. MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7654–7673, Online. Association for Computational Linguistics.
  15. Telmo Pires, Eva Schlinger, and Dan Garrette. 2019. How multilingual is multilingual BERT? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4996–5001, Florence, Italy. Association for Computational Linguistics. ↗
  16. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67. ↗
  17. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383–2392, Austin, Texas. Association for Computational Linguistics. ↗
  18. Tulika Saha, Dhawal Gupta, Sriparna Saha, and Pushpak Bhattacharyya. 2021. A unified dialogue management strategy for multi-intent dialogue conversations in multiple languages. ACM Trans. Asian Low-Resour. Lang. Inf. Process., 20(6). ↗
  19. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/pandya-2021-cascading-adaptors-leverage-english-data/