Elamur Research⌕
☰
home / papers / 2024 / TIGQA: An Expert-Annotated Question-Answering Dataset in Tigrinya

TIGQA: An Expert-Annotated Question-Answering Dataset in Tigrinya

Paper · 2024 · Hailay Kidu Teklehaymanot, Dren Fazlija, Niloy Ganguly et al.

Summary

The paper addresses the lack of curated reading-comprehension data for Tigrinya, a Ge'ez-script Semitic language with over 10 million speakers. The few existing resources are either drawn from Wikipedia, which has few reliable Tigrinya contributors, or produced by machine translation. The authors first tested whether machine-translating an English SQuAD-style dataset could produce a usable Tigrinya version. Translation quality proved inadequate, so they built TIGQA by hand. Contexts come from scanned Tigrinya-language and Biology school textbooks, digitised with OCR. Subject experts wrote the questions and marked the answer spans in SQuAD format. The authors show that many questions require inference across one or several sentences, not just lexical overlap. They benchmark DrQA variants, AfriBERTa Base and XLM-R against estimated human performance. XLM-R Large is the strongest model, but it remains well below human performance, especially on exact match. The resource matters because it is public and education-oriented, and it can serve as a template for other low-resource QA datasets.

Key points

  • TIGQA has 2,685 question–answer pairs drawn from 537 context paragraphs across 122 topics, such as climate, water and traffic. It is released on GitHub (hailaykidu/TigQA-Datasets).
  • The contexts were extracted from scanned textbooks with the Tesseract OCR engine and annotated by domain experts rather than crowd workers.
  • XLM-R Large scores best, with test EM 66.56 and F1 84.34. Humans reach about 92–94 F1 and about 86–87 EM.
  • Among the other baselines on test F1, DrQA+BERT reaches 60.03, AfriBERTa Base 60.02 and XLM-R Base 68.12. DrQA with Word2vec embeddings is weakest on EM (30.82).
  • In the breakdown analysis, very long answers and "why" questions are hardest, while short entity answers are handled well. The authors also note that Tigrinya tokenization choices affect results.

Interesting ideas

  • I1. Before building a dataset, run an empirical check of whether machine-translating an existing benchmark gives acceptable quality, and only fall back to manual annotation when it does not.
  • I2. Draw QA contexts from school textbooks rather than encyclopedic web text, so the passages are reliable, curriculum-aligned and suited to pedagogy.
  • I3. Label each question with a reasoning type (single-sentence vs. multi-sentence inference) as well as question and answer type, so that scores can be sliced to show where a system really fails.
  • I4. Treat a large gap between F1 and exact match as a signal that the system finds the right region of the text but misplaces the answer boundaries.
  • I5. Measure human performance on the same dev/test splits to provide a ceiling, rather than reporting model scores in isolation.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Mova could gate any translated or LLM-produced silver data behind a similar measured quality check before it enters training.
  • I2 (answer): Mova's school-course readers could be evaluated on textbook passages with expert-written questions, which matches the target use better than Wikipedia-style QA.
  • I3 (answer): tagging Mova's fable and test questions the same way would show whether errors come from cross-sentence linking in the world model or from local lookup.
  • I4 (back): when Mova turns a graph answer back into a text span, tracking an EM-vs-F1 gap would isolate span-boundary realisation errors from wrong-meaning errors.
  • I5 (learn): adding a human ceiling next to negative controls would put Mova's evaluation numbers in context and show how much headroom is real.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Indian Institute of Technology KharagpurKharagpur · India24 works · 714 citations🎓Leibniz University HannoverHanover · Germany5 works · 599 citations🔬L3S Research CenterHanover · Germany3 works · 4 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksQuestion answering · 441 works

Cite

Cite as Hailay Kidu Teklehaymanot, Dren Fazlija, Niloy Ganguly, Gourab Kumar Patro and Wolfgang Nejdl. 2024. TIGQA: An Expert-Annotated Question-Answering Dataset in Tigrinya. Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{teklehaymanot2024tigqa,
  title = {{TIGQA: An Expert-Annotated Question-Answering Dataset in Tigrinya}},
  author = {Hailay Kidu Teklehaymanot, Dren Fazlija, Niloy Ganguly, Gourab Kumar Patro and Wolfgang Nejdl},
  booktitle = {Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)},
  year = {2024},
  url = {https://aclanthology.org/2024.lrec-main.1404/}
}

References

103 references, 31 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Negin Abadani, Jamshid Mozafari, Afsaneh Fatemi, Mohammd Ali Nematbakhsh, and Arefeh Kazemi. 2021. Parsquad: machine translated squad dataset for persian question answering. In 2021 7th International Conference on Web Research (ICWR), pages 163–168. IEEE. find
  2. Nuredin Ali Abdelkadir, Negasi Haile Abadi, and Asmelash Teka Hadgu. 2023. Error analysis of tigrinya–english machine translation systems. In 4th Workshop on African Natural Language Processing. find
  3. Tilahun Abedissa, Ricardo Usbeck, and Yaregal Assabie. 2023. Amqa: Amharic question answering dataset. arXiv:2303.03290. arXiv preprint David Adelani, Md Mahfuz Ibn Alam, Antonios Anastasopoulos, Akshita Bhagia, Marta R Costa-jussà, Jesse Dodge, Fahim Faisal, Christian Federmann, Natalia Fedorova, Francisco Guzmán, et al. 2022. Findings of the wmt’22 shared task on large-scale machine translation evaluation for african languages. In Proceedings of the Seventh Conference on Machine Translation (WMT), pages 773–800. ↗
  4. Benjamin Akera, Jonathan Mukiibi, Lydia Sanyu Naggayi, Claire Babirye, Isaac Owomugisha, Solomon Nsumba, Joyce Nakatumba-Nabende, Engineer Bainomugisha, Ernest Mwebaze, and John Quinn. 2022. Machine translation for african languages: Community creation of datasets and models in uganda. find
  5. Chantal Amrhein, Nikita Moghe, and Liane Guillou. 2022. ACES: Translation accuracy challenge sets for evaluating machine translation metrics. In Proceedings of the Seventh Conference on Machine Translation (WMT), pages 479–513, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics. ↗
  6. Anuoluwapo Aremu, Jesujoba O. Alabi, and David Ifeoluwa Adelani. 2023. Yorc: Yoruba reading comprehension dataset. find
  7. Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. 2020. On the cross-lingual transferability of monolingual representations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4623– 4637, Online. Association for Computational Linguistics.
  8. Akari Asai, Jungo Kasai, Jonathan Clark, Kenton Lee, Eunsol Choi, and Hannaneh Hajishirzi. 2021. XOR QA: Cross-lingual open-retrieval question answering. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 547– 564, Online. Association for Computational Linguistics. ↗
  9. Fikre Belay, Desalegn Berhane, Haftom Teshale, Gebretsadkan Mulubrhan, Tsigab Hagos, Hagos Gebremariam, Tadesse Brhane, and Zubairul Islam. 2023. The effect of war on educational institutions of eastern tigray zone, tigray state, ethiopia. International Journal of Educational Development, 102:102864. find
  10. Asma Ben Abacha, Yassine Mrabet, Yuhao Zhang, Chaitanya Shivade, Curtis Langlotz, and Dina Demner-Fushman. 2021. Overview of the MEDIQA 2021 shared task on summarization in the medical domain. In Proceedings of the 20th Workshop on Biomedical Language Processing, pages 74–85, Online. Association for Computational Linguistics. find
  11. Terra Blevins and Luke Zettlemoyer. 2022. Language contamination helps explains the cross-lingual capabilities of English pretrained models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3563–3574, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. ↗
  12. B Barla Cambazoglu, Mark Sanderson, Falk Scholer, and Bruce Croft. 2021. A review of public datasets in question answering research. In ACM SIGIR Forum, volume 54, pages 1–23. ACM New York, NY, USA. ↗
  13. Casimiro Pio Carrino, Marta R. Costa-jussà, and José A. R. Fonollosa. 2020. Automatic Spanish translation of SQuAD dataset for multilingual question answering. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 5515–5523, Marseille, France. European Language Resources Association. find
  14. Casimiro Pio Carrino, Marta R Costa-Jussà, and José AR Fonollosa. 2019. Automatic spanish translation of the squad dataset for multilingual question answering. arXiv preprint arXiv:1912.05200. ↗
  15. Danqi Chen, Jason Bolton, and Christopher D. Manning. 2016. A thorough examination of the CNN/Daily Mail reading comprehension task. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2358–2367, Berlin, Germany. Association for Computational Linguistics.
  16. Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. Reading Wikipedia to answer open-domain questions. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1870–1879, Vancouver, Canada. Association for Computational Linguistics. ↗
  17. Eunsol Choi, He He, Mohit Iyyer, Mark Yatskar, Wen-tau Yih, Yejin Choi, Percy Liang, and Luke Zettlemoyer. 2018. QuAC: Question answering in context. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2174–2184, Brussels, Belgium. Association for Computational Linguistics. ↗
  18. Jonathan H Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, and Jennimaria Palomaki. 2020. Tydi qa: A benchmark for information-seeking question answering in ty pologically di verse languages. Transactions of the Association for Computational Linguistics, 8:454–470. find
  19. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Unsupervised cross-lingual representation learning at scale. CoRR, abs/1911.02116. ↗
  20. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8440–8451, Online. Association for Computational Linguistics. ↗
  21. Alexis Conneau and Guillaume Lample. 2019. Cross-lingual language model pretraining. Advances in neural information processing systems, 32. find
  22. Danilo Croce, Alexandra Zelenanska, and Roberto Basili. 2018a. Neural learning for question answering in italian. In AI* IA 2018–Advances in Artificial Intelligence: XVIIth International Conference of the Italian Association for Artificial Intelligence, Trento, Italy, November 20–23, 2018, Proceedings 17, pages 389–402. Springer. find
  23. Danilo Croce, Alexandra Zelenanska, and Roberto Basili. 2018b. Neural learning for question answering in italian. In AI*IA 2018 – Advances in Artificial Intelligence, pages 389–402, Cham. Springer International Publishing. find
  24. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. ↗
  25. Martin d’Hoffschmidt, Wacim Belblidia, Quentin Heinrich, Tom Brendlé, and Maxime Vidal. 2020. FQuAD: French question answering dataset. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1193–1208, Online. Association for Computational Linguistics. find
  26. Daria Dzendzik, Jennifer Foster, and Carl Vogel. 2021. English machine reading comprehension datasets: A survey. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8784–8804, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  27. Pavel Efimov, Andrey Chertok, Leonid Boytsov, and Pavel Braslavski. 2020. Sberquad–russian reading comprehension dataset: Description and analysis. In Experimental IR Meets Multilinguality, Multimodality, and Interaction: 11th International Conference of the CLEF Association, CLEF 2020, Thessaloniki, Greece, September 22–25, 2020, Proceedings 11, pages 3–15. Springer. find
  28. Roald Eiselen and Tanja Gaustad. 2023. Deep learning and low-resource languages: How much data is enough? a case study of three linguistically distinct South African languages. In Proceedings of the Fourth workshop on Resources for African Indigenous Languages (RAIL 2023), pages 42–53, Dubrovnik, Croatia. Association for Computational Linguistics. ↗
  29. Fitsum Gaim, Wonsuk Yang, Hancheol Park, and Jong Park. 2023. Question-answering in a low-resourced language: Benchmark dataset and models for Tigrinya. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11857–11870, Toronto, Canada. Association for Computational Linguistics.
  30. Tanja Gaustad and Martin Puttkammer. 2022. Development of linguistically annotated parallel language resources for four south african languages. Journal of the Digital Humanities Association of Southern Africa, 3(03). ↗
  31. Augusto Gonzalez-Bonorino, Eitel JM Lauría, and Edward Presutti. 2022. Implementing open-domain question-answering in a college setting: An end-to-end methodology and a preliminary exploration. In CSEDU (2), pages 66–75. find
  32. Marissa Griesel and Sonja Bosch. 2020. Navigating challenges of multilingual resource development for under-resourced languages: The case of the African Wordnet project. In Proceedings of the first workshop on Resources for African Indigenous Languages, pages 45–50, Marseille, France. European Language Resources Association (ELRA). find
  33. Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc. ↗
  34. Lynette Hirschman, Marc Light, Eric Breck, and John D Burger. 1999. Deep read: A reading comprehension system. In Proceedings of the 37th annual meeting of the Association for Computational Linguistics, pages 325–332. find
  35. Daniel Hládek, Ján Staš, Jozef Juhár, and Tomáš Koctúr. 2023. Slovak dataset for multilingual question answering. IEEE Access, 11:32869– 32881. ↗
  36. Daniel Hládek, Ján Staš, Jozef Juhár, and Tomáš Koctúr. 2023. Slovak dataset for multilingual question answering. IEEE Access, 11:32869– 32881. ↗
  37. Diana Constantina Hoefels, Çağrı Çöltekin, and Irina Diana Mădroane. 2022. CoRoSeOf - an annotated corpus of Romanian sexist and offensive tweets. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 2269–2281, Marseille, France. European Language Resources Association. find
  38. Naghme Jamali, Yadollah Yaghoobzadeh, and Heshaam Faili. 2022. PerCQA: Persian community question answering dataset. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 6083–6092, Marseille, France. European Language Resources Association. find
  39. Jing Jiang. 2008. Domain adaptation in natural language processing. University of Illinois at Urbana-Champaign. find
  40. Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1601–1611, Vancouver, Canada. Association for Computational Linguistics. ↗
  41. Divyanshu Kakwani, Anoop Kunchukuttan, Satish Golla, NC Gokul, Avik Bhattacharyya, Mitesh M Khapra, and Pratyush Kumar. 2020. Indicnlpsuite: Monolingual corpora, evaluation benchmarks and pre-trained multilingual language models for indian languages. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4948–4961.
  42. Alan S Kaye. 2007. Morphologies of Asia and Africa. Penn State Press. find
  43. Rachel Keraron, Guillaume Lancrenon, Mathilde Bras, Frédéric Allary, Gilles Moyse, Thomas Scialom, Edmundo-Pavel Soriano-Morales, and Jacopo Staiano. 2020. Project piaf: Building a native french question-answering dataset. arXiv preprint arXiv:2007.00968. ↗
  44. Vlado Keselj. 2009. Book review: Speech and language processing by daniel jurafsky and james h. martin. Computational Linguistics, 35(3). find
  45. Paul R Kingsbury and Martha Palmer. 2002. From treebank to propbank. In LREC, pages 1989– 1993.
  46. Tomáš Kočiský, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gábor Melis, and Edward Grefenstette. 2018. The NarrativeQA reading comprehension challenge. Transactions of the Association for Computational Linguistics, 6:317–328. ↗
  47. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. 2019. Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:453–466. ↗
  48. Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. 2017. RACE: Large-scale ReAding comprehension dataset from examinations. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 785–794, Copenhagen, Denmark. Association for Computational Linguistics. ↗
  49. Richard Lastrucci, Jenalea Rajab, Matimba Shingange, Daniel Njini, and Vukosi Marivate. 2023. Preparing the vuk’uzenzele and ZA-gov-multilingual South African multilingual corpora. In Proceedings of the Fourth workshop on Resources for African Indigenous Languages (RAIL 2023), pages 18–25, Dubrovnik, Croatia. Association for Computational Linguistics. find
  50. Patrick Lewis, Barlas Oguz, Ruty Rinott, Sebastian Riedel, and Holger Schwenk. 2020. MLQA: Evaluating cross-lingual extractive question answering. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7315–7330, Online. Association for Computational Linguistics. ↗
  51. Jiahua Liu, Yankai Lin, Zhiyuan Liu, and Maosong Sun. 2019. Xqa: A cross-lingual open-domain question answering dataset. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2358–2368. find
  52. Arle Lommel. 2018. Metrics for translation quality assessment: A case for standardising error typologies. Translation quality assessment: From principles to practice, pages 109–127. ↗
  53. Rooweither Mabuya, Don Mthobela, Mmasibidi Setaka, and Menno Van Zaanen. 2023. Proceedings of the fourth workshop on resources for african indigenous languages (rail 2023). In Proceedings of the Fourth workshop on Resources for African Indigenous Languages (RAIL 2023). find
  54. Kateřina Macková. 2022. Question answering in czech via machine translation and cross-lingual transfer. find
  55. Mack Makgatho, Vukosi Marivate, Tshephisho Sefara, and Valencia Wagner. 2022. Training cross-lingual embeddings for setswana and sepedi. Journal of the Digital Humanities Association of Southern Africa, 3(03). ↗
  56. Mitchell P Marcus, Beatrice Santorini, Mary Ann Marcinkiewicz, and Ann Taylor. 1999. Treebank-3. Linguistic Data Consortium, Philadelphia, 14. find
  57. Elvis Mboning Tchiaze, Jean Marc Bassahak, Daniel Baleba, Ornella Wandji, and Jules Assoumou. 2020. Building collaboration-based resources in endowed African languages: Case of NTeALan dictionaries platform. In Proceedings of the first workshop on Resources for African Indigenous Languages, pages 51–56, Marseille, France. European Language Resources Association (ELRA). find
  58. Timo Möller, Julian Risch, and Malte Pietsch. 2021a. GermanQuAD and GermanDPR: Improving non-English question answering and passage retrieval. In Proceedings of the 3rd Workshop on Machine Reading for Question Answering, pages 42–50, Punta Cana, Dominican Republic. Association for Computational Linguistics. ↗
  59. Timo Möller, Julian Risch, and Malte Pietsch. 2021b. Germanquad and germandpr: Improving non-english question answering and passage retrieval. arXiv preprint arXiv:2104.12741. ↗
  60. Mahmud Momoh. 2023. Vowels and the Igala language resources. In Proceedings of the Fourth workshop on Resources for African Indigenous Languages (RAIL 2023), pages 106– 114, Dubrovnik, Croatia. Association for Computational Linguistics. ↗
  61. Xiangyang Mou, Chenghao Yang, Mo Yu, Bingsheng Yao, Xiaoxiao Guo, Saloni Potdar, and Hui Su. 2021. Narrative question answering with cutting-edge open-domain qa techniques: A comprehensive study. Transactions of the Association for Computational Linguistics, 9:1032– 1046. ↗
  62. Kiet Nguyen, Vu Nguyen, Anh Nguyen, and Ngan Nguyen. 2020. A Vietnamese dataset for evaluating machine reading comprehension. In Proceedings of the 28th International Conference on Computational Linguistics, pages 2595–2605, Barcelona, Spain (Online). International Committee on Computational Linguistics. ↗
  63. Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. Ms marco: A human generated machine reading comprehension dataset. choice, 2640:660.
  64. Constantin Dragos Nicolae, Rohan Kumar Yadav, and Dan Tufiş. 2023. Evaluation of language models on romanian xquad and roitd datasets. INTERNATIONAL JOURNAL OF COMPUTERS COMMUNICATIONS & CON-TROL, 18(1). ↗
  65. Kelechi Ogueji, Yuxin Zhu, and Jimmy Lin. 2021a. Small data? no problem! exploring the viability of pretrained multilingual language models for low-resourced languages. In Proceedings of the 1st Workshop on Multilingual Representation Learning, pages 116–126, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  66. Kelechi Ogueji, Yuxin Zhu, and Jimmy Lin. 2021b. Small data? no problem! exploring the viability of pretrained multilingual language models for low-resourced languages. In Proceedings of the 1st Workshop on Multilingual Representation Learning, pages 116–126.
  67. Amandalynne Paullada, Inioluwa Deborah Raji, Emily M Bender, Emily Denton, and Alex Hanna. 2021. Data and its (dis) contents: A survey of dataset development and use in machine learning research. Patterns, 2(11). ↗
  68. Martin Plaut and Sarah Vaughan. 2023. Understanding Ethiopia’s Tigray War. Hurst Publishers. ↗
  69. Rifki Afina Putri and Alice Oh. 2022. IDK-MRC: Unanswerable questions for Indonesian machine reading comprehension. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 6918– 6933, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. find
  70. Martin Puttkammer and Jakobus S. Du Toit. 2022. Canonical segmentation and syntactic morpheme tagging of four resource-scarce nguni languages. Journal of the Digital Humanities Association of Southern Africa, 3(03). ↗
  71. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383–2392, Austin, Texas. Association for Computational Linguistics. ↗
  72. Siva Reddy, Danqi Chen, and Christopher D. Manning. 2019. CoQA: A conversational question answering challenge. Transactions of the Association for Computational Linguistics, 7:249– 266. ↗
  73. Matthew Richardson, Christopher JC Burges, and Erin Renshaw. 2013a. Mctest: A challenge dataset for the open-domain machine comprehension of text. In Proceedings of the 2013 conference on empirical methods in natural language processing, pages 193–203.
  74. Matthew Richardson, Christopher J.C. Burges, and Erin Renshaw. 2013b. MCTest: A challenge dataset for the open-domain machine comprehension of text. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 193–203, Seattle, Washington, USA. Association for Computational Linguistics.
  75. Anna Rogers, Matt Gardner, and Isabelle Augenstein. 2023. Qa dataset explosion: A taxonomy of nlp resources for question answering and reading comprehension. ACM Computing Surveys, 55(10):1–45. ↗
  76. Yuji Roh, Geon Heo, and Steven Euijong Whang. 2019. A survey on data collection for machine learning: a big data-ai integration perspective. IEEE Transactions on Knowledge and Data Engineering, 33(4):1328–1347. ↗
  77. Uma Roy, Noah Constant, Rami Al-Rfou, Aditya Barua, Aaron Phillips, and Yinfei Yang. 2020. LAReQA: Language-agnostic answer retrieval from a multilingual pool. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5919–5930, Online. Association for Computational Linguistics. find
  78. Amanuel Sahle. 1998. Sewasiw Tigrinya b’sefihu: A comprehensive Tigrinya grammar. Red Sea Press, US. find
  79. Priyanka Sen and Amir Saffari. 2020. What do models learn from question answering datasets? In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2429–2438, Online. Association for Computational Linguistics. find
  80. Lim Seungyoung, Kim Myungji, Lee Jooyoul, et al. 2019. Korquad1. 0: Korean qa dataset for machine reading comprehension. arXiv preprint arXiv: 1909.07005. ↗
  81. Johannes Sibeko. 2023. Exploring linguistic complexity of english first additional language grade 12 and grade 11 exam texts. Journal of the Digital Humanities Association of Southern Africa, 4(02). ↗
  82. Johannes Sibeko and Orphée De Clercq. 2023. A corpus-based list of frequently used words in sesotho. In Proceedings of the Fourth workshop on Resources for African Indigenous Languages (RAIL 2023), pages 32–41, Dubrovnik, Croatia. Association for Computational Linguistics. find
  83. Johannes Sibeko and Mmasibidi Setaka. 2023. Evaluating the sesotho rule-based syllabification system on sepedi and setswana words. In Proceedings of the Fourth workshop on Resources for African Indigenous Languages (RAIL 2023), pages 76–85, Dubrovnik, Croatia. Association for Computational Linguistics. ↗
  84. Ray Smith. 2007. An overview of the tesseract ocr engine. In ICDAR ’07: Proceedings of the Ninth International Conference on Document Analysis and Recognition, pages 629–633, Washington, DC, USA. IEEE Computer Society. find
  85. ByungHoon So, Kyuhong Byun, Kyungwon Kang, and Seongjin Cho. 2022. Jaquad: Japanese question answering dataset for machine reading comprehension. arXiv preprint arXiv:2202.01764. ↗
  86. Kenneth Steimel, Sandra Kübler, and Daniel Dakota. 2023. Towards a swahili universal dependency treebank: Leveraging the annotations of the helsinki corpus of swahili. In Proceedings of the Fourth workshop on Resources for African Indigenous Languages (RAIL 2023), pages 86– 96. find
  87. Ephrem Tadesse, Rosa Tsegaye, and Kuulaa Qaqqabaa. 2019. Event extraction from unstructured amharic text.
  88. Andrew Tadross and �Ābrehām Taxlu. 2017. The Essential Guide to Tigrinya, the Language of Eritrea and Tigray Ethiopia. find
  89. Yemane Tedla and Kazuhide Yamamoto. 2016. The effect of shallow segmentation on english-tigrinya statistical machine translation. In 2016 International Conference on Asian Language Processing (IALP), pages 79–82. IEEE. find
  90. Abrhalei Tela, Abraham Woubie, and Ville Hautamaki. 2020. Transferring monolingual model to low-resource language: The case of tigrinya.
  91. Atnafu Lambebo Tonja, Tadesse Destaw Belay, Israel Abebe Azime, Abinew Ali Ayele, Moges Ahmed Mehamed, Olga Kolesnikova, and Seid Muhie Yimam. 2023. Natural language processing in Ethiopian languages: Current state, challenges, and opportunities. In Proceedings of the Fourth workshop on Resources for African Indigenous Languages (RAIL 2023), pages 126–139, Dubrovnik, Croatia. Association for Computational Linguistics. ↗
  92. Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, and Kaheer Suleman. 2016a. Newsqa: A machine comprehension dataset. arXiv preprint arXiv:1611.09830. ↗
  93. Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, and Kaheer Suleman. 2017. NewsQA: A machine comprehension dataset. In Proceedings of the 2nd Workshop on Representation Learning for NLP, pages 191–200, Vancouver, Canada. Association for Computational Linguistics. ↗
  94. Adam Trischler, Zheng Ye, Xingdi Yuan, Jing He, and Philip Bachman. 2016b. A parallel-hierarchical model for machine comprehension on sparse data. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 432–441, Berlin, Germany. Association for Computational Linguistics. ↗
  95. Svitlana Vakulenko, Shayne Longpre, Zhucheng Tu, and Raviteja Anantha. 2021. Question rewriting for conversational question answering. In Proceedings of the 14th ACM international conference on web search and data mining, pages 355–363. find
  96. Barack W Wanjawa, Lilian DA Wanzare, Florence Indede, Owen McOnyango, Lawrence Muchemi, and Edward Ombui. 2023. Kenswquad—a question answering dataset for swahili low-resource language. ACM Transactions on Asian and Low-Resource Language Information Processing, 22(4):1–20. ↗
  97. Dirk Weissenborn, Georg Wiese, and Laura Seiffe. 2017. Making neural QA as simple as possible but not simpler. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 271–280, Vancouver, Canada. Association for Computational Linguistics. ↗
  98. Yi Yang, Wen-tau Yih, and Christopher Meek. 2015. Wikiqa: A challenge dataset for open-domain question answering. In Proceedings of the 2015 conference on empirical methods in natural language processing, pages 2013– 2018. ↗
  99. Hailemariam Mehari Yohannes and Toshiyuki Amagasa. 2022. A method of named entity recognition for tigrinya. ACM SIGAPP Applied Computing Review, 22(3):56–68. ↗
  100. Xinyan Yu, Trina Chatterjee, Akari Asai, Junjie Hu, and Eunsol Choi. 2022. Beyond counting datasets: A survey of multilingual dataset construction and necessary resources. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 3725–3743, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. find
  101. Munazza Zaib, Wei Emma Zhang, Quan Z Sheng, Adnan Mahmood, and Yang Zhang. 2022. Conversational question answering: A survey. Knowledge and Information Systems, 64(12):3151–3195. ↗
  102. Xinyu Zhang, Nandan Thakur, Odunayo Ogundepo, Ehsan Kamalloo, David Alfonso-Hermelo, Xiaoguang Li, Qun Liu, Mehdi Rezagholizadeh, and Jimmy Lin. 2023. Miracl: A multilingual retrieval dataset covering 18 diverse languages. Transactions of the Association for Computational Linguistics, 11:1114–1131. ↗
  103. Brian Zylich, Adam Viola, Brokk Toggerson, Lara Al-Hariri, and Andrew Lan. 2020. Exploring automated question answering methods for teaching assistance. In Artificial Intelligence in Education: 21st International Conference, AIED 2020, Ifrane, Morocco, July 6–10, 2020, Proceedings, Part I 21, pages 610–622. Springer. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/teklehaymanot-2024-tigqa-expert-annotated-question-answering/