Elamur Research⌕
☰
home / papers / 2026 / Benchmarking Models for Low-Resource Nepali Event Extraction with Trigger Phrase Identification and Event Classification

Benchmarking Models for Low-Resource Nepali Event Extraction with Trigger Phrase Identification and Event Classification

Paper · 2026 · Sujal Maharjan, Astha Shrestha, Lakshmojee Koduru et al.

Summary

Event extraction research has focused on high-resource languages, so Nepali has had no dedicated annotated resource for it. Nepali is a morphologically rich SOV language, and its event triggers are often multi-word verb clusters or nominalized verbs. To address this, the authors built NepEE, a corpus of 10,226 Devanagari sentences taken from the openly licensed Sanjaal Corps collection. Each sentence is annotated for two subtasks: marking the trigger span and assigning one of eight event types, such as Political, Economic Event, and Disaster and Accidents. Five native-speaker annotators followed an iterative three-phase protocol and reached near-perfect agreement.

The authors then benchmarked three groups of systems:

  • classical feature-based learners,
  • fine-tuned Transformer encoders,
  • instruction-tuned LLMs prompted zero-shot and with fixed few-shot examples.

Fine-tuned encoders performed best on both subtasks. Classical models and prompted LLMs struggled to recover exact trigger boundaries, although a tuned SVM still beat several zero-shot LLMs on event classification. The released data and code give Nepali event understanding its first solid baselines. They also show that span-level extraction is considerably harder than sentence-level labeling.

Key points

  • NepEE contains 10,226 sentences across 8 event categories. Annotators agreed at Fleiss' κ = 0.812 when identifying triggers and at κ = 0.855 when classifying events. Dataset and code are available at GitHub/SUJAL390/EEUCA-ACL-2026.
  • The evaluated encoders include mBERT, XLM-RoBERTa, IndicBERTv2 and nepaliBERT. The evaluated LLMs are Qwen-2.5, Gemma-3, Phi-4 and Llama-3.1.
  • Trigger identification: fine-tuned mBERT scored best (macro F1 0.7132). Among classical models, a token-wise Passive-Aggressive classifier (F1 0.3506) outperformed CRF (F1 0.2719).
  • Event type classification: IndicBERTv2 led with macro F1 0.8536, and the SVM baseline reached 0.7856.
  • Limitations: the data comes from a single source, there are no argument-role annotations, and LLMs are scored with strict exact-match metrics. Planned extensions include argument roles and event coreference.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢GoogleMountain View · United States430 works · 424,363 citations🎓Macquarie UniversitySydney · Australia42 works · 5,776 citations🎓Virginia TechBlacksburg · United States36 works · 43,036 citations🎓Tribhuvan UniversityKathmandu · Nepal5 works · 15 citations🎓IIMS CollegeKathmandu · Nepal5 works · 5 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Information extraction · 167 worksLow-resource languages · 267 works

Cite

Cite as Sujal Maharjan, Astha Shrestha, Lakshmojee Koduru, Sweta Poudel, Shuvam Shiwakoti, Rabin Thapa, Kritesh Rauniyar and Surendrabikram Thapa. 2026. Benchmarking Models for Low-Resource Nepali Event Extraction with Trigger Phrase Identification and Event Classification. Proceedings of the 9th Workshop on Event Extraction and Understanding: Challenges and Applications (EEUCA 2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eeuca-1.7

@inproceedings{maharjan2026benchmarking,
  title = {{Benchmarking Models for Low-Resource Nepali Event Extraction with Trigger Phrase Identification and Event Classification}},
  author = {Sujal Maharjan, Astha Shrestha, Lakshmojee Koduru, Sweta Poudel, Shuvam Shiwakoti, Rabin Thapa, Kritesh Rauniyar and Surendrabikram Thapa},
  booktitle = {Proceedings of the 9th Workshop on Event Extraction and Understanding: Challenges and Applications (EEUCA 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.eeuca-1.7},
  url = {https://doi.org/10.18653/v1/2026.eeuca-1.7}
}

References

40 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J Hewett, Mojan Javaheripi, Piero Kauffmann, and 1 others. 2024. Phi-4 technical report. arXiv preprint arXiv:2412.08905.
  2. George R Doddington, Alexis Mitchell, Mark A Przybocki, Lance A Ramshaw, Stephanie M Strassel, Ralph M Weischedel, and 1 others. 2004. The automatic content extraction (ace) program-tasks, data, and evaluation. In Lrec, volume 2, pages 837–840. Lisbon.
  3. Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 2623–2631.
  4. Seth Ebner, Patrick Xia, Ryan Culkin, Kyle Rawlins, and Benjamin Van Durme. 2020. Multi-sentence argument linking. In Proceedings of the 58th annual meeting of the association for computational linguistics, pages 8057–8077.
  5. Rosa Falotico and Piero Quatto. 2015. Fleiss’ kappa statistic without paradoxes. Quality & Quantity, 49(2):463–470. find
  6. Bal Krishna Bal. 2004. Structure of nepali grammar. PAN Localization, Madan Puraskar Pustakalaya, Kathmandu, Nepal, pages 332–396. find
  7. Joseph L Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological bulletin, 76(5):378. find
  8. Yoshua Bengio. 2012. Deep learning of representations for unsupervised and transfer learning. In Proceedings of ICML workshop on unsupervised and transfer learning, pages 17–36. JMLR Workshop and Conference Proceedings. find
  9. Rajan Ghimire. 2022. NepaliBERT. huggingface.co/Rajan/NepaliBERT. 2023-02-25. find
  10. Thorsten Brants. 2000. Tnt–a statistical part-of-speech tagger. In Sixth Applied Natural Language Processing Conference, pages 224–231. https://Accessed: Xavier Glorot, Antoine Bordes, and Yoshua Bengio. 2011. Domain adaptation for large-scale sentiment classification: A deep learning approach. In…
  11. Ruirui Chen, Chengwei Qin, Weifeng Jiang, and Dongkyu Choi. 2024. Is a large language model a good annotator for event extraction? In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages 17772–17780. find
  12. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.
  13. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th annual meeting of the association fo… find
  14. Ralph Grishman. 2019. Twenty-five years of information extraction. Natural Language Engineering, 25(6):677–692. find
  15. Maarten Grootendorst. 2022. Bertopic: Neural topic modeling with a class-based tf-idf procedure. arXiv preprint arXiv:2203.05794.
  16. Corinna Cortes and Vladimir Vapnik. 1995. Support-vector networks. Machine learning, 20(3):273–297.
  17. Koby Crammer, Ofer Dekel, Joseph Keshet, Shai Shalev-Shwartz, and Yoram Singer. 2006. Online passive-aggressive algorithms. Journal of Machine Learning Research, 7(Mar):551–585. find
  18. Ali Hürriyetoğlu, Surendrabikram Thapa, Hristo Tanev, and Surabhi Adhikari. 2025. Findings and insights from the 8th workshop on challenges and applications of automated extraction of socio-political events from text. In Proceedings of the 8th Workshop on Challenges and Applications of Automated Ext… find
  19. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies,…
  20. Divyanshu Kakwani, Anoop Kunchukuttan, Satish Golla, Gokul NC, Avik Bhattacharyya, Mitesh M Khapra, and Pratyush Kumar. 2020. Indicnlpsuite: Monolingual corpora, evaluation benchmarks and pre-trained multilingual language models for indian languages. In Findings of the association for computational…
  21. Yang, Jiaxi Yang, Jingren Zhou, and 25 others. 2025. Qwen2.5 technical report. Preprint, arXiv:2412.15115. find
  22. Lance Ramshaw and Mitch Marcus. 1995. Text chunking using transformation-based learning. In Third workshop on very large corpora.
  23. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles.
  24. Kritesh Rauniyar, Sweta Poudel, Shuvam Shiwakoti, Surendrabikram Thapa, Junaid Rashid, Jungeun Kim, Muhammad Imran, and Usman Naseem. 2023. Multi-aspect annotation and analysis of nepali tweets on anti-establishment election discourse. IEEE Access, 11:143092–143115. find
  25. John Lafferty, Andrew McCallum, and Fernando CN Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. find
  26. Sovan Kumar Sahoo, Saumajit Saha, Asif Ekbal, and Pushpak Bhattacharyya. 2020. A platform for event extraction in hindi. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 2241–2250. find
  27. J Richard Landis and Gary G Koch. 1977. The measurement of observer agreement for categorical data. biometrics, pages 159–174. find
  28. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penche… find
  29. Haonan Li, Fajri Koto, Minghao Wu, Alham Fikri Aji, and Timothy Baldwin. 2023. Bactrian-x: Multilingual replicable instruction-following models with low-rank adaptation. arXiv preprint arXiv:2305.15011.
  30. Qian Li, Jianxin Li, Jiawei Sheng, Shiyao Cui, Jia Wu, Yiming Hei, Hao Peng, Shu Guo, Lihong Wang, Amin Beheshti, and 1 others. 2022. A survey on deep learning event extraction: Approaches and applications. IEEE Transactions on Neural Networks and Learning Systems, 35(5):6301–6321. find
  31. Surendrabikram Thapa, Kritesh Rauniyar, Shuvam Shiwakoti, Sweta Poudel, Usman Naseem, and Mehwish Nasim. 2023. Nehate: Large-scale annotated data shedding light on hate speech in nepali local election discourse. Frontiers in Artificial Intelligence and Applications, 372:2346–2353. find
  32. Nobal Niraula and Jeevan Chapagain. 2022. Named entity recognition for nepali: data sets and algorithms. In The International FLAIRS Conference Proceedings, volume 35. find
  33. Sulav Timilsina, Milan Gautam, and Binod Bhattarai. 2022. Nepberta: Nepali language model trained in a large corpus. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Proce… find
  34. Jinu Nyachhyon, Mridul Sharma, Prajwal Thapa, and Bal Krishna Bal. 2025. Consolidating and developing benchmarking datasets for the nepali natural language understanding tasks. In Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia… find
  35. Christopher Walker, Stephanie Strassel, Julie Medero, and Kazuaki Maeda. 2006. find
  36. Ace 2005 multilingual training corpus. (No Title). find
  37. Xiaozhi Wang, Ziqi Wang, Xu Han, Wangyi Jiang, Rong Han, Zhiyuan Liu, Juanzi Li, Peng Li, Yankai Lin, and Jie Zhou. 2020. Maven: A massive general domain event detection dataset. In Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), pages 1652–1671.
  38. Kushal Paudyal. 2017. NepaliDataSets: Publicly released Nepali datasets of Sanjaal Corps. https://github.com/sanjaalcorps/NepaliDataSets. Accessed: 2025-04-17. ↗
  39. Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych, and Sebastian Ruder. 2020. Mad-x: An adapter-based framework for multi-task cross-lingual transfer. In Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), pages 7654–7673.
  40. Wei Xiang and Bang Wang. 2019. A survey of event extraction from text. IEEE Access, 7:173111–173137. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/maharjan-2026-benchmarking-models-low-resource-nepali/