Elamur Research⌕
☰
home / papers / 2026 / 100,000+ Movie Reviews from Kazakhstan: Russian, Kazakh, and Code-Switched Texts

100,000+ Movie Reviews from Kazakhstan: Russian, Kazakh, and Code-Switched Texts

Paper · 2026 · Rustem Yeshpanov

Summary

Public sentiment data for Kazakh is scarce, and no benchmark has captured the mixed Russian/Kazakh review culture of Kazakhstan. To fill this gap, the author scraped the ticketing and entertainment portal kino.kz and built a corpus of 100,502 movie reviews covering 2001–2025 and 4,943 titles. Each review was hand-labelled by a single annotator for language (Russian, Kazakh, or code-switched) and for three-way polarity. A GPT model was used as a reliability check on those labels. About 11% of reviews contain a rating stated explicitly by the user. These ratings were normalised and grouped into five classes.

Two tasks are benchmarked: polarity classification and score classification. The models include classical BoW/TF–IDF baselines and the multilingual transformers mBERT, XLM-RoBERTa and RemBERT. Transformers win on polarity, though only by a small margin, and the rare neutral class remains the main weakness. Score prediction is much harder once rating mentions are masked to prevent leakage. The released data, guidelines and models give a reference point for multilingual and code-switched sentiment work in Kazakhstan.

Key points

  • The corpus has 96,749 Russian, 2,639 Kazakh and 1,114 code-switched reviews from 31,453 reviewer identifiers, which were pseudonymised for release. Polarity is heavily skewed: 68,858 positive, 27,315 negative and 4,329 neutral.
  • Annotation took one person 110 days. As a cross-check, gpt-4.1-nano-2025-04-14 matched the human labels 89.54% of the time (Cohen's κ = 0.78), but the human labels are the official ones.
  • On polarity classification, RemBERT performed best, with macro-F1 = 0.82 and κ = 0.88 on the test set.
  • On leakage-controlled score classification (explicit scores replaced by a mask token), all models fell to macro-F1 0.51–0.55. Accuracy is 69%, but the F1 for score 2 is only 0.09.
  • The dataset, annotation guidelines and trained models are available on the Hugging Face Hub.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

Cite

Cite as Rustem Yeshpanov. 2026. 100,000+ Movie Reviews from Kazakhstan: Russian, Kazakh, and Code-Switched Texts. Proceedings of the 6th International Conference on Natural Language Processing for the Digital Humanities.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.nlp4dh-1.4

@inproceedings{yeshpanov2026100,
  title = {{100,000+ Movie Reviews from Kazakhstan: Russian, Kazakh, and Code-Switched Texts}},
  author = {Rustem Yeshpanov},
  booktitle = {Proceedings of the 6th International Conference on Natural Language Processing for the Digital Humanities},
  year = {2026},
  doi = {10.18653/v1/2026.nlp4dh-1.4},
  url = {https://doi.org/10.18653/v1/2026.nlp4dh-1.4}
}

References

19 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. PD Blinov, Maria Klekovkina, Eugeny Kotelnikov, and Oleg Pestov. 2013. Research of lexical approach and machine learning methods for sentiment analysis. Computational Linguistics and Intellectual Technologies, 2(12):48–58. find
  2. Hyung Won Chung, Thibault Fevry, Henry Tsai, Melvin Johnson, and Sebastian Ruder. 2021. Rethinking Embedding Coupling in Pre-trained Language Models. In International Conference on Learning Representations.
  3. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised Cross-lingual Representation Learning at Scale. In Proceedings of the Association for Computational Linguistics…
  4. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of 7 https://huggingface.co/datasets/ yeshpanovrustem/100k_movie_reviews_from_kz Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter… ↗
  5. Gerard Salton and Chris Buckley. 1988. Term-Weighting Approaches in Automatic Text Retrieval. Inf. Process. Manag., 24:513–523. find
  6. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013. Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP 2013)…
  7. Thorsten Joachims. 1999. Text Categorization with Support Vector Machines: Learning with Many Relevant Features. In European Conference on Machine Learning.
  8. Daniel Jurafsky and James H. Martin. 2009. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition (2nd Edition). Prentice-Hall, Inc., Upper Saddle River, NJ, USA. find
  9. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, and 3 others. 2019. Transf…
  10. J. Richard Landis and Gary G. Koch. 1977. The Measurement of Observer Agreement for Categorical Data. Biometrics, 33(1):159–174. find
  11. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv preprint arXiv:1907.11692.
  12. Yiming Yang. 2001. A Study on Thresholding Strategies for Text Categorization. In Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, pages 137–145. find
  13. Andrew Maas. 2011. Large Movie Review Dataset. https://ai.stanford.edu/~amaas/data/ sentiment/. Accessed 2026-02-26. ↗
  14. Rustem Yeshpanov and Huseyin Atakan Varol. 2024. KazSAnDRA: Kazakh Sentiment Analysis Dataset of Reviews and Attitudes. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 9657–9667. ELRA Language Resourc… find
  15. Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. Learning Word Vectors for Sentiment Analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 142–150. Association for…
  16. Bo Pang and Lillian Lee. 2008. Opinion Mining and Sentiment Analysis. Found. Trends Inf. Retr., 2(1– 2):1–135. find
  17. Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan. 2002. Thumbs up? Sentiment Classification using Machine Learning Techniques. In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002), pages 79– 86. Association for Computational Linguistics.
  18. Parth Patwa, Gustavo Aguilar, Sudipta Kar, Suraj Pandey, Srinivas PYKL, Björn Gambäck, Tanmoy Chakraborty, Thamar Solorio, and Amitava Das. 2020. SemEval-2020 Task 9: Overview of Sentiment Analysis of Code-Mixed Tweets. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 774–790.…
  19. Anna Rogers, Alexey Romanov, Anna Rumshisky, Svitlana Volkova, Mikhail Gronas, and Alex Gribov. 2018. RuSentiment: An Enriched Sentiment Analysis Dataset for Social Media in Russian. In Proceedings of the 27th International Conference on Computational Linguistics, pages 755–763. Association for Comp… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yeshpanov-2026-100-000-movie-reviews-kazakhstan/