Elamur Research⌕
☰
home / papers / 2026 / Pioneering Bot Detection on Polish Reddit at the Comment Level

Pioneering Bot Detection on Polish Reddit at the Comment Level

Paper · 2026 · Karmela Matyjaszek

Summary

Most bot-detection research targets English-language Twitter and relies on account-level or network signals. This study instead addresses comment-level bot detection in Polish-oriented Reddit communities, where text is heavily mixed with English. The author drew on a pool of roughly 10 million comments from Polish and partly Polish subreddits, collected up to the end of November 2025. From this pool, a manually labelled dataset of 40,791 comments by 1,418 users was built using a two-stage annotation protocol. Language mixing is not filtered out. Instead, it becomes part of the feature set, alongside Reddit-specific signals (such as conformity to the 'Bottiquette' conventions), culture-specific markers and stylometric repetition measures. Classifiers from several model families, including XGBoost, perform well and hold up on 2025 test data. SHAP analysis shows which of the new features matter most. The work gives a first baseline for a neglected setting. It is also practical for NLP corpus cleaning, because it needs only single-comment features and no full account crawling.

Key points

  • Dataset: 40,791 annotated comments from 1,418 users, with about 58% of comments labelled as bot-generated. Around 42% of the words are English.
  • Language-aware features: Polish and English word counts, Polish density (the language ratio) and diacritics usage. These treat a comment's language composition as a behavioural signal.
  • Most decisive features (by SHAP): cross-level interaction features, Bottiquette compliance, formatting markers, language indicators, and repetition/randomness measures, especially the entropy of non-alphabetic characters. Temporal features turned out to be weak.
  • Qualitative findings: the paper profiles the Polish Reddit bot ecosystem, covering self-identifying bot usernames, developer "cyborg" accounts, "bot wars" between anti-bot accounts, quote-repeating bots, and humans playfully imitating bots.
  • Stated limitations: no ablation of feature groups, unweighted model voting in the consensus analysis, concept drift that is not fully explained, and dictionary-based features that cope poorly with Polish morphology.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of GdańskGdansk · Poland2 works · 4 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksSocial Sciences · 1,772 worksInformation Systems · 1,437 worksSociology and Political Science · 577 works

Cite

Cite as Karmela Matyjaszek. 2026. Pioneering Bot Detection on Polish Reddit at the Comment Level. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 4: Student Research Workshop).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-srw.65

@inproceedings{matyjaszek2026pioneering,
  title = {{Pioneering Bot Detection on Polish Reddit at the Comment Level}},
  author = {Karmela Matyjaszek},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 4: Student Research Workshop)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-srw.65},
  url = {https://doi.org/10.18653/v1/2026.eacl-srw.65}
}

References

22 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Sanaz Adel Alipour, Rita Orji, and A. Zincir-Heywood. 2023. Behaviour and bot analysis on online social networks: Twitter, parler, and reddit. International Journal of Technology and Human Interaction, 19:1– 19. find
  2. Andrea Bacciu, Massimo La Morgia, Alessandro Mei, Eugenio Nerio Nemmi, Valerio Neri, and Julinda Stefa. 2019. Bot and gender detection of twitter accounts using distortion and lsa. In Conference and Labs of the Evaluation Forum. find
  3. David Beskow and Kathleen Carley. 2018a. Bot conversations are different: Leveraging network metrics for bot detection in twitter. In ASONAM, pages 825–832. find
  4. David M. Beskow and Kathleen M. Carley. 2018b. Its all in a name: detecting and labeling bots by their name. Computational and Mathematical Organization Theory, 25:24 – 35. find
  5. Nikan Chavoshi, Hossein Hamooni, and Abdullah Mueen. 2016. Debot: Twitter bot detection via warped correlation. In 2016 IEEE 16th International Conference on Data Mining (ICDM), pages 817–822. find
  6. Zi Chu, Steven Gianvecchio, Haining Wang, and Sushil Jajodia. 2010. Who is tweeting on twitter: human, bot, or cyborg? In Asia-Pacific Computer Systems Architecture Conference. find
  7. Gordon V. Cormack, Charles L A Clarke, and Stefan Buettcher. 2009. Reciprocal rank fusion outperforms condorcet and individual rank learning methods. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’09, page 758–759, New York,…
  8. Alceu Ferraz Costa, Yuto Yamaguchi, Agma J. M. Traina, Caetano Traina, and Christos Faloutsos. 2015. Rsc: Mining and modeling temporal activity in social media. Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. find
  9. Stefano Cresci. 2020. A decade of social bot detection. Communications of the ACM, 63:72 – 83. find
  10. Robert Gorwa and Douglas Guilbeault. 2018. Unpacking the social media bot: A typology to guide research and policy. Policy & Internet. find
  11. Christian Grimme, Mike Preuss, Lena Adam, and Heike Trautmann. 2017. Social bots: Human-like by means of human control? Big Data, 5:279 – 293. find
  12. Sofia Hurtado, Poushali Ray, and Radu Marculescu. 2019. Bot detection in reddit political discussion. Proceedings of the Fourth International Workshop on Social Sensing. find
  13. Anna Kapuścińska. 2020. O emotikonach raz jeszcze – na przykładzie emotikonu “xd” w j˛ezyku polskim. Prace J˛ezykoznawcze, 22(2):57–66. find
  14. Jürgen Knauth. 2019. Language-agnostic Twitter-bot detection. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019), pages 550–558, Varna, Bulgaria. INCOMA Ltd. find
  15. Sneha Kudugunta and Emilio Ferrara. 2018. Deep neural networks for bot detection. Information Sciences, 467:312–322. find
  16. Sarita Yardi, Daniel Romero, Grant Schoenebeck, and danah boyd. 2010. Detecting spam in a twitter network. First Monday, 15. find
  17. Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
  18. A Michele Mazza, Stefano Cresci, Marco Avvenuti, Walter Quattrociocchi, and Maurizio Tesconi. 2019. Rt-bust: Exploiting temporal patterns for botnet detection on twitter. Proceedings of the 10th ACM Conference on Web Science. find
  19. Lynnette Hui Xian Ng and Kathleen M. Carley. 2022. Botbuster: Multi-platform bot detection using a mixture of experts. ArXiv, abs/2207.13658. — [arXiv:2207.13658](https://arxiv.org/abs/2207.13658) find
  20. Lynnette Hui Xian Ng and Kathleen M. Carley. 2024. Assembling a multi-platform ensemble social bot detector with applications to us 2020 elections. Social Network Analysis and Mining, 14:1–16. find
  21. Juan Pizarro. 2019. Using n-grams to detect bots on twitter: Notebook for pan at clef 2019. In Working Notes Papers of the CLEF 2019 Evaluation Labs, volume 2380 of CEUR Workshop Proceedings. CEUR-WS.org. find
  22. Adrian Rauchfleisch and Jonas Kaiser. 2020. The false positive problem of automatic bot detection in social science research. PLoS ONE, 15. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/matyjaszek-2026-pioneering-bot-detection-polish-reddit/