Elamur Research⌕
☰
home / papers / 2026 / FadeIT at EVALITA 2026: Overview of the Fallacy Detection in Italian Social Media Texts Task

FadeIT at EVALITA 2026: Overview of the Fallacy Detection in Italian Social Media Texts Task

Paper · 2026 · Alan Ramponi and Sara Tonelli

Summary

FadeIT is the first shared task on detecting fallacies in Italian social media posts, a language with little prior work on the problem. It was organized at EVALITA 2026 and uses Faina, a dataset of tweets from 2019–2022 about migration, climate change and public health. Two expert annotators labeled Faina at the span level across 20 fallacy types. Overlapping spans are allowed, and disagreements the annotators judged genuine were kept rather than resolved into one label.

The task has two subtasks:

  • Subtask A: multi-label classification of whole posts.
  • Subtask B: locating and labeling fallacious text segments, which is harder.

Systems were scored against both annotators' labels as two equally valid gold standards, so the evaluation reflects human label variation. Seven teams submitted 25 runs, all based on transformers: encoders such as AlBERTo and UmBERTo, and LLMs such as Gemma 3, LLaMa 3.1 and GPT-5. Some teams added data augmentation, which gave mixed results. The organizers conclude that performance still leaves much room for improvement, especially at the span level. They offer the dataset and protocol as a benchmark for studying subjective, disagreement-aware argument analysis.

Key points

  • Faina dataset:
    • 1,440 posts, balanced by topic (480 each) and year (360 each).
    • The top 10 most-engaged posts were kept per month–topic pair, with at most one post per user.
    • Contains 11,064 annotated spans over 58,490 tokens, produced through five annotation-and-discussion rounds.
  • Splits and formats:
    • 1,152 training/development posts and 288 test posts (80/20).
    • Subtask A uses a TSV file; subtask B uses a CoNLL-style file with BIO tags. Both include parallel labels from each annotator.
  • Participation: 25 runs from 7 teams based in five countries.
  • Use of metadata and disagreement: only the Label team exploited annotator disagreement, predicting labels specific to each annotator. It also found topic metadata helpful.
  • Other approaches:
    • TiGRO used multi-task learning via MaChAmp and reported gains over a single-task setup.
    • MALTO's paraphrases generated with ChatGPT reduced performance, likely because they introduced label noise.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 works

🧭 Topics 1Research subjects and languages the work is about.Evaluation & benchmarks · 1,341 works

Cite

Cite as Alan Ramponi and Sara Tonelli. 2026. FadeIT at EVALITA 2026: Overview of the Fallacy Detection in Italian Social Media Texts Task. Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{ramponi2026fadeit,
  title = {{FadeIT at EVALITA 2026: Overview of the Fallacy Detection in Italian Social Media Texts Task}},
  author = {Alan Ramponi and Sara Tonelli},
  booktitle = {Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026)},
  year = {2026},
  url = {https://aclanthology.org/2026.evalita-1.62/}
}

References

32 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. C. L. Hamblin, Fallacies, Advanced Reasoning Forum, Socorro, USA, 2022. find
  2. C. W. Tindale, Fallacies and Argument Appraisal, Critical Reasoning and Argumentation, Cambridge University Press, Cambridge, UK, 2007. doi: https://doi.org/10.1017/ CBO9780511806544 . find
  3. E. Musi, M. Aloumpi, E. Carmi, S. Yates, K. O’Halloran, Developing fake news immunity: Fallacies as misinformation triggers during the pandemic, Online Journal of Communication and Media Technologies 12 (2022) e202217. doi: https://doi.org/10.30935/ojcmt/12083 . — [doi:10.30935/ojcmt/12083](https://doi.org/10.30935/ojcmt/12083) ↗
  4. U. Ecker, J. Roozenbeek, S. van der Linden, L. Q. Tay, J. Cook, N. Oreskes, S. Lewandowsky, Misinformation poses a bigger threat to democracy than you might think, Nature 630 (2024) 29–32. doi: https://doi.org/10.1038/d41586-024-01587-3 . ↗
  5. T. Alhindi, T. Chakrabarty, E. Musi, S. Muresan, Multitask instruction-based prompting for fallacy recognition, in: Y. Goldberg, Z. Kozareva, Y. Zhang (Eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Abu Dhabi,… find
  6. A. Ramponi, A. Daffara, S. Tonelli, Fine-grained fallacy detection with human label variation, in: L. Chiruzzo, A. Ritter, L. Wang (Eds.), Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1… find
  7. F. Cutugno, A. Miaschi, A. P. Aprosio, G. Rambelli, L. Siciliani, M. A. Stranisci, EVALITA 2026: Overview of the 9th evaluation campaign of natural language processing and speech tools for Italian, in: Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for I… find
  8. B. Plank, The “problem” of human label variation: On ground truth in data, modeling and evaluation, in: Y. Goldberg, Z. Kozareva, Y. Zhang (Eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Abu Dhabi, United Arab…
  9. P. Nakov, A. Barrón-Cedeño, G. Da San Martino, F. Alam, R. Míguez, T. Caselli, M. Kutlu, W. Zaghouani, C. Li, S. Shaar, H. Mubarak, A. Nikolov, Y. S. Kartal, Overview of the CLEF-2022 CheckThat! lab task 1 on identifying relevant claims in tweets, in: Proceedings of the Working Notes of CLEF 2022 -… find
  10. G. Da San Martino, S. Yu, A. Barrón-Cedeño, R. Petrov, P. Nakov, Fine-grained analysis of propaganda in news articles, in: K. Inui, J. Jiang, V. Ng, X. Wan (Eds.), Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Nat…
  11. M. Polignano, P. Basile, M. de Gemmis, G. Semeraro, V. Basile, AlBERTo: Italian BERT language understanding model for NLP challenging tasks based on tweets, in: R. Bernardi, R. Navigli, G. Semeraro (Eds.), Proceedings of the Sixth Italian Conference on Computational Linguistics (CLiC-it 2019), CEUR…
  12. L. Parisi, S. Francia, P. Magnani, UmBERTo: An Italian language model trained with whole word masking, https://github.com/musixmatchresearch/umberto, 2020. Accessed: 2026-01-01. ↗
  13. C. J. Scozzaro, M. Rinaldi, G. Mittone, M. A. Stranisci, Kenji-Endo: a BabyLM @EVALITA, in: Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026), CEUR.org, Bari, Italy, 2026.
  14. T. Labruna, E. Papadopulos, Label at FadeIT: Fallacy-aware LLM reasoning for score-based classification, in: Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026), CEUR.org, Bari, Italy, 2026. find
  15. M. Salami, L. M. Rodia, V. Schiau, E. A. Munis, C. Savelli, F. Giobergia, MALTO at FadeIT: A BERT-based system for multi-label fallacy detection in Italian social media, in: Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVAL…
  16. D. D. Phu, S. B. Hong, T. V. Dang, PuDy at FadeIT: Enhancing fine-grained fallacy detection with hierarchy-aware training and contextual enrichment, in: Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026), CEUR.org,…
  17. Meenakshi, J. R, R. U, B. G. HB, M. Ptaszynski, RBG-AI at FadeIT: Prompted LLMs with label abstraction for logical fallacy detection, in: Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026), CEUR.org, Bari, Italy, 20…
  18. S. Atzeni, G. Sarti, T. Caselli, M. Nissim, TiGRO at FadeIT: E pluribus unum – A multi-task approach to fallacy detection and span identification, in: Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026), CEUR.org, Ba… find
  19. M. Fenu, M. Atzori, Unica at FadeIT: Adapting large language models to fallacy identification in social networks, in: Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026), CEUR.org, Bari, Italy, 2026. find
  20. M. Marone, O. Weller, W. Fleshman, E. Yang, D. Lawrie, B. V. Durme, mmBERT: A modern multilingual encoder with annealed language learning, arXiv preprint arXiv:2509.06888 (2025). URL: https://arxiv.org/abs/2509.06888. ↗
  21. Gemma Team, et al., Gemma 3 technical report, arXiv preprint arXiv:2503.19786 (2025). URL: https://arxiv.org/abs/2503.19786. ↗
  22. A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughan, A. Yang, A. Fan, A. Goyal, A. Hartshorn, A. Yang, A. Mitra, A. Sravankumar, A. Korenev, A. Hinsvark, A. Rao, et al., The Llama 3 herd of models, arXiv preprint arXiv:2407.21783 (202…
  23. A. Q. Jiang, A. Sablayrolles, A. Roux, A. Mensch, B. Savary, C. Bamford, D. S. Chaplot, D. de las Casas, E. B. Hanna, F. Bressand, G. Lengyel, G. Bour, G. Lample, L. R. Lavaud, L. Saulnier, M.-A. Lachaux, P. Stock, S. Subramanian, S. Yang, S. Antoniak, et al., Mixtral of experts, arXiv preprint arXi… find
  24. Gemini Team Google, et al., Gemini: A family of highly capable multimodal models, arXiv preprint arXiv:2312.11805 (2025). URL: https://arxiv.org/abs/2312.11805. ↗
  25. R. van der Goot, A. Üstün, A. Ramponi, I. Sharaf, B. Plank, Massive choice, ample tasks (MaChAmp): A toolkit for multi-task learning in NLP, in: D. Gkatzia, D. Seddah (Eds.), Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstra…
  26. A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv, C. Zheng, D. Liu, F. Zhou, F. Huang, F. Hu, H. Ge, H. Wei, H. Lin, J. Tang, J. Yang, et al., Qwen3 technical report, arXiv preprint arXiv:2505.09388 (2025). URL: https://arxiv.org/abs/2505.09388. ↗
  27. A. Ramponi, B. Plank, Neural unsupervised domain adaptation in NLP—A survey, in: D. Scott, N. Bel, C. Zong (Eds.), Proceedings of the 28th International Conference on Computational Linguistics, International Committee on Computational Linguistics, Barcelona, Spain (Online), 2020, pp. 6838–6855. URL:… — Neural Unsupervised Domain Adaptation in NLP–-A Survey
  28. C. Casula, S. Vecellio Salto, A. Ramponi, S. Tonelli, Delving into qualitative implications of synthetic data for hate speech detection, in: Y. Al-Onaizan, M. Bansal, Y.-N. Chen (Eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Association for Computatio… find
  29. T. Wullach, A. Adler, E. Minkov, Fight fire with fire: Fine-tuning hate detectors using large samples of generated hate speech, in: M.-F. Moens, X. Huang, L. Specia, S. W.-t. Yih (Eds.), Findings of the Association for Computational Linguistics: EMNLP 2021, Association for Computational Linguistics,… find
  30. J. Tiedemann, S. Thottingal, OPUS-MT – building open translation services for the world, in: A. Martins, H. Moniz, S. Fumega, B. Martins, F. Batista, L. Coheur, C. Parra, I. Trancoso, M. Turchi, A. Bisazza, J. Moorkens, A. Guerberof, M. Nurminen, L. Marg, M. L. Forcada (Eds.), Proceedings of the 22n… find
  31. A. Daffara, A. Ramponi, S. Tonelli, WorthIt: Check-worthiness estimation of Italian social media posts, in: C. Bosco, E. Jezek, M. Polignano, M. Sanguinetti (Eds.), Proceedings of the Eleventh Italian Conference on Computational Linguistics (CLiC-it 2025), CEUR Workshop Proceedings, Cagliari, Italy,… find
  32. P. Goffredo, M. Chaves, S. Villata, E. Cabrio, Argument-based detection and classification of fallacies in political debates, in: H. Bouamor, J. Pino, K. Bali (Eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Si… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ramponi-2026-fadeit-evalita-2026-overview-fallacy/