Elamur Research⌕
☰
home / papers / 2026 / MALTO at FadeIT: A BERT-Based System for Multi-Label Fallacy Detection in Italian Social Media

MALTO at FadeIT: A BERT-Based System for Multi-Label Fallacy Detection in Italian Social Media

Paper · 2026 · Matin Salami, Luca M. Rodia, Vladimir Schiau et al.

Summary

The paper describes the MALTO team's entry to Subtask A of the FadeIT shared task at EVALITA 2026. The task is to detect, for each Italian social media post, which of 20 fallacy types it contains. A post can carry several fallacies or none, and the labels come from two annotators who sometimes disagree. The data is the FAINA corpus, which covers four years of posts on migration, climate change and public health. Its label distribution is strongly skewed toward a few fallacy types.

The system fine-tunes AlBERTo, an Italian BERT model pretrained on tweets. Token representations are mean-pooled and passed to a light linear head with one sigmoid output per fallacy. A single global decision threshold converts these probabilities into label sets. The authors also compared:

  • UmBERTo and XLM-RoBERTa-base as alternative encoders;
  • several loss functions (BCE, weighted BCE, Focal Loss);
  • ChatGPT-generated paraphrases as extra training data.

Plain BCE on the original data gave the best result, a micro-F1 of 0.54 on the official test set. Paraphrase augmentation consistently lowered scores, likely by blurring rhetorical cues and adding label noise. The work is useful as an Italian fallacy-detection baseline and as a warning about naive LLM-based data augmentation.

Key points

  • The FAINA corpus provides 1152 training and 288 test posts, with 20 fallacy types and annotations from two independent annotators. Micro-F1 is the official ranking metric.

  • On the internal test split, the three encoders trained with BCE performed almost identically on micro-F1: AlBERTo 0.575, UmBERTo 0.574, XLM-RoBERTa-base 0.567.

  • Weighted BCE and Focal Loss improved macro-F1, which helps rare classes, but reduced micro-F1.

  • On the official test set, the best run scored 0.54. A run using augmented data with a weighted loss fell to 0.374, and a variant aware of fallacy descriptions reached 0.466.

  • Per-label F1 tracked label frequency:

    FallacyF1
    Appeal to emotion71.06
    Doubt70.32
    Red herring16.11
    False analogy15.84
    Circular reasoning, False dilemma, Strawmannever correctly detected

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Politecnico di TorinoTurin · Italy11 works · 18 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSociology and Political Science · 577 works

Cite

Cite as Matin Salami, Luca M. Rodia, Vladimir Schiau, Evren A. Munis, Claudio Savelli and Flavio Giobergia. 2026. MALTO at FadeIT: A BERT-Based System for Multi-Label Fallacy Detection in Italian Social Media. Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{salami2026malto,
  title = {{MALTO at FadeIT: A BERT-Based System for Multi-Label Fallacy Detection in Italian Social Media}},
  author = {Matin Salami, Luca M. Rodia, Vladimir Schiau, Evren A. Munis, Claudio Savelli and Flavio Giobergia},
  booktitle = {Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026)},
  year = {2026},
  url = {https://aclanthology.org/2026.evalita-1.64/}
}

References

30 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Z. Jin, A. Lalwani, T. Vaidhya, X. Shen, Y. Ding, Z. Lyu, M. Sachan, R. Mihalcea, B. Schölkopf, Logical fallacy detection, in: Findings of the Association for Computational Linguistics: EMNLP 2022, Association for Computational Linguistics, 2022, pp. 7180–7198. URL: https://aclanthology. org/2022.fi…
  2. M.-H. Yeh, R. Wan, T.-H. K. Huang, Cocolofa: A dataset of news comments with common logical fallacies written by llm-assisted crowds, in: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, 2024, pp. 660–677. URL: https:/… find
  3. E. Cantín Larumbe, A. Chust Vendrell, Argumentative fallacy detection in political debates, in: Proceedings of the 12th Workshop on Argument Mining, Association for Computational Linguistics, Vienna, Austria, 2025, pp. 369–373. URL: https://aclanthology.org/2025.argmining-1. 36/. doi: 10.18653/v1/20… ↗
  4. A. Ramponi, A. Daffara, S. Tonelli, Fine-grained fallacy detection with human label variation, in: L. Chiruzzo, A. Ritter, L. Wang (Eds.), Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1… find
  5. F. Cutugno, A. Miaschi, A. P. Aprosio, G. Rambelli, L. Siciliani, M. A. Stranisci, Evalita 2026: Overview of the 9th evaluation campaign of natural language processing and speech tools for italian, in: Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for I… find
  6. M. Polignano, P. Basile, M. de Gemmis, G. Semeraro, V. Basile, Alberto: Italian bert language understanding model for nlp challenging tasks based on tweets, in: Proceedings of the Sixth Italian Conference on Computational Linguistics (CLiC-it 2019), CEUR Workshop Proceedings, 2019, pp. 312–317. URL:…
  7. J. Lanchantin, T. Wang, V. Ordonez, Y. Qi, General multi-label image classification with transformers, 2020. URL: https://arxiv.org/abs/2011.14027. arXiv:2011.14027 . — [arXiv:2011.14027](https://arxiv.org/abs/2011.14027) ↗
  8. P. Yang, X. Sun, W. Li, S. Ma, W. Wu, H. Wang, Sgm: Sequence generation model for multi-label classification, 2018. URL: https://arxiv.org/abs/1806.04822. arXiv:1806.04822 . ↗
  9. G. Da San Martino, S. Yu, A. Barrón-Cedeño, R. Petrov, P. Nakov, Fine-grained analysis of propaganda in news articles, 2019. URL: https://arxiv.org/abs/1910.02517. arXiv:1910.02517 . ↗
  10. T.-Y. Lin, P. Goyal, R. Girshick, K. He, P. Dollár, Focal loss for dense object detection, 2018. URL: https://arxiv.org/abs/1708.02002. arXiv:1708.02002 . ↗
  11. Y. Cui, M. Jia, T.-Y. Lin, Y. Song, S. Belongie, Class-balanced loss based on effective number of samples, 2019. URL: https://arxiv.org/abs/1901.05555. arXiv:1901.05555 . ↗
  12. F. Borra, C. Savelli, G. Rosso, A. Koudounas, F. Giobergia, MALTO at SemEval-2024 task 6: Leveraging synthetic data for LLM hallucination detection, in: A. K. Ojha, A. S. Doğruöz, H. Tayyar Madabushi, G. Da San Martino, S. Rosenthal, A. Rosá (Eds.), Proceedings of the 18th International Workshop on… find
  13. H. He, Y. Bai, E. A. Garcia, S. Li, Adasyn: Adaptive synthetic sampling approach for imbalanced learning, in: 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), 2008, pp. 1322–1328. doi: 10.1109/IJCNN.2008.4633969 . — [doi:10.1109/ijcnn.2008.4633969](https://doi.org/10.1109/ijcnn.2008.4633969) find
  14. J. Li, Synthetic minority oversampling technique based on adaptive local mean vectors and improved differential evolution, IEEE Access 10 (2022) 1–1. doi: 10.1109/ACCESS.2022.3187699 . — [doi:10.1109/access.2022.3187699](https://doi.org/10.1109/access.2022.3187699) find
  15. M. Poliakov, N. Shvai, Missynth: Improving missci logical fallacies classification with synthetic data, 2025. URL: https://arxiv.org/abs/2510.26345. arXiv:2510.26345 . — [arXiv:2510.26345](https://arxiv.org/abs/2510.26345) ↗
  16. L. Mouchel, D. Paul, S. Cui, R. West, A. Bosselut, B. Faltings, A logical fallacy-informed framework for argument generation, 2025. URL: https://arxiv.org/abs/2408.03618. arXiv:2408.03618 . ↗
  17. S. Saha, P. Yadav, L. Bauer, M. Bansal, ExplaGraphs: An explanation graph generation task for structured commonsense reasoning, in: M.-F. Moens, X. Huang, L. Specia, S. W.-t. Yih (Eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Association for Computati… find
  18. A. Glazkova, O. Zakharova, Evaluating llm prompts for data augmentation in multi-label classification of ecological texts, in: 2024 Ivannikov Ispras Open Conference (ISPRAS), IEEE, 2024, p. 1–7. URL: http://dx.doi.org/10.1109/ISPRAS64596.2024.10899128. doi: 10.1109/ispras64596. 2024.10899128 . — [doi:10.1109/ispras64596.2024.10899128](https://doi.org/10.1109/ispras64596.2024.10899128) ↗
  19. F. Wang, M. Beladev, O. Kleinfeld, E. Frayerman, T. Shachar, E. Fainman, K. L. Assaraf, S. Mizrachi, B. Wang, Text2topic: Multi-label text classification system for efficient topic detection in user generated content with zero-shot capabilities, 2023. URL: https://arxiv.org/abs/2310.14817. arXiv:231… ↗
  20. Ç. Çöltekin, M. Brivio, F. Can, Tübingen at politicit: Exploring SVMs, pretrained language models, and linguistic transfer for ideology detection in social media, in: M. G. Garcia, M. Passarotti (Eds.), Proceedings of the Eighth Evaluation Campaign of Natural Language Processing and Speech Tools for… find
  21. H. Fallah, P. Bellot, E. Bruno, E. Murisasco, Adapting Transformers for Multi-Label Text Classification, in: CIRCLE (Joint Conference of the Information Retrieval Communities in Europe) 2022, Samatan, France, 2022. URL: https://hal.science/hal-03727927. ↗
  22. T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y. Jernite, J. Plu, C. Xu, T. Le Scao, S. Gugger, M. Drame, Q. Lhoest, A. Rush, Transformers: State-of-the-art natural language processing, in: Q.…
  23. J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, 2019. URL: https://arxiv.org/abs/1810.04805. arXiv:1810.04805 . ↗
  24. L. Parisi, S. Francia, P. Magnani, Umberto: an italian language model trained with whole word masking, https://github.com/musixmatchresearch/umberto, 2020. ↗
  25. A. Conneau, K. Khandelwal, N. Goyal, V. Chaudhary, G. Wenzek, F. Guzmán, E. Grave, M. Ott, L. Zettlemoyer, V. Stoyanov, Unsupervised cross-lingual representation learning at scale, 2020. URL: https://arxiv.org/abs/1911.02116. arXiv:1911.02116 . ↗
  26. A. Ramponi, S. Tonelli, FadeIT at EVALITA 2026: Overview of the fallacy detection in italian social media texts task, in: Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026), CEUR.org, Bari, Italy, 2026.
  27. J. Xing, D. Luo, C. Xue, R. Xing, Comparative analysis of pooling mechanisms in llms: A sentiment analysis perspective, 2025. URL: https://arxiv.org/abs/2411.14654. arXiv:2411.14654 . — [arXiv:2411.14654](https://arxiv.org/abs/2411.14654) ↗
  28. J. L. Ba, J. R. Kiros, G. Hinton, Layer normalization, arXiv preprint arXiv:1607.06450 (2016).
  29. I. Loshchilov, F. Hutter, Decoupled weight decay regularization, arXiv preprint arXiv:1711.05101 (2017). find
  30. X. Zhang, Y. Song, F. Zuo, X. Wang, Towards imbalanced large scale multi-label classification with partially annotated labels, 2023. URL: https://arxiv.org/abs/2308.00166. arXiv:2308.00166 . A. Model configurations and training hyperparameters The table below explains all model configurations that a… — [arXiv:2308.00166](https://arxiv.org/abs/2308.00166) ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/salami-2026-malto-fadeit-bert-based-system/