Elamur Research⌕
☰
home / papers / 2026 / Beyond Logical Forms: LLM-Extracted Patterns for Fallacy Classification

Beyond Logical Forms: LLM-Extracted Patterns for Fallacy Classification

Paper · 2026 · Eleni Papadopulos, Firoj Alam and Giovanni Da San Martino

Summary

Automatic fallacy classification is hard because fallacies, especially informal ones, take many surface forms. Earlier structure-based methods, such as the Electra model of Jin et al. (2022), map each fallacy to a single abstract logical form, which misses this variety. The authors propose a weakly supervised, inference-only framework. They use the labelled training portion of the LOGIC dataset in two steps. First, llama-3.3-70B-Instruct explains why each example is fallacious. Second, o4-mini derives "structural patterns" per class from the examples and their explanations. These patterns combine formal logical skeletons with recurring lexical and rhetorical cues. The patterns, together with LLM-written class definitions, are added to classification prompts in zero- and one-shot settings, with no fine-tuning. Prompts that include the patterns beat zero-shot baselines and earlier unsupervised approaches on LOGIC. The patterns also transfer to the REDDIT and ELECDEBATE datasets without being re-extracted. The study shows that data-driven patterns can serve as reusable, generalizable logical representations for low-cost fallacy detection.

Key points

  • Source data: Patterns were extracted only from LOGIC, which has 2,449 short educational examples across 13 fallacy classes, split 1807/299/299. Several LOGIC classes merge distinct sub-fallacies; for example, Hasty Generalization also covers Slippery Slope.
  • Pipeline: llama-3.3-70B-Instruct generates the explanations, and o4-mini extracts the patterns. During extraction, function words are kept and content words are replaced with placeholders.
  • Main results: Pattern-based prompting reaches 73.5% accuracy on LOGIC, rising to 74.2% with one-shot examples. Reasoning models perform best in every configuration.
  • Cross-dataset tests: The patterns were evaluated on REDDIT (Sahai et al., 2021; subreddit comments) and ELECDEBATE 60TO16 (Goffredo et al., 2023; U.S. presidential debates from 1960 to 2016).
  • Resources: The prompts are available at github.com/elenipapadopulos/fallacy-patterns.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of PaduaPadua · Italy18 works · 4,260 citations🎓Politecnico di TorinoTurin · Italy11 works · 18 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSociology and Political Science · 577 worksPhilosophy · 284 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksSemantic parsing & meaning representation · 185 works

Cite

Cite as Eleni Papadopulos, Firoj Alam and Giovanni Da San Martino. 2026. Beyond Logical Forms: LLM-Extracted Patterns for Fallacy Classification. Proceedings of the 13th Workshop on Argument Mining and Reasoning.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.argmining-1.2

@inproceedings{papadopulos2026beyond,
  title = {{Beyond Logical Forms: LLM-Extracted Patterns for Fallacy Classification}},
  author = {Eleni Papadopulos, Firoj Alam and Giovanni {Da San Martino}},
  booktitle = {Proceedings of the 13th Workshop on Argument Mining and Reasoning},
  year = {2026},
  doi = {10.18653/v1/2026.argmining-1.2},
  url = {https://doi.org/10.18653/v1/2026.argmining-1.2}
}

References

26 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Tariq Alhindi, Smaranda Muresan, and Preslav Nakov. 2024. Large language models are few-shot training example generators: A case study in fallacy recognition. find
  2. Ivan Habernal, Henning Wachsmuth, Iryna Gurevych, and Benno Stein. 2018. Before name-calling: Dynamics and triggers of ad hominem fallacies in web argumentation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Techn…
  3. Paul Reisert, Naoya Inoue, Tatsuki Kuribayashi, and Kentaro Inui. 2018. Feasible annotation scheme for capturing policy argument reasoning using argument templates. In Proceedings of the 5th Workshop on Argument Mining, pages 79–89, Brussels, Belgium. Association for Computational Linguistics. find
  4. Irfan Robbani, Paul Reisert, Surawat Pothong, Naoya Inoue, Camélia Guerraoui, Wenzhi Wang, Shoichi Naito, Jungmin Choi, and Kentaro Inui. 2024. Flee the flaw: Annotating the underlying logic of fallacious arguments through templates and slot-filling. In Proceedings of the 2024 Conference on Empirica… find
  5. C.L. Hamblin. 1970. Fallacies. University paperbacks. Methuen. find
  6. William L. Hamilton, Rex Ying, and Jure Leskovec. 2017. Representation learning on graphs: Methods and applications. find
  7. Ruixin Hong, Hongming Zhang, Xinyu Pang, Dong Yu, and Changshui Zhang. 2024. A closer look at the self-verification abilities of large language models in logical reasoning. find
  8. Ramon Ruiz-Dolz and John Lawrence. 2023. Detecting argumentative fallacies in the wild: Problems and limitations of large language models. In Proceedings of the 10th Workshop on Argument Mining, pages 1–10, Singapore. Association for Computational Linguistics. find
  9. Jiwon Jeong, Hyeju Jang, and Hogun Park. 2025. Large language models are better logical fallacy reasoners with counterargument, explanation, and goal-aware prompt formulation. find
  10. Ramon Ruiz-Dolz and John Lawrence. 2025. An explainable framework for misinformation identification via critical question answering. Preprint, arXiv:2503.14626. — [arXiv:2503.14626](https://arxiv.org/abs/2503.14626) find
  11. Zhijing Jin, Abhinav Lalwani, Tejas Vaidhya, Xiaoyu Shen, Yiwen Ding, Zhiheng Lyu, Mrinmaya Sachan, Rada Mihalcea, and Bernhard Schölkopf. 2022. Logical fallacy detection. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 7180–7198, Abu Dhabi, United Arab Emirates. Asso…
  12. Devendra Sachan, Yuhao Zhang, Peng Qi, and William L. Hamilton. 2021. Do syntax trees help pre-trained transformers extract information? find
  13. Saumya Sahai, Oana Balalau, and Roxana Horincar. 2021. Breaking down the invisible wall of informal fallacies in online discussions. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (… find
  14. Ralph Henry Johnson and J. Anthony Blair. 1977. Logical Self-Defense. Toronto, Canada. find
  15. Yuanyuan Lei and Ruihong Huang. 2024. Boosting logical fallacy reasoning in LLMs via logical structure tree. find
  16. Yanda Li, Dixuan Wang, Jiaqing Liang, Guochao Jiang, Qianyu He, Yanghua Xiao, and Deqing Yang. 2024. Reason from fallacy: Enhancing large language models’ logical reasoning through logical fallacy understanding. find
  17. Zhivar Sourati, Filip Ilievski, Hông-Ân Sandlin, and Alain Mermoud. 2023a. Case-based reasoning with language models for classification of logical fallacies. find
  18. Gionnieve Lim and Simon T. Perrault. 2024. Evaluation of an llm in identifying logical fallacies: A call for rigor when adopting llms in hci research. find
  19. Zhivar Sourati, Vishnu Priya Prasanna Venkatesh, Darshan Deshpande, Himanshu Rawlani, Filip Ilievski, Hông-Ân Sandlin, and Alain Mermoud. 2023b. Robust and explainable identification of logical fallacies in natural language arguments. find
  20. OpenAI. 2023. GPT-4 technical report. arXiv preprint arXiv:2303.08774.
  21. OpenAI. 2025. Openai o3 and o4-mini system card. find
  22. Fengjun Pan, Xiaobao Wu, Zongrui Li, and Anh Tuan Luu. 2024. Are LLMs good zero-shot fallacy classifiers? find
  23. Thomas Storer. 1949. Carl g. hempel and paul oppenheim. studies in the logic of explanation. philosophy of science, vol. 15 (1948), pp. 135–175. Journal of Symbolic Logic, 14(2):133–133. find
  24. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics.
  25. Nicole Teo, Donghao Huang, Erik Cambria, and Zhaoxia Wang. 2025. Large language models for logical fallacy detection. In Trends and Applications in Knowledge Discovery and Data Mining, pages 387–398, Singapore. Springer Nature Singapore. find
  26. Prashanth Vijayaraghavan and Soroush Vosoughi. 2022. TWEETSPIN: Fine-grained propaganda detection in social media using multi-view representations. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/papadopulos-2026-beyond-logical-forms-llm-extracted/