Elamur Research⌕
☰
home / papers / 2026 / Teaching LLMs to unveil tendentious implicit contents of Italian political communication

Teaching LLMs to unveil tendentious implicit contents of Italian political communication

Summary

Political speakers often slip debatable claims into speeches through presuppositions and implicatures, so they never openly commit to them. Earlier work on Italian data showed that general-purpose LLMs, used without task training, mostly fail to reconstruct such manipulative meanings. This paper tests whether supervised instruction tuning can close that gap. The authors start from IMPAQTS-PID, a benchmark built on the expert-annotated IMPAQTS corpus of Italian political speeches. They remove its multiple-choice options and turn it into an open generation task: the model receives a speech excerpt with the implicit segment marked by tags and must state the hidden content explicitly. Llama3.1-8B-Instruct and Qwen2.5-7B-Instruct are fine-tuned on 25,000 such pairs. They are compared against zero-shot GPT-4o-mini results from prior work. Three independent annotators rate the outputs on a 0–4 correctness scale, and BERTScore is also examined. The fine-tuned models clearly beat the earlier benchmark results, and both explain presuppositions much better than implicatures. The authors conclude that task-specific pragmatic supervision, not only model scale, drives progress in interpreting implicit meaning.

Key points

  • IMPAQTS corpus. It contains 1,403 monologic speeches (about 2 million words) by 150 Italian politicians from 1946 to 2023. The speeches cover six types, from parliamentary speeches to new media statements, and three historical periods. Only non-bona fide true (potentially manipulative) implicit content is annotated, each instance with a class label and an explanation.
  • IMPAQTS-PID dataset. It has 31,822 instances: 14,925 implicatures and 16,897 presuppositions. The split is 25,000 for training and 6,822 for testing.
  • Fine-tuning setup. Llama3.1-8B-Instruct and Qwen2.5-7B-Instruct were tuned with an Italian instruction prompt for 3 epochs. Settings: learning rate 2e-5, cosine scheduler, batch size 2, gradient accumulation 8, warmup 0.1.
  • Evaluation. Each model received 50 manually rated explanations from 3 annotators on a 0–4 scale: 0 = missing output, 4 = fully correct. BERTScore was evaluated as an automatic metric. The baseline is zero-shot GPT-4o-mini from Paci et al. (2025).
  • Main findings. Fine-tuning substantially improves on prior zero-shot benchmarks, and presuppositions are explained considerably better than implicatures. The work continues Paci (2025), Paci et al. (2025) and the IMPOLS shared task at EVALITA 2026.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Implicature

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/4da4mm3sngm7

@inproceedings{paci2026teaching,
  title = {{Teaching LLMs to unveil tendentious implicit contents of Italian political communication}},
  author = {Paci, Walter and Gregori, Lorenzo and Panunzi, Alessandro},
  booktitle = {Proceedings of the 15th Workshop on Cognitive Modeling and Computational Linguistics},
  year = {2026},
  doi = {10.63317/4da4mm3sngm7},
  url = {https://doi.org/10.63317/4da4mm3sngm7}
}

References

24 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. David I Beaver. 2001. Presupposition and assertion in dynamic semantics, volume 29. CSLI publications Stanford. find
  2. Ye-eun Cho and Seong mook Kim. 2024. Pragmatic inference of scalar implicature by llms. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop), pages 10–20. find our card
  3. Federica Cominetti, Lorenzo Gregori, Edoardo Lombardi Vallauri, and Alessandro Panunzi. 2024. Impaqts: a multimodal corpus of parliamentary and other political speeches in italy (1946-2023), annotated with implicit strategies. In Proceedings of the IV Workshop on Creating, Analysing, and Increasing… find
  4. Lorenzo Gregori, Walter Paci, and Saccone Valentina. 2026. IMPOLS at EVALITA 2026: Overview of the IMPOLS Task. In Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. find
  5. Final Workshop (EVALITA 2026), Bari, Italy. CEUR.org. find
  6. H. P. Grice. 1975. Logic and conversation. In Peter Cole and Jerry L. Morgan, editors, Syntax and Semantics: Vol. 3: Speech Acts, pages 41–58. Academic Press, New York. find
  7. Jennifer Hu, Sammy Floyd, Olessia Jouravlev, Evelina Fedorenko, and Edward Gibson. 2023. A fine-grained comparison of pragmatic language understanding in humans and language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pa… find our card
  8. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hallucination in natural language generation. ACM computing surveys, 55(12):1–38. find our card
  9. Stephen C Levinson. 2000. Presumptive meanings: The theory of generalized conversational implicature. MIT press. find
  10. Edoardo Lombardi Vallauri. 2017. Implicits as evolved persuaders. In Pragmemes and theories of language use, pages 725–748. Springer. find
  11. Bolei Ma, Yuting Li, Wei Zhou, Ziwei Gong, Yang Janet Liu, Katja Jasinskaja, Annemarie Friedrich, Julia Hirschberg, Frauke Kreuter, and Barbara Plank. 2025. Pragmatics in the era of large language models: A survey on datasets, evaluation, opportunities and challenges. arXiv preprint arXiv:2502.12378… find
  12. Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022. Rethinking the role of demonstrations: What makes in-context learning work? In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Association for Compu… find our card
  13. Evaluating text generation with bert. preprint arXiv:1904.09675. find
  14. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in neural information processing systems, 36:46595– 46623. 9. find
  15. Appenidix Models fine-tuning parameters The following prompt has been used to perform instruction tuning of Llama and Qwen models. Il seguente estratto di un discorso politico contiene un contenuto implicito non bona fide vero tra i tag \<s\> e \</s\>. Esplicita questo contenuto implicito. find
  16. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744. find
  17. Walter Paci. 2025. "it’s a further exercise in futility": implicit content detection and classification in italian political discourse. a pilot study. AI-Linguistica. Linguistic Studies on AI-Generated Texts and Discourses, 2(2). arXiv The following excerpt from a political speech contains a non-bon… find
  18. The following parameters have been used for Llama and Qwen models fine-tuning. num_train_epochs=3 per_device_train_batch_size=2 find
  19. Walter Paci, Alessandro Panunzi, and Sandro Pezzelle. 2025. They want to pretend not to understand: The limits of current LLMs in interpreting implicit content of political discourse. In Findings of the Association for Computational Linguistics: ACL 2025, pages 15569–15593, Vienna, Austria. Associat… find
  20. Manual evaluation results Dilyorjon Solidjonov. 2025. Pragmatic competence without embodiment? evaluating llm performance on implicature, presupposition, and speech acts. find
  21. The following tables reports the judgments of each annotator per score. Both partial values on presuppositions and implicatures, and the overall values are reported. gradient_accumulation_steps=8 learning_rate=2e-5 lr_scheduler_type="cosine" warmup_ratio=0.1 find
  22. Dan Sperber and Deirdre Wilson. 1986. Relevance: Communication and cognition, volume 142. Harvard University Press Cambridge, MA. find
  23. Robert Stalnaker. 1978. Assertion. Syntax and Semantics (New York Academic Press), 9:315– 332. find
  24. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Score Total items annot 1 annot 2 annot 3 Implicatures annot 1 annot 2 annot 3 Presuppositions annot 1 annot 2 annot 3 22 19 24 0 0 0 15 15 13 13 16 13 0 0 0 4 3 6 0 0 0 10 7 8 10 14 10 0 0 0 18 16 18 0 0 0… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/paci-2026-teaching-llms-unveil-tendentious-implicit/