Elamur Research⌕
☰
home / papers / 2020 / Multilingual Irony Detection with Dependency Syntax and Neural Models

Multilingual Irony Detection with Dependency Syntax and Neural Models

Summary

Irony is a gap between what an utterance literally says and what it means. Automatic irony detection usually relies on lexical, semantic or pragmatic cues and rarely uses syntax. This study tests whether morphological and dependency-syntactic information on its own can support irony detection across languages. The authors frame the task as binary classification of tweets in English, Spanish, French and Italian, using benchmark data from earlier shared tasks, which they enrich with Universal Dependencies (UD) annotation. They compare three setups:

  • classical machine learning classifiers using a range of dependency-based features;
  • two types of word embeddings trained on parsed data;
  • Multilingual BERT extended with syntactic features, either a selected subset or an autoencoder-compressed version.

The syntax-aware models beat syntax-agnostic baselines and compare well with the top shared-task systems, although the best configuration differs by language. The work shows that language-independent UD-based syntax carries a useful irony signal. It also lays groundwork for future models that combine syntax with pragmatics.

Key points

  • Datasets: DEFT 2017 (French), SemEval-2018 Task 3 (English), IroSvA (Spanish) and IronITA (Italian), all enriched with UD morpho-syntactic annotation.
  • Feature selection with M-BERT: Adding the best feature subset to M-BERT (M-BERT+best_feats) raised macro F1 from .682 to .694 in English and from .668 to .677 in Spanish. French and Italian did not benefit.
  • Autoencoder variant: M-BERT+autoencoder came close to, and in places slightly exceeded, state-of-the-art results for English. For French and Italian it gave the weakest scores of all settings.
  • Error analysis: Misclassified tweets contain more SYM and X tags and more parataxis, flat and expl relations than average. For example, SYM was 22% above average in English errors, 7% in Spanish and 4% in Italian.
  • Parser limitation: These categories are hard for UD parsers trained on standard text, which points to social-media parsing quality as a key limitation.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

7 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 6 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.coling-main.116 · arXiv 2011.05706

@inproceedings{cignarella2020multilingual,
  title = {{Multilingual Irony Detection with Dependency Syntax and Neural Models}},
  author = {Cignarella, Alessandra Teresa and Basile, Valerio and Sanguinetti, Manuela and Bosco, Cristina and Rosso, Paolo and Benamara, Farah},
  booktitle = {Proceedings of the 28th International Conference on Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.coling-main.116},
  eprint = {2011.05706},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2020.coling-main.116}
}

References

47 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Salvatore Attardo. 2000. Irony markers and functions: Towards a goal-oriented theory of irony and its processing. Rask: International journal of language and communication, 12(1):3–20. find
  2. Francesco Barbieri, Valerio Basile, Danilo Croce, Malvina Nissim, Nicole Novielli, and Viviana Patti. 2016. Overview of the EVALITA 2016 SENTIment POLarity Classification task. In Proceedings of EVALITA 2016 Evaluation of NLP and Speech Tools for Italian. CEUR-WS.org. find
  3. Valerio Basile, Andrea Bolioli, Malvina Nissim, Viviana Patti, and Paolo Rosso. 2014. Overview of the EVALITA 2014 SENTIment POLarity Classification task. In Proceedings of EVALITA 2014 Evaluation of NLP and Speech Tools for Italian. CEUR-WS.org. find
  4. Farah Benamara, Cyril Grouin, Jihen Karoui, Véronique Moriceau, and Isabelle Robba. 2017. Analyse d’opinion et langage figuratif dans des tweets: présentation et résultats du Défi Fouille de Textes DEFT2017. In Actes de l’atelier DEFT2017 associé à la conférence TALN, pages 1–12. Association pour le… find
  5. Yoshua Bengio. 2009. Learning Deep Architectures for AI. Foundations and Trends® in Machine Learning, 2(1):1–127. find
  6. Raymond Chakhachiro. 2018. Translating Irony between English and Arabic. Cambridge Scholars Publishing. find
  7. Alessandra Teresa Cignarella and Cristina Bosco. 2019. ATC at IroSvA 2019: Shallow Syntactic Dependency-based Features for Irony Detection in Spanish Variants. In Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2019), pages 257–263. CEUR-WS.org. find
  8. Alessandra Teresa Cignarella, Simona Frenda, Valerio Basile, Cristina Bosco, Viviana Patti, and Paolo Rosso. 2018. Overview of the Evalita 2018 task on irony detection in Italian tweets (IRONITA). In Proceedings of EVALITA 2018 Evaluation of NLP and Speech Tools for Italian. CEUR-WS.org. find our card
  9. Alessandra Teresa Cignarella, Cristina Bosco, and Paolo Rosso. 2019. Presenting TWITTIRO-UD: An Italian Twitter Treebank in Universal Dependencies. In Proceedings of the Fifth International Conference on Dependency Linguistics (Depling, SyntaxFest 2019), pages 190–197. Association for Computational… find
  10. Andrea Cimino, Lorenzo De Mattei, and Felice Dell’Orletta. 2018. Multi-task learning in deep neural networks at Evalita 2018. In Proceedings of EVALITA 2018 Evaluation of NLP and Speech Tools for Italian, pages 86–95. CEUR-WS.org. find
  11. Bilal Ghanem, Alessandra Teresa Cignarella, Cristina Bosco, Paolo Rosso, and Francisco Manuel Rangel Pardo. 2019. UPV-28-UNITO at SemEval-2019 Task 7: Exploiting Post’s Nesting and Syntax Information for Rumor Stance Classification. In Proceedings of the 13th International Workshop on Semantic Evalu… find
  12. Bilal Ghanem, Jihen Karoui, Farah Benamara, Paolo Rosso, and Véronique Moriceau. 2020. Irony detection in a multilingual context. In Proceedings of the European Conference on Information Retrieval, pages 141–149. Springer. find
  13. Aniruddha Ghosh, Guofu Li, Tony Veale, Paolo Rosso, Ekaterina Shutova, John Barnden, and Antonio Reyes. 2015. SemEval-2015 task 11: Sentiment analysis of figurative language in Twitter. In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 470–478. Association… find
  14. José-Ángel González, Lluís-Felip Hurtado, and Ferran Pla. 2019. ELiRF-UPV at IroSvA: Transformer Encoders for Spanish Irony Detection. In Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2019). CEUR-WS.org. find
  15. Herbert P. Grice. 1975. Logic and conversation. In P. Cole and J. L. Morgan, editors, Speech acts, pages 41–58. New York: Academic Press. find
  16. Aditya Joshi, Vinita Sharma, and Pushpak Bhattacharyya. 2015. Harnessing context incongruity for sarcasm detection. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Pa… find our card
  17. Armand Joulin, Edouard Grave, Piotr Bojanowski, Matthijs Douze, Hérve Jégou, and Tomas Mikolov. 2016. Fasttext. zip: Compressing text classification models. arXiv preprint arXiv:1612.03651. find our card
  18. Hiroshi Kanayama and Ran Iwamoto. 2020. How Universal are Universal Dependencies? Exploiting Syntax for Multilingual Clause-level Sentiment Detection. In Proceedings of The 12th Language Resources and Evaluation Conference (LREC 2020), pages 4063–4073. European Language Resources Association (ELRA). find
  19. Jihen Karoui, Farah Benamara Zitoune, Véronique Moriceau, Nathalie Aussenac-Gilles, and Lamia Hadrich Belguith. 2015. Towards a contextual pragmatic model to detect irony in tweets. In The 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference… find our card
  20. Omer Levy and Yoav Goldberg. 2014. Dependency-based word embeddings. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 302–308. Association for Computational Linguistics (ACL). find
  21. Lizhen Liu, Donghai Zhang, and Wei Song. 2018a. Exploiting syntactic structures for humor recognition. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1875–1883. Association for Computational Linguistics (ACL). find
  22. Yijia Liu, Yi Zhu, Wanxiang Che, Bing Qin, Nathan Schneider, and Noah A. Smith. 2018b. Parsing tweets into Universal Dependencies. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)… find our card
  23. Sean MacAvaney and Amir Zeldes. 2018. A Deeper Look into Dependency-Based Word Embeddings. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop, pages 40–45. Association for Computational Linguistics (ACL). find
  24. Taylor Mahler, Willy Cheung, Micha Elsner, David King, Marie-Catherine de Marneffe, Cory Shain, Symon Stevens-Guille, and Michael White. 2017. Breaking NLP: Using Morphosyntax, Semantics, Pragmatics and World Knowledge to Fool Sentiment Analysis Systems. In Proceedings of the First Workshop on Build… find our card
  25. Ryan McDonald, Joakim Nivre, Yvonne Quirmbach-Brundage, Yoav Goldberg, Dipanjan Das, Kuzman Ganchev, Keith Hall, Slav Petrov, Hao Zhang, Oscar Täckström, et al. 2013. Universal dependency annotation for multilingual parsing. In Proceedings of the 51st Annual Meeting of the Association for Computatio… find
  26. Laura A. Michaelis and Hanbing Feng. 2015. What is this, sarcastic syntax? Constructions and Frames, 7(2):148– 180. find
  27. Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Jan Hajic, Christopher D. Manning, Sampo Pyysalo, Sebastian Schuster, Francis M. Tyers, and Daniel Zeman. 2020. Universal Dependencies v2: An Evergrowing Multilingual Treebank Collection. In Proceedings of The 12th Language Resources and Evalu… find
  28. Reynier Ortega-Bueno, Francisco Rangel, Delia Irazú Hernández Farıas, Paolo Rosso, Manuel Montes-y Gómez, and José E Medina Pagola. 2019. Overview of the task on irony detection in Spanish variants. In Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2019), co-located with 34th Confere… find
  29. Juan-Pablo Posadas-Duran, Grigori Sidorov, and Ildar Batyrshin. 2014. Complete syntactic n-grams as style markers for authorship attribution. In Proceedings of the Mexican International Conference on Artificial Intelligence, pages 9–17. Springer. find
  30. Ines Rehbein, Josef Ruppenhofer, and Bich-Ngoc Do. 2019. tweeDe – A Universal Dependencies treebank for German tweets. In Proceedings of the 18th International Workshop on Treebanks and Linguistic Theories (TLT, SyntaxFest 2019), pages 100–108. Association for Computational Linguistics (ACL). find
  31. Mickael Rouvier and Pierre-Michel Bousquet. 2017. LIA@DEFT’2017: Multi-view Ensemble of Convolutional Neural Network. In Actes de l’atelier « Défi Fouille de Textes » (DEFT 2017), 24éme Conférence TALN: Traitement Automatique des Langues Naturelles, pages 13–26. find
  32. Manuela Sanguinetti, Cristina Bosco, Alberto Lavelli, Alessandro Mazzei, Oronzo Antonelli, and Fabio Tamburini. 2018. PoSTWITA-UD: An Italian Twitter treebank in Universal Dependencies. In Proceedings of the 11th International Conference on Language Resources and Evaluation (LREC 2018). European Lan… find
  33. Manuela Sanguinetti, Cristina Bosco, Lauren Cassidy, Özlem Çetinoğlu, Alessandra Teresa Cignarella, Teresa Lynn, Ines Rehbein, Özlem Ruppenhofer, Djamé Seddah, and Amir Zeldes. 2020. Treebanking User-Generated Content: A Proposal for a Unified Representation in Universal Dependencies. In Proceeding… find
  34. Djamé Seddah, Farah Essaidi, Amal Fethi, Matthieu Futeral, Benjamin Muller, Pedro Javier Ortiz Suárez, Benoît Sagot, and Abhishek Srivastava. 2020. Building a user-generated content north-african arabizi treebank: Tackling hell. In Proceedings of the 58th Annual Meeting of the Association for Comput… find
  35. Grigori Sidorov, Francisco Velasquez, Efstathios Stamatatos, Alexander Gelbukh, and Liliana Chanona-Hernández. 2012. Syntactic dependency-based n-grams as classification features. In Proceedings of the Mexican International Conference on Artificial Intelligence, pages 1–11. Springer. find
  36. Grigori Sidorov, Francisco Velasquez, Efstathios Stamatatos, Alexander Gelbukh, and Liliana Chanona-Hernández. 2013. Syntactic dependency-based n-grams: More evidence of usefulness in classification. In Proceedings of the International Conference on Intelligent Text Processing and Computational Ling… find
  37. Grigori Sidorov, Francisco Velasquez, Efstathios Stamatatos, Alexander Gelbukh, and Liliana Chanona-Hernández. 2014. Syntactic n-grams as machine learning features for natural language processing. Expert Systems with Applications, 41(3):853–860. find
  38. Grigori Sidorov. 2014. Should syntactic n-grams contain names of syntactic relations?. International Journal of Computational Linguistics Applications, 5(2):25–47. find
  39. Natalia Silveira, Timothy Dozat, Marie-Catherine De Marneffe, Samuel R Bowman, Miriam Connor, John Bauer, and Christopher D Manning. 2014. A gold standard dependency corpus for English. In Proceedings of the 9th International Conference on Language Resources and Evaluation (LREC 2014), pages 2897–29… find
  40. Maria Simi, Cristina Bosco, and Simonetta Montemagni. 2014. Less is More? Towards a Reduced Inventory of Categories for Training a Parser for the Italian Stanford Dependencies. In Proceedings of the 9th International Conference on Language Resources and Evaluation (LREC 2014), page 83–90. European L… find
  41. Richard Socher, Alex Perelygin, Jean Y. Wu, Jason Chuang, Christopher D. Manning, Andrew Y. Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP… find
  42. Dan Sperber and Deirdre Wilson. 1981. Irony and the use-mention distinction. Philosophy, 3:143–184. find
  43. Mariona Taulé, Maria Antònia Martí, and Marta Recasens. 2008. AnCora: Multilevel Annotated Corpora for Catalan and Spanish. In Proceedings of the 6th Language Resources and Evaluation Conference (LREC 2008), pages 96–101. European Language Resources Association (ELRA). find
  44. Akira Utsumi. 1996. A unified theory of irony and its computational formalization. In Proceedings of the 16th Conference on Computational Linguistics - Volume 2, pages 962–967. Association for Computational Linguistics (ACL). find our card
  45. Cynthia Van Hee, Els Lefever, and Véronique Hoste. 2018. SemEval-2018 task 3: Irony detection in English tweets. In Proceedings of The 12th International Workshop on Semantic Evaluation (SemEval 2018), pages 39–50. Association for Computational Linguistics (ACL). find our card
  46. Chuhan Wu, Fangzhao Wu, Sixing Wu, Junxin Liu, Zhigang Yuan, and Yongfeng Huang. 2018. THU_NGN at SemEval-2018 Task 3: Tweet Irony Detection with Densely Connected LSTM and Multi-task Learning. In Proceedings of The 12th International Workshop on Semantic Evaluation (SemEval 2018), pages 51–56. Asso… find
  47. Jaime Zabalza, Jinchang Ren, Jiangbin Zheng, Huimin Zhao, Chunmei Qing, Zhijing Yang, Peijun Du, and Stephen Marshall. 2016. Novel segmented stacked autoencoder for effective dimensionality reduction and feature extraction in hyperspectral imaging. Neurocomputing, 185:1–10. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/cignarella-2020-multilingual-irony-detection-dependency-syntax/