Elamur Research⌕
☰
home / papers / 2015 / Sentiment Analyzer with Rich Features for Ironic and Sarcastic Tweets

Sentiment Analyzer with Rich Features for Ironic and Sarcastic Tweets

Paper · 2015 · Piyoros Tungthamthiti, Enrico Santus, Hongzhi Xu et al.

Summary

Tweets are hard to score for sentiment, and figurative language makes it harder because the literal wording often points the opposite way from the attitude the writer means. The authors build a sentiment system aimed at ironic and sarcastic tweets. It predicts an intensity score on an 11-point scale from −5 to +5, using the SemEval 2015 Task 11 data.

Two earlier systems, from Xu et al. 2015 and Tungthamthiti et al. 2014, are refined and merged. The first module supplies token, n-gram, dependency-pair, emphasis and lexicon-polarity features. The second adds concept expansion with common-sense knowledge, counts of punctuation and special symbols, and a new cross-sentence coherence feature that supports contradiction detection. A RepTree regression tree in Weka makes the final prediction.

The combined system reaches cosine 0.953 on sarcasm and 0.921 on irony, beating all 15 task participants on those categories, and ranks second overall. Its weak results on metaphor and literal tweets suggest that methods built around incongruity need a separate treatment for texts that are congruous.

Key points

  • Data and evaluation: SemEval 2015 Task 11, with a test set of about 4,000 tweets across sarcasm, irony, metaphor and non-figurative categories. Scores are measured by cosine similarity and RMSE.
  • Pre-processing:
    • Stanford Lemmatizer for lemmas and Stanford parser for POS tags and dependencies.
    • Repeated vowels and capitals are normalized, while emphasized words are kept in a separate feature bag.
    • Runs of "?!" are collapsed, and glued words are split by maximal matching (e.g. "yeahright").
    • Usernames, URLs and hashtags are dropped.
  • Lexicon features:
    • Four lexicons are used: Opinion Lexicon, Afinn, MPQA and SentiWordnet. Their union and intersection serve as two extra dictionaries.
    • Polarity is summed per POS and per dictionary.
    • Negation is checked either with a 5-token window or along the dependency tree.
    • Polarity shift is measured within the window and across parent–child pairs.
  • Results:
    • Module 1 alone scores cosine 0.896 on sarcasm and 0.918 on irony.
    • Module 2 alone scores 0.948 on sarcasm and 0.912 on irony.
    • The integrated system scores 0.953 on sarcasm, 0.921 on irony and 0.736 overall (RMSE 1.382), against 0.758 overall for the top system, ClaC.
  • Significance: Paired t-tests show that the integrated system differs from each single module at roughly the 90% level. The p-values in the table and in the text are reported inconsistently.

Interesting ideas

  • I1. Normalize lengthened vowels and all-caps words so the lemmatizer and parser work properly, but store the original emphasis as a separate signal instead of discarding it.
  • I2. Decide negation scope along dependency arcs rather than with a fixed word window, and report both variants as separate features.
  • I3. Measure the polarity clash between a dependency head and its child (e.g. a positive verb with a negative object) as a cue for incongruity.
  • I4. Before calling two sentences contradictory, first check that they actually refer to the same things; the coherence feature measurably improved sarcasm scoring.
  • I5. Break down evaluation by figurative category: this exposed that incongruity-based systems collapse on metaphor and literal texts.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): The en tokenizer could output a clean lemma plus an "emphasis" attribute per token, so parsing stays robust while prag still sees the writer's intensity.
  • I2 (frames): Prag could compute polarity under negation directly over the UD tree and compare it with a window baseline as an explainable sanity check.
  • I3 (frames): A head–dependent polarity-contrast score could become one explicit, traceable trigger in the irony detector instead of an opaque classifier score.
  • I4 (frames): Irony and contradiction rules in prag could be gated on coreference links, so that unrelated sentences with opposite sentiment are not flagged.
  • I5 (learn): Mova's evaluations could report results per phenomenon class, including congruous controls, so that an irony heuristic which hurts literal readings shows up immediately.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Hong Kong Polytechnic UniversityHong Kong · China34 works · 4,476 citations🎓Japan Advanced Institute of Science and TechnologyNomi Shi · Japan3 works · 934 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksPsychology · 798 worksExperimental and Cognitive Psychology · 463 works

🧭 Topics 1Research subjects and languages the work is about.Sentiment & emotion · 608 works

Cite

Cite as Piyoros Tungthamthiti, Enrico Santus, Hongzhi Xu, Chu-Ren Huang and Kiyoaki Shirai. 2015. Sentiment Analyzer with Rich Features for Ironic and Sarcastic Tweets. Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{tungthamthiti2015sentiment,
  title = {{Sentiment Analyzer with Rich Features for Ironic and Sarcastic Tweets}},
  author = {Piyoros Tungthamthiti, Enrico Santus, Hongzhi Xu, Chu-Ren Huang and Kiyoaki Shirai},
  booktitle = {Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation},
  year = {2015},
  url = {https://aclanthology.org/Y15-1021/}
}

References

14 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Baccianella S., Esuli A., and Sebastiani F.. 2010. Sentiwordnet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining. In LREC, volume 10, pages 2200–2204.
  2. Barbieri F. and Saggion H. 2014, Modelling irony in twitter, In Proceedings of the Student Research Workshop at the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 56–64
  3. Buschmeier K., Cimiano P., and Klinger R. 2014, An impact analysis of features in a classification approach to irony detection in product reviews Cambria E., Speer R., Havasi C. and Hussain A. 2010, SenticNet: A Publicly Available Semantic Resource for Opinion Mining, Commonsense Knowledge: Papers from the AAAI Fall Symposium Carvalho P., Sarmento L., Silva J. M. and Oliveira D. E. 2009, Clues for detecting irony in user-generated contents: oh...!! its so easy;-), In Proceedings of the 1st international CIKM workshop on Topic-sentiment analysis for mass opinion, pages 53–56
  4. Davidov D., Tsur O., and Rappoport A. 2010, Semi-supervised recognition of sarcastic sentences in twitter and amazon, In Proceedings of the Fourteenth Conference on Computational Natural Language Learning, pages 107–116, Association for Computational Linguistics Ellen R., Ashequl Q., Prafulla S., Lalindra S., Gilbert D. S., Gilbert N., Ruihong H. 2013, Sarcasm as Contrast between a Positive Sentiment and Negative Situation, In Proceedings of the Conference on Empirical Methods on Natural Language Processing (EMNLP), pages 704–714, Seattle, Washington Finn Årup Nielsen. 2011, A new anew: Evaluation of a word list for sentiment analysis in microblogs, In Proceedings of the ESWC2011 Workshop on ‘Making Sense of Microposts’: Big things come in small packages.
  5. Ghosh A., Li G., Veale T., Rosso P., Shutova E., Reyes A., and Barnden J. 2015, Semeval-2015 task 11: Sentiment analysis of figurative language in twitter, In Proceedings of the International Workshop on Semantic Evaluation (SemEval-2015), Co-located with NAACL and *SEM, Denver, Colorado, USA Gimenez M., Pla F. and Hurtado L.F. 2015, ELiRF: A Support Vector Machine Approach for Sentiment Analysis Tasks in Twitter at SemEval-2015, In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), Association for Computational Linguistics, pages 673–678, Denver, Colorado Grice H. P. 1975, Logic and conversion, Syntax and semantics 3: Speech arts, pages: 41–58
  6. Haiman J. 1998, Talk is cheap: Sarcasm, alienation, and the evolution of language, Language Arts & Disciplines, Oxford, Oxford University Press Hall M., Frank E., Holmes G., Pfahringer B., Reutemann P., and Witten I.H. 2009, The weka data mining software: An update, SIGKDD Explorations Hao Y. and Veale T. 2010, An ironic fist in a velvet glove: Creative mis-representation in the construction of ironic similes, Minds and Machines, 20(4):635–650 find
  7. Hart L. 2013, The Linguistics of Sentiment Analysis, Portland State University, PDX Scholar 2013, http://pdxscholar.library.pdx.edu/honorstheses/20 ↗
  8. Martin J. R. and White P. R.R. 2005, The Language of Evaluation: Appraisal in English, Palgrave, London, UK Miller A. G. 1995 WordNet: A Lexical Database for English Communications of the ACM Hu M. and Liu B., 2004. Mining and summarizing customer reviews, In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 168–177, ACM.
  9. Pak E. and Paroubek P. 2010, Twitter as a corpus for sentiment analysis and opinion mining, In Proceedings of the Seventh Conference on International Language Resources and Evaluation Pang B. and Lee L. 2008, Opinion mining and sentiment analysis, Foundations and Trends in Information Retrieval Vol. 2, pages 1–135
  10. Quintilian 1953, The institutio Oratoria of Quintilian. With an English Translation by Harold Edgeworth Butler. London: William Heinemann Resnik P. 1995, Using Information Content to Evaluate Semantic Similarity in a Taxonomy, CoRR Reyes A., Rosso P. 2012, Making objective decisions from subjective data: detecting irony in customer reviews, Decision Support System 2012, 53:754–760.
  11. Reyes A., Rosso P., and Veale T. 2013, A multidimensional approach for detecting irony in twitter, Language Resources and Evaluation, 47(1):239–268. find
  12. Soon W. M., Ng H. T, Lim D. C. Y. 2001 A Machine Learning Approach to Coreference Resolution of Noun Phrases Computational Linguistics, pages 521–544, Cambridge, MA, USA Stevenson A. 2010, Oxford dictionary of English, Oxford University Press Stringfellow, F. J. 1994. The Meaning of Irony NewYork: State University of NY.
  13. Thaseen S. and Kumar C. A. 2013, An analysis of supervised tree based classifiers for intrusion detection system, Pattern Recognition, Informatics and Mobile Engineering (PRIME), 2013 International Conference on, pages 294–299, Salem, MA, USA Tsur O., Davidov D. 2010 ICWSM - a great catchy name: Semi-supervised recognition of sarcastic sentences in product reviews In International AAAI Conference on Weblogs and Social Tungthamthiti P., Kiyoaki S., and Masnizah M. 2014, Recognition of Sarcasm in Tweets Based on Concept Level Sentiment Analysis and Supervised Learning Approaches, In Proceedings of the 28th Pacific Asia Conference on Language, Information and Computation, pages 404–413, Phuket, Thailand Wiebe J., Wilson T., and Cardie C. 2005. Annotating expressions of opinions and emotions in language, Language resources and evaluation, 39(2-3):165–210. find
  14. Xu, Hongzhi and Santus, Enrico and Laszlo, Anna and Huang, Chu-Ren 2015, LLT-PolyU: Identifying Sentiment Intensity in Ironic Tweets, In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), Association for Computational Linguistics, pages 673–678, Denver, Colorado

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tungthamthiti-2015-sentiment-analyzer-rich-features-ironic/