Elamur Research⌕
☰
home / papers / 2015 / Your Sentiment Precedes You: Using an author's historical tweets to predict sarcasm

Your Sentiment Precedes You: Using an author's historical tweets to predict sarcasm

Summary

Some sarcastic tweets cannot be recognized from their wording alone. Whether "I love spending four hours cooking" is ironic depends on the author's real attitude. Khattri et al. address this by bringing in the author's own Twitter timeline as extra context.

Their rule-based system has two parts:

  • Contrast-based predictor: looks for opposing sentiment inside the tweet, following Riloff et al. (2013).
  • Historical tweet-based predictor: extracts NNP sequences as target phrases, retrieves the author's earlier tweets that mention them, and takes a majority vote of their sentiment. A conflict between the current tweet's sentiment and that past opinion signals sarcasm.

Four integrators combine the two predictors, and each setup is tested with two sentiment lexicons. On Riloff et al.'s annotated test corpus, the Relaxed-AND integrator reaches an F-score of 0.882, far above the 0.51 benchmark. The authors present this as the first quantitative evidence that an author's past writing helps detect sarcasm. The idea also carries over to forums and review sites.

Key points

  • Test data: Riloff et al. (2013) corpus with 2278 retrievable tweets, of which 506 are sarcastic. Timelines could not be obtained for 248 of 2273 authors, because accounts were private, renamed, or deactivated. The Twitter API returns only the latest 3500 tweets per user.

  • Implicit sentiment phrases: 445 phrases, mined as 3- to 10-grams that occur at least three times in 8000 #sarcasm tweets. The in-house rule-based sentiment classifier scores 58.49% accuracy on Sentiment140.

  • Integrators: Only historical tweet-based, OR, AND, and Relaxed-AND. Relaxed-AND requires both predictors to agree, but falls back on the contrast predictor when the author has no relevant history.

  • Lexicons: L1 from Pang and Lee (2004), used in setup SD1, and L2 from Mohammad and Turney (2013), used in setup SD2.

  • Results (F-score):

    ConfigurationSD1SD2
    History alone≈0.50≈0.50
    AND0.6170.627
    Relaxed-AND0.8260.882

    Relaxed-AND also has positive-class precision of 0.777 (SD1) and 0.811 (SD2). The method assumes that past tweets express the author's sincere sentiment.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

21 papers of our library cite it (of the 140673 whose bibliographies we read); 11 of them have a page here, 10 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w15-2905

@inproceedings{khattri2015your,
  title = {{Your Sentiment Precedes You: Using an author's historical tweets to predict sarcasm}},
  author = {Khattri, Anupam and Joshi, Aditya and Bhattacharyya, Pushpak and Carman, Mark},
  booktitle = {Proceedings of the 6th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis},
  year = {2015},
  doi = {10.18653/v1/w15-2905},
  url = {https://doi.org/10.18653/v1/w15-2905}
}

References

12 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Francesco Barbieri, Horacio Saggion, and Francesco Ronzano. 2014. Modelling sarcasm in twitter, a novel approach. ACL 2014, page 50. find our card
  2. Paula Carvalho, Luı́s Sarmento, Mário J Silva, and Eugénio de Oliveira. 2009. Clues for detecting irony in user-generated contents: oh...!! it’s so easy;-). In Proceedings of the 1st international CIKM workshop on Topic-sentiment analysis for mass opinion, pages 53–56. ACM. find
  3. Dmitry Davidov, Oren Tsur, and Ari Rappoport. 2010. Semi-supervised recognition of sarcastic sentences in twitter and amazon. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning, pages 107–116. Association for Computational Linguistics. find our card
  4. Roberto Gonzalez-Ibanez, Smaranda Muresan, and Nina Wacholder. 2011. Identifying sarcasm in twitter: a closer look. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers-Volume 2, pages 581–586. Association for Computational Linguistics. find our card
  5. Aditya Joshi, Vinita Sharma, and Pushpak Bhattacharyya. 2015. Harnessing context incongruity for sarcasm detection. In ACL-IJCNLP 2015. find our card
  6. Gregory Malecha and Ian Smith. 2010. Maximum entropy part-of-speech tagging in nltk. unpublished course-related report. find
  7. Diana Maynard and Mark A Greenwood. 2014. Who cares about sarcastic tweets? investigating the impact of sarcasm on sentiment analysis. In Proceedings of LREC. find our card
  8. Saif M. Mohammad and Peter D. Turney. 2013. Crowdsourcing a word-emotion association lexicon. 29(3):436–465. ↗
  9. Bo Pang and Lillian Lee. 2004. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In Proceedings of the 42nd annual meeting on Association for Computational Linguistics, page 271. Association for Computational Linguistics. ↗
  10. Ellen Riloff, Ashequl Qadir, Prafulla Surve, Lalindra De Silva, Nathan Gilbert, and Ruihong Huang. 2013. Sarcasm as contrast between a positive sentiment and negative situation. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 704–714. Association for Computational Linguistics. find our card
  11. Oren Tsur, Dmitry Davidov, and Ari Rappoport. 2010. Icwsm-a great catchy name: Semi-supervised recognition of sarcastic sentences in online product reviews. In ICWSM. find
  12. Byron C Wallace, Do Kook Choe, and Eugene Charniak. 2015. Sparse, contextually informed models for irony detection: Exploiting user communities, entities and sentiment. In ACL-IJCNLP 2015. find

↑ back to top

This page in Markdown →