Elamur Research
☰
home / papers / 2016 / Modelling Context with User Embeddings for Sarcasm Detection in Social Media

Modelling Context with User Embeddings for Sarcasm Detection in Social Media

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/K16-1017.pdf

Summary

Sarcasm is hard to detect from wording alone, because the same sentence can be sincere or ironic depending on who writes it. Earlier systems added information about the speaker through hand-built features such as sentiment lexicons, profile data and behavioural traits. Those features take a lot of work to build and rely on external tools. The authors replace them with learned user embeddings. Each Twitter author gets a vector trained only on the text of their earlier posts, using an objective similar to Paragraph Vector: the model predicts a user's words from that user's vector. The model, CUE-CNN, encodes the tweet with a convolutional network, combines that encoding with the author's vector, and passes both through a hidden layer before classification. Its accuracy is more than 2 points higher than the feature-rich model of Bamman and Smith (2015), and its results vary less across cross-validation folds. The authors also show that the learned user vectors reflect real user traits. The main point is that speaker context can be learned automatically instead of hand-engineered.

Key points

  • CUE-CNN makes its prediction from two inputs together: CNN features of the tweet (ReLU filters, concatenated) and an embedding of the author.
  • User embeddings are pre-trained with a hinge-loss objective over word usage and negative sampling. Negatives drawn from a unigram distribution over all users' posts gave 0.8% more accuracy than uniformly random negatives.
  • Each step helped: adding author context, pre-training the user vectors, and adding a hidden layer that models how user and content interact.
  • In a t-SNE plot, users fall into groups by political leaning (based on following Democratic vs. Republican politicians) and by interest in sports. This suggests the vectors capture homophily, meaning similar users end up close together.
  • Training used cross-entropy loss with AdaDelta. Word vectors were learned with Hierarchical Softmax as implemented in Gensim.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

77 papers of our library cite it (of the 140673 whose bibliographies we read); 16 of them have a page here, 61 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/k16-1017 · arXiv 1607.00976

@inproceedings{amir2016modelling,
  title = {{Modelling Context with User Embeddings for Sarcasm Detection in Social Media}},
  author = {Amir, Silvio and Wallace, Byron C. and Lyu, Hao and Carvalho, Paula and Silva, Mário J.},
  booktitle = {Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning},
  year = {2016},
  doi = {10.18653/v1/k16-1017},
  eprint = {1607.00976},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/k16-1017}
}

References

1 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ramón Astudillo, Silvio Amir, Wang Ling, Mario Silva, and Isabel Trancoso. 2015. Learning word representations from scarce and noisy data with embedding subspaces. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1074–1084, Beijing, China, July. find