Elamur Research⌕
☰
home / papers / 2018 / THU_NGN at SemEval-2018 Task 2: Residual CNN-LSTM Network with Attention for English Emoji Prediction

THU_NGN at SemEval-2018 Task 2: Residual CNN-LSTM Network with Attention for English Emoji Prediction

Paper · 2018 · Chuhan Wu, Fangzhao Wu, Sixing Wu et al.

Summary

This paper describes the THU_NGN entry to SemEval-2018 Task 2, subtask 1. The task is to guess which of 20 emojis originally accompanied an English tweet after the emoji has been removed. The team built RCLA, a residual CNN-LSTM with attention, which processes the tweet in stages:

  • Word embeddings, pre-trained on tweets and fine-tuned, are joined with POS tags from Ark-Tweet-NLP.
  • A Bi-LSTM reads the tweet for long-range context.
  • An attention layer reweights each hidden state.
  • A three-layer CNN with residual links and several window sizes picks up local patterns, followed by max pooling.
  • 45 lexicon-based sentiment features from the AffectiveTweets package are concatenated before a softmax classifier.

Because the emoji classes are heavily skewed, the cross-entropy loss gives rarer labels larger weights. The system scored 30.25% macro-F on the test set, placing 7th of 48 teams. An ablation shows that each component adds a modest gain. The work is useful as a careful, well-ablated recipe for noisy short-text classification with imbalanced labels.

Key points

  • The data has 20 emoji classes, 489,277 training tweets, and 50,000 tweets each in the trial and test sets. Evaluation uses macro-averaged F-score. The top emoji covers over 20% of the data, while some classes have only about 2–3%.
  • Embeddings come from Barbieri et al. (2016b): 300-dimensional vectors trained on 20 million geo-localized tweets.
  • Hyperparameters were tuned on the trial set:
    • Bi-LSTM hidden size 300
    • CNN windows of 2/3/4 with 200 filters
    • dropout 0.2 and batch size 500
    • RMSProp optimizer, up to 100 epochs
  • The architecture ablation gives these macro-F scores: LSTM 28.72, CNN 28.86, CL 29.35, RCL 29.61, RCLA 30.25.
  • The feature ablation gives: no extra features 29.08, +POS 29.76, +Sentiment 29.55, both 30.25. The sentiment features come from the TweetToLexiconFeatureVector and TweetToSentiStrengthFeatureVector filters in Weka.

Interesting ideas

  • I1. Class weights in the loss grow as a label's frequency shrinks, using a square-root scaling, so rare classes get more weight without letting them dominate training.
  • I2. The attention weights rescale each position's hidden state rather than collapsing the sentence into one vector, so later layers still see word order while the per-token weights remain inspectable.
  • I3. Hand-built lexicon features (sentiment scores, emoticon and hashtag cues) are concatenated with the learned representation just before the classifier, giving a transparent channel next to the neural one.
  • I4. An incremental ablation ladder adds one component at a time (CNN or LSTM alone, then CL, then residual, then attention, then each feature group) and reports each step's macro-F, which makes the contribution of every part visible.
  • I5. The error analysis separates two kinds of confusion: label pairs that share meaning (both used for happy tweets) and pairs that merely look alike.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Mova could weight rare UD relations or uncommon VerbNet roles this way when training parsers on skewed silver data, instead of letting frequent labels swamp the gradient.
  • I2 (frames): A per-token salience score that keeps word order could flag which words drive an irony or speech-act decision, while leaving the full sequence available for role assignment.
  • I3 (frames): Lexicon-based polarity features could be fed into irony and implicature detection as named, inspectable inputs, which helps keep each decision explainable.
  • I4 (learn): Mova's evaluation could adopt this stepwise ladder, paired with negative controls, to prove that each repair rule or witness parser actually earns its place.
  • I5 (learn): Splitting parser or frame confusions into "semantically overlapping" and "near-duplicate inventory" cases could show where label sets should be merged, rather than modelled harder.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tsinghua UniversityBeijing · China333 works · 87,019 citations🏢Microsoft Research AsiaBeijing · China73 works · 60,539 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksHuman-Computer Interaction · 141 works

🧭 Topics 1Research subjects and languages the work is about.English · 2,263 works

Cite

Cite as Chuhan Wu, Fangzhao Wu, Sixing Wu, Zhigang Yuan, Junxin Liu and Yongfeng Huang. 2018. THU_NGN at SemEval-2018 Task 2: Residual CNN-LSTM Network with Attention for English Emoji Prediction. Proceedings of the 12th International Workshop on Semantic Evaluation.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/s18-1063

@inproceedings{wu2018thu,
  title = {{THU\_NGN at SemEval-2018 Task 2: Residual CNN-LSTM Network with Attention for English Emoji Prediction}},
  author = {Chuhan Wu, Fangzhao Wu, Sixing Wu, Zhigang Yuan, Junxin Liu and Yongfeng Huang},
  booktitle = {Proceedings of the 12th International Workshop on Semantic Evaluation},
  year = {2018},
  doi = {10.18653/v1/s18-1063},
  url = {https://doi.org/10.18653/v1/s18-1063}
}

References

15 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Petra Kralj Novak, Jasmina Smailovic, Borut Sluban, and Igor Mozetic. 2015. Sentiment of emojis. PLOS ONE, 10(12). ↗
  2. Sho Aoki and Osamu Uchida. 2011. A method for automatically generating the emotional vectors of emoticons using weblog articles. In Proc. 10th WSEAS Int. Conf. on Applied Computer and Applied Computational Science, Stevens Point, Wisconsin, USA, pages 132–136. find
  3. Raúl Santos-Rodrı́guez, Darı́o Garcı́a-Garcı́a, and Jesús Cid-Sueiro. 2009. Cost-sensitive classification based on bregman divergences for medical diagnosis. In Machine Learning and Applications, 2009. ICMLA’09. International Conference on, pages 551–556. IEEE. find
  4. Francesco Barbieri, Luis Espinosa Anke, and Horacio Saggion. 2016a. Revealing patterns of twitter emoji usage in barcelona and madrid. In CCIA, pages 239– 244. ↗
  5. Mike Thelwall, Kevan Buckley, and Georgios Paltoglou. 2012. Sentiment strength detection for the social web. Journal of the Association for Information Science and Technology, 63(1):163–173. ↗
  6. Francesco Barbieri, Miguel Ballesteros, and Horacio Saggion. 2017. Are emojis predictable? In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 105–111. Association for Computational Linguistics. find
  7. Sanjaya Wijeratne, Lakshika Balasuriya, Amit Sheth, and Derek Doran. 2017. A semantics-based measure of emoji similarity. In 2017 IEEE/WIC/ACM International Conference on Web Intelligence (WI), Leipzig, Germany. ACM, ACM. find
  8. Chuhan Wu, Fangzhao Wu, Yongfeng Huang, Sixing Wu, and Zhigang Yuan. 2017. Thu ngn at ijcnlp-2017 task 2: Dimensional sentiment analysis for chinese phrases with deep lstm. Proceedings of the IJCNLP 2017, Shared Tasks, pages 47–52. find
  9. Francesco Barbieri, Jose Camacho-Collados, Francesco Ronzano, Luis Espinosa-Anke, Miguel Ballesteros, Valerio Basile, Viviana Patti, and Horacio Saggion. 2018. SemEval-2018 Task 2: Multilingual Emoji Prediction. In Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), New Orleans, LA, United States. Association for Computational Linguistics.
  10. Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierarchical attention networks for document classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1480–1489.
  11. Francesco Barbieri, German Kruszewski, Francesco Ronzano, and Horacio Saggion. 2016b. How cosmopolitan are emojis?: Exploring emojis usage and meaning over different languages with distributional semantics. In Proceedings of the 2016 ACM on Multimedia Conference, pages 531–535. ACM. find
  12. Francesco Barbieri, Francesco Ronzano, and Horacio Saggion. 2016c. What does this emoji mean? a vector space skip-gram model for twitter emojis. In LREC. find
  13. Felipe Bravo-Marquez, Marcelo Mendoza, and Barbara Poblete. 2014. Meta-level sentiment models for big social data analysis. Knowledge-Based Systems, 69:86–99. ↗
  14. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770– 778. ↗
  15. Nikola Ljubešić and Darja Fišer. 2016. A global analysis of emoji usage. In Proceedings of the 10th Web as Corpus Workshop, pages 82–89. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wu-2018-thu-ngn-semeval-2018-task-s18-1063/