Elamur Research⌕
☰
home / papers / 2017 / NileTMRG at SemEval-2017 Task 4: Arabic Sentiment Analysis

NileTMRG at SemEval-2017 Task 4: Arabic Sentiment Analysis

Paper · 2017 · Samhaa R. El-Beltagy, Mona El Kalamawy and Abu Bakr Soliman

Summary

The paper describes the NileTMRG entries for the Arabic subtasks of SemEval-2017 Task 4. These are three-way tweet polarity (A), two-way polarity toward a given topic (B) and estimating the positive/negative distribution of tweets about a topic (D).

For A, the team reused its earlier NU sentiment analyzer, a Complement Naïve Bayes model over idf-weighted unigrams and bigrams plus hand-crafted lexical cues. They added two new features drawn from a weighted sentiment lexicon and enlarged the training data well beyond the cleaned official set.

For B, three classifiers vote on each label:

  • a Kim-style CNN over word2vec embeddings trained on 4 million Arabic tweets;
  • a multilayer perceptron using bag-of-words plus target-aware features;
  • a logistic regression model using the same input as the perceptron.

For D, the team built no separate model. They simply counted the labels predicted for B.

The team ranked first in all three Arabic subtasks. The result shows that careful data cleaning, lexicon resources and modest ensembles can lead a shared task for a lower-resource language.

Key points

  • Subtask A data:
    • The official data had 3355 training and 671 development tweets.
    • Removing conflicting duplicates, repeated copies and tweets overlapping the dev set left 2499 training tweets.
    • Adding the NBI and NU datasets and lexicon entries scoring above 0.8 raised the total to 13,292.
  • Subtask A result: average recall ρ 0.583, F1PN 0.610 and accuracy 0.581 on 6100 test tweets, ranked 1st.
  • Subtask B model:
    • The CNN has filter windows of 3 and 4, max pooling and a 25-node dense layer, built in Keras.
    • Inputs are 100-dimensional embeddings, fixed at 35 tokens, with the tweet centered and zero-padded.
    • Words missing from word2vec get random vectors.
  • Subtask B result: on 2757 test tweets the vote reached ρ 0.768, F1PN 0.767 and accuracy 0.770 (1st). No single classifier on its own matched the ensemble's ρ.
  • Subtask D result: KLD 0.127, AE 0.170 and RAE 2.462, ranked 1st by KLD.

Interesting ideas

  • I1. Before training, drop duplicate tweets whose labels disagree, keep one copy of consistent duplicates, and delete anything that also appears in the dev set; this removes label noise and leakage cheaply.
  • I2. High-confidence lexicon terms were injected into training as tiny one-word labeled examples, turning a dictionary resource directly into supervision.
  • I3. For target-dependent polarity, the topic mention is swapped for a fixed placeholder and features describe what sits near the target, where it occurs, and which half of the text holds sentiment words.
  • I4. A flag records whether the last sentiment-bearing word was positive or negative, a cheap proxy for the final word winning in mixed messages.
  • I5. The distribution-estimation task was solved by aggregating per-item classifier decisions rather than training a separate quantifier.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Apply the same dedup-and-conflict filter to silver sentences from the witness parsers, so disagreeing duplicates are discarded rather than averaged and evaluation sets stay clean.
  • I2 (learn): Entries from compiled resources (e.g., VerbNet class members) could become minimal training items for a tagger or frame classifier, covering rare words the corpus never shows.
  • I3 (frames): Frames could record evaluative words tied to a specific argument, not to the whole sentence, telling us which participant the text judges.
  • I4 (frames): Speech-act and irony detection could weigh the closing evaluative clause more heavily and expose that choice as an explainable rule.
  • I5 (answer): Questions asking how many characters felt or did something can be answered by counting per-entity states in the world model, keeping every count traceable to individual decisions.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Cairo UniversityGiza · Egypt42 works · 1,350 citations🎓Nile UniversityGiza · Egypt14 works · 158 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Sentiment & emotion · 972 worksSemitic languages · 2,908 works

Cite

Cite as Samhaa R. El-Beltagy, Mona El Kalamawy and Abu Bakr Soliman. 2017. NileTMRG at SemEval-2017 Task 4: Arabic Sentiment Analysis. Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/s17-2133

@inproceedings{elbeltagy2017niletmrg,
  title = {{NileTMRG at SemEval-2017 Task 4: Arabic Sentiment Analysis}},
  author = {Samhaa R. El-Beltagy, Mona {El Kalamawy} and Abu Bakr Soliman},
  booktitle = {Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017)},
  year = {2017},
  doi = {10.18653/v1/s17-2133},
  url = {https://doi.org/10.18653/v1/s17-2133}
}

References

24 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abdulla, N.A. et al., 2013. Arabic sentiment analysis: Lexicon-based and corpus-based. In Applied Electrical Engineering and Computing Technologies (AEECT), 2013 IEEE Jordan Conference on. Amman: IEEE, pp. 1–6. find
  2. Anon, 2012. Facebook Statistics by Country. Available at: http://www.socialbakers.com/facebook-5 794 statistics/. ↗
  3. Chollet, F., 2015. Keras: Deep Learning library for Theano and TensorFlow. GitHub Repository. find
  4. Duwairi, R. et al., 2014. Sentiment Analysis in Arabic tweets. In Information and Communication Systems (ICICS), 2014 5th International Conference. Irbid: IEEE, pp. 1–6. find
  5. El-Beltagy, S.R. et al., 2016. Combining Lexical Features and a Supervised Learning Approach for Arabic Sentiment Analysis. In CICLing 2016. Konya, Turkey. find
  6. El-Beltagy, S.R., 2016. NileULex: A Phrase and Word Level Sentiment Lexicon for Egyptian and Modern Standard Arabic. In proceedings of LREC 2016. Portorož , Slovenia.
  7. El-Beltagy, S.R., 2017. WeightedNileULex: A Scored Arabic Sentiment Lexicon for Improved Sentiment Analysis. In N. El-Gayar & C. Suen, eds. Language Processing, Pattern Recognition and Intelligent Systems. Special Issue on Computational Linguistics, Speech& Image Processing for Arabic Language. World Scientific Publishing Co. ↗
  8. El-Beltagy, S.R. & Ali, A., 2013. Open Issues in the Sentiment Analysis of Arabic Social Media : A Case Study. In Proceedings of 9th the International Conference on Innovations and Information Technology (IIT2013). Al Ain, UAE. find
  9. Farid, D., 2013. Egypt has the largest number of Facebook users in the Arab world. Daily News Egypt. Available at: http://www.dailynewsegypt.com/2013/09/25/eg ypt-has-the-largest-number-of-facebook-users-in-the-arab-world-report/. ↗
  10. Khalil, T. et al., 2015. Which configuration works best? An experimental study on Supervised Arabic Twitter Sentiment Analysis. In Proceedings of the First Conference on Arabic Computational Liguistics (ACLing 2015), co-located with CICLing 2015. Cairo, Egypt, pp. 86–93. find
  11. Kim, Y., 2014. Convolutional Neural Networks for Sentence Classification. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP 2014), pp.1746–1751.
  12. Mikolov, T., Sutskever, I., et al., 2013. Distributed Representations of Words and Phrases and their Compositionality. Advances in neural information processing systems, pp.3111–3119. ↗
  13. Mikolov, T., Corrado, G., et al., 2013. word2vec-v1. Proceedings of the International Conference on Learning Representations (ICLR 2013), pp.1– 12. find
  14. Neal, R.W., 2013. Twitter Usage Statistics: Which Country Has The Most Active Twitter Population? International Business Times. Available at: http://www.ibtimes.com/twitter-usage-statistics-which-country-has-most-active-twitter-population-1474852. ↗
  15. Palogiannidi, E. et al., 2016. Tweester at SemEval-2016 Task 4 : Sentiment Analysis in Twitter using Semantic-Affective Model Adaptation. , pp.160–168. find
  16. Refaee, E. & Rieser, V., 2014. Subjectivity and Sentiment Analysis of Arabic Twitter Feeds with Limited Resources. In Proceedings of the Workshop on Free/Open-Source Arabic Corpora and Corpora Processing Tools. Reykjavik, Iceland, pp. 16–21. find
  17. Rennie, J.D.M. et al., 2003. Tackling the Poor Assumptions of Naive Bayes Text Classifiers. Proceedings of the Twentieth International Conference on Machine Learning (ICML)-2003), 20(1973), pp.616–623. find
  18. Rosenthal, S., Farra, N. & Nakov, P., 2017. {SemEval}-2017 Task 4: Sentiment Analysis in {T}witter. In Proceedings of the 11th International Workshop on Semantic Evaluation. SemEval ’17. Vancouver, Canada: Association for Computational Linguistics. find
  19. Salamah, J. Ben & Elkhlifi, A., 2014. Microblogging Opinion Mining Approach for Kuwaiti Dialect. In The International Conference on Computing Technology and Information Management (ICCTIM2014). pp. 388–396. find
  20. Shoukry, A. & Rafea, A., 2015. A Hybrid Approach for Sentiment Classification of Egyptian Dialect Tweets. In First International Conference on Arabic Computational Linguistics (ACLing). Cairo, Egypt, pp. 78–85. find
  21. Shoukry, A. & Rafea, A., 2012. Preprocessing Egyptian Dialect Tweets for Sentiment Mining. In Proceedings of the fourth workshop on Computational Approaches to Arabic Script-Based Languages. San Diego, California, USA, pp. 47–56.
  22. Stojanovski, D. et al., 2016. Finki at SemEval-2016 Task 4 : Deep Learning Architecture for Twitter Sentiment Analysis. Proceedings of SemEval-2016, (November), pp.154–159. find
  23. Vilares, D. et al., 2016. LyS at SemEval-2016 Task 4 : Exploiting Neural Activation Values for Twitter Sentiment Classification and Quantification ∗. , pp.79–84. find
  24. Yadav, V., 2016. thecerealkiller at SemEval-2016 Task 4 : Deep Learning based System for Classifying Sentiment of Tweets on Two Point Scale. , pp.100–102. 6 795 find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/beltagy-2017-niletmrg-semeval-2017-task-4/