Elamur Research⌕
☰
home / papers / 2016 / NileTMRG at SemEval-2016 Task 7: Deriving Prior Polarities for Arabic Sentiment Terms

NileTMRG at SemEval-2016 Task 7: Deriving Prior Polarities for Arabic Sentiment Terms

Paper · 2016 · Samhaa R. El-Beltagy

Summary

This paper describes the NileTMRG entry to the Arabic part of SemEval-2016 Task 7, where each Arabic Twitter term must receive a score showing how strongly it signals positive sentiment. The core assumption is that a word's prior polarity can be inferred from the polarity of the tweets it usually appears in. For each term, the author gathered up to 500 tweets and removed near-duplicates. Each tweet was then labelled by an in-house Arabic sentiment classifier, and the labelled tweets were indexed in Lucene. Each term was scored with normalized PMI (nPMI) against the positive class, with neutral tweets excluded, and the scores were rescaled to the range 0–1. Terms that returned few tweets went through fallback querying, which stripped hashtags or stemmed the term, and their scores were discounted. The system placed second of three, with Kendall 0.475. The work matters because it builds an intensity lexicon for Arabic, where such resources are scarce, by reusing an existing classifier instead of relying on manual annotation.

Key points

  • About 249K tweets were collected for the 1166 test phrases. Of these phrases, 141 needed fallback processing (hashtag/underscore removal, then stemming with a light stemmer), and 15 returned no tweets at all.
  • The tweet labeller was a Complement Naïve Bayes classifier with smoothing 1. It was trained on 11,242 roughly balanced positive/negative/neutral Arabic tweets and reached 79.4% in 10-fold cross-validation. Its lexicon features came from NileULex, which contains 5953 MSA and Egyptian Arabic entries.
  • Terms with no retrieved tweets were given nPMI 0. Scores from fallback queries were multiplied by 0.75, a factor tuned on the 200-term development set.
  • Simple min-max rescaling slightly outperformed a linear regression fitted to the development gold scores, even when both were evaluated on that same development set.
  • Official results were Kendall 0.47515 and Spearman 0.65763, giving 2nd place. iLab-Edinburgh scored 0.5362 Kendall, and LSIS scored 0.42431.

Interesting ideas

  • I1. A word's prior polarity can be derived by retrieving many contexts for it, labelling each context with an existing sentence-level classifier, and measuring association with nPMI, which turns a classifier into a lexicon builder.
  • I2. Neutral contexts are removed from both the term counts and the corpus size before computing PMI, so the score reflects only the contrast between positive and negative usage.
  • I3. Rare terms get a backoff cascade (exact form, then a normalized form, then a stem), and any score obtained through backoff is explicitly discounted rather than trusted at full weight.
  • I4. If a context contains the negated form of the target term, its sentiment label is flipped before counting, so the evidence is attributed to the term itself.
  • I5. A trivial rescaling beat a regression trained on the development data even when both were evaluated on that data, which is a useful reminder to always compare against the simplest baseline.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (world): we could compile evaluative values for lemmas by counting how often their frames occur in clauses already judged positive or negative, with each score traceable back to the contexts behind it.
  • I2 (world): when building value or connotation entries for the knowledge layer, discarding non-evaluative occurrences would keep neutral usage from diluting the signal.
  • I3 (tokens): unknown-word lemma and morphology backoff could carry a visible confidence penalty, so later stages can see that a value came from a fallback.
  • I4 (frames): polarity evidence could be credited to a predicate only after checking whether it sits under a negation in the UD tree, inverting it when it does.
  • I5 (learn): every learned calibration step should be reported next to a naive baseline acting as a control, and kept only if it clearly wins.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Nile UniversityGiza · Egypt14 works · 158 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Sentiment & emotion · 972 worksSemitic languages · 2,908 works

Cite

Cite as Samhaa R. El-Beltagy. 2016. NileTMRG at SemEval-2016 Task 7: Deriving Prior Polarities for Arabic Sentiment Terms. Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/s16-1079

@inproceedings{elbeltagy2016niletmrg,
  title = {{NileTMRG at SemEval-2016 Task 7: Deriving Prior Polarities for Arabic Sentiment Terms}},
  author = {Samhaa R. El-Beltagy},
  booktitle = {Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016)},
  year = {2016},
  doi = {10.18653/v1/s16-1079},
  url = {https://doi.org/10.18653/v1/s16-1079}
}

References

20 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Apache, 2011. Lucene. Available at: http://lucene.apache.org/. ↗
  2. Astudillo, R.F. et al., 2015. INESC-ID: A Regression Model for Large Scale Twitter Sentiment Lexicon Induction. In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015). pp. 613–618. Available at: http://alt.qcri.org/semeval2015/cdrom/pdf/SemEv al102.pdf. ↗
  3. Baccianella, S., Esuli, A. & Sebastiani, F., 2010. SentiWordNet 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC’10). pp. 2200–2204.
  4. Bouma, G., 2009. Normalized (Pointwise) Mutual Information in Collocation Extraction. In the Biennial GSCL Conference. pp. 31–40. find
  5. El-Beltagy, S.R. et al., 2016. Combining Lexical Features and a Supervised Learning Approach for Arabic Sentiment Analysis. In CICLing 2016 -submitted. Konya, Turkey. find
  6. El-Beltagy, S.R., 2016. NileULex: A Phrase and Word Level Sentiment Lexicon for Egyptian and Modern Standard Arabic. In to appear in proceedings of LREC 2016. Portorož , Slovenia.
  7. El-Beltagy, S.R. & Ali, A., 2013. Open Issues in the Sentiment Analysis of Arabic Social Media : A Case Study. In Proceedings of 9th the International Conference on Innovations and Information Technology (IIT2013). Al Ain, UAE. find
  8. El-Beltagy, S.R. & Rafea, A., 2011. An Accuracy Enhanced Light Stemmer for Arabic Text. ACM Transactions on Speech and Language Processing, 7(2), pp.2 – 23. ↗
  9. Eskander, R. & Rambow, O., 2015. SLSA: A Sentiment Lexicon for Standard Arabic. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, (September), pp.2545–2550. Available at: http://aclweb.org/anthology/D15-1304. ↗
  10. Hamdan, H., Bellot, P. & Bechet, F., 2015. lsislif: Feature extraction and label weighting for sentiment analysis in twitter. In Proceedings of the 9th International Workshop on Semantic Evaluation. pp. 568–573. find
  11. Khalil, T. et al., 2015. Which configuration works best? An experimental study on Supervised Arabic Twitter Sentiment Analysis. In Proceedings of the First Conference on Arabic Computational Liguistics (ACLing 2015), co-located with CICLing 2015. Cairo, Egypt, pp. 86–93. find
  12. Kiritchenko, S., Zhu, X. & Mohammad, S., 2014. Sentiment Analysis of Short Informal Texts. Journal of Artificial Intelligence Research, 50, pp.723–762. ↗
  13. Leskovec, J., Rajaraman, A. & Ullman, J.D., 2014. Mining of Massive Datasets 2 edition., Cambridge, UK: Cambridge University Press. Available at: http://ebooks.cambridge.org/ref/id/CBO97811390 58452. ↗
  14. Liu, B., 2010. Sentiment Analysis and Subjectivity. In N. I. and F. J. Damerau, ed. Handbook of Natural Language Processing, Second Edition. find
  15. Mohammad, S. & Turney, P., 2013. Crowdsourcing a Word-Emotion Association Lexicon. Computational Intelligence, 29(3), pp.436–465. ↗
  16. Mohammad, S.M., Kiritchenko, S. & Zhu, X., 2013. NRC-Canada: Building the State-of-the-Art in Sentiment Analysis of Tweets. In Proceedings of the seventh international workshop on Semantic Evaluation Exercises (SemEval-2013). Atlanta, Georgia, USA. ↗
  17. Rennie, J.D.M. et al., 2003. Tackling the Poor Assumptions of Naive Bayes Text Classifiers. Proceedings of the Twentieth International Conference on Machine Learning (ICML)-2003), 20(1973), pp.616–623. find
  18. Rosenthal, S. et al., 2015. SemEval-2015 Task 10: Sentiment Analysis in Twitter. In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015). pp. 451–463. Available at: http://alt.qcri.org/semeval2015/cdrom/pdf/SemEv al078.pdf. ↗
  19. Twitter, 2016. Twitter Search API. Available at: https://dev.twitter.com/rest/public/search. ↗
  20. Wilson, T., Wiebe, J. & Hoffmann, P., 2005. Recognizing contextual polarity in phrase-level sentiment analysis. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP). Vancouver, Canada, pp. 347–354.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/beltagy-2016-niletmrg-semeval-2016-task-7/