Elamur Research⌕
☰
home / papers / 2018 / Des représentations continues de mots pour l'analyse d'opinions en arabe: une étude qualitative (Word embeddings for Arabic sentiment analysis : a qualitative study)

Des représentations continues de mots pour l'analyse d'opinions en arabe: une étude qualitative (Word embeddings for Arabic sentiment analysis : a qualitative study)

Paper · 2018 · Amira Barhoumi, Nathalie Camelin and Yannick Estève

Summary

The paper looks at polarity classification of Arabic book reviews with neural models. Arabic has few openly available sentiment resources, so the authors ask how far freely released word embeddings can take a system that has no expert lexicons. They build two classifiers: a Kim-style CNN with filter windows of 3, 5 and 7 plus max-pooling, and a single-layer unidirectional LSTM. Each is fed with one of several public 300-dimensional embedding sets: the CBOW vectors of Dahou et al. (2016) and the six AraVec models of Soliman et al. (2017).

Evaluation uses the official binary split of the LABR corpus. The baseline is Dahou et al.'s CNN, which the authors re-ran on that split. The CNN beats the LSTM with every embedding set, and the best pairing is CNN_Soliman_CBOW_Web. A qualitative study of errors and embedding quality suggests that generic embeddings may not encode what sentiment detection needs. This motivates two future directions: building polarity lexicons automatically and training sentiment-specific embeddings.

Key points

  • Corpus: LABR contains about 63k book reviews with star ratings. Reviews with 1–2 stars count as negative and 4–5 stars as positive; neutral reviews are dropped. This leaves 40,845 training reviews (68% positive) and 10,211 test reviews (69% positive), with 10% of the training set held out for development.
  • Corpus statistics: The data has over 3M tokens and a vocabulary of 324k. Words seen at least 5 times make up 13% of the vocabulary yet cover 86.5% of tokens.
  • Baseline: The authors re-ran Dahou et al.'s released CNN code on the official split and got 77.39% accuracy. It is the best reported result that uses no prior lexical knowledge. ElSahar & El-Beltagy score higher but depend on lexicons that are not publicly available.
  • Embeddings tested:
    • Dahou CBOW vectors trained on web text.
    • AraVec, which covers CBOW and Skip-gram, each trained on Twitter, Wikipedia and web corpora.
  • Results: The CNN outperforms the LSTM for every embedding source. The top system is CNN_Soliman_CBOW_Web.

Interesting ideas

  • I1. Re-run the strongest prior system yourself. Running a competitor's released code on the official split, instead of trusting numbers obtained on a private split, gives a baseline you can actually compare against.
  • I2. Separate baselines by the resources they use. Systems that rely on hand-built expert knowledge are kept apart from those that do not, so the comparison is fair.
  • I3. Profile vocabulary frequency before investing effort. Knowing that a small slice of the vocabulary covers most tokens shows where to concentrate work and how large the rare-word tail is.
  • I4. Check whether generic resources fit the specific task. Inspecting a general-purpose representation qualitatively can reveal that it misses the distinction a task depends on.
  • I5. Bootstrap missing lexicons by translation. The authors propose translating existing polarity resources from other languages to create prior knowledge where none exists.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Our evaluation harness should re-run competing parsers and taggers on our exact test splits before we claim any gain over them.
  • I2 (answer): School-test and QA scores could be reported separately for runs with and without hand-written knowledge, making clear what the reasoning contributes by itself.
  • I3 (tokens): A similar coverage profile over our lemma lexicon would show which entries most deserve hand checking and how many tokens fall to fallback morphology.
  • I4 (world): Before plugging WordNet- or Wikidata-derived knowledge into a reasoning step, we can spot-check whether it actually separates the categories that step needs.
  • I5 (world): Evaluative or affect lexicons available in other languages could be mapped through translation into our ontology, giving prag a polarity layer for irony and implicature cues.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Laboratoire d’Informatique de l’Université du MansLe Mans · France12 works · 2,995 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Sentiment & emotion · 972 worksSemitic languages · 2,908 works

Cite

Cite as Amira Barhoumi, Nathalie Camelin and Yannick Estève. 2018. Des représentations continues de mots pour l'analyse d'opinions en arabe: une étude qualitative (Word embeddings for Arabic sentiment analysis : a qualitative study). Actes de la Conférence TALN. Volume 1 - Articles longs, articles courts de TALN.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{barhoumi2018des,
  title = {{Des représentations continues de mots pour l'analyse d'opinions en arabe: une étude qualitative (Word embeddings for Arabic sentiment analysis : a qualitative study)}},
  author = {Amira Barhoumi, Nathalie Camelin and Yannick Estève},
  booktitle = {Actes de la Conférence TALN. Volume 1 - Articles longs, articles courts de TALN},
  year = {2018},
  url = {https://aclanthology.org/2018.jeptalnrecital-court.3/}
}

References

26 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. A BDULLA N. A., A HMED N. A., S HEHAB M. A., A L -A YYOUB M., A L -K ABI M. N. & A L - RIFAI S. (2014a). Towards improving the lexicon-based approach for arabic sentiment analysis. International Journal of Information Technology and Web Engineering (IJITWE), 9(3), 55–71. find
  2. A BDULLA N. A., A L -A YYOUB M. & A L -K ABI M. N. (2014b). An extended analytical study of arabic sentiments. International Journal of Big Data Intelligence 1, 1(1-2), 103–113. find
  3. A L -K ABI M., A L -A YYOUB M., A LSMADI I. & W AHSHEH H. (2016). A prototype for a standard arabic sentiment analysis corpus. Int. Arab J. Inf. Technol., 13(1A), 163–170. find
  4. A L -K ABI M. N., G IGIEH A. H., A LSMADI I. M., W AHSHEH H. A. & H AIDAR M. M. (2014). Opinion mining and analysis for arabic language. International Journal of Advanced Computer Science and Applications (IJACSA), SAI Publisher, 5(5), 181–195. find
  5. A LMAS Y. & A HMAD K. (2007). A note on extracting ‘sentiments’ in financial news in english, arabic & urdu. In The Second Workshop on Computational Approaches to Arabic Script-based Languages, p. 1–12. find
  6. B ARHOUMI A., E STÈVE Y., A LOULOU C. & B ELGUITH L. H. (2017). Document embeddings for arabic sentiment analysis. In Proceedings of the First Conference on Language Processing and Knowledge Management, LPKM 2017, Kerkennah (Sfax), Tunisia, September 8-10, 2017. find
  7. B AYOUDHI A., G HORBEL H. & B ELGUITH L. H. (2015). Sentiment classification of arabic documents : Experiments with multi-type features and ensemble algorithms. In Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation, p. 196–205.
  8. D AHOU A., X IONG S., Z HOU J., H ADDOUD M. H. & D UAN P. (2016). Word embeddings and convolutional neural network for arabic sentiment classification. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics : Technical Papers, p. 2418–2427.
  9. D ERIU J., L UCCHI A., D E L UCA V., S EVERYN A., M ÜLLER S., C IELIEBAK M., H OFMANN T. & J AGGI M. (2017). Leveraging large amounts of weakly supervised data for multi-language sentiment classification. International World Wide Web Conference Committee (IW3C2). find
  10. E L -H ALEES A. (2011). Arabic opinion mining using combined. Proceeding the International Arab Conference On Information Technology. find
  11. E L S AHAR H. & E L -B ELTAGY S. R. (2015). Building large arabic multi-domain resources for sentiment analysis. In International Conference on Intelligent Text Processing and Computational Linguistics, p. 23–34 : Springer. find
  12. F ARRA N., C HALLITA E., A SSI R. A. & H AJJ H. (2010). Sentence-level and document-level sentiment mining for arabic texts. In Data Mining Workshops (ICDMW), 2010 IEEE International Conference on, p. 1114–1119 : IEEE. find
  13. G AURANGI P., V ARSHA G., V EDANT K. & K ALPANA D. (2014). Sentiment analysis using support vector machine. International Journal of Innovative Research in Computer and Communication Engineering. find
  14. H ASSAN A. (2017). Sentiment analysis with recurrent neural network and unsupervised neural language model. find
  15. I BRAHIM H. S., A BDOU S. M. & G HEITH M. (2015). Sentiment analysis for modern standard arabic and colloquial. International Journal on Natural Language Computing (IJNLC), 4(2).
  16. K IM Y. (2014). Convolutional neural networks for sentence classification. arXiv preprint arXiv :1408.5882. ↗
  17. M IKOLOV T., S UTSKEVER I., C HEN K., C ORRADO G. S. & D EAN J. (2013). Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, p. 3111–3119.
  18. N ABIL M., A LY M. & A TIYA A. (2014). Labr : A large scale arabic sentiment analysis benchmark. arXiv preprint arXiv :1411.6718. ↗
  19. P ANG B., L EE L. et al. (2008). Opinion mining and sentiment analysis. Foundations and Trends R in Information Retrieval, 2(1–2), 1–135. find
  20. R EFAEE E. & R IESER V. (2016). ilab-edinburgh at semeval-2016 task 7 : A hybrid approach for determining sentiment intensity of arabic twitter phrases. Proceedings of SemEval-2016, p. 474–480.
  21. S OLIMAN A. B., E ISSA K. & E L -B ELTAGY S. R. (2017). Aravec : A set of arabic word embedding models for use in arabic nlp. Procedia Computer Science, 117, 256–265. find
  22. T RIPATHY A., A GRAWAL A. & R ATH S. K. (2015). Classification of sentimental reviews using machine learning techniques. 3rd International Conference on Recent Trends in Computing (ICRTC-2015), p. 821––829. find
  23. W ILSON T., W IEBE J. & R WA R. (2004). Just how mad are you ? finding strong and weak opinion clauses. Proceedings of the Nineteenth National Conference on Artificial Intelligence, Sixteenth Conference on Innovative Applications of Artificial Intelligence, p. 761–769. find
  24. Y U L.-C., W ANG J., L AI K. R. & Z HANG X. (2017). Refining word embeddings for sentiment analysis. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, p. 534–539. find
  25. Z AINUDDIN N. & S ELAMAT A. (2014). Sentiment analysis using support vector machine. International Conference on Computer, Communications, and Control Technology (I4CT). find
  26. Z HOU P., Q I Z., Z HENG S., X U J., B AO H. & X U B. (2016). Text classification improved by integrating bidirectional lstm with two-dimensional max pooling. arXiv preprint arXiv :1611.06639. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/barhoumi-2018-des-representations-continues-de-mots/