Elamur Research⌕
☰
home / papers / 2019 / Plongements lexicaux spécifiques à la langue arabe : application à l'analyse d'opinions (Arabic-specific embedddings : application in Sentiment Analysis)

Plongements lexicaux spécifiques à la langue arabe : application à l'analyse d'opinions (Arabic-specific embedddings : application in Sentiment Analysis)

Paper · 2019 · Amira Barhoumi, Nathalie Camelin, Chafik Aloulou et al.

Summary

The paper asks whether Arabic sentiment classification improves when word embeddings respect the language's agglutinative, morphologically rich structure. Pretrained Arabic vectors usually treat each whitespace-delimited form as one unit, even though such a form may bundle clitics, an inflected stem and pronouns. The authors merged several Arabic corpora (opinion reviews plus a large news collection) into one training set called Global. They cleaned and normalized it, then segmented it in three ways: Farasa tokens, Farasa lemmas, and light stems. For each unit type they trained a separate 300-dimensional Skip-gram word2vec space. These embeddings feed a Kim-style CNN that labels LABR book reviews as positive or negative. Input length and the padding/truncation strategy were chosen from an analysis of the corpus. All three representations reach about 91% accuracy, roughly 2 points above an 89.34% baseline that used ordinary word embeddings. Tokens score best on the dev set, while lemmas or light stems score best on test. The authors present this as the first work to build and compare Arabic embeddings over these unit types.

Key points

  • The Global training corpus combines BRAD (510k book reviews), HARD (373K reviews), the LABR training split (23K book reviews) and the AbuElKhair news corpus (5222k articles).
  • Cleaning steps:
    • removal of URLs, mentions, hashtags, numbers, punctuation and non-Arabic words;
    • deletion of kashida, with runs of a repeated character capped at two;
    • unification of alif variants;
    • removal of short vowels, soukoun and chadda.
  • The three Skip-gram spaces (dim 300) have these vocabulary sizes, against about 3M for raw words:
    • tokens: 1,980,255;
    • lemmas: 946,171;
    • light stems: 1,997,601.
  • The CNN uses filter windows {3, 4, 5}, max-pooling, a fully connected layer and a sigmoid output. Document length is set by mean + 2×std on LABR train (64 and 117.71), giving 300 words, which covers more than 96% of reviews.
  • Accuracy is about 91% for all three unit types, roughly +2% over the 89.34% baseline. The best unit differs between dev (tokens) and test (lemmas or light stems).

Interesting ideas

  • I1. Build separate embedding spaces for surface tokens, lemmas and light stems over the same corpus and compare them on one downstream task; lemmatizing roughly halves the vocabulary and raises per-unit counts.
  • I2. Set the fixed input length from training-set statistics (mean plus two standard deviations), so almost all inputs fit and the matrix stays bounded.
  • I3. Pick pre, post or symmetric padding/truncation by measuring, with a polarity lexicon, where opinion words and negations sit inside documents, rather than using a library default.
  • I4. Normalize expressive spelling (elongation characters, long runs of repeated letters capped at two, letter variants) before building a vocabulary to cut spurious hapaxes.
  • I5. Report dev and test results separately even when they disagree on the winning configuration, since that disagreement shows the differences between representations are small.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): Lexical features in the en tagger and lemmatizer could be keyed alternately on surface form, lemma and a crude stem, and compared to see which granularity reduces sparsity for rare English inflections.
  • I2 (tree): gparse could derive its maximum sentence length or batching bucket edges from the same rule applied to treebank statistics, and log the small fraction of sentences that exceed it.
  • I3 (answer): When a long tale or test passage must be shortened for a reader, the kept segment could be chosen by measuring where question-relevant cue words usually occur, and the choice recorded as an explainable decision.
  • I4 (tokens): Informal English such as "soooo" or "!!!!" could be capped at two repeats and mapped to a canonical lemma, while a flag keeps the emphasis available for the irony and speech-act layer.
  • I5 (learn): Evaluations should flag cases where the best variant changes between splits and treat such a gap as a sign that the comparison is not decisive.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Laboratoire d’Informatique de l’Université du MansLe Mans · France13 works · 2,995 citations🎓University of SfaxSfax · Tunisia17 works · 233 citations🔬Laboratoire Informatique d'AvignonAvignon · France8 works · 436 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 2Research subjects and languages the work is about.Sentiment & emotion · 1,035 worksSemitic languages · 3,066 works

Cite

Cite as Amira Barhoumi, Nathalie Camelin, Chafik Aloulou, Yannick Estève and Lamia Hadrich Belguith. 2019. Plongements lexicaux spécifiques à la langue arabe : application à l'analyse d'opinions (Arabic-specific embedddings : application in Sentiment Analysis). Actes de la Conférence sur le Traitement Automatique des Langues Naturelles (TALN) PFIA 2019. Volume II : Articles courts.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{barhoumi2019plongements,
  title = {{Plongements lexicaux spécifiques à la langue arabe : application à l'analyse d'opinions (Arabic-specific embedddings : application in Sentiment Analysis)}},
  author = {Amira Barhoumi, Nathalie Camelin, Chafik Aloulou, Yannick Estève and Lamia Hadrich Belguith},
  booktitle = {Actes de la Conférence sur le Traitement Automatique des Langues Naturelles (TALN) PFIA 2019. Volume II : Articles courts},
  year = {2019},
  url = {https://aclanthology.org/2019.jeptalnrecital-court.24/}
}

References

23 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. A BDULLA N. A., A HMED N. A., S HEHAB M. A., A L -A YYOUB M., A L -K ABI M. N. & A L -RIFAI S. (2014a). Towards improving the lexicon-based approach for arabic sentiment analysis. International Journal of Information Technology and Web Engineering (IJITWE), 9(3), 55–71. find
  2. A BDULLA N. A., A L -A YYOUB M. & A L -K ABI M. N. (2014b). An extended analytical study of arabic sentiments. International Journal of Big Data Intelligence 1, 1(1-2), 103–113. find
  3. A L S ALLAB A., H AJJ H., B ADARO G., B ALY R., E L H AJJ W. & S HABAN K. B. (2015). Deep learning models for sentiment analysis in arabic. In Proceedings of the Second Workshop on Arabic Natural Language Processing, p. 9–17.
  4. A LMAS Y. & A HMAD K. (2007). A note on extracting ‘sentiments’ in financial news in english, arabic & urdu. In The Second Workshop on Computational Approaches to Arabic Script-based Languages, p. 1–12. find
  5. B ARHOUMI A., C AMELIN N. & E STÈVE Y. (2018). Des représentations continues de mots pour l’analyse d’opinions en arabe : une étude qualitative. In Actes de la conférence Traitement Automatique de la Langue Naturelle, TALN 2018, p. 215. find
  6. B AYOUDHI A., G HORBEL H. & B ELGUITH L. H. (2015). Sentiment classification of arabic documents : Experiments with multi-type features and ensemble algorithms. In Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation, p. 196–205.
  7. D AHOU A., X IONG S., Z HOU J., H ADDOUD M. H. & D UAN P. (2016). Word embeddings and convolutional neural network for arabic sentiment classification. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics : Technical Papers, p. 2418–2427.
  8. E L -H ALEES A. (2011). Arabic opinion mining using combined. Proceeding the International Arab Conference On Information Technology. find
  9. E L -K HAIR I. A. (2016). 1.5 billion words arabic corpus. arXiv preprint arXiv :1611.04033. ↗
  10. E LNAGAR A. & E INEA O. (2016). Brad 1.0 : Book reviews in arabic dataset. In Computer Systems and Applications (AICCSA), 2016 IEEE/ACS 13th International Conference of, p. 1–8 : IEEE. find
  11. E LNAGAR A., K HALIFA Y. S. & E INEA A. (2018). Hotel arabic-reviews dataset construction for sentiment analysis applications. In Intelligent Natural Language Processing : Trends and Applications, p. 35–52. Springer. find
  12. F ARRA N., C HALLITA E., A SSI R. A. & H AJJ H. (2010). Sentence-level and document-level sentiment mining for arabic texts. In Data Mining Workshops (ICDMW), 2010 IEEE International Conference on, p. 1114–1119 : IEEE. find
  13. G RAVE E., B OJANOWSKI P., G UPTA P., J OULIN A. & M IKOLOV T. (2018). Learning word vectors for 157 languages. In Proceedings of the International Conference on Language Resources and Evaluation (LREC 2018).
  14. I BRAHIM H. S., A BDOU S. M. & G HEITH M. (2015). Sentiment analysis for modern standard arabic and colloquial. International Journal on Natural Language Computing (IJNLC), 4(2).
  15. K IM Y. (2014). Convolutional neural networks for sentence classification. arXiv :1408.5882. arXiv preprint M IKOLOV T., S UTSKEVER I., C HEN K., C ORRADO G. S. & D EAN J. (2013). Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, p. 3111–3119. ↗
  16. N ABIL M., A LY M. & A TIYA A. (2014). Labr : A large scale arabic sentiment analysis benchmark. arXiv preprint arXiv :1411.6718. ↗
  17. P ANG B., L EE L. et al. (2008). Opinion mining and sentiment analysis. Foundations and Trends R in Information Retrieval, 2(1–2), 1–135. find
  18. P ENNINGTON J., S OCHER R. & M ANNING C. (2014). Glove : Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), p. 1532–1543.
  19. P ETERS M. E., N EUMANN M., I YYER M., G ARDNER M., C LARK C., L EE K. & Z ETTLEMOYER L. (2018). Deep contextualized word representations. In Proc. of NAACL.
  20. R EFAEE E. & R IESER V. (2016). ilab-edinburgh at semeval-2016 task 7 : A hybrid approach for determining sentiment intensity of arabic twitter phrases. Proceedings of SemEval-2016, p. 474–480.
  21. S OLIMAN A. B., E ISSA K. & E L -B ELTAGY S. R. (2017). Aravec : A set of arabic word embedding models for use in arabic nlp. Procedia Computer Science, 117, 256–265. find
  22. Y ANG K., C AI Y., H UANG D., L I J., Z HOU Z. & L EI X. (2017). An effective hybrid model for opinion mining and sentiment analysis. In Big Data and Smart Computing (BigComp), 2017 IEEE International Conference on, p. 465–466 : IEEE. find
  23. Y U L.-C., W ANG J., L AI K. R. & Z HANG X. (2017). Refining word embeddings for sentiment analysis. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, p. 534–539. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/barhoumi-2019-plongements-lexicaux-specifiques-la-langue/