Elamur Research⌕
☰
home / papers / 2018 / SentiALG: Automated Corpus Annotation for Algerian Sentiment Analysis

SentiALG: Automated Corpus Annotation for Algerian Sentiment Analysis

Paper · 2018 · Imane Guellil; Ahsan Adeel; Faical Azouaou; and Amir Hussain

Summary

Algerian dialect (DALG), a Maghrebi Arabic variety, lacks labelled data for sentiment analysis. Social media users also write it in two scripts: Arabic letters and Arabizi, a romanization that mixes Latin letters and digits. The authors avoid manual labelling by building both a sentiment lexicon and an annotated corpus automatically. They translate the English SOCAL lexicon into Algerian dialect through the crowd-sourced Glosbe API. Each dialect word gets the mean score of all English words it was translated from. A rule-based scorer then labels Facebook messages. It normalizes the text, looks up n-grams and falls back to stems, flips polarity after negation markers, and assigns a label from the sign of the total. From over 15 million collected messages they sample a balanced corpus of 8000 messages. Shallow classifiers on BOW and Doc2vec features are trained on this corpus and tested on held-out automatic labels and on a manually annotated external set. The abstract reports F1 up to 72% (Arabic) and 78% (Arabizi). External results are lower. The work matters as a low-cost baseline resource for an under-resourced, two-script dialect.

Key points

  • Lexicon:
    • SOCAL has 6769 English terms scored from −5 to +5, and Glosbe translated only 3968 of them.
    • The resulting DALG lexicon has 4873 entries: 2390 in Arabic script and 2483 in Arabizi.
  • Raw data:
    • The authors chose 226 popular Algerian Facebook pages via Socialbakers and added dialect keywords from PADIC.
    • They crawled with RestFB and collected 15,407,910 messages (7,926,504 Arabic, 3,976,700 Arabizi).
  • Annotated corpus: 8000 messages, 4000 per script, each script split 2000 positive and 2000 negative.
  • Classification setup:
    • Features: BOW versus Doc2vec (PV-DM, PV-DBOW and their combination), with several shallow classifiers.
    • BOW generally beat Doc2vec.
    • Internal test sets scored higher than the external hand-labelled one, roughly 68% and 66% F1 for Arabic and Arabizi.
  • Inconsistency in the paper: the text says Arabic usually outperformed Arabizi, which sits oddly with the higher headline Arabizi F1.
  • Error sources the authors name:
    • Arabizi spelling variation (sometimes more than 100 forms of one word).
    • Lexicon noise, such as neutral function words scored negative.
    • Irregular plurals that soft stemming misses.
    • The fixed threshold of zero.

Interesting ideas

  • I1. When one target word is reached from several scored source words via translation, it receives the average of their scores, which smooths out any single noisy alignment.
  • I2. Lookup tries the longest n-gram first and only falls back to affix-stripped stems when nothing matches, with a repair rule that restores a dropped final letter in past-tense forms.
  • I3. Negation reverses the polarity of the words that follow the marker rather than only its neighbour, a crude but explicit scope rule.
  • I4. Models trained on auto-labelled data are checked against both held-out auto labels and a separate hand-labelled set, and the gap between the two exposes how much the labeller's own bias inflates scores.
  • I5. Polarity disagreements between script variants of the same word (e.g., the Arabizi fragment of the word for "god" scored negative while the Arabic form is positive) reveal lexicon noise.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (world): Value or measure entries imported through translation or synonym links (WordNet/Wikidata) could be scored by averaging over all source links, with the spread recorded as an explainable confidence.
  • I2 (tokens): The lemmatizer could use the same ordered cascade (multiword expression, then full form, then stripped stem, then repaired stem) and log which tier fired for each token.
  • I3 (frames): Negation scope could be computed over the UD subtree of the negated head instead of a linear window, and the authors' linear rule could serve as a baseline for comparison.
  • I4 (learn): Silver-trained parsers or readers should always report the silver-test versus gold-test gap as a standard negative control against circular evaluation.
  • I5 (learn): Self-induced repair rules could flag lexicon or knowledge entries whose values conflict across spelling variants or aliases of one concept, and route them for review.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of StirlingStirling · United Kingdom5 works · 2,653 citations🎓École Nationale Supérieure d'InformatiqueAlgiers · Algeria8 works · 430 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksPsychology · 1,552 worksExperimental and Cognitive Psychology · 870 works

🧭 Topics 2Research subjects and languages the work is about.Sentiment & emotion · 972 worksSemitic languages · 2,908 works

Cite

Cite as Imane Guellil; Ahsan Adeel; Faical Azouaou; and Amir Hussain. 2018. SentiALG: Automated Corpus Annotation for Algerian Sentiment Analysis. arXiv:1808.05079.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1808.05079 · arXiv 1808.05079

@misc{azouaou2018sentialg,
  title = {{SentiALG: Automated Corpus Annotation for Algerian Sentiment Analysis}},
  author = {Imane Guellil; Ahsan Adeel; Faical Azouaou; and Amir Hussain},
  year = {2018},
  doi = {10.48550/arxiv.1808.05079},
  eprint = {1808.05079},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1808.05079}
}

References

26 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abdul-Mageed, M., Diab, M.T.: Awatif: A multi-genre corpus for modern standard arabic subjectivity and sentiment analysis. In: LREC. pp. 3907–3914. Citeseer (2012)
  2. Abdulla, N., Mohammed, S., Al-Ayyoub, M., Al-Kabi, M., et al.: Automatic lexicon construction for arabic sentiment analysis. In: Future Internet of Things and Cloud (FiCloud), 2014 International Conference on. pp. 547–552. IEEE (2014) find
  3. Al-Ayyoub, M., Essa, S.B., Alsmadi, I.: Lexicon-based sentiment analysis of arabic tweets. International Journal of Social Network Mining 2(2), 101–114 (2015) find
  4. Al-Twairesh, N., Al-Khalifa, H., Al-Salman, A., Al-Ohali, Y.: Arasenti-tweet: A corpus for arabic sentiment analysis of saudi tweets. Procedia Computer Science 117, 63–72 (2017) find
  5. Alayba, A.M., Palade, V., England, M., Iqbal, R.: Arabic language sentiment analysis on health services. In: Arabic Script Analysis and Recognition (ASAR), 2017 1st International Workshop on. pp. 114–118. IEEE (2017)
  6. AlKhateeb, J.H., Jiang, J., Ren, J., Ipson, S.: Component-based segmentation of words from handwritten arabic text. International Journal of Computer Systems Science and Engineering 5(1) (2009) find
  7. AlKhateeb, J.H., Pauplin, O., Ren, J., Jiang, J.: Performance of hidden markov model and dynamic bayesian network classifiers on handwritten arabic word recognition. knowledge-based systems 24(5), 680–688 (2011) find
  8. AlKhateeb, J.H., Ren, J., Jiang, J., Al-Muhtaseb, H.: Offline handwritten arabic cursive text recognition using hidden markov models and re-ranking. Pattern Recognition Letters 32(8), 1081–1088 (2011) find
  9. Aly, M., Atiya, A.: Labr: A large scale arabic book reviews dataset. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). vol. 2, pp. 494–498 (2013)
  10. Badaro, G., Baly, R., Hajj, H., Habash, N., El-Hajj, W.: A large scale arabic sentiment lexicon for arabic opinion mining. In: Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP). pp. 165–173 (2014)
  11. Duwairi, R.M., Alfaqeh, M., Wardat, M., Alrabadi, A.: Sentiment analysis for arabizi text. In: Information and Communication Systems (ICICS), 2016 7th International Conference on. pp. 127–132. IEEE (2016) find
  12. GUELLIL, I., AZOUAOU, F.: Bilingual lexicon for algerian arabic dialect treatment in social media (2017) find
  13. Guellil, I., Azouaou, F., Abbas, M.: Comparison between neural and statistical translation after transliteration of algerian arabic dialect. In: WiNLP: Women & Underrepresented Minorities in Natural Language Processing (co-located withACL 2017). pp. 1–5 (2017) find
  14. Guellil, I., Azouaou, F., Abbas, M., Fatiha, S.: Arabizi transliteration of algerian arabic dialect into modern standard arabic. In: Social MT 2017/First workshop on Social Media and User Generated Content Machine Translation. pp. 1–8 (2017) find
  15. Guellil, I., Azouaou, F.: Arabic dialect identification with an unsupervised learning (based on a lexicon). application case: Algerian dialect. In: Computational Science and Engineering (CSE) and IEEE Intl Conference on Embedded and Ubiquitous Computing (EUC) and 15th Intl Symposium on Distributed Computing and Applications for Business Engineering (DCABES), 2016 IEEE Intl Conference on. pp. 724–731. IEEE (2016) find
  16. Guellil, I., Azouaou, F.: Asda: Analyseur syntaxique du dialecte alg {\’e} rien dans un but d’analyse s {\’e} mantique pp. 87–94 (2017) find
  17. Guellil, I., Boukhalfa, K.: Social big data mining: A survey focused on opinion mining and sentiments analysis. In: Programming and Systems (ISPS), 2015 12th International Symposium on. pp. 1–10. IEEE (2015) find
  18. Khan, A.Z., Atique, M., Thakare, V.: Combining lexicon-based and learning-based methods for twitter sentiment analysis. International Journal of Electronics, Communication and Soft Computing Science & Engineering (IJECSCSE) p. 89 (2015) find
  19. Le, Q., Mikolov, T.: Distributed representations of sentences and documents. In: International Conference on Machine Learning. pp. 1188–1196 (2014)
  20. Maas, A.L., Daly, R.E., Pham, P.T., Huang, D., Ng, A.Y., Potts, C.: Learning word vectors for sentiment analysis. In: Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies-volume 1. pp. 142–150. Association for Computational Linguistics (2011)
  21. Mataoui, M., Zelmati, O., Boumechache, M.: A proposed lexicon-based sentiment analysis approach for the vernacular algerian arabic. Research in Computing Science 110, 55–70 (2016) find
  22. Medhaffar, S., Bougares, F., Esteve, Y., Hadrich-Belguith, L.: Sentiment analysis of tunisian dialects: Linguistic ressources and experiments. In: Proceedings of the Third Arabic Natural Language Processing Workshop. pp. 55–61 (2017)
  23. Meftouh, K., Harrat, S., Jamoussi, S., Abbas, M., Smaili, K.: Machine translation experiments on padic: A parallel arabic dialect corpus. In: The 29th Pacific Asia conference on language, information and computation. pp. 1–9 (2015)
  24. Mohammad, S.M., Turney, P.D.: Crowdsourcing a word–emotion association lexicon. Computational Intelligence 29(3), 436–465 (2013) find
  25. Rushdi-Saleh, M., Martín-Valdivia, M.T., Ureña-López, L.A., Perea-Ortega, J.M.: Oca: Opinion corpus for arabic. Journal of the Association for Information Science and Technology 62(10), 2045–2054 (2011) find
  26. Taboada, M., Brooke, J., Tofiloski, M., Voll, K., Stede, M.: Lexicon-based methods for sentiment analysis. Computational linguistics 37(2), 267–307 (2011)

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/guellil-2018-sentialg-automated-corpus-annotation-algerian/