Elamur Research⌕
☰
home / papers / 2015 / Idioms-Proverbs Lexicon for Modern Standard Arabic and Colloquial Sentiment Analysis

Idioms-Proverbs Lexicon for Modern Standard Arabic and Colloquial Sentiment Analysis

Paper · 2015 · Hossam S. Ibrahim, Sherif M. Abdou and Mervat Gheith

Summary

Sentiment classifiers for Arabic often miss opinions carried by idioms and proverbs. These fixed expressions mean something different from their words taken one by one, and they frequently contain no explicit sentiment words at all. As a result, a word-level system may label a strongly negative comment as neutral. The authors address this for Modern Standard Arabic (MSA) and Egyptian colloquial by building AIPSeLEX, a sentiment lexicon of idioms and proverbs. They gathered a large pool of expressions from specialized books and websites and kept those that are both opinion-bearing and in common use. Native-speaker annotators then labeled each kept expression as positive or negative. On top of the lexicon, they built an extractor that locates these phrases in running text using n-grams and similarity measures. They tested it on tweets and on comments about hotels, products and TV programs to assess coverage and accuracy. The resource matters because figurative phrases, by the authors' count, make up a sizeable share of Arabic online opinions, and the annotation work only has to be done once for the whole community.

Key points

  • About 32,785 idioms and proverbs were gathered between January and September 2014. Of these, 3,632 frequent, sentiment-bearing ones were kept and labeled positive (PO) or negative (NG).
  • Three native-speaker raters with Arabic linguistic expertise annotated the entries. Disagreements went to majority vote and then discussion until full consensus. Observed agreement was 98%, and Cohen's Kappa was also reported.
  • The test data came from Twitter (via the API, filtered to lang=ar) and from sites such as booking.com, fatakat forums and ejabat.google.com.
    • Two native speakers picked roughly 3,600 items that met explicit criteria: subjective, a single opinion, MSA or Egyptian dialect, no sarcasm, no insults, and containing an idiom or proverb.
  • Preprocessing had three steps:
    • normalizing Arabic letter variants and stripping diacritics;
    • removing symbols and numbers;
    • collapsing elongations by detecting repeated single-character and two-character patterns.
  • In the authors' data, figurative expressions (sarcasm, sayings, idioms) accounted for over 20% of opinions. Sarcastic items were deliberately left out as too hard to handle.

Interesting ideas

  • I1. Store non-compositional expressions as whole lexicon entries with their own polarity, so they override word-by-word interpretation of the span.
  • I2. Find lexicon phrases in text by matching n-gram windows with a similarity score rather than exact strings, which tolerates small surface variation.
  • I3. Collapse character elongation by detecting repeated one- or two-character patterns, which handles both "gamiiiil"-style stretching and "hahaha"-style repetition.
  • I4. Resolve label disagreements with a majority vote first, then discuss the remaining cases until there is full agreement, and report a chance-corrected agreement statistic.
  • I5. Build the evaluation set with explicit inclusion and exclusion criteria (one opinion, no sarcasm, idiom present) so it measures exactly the phenomenon under study.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): A multiword-idiom table could mark spans like "cost an arm and a leg" as a single evaluative predicate, so prag does not build literal who-did-what frames from them.
  • I2 (tokens): An idiom matcher over lemmas with an edit-distance threshold could catch inflected or slightly altered idioms ("pinched pennies", "pinching every penny") before parsing.
  • I3 (tokens): The same rule could normalize informal English ("sooooo", "lolol") before tagging and lemmatization, cutting down unknown-word noise.
  • I4 (learn): Silver data from several witness parsers could use the same flow: take the majority arc, send residual conflicts to an LLM teacher tier, and report kappa-style agreement between witnesses.
  • I5 (answer): Fable and test QA sets could have targeted slices, such as items whose answer depends on understanding an idiom, so figurative-language failures show up separately from other errors.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Cairo UniversityGiza · Egypt42 works · 1,350 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 3Research subjects and languages the work is about.Sentiment & emotion · 972 worksSemitic languages · 2,908 worksIdioms & multiword expressions · 217 works

Cite

Cite as Hossam S. Ibrahim, Sherif M. Abdou and Mervat Gheith. 2015. Idioms-Proverbs Lexicon for Modern Standard Arabic and Colloquial Sentiment Analysis. arXiv:1506.01906 · International Journal of Computer Applications 118(11):26-31, May 2015.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.5120/20790-3435 · arXiv 1506.01906

@misc{ibrahim2015idioms,
  title = {{Idioms-Proverbs Lexicon for Modern Standard Arabic and Colloquial Sentiment Analysis}},
  author = {Hossam S. Ibrahim, Sherif M. Abdou and Mervat Gheith},
  year = {2015},
  doi = {10.5120/20790-3435},
  eprint = {1506.01906},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.5120/20790-3435}
}

References

30 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. X. Song and W. Ting, "Construction of unsupervised sentiment classifier on idioms resources," Journal of Central South University, vol. 21, pp. 1376-1384, 2014. find
  2. V. Hatzivassiloglou and K. R. McKeown, "Predicting the semantic orientation of adjectives," in Proceedings of the Joint ACL / EACL Conference, 1997, pp. 174–181. find
  3. P. Turney, "Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews," in Proceedings of the 40th Annual Meeting on Association for Computational Linguistics ACL '02, Stroudsburg, PA, USA, 2002, pp. 417-424. ↗
  4. B. Pang, L. Lee, and S. Vaithyanathan, "Thumbs up? Sentiment classification using machine learning techniques," in Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 2002, pp. 79–86. ↗
  5. M. Hu and B. Liu, "Mining and summarizing customer reviews " in Proceedings of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2004, pp. 168–177. find
  6. P. Alexander and P. Patrick, "Twitter as a Corpus for Sentiment Analysis and Opinion Mining " in Proceedings International Journal of Computer Applications (0975 – 8887) Volume 118 – No. 11, May 2015 of the Seventh conference on International Language Resources and Evaluation (LREC'10), European Language Resources Association ELRA, Valletta, Malta, 2010.
  7. C. Scheible and H. Schütze, "Bootstrapping Sentiment Labels For Unannotated Documents With Polarity PageRank," in Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC 2012), Istambol-Turki, 2012. find
  8. D. Davidiv, O. Tsur, and A. Rappoport, "Enhanced Sentiment Learning Using Twitter Hash-tags and Smileys," in Proceedings of the 23rd International Conference on Computational Linguistics (Coling2010), Beijing, China, 2010, pp. 241–249. find
  9. L. Barbosa and J. Feng, "Robust Sentiment Detection on Twitter from Biased and Noisy Data " in Proceedings of the 23rd International Conference on Computational Linguistics (Coling), 2010.
  10. M. Abdul-Mageed and M. Diab, "Subjectivity and Sentiment Annotation of Modern Standard Arabic Newswire," in Proceedings of the Fifth Law Workshop (LAW V), Association for Computational Linguistics, Portland, Oregon, 2011, pp. 110–118.
  11. M. Abdul-Mageed, M. Diab, and M. Korayem, "Subjectivity and sentiment analysis of modern standard Arabic," in Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, 2011.
  12. A. Shoukry and A. Rafea, "Sentence-level Arabic sentiment analysis," in Collaboration Technologies and Systems (CTS) International Conference, Denver, CO, USA, 2012, pp. 546-550. find
  13. A. Mourad and K. Darwish, "Subjectivity and Sentiment Analysis of Modern Standard Arabic and Arabic Microblogs," in Proceedings of the 4th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (WASSA), Atlanta, Georgia, 2013, pp. 55–64.
  14. E. Refaee and V. Rieser, "An Arabic Twitter Corpus for Subjectivity and Sentiment Analysis," in Proceedings of The 9th edition of the Language Resources and Evaluation Conference (LREC 2014), Reykjavik, Iceland, 2014.
  15. A. S. Zibin and A. R. M. Altakhaineh, "Informativity of Arabic Proverbs in Context: An Insight into Palestinian Discourse," International Journal of Linguistics (IJL), vol. 6, p. 67, 2014. ↗
  16. Z. Kovecses, Metaphor:A Practical Introduction, First ed.: Oxford University Press, 2002. find
  17. H. S. Ibrahim, S. M. Abdou, and M. Gheith, "Automatic expandable large-scale sentiment lexicon of Modern Standard Arabic and Colloquial," in 16th International Conference on Intelligent Text Processing and Computational Linguistics (CICLING), Cairo - Egypt, 2015. find
  18. O. F. Zaidan and C. Callison-Burch, "Arabic dialect identification," Computational Linguistics, vol. 40, pp. 171-202, March 2014 2012. ↗
  19. M. Elmahdy, G. Rainer, M. Wolfgang, and A. Slim, "Survey on common Arabic language forms from a speech recognition point of view," in proceeding of International conference on Acoustics (NAG-DAGA), Rotterdam, Netherlands, 2009, pp. 63-66. find
  20. B. Liu, Sentiment Analysis and Opinion Mining Morgan &Claypool Publishers, 2012. find
  21. A. Basha, لٔلاا فرحنا ةسح حثذرئ ححٔرشي : حًٛناعنا لاثيلاا ٗعٕضٕي فاشك عي مثًنا ٍي [Colloquial sayings: an annotated and arranged by the first letter of ideals with the Scout TOPICAL]. Egypt: Al-Ahram Foundation - Al-Ahram Center for Translation and Publishing, 1986. find
  22. A. Saalan, حٚرصًنا حٛثعشنا لاثيلاا حعٕسٕي [Encyclopedia of Egyptian popular sayings], First ed. Egypt: Dar-alafkalarabia press, 2003. find
  23. F. Husain, , حٛياعنا لاثيلاا , حٛثعشنا صصقنا , حٛترعنا ردإُنا ٖرصًنا رٕهكنٕفنا [Colloquial sayings, Egyptian folklore]. Egypt: General Egyptian Book Organization GEBO, 1984. find
  24. G. Taher. (2006). حًٛهع حسارد - حٛثعشنا لاثيلأا حعٕسٕي [Encyclopedia of public sayings - a scientific study]. Available: http://books.google.com.eg/books?id=2CR\_EKTjxRgC IJCA TM : www.ijcaonline.org International Journal of Computer Applications (0975 – 8887) Volume 118 – No. 11, May 2015 ↗
  25. PROz. (2014). PROz website for Arabic Idioms/Maxims/Sayings (Jan 2014). Available: http://www.proz.com/glossary-translations/ ↗
  26. J. C. Carletta, "Assessing agreement on classification tasks: the KAPPA statistic " Computational Linguistics, vol. 22, pp. 249- 254, 1996. ↗
  27. G. Salton, A. Wong, and C. S. Yang, "A vector space model for information retrieval," Communications of the ACM, vol. 11, pp. 613–620, November 1975. find
  28. V. Levenshtein, "Binary codes capable of correcting deletions, insertions and reversals," Soviet Physics Doklady, 10(8):707-710, 1966. Original in Russian in Doklady Akademii Nauk SSSR, vol. 163, pp. 845-848, 1965. find
  29. X. Ding, B. Liu, and P. S. Yu., "A holistic lexicon-based approach to opinion mining," in Proceeding WSDM '08 Proceedings of the 2008 International Conference on Web Search and Data Mining, New York, NY, USA, 2008, pp. 231-240. find
  30. A. Mudinas, D. Zhang, and M. Levene, "Combining lexicon and learning based approaches for concept-level sentiment analysis," in Proceeding WISDOM '12 Proceedings of the First International Workshop on Issues of Sentiment Discovery and Opinion Mining Article, New York, NY, USA, 2012. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ibrahim-2015-idioms-proverbs-lexicon-modern-standard/