Elamur Research⌕
☰
home / papers / 2011 / Subjectivity and Sentiment Annotation of Modern Standard Arabic Newswire

Subjectivity and Sentiment Annotation of Modern Standard Arabic Newswire

Paper · 2011 · Muhammad Abdul-Mageed and Mona Diab

Summary

Research on subjectivity and sentiment analysis (SSA) has mostly focused on English and on opinion-heavy genres such as reviews and blogs. Morphologically-rich languages (MRL) and news text have received little attention, largely because annotated data was missing. The authors address this for Modern Standard Arabic (MSA) by adding a sentence-level annotation layer to newswire that was already part of the Penn Arabic TreeBank (PATB).

Two native-speaker annotators gave each sentence one of four labels: objective, or subjective with positive, negative or neutral polarity. They also assigned a news domain label. The guidelines rest on linguistic ideas:

  • Good or bad news is not automatically sentiment.
  • Sentiment is judged from the author's perspective.
  • Hedges and boosters can make a sentence subjective or change its strength.
  • Speech acts such as praise or banning carry attitude.
  • Annotators must set aside their own beliefs.

Agreement was high in raw terms but modest by Kappa. The authors describe the result as the first SSA resource for MSA newswire and promise to release it.

Key points

  • The corpus has 2855 sentences from PATB Part 1 v3.0. They come from its first 400 documents, about 54.5% of that part.
  • The sentence labels are OBJ, S-POS, S-NEG and S-NEUT. The rare mixed-polarity sentences (under 0.005%) were folded into S-NEUT.
  • SSA inter-annotator agreement was 88.06% raw, but Kappa was only 0.38.
  • Nine news domains were labeled, including Politics, Sports, Military & political violence, Economy, Crime and This day in history. Domain agreement was 97%, and the annotators settled all disagreements by discussion.
  • The authors contrast their work with fine-grained English news annotation (Wiebe et al., 2005) and with Abbasi et al. (2008). The latter covers Arabic web forums at document level and is not publicly available.

Interesting ideas

  • I1. Reporting a bad event (deaths, accidents) is kept separate from expressing negative sentiment, so plainly reported bad news stays objective.
  • I2. Hedges ("may", "it seems") and boosters ("definitely") are treated as operators: they turn a factual claim into a subjective one, or strengthen or weaken its polarity.
  • I3. When a sentence quotes someone, the quoted speaker's attitude (e.g., the minister being "glad") can decide the label, so one sentence can hold several attitude holders.
  • I4. Sentiment is labeled from the author's perspective rather than the reader's, and annotators are told to suppress their own background beliefs.
  • I5. High raw agreement (88%) alongside a low Kappa (0.38) shows that raw agreement can mislead on skewed label sets.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): Keep the polarity of an event in the world model separate from the attitude a speaker shows toward it, so a fable's sad event isn't mistaken for the narrator's judgement.
  • I2 (world): Store a certainty value on each asserted state, driven by modal and adverbial cues in the UD tree, so answers can tell "X happened" apart from "X may have happened".
  • I3 (frames): Attach sentiment and speech-act frames to the holder in the reporting clause, not to the text as a whole, so who-feels-what stays explicit.
  • I4 (learn): When an LLM teacher produces silver sentiment or stance labels, spell out this perspective rule in the instructions and test it with negative controls (ideologically flipped sentences) to catch reader bias.
  • I5 (learn): When evaluating agreement between witness parsers or teachers on imbalanced labels, report chance-corrected scores next to raw agreement.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Columbia UniversityNew York · United States241 works · 54,053 citations🎓Indiana University BloomingtonBloomington · United States37 works · 3,685 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksExperimental and Cognitive Psychology · 870 works

🧭 Topics 2Research subjects and languages the work is about.Sentiment & emotion · 972 worksSemitic languages · 2,908 works

Cite

Cite as Muhammad Abdul-Mageed and Mona Diab. 2011. Subjectivity and Sentiment Annotation of Modern Standard Arabic Newswire. Proceedings of the 5th Linguistic Annotation Workshop.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{abdulmageed2011subjectivity,
  title = {{Subjectivity and Sentiment Annotation of Modern Standard Arabic Newswire}},
  author = {Muhammad Abdul-Mageed and Mona Diab},
  booktitle = {Proceedings of the 5th Linguistic Annotation Workshop},
  year = {2011},
  url = {https://aclanthology.org/W11-0413/}
}

References

26 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. A. Abbasi, H. Chen, and A. Salem. 2008. Sentiment analysis in multiple languages: Feature selection for opinion classification in web forums. ACM Trans. Inf. Syst., 26:1–34. find
  2. M. Abdul-Mageed. 2008. Online News Sites and Journalism 2.0: Reader Comments on Al Jazeera Arabic. tripleC-Cognition, Communication, Cooperation, 6(2):59. find
  3. K. Bach and R.M. Harnish. 1979. Linguistic communication and speech acts. find
  4. A. Balahur and R. Steinberger. 2009. Rethinking Sentiment Analysis in the News: from Theory to Practice and back. Proceeding of WOMSA. find
  5. A. Balahur, R. Steinberger, E. van der Goot, B. Pouliquen, and M. Kabadjov. 2009. Opinion mining on newspaper quotations. In 2009 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology, pages 523–526. IEEE. find
  6. A. Banfield. 1982. Unspeakable Sentences: Narration and Representation in the Language of Fiction. Routledge Kegan Paul, Boston. find
  7. L. Chouliaraki and N. Fairclough. 1999. Discourse in late modernity: Rethinking critical discourse analysis. Edinburgh Univ Pr. find
  8. M. Diab, K. Hacioglu, and D. Jurafsky. 2007. Automatic processing of Modern Standard Arabic text. Arabic Computational Morphology, pages 159–179. ↗
  9. M.T. Diab, L. Levin, T. Mitamura, O. Rambow, V. Prabhakaran, and W. Guo. 2009. Committed belief annotation and tagging. In Proceedings of the Third Linguistic Annotation Workshop, pages 68–73. Association for Computational Linguistics.
  10. R. Fowler. 1991. Language in the News: Discourse and Ideology in the Press. Routledge. find
  11. N. Habash, O. Rambow, and R. Roth. 2009. Mada+tokan: A toolkit for arabic tokenization, diacritization, morphological disambiguation, pos tagging, stemming and lemmatization. In Proceedings of the 2nd International Conference on Arabic Language Resources and Tools (MEDAR), Cairo, Egypt. find
  12. S. Kim and E. Hovy. 2004. Determining the sentiment of opinions. In Proceedings of the 20th International Conference on Computational Linguistics, pages 1367–1373.
  13. W.H. Lin, T. Wilson, J. Wiebe, and A. Hauptmann. 2006. Which side are you on?: identifying perspectives at the document and sentence levels. In Proceedings of the Tenth Conference on Computational Natural Language Learning, pages 109–116. Association for Computational Linguistics. find
  14. M. Maamouri, A. Bies, T. Buckwalter, and W. Mekki. 2004. The penn arabic treebank: Building a large-scale annotated arabic corpus. In NEMLAR Conference on Arabic Language Resources and Tools, pages 102–109. find
  15. F. Palmer. 1986. Mood and Modality. 1986. Cambridge: Cambridge University Press. find
  16. L. Polanyi and A. Zaenen. 2006. Contextual valence shifters. Computing attitude and affect in text: Theory and applications, pages 1–10. ↗
  17. R. Quirk, S. Greenbaum, R.A. Close, and R. Quirk. 1974. A university grammar of English, volume 1985. Longman. find
  18. J.R. Searle. 1975. A taxonomy of speech acts. In K. Gunderson, editor, Language, mind, and knowledge, pages 344–369. Minneapolis: University of Minnesota Press. find
  19. S. Somasundaran, J. Ruppenhofer, and J. Wiebe. 2007. Detecting arguing and sentiment in meetings. In Proceedings of the SIGdial Workshop on Discourse and Dialogue, volume 6. Citeseer. find
  20. H. Tanev. 2007. Unsupervised learning of social networks from a multiple-source news corpus. MuLTI-SOuRcE, MuLTILINguAL INfORMATION ExTRAc-TION ANd SuMMARIzATION, page 33. find
  21. R. Tsarfaty, D. Seddah, Y. Goldberg, S. Kuebler, Y. Versley, M. Candito, J. Foster, I. Rehbein, and L. Tounsi. 2010. Statistical parsing of morphologically rich languages (spmrl) what, how and whither. In Proceedings of the NAACL HLT 2010 First Workshop on Statistical Parsing of Morphologically-Rich Languages, Los Angeles, CA.
  22. T.A. Van Dijk. 1988. News as discourse. Lawrence Erlbaum Associates. find
  23. J. Wiebe, R. Bruce, and T. O’Hara. 1999. Development and use of a gold standard data set for subjectivity classifications. In Proc. 37th Annual Meeting of the Assoc. for Computational Linguistics (ACL-99), pages 246– 253, University of Maryland: ACL.
  24. J. Wiebe, T. Wilson, and C. Cardie. 2005. Annotating expressions of opinions and emotions in language. Language Resources and Evaluation, 39(2):165–210. ↗
  25. J. Wiebe. 1994. Tracking point of view in narrative. Computional Linguistics, 20(2):233–287. ↗
  26. R. Wodak and M. Meyer. 2009. Critical discourse analysis: History, agenda, theory and methodology. Methods of critical discourse analysis, pages 1–33. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mageed-2011-subjectivity-sentiment-annotation-modern-standard/