Elamur Research⌕
☰
home / papers / 2020 / A Sentiment-annotated Dataset of English Causal Connectives

A Sentiment-annotated Dataset of English Causal Connectives

Paper · 2020 · Marta Andersson, Murathan Kurfalı and Robert Östling

Summary

The paper asks whether three near-synonymous English causal connectives, because of, due to and owing to, have a semantic prosody. Semantic prosody here means a habitual tendency to appear with pleasant or unpleasant content, even though dictionary senses suggest the connectives are neutral. The authors sampled instances from the web-based GloWbE corpus across seven English varieties and annotated them with PDTB Annotator. Each connective's two arguments, the effect and the cause, received a separate polarity label. Annotators marked the smallest span that carries the evaluative meaning.

All three connectives lean clearly negative. Differences between connectives, between varieties and between cause and effect turned out small or absent, which went against the authors' hypotheses.

They also fine-tuned BERT to predict argument polarity. It reached only moderate agreement with the human annotators. The authors still see this as promising given the small amount of training data.

The resulting resource ties discourse relations to sentiment. It suggests that connective choice carries evaluative signal that sentiment systems could use.

Key points

  • Data comes from GloWbE (about 1.9 billion words from 20 countries), with blogs excluded. Each hit got 250 words of context on either side, and hits with gaps or repeated connectives were dropped. The dataset has 1405 annotated relations.
  • Negative shares across varieties were 64–80% for due to, 48–72% for because of and 54–69% for owing to.
  • For due to, variety made no significant difference (p = 0.26). Contrary to expectation, the major varieties were slightly less negative than the Asian ones (70% vs 80%, p = 0.04).
  • Arguments with opposite polarity were rare: 30 negative/positive (2.1%) and 28 positive/negative (2.0%). The two orders were roughly balanced, which argues against the connective binding more to one argument.
  • The model was bert-large-cased via Huggingface Transformers, trained on annotated spans only. US English because of was held out as the test set. Cohen's Kappa against annotators ranged from 0.235 to 0.529.

Interesting ideas

  • I1. Label polarity separately for the cause and the effect of a causal relation, rather than giving one score to the whole sentence; mixed cases then become visible and countable.
  • I2. Annotators mark the minimal span that is both necessary and sufficient to fix the polarity, which gives an explainable anchor instead of a holistic judgement.
  • I3. Before measuring a word's contextual bias, first confirm with a dictionary that its core sense is neutral, because inherently polar words override their context.
  • I4. Fine-tuning is unstable, so the authors train several runs and report the one with median development score rather than the best.
  • I5. The model is scored against each human annotator with Kappa, and its confusion pattern is compared with human disagreements (the model under-uses the positive label).

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): a causal frame could carry a polarity slot on each argument, so the explanation of a "because of" relation shows which side is bad or good.
  • I2 (frames): evaluative cues could be attached to specific UD subtrees, so every sentiment claim points to the exact phrase that justifies it.
  • I3 (world): the compiled lexicon could store a "prior prosody" per connective or verb (e.g. cause and due to lean negative) as a soft default for inferring how a text evaluates events.
  • I4 (learn): adopt median-of-runs reporting for gparse and teacher-tier experiments, so results are not inflated by a lucky seed.
  • I5 (learn): treat a trained component as one more witness and inspect how it disagrees, not just how often, to find systematic biases worth repair rules.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Stockholm UniversityStockholm · Sweden21 works · 1,832 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksLiterature and Literary Theory · 541 works

🧭 Topics 2Research subjects and languages the work is about.English · 2,379 worksSentiment & emotion · 1,035 works

Cite

Cite as Marta Andersson, Murathan Kurfalı and Robert Östling. 2020. A Sentiment-annotated Dataset of English Causal Connectives. Proceedings of the 14th Linguistic Annotation Workshop.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{andersson2020sentiment,
  title = {{A Sentiment-annotated Dataset of English Causal Connectives}},
  author = {Marta Andersson, Murathan Kurfalı and Robert Östling},
  booktitle = {Proceedings of the 14th Linguistic Annotation Workshop},
  year = {2020},
  doi = {10.48448/sq70-ha05},
  url = {https://doi.org/10.48448/sq70-ha05}
}

References

30 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Marta Andersson. 2019. Subjectivity of english connectives. Empirical Studies of the Construction of Discourse, 305:299. ↗
  2. Farah Benamara, Maite Taboada, and Yannick Mathieu. 2017. Evaluative language beyond bags of words: Linguistic insights and computational applications. Computational Linguistics, 43(1):201–264. ↗
  3. Diane Blakemore and Robyn Carston. 2005. The pragmatics of sentential coordination with and. Lingua, 115(4):569–589. find
  4. Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and psychological measurement, 20(1):37–46. ↗
  5. Mark Davies. 2013. Corpus of global web-based english: 1.9 billion words from speakers in 20 countries (glowbe). Available online atcorpus. byu. edu/glowbe/. RetrievedDecember, 15:2015. find
  6. Rahim Dehkharghani, Hanefi Mercan, Arsalan Javeed, and Yucel Saygin. 2014. Sentimental causal rule discovery from twitter. Expert Systems with Applications, 41(10):4950–4958. ↗
  7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. ↗
  8. David J Hauser and Norbert Schwarz. 2016. Semantic prosody and judgment. Journal of Experimental Psychology: General, 145(7):882. ↗
  9. Judith Kamalski, Leo Lentz, Ted Sanders, and Rolf A Zwaan. 2008. The forewarning effect of coherence markers in persuasive discourse: Evidence from persuasion and processing. Discourse Processes, 45(6):545–579. ↗
  10. Dongyeop Kang, Varun Gangal, Ang Lu, Zheng Chen, and Eduard Hovy. 2017. Detecting and explaining causes from text for a time series event. arXiv preprint arXiv:1707.08852. ↗
  11. Alan Lee, Rashmi Prasad, Bonnie Webber, and Aravind K Joshi. 2016. Annotating discourse relations with the pdtb annotator. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: System Demonstrations, pages 121–125. find
  12. Yen-Yu Lin and Siaw-Fong Chung. 2016. A corpus-based study on the semantic prosody of challenge. Taiwan Journal of TESOL, 13(2):99–146. find
  13. Mary L McHugh. 2012. Interrater reliability: the kappa statistic. Biochemia medica: Biochemia medica, 22(3):276–282. ↗
  14. Saif Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016. Semeval-2016 task 6: Detecting stance in tweets. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 31–41.
  15. Saif Mohammad. 2016. A practical guide to sentiment annotation: Challenges and solutions. In Proceedings of the 7th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, pages 174–179. find
  16. Subhabrata Mukherjee and Pushpak Bhattacharyya. 2012. Sentiment analysis in twitter with lightweight discourse analysis. In Proceedings of COLING 2012, pages 1847–1864. find
  17. Alan Partington. 2004. ” utterly content in each other’s company” semantic prosody and semantic preference. International journal of corpus linguistics, 9(1). find
  18. Rashmi Prasad, Eleni Miltsakaki, Nikhil Dinesh, Alan Lee, Aravind Joshi, Livio Robaldo, and Bonnie L Webber. 2007. The penn discourse treebank 2.0 annotation manual. ↗
  19. Peter G Preethi, Vilma Uma, et al. 2015. Temporal sentiment analysis and causal rules extraction from tweets for event prediction. Procedia computer science, 48:84–89. find
  20. Sara Rosenthal, Preslav Nakov, Svetlana Kiritchenko, Saif Mohammad, Alan Ritter, and Veselin Stoyanov. 2015. Semeval-2015 task 10: Sentiment analysis in twitter. In Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015), pages 451–463.
  21. Merel Scholman and Vera Demberg. 2017. Crowdsourcing discourse interpretations: On the influence of context and the reliability of a connective insertion task. In Proceedings of the 11th Linguistic Annotation Workshop, pages 24–33. ↗
  22. Antonios Siganos, Evangelos Vagenas-Nanos, and Patrick Verwijmeren. 2014. Facebook’s daily sentiment and international stock markets. Journal of Economic Behavior & Organization, 107:730–743. ↗
  23. John Sinclair. 1998. The lexical item. Amsterdam Studies In The Theory And History Of Linguistic Science Series 4, pages 1–24. ↗
  24. Michael Stubbs. 1995. Collocations and semantic profiles: On the cause of the trouble with quantitative studies. Functions of language, 2(1):23–55. find
  25. Michael Stubbs. 2001. Words and phrases: Corpus studies of lexical semantics. Blackwell Publishers Oxford. find
  26. Eve Sweetser. 1990. From etymology to pragmatics: Metaphorical and cultural aspects of semantic structure, volume 54. Cambridge University Press. find
  27. Fei Wang, Yunfang Wu, and Likun Qiu. 2012. Exploiting discourse relations for sentiment analysis. In Proceedings of COLING 2012: Posters, pages 1311–1320.
  28. Anna Wierzbicka. 1998. Anchoring linguistic typology in universal semantic primes. ↗
  29. Richard Xiao and Tony McEnery. 2006. Collocation, semantic prosody, and near synonymy: A cross-linguistic perspective. Applied linguistics, 27(1):103–129. ↗
  30. Farah Benamara Zitoune, Nicholas Asher, Yvette Yannick Mathieu, Vladimir Popescu, and Baptiste Chardon. 2016. Evaluation in discourse: A corpus-based study. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/andersson-2020-sentiment-annotated-dataset-english-causal/