Elamur Research⌕
☰
home / papers / 2012 / Hedge Detection as a Lens on Framing in the GMO Debates: A Position Paper

Hedge Detection as a Lens on Framing in the GMO Debates: A Position Paper

Paper · 2012 · Eunsol Choi, Chenhao Tan, Lillian Lee et al.

Summary

This position paper urges computational linguists to study framing, meaning how the way an issue is presented shapes opinion. It proposes a concrete test case: in the GMO controversy, do pro- and anti-GMO articles differ in how "scientific" they sound? The authors use hedging, a writer's partial withholding of commitment to a claim, as a measurable stand-in for scientific tone. Their plan has two stages. First, check whether hedge frequency actually separates professional science writing from popular science writing on GMOs. Only then, compare pro- and anti-GMO texts.

For the first stage they built and released two corpora: Web of Science abstracts and LexisNexis news articles. They trained a cue-word hedge classifier, reimplemented after Georgescul, on the CoNLL 2010 data. It transferred poorly to the new domains, so no firm conclusions were possible. Still, one model hinted that scientific prose hedges less than popular text, which runs against claims in the rhetoric literature. The paper matters mainly as a research agenda plus data, and as a frank report on domain shift in hedge detection.

Key points

  • New corpora:
    • WOS: 648 abstracts, 5596 sentences, Flesch 23.39. Found with the query "transgenic foods", with off-topic terms such as "mice" and "rats" filtered out, then cleaned by an expert.
    • LEXIS: 928 news articles, 36,795 sentences, Flesch 45.78.
  • Training data: CoNLL 2010 hedge corpora.
    • Bio train: 14,541 sentences, 18% uncertain.
    • Bio eval: 5003 sentences, 16% uncertain.
    • Wiki train: 11,111 sentences, 22% uncertain.
    • Wiki eval: 9634 sentences, 23% uncertain.
  • Domain transfer: Results were checked on 153 hand-labelled WOS+LEXIS sentences.
    • Accuracy fell sharply compared with in-domain cross-validation.
    • Bio was the best training set.
    • Bio+Wiki gave high recall but many false positives.
    • The classifier did better on WOS than on LEXIS.
  • Uncertain-sentence rates:
    • Bio model: WOS 16%, LEXIS 19%, a statistically significant gap.
    • Model tuned on the 153 sentences: WOS 15%, LEXIS 14%, not significant.
    • The authors consider both unreliable.
  • Length bias: False positives were longer than other sentences (35.09 tokens on average, against 28.6 for true positives and 22.0 for false negatives). Cue counts grow with sentence length.

Interesting ideas

  • I1. Before using a linguistic feature as a proxy for a high-level property, first confirm on a controlled contrast (prof- vs pop-science) that the proxy really separates the classes. Only then apply it to the target question.
  • I2. Cue-frequency detectors can secretly measure sentence length, because longer sentences simply contain more chances to match a cue. Splitting errors by length exposed this.
  • I3. A small hand-labelled in-domain sample (153 sentences) was enough to expose cross-domain collapse. Tuning on it improved one target domain while hurting the other.
  • I4. Pooling cue lexicons from two domains raised recall but flooded the output with false positives. Uncertainty markers behave differently across domains.
  • I5. Hedging is defined as the writer withholding full commitment to a statement, which is an extra-propositional layer on top of the claim itself.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Any signal used to score silver data or repair rules should first be checked on a known contrast set with a negative control, so we never build on an unvalidated proxy.
  • I2 (learn): Evaluation reports for parsers and frame extraction should break errors down by sentence length, which would reveal length confounds instead of hiding them in one score.
  • I3 (tree): Keep tiny labelled samples per genre (news, school texts, fables) for gparse and the perceptron parser, and check that tuning for one genre does not quietly degrade another.
  • I4 (frames): The hedge and modality lexicon in prag should be genre-scoped or weighted, not one merged list.
  • I5 (world): Facts entered into the world of a text could carry a commitment level taken from hedges ("seems", "suggest", "probably"), so answers can say how firmly the text asserts something.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Cornell UniversityIthaca · United States89 works · 58,422 citations

📚 Shelves 6 Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works Social Sciences · 744 works Philosophy · 211 works General Social Sciences · 87 works Literature and Literary Theory · 340 works

Cite

Cite as Eunsol Choi, Chenhao Tan, Lillian Lee, Cristian Danescu-Niculescu-Mizil and Jennifer Spindel. 2012. Hedge Detection as a Lens on Framing in the GMO Debates: A Position Paper. Proceedings of the Workshop on Extra-Propositional Aspects of Meaning in Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. arXiv 1206.1066

@inproceedings{choi2012hedge,
  title = {{Hedge Detection as a Lens on Framing in the GMO Debates: A Position Paper}},
  author = {Eunsol Choi, Chenhao Tan, Lillian Lee, Cristian Danescu-Niculescu-Mizil and Jennifer Spindel},
  booktitle = {Proceedings of the Workshop on Extra-Propositional Aspects of Meaning in Computational Linguistics},
  year = {2012},
  eprint = {1206.1066},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1206.1066}
}

References

32 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Amr Ahmed and Eric P Xing. Staying informed: supervised and semi-supervised multi-view topical analysis of ideological perspective. In EMNLP, pages 1140–1150, 2010. find
  2. Charles Bazerman. Shaping Written Knowledge: The Genre and Activity of the Experimental Article in Science. University of Wisconsin Press, Madison, Wis., 1988. find
  3. Beata Beigman Klebanov, Eyal Beigman, and Daniel Diermeier. Vocabulary choice as an indicator of perspective. In ACL Short Papers, pages 253–257, Stroudsburg, PA, USA, 2010. Association for Computational Linguistics. find
  4. Robert D. Benford and David A. Snow. Framing processes and social movements: An overview and assessment. Annual Review of Sociology, 26: 611–639, 2000. ↗
  5. Richard Harvey Brown. Toward a democratic science: Scientific narration and civic communication. Yale University Press, New Haven, 1998. find
  6. Dennis Chong and James N. Druckman. Framing theory. Annual Review of Political Science, 10: 103–126, 2007. find
  7. Jeanne Fahnestock. Accommodating Science. Written Communication, 3(3):275–296, 1986. find
  8. Jeanne Fahnestock. Preserving the figure: Consistency in the presentation of scientific arguments. Written Communication, 21(1):6–31, 2004. find
  9. Rong-En Fan, Pai-Hsuen Chen, and Chih-Jen Lin. Working set selection using second order information for training support vector machines. JMLR, 6:1889–1918, December 2005. ISSN 1532-4435. ↗
  10. Richárd Farkas, Veronika Vincze, György Móra, János Csirik, and György Szarvas. The CoNLL-2010 shared task: Learning to detect hedges and their scope in natural language text. In CoNLL— Shared Task, pages 1–12, 2010. our card
  11. Joseph L. Fleiss. Statistical Methods for Rates and Proportions. Wiley series in probability and mathematical statistics. John Wiley & Sons, New York, second edition, 1981. find
  12. Maria Georgescul. A hedgehop over a max-margin framework using hedge cues. In CONLL— Shared-Task, pages 26–31, 2010. our card
  13. G. Nigel Gilbert and Michael Joseph Mulkay. Opening Pandora’s box: A sociological analysis of scientists’ discourse. CUP Archive, 1984. find
  14. Erving Goffman. Frame analysis: An essay on the organization of experience. Harvard University Press, 1974. find
  15. Alan G. Gross. The rhetoric of science. Harvard University Press, Cambridge, Mass., 1990. find
  16. Michael Alexander Kirkwood Halliday and James R. Martin. Writing science: Literacy and discursive power. Psychology Press, London [u.a.], 1993. find
  17. Eric A Hardisty, Jordan Boyd-Graber, and Philip Resnik. Modeling perspective using adaptor grammars. In EMNLP, pages 284–292, 2010. find
  18. Ken Hyland. Hedging in scientific research articles. John Benjamins Pub. Co., Amsterdam; Philadelphia, 1998. ↗ our card
  19. Napoleon K. Juanillo, Jr. Frames for Public Discourse on Biotechnology. In Genetically Modified Food and the Consumer: Proceedings of the 13th meeting of the National Agricultural Biotechnology Council, pages 39–50, 2001. find
  20. J. Peter Kincaid, Robert P. Fishburne, Richard L. Rogers, and Brad S. Chissom. Derivation of new readability formulas for navy enlisted personnel. Technical report, National Technical Information Service, Springfield, Virginia, February 1975. find
  21. George Lakoff. Hedges: A study in meaning criteria and the logic of fuzzy concepts. Journal of Philosophical Logic, 2(4):458–508, 1973. ↗ our card
  22. Bruno Latour. Science in action: How to follow scientists and engineers through society. Harvard University Press, Cambridge, Mass., 1987. find
  23. Bruno Latour and Steve Woolgar. Laboratory life: The social construction of scientific facts. Sage Publications, Beverly Hills, 1979. find
  24. Beverly A. Lewin. Hedging: Form and function in scientific research texts. In Genre Studies in English for Academic Purposes, volume 9, pages 89–108. Universitat Jaume I, 1998. find
  25. Wei-Hao Lin, Theresa Wilson, Janyce Wiebe, and Alexander Hauptmann. Which side are you on? identifying perspectives at the document and sentence levels. In CoNLL, 2006. find
  26. Greg Myers. The pragmatics of politeness in scientific articles. Applied Linguistics, 10(1):1–35, 1989. ↗
  27. Greg Myers. Discourse studies of scientific popularization: Questioning the boundaries. Discourse Studies, 5(2):265–279, 2003. ↗
  28. Françoise Salager-Meyer. Scientific discourse and contrastive linguistics: hedging. European Science Editing, 37(2):35–37, 2011. find
  29. Dietram A. Scheufele. Framing as a theory of media effects. Journal of Communication, 49(1):103– 122, 1999. ↗
  30. Vladimir N. Vapnik. Statistical Learning Theory. Wiley-Interscience, 1998. find
  31. Teppo Varttala. Remarks on the communicative functions of hedging in popular scientific and specialist research articles on medicine. English for Specific Purposes, 18(2):177–200, 1999. ↗
  32. Teppo Varttala. Hedging in scientifically oriented discourse: Exploring variation according to discipline and intended audience. PhD thesis, University of Tampere, 2001. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/choi-2012-hedge-detection-as-lens-framing/