Elamur Research⌕
☰
home / papers / 2026 / Best-Worst Scaling of Hype in Biomedical Research: Building an Intensity Lexicon of Promotional Adjectives

Best-Worst Scaling of Hype in Biomedical Research: Building an Intensity Lexicon of Promotional Adjectives

Summary

Biomedical papers, grant texts and press materials increasingly use promotional wording ("hype"). Automatic hype assessment needs to know how strongly each term promotes, because hype is graded rather than binary, but no such graded resource existed. The authors built Hyplex, a lexicon of 303 adjectives from biomedical writing, each with a human-derived promotional intensity score. They started from 139 hype adjectives previously found in the NIH corpus (901,717 successful grant abstracts, 1985–2020). They added near-synonyms from WordNet and ChatGPT and kept those that occurred more than ten times in the corpus with a promotional sense. Ten expert English users then rated the terms with Best–Worst Scaling. In category-specific Balanced Incomplete Block Designs, they chose the most and least promotional word in each set of four. High, neutral and low anchors calibrated scores across categories. The ratings are reliable (split-half r = 0.87). When compared against the NRC VAD Lexicon, the scores track arousal and dominance more closely than valence, so hype reflects activation and assertiveness more than plain positivity. This enables graded hype detection and feedback tools.

Key points

  • Hyplex covers 303 adjectives in eight semantic categories: IMPORTANCE, NOVELTY, SCALE, RIGOUR, UTILITY, QUALITIES, ATTITUDE and PROBLEM. Category sizes range from 20 to 57 items.
  • Annotation used Best–Worst Scaling on 4-item sets. The BIBDs were generated with the R package bwsTools, and all 10 annotators saw identical blocks within each category over five sessions.
  • Internal consistency is high: split-half reliability r = 0.87, 95% CI [0.85, 0.89].
  • Validation against the NRC VAD Lexicon shows the strongest correlations with arousal and dominance.
  • The authors release the lexicon and a web-based BWS annotation platform that integrates with bwsTools. Stated limitations are that the resource covers only adjectives and only the biomedical domain.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/3cdc4v7fff2y

@inproceedings{millar2026best,
  title = {{Best-Worst Scaling of Hype in Biomedical Research: Building an Intensity Lexicon of Promotional Adjectives}},
  author = {Millar, Neil and Satav, Dipesh and Batalo, Bojan and Shimomoto, Erica K. and Ohniwa, Ryosuke L.},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/3cdc4v7fff2y},
  url = {https://doi.org/10.63317/3cdc4v7fff2y}
}

References

28 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Bojan Batalo, Erica K. Shimomoto, Dipesh Satav, and Neil Millar. 2026. Hype or not? formalizing automatic promotional language detection in biomedical research. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics. Association for Computation… find
  2. Howard Bauchner and Frederick P Rivara. 2022. The scientific communication ecosystem: the responsibility of investigators. The Lancet, 400(10360):1289–1290. find
  3. Godfred O Boateng, Torsten B Neilands, Edward A Frongillo, Hugo R Melgar-Quiñonez, and Sera L Young. 2018. Best practices for developing and validating scales for health, social, and behavioral research: a primer. Frontiers in public health, 6:149. find
  4. William G Cochran and Gertrude M Cox. 1957. Experimental designs. John Wiley & Sons. find
  5. Christopher Crocker and David MH Thomson. 2014. Anchored scaling in best–worst experiments: A process for facilitating comparison of conceptual profiles. Food Quality and Preference, 33:37–53. find
  6. Adam Finn and Jordan J Louviere. 1992. Determining the appropriate response to evidence of public concern: the case of food safety. Journal of Public Policy & Marketing, 11(2):12–25. find
  7. Terry N Flynn and Anthony AJ Marley. 2014. Best-worst scaling: theory and methods. In Handbook of choice modelling, pages 178–201. Edward Elgar Publishing. find
  8. Susan Hunston and Geoffrey Thompson. 2000. Evaluation in text: Authorial stance and the construction of discourse: Authorial stance and the construction of discourse. Oxford University Press, UK. find
  9. Ken Hyland. 2005. Metadiscourse: Exploring interaction in writing. Continuum. find
  10. Jordan Louviere, Ian Lings, Towhidul Islam, Siegfried Gudergan, and Terry Flynn. 2013. An introduction to the application of (case 1) best– worst scaling in marketing research. International journal of research in marketing, 30(3):292–303. find
  11. Jordan J. Louviere, Terry N. Flynn, and A. A. J. Marley. 2015. Best-Worst Scaling: Theory, Methods and Applications. Cambridge University Press. find
  12. Jordan J Louviere and George G Woodworth. 1991. Best-worst scaling: A model for the largest difference judgments. Technical report, working paper. find
  13. James R Martin and Peter R White. 2003. The language of evaluation, volume 2. Springer. find
  14. Neil Millar, Bojan Batalo, and Brian Budgell. 2022a. Trends in the use of promotional language (hype) in abstracts of successful national institutes of health grant applications, 1985-2020. JAMA network open, 5(8):e2228676–e2228676. find
  15. Neil Millar, Bojan Batalo, and Brian Budgell. 2022b. Trends in the use of promotional language (hype) in national institutes of health funding opportunity announcements, 1992-2020. JAMA Network Open, 5(11):e2243221–e2243221. find
  16. Neil Millar, Bojan Batalo, and Brian Budgell. 2023. Promotional language (hype) in abstracts of publications of national institutes of health–funded research, 1985-2020. JAMA Network Open, 6(12):e2348706–e2348706. find
  17. Neil Millar, Françoise Salager-Meyer, and Brian Budgell. 2019. “it is important to reinforce the importance of. . . ”:‘hype’in reports of randomized controlled trials. English for Specific Purposes, 54:139–151. find
  18. Hao Peng, Huilian Sophie Qiu, Henrik Barslund Fosse, and Brian Uzzi. 2024. Promotional language and the adoption of innovative ideas in science. Proceedings of the National Academy of Sciences, 121(25):e2320066121. find
  19. Huilian Sophie Qiu, Hao Peng, Henrik Barslund Fosse, Teresa K Woodruff, and Brian Uzzi. 2024. Use of promotional language in grant applications and grant success. JAMA network open, 7(12):e2448696–e2448696. find
  20. Petroc Sumner, Solveiga Vivian-Griffiths, Jacky Boivin, Andy Williams, Christos A Venetis, Aimée Davies, Jack Ogden, Leanne Whelan, Bethan Hughes, Bethan Dalton, et al. 2014. The association between exaggeration in health related science news and academic press releases: retrospective observational… find
  21. Nils B Weidmann, Sabine Otto, and Lukas Kawerau. 2018. The use of positive words in political science language. PS: Political Science & Politics, 51(3):625–628. find
  22. Mark H White II. 2021. bwstools: An r package for case 1 best-worst scaling. Journal of choice modelling, 39:100289. 8. find
  23. Language Resource References Feng, Song and Kang, Jun Seok and Kuznetsova, Polina and Choi, Yejin. 2013. Connotation lexicon: A dash of sentiment beneath the surface meaning. PID https://www3.cs.stonybro ok.edu/~ychoi/connotation/. ↗
  24. Kiritchenko, Svetlana and Mohammad, Saif. 2016. Capturing Reliable Fine-Grained Sentiment Associations by Crowdsourcing and Best–Worst Scaling. PID https://www.saifmohammad.com/WebPages/BestWorst.html. ↗
  25. Miller, George A. 1995. WordNet: a lexical database for English. ACM New York, NY, USA. PID https://wordnet.princeton.edu/. ↗ our card
  26. Mohammad, Saif. 2018. Obtaining Reliable Human Ratings of Valence, Arousal, and Dominance for 20,000 English Words. Association for Computational Linguistics. PID https://saifmoha mmad.com/WebPages/nrc-vad.html. find our card
  27. Mohammad, Saif and Bravo-Marquez, Felipe. 2017. Emotion Intensities in Tweets. Association for Computational Linguistics. PID https://www.saifmohammad.com/WebPages/BestWors t.html. ↗ our card
  28. Wilson, Theresa and Wiebe, Janyce and Hoffmann, Paul. 2009. Recognizing contextual polarity: An exploration of features for phrase-level sentiment analysis. MIT Press One Rogers Street, Cambridge, MA 02142-1209, USA journals-info . . . . PID https://mpqa.cs.pitt.edu/lexic ons/subj_lexicon/. ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/millar-2026-best-worst-scaling-hype-biomedical/