Elamur Research⌕
☰
home / papers / 2023 / GPT detectors are biased against non-native English writers

GPT detectors are biased against non-native English writers

Summary

AI-text detectors are increasingly used to check essays and papers, yet little was known about whether they treat all human writers equally. The authors tested seven popular GPT detectors on two sets of human writing. The first was 91 TOEFL essays by non-native English speakers, collected from a Chinese education forum. The second was 88 US 8th-grade essays from the Hewlett Foundation ASAP dataset.

The detectors classified the native essays almost perfectly. However, they marked most TOEFL essays as machine-written, with an average false positive rate of 61.22%. The authors link this to perplexity. Non-native writing tends to use simpler, more predictable wording, and detectors treat that as a sign of AI authorship. When ChatGPT enriched the vocabulary of the TOEFL essays, false positives dropped sharply. When it simplified the native essays, false positives rose.

The same mechanism enables evasion: one self-editing prompt let ChatGPT-3.5 essays and abstracts get past most detectors. The study warns against using such tools in education or assessment, where they could unfairly penalize non-native speakers.

Key points

  • Unanimous and partial flagging of TOEFL essays: all seven detectors flagged 18 of the 91 TOEFL essays (19.78%), and at least one detector flagged 89 of them (97.80%). The unanimously flagged essays had significantly lower perplexity (P = 9.74E-05).
  • Effect of vocabulary enrichment: after ChatGPT made the TOEFL wording more native-like, the average false positive rate fell from 61.22% to 11.77%. Only 1 essay was then flagged by every detector.
  • Effect of simplification: rewriting the US 8th-grade essays in simplified, "non-native" style raised misclassification from 5.19% to 56.65% and lowered their perplexity.
  • Academic writing check: among 1574 accepted ICLR 2023 papers (submitted before ChatGPT's release), abstracts from authors in non-native English-speaking countries had lower perplexity (P = 0.035). The gap remained after controlling for review scores (P = 0.033).
  • Evasion by self-editing: a second-round self-edit prompt cut detection of 31 generated Common App essays from 100% to 13%. For abstracts generated from 145 Stanford CS224n titles, the highest detection rate across detectors dropped from 68% to 28%.

Interesting ideas

  • I1. Bidirectional style intervention as a causal test: the authors shift text in both directions, enriching non-native essays and simplifying native ones. This gives stronger evidence that a specific property (lexical variety) drives a classifier's errors, rather than just observing a correlation. It is a reusable design for auditing any text classifier.
  • I2. Comparing items that all models agree on: the authors examine the essays that every detector flagged and compare them with the rest along one interpretable variable (perplexity). This is a cheap way to find what a group of models is jointly reacting to.
  • I3. Using a time cutoff to get clean human data: choosing a conference whose deadlines predate ChatGPT ensures the corpus contains no machine-written text. The same trick suits any study that needs reliably human text, for example Ukrainian corpora collected before a given model's release.
  • I4. Controlling for a quality confounder: the authors adjusted for review ratings when comparing native and non-native abstracts, so that lower perplexity is not simply explained by weaker papers. Separating style from quality in this way matters in any fairness audit.
  • I5. Testing robustness with simple self-edit prompts: a single rewrite instruction is a cheap baseline attack for any perplexity-based detector. It is worth including in standard evaluation before claiming a detector is reliable.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

66 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 62 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2304.02819 · arXiv 2304.02819

@misc{liang2023gpt,
  title = {{GPT detectors are biased against non-native English writers}},
  author = {Liang, Weixin and Yuksekgonul, Mert and Mao, Yining and Wu, Eric and Zou, James},
  year = {2023},
  doi = {10.48550/arxiv.2304.02819},
  eprint = {2304.02819},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2304.02819}
}

References

36 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. OpenAI. ChatGPT. https://chat.openai.com/ (2022). Accessed: 2022-12-31. ↗
  2. Hu, K. Chatgpt sets record for fastest-growing user base - analyst note. Reuters (2023). find
  3. Paris, M. Chatgpt hits 100 million users, google invests in ai bot and catgpt goes viral. Forbes (2023). find
  4. Lee, M. et al. Evaluating human-language model interaction. arXiv preprint arXiv:2212.09746 (2022). ↗
  5. Kung, T. H. et al. Performance of chatgpt on usmle: Potential for ai-assisted medical education using large language models. PLoS digital health 2, e0000198 (2023). find
  6. Terwiesch, C. Would chat gpt3 get a wharton mba? a prediction based on its performance in the operations management course. Mack Inst. for Innov. Manag. at Whart. Sch. Univ. Pennsylvania (2023). find
  7. Else, H. Abstracts written by chatgpt fool scientists. Nature (2023). find
  8. Gao, C. A. et al. Comparing scientific abstracts generated by chatgpt to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers. bioRxiv 2022–12 (2022). find
  9. Kreps, S., McCain, R. & Brundage, M. All the news that’s fit to fabricate: Ai-generated text as a tool of media misinformation. J. Exp. Polit. Sci. 9, 104–117, DOI: 10.1017/XPS.2020.37 (2022). ↗
  10. Editorial, N. Tools such as chatgpt threaten transparent science; here are our ground rules for their use. Nature 613, 612–612 (2023). find
  11. ICML. Clarification on large language model policy LLM. https://icml.cc/Conferences/2023/llm-policy (2023). ↗
  12. Clark, E. et al. All that’s ‘human’is not gold: Evaluating human evaluation of generated text. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 7282–7296 (2021). find
  13. OpenAI. GPT-2: 1.5B release. https://openai.com/research/gpt-2-1-5b-release (2019). Accessed: 2019-11-05. ↗
  14. Jawahar, G., Abdul-Mageed, M. & Lakshmanan, L. V. Automatic detection of machine generated text: A critical survey. arXiv preprint arXiv:2011.01314 (2020). ↗
  15. Fagni, T., Falchi, F., Gambini, M., Martella, A. & Tesconi, M. Tweepfake: About detecting deepfake tweets. Plos one 16, e0251415 (2021). find
  16. Ippolito, D., Duckworth, D., Callison-Burch, C. & Eck, D. Automatic detection of generated text is easiest when humans are fooled. arXiv preprint arXiv:1911.00650 (2019). ↗ our card
  17. Mitchell, E., Lee, Y., Khazatsky, A., Manning, C. D. & Finn, C. DetectGPT: Zero-shot machine-generated text detection using probability curvature. arXiv preprint arXiv:2301.11305 (2023). ↗ our card
  18. Solaiman, I. et al. Release strategies and the social impacts of language models. arXiv preprint arXiv:1908.09203 (2019). ↗ our card
  19. Gehrmann, S., Strobelt, H. & Rush, A. M. Gltr: Statistical detection and visualization of generated text. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, 111–116 (2019). find our card
  20. Heikkil"a, M. How to spot ai-generated text. MIT Technol. Rev. (2022). find
  21. Crothers, E., Japkowicz, N. & Viktor, H. Machine generated text: A comprehensive survey of threat models and detection methods. arXiv preprint arXiv:2210.07321 (2022). ↗
  22. Rosenblatt, K. Chatgpt banned from new york city public schools’ devices and networks. NBC News (2023). Accessed: 22.01.2023. find
  23. Kasneci, E. et al. Chatgpt for good? on opportunities and challenges of large language models for education. Learn. Individ. Differ. 103, 102274 (2023). find our card
  24. Kaggle. The hewlett foundation: Automated essay scoring. https://www.kaggle.com/c/asap-aes (2012). Accessed: 2023-03-15. ↗
  25. Laufer, B. & Nation, P. Vocabulary size and use: Lexical richness in l2 written production. Appl. linguistics 16, 307–322 (1995). find
  26. Jarvis, S. Short texts, best-fitting curves and new measures of lexical diversity. Lang. Test. 19, 57–84 (2002). find
  27. Daller, H., Van Hout, R. & Treffers-Daller, J. Lexical richness in the spontaneous speech of bilinguals. Appl. linguistics 24, 197–222 (2003). find
  28. Lu, X. A corpus-based evaluation of syntactic complexity measures as indices of college-level esl writers’ language development. TESOL quarterly 45, 36–62 (2011). find
  29. Crossley, S. A. & McNamara, D. S. Does writing development equal writing quality? a computational investigation of syntactic complexity in l2 learners. J. Second. Lang. Writ. 26, 66–79 (2014). find
  30. Ortega, L. Syntactic complexity measures and their relationship to l2 proficiency: A research synthesis of college-level l2 writing. Appl. linguistics 24, 492–518 (2003). find
  31. Biber, D., Gray, B. & Poonpon, K. Should we use characteristics of conversation to measure grammatical complexity in l2 writing development? Tesol Q. 45, 5–35 (2011). find
  32. Krishna, K., Song, Y., Karpinska, M., Wieting, J. & Iyyer, M. Paraphrasing evades detectors of ai-generated text, but retrieval is an effective defense. arXiv preprint arXiv:2303.13408 (2023). ↗ our card
  33. Sadasivan, V. S., Kumar, A., Balasubramanian, S., Wang, W. & Feizi, S. Can ai-generated text be reliably detected? arXiv preprint arXiv:2303.11156 (2023). ↗ our card
  34. Kirchenbauer, J. et al. A watermark for large language models. arXiv preprint arXiv:2301.10226 (2023). ↗ our card
  35. Gu, C., Huang, C., Zheng, X., Chang, K.-W. & Hsieh, C.-J. Watermarking pre-trained language models with backdooring. arXiv preprint arXiv:2210.07543 (2022). ↗
  36. Liang, W., Yuksekgonul, M., Mao, Y., Wu, E. & Zou, J. ChatGPT-Detector-Bias: v1.0.0, DOI: 10.5281/zenodo.7893958 (2023). a c ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liang-2023-gpt-detectors-biased-against-non/