Elamur Research⌕
☰
home / papers / 2020 / It's Morphin' Time! Combating Linguistic Discrimination with Inflectional Perturbations

It's Morphin' Time! Combating Linguistic Discrimination with Inflectional Perturbations

Paper · 2020 · Samson Tan, Shafiq Joty, Min-Yen Kan et al.

Summary

NLP models trained mostly on polished Standard English often fail for people whose English uses inflections differently, such as L2 learners and speakers of dialects like AAVE or Colloquial Singapore English. The authors treat this as a form of linguistic discrimination and test it with MORPHEUS, a black-box attack. MORPHEUS visits each noun, verb and adjective in a sentence and substitutes the inflected form of the same lemma that most increases the target model's loss. It only considers forms with the same universal POS. Candidate forms come from lemminflect, and the attack needs nothing beyond the model's logits.

The attack was run against BERT and SpanBERT trained on SQuAD 1.1/2.0, and against Transformer-big translating English to French. In all cases it seriously damaged model output.

As a fix, the authors sample inflections according to the distribution seen in successful adversaries, build an augmented training set, and fine-tune the existing model for a single epoch. This gives large robustness gains at a small cost on clean data. The result matters because it offers an inexpensive route to fairer systems with no extra cost at inference time.

Key points

  • MORPHEUS searches greedily and sequentially, so loss increases build up across tokens. With contextual models this beat perturbing each token independently in parallel. Most words have fewer than 5 inflections, so querying a black-box model stays cheap.
  • Search should be driven by the loss, not by step-like metrics. Exact match and F1 in extractive QA are too flat to guide the search toward full model failure.
  • SQuAD 2.0 models proved more fragile than SQuAD 1.1 models and tended to switch to "no answer". Transformer-big sometimes produced completely unrelated French, and for about 1.4% of the test set the adversaries made it echo the English source.
  • The adversarial training set is built by weighted random sampling: 4 samples per example, giving a 1:4 clean-to-adversarial ratio, in linear time.
    • SpanBERT: one epoch of fine-tuning cost under 2 points on the clean dev set.
    • Transformer-big: the degradation under attack fell from 56.25% to 20.20%, at roughly 3 BLEU on clean data. Keeping the full original training set was needed to hold clean performance.
    • Retraining from scratch gave no advantage.
  • Code and robust models are released at github.com/salesforce/morpheus.

Interesting ideas

  • I1. Restricting a perturbation to inflections of the same lemma and same universal POS yields test inputs whose meaning is essentially preserved, so any change in output signals a real weakness.
  • I2. Sequential greedy search, where each token's change is chosen given the earlier changes, finds combinations of edits that hurt a contextual model far more than independent per-token edits.
  • I3. An adversarial search needs a smooth objective, because a 0/1-style metric gives no direction to climb.
  • I4. Building augmentation data by sampling from the distribution of forms that actually broke the model, then fine-tuning briefly instead of retraining, buys robustness cheaply while keeping clean accuracy.
  • I5. The authors argue against adding a separate error-correction model in front, since it doubles inference cost; making the main model tolerant is cheaper.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): Inflect test sentences this way and check that the tagger and lemmatizer in crate en still return the original lemmas.
  • I2 (learn): Use the same accumulating search to hunt worst-case inputs that can feed self-induced repair rules.
  • I3 (learn): When searching for parser failures, rank candidates by parser score margins or attachment confidence rather than per-sentence exact LAS.
  • I4 (tree): Add inflection-perturbed copies of treebank sentences (gold heads kept) to fine-tuning for the perceptron parser and gparse, weighted toward the forms that caused errors.
  • I5 (frames): Make frame and role extraction in crate prag rely on lemmas and dependency relations rather than agreement morphology, so non-standard inflection still yields the same who-did-what-to-whom.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓National University of SingaporeSingapore · Singapore84 works · 32,204 citations🏢SalesforceSan Francisco · United States37 works · 15,019 citations🎓Nanyang Technological UniversitySingapore · Singapore73 works · 24,390 citations

📚 Shelves 4 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Social Sciences · 744 works Safety Research · 221 works

🧭 Topics 1 Morphology · 277 works

Cite

Cite as Samson Tan, Shafiq Joty, Min-Yen Kan and Richard Socher. 2020. It's Morphin' Time! Combating Linguistic Discrimination with Inflectional Perturbations. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.acl-main.263

@inproceedings{tan2020it,
  title = {{It's Morphin' Time! Combating Linguistic Discrimination with Inflectional Perturbations}},
  author = {Samson Tan, Shafiq Joty, Min-Yen Kan and Richard Socher},
  booktitle = {Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.acl-main.263},
  url = {https://doi.org/10.18653/v1/2020.acl-main.263}
}

References

53 references, 35 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Eneko Agirre, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, and Weiwei Guo. 2013. *SEM 2013 shared task: Semantic textual similarity. In Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 1: Proceedings of the Main Conference and the Shared Task: Semantic Textual Similarity, pages 32–43, Atlanta, Georgia, USA. Association for Computational Linguistics. our card
  2. Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani Srivastava, and Kai-Wei Chang. 2018. Generating natural language adversarial examples. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2890–2896, Brussels, Belgium. Association for Computational Linguistics. ↗ our card
  3. Yonatan Belinkov and Yonatan Bisk. 2018. Synthetic and natural noise both break neural machine translation. In 6th International Conference on Learning Representations, Vancouver, BC, Canada. ↗
  4. Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural Language Processing with Python. O’Reilly Media. find
  5. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems 29, pages 4349–4357. Curran Associates, Inc. ↗ our card
  6. Shikha Bordia and Samuel R. Bowman. 2019. Identifying and reducing gender bias in word-level language models. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop, pages 7–15, Minneapolis, Minnesota. Association for Computational Linguistics. our card
  7. Christopher Bryant, Mariano Felice, Øistein E. Andersen, and Ted Briscoe. 2019. The BEA-2019 shared task on grammatical error correction. In Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 52–75, Florence, Italy. Association for Computational Linguistics. our card
  8. David Crystal. 2003. English as a Global Language. Cambridge University Press. ↗
  9. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. ↗ our card
  10. David M. Eberhard, Gary F. Simons, and Charles D. Fennig, editors. 2019. Ethnologue: Languages of the World, 22 edition. SIL International. find
  11. Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. 2018. HotFlip: White-box adversarial examples for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 31–36, Melbourne, Australia. Association for Computational Linguistics. ↗ our card
  12. Steffen Eger, Gözde Gül Sahin, Andreas Rücklé, Ji-Ung Lee, Claudia Schulz, Mohsen Mesgar, Krishnkant Swarnkar, Edwin Simpson, and Iryna Gurevych. 2019. Text processing like humans do: Visually attacking and shielding nlp systems. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1634–1647, Minneapolis, Minnesota. Association for Computational Linguistics. ↗ our card
  13. Matt Gardner, Joel Grus, Mark Neumann, Oyvind Tafjord, Pradeep Dasigi, Nelson F. Liu, Matthew Peters, Michael Schmitz, and Luke Zettlemoyer. 2018. AllenNLP: A deep semantic natural language processing platform. In Proceedings of Workshop for NLP Open Source Software (NLP-OSS), pages 1– 6, Melbourne, Australia. Association for Computational Linguistics. our card
  14. Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N. Dauphin. 2017. Convolutional sequence to sequence learning. In Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 1243–1252, International Convention Centre, Sydney, Australia. PMLR. ↗ our card
  15. Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and harnessing adversarial examples. In 3rd International Conference on Learning Representations, San Diego, California. ↗ our card
  16. J. Hartshorne, B. Tenenbaum, J., and S. Pinker. 2018. A critical period for second language acquisition: Evidence from 2/3 million english speakers. Cognition, 177:263–277. ↗
  17. Belma Haznedar. 2002. Missing surface inflection in adult and child l2 acquisition. In Proceedings of the 6th Generative Approaches to Second Language Acquisition Conference, pages 140–149, Somerville, Massachusetts. Cascadilla Proceedings Project. find
  18. Alex Hern. 2017. Facebook translates ’good morning’ into ’attack them’, leading to arrest. The Guardian. find
  19. Dirk Hovy and Shannon L. Spruit. 2016. The social impact of natural language processing. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 591–598, Berlin, Germany. Association for Computational Linguistics. ↗ our card
  20. Mohit Iyyer, John Wieting, Kevin Gimpel, and Luke Zettlemoyer. 2018. Adversarial example generation with syntactically controlled paraphrase networks. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1875–1885, New Orleans, Louisiana. Association for Computational Linguistics. ↗ our card
  21. Robin Jia and Percy Liang. 2017. Adversarial examples for evaluating reading comprehension systems. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2021–2031, Copenhagen, Denmark. Association for Computational Linguistics. ↗ our card
  22. Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. 2019. Is bert really robust? natural language attack on text classification and entailment. arXiv e-prints, arXiv:1907.11932. ↗ our card
  23. Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, and Omer Levy. 2019. SpanBERT: Improving pre-training by representing and predicting spans. arXiv e-prints, arXiv:1907.10529. ↗ our card
  24. Braj B. Kachru, Yamuna Kachru, and Cecil Nelson, editors. 2009. The Handbook of World Englishes. Wiley-Blackwell. find
  25. Nitish Shirish Keskar, Bryan McCann, Lav Varshney, Caiming Xiong, and Richard Socher. 2019. CTRL - A Conditional Transformer Language Model for Controllable Generation. arXiv e-prints, arXiv:1909.05858. ↗ our card
  26. Donna Lardiere. 1998. Case and tense in the ‘fossilized’ steady state. Second Language Research, 14(1):1–26. find
  27. Jacob RE Leimgruber. 2009. Modelling variation in Singapore English. Ph.D. thesis, Oxford University. find
  28. Chandler May, Alex Wang, Shikha Bordia, Samuel R. Bowman, and Rachel Rudinger. 2019. On measuring social biases in sentence encoders. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 622–628, Minneapolis, Minnesota. Association for Computational Linguistics. our card
  29. Paul Michel, Xian Li, Graham Neubig, and Juan Pino. 2019. On evaluation of adversarial perturbations for sequence-to-sequence models. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3103–3114, Minneapolis, Minnesota. Association for Computational Linguistics. ↗ our card
  30. Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), pages 48–53, Minneapolis, Minnesota. Association for Computational Linguistics. ↗ our card
  31. Myle Ott, Sergey Edunov, David Grangier, and Michael Auli. 2018. Scaling neural machine translation. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 1–9, Brussels, Belgium. Association for Computational Linguistics. our card
  32. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics. ↗ our card
  33. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 2227–2237, New Orleans, Louisiana. Association for Computational Linguistics. ↗ our card
  34. Maja Popović. 2015. chrF: character n-gram f-score for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395, Lisbon, Portugal. Association for Computational Linguistics. ↗ our card
  35. Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186– 191, Brussels, Belgium. Association for Computational Linguistics. ↗ our card
  36. Philippe Prévost and Lydia White. 2000. Missing surface inflection or impairment in second language acquisition? evidence from tense and agreement. Second Language Research, 16:103–133. ↗
  37. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. find
  38. Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. Know what you don’t know: Unanswerable questions for SQuAD. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 784– 789, Melbourne, Australia. Association for Computational Linguistics. ↗ our card
  39. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383–2392, Austin, Texas. Association for Computational Linguistics. ↗ our card
  40. Lydia White. 2003. Fossilization in steady state l2 grammars: Persistent problems with inflectional morphology. Bilingualism: Language and Cognition, 6:129 – 141. ↗
  41. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2018. Semantically equivalent adversarial rules for debugging NLP models. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 856–865, Melbourne, Australia. Association for Computational Linguistics. ↗
  42. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, R’emi Louf, Morgan Funtowicz, and Jamie Brew. 2019. Huggingface’s transformers: State-of-the-art natural language processing. arXiv e-prints, arXiv:1910.03771. ↗ our card
  43. John Rickford and Sharese King. 2016. Language and linguistics on trial: Hearing rachel jeantel (and other vernacular speakers) in the courtroom and beyond. Language, 92:948–988. ↗
  44. Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018. Gender bias in coreference resolution. In Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL), pages 8–14, New Orleans, Louisiana. Association for Computational Linguistics. our card
  45. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715– 1725, Berlin, Germany. Association for Computational Linguistics. ↗ our card
  46. Walt Wolfram. 2004. The grammar of urban African American Vernacular English. Handbook of varieties of English, 2:111–32. find
  47. Wei Emma Zhang, Quan Z. Sheng, Ahoud Alhazmi, and Chenliang Li. 2019a. Adversarial attacks on deep learning models in natural language processing: A survey. arXiv e-prints, arXiv:1901.06796. ↗
  48. Yuan Zhang, Jason Baldridge, and Luheng He. 2019b. PAWS: paraphrase adversaries from word scrambling. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1298–1308, Minneapolis, Minnesota. Association for Computational Linguistics. ↗ our card
  49. Min Joon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2017. Bidirectional attention flow for machine comprehension. In 5th International Conference on Learning Representations, Toulon, France. ↗ our card
  50. Harry Seymour. 2004. The challenge of language assessment for african american english-speaking children: A historical perspective. Seminars in Speech and Language, 25:3–12. ↗
  51. Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. 2014. Intriguing properties of neural networks. In 2nd International Conference on Learning Representations, Banff, AB, Canada. ↗ our card
  52. Rachael Tatman. 2017. Gender and dialect bias in YouTube’s automatic captions. In Proceedings of the First ACL Workshop on Ethics in Natural Language Processing, pages 53–59, Valencia, Spain. Association for Computational Linguistics. our card
  53. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 5998–6008. Curran Associates, Inc. our card

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tan-2020-it-s-morphin-time-combating/