Elamur Research⌕
☰
home / papers / 2019 / On Evaluation of Adversarial Perturbations for Sequence-to-Sequence Models

On Evaluation of Adversarial Perturbations for Sequence-to-Sequence Models

Paper · 2019 · Paul Michel, Xian Li, Graham Neubig et al.

Summary

Adversarial attacks on seq2seq models are usually judged only by how much they degrade the output. That ignores whether the altered input still means the same thing. If the source meaning changes, a different translation is expected, so this is no model failure. Using untargeted word-substitution attacks on machine translation (cs-en, de-en, fr-en), the authors propose an evaluation framework that scores both how well source meaning is kept and how much target quality drops. They also design a 6-level human similarity scale and test automatic metrics as proxies for it. chrF agrees with human ratings better than BLEU or METEOR. Under this framework, unconstrained gradient-based substitutions often distort the source meaning. Adding constraints (nearest-neighbour substitutions, or CharSwap typo-like character swaps) yields perturbations that keep meaning better but still disrupt translations. Adversarial training with CharSwap examples makes a word-based LSTM more robust without hurting clean test accuracy. Unconstrained adversarial training lacks that benefit. The work matters because it provides a principled way to report attack results.

Key points

  • Scoring: The relative target score decrease d_tgt is combined with source similarity s_src into a single success score, S = s_src + d_tgt. An attack with S > 1 damages the target more than it damages the source.
  • Metric choice: chrF (chrF2) had the best Pearson correlation with human judgments, ahead of BLEU and METEOR. RDchrF is used as the main robustness measure.
  • Constraints: CharSwap was the only constraint that consistently achieved an average success above 1.
  • Clean test performance (word-based LSTM, α = 1.0):
    • CharSwap-adv stays close to the baseline, e.g. cs-en 43.74 vs 44.21 chrF.
    • Unconstrained-adv falls to 41.38 chrF.
  • Robustness: CharSwap-adv lowers RDchrF under CharSwap attacks from about 24 to about 16. The evaluation toolkit is released as teapot-nlp.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Carnegie Mellon UniversityPittsburgh · United States365 works · 158,048 citations🏢MetaMenlo Park · United States158 works · 211,459 citations🏢Meta (United Kingdom)London · United Kingdom39 works · 11,541 citations

📚 Shelves 4 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works

🧭 Topics 1 Evaluation & benchmarks · 685 works

Cite

Cite as Paul Michel, Xian Li, Graham Neubig and Juan Pino. 2019. On Evaluation of Adversarial Perturbations for Sequence-to-Sequence Models. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/n19-1314

@inproceedings{michel2019evaluation,
  title = {{On Evaluation of Adversarial Perturbations for Sequence-to-Sequence Models}},
  author = {Paul Michel, Xian Li, Graham Neubig and Juan Pino},
  booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)},
  year = {2019},
  doi = {10.18653/v1/n19-1314},
  url = {https://doi.org/10.18653/v1/n19-1314}
}

References

39 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani Srivastava, and Kai-Wei Chang. 2018. Generating natural language adversarial examples. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2890–2896. our card
  2. Yonatan Belinkov and Yonatan Bisk. 2018. Synthetic and natural noise both break neural machine translation. In Proceedings of the International Conference on Learning Representations (ICLR). find
  3. Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski, Bernhard Firner, Beat Flepp, Prasoon Goyal, Lawrence D Jackel, Mathew Monfort, Urs Muller, Jiakai Zhang, et al. 2016. End to end learning for self-driving cars. arXiv preprint arXiv:1604.07316. — [arXiv:1604.07316](https://arxiv.org/abs/1604.07316) find
  4. Daniel Cer, Mona Diab, Eneko Agirre, Inigo Lopez-Gazpio, and Lucia Specia. 2017. Semeval-2017 task 1: Semantic textual similarity multilingual and crosslingual focused evaluation. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 1–14. our card
  5. Mauro Cettolo, Jan Niehues, Sebastian Stüker, Luisa Bentivogli, Roldano Cattoni, and Marcello Federico. 2016. The iwslt 2016 evaluation campaign. find
  6. Minhao Cheng, Jinfeng Yi, Huan Zhang, Pin-Yu Chen, and Cho-Jui Hsieh. 2018. Seq2sick: Evaluating the robustness of sequence-to-sequence models with adversarial examples. arXiv preprint arXiv:1803.01128. — [arXiv:1803.01128](https://arxiv.org/abs/1803.01128) find
  7. Moustapha Cissé, Piotr Bojanowski, Edouard Grave, Yann Dauphin, and Nicolas Usunier. 2017. Parseval networks: Improving robustness to adversarial examples. In International Conference on Machine Learning. find
  8. Michael Denkowski and Alon Lavie. 2014. Meteor universal: Language specific translation evaluation for any target language. In Proceedings of the EACL 2014 Workshop on Statistical Machine Translation. our card
  9. Javid Ebrahimi, Daniel Lowd, and Dejing Dou. 2018a. On adversarial examples for character-level neural machine translation. In COLING. our card
  10. Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. 2018b. Hotflip: White-box adversarial examples for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 31–36. our card
  11. Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014. Explaining and harnessing adversarial examples. Proceedings of the International Conference on Learning Representations (ICLR). our card
  12. Isao Goto, Ka-Po Chow, Bin Lu, Eiichiro Sumita, and Benjamin Ka-Yin T’sou. 2013. Overview of the patent machine translation task at the ntcir-10 workshop. In NII Test Collection for IR Systems. find
  13. Kathrin Grosse, Nicolas Papernot, Praveen Manoharan, Michael Backes, and Patrick McDaniel. 2016. Adversarial perturbations against deep neural networks for malware classification. arXiv preprint arXiv:1606.04435. — [arXiv:1606.04435](https://arxiv.org/abs/1606.04435) find
  14. Hossein Hosseini, Sreeram Kannan, Baosen Zhang, and Radha Poovendran. 2017. Deceiving google’s perspective api built for detecting toxic comments. arXiv preprint arXiv:1702.08138. — [arXiv:1702.08138](https://arxiv.org/abs/1702.08138) find
  15. Mohit Iyyer, Varun Manjunatha, Jordan Boyd-Graber, and Hal Daumé III. 2015. Deep unordered composition rivals syntactic methods for text classification. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural L… our card
  16. Mohit Iyyer, John Wieting, Kevin Gimpel, and Luke Zettlemoyer. 2018. Adversarial example generation with syntactically controlled paraphrase networks. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vo… our card
  17. Robin Jia and Percy Liang. 2017. Adversarial examples for evaluating reading comprehension systems. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2021–2031. our card
  18. Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Representations (ICLR). find
  19. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondřej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In Pro… find
  20. J. Zico Kolter and Eric Wong. 2017. Provable defenses against adversarial examples via the convex outer adversarial polytope. arXiv preprint arXiv:1711.00851. — [arXiv:1711.00851](https://arxiv.org/abs/1711.00851) find
  21. Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective approaches to attention-based neural machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1412–1421. our card
  22. Xiaoyi Ma and Christopher Cieri. 2006. Corpus support for machine translation at ldc. In Language Resources and Evaluation Conference. find
  23. Leo X McCusker, Philip B Gough, and Randolph G Bias. 1981. Word recognition inside out and outside in. Journal of Experimental Psychology: Human Perception and Performance, 7(3):538. find
  24. Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. 2016. Deepfool: A simple and accurate method to fool deep neural networks. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2574–2582. find
  25. Aakanksha Naik, Abhilasha Ravichander, Norman Sadeh, Carolyn Rose, and Graham Neubig. 2018. Stress test evaluation for natural language inference. In Proceedings of the 27th International Conference on Computational Linguistics, pages 2340–2353. find
  26. Nicolas Papernot, Patrick McDaniel, Ananthram Swami, and Richard Harang. 2016. Crafting adversarial input sequences for recurrent neural networks. In Military Communications Conference, MILCOM 2016-2016 IEEE, pages 49–54. IEEE. find
  27. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318. our card
  28. Maja Popović. 2015. chrf: character n-gram f-score for automatic mt evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395. our card
  29. Maja Popović. 2016. chrf deconstructed: beta parameters and n-gram weights. In Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers, pages 499–504. our card
  30. Matt Post. 2018. A call for clarity in reporting bleu scores. arXiv preprint arXiv:1804.08771. our card
  31. Sravana Reddy and Kevin Knight. 2016. Obfuscating gender in social media writing. In Proceedings of the First Workshop on NLP and Computational Social Science. Association for Computational Linguistics. find
  32. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2018. Semantically equivalent adversarial rules for debugging nlp models. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 856–865. find
  33. Suranjana Samanta and Sameep Mehta. 2017. Towards crafting text adversarial samples. arXiv preprint arXiv:1707.02812. find
  34. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715– 1725. our card
  35. Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research (JMLR), 15(1):1929–1958. find
  36. Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wojna. 2016. Rethinking the inception architecture for computer vision. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2818–2826. find
  37. Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. 2013. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199. our card
  38. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS), pages 5998–6008. our card
  39. Zhengli Zhao, Dheeru Dua, and Sameer Singh. 2018. Generating natural adversarial examples. In Proceedings of the International Conference on Learning Representations (ICLR). find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/michel-2019-evaluation-adversarial-perturbations-sequence-sequence/