📄 Trick Me If You Can: Human-in-the-Loop Generation of Adversarial Examples for Question Answering

Summary

Automatic adversarial attacks in NLP usually apply narrow, meaning-preserving tweaks such as typos, word swaps, or distractor sentences. These reveal only shallow weaknesses and scale poorly to paragraph-length inputs. Wallace et al. instead put people in charge of breaking models. Experienced Quizbowl question writers drafted trivia questions in an interactive interface. The interface displayed the model's top guesses, the point in the question where it first answered correctly, and word-level saliency highlights showing which words drove its prediction. Writers kept editing until the system failed while the question stayed fair for human players.

The work ran in two rounds, targeting an ElasticSearch-based IR baseline and a GRU-based neural classifier. The resulting adversarial questions were no harder for humans than ordinary ones. Strong QA systems, however, lost up to 40% relative accuracy on them and failed badly in live human–computer matches they normally win. The edits exposed phenomena such as multi-hop reasoning and misleading entity types. The work shows that guided human creativity can produce diverse, semantically grounded challenge data that rule-based or generative attacks cannot easily reach.

Key points

• Testbed: Quizbowl, whose pyramid-shaped questions move from obscure clues to easier ones. Models were trained on roughly 110,000 questions, and the answer space grew from 11,000 entities in round one to 25,000 in round two.
• Models attacked: the IR baseline from the NIPS 2017 Quizbowl shared task, and a bidirectional GRU RNN using 300-dimensional GloVe embeddings.
• Interpretations:
  • For the RNN, a gradient-based, first-order Taylor approximation estimates the effect of removing each word.
  • For the IR model, the ElasticSearch Highlight API provides the word scores.
  • Writers found that spotting overused "stock" clues was the most valuable feature.
• Main result: relative accuracy of strong QA systems dropped by as much as 40%, while human difficulty was unchanged.
• Motivation: an automatic syntactic paraphrase network produced valid Quizbowl paraphrases only about 3% of the time, which illustrates the limits of model-generated attacks on long texts.