Elamur Research
☰
home / papers / 2018 / Adversarial Removal of Demographic Attributes from Text Data

Adversarial Removal of Demographic Attributes from Text Data

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/D18-1002.pdf

Summary

Classifiers trained on text can quietly absorb author demographics, which is a fairness risk for decisions such as hiring or credit scoring. The authors test whether adversarial training can strip race, gender and age from neural text representations. Using Twitter data, they build two harmless binary main tasks, emoji-derived sentiment and @mention detection. They pair these with protected labels from the DIAL corpus (a dialect-based race proxy) and PAN16 (age, gender). A one-layer LSTM encoder with MLP heads is trained jointly with an adversary through a gradient-reversal layer. Afterwards, a separate attacker network is fit on the frozen representations to measure how much of the attribute remains. Demographic signals turn out to be encoded even for unrelated tasks on balanced data. Adversarial training reduces this leakage, but although the adversary falls to chance level during training, the post-hoc attacker still recovers the attribute well above 50%. Stronger or more heavily weighted adversaries and adversary ensembles help only partially. The lesson is that adversary accuracy does not prove invariance, so results must be checked with an independent attacker.

Key points

  • Data: DIAL (Blodgett et al., 2016) contains 59.2M tweets from 2.8M users. AAE vs. SAE labels with confidence above 80% serve as proxies for Black vs. White authors, giving 166K train and 10K dev tweets. PAN16 covers 436 users annotated via LinkedIn, giving 160K train and 10K dev tweets. Every split is balanced on both main and protected labels, so chance accuracy is 50%.
  • Model: The encoder is an LSTM and the classifier and adversary are MLPs with a tanh hidden layer. Gradient reversal (GRL, Ganin & Lempitsky 2015) uses a strength parameter λ. The attacker reuses the same train/dev split, which is the hardest setting for it.
  • Gap on a 10% held-out set: The attacker beat the adversary's accuracy by 12.2 points (DIAL, sentiment/race), 14.3 (DIAL, mention/race), 8.1 (PAN16, mention/gender) and 9.7 (PAN16, mention/age).
  • Mitigation and theory: Tuning adversary capacity and weight, and ensembling adversaries, improves removal, but no method works best in every case. An appendix shows that a truly guarded predictor would meet three fairness criteria: parity of outcomes across demographic groups, equal odds, and equal opportunity.
  • Code: github.com/yanaiela/demog-text-removal

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

120 papers of our library cite it (of the 140673 whose bibliographies we read); 7 of them have a page here, 113 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d18-1002

@inproceedings{elazar2018adversarial,
  title = {{Adversarial Removal of Demographic Attributes from Text Data}},
  author = {Elazar, Yanai and Goldberg, Yoav},
  booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
  doi = {10.18653/v1/d18-1002},
  url = {https://doi.org/10.18653/v1/d18-1002}
}

References

36 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Omar Ghailan, Hoda MO Mokhtar, and Osman Hegazy. 2016. Improving credit scorecard modeling through applying text analysis. institutions, 7(4). find
  2. Alex Beutel, Jilin Chen, Zhe Zhao, and Ed H Chi. 2017. Data decisions and theoretical implications when adversarially learning fair representations. arXiv preprint arXiv:1707.00075. — [arXiv:1707.00075](https://arxiv.org/abs/1707.00075) find
  3. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in neural information processing systems, pages 2672–2680. find
  4. Su Lin Blodgett, Lisa Green, and Brendan O’Connor. 2016. Demographic dialectal variation in social media: A case study of african-american english. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1119–1130, Austin, Texas. Association for Computational… find our card
  5. Moritz Hardt, Eric Price, Nati Srebro, et al. 2016. Equality of opportunity in supervised learning. In Advances in neural information processing systems, pages 3315–3323. find our card
  6. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in Neural Information Processing Systems, pages 4349–4357. find our card
  7. Dirk Hovy, Anders Johannsen, and Anders Søgaard. 2015. User review sites as a resource for large-scale sociolinguistic studies. In Proceedings of the 24th International Conference on World Wide Web, pages 452–461. International World Wide Web Conferences Steering Committee. find
  8. John D Burger, John Henderson, George Kim, and Guido Zarrella. 2011. Discriminating gender on twitter. In Proceedings of the conference on empirical methods in natural language processing, pages 1301–1309. Association for Computational Linguistics. find
  9. Ryan Kiros, Yukun Zhu, Ruslan R Salakhutdinov, Richard Zemel, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015. Skip-thought vectors. In Advances in neural information processing systems, pages 3294–3302. find
  10. Moshe Koppel, Shlomo Argamon, and Anat Rachel Shimoni. 2002. Automatically categorizing written texts by author gender. Literary and linguistic computing, 17(4):401–412. find
  11. Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186. find
  12. Yitong Li, Timothy Baldwin, and Trevor Cohn. 2018. Towards robust and privacy-preserving text representations. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 25–30. Association for Computational Linguistics. find
  13. Xilun Chen, Yu Sun, Ben Athiwaratkun, Claire Cardie, and Kilian Weinberger. 2016. Adversarial deep averaging networks for cross-lingual sentiment classification. arXiv preprint arXiv:1606.01614. find
  14. Alexis Conneau, Guillaume Lample, Marc’Aurelio Ranzato, Ludovic Denoyer, and Hervé Jégou. 2017. Word translation without parallel data. arXiv preprint arXiv:1710.04087. find
  15. Christos Louizos, Kevin Swersky, Yujia Li, Max Welling, and Richard Zemel. 2015. The variational fair autoencoder. arXiv preprint arXiv:1511.00830. — [arXiv:1511.00830](https://arxiv.org/abs/1511.00830) find
  16. Harrison Edwards and Amos Storkey. 2015. Censoring representations with an adversary. arXiv preprint arXiv:1511.05897. — [arXiv:1511.05897](https://arxiv.org/abs/1511.05897) find
  17. Henry B Mann and Donald R Whitney. 1947. On a test of whether one of two random variables is stochastically larger than the other. The annals of mathematical statistics, pages 50–60. find
  18. Bjarke Felbo, Alan Mislove, Anders Søgaard, Iyad Rahwan, and Sune Lehmann. 2017. Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm. In Conference on Empirical Methods in Natural Language Processing (EMNLP). find our card
  19. Oren Melamud, Jacob Goldberger, and Ido Dagan. 2016. context2vec: Learning generic context embedding with bidirectional lstm. In Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pages 51–61. find our card
  20. Clément Feutry, Pablo Piantanida, Yoshua Bengio, and Pierre Duhamel. 2018. Learning anonymized representations with adversarial neural networks. arXiv preprint arXiv:1802.09386. — [arXiv:1802.09386](https://arxiv.org/abs/1802.09386) find
  21. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111–3119. find
  22. Yaroslav Ganin and Victor Lempitsky. 2015. Unsupervised domain adaptation by backpropagation. In International Conference on Machine Learning, pages 1180–1189. find
  23. Dong Nguyen, Rilana Gravel, Dolf Trieschnigg, and Theo Meder. 2013. ” how old do you think i am?” a study of language and age in twitter. In ICWSM. find
  24. Yuan Zhang, Regina Barzilay, and Tommi Jaakkola. 2017. Aspect-augmented adversarial networks for domain adaptation. Transactions of the Association of Computational Linguistics, 5(1):515–528. find
  25. Dino Pedreshi, Salvatore Ruggieri, and Franco Turini. 2008. Discrimination-aware data mining. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 560–568. ACM. find
  26. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2017. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2979–2989. find our card
  27. Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. arXiv preprint arXiv:1802.05365. find
  28. Francisco Rangel, Paolo Rosso, Ben Verhoeven, Walter Daelemans, Martin Potthast, and Benno Stein. 2016. Overview of the 4th author profiling task at pan 2016: cross-genre evaluations. In Working Notes Papers of the CLEF 2016 Evaluation Labs. CEUR Workshop Proceedings/Balog, Krisztian [edit.]; et al.… find
  29. Rachel Rudinger, Chandler May, and Benjamin Van Durme. 2017. Social bias in elicited natural language inferences. In Proceedings of the First ACL Workshop on Ethics in Natural Language Processing, pages 74–79. find our card
  30. Ben Verhoeven and Walter Daelemans. 2014. Clips stylometry investigation (csi) corpus: A dutch corpus for the detection of age, gender, personality, sentiment and deception in text. In LREC 2014-NINTH INTERNATIONAL CONFER-ENCE ON LANGUAGE RESOURCES AND EVAL-UATION, pages 3081–3085. find
  31. Ben Verhoeven, Walter Daelemans, and Barbara Plank. 2016. Twisty: A multilingual twitter stylometry corpus for gender and personality profiling. In LREC. find
  32. Rob Voigt, David Jurgens, Vinodkumar Prabhakaran, Dan Jurafsky, and Yulia Tsvetkov. 2018. Rtgender: A corpus for studying differential responses to gender. In LREC. find
  33. Edson RD Weren, Anderson U Kauer, Lucas Mizusaki, Viviane P Moreira, J Palazzo M de Oliveira, and Leandro K Wives. 2014. Examining multiple features for author profiling. Journal of information and data management, 5(3):266. find
  34. Qizhe Xie, Zihang Dai, Yulun Du, Eduard Hovy, and Graham Neubig. 2017. Controllable invariance through adversarial feature learning. In Advances in Neural Information Processing Systems, pages 585– 596. find
  35. Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. 2013. Learning fair representations. In International Conference on Machine Learning, pages 325–333. find
  36. Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. 2018. Mitigating unwanted biases with adversarial learning. arXiv preprint arXiv:1801.07593. — [arXiv:1801.07593](https://arxiv.org/abs/1801.07593) find