Elamur Research
☰
home / papers / 2017 / Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints

Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/D17-1323.pdf

Summary

Vision tasks grounded in language often use structured predictors that exploit correlations between labels. Zhao et al. show that these models can pick up gender stereotypes from web-sourced training data and then exaggerate them. They examine two settings: visual semantic role labeling (vSRL) on imSitu and multilabel object classification (MLC) on MS-COCO. They define a bias score, the share of an output (a verb or object) co-occurring with a given gender. Amplification is measured as how far a model's predicted bias exceeds the training bias. Both datasets are strongly skewed, and CRF-based models widen the skew further.

To counter this, the authors propose RBA (Reducing Bias Amplification). This inference-time calibration adds corpus-level constraints that keep predicted gender ratios within a margin of the training statistics. The joint problem is solved with Lagrangian relaxation, which splits it into per-instance subproblems that the original decoder can handle. On the test sets, bias amplification drops by 40.5% for vSRL and 47.5% for MLC, with almost no accuracy cost. This matters because it offers a general, model-agnostic way to stop structured models from magnifying social bias.

Key points

  • Dataset skew: more than 45% of imSitu verbs and 37% of MS-COCO objects (80 classes) favor one gender by over 2:1. In the imSitu training data, man is the agent in 33% of cooking images, but a trained CRF predicts man in only 16%. With RBA, this rises to 20%.

  • Amplification measure: the bias score for an output–gender pair is its count with that gender divided by its count across all genders. Mean amplification averages the predicted-minus-training bias over pairs that were already biased in training.

  • vSRL test results (top-1 semantic role accuracy):

    ViolationsMean amplificationAccuracy
    CRF1490.04224.14
    CRF + RBA1020.02524.01

  • MLC test results (top-1 mean average precision):

    ViolationsMean amplificationmAP
    CRF380.04045.40
    CRF + RBA160.02145.38

  • Method details: RBA uses a 0.05 margin and works as a meta-algorithm over any structured predictor. It is weakest where the initial training bias is low. Code and data are released at github.com/uclanlp/reducingbias.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Semantic role labeling

Cited by

215 papers of our library cite it (of the 140673 whose bibliographies we read); 26 of them have a page here, 189 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d17-1323 · arXiv 1707.09457

@inproceedings{zhao2017men,
  title = {{Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints}},
  author = {Zhao, Jieyu and Wang, Tianlu and Yatskar, Mark and Ordonez, Vicente and Chang, Kai-Wei},
  booktitle = {Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing},
  year = {2017},
  doi = {10.18653/v1/d17-1323},
  eprint = {1707.09457},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/d17-1323}
}

References

34 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. 2015. Vqa: Visual question answering. In Proceedings of the IEEE International Conference on Computer Vision, pages 2425–2433. find
  2. Collin F Baker, Charles J Fillmore, and John B Lowe. 1998. The Berkeley framenet project. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pages 86–90. find our card
  3. Solon Barocas and Andrew D Selbst. 2014. Big data’s disparate impact. Available at SSRN 2477899. find
  4. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. find
  5. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In The Conference on Advances in Neural Information Processing Systems (NIPS), pages 4349–4357. find our card
  6. Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186. find
  7. Kai-Wei Chang, S. Sundararajan, and S. Sathiya Keerthi. 2013. Tractable semi-supervised learning of complex structured prediction models. In Proceedings of the European Conference on Machine Learning (ECML), pages 176–191. find
  8. Yin-Wen Chang and Michael Collins. 2011. Exact decoding of phrase-based translation models through Lagrangian relaxation. In EMNLP, pages 26–37. find
  9. Xinlei Chen, Hao Fang, Tsung-Yi Lin, Ramakrishna Vedantam, Saurabh Gupta, Piotr Dollár, and C Lawrence Zitnick. 2015. Microsoft coco captions: Data collection and evaluation server. arXiv preprint arXiv:1504.00325. find our card
  10. Bhavana Bharat Dalvi. 2015. Constrained Semi-supervised Learning in the Presence of Unanticipated Classes. Ph.D. thesis, Google Research. find
  11. Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. 2012. Fairness through awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, pages 214–226. ACM. find
  12. Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In European Conference on Computer Vision, pages 740–755. Springer. find our card
  13. G. Miller, R. Beckwith, C. Fellbaum, D. Gross, and K.J. Miller. 1990. Wordnet: An on-line lexical database. International Journal of Lexicography, 3(4):235–312. find
  14. Emiel van Miltenburg. 2016. Stereotyping and bias in the flickr30k dataset. MMC. find
  15. Ishan Misra, C Lawrence Zitnick, Margaret Mitchell, and Ross Girshick. 2016. Seeing through the human reporting bias: Visual classifiers from noisy human-centric labels. In Conference on Computer Vision and Pattern Recognition (CVPR), pages 2930–2939. find
  16. Ruslan Mitkov. 2014. Anaphora resolution. Routledge. find
  17. Nanyun Peng, Ryan Cotterell, and Jason Eisner. 2015. Dual decomposition inference for graphical models over strings. In Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 917–927. find
  18. John Podesta, Penny Pritzker, Ernest J. Moniz, John Holdren, and Jefrey Zients. 2014. Big data: Seizing opportunities and preserving values. Executive Office of the President. find
  19. Karen Ross and Cynthia Carter. 2011. Women and news: A long and winding road. Media, Culture & Society, 33(8):1148–1165. find
  20. Michael Feldman, Sorelle A Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. 2015. Certifying and removing disparate impact. In Proceedings of International Conference on Knowledge Discovery and Data Mining (KDD), pages 259–268. find
  21. Alexander M Rush and Michael Collins. 2012. A Tutorial on Dual Decomposition and Lagrangian Relaxation for Inference in Natural Language Processing. Journal of Artificial Intelligence Research, 45:305– 362. find
  22. Jonathan Gordon and Benjamin Van Durme. 2013. Reporting bias and knowledge extraction. Automated Knowledge Base Construction (AKBC). find
  23. Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. find our card
  24. Inc. Gurobi Optimization. 2016. Gurobi optimizer reference manual. find
  25. David Sontag, Amir Globerson, and Tommi Jaakkola. 2011. Introduction to dual decomposition for inference. Optimization for Machine Learning, 1:219– 254. find
  26. Moritz Hardt, Eric Price, Nati Srebro, et al. 2016. Equality of opportunity in supervised learning. In Conference on Neural Information Processing Systems (NIPS), pages 3315–3323. find our card
  27. Matthew Kay, Cynthia Matuszek, and Sean A Munson. 2015. Unequal representation and gender stereotypes in image search results for occupations. In Human Factors in Computing Systems, pages 3819– 3828. ACM. find
  28. Bernhard Korte and Jens Vygen. 2008. Combinatorial Optimization: Theory and Application. Springer Verlag. find
  29. Latanya Sweeney. 2013. Discrimination in online ad delivery. Queue, 11(3):10. find
  30. Benjamin D Van Durme. 2010. Extracting implicit knowledge from text. Ph.D. thesis, University of Rochester. find
  31. Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. 2015. Show and tell: A neural image caption generator. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3156–3164. find our card
  32. Mark Yatskar, Vicente Ordonez, Luke Zettlemoyer, and Ali Farhadi. 2017. Commonly uncommon: Semantic sparsity in situation recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). find
  33. Mark Yatskar, Luke Zettlemoyer, and Ali Farhadi. 2016. Situation recognition: Visual semantic role labeling for image understanding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5534–5542. find
  34. Indre Zliobaite. 2015. A survey on measuring indirect discrimination in machine learning. arXiv preprint arXiv:1511.00148. — [arXiv:1511.00148](https://arxiv.org/abs/1511.00148) find