Elamur Research
☰
home / papers / 2016 / Equality of Opportunity in Supervised Learning

Equality of Opportunity in Supervised Learning

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive license to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/1610.02413

Summary

The paper asks how to formally check, and correct, whether a supervised predictor treats a protected group unfairly. The authors argue that ignoring the protected attribute fails because other features can encode it indirectly. They also argue that demographic parity, which requires equal acceptance rates, can still be unfair and can sharply reduce utility. Their alternative is equalized odds: predictions must be independent of the protected attribute once the true outcome is known. For binary tasks, this means matching true positive and false positive rates across groups. A weaker variant, equal opportunity, matches only true positive rates. Both criteria are oblivious, meaning they use only the joint statistics of prediction, outcome and group. Any existing score can be made compliant through a cheap post-processing step that picks group-specific, possibly randomized, thresholds. The authors prove this step is optimal when applied to a Bayes-optimal score. They also show that no oblivious test can tell apart some causal structures that differ in intuitive fairness. A FICO credit-score case study shows the criteria keep much more profit than demographic parity.

Key points

  • Equalized odds always allows a perfect predictor (Ŷ = Y), so it fits the usual supervised-learning goal of higher accuracy. Demographic parity rules this out whenever the outcome correlates with group membership.
  • In the binary case, the full joint distribution of (Ŷ, A, Y) has only 8 parameters. This makes the criteria easy to estimate and audit from aggregate data, potentially even under Differential Privacy.
  • The post-processing method chooses operating points from per-group ROC curves:
    • Equal opportunity selects points with the same true positive rate across groups.
    • Equalized odds selects a point lying under every group's curve, which pushes decision makers to build models that are accurate for all groups.
  • The FICO case study covers four groups (Asian, White, Hispanic, Black). It compares five lending policies: max profit, a single race-blind threshold, demographic parity, equal opportunity and equalized odds.
  • The authors present the framework as a way to detect and measure possible bias, not as proof of fairness. They stress the need for reliable target labels and domain-specific scrutiny.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

159 papers of our library cite it (of the 140673 whose bibliographies we read); 17 of them have a page here, 142 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1610.02413 · arXiv 1610.02413

@misc{hardt2016equality,
  title = {{Equality of Opportunity in Supervised Learning}},
  author = {Hardt, Moritz and Price, Eric and Srebro, Nathan},
  year = {2016},
  doi = {10.48550/arxiv.1610.02413},
  eprint = {1610.02413},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1610.02413}
}

References

13 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P Gummadi. Learning fair classifiers. CoRR, abs:1507.05259, 2015. find
  2. T. Calders, F. Kamiran, and M. Pechenizkiy. Building classifiers with independency constraints. In In Proc. IEEE International Conference on Data Mining Workshops, pages 13–18, 2009. find
  3. Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard S. Zemel. Fairness through awareness. In Proc. ACM ITCS, pages 214–226, 2012. find
  4. Michael Feldman, Sorelle A. Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. Certifying and removing disparate impact. In Proc. 21st ACM SIGKDD, pages 259–268. ACM, 2015. find
  5. Jon M. Kleinberg, Sendhil Mullainathan, and Manish Raghavan. Inherent trade-offs in the fair determination of risk scores. CoRR, abs/1609.05807, 2016. — [arXiv:1609.05807](https://arxiv.org/abs/1609.05807) find
  6. Christos Louizos, Kevin Swersky, Yujia Li, Max Welling, and Richard S. Zemel. The variational fair autoencoder. CoRR, abs/1511.00830, 2015. — [arXiv:1511.00830](https://arxiv.org/abs/1511.00830) find
  7. John Podesta, Penny Pritzker, Ernest J. Moniz, John Holdren, and Jefrey Zients. Big data: Seizing opportunities and preserving values. Executive Office of the President, May 2014. find
  8. Dino Pedreshi, Salvatore Ruggieri, and Franco Turini. Discrimination-aware data mining. In Proc. 14th ACM SIGKDD, 2008. find
  9. US Federal Reserve. Report to the congress on credit scoring and its effects on the availability and affordability of credit, 2007. find
  10. Andrea Romei and Salvatore Ruggieri. A multidisciplinary survey on discrimination analysis. The Knowledge Engineering Review, 29:582–638, 11 2014. Larry Wasserman. All of Statistics: A Concise Course in Statistical Inference. Springer, 2010. find
  11. Big data: A report on algorithmic systems, opportunity, and civil rights. Executive Office of the President, May 2016. find
  12. Indre Zliobaite. On the relation between accuracy and fairness in binary classification. CoRR, abs/1505.05723, 2015. — [arXiv:1505.05723](https://arxiv.org/abs/1505.05723) find
  13. Richard S. Zemel, Yu Wu, Kevin Swersky, Toniann Pitassi, and Cynthia Dwork. Learning fair representations. In Proc. 30th ICML, 2013. find