Let’s say there is a population of dogs living in Puppy City; a new job as a customer support representative at a company has opened up, and the company wants to use machine learning to making a hiring decision. By using machine learning, the company says, they’ll be able to reduce bias in the hiring manager’s decision making process.
But there’s a problem — Puppy City anti-discrimination law prohibits hiring on the basis of dog breed. So, the company makes a simple fix: they simply won’t include dog breed as a feature in the machine learning model. Thus, they have marked dog breed as a protected attribute. This strategy is known as “fairness through unawareness”.
However, the problem of redundant encodings emerges — features in the dataset that account for the protected attribute. For instance, the company may hire based on two main features:
- Dog college GPA (they’ll need to be smart & think quick)
- Dog speaking pitch and tone (they’ll be talking to customers all day)
However, (as an example) German Shepherds have a deeper voice while chihuahuas have a higher voice. In dog college, German Shepherds generally get a better GPA than chihuahuas because of inequities in dog society. Hence, it is entirely possible to predict the dog breed based on existing features.
Given that these complex models often obscure their complicated internal thinking processes, how are we to know whether or not the model is relying on dog breed to make a decision?
Demographic parity is a method to measure if protected attributes are really being protected. Simply put, there should be no correlation between the target attribute y and the protected attribute p. As an equation:
probability y=1 given p=0 is equal to probability y=1 given p=1
(Note — we’re assuming y and p are binary here.)
There are two problems measuring fairness by demographic parity:
- It doesn’t ensure fairness. We can just accept unqualified candidates or reject qualified candidates in p=0 and p=1 to make the percentages match up. This can happen easily if there is lack of training data in one group.
- It restricts performance. Despite the fact that we need machine learning models to be fair, they need to perform well. What if p really does provide significant information to predict y? — optimizing demographic parity will shoot the performance down significantly.
In “Equality of Opportunity in Supervised Learning”, Moritz Hardt, Eric Price, and Nathan Srebro suggest another approach to protecting protected attributes. It seeks to address the two limitations of demographic parity by allowing for high performance while preventing discrimination with respect to the protected attribute p.
They introduce the concept of equalized odds, which states that
probability y = 1 given p = 0 and actual y = Y (the real target)
is equal to
probability y = 1 given p = 1 and actual y = Y (the real target)
Essentially, the model should have the same true positive rate and false positive rate between p=0 and p=1. When it predicts y=1 (usually something like “yes, make the hire” or “yes, grant the loan”), it should make as many correct predictions as false predictions in each group.
Equalized odds is a more nuanced view of fairness than demographic parity or “fairness by unawareness”; it allows for the model to use the information in the protected attribute p while preventing the model from abusing p in a discriminatory fashion.
By optimizing the model on equalized odds, the model will be fairer in respect to p while capable of reaching great performance.
This is a more statistical view of allowing for fairness with respect to a protected attribute; however, it has a machine-learning flavor to it in that it also allows models to extract more latent insights from a protected attribute. While it’s certainly not a perfect solution, it certainly is an effective one.
A variant of equalized odds, equal opportunity, is now a standard for measuring fairness in machine learning models.
Read the full paper here: Equality of Opportunity in Supervised Learning