Why do biases exist? We tend to form biases towards something for particular reasons we believe are right or correct. We categorize new information we obtain to either be positive or negative. This happens through our experiences or from additional information we have gathered from our surroundings, such as news, books, and word of mouth. We form these biases and store them in our minds like a preference list. We believe that information that ranks highest on our list of preferences will grant us the best satisfaction and outcome.
In this coming age of technology, we have taught and programmed machines to form and store biases within their memory so that they may serve humanity for a better future. Unfortunately, certain machines, which we will call Weapons of Math Destruction (WMD), in accordance with Cathy O’Neil’s work, “Weapons of Math Destruction,” are causing harm to our society.
WMDs are causing harm to our society by increasing the inequality between the privileged and underprivileged. One WMD that O’Neil presented was a teacher assessment tool known as IMPACT. IMPACT assessed teachers based on their students’ scores. Teachers that ranked very low on the assessment were fired from their teaching position. Teachers who were negatively affected by this WMD believed that the algorithm was unfair. IMPACT focused solely on the students’ scores. Still, sometimes, low scores do not necessarily mean that the teacher is horrible at teaching. Low scores can be due to several factors such as living in a bad home environment and personal troubles.
From the previous paragraph, IMPACT is biased towards students’ scores. It takes no account of each of the student’s backgrounds. According to O’Neil, “Models are opinions embedded in mathematics.” AI models, such as IMPACT, do not accurately represent reality itself because they are merely abstractions based on math. Some may believe that IMPACT serves its purpose and has helped the school system significantly. In contrast, others believe that IMPACT fails on its objective. Biases are opinions. Each of us as humans have different biases and different preference list in our minds. Some of our biases may be similar, and some of us may be completely different; it is what makes us unique. The programmers who created the IMPACT algorithm have different biases, and they incorporated their biases into IMPACT’s algorithm.
When creating a model that is aimed towards a certain group of people, it is important to ensure that the programmers communicate with the target group to understand their backgrounds. Programmers should not only focus on the correctness and efficiency of an AI model’s algorithm, but they should also think about how it could affect groups that will be affected by the AI model. Without this understanding, WMDs will continue to cause long-term harm to countless individuals in our society.