How we made machines racist and they in turn biased humans back
A cautionary tale of the blind spots of data and artificial intelligence
But who taught them to be biased?
Is this only about not finding the right fit? The significance of this incident is slightly beyond that, it’s regarding how many practices and processes are designed with keeping only one gender in mind. And how it affects breakthroughs such as this because the data that we collected designing those processes, we missed out on the other half.
Artificial Intelligence (AI) is making spectacular leaps: from voice-activated AI assistants to autopilots that can land a damaged plane during an emergency. But developing an AI algorithm comes with a responsibility many are not aware of – to watch out for instances where human cognitive biases can affect AI applications.
AI is still in its infancy in India, but the trend is clear. More and more applications will start incorporating smart algorithms, and as a society, we need to be aware of its limitations.
AI, or machine learning (ML) has in its heart an algorithm which collects vast amounts of examples and uses those examples to specify a correct output given an input. ML is best at recognizing patterns (facial recognition, identifying spoken words), recognizing anomalies (unusual credit card transactions) and predictive analysis (next Netflix show you are likely to watch). The autopilot for example records the actions of human pilots generating learning models, which then iteratively grows better till it’s able to mimic the human in landing a plane. In essence, the machine is always learning to continuously make itself perfect.
But it’s a machine, how can it have bias?
Machines by definition are supposed to be devoid of human emotions, in other words, be objective. But here comes the most essential question, the machine is learning but who is teaching it? There are two answers. One, behind every ML algorithm there is a set of data, data to train the algorithms to be perfect, data that is provided by humans.
And second, it is this human who is deciding which questions the machine should answer. Now take safety as an example. If law enforcement is able to predict which areas are more crime-prone would they not be able to allocate resources wisely? CrimeScan and PredPol are predictive policing softwares which analyse crime data to predict patterns of criminal behaviour. But here is what went wrong – in an alarming report, investigative watchdog ProPublica found that courts in the US used these AI tools to predict the likelihood of future crimes during sentencing, and they were significantly biased against African Americans.
Cognitive biases arise when people rely on their past experiences to make decisions. The mental shortcuts or stereotypes that often make us prone to showing biases. Confirmation bias is exhibited when one has a hypothesis in mind and looks for patterns to support or conform to that hypothesis. So if one’s experience has been that African Americans are more prone to violence, the underlying data set would unconsciously train on that human bias.
Tech start-ups in India and beyond are now pitching AI-led recruitment tools for hiring. The data that powers these algorithms is gathered from social media sites and other public data to identify candidates that best fit a profile. But affinity bias and an insufficient dataset can easily make these hiring practices the least objective.
Studies by MIT Media Labs showed how facial recognition algorithms discriminate based on classes such as race and gender. They analysed three commercial gender classification systems and found that darker-skinned females were the most misclassified group (error rates upto 35%), whereas the maximum error rate for lighter-skinned males is 0.8%. Perhaps one of the most alarming errors was committed when Google’s first facial recognition software tagged African American faces as gorillas, the training data had no information on how to identify darker skin tones.
It’s the maker’s fault!
Let’s take the most widely used example of AI – virtual assistants. Amazon’s Alexa, Apple’s Siri, and Microsoft’s Cortana were all launched with female voices. Male assistants such as IBM’s Watson, Salesforce’s Einstein, and Samsung’s Bixby were built to assist in grander business strategies and complex problem-solving. It’s not just that it exhibits bias of the maker but it reinforces the stereotype of a female assistant taking orders from the user as well.
Applications are slowly and steadily getting into sensitive territories. Affectiva helps in identifying emotions from images of people’s faces. Researchers Kosinski and Wang claimed to determine the sexuality of Caucasian males from profile pictures on Facebook or dating sites. Israeli start-up Faception has developed software that helps determine an individual’s characteristics – propensity towards crime and even terrorism. These are spaces where an error in judgement can have deep consequences.
Microsoft’s conversational AI-bot Tay used data from Twitter conversations and began repeating racist and misogynist phrases in less than twenty-four hours. We need to learn from these soon.
Responsible AI
San Francisco-based think tank Open AI published a paper titled “AI Safety Needs Social Scientists”. While going through the paper one thing that stood out most to me is why AI needs empathy.
IBM recently stated that they are planning to fight bias in facial recognition with a new diverse dataset, obtained through Flickr. MIT’s Joy Buolamwini and her Algorithmic Justice League develop practices for accountability during the design and development of machine learning codes. India is still far behind in conversations of this kind. The instances where AI has the potential to deepen social fissures have particular significance in India. With religion, caste, economic and even sexual-orientation-based discrimination, the blind adoption of AI has the potential to further fragment our society.
Let’s take a cue from Dall E, an AI system creates highly realistic art from a description given by a user. Dall E 2’s “a painting of a fox sitting in a field at sunrise in the style of Claude Monet” will practically take your breath away. But here is what Dall E 2 creators had to specify to let Dall E not get violent or offensive.
“Preventing Harmful Generations
We’ve limited the ability for DALL·E 2 to generate violent, hateful, or adult images. By removing the most explicit content from the training data, we minimized DALL·E 2’s exposure to these concepts. We also used advanced techniques to prevent photorealistic generations of real individuals’ faces, including those of public figures.”
