Gender and racial bias in computer vision
There is a wide consensus among the AI community that bias is undesirable and that it may lead to very harmful consequences for users. So much has been written on building fair, transparent, and explainable AI. Then why do we keep hearing news of racial and gender bias in AI over and over again? What are we doing wrong?
In the case of tabular data, bias may be easier to spot and handle by using “blind taste tests” and by excluding variables which may lead to biased outcomes. However, when it comes to unstructured data such as images and video, current supervised learning approaches rely on patterns and annotations where bias is much more difficult to eradicate.
Bias in computer vision is a hard problem to solve also because AI, like its human creators, may be led to judge by appearance. When it comes to protected attributes such as race and gender, visual appearances may end up being tied to harmful assumptions and stereotypes, especially in controversial applications of AI such as surveillance and profiling.
There are tons of blunders produced by biased computer vision systems. Take for example the case of Google Photos tagging human faces as “gorillas” or Flickr tagging them as “ape”, as a result of the lack of ethnic balance in face detection models.
Another famous case is the saliency detection model used by Twitter which prioritizes white faces over black faces when deciding how photos are cropped. Zoom has also faced backlash against its virtual backgrounds feature which failed to recognize a black professor’s head and was automatically erasing it.
Models can also learn harmful and stereotypical correlations between humans and their context. Recently, AlgorithmWatch showed that Google Cloud Vision labelled an image of a dark-skinned individual holding a thermometer as “gun” while a similar image with a light-skinned individual was labelled “electronic device”.