Twitter got a lot of fanfare in 2018 when they announced that they were using machine learning to automatically crop photos on the site. In a lengthy blog post, they said the tool would automatically crop photos to focus on "saliency," which they explained was the most interesting part of a photo.

Google Brain researcher Lucas Theis and machine learning lead Zehan Wang said they used academic studies to figure out what part of a photo eyes generally focus on first.

“This data can be used to train neural networks and other algorithms to predict what people might want to look at,” the two wrote.

In response to the recent complaints, Twitter Chief Technology Officer Parag Agrawal said it was good that people spotted the problem and vowed to fix it.

This is a very important question. To address it, we did analysis on our model when we shipped it, but needs continuous improvement.

Love this public, open, and rigorous test — and eager to learn from this. https://t.co/E8Y71qSLXa

— Parag Agrawal (@paraga) September 20, 2020

But Twitter's Chief Design Officer Dantley Davis caused even more criticism when he defended the site's process, writing that Madland's original problems were occurring "because of the contrast with his skin."

"I know you think it's fun to dunk on me – but I'm as irritated about this as everyone else. However, I'm in a position to fix it and I will. It's 100% our fault. No-one should say otherwise," he later said.

While this may seem like a fairly mundane problem, it has roots that are already causing societal issues. The problem highlighted the bigger issue that tech firms face, which is that the lack of diversity leaves blindspots in technology that are only noticed once its deployed.

Facial recognition software is now used widely by police forces and armies across the world despite dozens of studies proving that it still cannot identify the faces of Black people, particularly Black women.

People have already been unfairly arrested due to mistakes made by facial recognition software, and there are dozens of other instances where algorithms with inherent racial bias were deployed with devastating consequences.

"Predictive policing" software is being rolled out despite outrage from scientists who say the technology is not ready. Other studies have proven that Black people were discriminated against by an algorithm widely used in U.S. hospitals to allocate health care to patients.

ProPublica released a groundbreaking study in 2016 of a software used in U.S. courts to "predict" recidivism, showing that it overwhelmingly discriminated against Black people.

Algorithmic-Bias

Get relevant content delivered to you once a week