Deb Raji can’t remember the number of times she’s been forced to wait at airports because of her black skin.
She said she’s stood in front of the biometric face scanners at least three times — all part of the U.S. Customs and Border Patrol’s effort to ensure your passport does, indeed, belong to you — and that the machines have been unable to verify her identity. The problem is particularly acute when she’s gotten some sun, Raji said. Each time, she’s forced to move to another line, and wait for a border attendant to manually verify her passport matches her face.
“It’s like a mosquito bite level of annoyance,” Raji told Built In. “It’s more just the undercurrent of it, it’s a frustrating reminder of the fact that this technology was not built for you.”
Raji, a technology fellow at the New York City-based AI Now Institute, said she does not believe in the use of facial recognition under any circumstance.
But this week, she had reason to welcome influential tech companies’ actions on the controversial technology.
On Monday, IBM CEO Arvind Krishna wrote in a letter to Congress that the New York company would end its facial recognition program over fears the technology would be used in mass surveillance and racial profiling. On Thursday, Amazon CEO Jeff Bezos followed suit, saying the Seattle company would pause police use of its facial recognition technology for a year. Raji said these moves have been a long time coming.
Last year, more than 25 AI experts from Google, Facebook, Microsoft and more called for Amazon to stop selling Rekognition to law enforcement, saying it was biased against women and people of color. In September of 2019, IBM quietly discontinued its public-facing facial recognition model, Raji said, which allowed companies to plug into IBM’s API and use their tech for a fee. When IBM ended that program, she said the company kept its private contracts with businesses and governments. This week’s announcement presumptively ends those agreements, she said.
Scott Urban, founder of the Chicago-based Reflectacles, said he found IBM and Amazon’s announcements “very convenient” amid growing public suspicion over facial recognition tech.
Recently, sales of his anti-facial recognition glasses have been up drastically, Urban said. He attributes the increase to consumers’ privacy concerns with COVID-19 contact tracing, and citizen protests against police use of excessive force. Urban said he’s working on plans to introduce additional tech that helps people distance themselves from AI, machine learning and other systems that “want to make us into simple numbers with a profile.” He said that IBM was not a big player in the facial recognition arena, and that the move was motivated by profit margins.
“In the past five years, many smaller and more dedicated companies popped up in the facial recognition realm,” Urban wrote in an email to Built In. “I assume the newer, more dedicated companies would be able to take out a conglomerate like IBM in this regard.”
IBM and Amazon have long faced criticism over their facial recognition models, Raji said, particularly after the 2018 “Gender Shades” project famously called out the tech’s inability to identify black women, other minorities and continual misgender of trans people. The impact of these biased algorithms is disproportionately felt by people of color, Raji said.
She pointed to instances of police relying on the technology to make arrests, which has led to false convictions; immigration authorities leaning on the technology to surveil individuals they suspect of living in the country without documentation, which she said could lead to wrongful deportations; and companies using facial recognition during the hiring process, which could impact black and brown candidates’ chances of landing a job.
“The tech itself falls short, it is not a reliable source of identification in any situation right now and, because it’s so premature and a tech used so widely, it’s an alarming thing,” Raji said. “We don’t know enough about facial recognition. We just know it doesn’t work and you’re already using it for a vulnerable population. I think that it just needs to be put on pause.”
She hoped other tech companies will stop marketing the technology to private companies and governments. Raji also hoped other tech companies would look to IBM’s announcement as a model for what a response to the death of George Floyd should look like.
In the statements she’s seen, Raji said many tech companies fail to understand that racism does not happen in a moment, but that it’s a “felt reality” embedded in the U.S. tech industry and society. Although she said IBM is not perfect — pointing to the company’s alleged history of providing the Nazis technology during World War II, and what she said called a continual lack of citation to black women technologists — the fact that the company acknowledged its role in perpetuating systemic institutional racism should be held up.
“Just because you suddenly hear a scream doesn’t mean it hasn’t been here the whole time,” Raji said.
