“One false match can lead to missed flights, lengthy interrogations, watchlist placements, tense police encounters, false arrests, or worse.” That was Jay Stanley of the American Civil Liberties Union, reacting to a federal study that MIT Technology Review called the most comprehensive evaluation to date of how face recognition performs across race, age and sex. The test, from the National Institute of Standards and Technology, came out on December 19, 2019. Its answer was uncomfortable for an industry that sells to border agencies and police departments.
What was tested
NIST evaluated 189 software algorithms from 99 developers, a majority of the industry. It ran them on four collections of photographs drawn from State Department, Homeland Security and FBI databases: 18.27 million images of 8.49 million people. Mug shots, immigration applications, visa applications and border-crossing photos were all in the mix. NIST did not use pictures scraped from social media or video surveillance.
Each algorithm was put through two jobs. One-to-one matching asks whether a photo fits another photo of the same person, the check behind unlocking a phone or inspecting a passport. One-to-many searching asks whether a face matches anyone in a database, which is how investigators hunt for suspects.
Where the errors landed
For one-to-one matching, NIST found higher false-positive rates for Asian and African American faces than for Caucasian ones. The gap often ran from a factor of 10 to 100 times, depending on the algorithm. Among algorithms developed in the United States, similar spikes appeared for Asians, African Americans and native groups. American Indians saw the highest rates. Business Insider added that women were misidentified more often than men across the board.
One-to-many searching produced the finding with the sharpest legal edge. In a test against an FBI database of 1.6 million domestic mugshots, false positives were more common for African American women. As Patrick Grother, the report’s primary author, put it, a false positive in that kind of search “puts an incorrect match on a list of candidates that warrant further scrutiny.” A false negative, by contrast, is mostly an inconvenience. You try again.
There was an exception worth noting. Some algorithms developed in Asian countries showed no dramatic difference in false positives between Asian and Caucasian faces. Grother said NIST had not examined cause and effect, but called this “an encouraging sign that more diverse training data may produce more equitable outcomes.” He also said the most equitable algorithms were among the most accurate. Different algorithms, in other words, perform differently.
Who was missing, and what follows
Amazon was not among the 99 vendors. According to The Washington Post, as cited by Business Insider, the company did not make its software available for the study. Amazon has said its cloud-based service would have to be altered to run NIST’s test. Intel, Microsoft, Panasonic, SenseTime and Vigilant Solutions did take part.
Stanley called the findings evidence of a “dystopian technology” and urged agencies to stop using it. NIST took a different line. It left regulation to policymakers and asked developers to research how the biases could be reduced.
Sources
- NIST Study Evaluates Effects of Race, Age, Sex on Face Recognition Software (NIST)
- Facial-Recognition Technology Has a Racial-Bias Problem, Study Finds (Business Insider)
- A US government study confirms most face recognition systems are racist (MIT Technology Review)
