Home | News | Editorials

Facial Recognition and How it Works

By Susan Gonzales and Zachary Solomon

Facial recognition seems like the stuff of science fiction. Technology that’s capable of identifying a human face from a photograph or video and matching it from among hundreds of millions of other faces stored in a database of faces—if that doesn’t sound futuristic, we don’t know what does.

Here’s how it works: facial recognition systems use artificial intelligence algorithms (step-by-step instructions written by a programmer to be followed by a computer to solve a problem) to identify specific details in a person’s face in an image, details such as the distance between one’s eyes or the distance between their ears. That image is then cropped to size, converted into a format that is easily readable by a search engine, and then matched against all the other images in a database. Sometimes, if the algorithm can’t find a positive match, it will suggest several options—images of people who look similar enough to the person in the original image.

For a real-life example of facial recognition, look to the January 6th attack on the U.S. Congress.. The FBI, which itself has access to over 640 million photos of American citizens in its facial recognition database, has been working to identify the hundreds of people who terrorized the Capitol Building. And they have had some success. Using facial recognition technology, law enforcement has been matching faces from the images and videos taken during the insurgency—often by the insurgents themselves—against the faces in the various databases available to them.

With law enforcement, facial recognition technology has the potential to be a force for good in the world. Being able to identify criminals through a random image or blurry video using intelligent algorithms can make justice swifter and safer.

Facial recognition technology can help identify participants of the mob inside the Capitol Building — even with masks. Photo: Saul Loeb/Agence France-Presse — Getty Images

But unfortunately, it’s not all roses. The common belief is that facial recognition can be a tool for good as long as it is heavily regulated. Facial recognition tech has a lot of room for improvement. There have been numerous reports of inaccurate matches; people are routinely misidentified, especially people of color and women. A recent study showed that a popular facial recognition system was incorrect 34.7% of the time when it came to identifying black women. And last year, the city of San Francisco banned the use of facial recognition technology, citing its potential abuse.

There are countless examples where facial recognition has gone wrong. Elected officials and community leaders are demanding changes based on the unintended fallout of facial recognition tech on people of color and women in particular. Improvements to the technology are expected in the coming months and years that may address these issues. Stay tuned.

For more detailed information about facial recognition technology and bias, check out the Algorithmic Justice League’s primer and the ACLU factsheet.

Join us.

To stay informed about the ways in which AI is affecting you and your community, sign up for our newsletter. We can't have the conversation without you.

Our Partners:

AIandYou proudly partners with a diverse group of scientists, researchers and engineers to amplify their work, support their programs and create a platform for them to discuss AI with the community.

Black in AI
Latinx in AI
Queer in AI
Women in Machine Learning
Women in AI

Proudly Supported By:

Amazon
Chan Zuckerberg Initiative
Microsoft

We are proud to be sponsored by some of the world's leaders in AI and AI-related fields. These organizations are drawing the maps for an unknown world. By recognizing the need to engage communities of color, these partners are ensuring a more equitable AI future for everyone.

Become a Sponsor