Computer Model Makes Light Work of Telling Faces Apart

There currently are 7.9 billion human faces on the planet. All are variations on the same template: two eyes flanking a nose above a mouth. Yet with a mere glance, most of us can tell the difference between any two faces. How do our brains make these lightning-fast judgments?

Spoiler alert: No one knows. And although computer programs today are excellent at putting names to faces, none is particularly good at judging just how similar (or dissimilar) different faces appear. But in a study published today in the Proceedings of the National Academy of Sciences (PNAS), an international team of researchers testing object-recognition programs report clues about the kinds of computations brains might be making when assessing the degree of similarity between faces.

“Distinguishing faces from one another is a very very fine visual discrimination, yet we are masters at it even when the parts of faces, like noses and cheeks, can look quite alike,” said Kamila M. Jozwik, PhD, of the University of Cambridge and the first lead author of the paper. “Our study could help us understand how we recognize and perceive people, which influences how we think about them and treat them.”

“The ability to notice differences even in the same face, as in facial expressions or in familiar faces that have aged, influences our emotions and how we interact with people,” added Katherine R. Storrs, PhD, of the Justus Liebig University Giessen, Germany, and the second lead author of the study.

In their paper, Drs. Jozwik and Storrs, Nikolaus Kriegeskorte, PhD, a principal investigator at Columbia’s Zuckerman Institute and senior author of the study, and several coauthors, identified a “surprisingly simple” computer model that proved to be quite good at gauging facial differences.

“We hope this provides us with theoretical insight into the computational process our brains make when recognizing familiar faces or encountering new ones,” said Dr. Kriegeskorte.

Toward that end, the researchers recruited 26 undergraduates at the University of Cambridge and asked them to rank many pairs of realistic computer-generated faces (based on scans of real faces) with respect to how similar the two faces in each pair appeared.

The researchers then tasked 16 different computer programs, each running a model that represented faces in a different way, to make the same face similarity ratings. Some of the models represented faces as digital images, massive arrangements of pixels. Some models relied on geometric meshes, whose facets can be adjusted to represent faces. Still others made comparisons of facial landmarks. Finally, some models used information about textures and the shapes of features such as eyebrows, or were themselves based on artificial intelligence systems.

“We were looking for a computer model that would make the same judgments people do when comparing faces,” said Dr. Jozwik. “That would put us in a great starting place to ask how the brain compares different faces.”

The researchers found that two types of models were best at replicating the students’ similarity rankings. One type, deep neural networks (DNNs), is used on our mobile phones to recognize faces in photos and is often depicted in movies and TV shows whose storylines include AI.

Programmers train DNNs to recognize an object, say, a cat or a human face, with galleries of digital images that have previously been annotated by a person as being examples of the object of interest. The training phase continuously readjusts the sequence of calculations that DNNs make until these artificial intelligence programs can spot the target object in new images presented to the models. In the study, the DNN models compared the many thousands of pixels comprising different faces. From those comparisons, the DNNs calculated similarity rankings between the same face pairs the students rated.

The other type of program that was especially good at replicating the student’s facial similarity judgements was derived from the Basel Face Model (BFM). Think of the BFM as a kind of digital clay for faces. By massaging various portions of digital faces (scanned from real persons), it becomes possible to morph a face, more or less subtly, into a different face; and shapes and textures can be precisely and mathematically specified. For the PNAS study, researchers created pairs of faces from this BFM model and asked students to arrange them on a large computer touch screen according to how similar the face pairs appear.

The most striking result, the researchers said, is that the BFM was as good as the far more computationally intensive DNN models in replicating the facial-similarity perceptions of the students. This result suggests that the types of statistical variations between faces assessed by the BFM model are important to our brains, said Dr. Storrs.

The researchers stress that their study has limitations. For one thing, the BFM was built by researchers in Basel, Switzerland, based on scans of 200 mostly young, White faces. “The natural variation in a population of faces is different for different people in different places,” said Dr. Kriegeskorte. Unavailable at the moment are tools and datasets that are representative of the world’s facial diversity. That limits the confidence the researchers can currently have that their work does, in fact, point toward the brain’s own computational techniques for assessing faces.

“Our hope is that these findings can guide us toward research questions and methods that will unveil more precisely where and how in the brain this crucial information-processing task is going on,” said Dr. Kriegeskorte. “We also hope research like ours will help us understand the inner workings and shortcomings of artificial intelligence systems for recognizing faces, which are becoming more prevalent in our technological landscapes.”  

Reference: Jozwik KM, O’Keeffe J, Storrs KR, Guo W, Golan T, Kriegeskorte N. Face dissimilarity judgments are predicted by representational distance in morphable and image-computable models. PNAS. 2022;119(27):e2115047119. doi: 10.1073/pnas.2115047119

This article has been republished from the following materials. Note: material may have been edited for length and content. For further information, please contact the cited source.