‘Averaged’ Image Could Boost Computer Face Recognition Accuracy

Psychologists Rob Jenkins and Mike Burton of the University of Glasgow found that by feeding a commercially available face recognition computer program these “averaged” faces rather than individual photos, they could increase its accuracy from 54 to 100 percent, according to a study published Friday in the journal Science.

But computer face recognition experts, including those at the company that makes the software used by the researchers, caution that technical challenges could make the research difficult to apply on a commercial scale.

Border control security systems that rely on computer face recognition are already being rolled out in airports worldwide. The Australian customs service, for instance, has used a system called SmartGate to speed Qantas airline staff through customs since 2002, and expanded the service to some Qantas frequent flyers in 2004.

These systems work by storing an electronic image of each traveler in a large database, then taking a picture of the travelers’ face each time he or she goes through customs and comparing it to the stored image.

The difficulty such systems must overcome is that two pictures of the same person can look very different. Differences in facial expression, lighting and the angle of the head make it hard for both computers and humans to figure out whether two photos truly depict the same person.

For now, most face recognition systems try to overcome this problem by carefully controlling variables like lighting and head position in both database and identification photos.

But Jenkins and Burton thought there might be a better solution. The researchers usually study human face perception — not computer face recognition. But it turns out that humans and computers struggle with the same problem. Humans are not particularly good at recognizing the same face in different pictures, at least not if that face belongs to a stranger.

“We’re a lot worse at it than we think,” Jenkins says, “and I think part of the reason for that is that we’re so good at recognizing familiar faces.”

We can recognize family, friends and acquaintances in photographs taken at any age and under almost any condition. But studies have shown that when people are asked to match a photo of a stranger to another picture of that stranger — shown in a field of photos of many strangers — they’re only accurate about 70 percent of the time.

“You go from that, being unable to match two pictures taken under good conditions, to being able to recognize someone you know in any old grotty picture,” says cognitive psychologist Peter Hancock, of the University of Stirling in Britain. “So the question is: How on earth do we do that?”

Previous research by Hancock and Jenkins suggested that we may do it, at least in part, by forming our own “average” internal image of the faces of people we know. This average wipes out the variations caused by bad lighting and funny angles, leaving only the salient, identifying aspects of a person’s face.

In a 2005 study, Hancock and Jenkins created an approximation of this internal average by averaging pixel intensity and face shape information, such as the distance between eyes and the position of the tip of the nose, from many photos of the same person. They found that people were much better at matching an averaged image of a stranger to another photo of that stranger than they were at matching two individual photos.

“When we saw that we thought, ‘it looks like we’re really capturing something,'” Jenkins says. “Then we started thinking — computers fall down too when they confront this kind of issue […] maybe we can also boost machine performance.”

In the new study, Jenkins and co-author Mike Burton tested the commercially available system FaceVACS, developed by the German company Cognitec Systems. FaceVACS software underlies Australia’s SmartGate system, but it also underlies a playful application on the Web site MyHeritage.com. The site allows users to upload personal photos, then compares the photos to a database of 31,000 pictures of more than 3,600 celebrities, and tells users which celebrity they most resemble.

The researchers tested the software by feeding it photographs of 25 celebrities who they knew were included in the database. If the computer returned a photo of the same celebrity, it was a match — if it returned a different celebrity’s photo, it was an error.

The researchers tested 500 photographs (20 each of the 25 celebrities) and found that the software had a 54 percent accuracy rate. But when they combined the 20 photos into one average image for each celebrity and fed the program the averaged images instead, that accuracy rated spiked to 100 percent.

“This is the first time getting anything like that performance using naturally varying photos,” Jenkins says.

Frank Weber, the head of Cognitec’s algorithm development group, says that the research is impressive.

“It’s clearly truly remarkable what they achieved here,” he says. But, he cautions, the research could be difficult to apply on a large scale, because the process of averaging the photos requires manual work–the photos have to be resized and rotated so that they line up correctly.

Jenkins also says he’d like to see that part of the process automated as well. That “would be a fantastic future development,” he says.

Support PBS NewsHour: