USC News

Menu Search

Here’s scanning at you, kid – computers learn to recognize faces

A German-made “electronic doorman,” running Eidos software, scans a face and recognizes it.

Celebrities at the supermarket, take heed. A computer system developed at USC and in Germany may blow your cover – despite sunglasses, hats and other camouflage. The new system, the inventors say, can identify people’s faces as accurately as the human eye – and see past most disguises.

“It is not infallible,” said neurobiologist Christoph von der Malsburg, who led teams at USC and the Bochum University in the transatlantic collaborative effort which developed the system. “But neither are people in this task. In tests of specific abilities – a task like matching a recent photograph of a criminal suspect with those in mug books – our system works very well, comparable in all ways to even superior human ability.”

The system – named Eidos, after the Greek word for “essence” or “form” – works even if subjects have changed their hairstyles, grown or shaved beards or mustaches, or put on or taken off glasses. It works even if subjects are making faces in a deliberate attempt to distort their features. It becomes less accurate if less than a full-face view is available for analysis – as do human perceptions.

Von der Malsburg, who holds joint appointments in computer science and neurobiology, said Eidos has potential applications for immigration checkpoints, where it might help border guards identify wanted suspects; and in law enforcement, where it could compare composite pictures drawn by police artists with photos in mug books. A version might also be used at ATM machines to verify withdrawals. The U.S. Army is presently evaluating the system for possible uses, and a German spinoff firm has already licensed a version of the system – called ZN-FACE – to the giant Deutsche Bank for security applications.

Face recognition is now a heavily researched field. What distinguishes Eidos from a large number of alternative systems, however, is its success and its unique approach. Only one competitor – the “Photobook” system, developed by a team from the Massachusetts Institute of Technology’s Media Lab – is at a comparable commercial stage. But only Eidos uses a technique that mimics the way scientists believe the brain recognizes images.

Von der Malsburg’s principal research interests lie in the investigation of brain function. In fact, his original goal in developing Eidos was to study the mechanics of human face recognition. The system is the outgrowth of a computer model he created to replicate the way the brain’s visual cortex processes information.

Recognizing faces is a complex process. Some people endowed with a “photographic memory” for words and numbers have reported having trouble with faces. “They change too much,” said one famous memorizer.

For effective recognition, the viewer – whether man or machine – needs “to remove the meaningless variations found in any pair of images of the same subject: to extract the essence, the form of the face,” said Michael J. Lyons, a research associate in the Neural, Information and Behavioral Science Program, who works with von der Malsburg on Eidos and also lectures in the Department of Physics.

The computers used with the Eidos system “see” images much as people do. Focusing on any given point, the system creates what might be called a keyhole view – an image sharp and detailed at the center, but blurring off toward the sides.

The computer turns this view into an electronic proxy called a “wavelet transform.” If two keyhole views are similar (though not identical), a simple program that compares the wavelet transforms will show the likeness. Thus, it can compare any two points on any two images. In contrast, comparing all points would require processing impossible amounts of data and complicate the interpretation of the results.

In a two-step process, the Eidos system uses such comparisons between wavelet transforms to recognize faces. First, it uses them to recognize features on a given image of a face – to find what parts of the image are the nose, the eyes, the mouth and so forth.

For this step, the system draws on what Von der Malsburg calls “general face information,” which blends parts from many faces to create a composite of some 45 facial landmarks.

The spatial relationships among these 45 points are coded into the computer. Thus, the system knows that points representing the eyes must be above those representing the mouth; the point representing the left eye should be to the left of the one representing the nose; and so on.

Next, the Eidos program fits this gridwork to the image being analyzed – trying to find various points that most resemble, say, the wavelet-transform average left eye; then looking to the right of that point for one closest to the wavelet-transform average right eye. In the end, all or as many of the image points as possible are matched to the “average.”

When the features in another image have been similarly identified, the program can then compare apples to apples, oranges to oranges – or, in this case, left eyes to left eyes, right eyes to right eyes.


The computer’s task is made more difficult if some of the 45 landmarks it looks for are absent in one or both images – as in a profile of a face, which shows only one eye, one side of the nose, etc. But then, a partial view gives human identifiers problems, too.

And just like humans, the computer does better if it can work from multiple images of the same face.

Experts distinguish two different types of procedure for face recognition. One, known as identification, tests a single unknown photograph against a large number of others – like those in rogues’ galleries maintained by police.

A second procedure is verification, typically associated with a security system that checks an employee’s name or identification number against his or her face, looking for an impersonation.

The Eidos system can perform both functions, though the second – which typically involves only a small gallery of faces for comparison – is faster, needs smaller computers and results in fewer errors.

The impressive results of the Eidos system have been achieved on conventional computers of limited power, although very large database sets – for example, national mug books containing millions of faces – will require very fast computers. The system also adapts well to so-called “parallel computing” architectures, which harness numerous small computers to work simultaneously on a problem.

But for verification situations – using a computer to check employees entering a secure room in a bank – an ordinary PC could support an effective system. The ZN-FACE commercial system now in use in Germany runs on an ordinary 486 or Pentium PC under the Windows operating system. With the Pentium chip, it takes the system just three seconds to determine if a given face matches any of 500 in a database.

The ZN company claims greater than 99 percent accuracy for its system, which it calls “the electronic doorman.”

In addition to von der Malsburg and Lyons, the USC Eidos team included graduate students Jean-Marc Fellous and Ladan Shams. The German group comprised scientists Martin Lades, Jan C. Vorbrueggen, Joachim Buhmann, Joerg Lange, Rolf P. Wuertz and Wolfgang Konen.

The researchers coordinated their efforts largely via the Internet – with computer programs, digital images, comments and drafts passing continually back and forth between the two continents and laboratories.

Future enhancements may give Eidos the ability to recognize human expressions, von der Malsburg said.

This means the system might have a future in interactive, virtual environments – where the goal is to have the computer respond to your actions.

“I can imagine a system in which, if you smile, you will see one thing, while if you look frightened, you may see something else,” von der Malsburg said.

Here’s scanning at you, kid – computers learn to recognize faces

Top stories on USC News