Is That Your Fingerprint or a Fake? This AI Can Tell

As high-level security systems transition to gathering biometric data via facial recognition, iris scans, and fingerprints, researchers such as USC's Dr. Wael Abd-Almageed are creating AI neural nets to spot fakes.

Because these days, downloading a CTO's headshot to create a 3D replica and break into a secret lab is not just the stuff of espionage movies—it could happen.

We spoke with Dr. Abd-Almageed, Research Team Lead and Senior Scientist at USC's Viterbi School of Engineering Information Sciences Institute, from his office in Marina del Rey about creating new artificial intelligence techniques and applying them to computer vision, biometrics, and multimedia problems. Here are edited and condensed excerpts of our conversation.

Dr. Abd-Almageed, is it an oversimplification to say that you are not teaching machines to think like humans but to approach vision, identity, criminal biometric fakes in an entirely new way?
[WAA] That's exactly right. We are not teaching machines to think like humans, simply because we do not yet know how humans think. We are developing AI algorithms, software, and sensors that will enable biometric authentications systems to be more robust and resilient against spoofing attacks.

To do that, we're inspired by how the human brain is wired. We're teaching AI to mimic one of the human functions. As an example, humans recognize faces, even if they're occluded with a beard, glasses, or hat, and can also tell if the face is a real person or printed face on a piece of paper. It's very difficult for computers to differentiate between real and fake people, but that's what we're training our system to do.

Currently you're the Principal Investigator on a project known as Biometric Authentication with a Timeless Learner (BATL), which runs until 2021. Can you talk about your work there, and what you've been tasked to achieve?
Existing biometric sensors, like cameras, fingerprint, and iris sensors are not inherently designed to be anti-spoof. The image can be great for identification, but it doesn't tell you whether this person is an actual person or maybe just a 2D image. In BATL, we are developing new anti-spoof face, fingerprint, and iris sensors that provide rich data, and AI algorithms that use the sensor data to tell if the biometric presentation is real or fake.

Or a fine mesh scan of the CTO's face held up to a camera to gain entry?
Right. So there's a problem with the existing technology and we were tasked with fixing that using a system which contains new sensing modalities like laser-speckle contrast imaging (LSCI) and short wave infrared (SWIR) to make authentication more accurate. In the case of fingerprint sensors, the laser can tell if there's blood pulsing through the digit.

As opposed to someone using a 3D-printed silicon device with the correct fingerprint. Or, being a bit gruesome here, the finger off a cadaver?
Exactly. Our systems combine the laser technology—to see if there's blood present—with AI and neural networks which make predictions on whether it's a live or fake sample each time.

How often does your system get it right?
We're improving all the time, reaching new milestones, but the last number we hit for fingerprint was a 97 attack percent detection accuracy, at very low false alarm rate (one in 500).

Impressive. On the way to being foolproof enough for high-level security edifices.
Indeed. But I must say, everything we do is a result of hard work of a great team of students, engineers, and scientists and not just my work.

What other biometrics research have you been involved in?
Large-scale face recognition. We developed a complex system in which we identify 68 facial "landmarks"—eyebrows, mouth, nose, and so on—and build a 3D render of the face. We use the rendered face along with an AI system to digitally rotate the face and identify the person, even when occluded by glasses or so on. A paper connected to this research was published recently by IEEE, and there we propose our "Pose-Aware Models (PAM) process," a face image using several pose-specific, deep convolutional neural networks.

The AI you're using was trained on significantly large pictorial datasets, including—curiously—Painter by Numbers?
Right. This was a very interesting problem to address and this paper is part of a series of papers that discusses the issue of "fake news" and repurposed images with corresponding fake text. In computer vision, this is pretty much a new area so we have to use proxy data sets, [and] one of them is painter by numbers. The system learns to question whether a painting is a Van Gogh or a Renoir by examining the image itself, along with the metadata or captions, which could be faked. Our framework for doing this is called Adversarial Image Repurposing Detection (AIRD) and is designed for large-scale image repurposing detection.

Recommended by Our Editors

AI and Machine Learning Exploit, Deepfakes, Now Harder to Detect

What Is Artificial Intelligence (AI)?

How OpenAI's Fake News Warnings Triggered Actual Fake News

You grew up in Egypt then came to the US to continue your education, receiving your PhD. with Distinction from the University of New Mexico in 2003. When did you first become interested in this field of biometric spoofing and machine vision?
Actually many years ago. When I finished my bachelors in Egypt and started on my masters, computer memory was so limited so I became interested in image processing and compression. That led me to early interest in computer vision.

What was the computing power back then?
Intel had just brought out the 486DX processors but, in 1989 when I got my first personal computer, computers didn't even have a hard drive. I used to write assembly and C code. Essentially it was very primitive compared to today, but we learned coding the hard way.

What brought you to USC ISI?
It's an interesting question because, when I joined ISI in 2013 they don't even have a computer vision department back then. But I had worked with Dr. Prem Natarajan, the incoming director, during my time at the University of Maryland and he brought me here to start this group.

When not working, do you have any geek credential worthy pursuits?
Quite honestly, not really. Essentially I have an engineering background and, even when I was a child, preferred to build things with Lego and Meccano. My desire, and dream, was to be an engineer and build systems. I switched to computer science later on, but then came full circle because for biometrics, we are not just developing algorithms on standard datasets and publishing papers, we are actually building prototypes and systems. They're not production-quality systems but could be easily converted to be so, in time. So what I do now is a good combination of engineering and computer science.

Dr. Wael Abd-Almageed will be speaking at the 12th IAPR International Conference On Biometrics (ICB), which runs until June 7 in Crete, Greece.

AI tech can identify genetic disorders from a person's face