By oVio Tech
Image Credit: Adrian Owen
The New York Police Department was searching for a suspect caught on camera stealing beer from a CVS in New York City in April 2017. Efforts to utilize facial recognition to locate the suspect were hampered by surveillance camera footage that recorded the perpetrator from an overhead angle, partially obscuring the suspect’s face. When detectives submitted the surveillance footage to the NYPD’s facial recognition application, it was unable to return matches deemed useful to the case.
“Photos that are pixelated, distorted, or of partial faces provide less data for a face recognition system to analyze than high-quality, passport-style photos, increasing room for error.”
Claire Garvie, Georgetown Law Center
On May 16, 2019, Claire Garvie of Georgetown Law’s Center on Privacy & Technology released a report highlighting this notable NYPD case. The report, titled “Garbage In, Garbage Out: Face Recognition on Flawed Data”, found that police departments routinely edited or modified photos, including the NYPD, to aid facial recognition software to get a match.
Seems obvious enough, but what’s not immediately obvious is that facial recognition software performs much better if the database of known faces includes an image taken at the same relative angles of pitch and yaw as the probe image submitted for recognition, but more on that later.
In pursuit of justice, the NYPD detectives didn’t let the initial failure of the facial recognition system stop them, instead understanding the system limitations they tried another approach. The suspect shared a likeness with actor Woody Harrelson so the detectives submitted a higher quality photo of the suspect’s celebrity doppelgänger, Mr. Harrison in place of the low quality surveillance image. A list of candidates were generated based on the celebrity match and provided to investigators, ultimately resulting in the arrest of someone that was not Woody Harrelson.
And according to Georgetown Law’s Center on Privacy and Technology, this wasn’t the first time the NYPD resorted to using a celebrity photo to serve as a stand-in for the actual perpetrator when conducting a facial recognition search. In fact, the NYPD’s Facial Identification Section (FIS) uses techniques such as “Mirroring partial faces”, “Insertion of Eyes”, “Removal of Facial Expression”, and use of the “Clone Stamp Tool” all of which are intended to modify a probe image to more closely resemble the features and full frontal face image commonly found in mugshot photos.
So, through trial and error, the technicians at FIS discovered that facial recognition software performs best when the probe and known gallery images are reasonably similar in pitch and yaw angle. But rather than approach the problem by providing better inputs to the system, i.e. improved mugshots of multiple angles, they took the clumsy and rather questionable approach of synthetically “improving” the probe image themselves.
But, a solution for the problem encountered by the FIS exists, and it’s commercially available. oVio Technologies’ 360° imaging platform captures a subject 360 times from 360 unique angles in 12 seconds or 190 unique angles in half the time providing an enhanced mugshot file containing every possible view that may be necessary to complete a future facial recognition investigation.
Visual Identification for the 21st Century
In conclusion, Ms. Garvie of Georgetown Law’s Center on Privacy & Technology recommends improvements to standards governing what police departments feed into facial recognition systems. And barring the creation of those guardrails, she proposes a moratorium on local, state, and federal law enforcement use of facial recognition. But, instead of simply tying the hands of law enforcement the more effective solution is pairing the creation of input standards with the implementation of 21st century mugshots by adopting oVio’s 360 imaging technology.
The introduction of 360° mugshots removes the impetus for police departments to “photoshop” surveillance photos in the first place, saving them loads of time, and improves the accuracy and effectiveness of the underlying facial recognition technology at the same time.