A real-time facial recognition software tested by London Metropolitan Police at this year's Notting Hill Carnival was labeled as a "resounding success" by police officers, but privacy groups members who were invited to view it in action called it "inaccurate and painfully crude."
"I watched the facial recognition screen in action for less than 10 minutes," Silkie Carlo, a member of Liberty, a UK-based human rights group, says.
"In that short time, I witnessed the algorithm produce two ‘matches’ – both immediately obvious, to the human eye, as false positives," she added. "In fact both alerts had matched innocent women with wanted men. The software couldn’t even differentiate sex. I was astonished."
Carlo, who detailed her findings on Liberty's blog, says project leads called the ssytem "a resounding success," even if the software yielded only false positives, with 35 false matches in a single day.
More worrisome, according to Carlo, is that police officers pursued several of these matches, only to find out the system had misidentified suspects.
Met Police did make an arrest based on a positive (correct) match by the automated facial recognition (AFR) software, but Carlo says officers screwed up.
Police arrested a man sought on rioting charges, but that man had been arrested previous to the Notting Hill Carnival and had resolved his legal situation, being arrested for the second time for the same crime.
Carlo says this happened because Met Police fed the AFR software old data. In a statement before the Notting Hill Carnival, police said they were going to test the software with 500 images of people sought on various charges or have been banned from attending public events.
"This ‘trial’ showed all the hallmarks of the very basic pitfalls technologists have warned of for years – policing led by low-quality data and low-quality algorithms," Carlo explained the Met Police's problem.
This year's trial is the AFR's second live test. UK police first tested the system at last year's Notting Hill Carnival. Met Police reported its highest ever arrest total for the Notting Hill Carnival with 454 arrests, but the software did not help police in one single case. As it stands, this year's trial was as unproductive as last year's.
Similar to warnings from US privacy groups who are currently battling the FBI and trying to prevent it from building an all-encompassing biometrics database, Liberty and fellow UK privacy groups warn about the rollout of facial recognition software without any proper legislation in place.
"Like GPS surveillance, if facial recognition were rolled out across the country, the State would potentially have a biometric record of who goes where, when and with whom," Carlo writes.
Furthermore, the technology powering the facial recognition system is grossly inaccurate, as it cannot differentiate male from female apart, and is more likely affected by the classic problem of all facial recognition software — lower accuracy when identifying black people, which usually yields many false matches that waste police time and resources.
"The technology isn’t there yet – as I observed, it’s offensively crude – but the risk to our freedom posed by this ‘trial’ is current and real."