How Accurate Are Flock's AI-Powered License-Reading Cameras?

Futurism reports: 404 Media revealed that a July 10 audit by the Los Angeles Police Department Office of the Inspector General caught the department's ALPR cameras generating 161 false stolen-vehicle alerts in just two months — each one ending with officers pulling over an innocent driver. Factoring in 337 alerts which "resulted in the recovery of stolen vehicles," the LAPD's cameras carry an error rate of 32.3 percent, effectively giving officers a one-in-three chance at pulling an innocent person over. "The biggest issue remains this national database at the FBI called NCIC," Flock's CEO Garrett Langley recently told The Drive, complaining about "how archaic its structure is." Flock CEO: There's no feedback loop mechanisms; it's really just a static comma-separated file. We've tried to build around it. We have this concept called "suppression." A local agency — let's take Atlanta, where I live — might see that there's a stolen plate out of California, but it's already been resolved because it's a rental car or a dealer car. They can, in Flock, suppress that: "Never alert us on that for the next year." That's to get around the fact that they can't force another agency to remove it from NCIC. Ideally, the way it would work is if enough agencies — let's call it one, two, or three — flag a tag as no longer valid or a bad entry, it should just get removed. Or at least it should get pushed more aggressively by the FBI... I'm hopeful that they'll see there's an opportunity to make something like NCIC, which is an important tool not just for companies like Flock, but for police departments to work together, start to make some enhancements to modernize what's a central part of our policing system. Later in the interview he says "It's less than one in a million alerts that an officer says, 'I'm not sure if that's right.' Less than one in a million." A recent report from Business Insider shares a worst-case scenario. In the summer of 2024 a Sacramento police officer said Flock's cameras had sent six false alerts for the same vehicle, wrongly saying it had been stolen or used in a felony crime in the nearby suburb of Roseville: Roseville police could see that Flock's software kept confusing the "9" on the man's license plate for an "8." The car owner said he would take off his license plate cover. Soon after, it happened again. It's "easier at this point we have it memorized," a dispatch supervisor in Roseville wrote in an email to a colleague. Flock says that in optimal conditions, its cameras accurately read more than 96% of license plate characters. Hundreds of pages of records from the Roseville Police Department show a different picture. In 2023 and 2024, Flock sent 1,427 alerts to Roseville police, flagging vehicles as stolen or used in a felony after they passed one of the city's Flock cameras. An analysis by the police department found that in 71% of those alerts, Flock's machine-learning software incorrectly read the license plates... The cameras regularly missed vehicles, captured blurry images, misread license plate characters and states, and sent delayed alerts to police about vehicles possibly connected to crimes, the records show... A factor that contributed to the misread problems in Roseville was the "particularly unique deployment" that the city requested, a Flock spokeswoman said. Roseville said it has its cameras configured so that they capture only the backs of vehicles, a setup intended to avoid capturing personally identifiable information like faces. Roseville's setup included older hardware and placement of cameras higher and further from vehicles than the company typically recommends, Flock added... In Toledo, Ohio, driver Brandon Upchurch was mauled by a police dog after a Flock camera misread the "7" on his license plate as a "2." As a result of his injuries, he said, he lost his job and was evicted from his home. He settled a lawsuit against the city and police officer for $35,000. None of the incorrect Flock alerts in Roseville resulted in a traffic stop or arrest, a department spokesman said. Roseville requires police officers to verify that a license plate is stolen or have independent reasonable suspicion of a crime before making a traffic stop... Flock said Roseville's camera performance has significantly improved, which the police department disputed... Flock's cameras also missed vehicles altogether... At one point, Flock's product director for machine learning suggested that officers ask drivers to remove license plate frames that made it "very difficult for the machine vision to tell it is actually a 'E' and not an 'F.'" Roseville told Flock at the time that it wouldn't do so, according to the department spokesman. Flock in 2024 shared an analysis with Roseville's police department, outlining reasons for the misreads. Vehicles far from a camera were sometimes blurry. Plates were cut off by trees or license plate frames. And Flock's software confused similar characters, mistaking an "N" for a "V", or a "1" for a "4...." Roseville isn't the first organization to flag inaccuracies in Flock's technology. In 2021, the research firm IPVM independently tested Flock's license plate readers, concluding that Flock misidentified the state in about one of 10 reads, and that the system regularly misclassified vehicles' type and make. IPVM said that Flock subsequently blocked it from purchasing its cameras for testing. Thanks to long-time Slashdot reader Cognitive Dissident for sharing the article.Read more of this story at Slashdot.