When Keith Rose joined the Occupy St. Louis movement in 2011, he didn't heed the advice of a fellow demonstrator to wear bandanas to cover his face. The police had cameras out, but Rose didn't think they were doing much with the photos they were taking.
When he joined the Ferguson protests a few years later, it seemed to him that law enforcement had indeed started to put together a file on him. He had not been arrested at that point, but after a few run-ins with police where they knew things like where he had been, who he had been with, or what he had said, he thought it seemed pretty clear that they were monitoring him.
"It really upset me and got me more concerned with surveillance," Rose says.
During the protests, the St. Louis Metropolitan Police Department opened a surveillance hub called the Real Time Crime Center (RTCC). This data center aggregates information gathered through technologies like license plate readers, sensors that can detect and locate gunfire, and cameras spread across the city. Most of RTCC, as well as much of the technology that powers it, was not funded by the city, but through grants and public-private partnerships with companies like Motorola. That approach, according to the American Civil Liberties Union (ACLU), allowed the police department to sidestep going to the city's elected Board of Aldermen for approval and city funds.
"It's really disturbing what this says about the future of big tech being able to expand its own power," Rose says.
But with an effort called Community Control Over Police Surveillance (CCOPS), the ACLU is attempting to give citizens more control over the technology that police could use to monitor them. Launched in 2016, CCOPS aims to help local communities draft and pass laws giving them oversight of any surveillance technology that law enforcement agencies or the government want to deploy.
The ACLU works with local lawmakers to craft each law, changing it to meet the community's specific needs. A dozen cities, plus San Francisco's BART system, have passed CCOPS laws. In addition to St. Louis, at least another dozen cities, plus the state of Maine, are in the process of adopting their own versions of the law.
St. Louis Alderman John Collins-Muhammad helped introduce CCOPS legislation in 2017 and 2018, and recently reintroduced an updated version. Collins-Muhammad said that he and the rest of the Board got community input and updated each iteration. He thinks the latest version of the bill will pass this year. The law would create a process for board approval of any new surveillance technology, require annual reports for equipment purchases, strengthen sunshine laws around law enforcement data, and call for publishing crime reduction statistics. The bill tries to balance the concerns of both the public and the police, Collins-Muhammad says.
"We don't want to invade privacy," he says. "But we also don't want to limit the police in doing their jobs."
The St. Louis Metropolitan Police Department has also contributed comments on the bill and has been supportive of it in general, Collins-Muhammad says. In a statement, Keith Barrett, a sergeant with the Department, said that civil liberties and surveillance technology can coexist if used in a responsible manner while still helping police do their job.
“The advancement of technological resources continues to allow our department to deter, document, and reduce crime,” Barrett says. “The technology that is deployed continues to aid our efforts to prevent crime and apprehend criminals. We look forward to working with all our community partners in ensuring transparency and accountability.”
Privacy Watch, a group of nonprofits in St. Louis, took part in the discussion around the bill. It wanted the law to address issues related to bias with the technology, privacy, freedom of speech, and the cost to taxpayers.
"There are so many layers of concern," says Kendra Tatum, a St. Louis resident and an organizer for the Organization for Black Struggle, a member group of Privacy Watch. "People refer to the phrase ‘Big Brother’ jokingly a lot, but we all know that this type of stuff happens."
Collins-Muhammad says he knows the threat personally, having experienced surveillance when he was an activist during the Ferguson protests. He said it was “scary” and got him interested in creating policy that could limit that experience for others in the future.
"This is something very important to me," he says.
An AI Panopticon
While the Real Time Crime Center doesn't use AI-powered tools yet, it is technically easy to overlay face recognition software after cameras are in use. The proposed CCOPS legislation in St. Louis wouldn't outlaw the technology, but would require law enforcement to go through the Board of Aldermen if they want to use it.
AI-powered surveillance technology has spread across the world quickly. ACLU'S Dawn of Robot Surveillance report points out that this is occurring as camera technology is improving with higher resolution and night-vision capabilities. At the same time, more cities are installing centralized surveillance systems, with about 350 million surveillance cameras across the world in 2016.
The potential of facial recognition technology is perhaps most prominently on display in China. The government not only uses it to give out traffic tickets, and has passed laws mandating face scans for those signing up for internet or a new phone number, but has put it to work monitoring and imprisoning the Uighur minority. By 2020, China plans to have more than 600 million cameras operating.
Beyond concerns over increased surveillance, facial recognition software has been shown to perform less well on dark skin. In a 2018 report called "Gender Shades," researchers Joy Buolamwini and Timnit Gebru found that three commercially available systems performed no better than chance at identifying women with darker skin tones.
The ACLU did its own study showing that Rekognition, Amazon's face recognition software, misidentified 28 members of Congress. Forty percent of the algorithm’s mistakes were made in attempting to identify people of color, even though they make up only 20 percent of Congress. Amazon disputed the study and showed that with the recommended higher confidence threshold, which only lets the algorithm offer up matches if it is very sure it is right, the misidentifications disappear. However, one of the police departmentstesting the software admitted to not setting any confidence threshold.
That isn't the only case of police departments using software in a way that the developers didn't intend. A study by Georgetown Law Center on Privacy and Technology found that police fed celebrity photos into face recognition software, along with sketches and photoshopped images.
There is a long history of surveillance in the U.S. according to Matt Cagle, an attorney at the ACLU of Northern California. Public and civil rights groups like the ACLU have fought against much of it, with varying levels of success. With AI, however, a lot of surveillance work can be automated, making AI game-changing technology.
"If they can watch us, they can control us," Cagle says.
In fact, the face recognition conversation can be part of the larger conversation about government surveillance according to Chad Marlow, a senior advocacy and policy counsel at the ACLU, who is also overseeing CCOPS. The proliferation of technology like surveillance cameras and Stingrays—which mimic cell phone towers and send out signals that trick nearby phones into transmitting identifying information and data about their locations—can be reevaluated. Also, anything new developed, with or without AI capabilities, can be integrated in a way that the community is aware of and has some control over.
“In that way, we aren’t playing whack-a-mole,” Marlow says.
Drawbacks of Bans
Four cities have gone further than community oversight with their CCOPS laws: Berkeley, Oakland, San Francisco, and Somerville, Massachusetts have outright banned face recognition by the government. Earlier this year, the ACLU, along with another 60 groups, sent a letter to Congress asking for a federal moratorium on the use of facial recognition for law enforcement and immigration enforcement until Congress fully debates which practices should be allowed.
“This capability threatens to create a world where people are watched and identified as they attend a protest, congregate outside a place of worship, visit a medical provider, or simply go about their daily lives,” the letter said.
But Daniel Castro, vice president of the Information Technology and Innovation Foundation and director of the Center for Data Innovation, says cities could also end up blocking better potential uses of face recognition tools out of fear of a surveillance state.
"We're not there yet," he says. "A lot of the concerns there are premature."
Other law enforcement applications of the technology, like finding suspects or missing children, can be made safer with laws that create guardrails, like establishing accuracy rates for any software used. The areas that do matter get conflated with broader issues, like whether the police should be surveilling protestors at all, Castro said.
"We can separate some of these issues without banning the technology," he says.
CCOPS laws only regulate government use of technology, and don't stop private companies from using it. St. Louis has a handful of businesses, like corner stores, using cameras run by a local company called Blue Line Technology that says it uses face recognition. Another company called Persistent Surveillance Systems (PSS) offered millions of dollars in crime-fighting technology to St. Louis for free.
But if the police wanted to plug AI-powered cameras from private property into their Real Time Crime Center or take PSS up on their offer, CCOPS would kick in, forcing the police to bring the project to the Board of Aldermen, points out Marlow. Then the community and its elected officials would decide the future they want to see in their city.
"CCOPS sets up the conversation; it isn't the conversation itself," he says.