Zeroing in on Surveillance Video

  • By Matt Bell
  • Posted 05.30.13
  • NOVA

After a crime or terrorist attack, authorities looking for the perpetrators may need to review many hours of surveillance video—in just minutes. But how can they speed up the video without speeding up the action? That's where "video synopsis" can help.

Listen to the story.

New software makes it possible for law enforcement to search through hours of surveillance footage in mere minutes.

Shmuel Peleg, a computer scientist at Hebrew University in Jerusalem, did not set out to help law enforcement when he first conceived the idea of what he calls "video synopsis."

"We started to try to shorten the time for watching family videos," he explained.

img tag

Computer scientist Shmuel Peleg in his office at Hebrew University. Enlarge Photo credit: © WGBH Educational Foundation

Peleg said he has never shot much home video himself, but his students have, and they wanted to figure out a way to go through long videos more quickly. Tackling the problem seemed like it might be a fun project for Peleg and his students, but it has turned out to have serious applications.

Shrinking Time

For law enforcement agencies trying to solve crimes or prevent terrorist attacks, surveillance video can be extremely valuable for detecting suspicious behavior and identifying suspects. But the amount of video can be overwhelming, and the time to review it can be limited.

Peleg's software, sold by the Israeli company BriefCam (which Peleg helped found), addresses this problem not by dramatically speeding up the action, but by combining actions that occur at different times.

Consider this surveillance video from an American office building.

Surveillance video

"At the first stage, we separate automatically the moving object[s] from the static background," Peleg explained. In this case, the software identifies the people entering and exiting the building.

The software then replays the scene as if the people moved through the area simultaneously.

The technology goes beyond simply shortening time, it reveals patterns of behavior.

"We pack as many objects as we can from as many different times as we can, so that we can see all the object[s] and all the activities in as short as possible [a] time," he said.

The summarized surveillance video from the office building, for example, compresses about two hours of activity into just a minute and a half.

Surveillance video

Revealing Patterns

Mordechai Kedar, a former Israeli military intelligence officer, says the technology offers an enormous benefit for investigators. Beyond simply shortening time, it reveals patterns of behavior.

"I'm looking for anybody who doesn't behave like the others," he said. "Let's say if you come to a situation in which you have half percent of the people behaving not in the usual way, it's a big advantage. You don't have to look at 99.5 percent of the people."

Law enforcement agencies around the world have purchased the technology, according to executives at BriefCam. They say authorities in Norway used the software following the shooting and bombing attacks by Anders Breivik, who killed dozens of people in 2011. Company officials suspect the software was also used by authorities in the United States after the Boston Marathon bombings in April.

As for Professor Peleg, he said he is pleased that his software is being used for law enforcement, but he does not think that fighting terrorism will be the real driver of the technology.

He explained that business owners use BriefCam's technology to keep an eye on their buildings and parking lots, and supermarkets use it to track the flow of customers in their stores. Peleg added that the software can also be used at home to enhance videos of new babies, pets, and elderly parents—or simply to shorten the time required for sitting through family videos in their entirety.

"I'll be much happier if more people use it for more daily cases rather than terrorism," he said.

Find more of The World's ongoing coverage of the Boston Marathon bombings.

Related Links

  • Manhunt—Boston Bombers

    Which technologies worked—and which didn't—in the race to track down the men behind the marathon attack?

  • The Limits of Facial Recognition

    Our incomplete understanding of how humans perceive faces may be hindering advances in automated face recognition.

  • Can Science Stop Crime?

    Explore the genetics behind criminal minds, the latest in lie detection, a human corpse "farm," and more.

  • Forensics on Trial

    Virtual autopsies, 3-D fingerprints, and digital crime scenes are making crime-solving into a more precise science.