NOVA

Northern Mariana Islands

United States

U.S. Minor Outlying Islands

U.S. Virgin Islands

Guam

American Samoa

Puerto Rico

US & Territories

Vision as we know it remains elusive for machines. But as this video short shows, they are starting to get the picture.

Vision as we know it remains elusive for machines.

Getting Computers to See

Carlisle Companies

Viking

NOVA Science Trust Logo

<div class="rich-text">
	Building vision into computers has been a greater hurdle than AI experts ever thought it would be, and they haven't cleared it yet. Simply recognizing everyday objects like a shoe or a chair—an ability than is simple for even a two-year-old child—remains difficult if not impossible for even the most sophisticated computers. As this video short reveals, before we can program them to do see like we do, we need to decode how our brain sees. With that in mind, AI experts are beginning to make some significant advances.
</div>

<div class="rich-text">Vision as we know it remains elusive for machines. But as this video short shows, they are starting to get the picture.</div>

Tech & Engineering

<div class="rich-text"><h3>
 Getting Computers to See
</h3>

 Posted: February 9, 2011



 GAME SHOW HOST:
 
 What do you say we play
 <cite>
 Jeopardy!
 </cite>
 ?



 NARRATOR:
 
 Watson is an intelligent machine, and a whiz at
 <cite>
 Jeopardy!
 </cite>
 .



 GAME SHOW HOST:
 
 Watson!



 WATSON:
 
 What is Jericho?



 GAME SHOW HOST:
 
 Correct!



 WATSON:
 
 $400, same category...



 NARRATOR:
 
 But his knowledge of the world comes only from the words he processes, alone in this room at IBM. In order to build a true artificial intelligence, one that comes close to matching our own, computers will have to learn from what they see in the world. And it turns out this is more challenging than anyone ever expected.



 RODNEY BROOKS:
 
 The biggest disappointment to me is how hard it has been to build general-purpose vision.



 RAJIT RAO:
 
 Vision is utterly subconscious. You open your eyes and the world is there. Almost half of our brain is working to make it easy for us.



 NARRATOR:
 
 And our eyes are only the start. What we actually see is determined by what we know.



 PATRICK WINSTON:
 
 We sometimes say that vision is a kind of controlled hallucination.



 ALEXEI EFROS:
 
 It's not just the pixels. A large part of it is defined by our previous experience. Our visual memory. The example I always use is the Monet paintings. So you have this train station. You have a train extending into the scene. And then when you look close, you realize that there is basically nothing there, some splotch of paint. And yet we all see a train because of the previous experience that we've had.



 NARRATOR:
 
 Until scientists can duplicate how humans see, they are training computers to see by example. Millions of examples. Thanks to the seemingly endless samples of digital images on the Internet.


 Hartmut Neven helped create one of the best computer vision programs available, Google Goggles.



 HARTMUT NEVEN:
 
 There's sort of a laundry list of things it can recognize at this point.



 NARRATOR:
 
 Goggles starts by looking for features in an image—points, angles, and pixels—and compares those features to hundreds of possible look-alikes it identifies in its databases.



 HARTMUT NEVEN:
 
 Any incoming picture is analyzed, and the most salient features of those are compared against similar features extracted from the database images.



 NARRATOR:
 
 Because it was trained to hunt for features among millions of different images, it now uses statistics to designate the most likely match. It may not look anything like human vision, but computers are starting to see the world in their own way.



 HARTMUT NEVEN:
 
 'Cause there's probably a couple billion different objects that Google Goggles can recognize at this point.



 NARRATOR:
 
 Including more paintings than a human ever could, with surprising accuracy. But it's not close to perfect. Give it a three-dimensional object without sharp edges, and it's lost.



 HARTMUT NEVEN:
 
 A little dog. [laughs]



 RODNEY BROOKS:
 
 The stuff that anyone could do, a two-year-old could do, like recognize a shoe, or a chair...



 HARTMUT NEVEN:
 
 The chair is yellow, I get things like sunsets and even a yellow fish.



 RODNEY BROOKS:
 
 It turns out that's the really hard stuff, that even with today's machine learning, we can't do well yet.



 HARTMUT NEVEN:
 
 Once we have a perfect vision system, we will have perfect A.I.



 NARRATOR:
 
 But perfect vision may not come until scientists decode how our brain sees.



 RAJIT RAO:
 
 What I would like is to go out at some point with a little robot or a computer which is looking at the world much in the same way as a kid, and come back learning these things with minimal, minimal supervision.
</div>

Colossal explosions shake a remote corner of the Siberian tundra, leaving behind massive craters. In Alaska, a huge lake erupts with bubbles of inflammable gas. Scientists are discovering that these mystifying phenomena add up to a ticking time bomb, as long-frozen permafrost melts and releases vast amounts of methane, a potent greenhouse gas. What are the implications of these dramatic developments in the Arctic? Scientists and local communities alike are struggling to grasp the scale of the methane threat and what it means for our climate future. (Premiered February 2, 2022)

In the Arctic, enormous releases of methane, a potent greenhouse gas, threaten the climate.

Arctic Sinkholes

The top half of Jupiter in blue and green against a night sky

A composite image of Jupiter using images captured by the James Webb Space Telescope. Image Credit: NASA, ESA, CSA, Jupiter ERS Team; image processing by Judy Schmidt.

Explore the cosmos, delve into ancient history, and follow an extreme rescue with NOVA’s most-watched documentaries released in 2022.

NOVA’s most popular science documentaries of 2022

Four bright orange loops brand off of a dark blue blob

A transmission electron micrograph of HIV-1 virus particles (orange) replicating from the plasma membrane of an infected H9 T cell. Credit: NIAID, Flickr

Scientists have cured a handful of people of HIV by piggybacking on treatments they received for blood cancer. But does that bring a widespread cure any closer?

Transcript

Credits

Support Provided By