Mazes and Squiggles
Look No Hands!
Toddler's First Steps
ALAN ALDA Sixty miles an hour and the car's driving itself. On
this edition of Scientific American Frontiers we'll explore
a world where machines don't need people.
We'll see robots that get around, that look, that grab --
all on their own. We'll meet Toddler as he takes his first
steps. We'll watch helicopters trying to fly without pilots.
And we'll get to know Cog, who's teaching himself about life.
ALAN ALDA I'm Alan Alda. Join me now for Robots Alive!
IGOR I am looking for tennis balls.
ALAN ALDA (NARRATION) We're on the track of robots with a mind
of their own.
ALAN ALDA Why did he stay in with it?
In case it goes wild.
ALAN ALDA Well, is it liable to attack us? What do you mean wild?
ALAN ALDA (NARRATION) At the fifth annual artificial intelligence
convention in Portland, Oregon robots are competing to show
how they can recognize and manipulate... And how they can
find their way in a maze of corridor and rooms. It's a forum
for some of the world's top robot scientists to see what they
can do. Here we are at the preliminaries of the maze contest.
AMELIA I have arrived at Conference Room One.
ALAN ALDA (NARRATION) Things are already getting tense.
It's doing better than I am. I feel like I'm about to pass
out. AMELIA If there is someone here they are probably pacing
up and down - giving a talk. This room is occupied. Excuse
me for interrupting your meeting.
ALAN ALDA (NARRATION) In the maze contest, robots have to: start
in the director's office; find which one of two conference
rooms is empty; set a meeting time; invite two professors
to it; and then return to summon the director exactly one
minute before the meeting. The robots are given the rough
layout but not exact sizes -- they've got to finds rooms and
doorways for themselves, and watch out for the foyer, a notorious
trap. In the preliminaries, the foyer was Amelia's downfall.
She thought it was a room, and got stuck. The team will work
on it tonight.
REID SIMMONS One more chance!
ALAN ALDA (NARRATION) Most entries are single robots, but in this
innovative approach two small robots find the rooms, under
the direction of a central planning computer. They use a system
like the Internet to talk to each other -- at least that's
KURT KONOLIGE Didier, it doesn't look like he's
got the right plan here at all. It's just more than twice
as hard to do two robots because you have to coordinate them
and communicate between them... ah...but it's pushing us.
ALAN ALDA (NARRATION) Now the finals. Early on the pace is set
by Kansas State. KSU 1 I am going to search for people in
ALAN ALDA (NARRATION) Throughout the contest the judges will demonstrate
that Conference Room One is busy.
ALAN ALDA How did he know that there were people in here?
PETE BONASSO So this is one of the teams that is using the camera
to follow motion. Basically, in successive frames of the camera
they'll see a change to detect that there must be someone
in the room, because they're moving.
ALAN ALDA (NARRATION) Rows of sonar range-finders are used by all
the robots to sense their surroundings. Not hitting obstacles
is a contest requirement.
ALAN ALDA What if it hadn't stopped?
It was interesting.
ALAN ALDA He came very close.
ALAN ALDA (NARRATION) Kansas State breezes through the foyer, finds
Conference Room Two, sets up the meeting time, and alerts
1 The meeting will be in Conference Room 2. The meeting will
be held in two minutes thirty-one seconds. Two minutes thirty-one
ALAN ALDA (NARRATION) The final arrival is perfectly timed.
1 Excuse me director. The meeting will be in Conference Room
Two. The meeting is scheduled to take place in one minute.
They have the fastest time of any one robot entrant - nine
minutes twenty-five seconds, and they have maximum possible,
295 points. So give them a big round of applause.
ALAN ALDA (NARRATION) The only way to beat this score is speed
-- there's a fifty point bonus for the fastest round. On Yoda,
the cameras are not working.
YODA I'm going to conference
room one. The next landmark after I start moving. Find the
ALAN ALDA This robot has multiple-personality disorder. Every time
it talks it's got a different voice.
YODA Is there anybody
in the room? Please come and get an M and M's.
ALAN ALDA (NARRATION) To detect motion Yoda has to entice people
within range of his sonar.
YODA Thank you.
ALAN ALDA Oh, thank you. So far, this is my personal favorite.
ALAN ALDA (NARRATION) Yoda also makes a flawless round.
meeting starts in one minute.
ALAN ALDA (NARRATION) Perfect timing, too. But Yoda took twice
as long as Kansas State. SRI's multi-robot strategy has the
greatest speed potential, simply because two traveling robots
can get done in half the time. Blue moves out to check Conference
Room One... While Red heads immediately to Conference Room
ALAN ALDA He's passing right by... oh no there he goes.
ALAN ALDA (NARRATION) SRI's prepared to take a ten-point penalty
for demanding red shorts on the judge -- it's easier for the
camera to detect.
ALAN ALDA Was he told beforehand to first go to the conference
room and then the professor's room?
KURT KONOLIGE He was told
to go to the conference room and check it and then check back
to see what to do next.
ALAN ALDA (NARRATION) At the same time, Red was discovering that
Conference Room Two was free. Both robots now talk to the
central computer which sets the meeting time and sends each
robot to a different professor's office. Nobody gets lost,
and the total time is less than half Kansas State's.
ALAN ALDA The thing is over now, they did it huh?
And the time is incredible!
ALAN ALDA That's amazing!
ALAN ALDA (NARRATION) Poor old Amelia is off to a bad start.
REID SIMMONS Just go back and restart.
ALAN ALDA What happened here?
REID SIMMONS One of the software
programs crashed - something that's never happened to us before.
ALAN ALDA Is that it now, are you...
REID SIMMONS Well, we're gonna
ALAN ALDA Restart. How many times does the competition allow you
REID SIMMONS I think you... I think you can do
it as many times as you want but you get points deducted each
time you restart...
ALAN ALDA I see...
REID SIMMONS This is a fairly major hit.
ALAN ALDA (NARRATION) Once restarted, Amelia is fine at the first
conference room and heads across the dreaded foyer. It looks
like the same problem as the preliminaries. But with overnight
software changes, Amelia can now recover if she gets lost.
ALAN ALDA Amelia come back. Amelia is headstrong here. Come back,
you've gone to far.
ALAN ALDA (NARRATION) She gets out of the foyer and slowly figures
out where she is.
ALAN ALDA That process was really fascinating. I mean that was
like thinking practically!
ALAN ALDA (NARRATION) But it's all too much for Amelia. She crashes
for the second and last time. With their time bonus, SRI's
multiple robots came first. But that's not really the point
-- pushing the state of the art is what counts, and by that
measure everybody won. It's hard to say where research like
this will lead, but this can give you an idea. Many machines
can be hard for people to handle, especially if you're new
ALAN ALDA I'm not trying to do this. I'm... I swear I really am
not... I'm not trying to crash. Ow!
ALAN ALDA (NARRATION) Now look at this. Same wheelchair but with
a robot navigator added.
ALAN ALDA It won't let me race as fast as I went before...
RICH SIMPSON Right, but part of that is that this is a crowded
environment and it... it reduces the speed in proportion to
how crowded its surroundings are.
ALAN ALDA Yeah, I'm driving as wildly as I did before but it's
not letting me.
ALAN ALDA (NARRATION) Based directly on the systems in the contest,
it's a first glimpse into a world with robots, where humans
and smart machines cooperate. For the second contest it's
another deceptively simple task: find ten tennis balls spread
around the court; pick them up; put them in the corner. Precise
but slow-moving machines like this will have a hard time,
because there's a ten minute time limit... Plus a couple of
moving squiggle balls for an extra bonus.
PETE BONASSO Hand-eye
coordination is an intelligent thing and if you can get the
robots to do that, we're... we're advancing the... the state
of the art.
CONTESTANT You can pick up trash on a cluttered
desk or something like that, you know, maneuver around various
objects on your desk...
ALAN ALDA So if you said, while you're at it catch my two-year
old, forget it!
ALAN ALDA (NARRATION) Small agile robots are the favorites in this
contest. Here the Walleye team has opted to paint the balls
black so their black and white camera can see them. There's
a twenty point penalty though.
CONTESTANT There's a very wide
gripper and we have a tendency to catch the gripper on the
wall. OK, there, it got it.
ALAN ALDA (NARRATION) In a couple of minutes, the first load's
delivered. But then...
CONTESTANT Ah... it looks like it just
ALAN ALDA (NARRATION) You just know it's going to crash at the
worst possible minute. Restart, and then try again.
Alright, we're ready to start up again.
ALAN ALDA (NARRATION) Right off the bat, the Walleye traps and
captures a moving squiggle ball, worth fifty points. Then
CONTESTANT Ah... this never happened in practice,
so I don't know what's going on here.
ALAN ALDA (NARRATION) At the second restart, the team leaves the
balls in the grippers. But after restarting the robot thinks
it has to start looking again It's not smart enough to know
it's already loaded up.
ALAN ALDA This machine is a glutton.
ALAN ALDA (NARRATION) As it grabs its sixth ball, it finally reaches
overload and stops for good. This robot is based on a powerful
carrying platform used in hospitals.
IGOR I am looking for
ALAN ALDA (NARRATION) It efficiently gathers tennis balls, but
it's too slow to get squiggle balls. In a piece of luck, it
runs into one by accident. Then against all odds, the second
squiggle ball runs into the collector as well. Accident or
not, it's worth points. Next up, a robot using a powerful
combination of a systematic search pattern, with a TV camera
ball-spotter. Amazingly, first thing there's another lucky
squiggle encounter. And then... yet another. It's annoying
for the team because their efficient machine can track and
catch squiggle balls on its own. And it also has a special
strategy for those hard-to-get balls placed along the edge
by the judges. It's an almost perfect score.
ALAN ALDA So now it's going to try to find any more... with the
CONTESTANT Yeah, because the tennis game could
go on and there could be new balls.
ALAN ALDA You mean you picture somebody playing tennis in the middle
of this, with this thing roaming around under your feet?
ALAN ALDA (NARRATION) They saved the best for last...
ALAN ALDA All right! There it goes, there it goes! Woah!
ALAN ALDA (NARRATION) At least the most entertaining.
So the robot, in fact, with it's camera, can see about halfway
um.. through the rink.
ALAN ALDA (NARRATION) This little guy really seems alive. Helped
by a super-fast color vision system, chasing down a squiggle
ball, it's like a cat chasing a mouse. What seems like a victory
dance is actually a quick search for the dumping corner, now
too far for the robot to see. So, it decides to follow the
wall all the way around. After delivering the first squiggle
ball, it quickly spots another. It's single-minded, ignoring
a low-value tennis ball for the bigger prize.
there are two little stops under the gripper with white ends
ALAN ALDA (NARRATION) The robot makes quick work of nine of the
ten tennis balls on the court. And then, it's faced with a
serious problem. There's just one ball left, but it's dead
center of the court, just out of range of the robot's camera
-- a camera that the team had to angle downward to avoid the
glare of the convention hall's lights.
ALAN ALDA Oh no... oh no, it's gonna miss it now, huh?
The ball is exactly where it has to be so it can't be seen
by that field of view from around the walls. And that's a
singularity. It just happens to be just exactly the right
circumstances to beat this machine.
ALAN ALDA (NARRATION) Until, that is, the robot turns in a slightly
different place... It was artificial intelligence -- or AI
as the scientists say -- but it sure seemed alive.
There's something about these robots!
ALAN ALDA It's true, you know, when that machine caught that tennis
ball today at the last second, when I didn't think it was
going to get it and it did it, it was like Rocky, and I was
cheering for it. And the combination of AI and a machine that
almost seemed to have a human will was exciting, and that
alone can make people want to do more in both these fields.
ALAN ALDA (NARRATION) Robotics and artificial intelligence are
the two frontiers of computer science. And it's got to be
the robot people who have the most fun.
ALAN ALDA Hello Cybot! CYBOT Would you like a drink?
ALAN ALDA Yes, please... so, you here for the whole convention?
back to top
ALAN ALDA (NARRATION) I guess this has to be one of the strangest
things I've ever taken a ride in. Nobody seems to be in charge.
ALAN ALDA Is this driving itself now?
MARTIAL HEBERT It's driving
itself now, you can see the steering wheel turning.
ALAN ALDA How does it know where to go?
MARTIAL HEBERT I designated
the goal point about a hundred meters away.
ALAN ALDA (NARRATION) The goal point that Martial has chosen cannot
be reached directly -- there are obstacles in the way. Using
pictures from a set of TV cameras, the onboard system generates
height and position information for the terrain ahead, decides
which features are too big to be ignored, and then plots a
course around them. This is one of a long series of experiments
to develop autonomous vehicles -- vehicles which can get around
on their own. The work has been continuing for more than a
decade, at Carnegie-Mellon University in Pittsburgh, directed
by Chuck Thorpe.
ALAN ALDA How did everything change when you went to this next
model? CHUCK THORPE OK... when we were working on this one
we were thinking a lot about driving in hazardous environments
and our picture of hazardous environments was really military
environments. But we started to realize that the highways
are pretty hazardous environments too. We still kill 40,000
people per year in the US on the highways.
ALAN ALDA (NARRATION) This is now their fifth generation of autonomous
vehicles. It uses a simple personal computer, hooked in to
a radar range-finder for detecting obstacles, and a single
miniature TV camera. Dean Pomerleau, who wrote most of the
software that ties it all together, took me out for a spin.
ALAN ALDA What's this big, blue square here?
DEAN POMERLEAU That's
actually a representation of the back of the vehicle... um,
you can see the vehicle and the two tires below it - the little
gray squares - and then below that the two yellow markings
that are dancing back and forth - those represent the uh...
where it believes the edges of the lane to be. Because we're
out in a parking lot right now, you don't actually... it's
not finding any road, so those are sort of dancing around
as it's hunting for the road ahead.
ALAN ALDA Now how about this red...
DEAN POMERLEAU Trapezoid?
ALAN ALDA Trapezoid, yes. I knew there was a word for it.
DEAN POMERLEAU That's the portion of the scene ahead that the system
is actually processing. So it isn't processing the whole image.
ALAN ALDA (NARRATION) For computer vision systems the real world
is a mess -- it's cluttered and complicated. So this system
tries to make it simpler. It takes just the central part of
the picture -- showing the road -- and looks for some kind
of structure. Right now it's finding this series of intersections
just as confusing as the parking lot. But then it settles
DEAN POMERLEAU So right now we're in a fairly well structured
setting. There are lane boundaries on the sides and dash marking
down the middle. So the system is... quite confident and as
you can see from the display, has locked on quite well to
the lane center and the two ah... lane boundaries.
ALAN ALDA So you would feel confident here letting the car go by
DEAN POMERLEAU For that stretch yes, but here we come
up to ah... an intersection, there's a stop light, in fact
this stop light appears not to be working. That guy giving
me hand signals that it's safe to go through the intersection,
is... is far beyond the kind of stuff that any machine vision
system is capable of now.
ALAN ALDA (NARRATION) Even so, the system is not entirely dumb.
DEAN POMERLEAU The most interesting thing about the system
is it in fact is not explicitly tracking those lane markings.
There isn't a hand-programmed, lane-marker detector in here.
The system basically adapts to utilize whatever features are
visible on the road ahead.
ALAN ALDA What other features would it... would it track?
DEAN POMERLEAU Things like what we see here, like the uh... the
uh cracks in the road where they've been filled - the dark
discoloration - even the uh... slight discoloration down the
ALAN ALDA (NARRATION) The system works best in the simpler setting
of the highway. That's where I'm going to try it out in what
they call warning mode. This is probably the way smart cars
like this will first come into widespread use.
ALAN ALDA Now I'm going to deliberately drift here, I'm gonna drift
DEAN POMERLEAU Yeah, see what happens. Right as
your tire crossed the lane boundary, it basically gave you
the audible warning that ah... there's, there's something
wrong here - you want to steer back toward the lane's center.
ALAN ALDA All right, here I go again. Drifting.
The goal of this initially is to uh... basically deploy it
on trucks. They're the... where a large problem is with drowsy
drivers in particular.
ALAN ALDA So we're getting down to one lane, and there's no stripe
on the left so it must just be reading the oil spots on the
DEAN POMERLEAU Uh huh... the oil spots and there is
a slight seam, a crack in the road that the system is also
ALAN ALDA Right, yeah.
DEAN POMERLEAU So it will adapt what features
it utilizes to the circumstances.
ALAN ALDA (NARRATION) Now we're going to go to full, autonomous
steering. Better to have Dean in the hot seat.
ALAN ALDA A little bit, yes - just a little bit.
So what I'm going to do now is change the mode now into the
mode where we can automatically drive the vehicle.
ALAN ALDA How many times have you done this? Have you done this
DEAN POMERLEAU Hundreds...
ALAN ALDA Hundreds? A hundred times?
DEAN POMERLEAU Oh, minimum.
ALAN ALDA (NARRATION) We rejoin the high-speed traffic on the highway.
DEAN POMERLEAU So we can see the system has locked on to the
ALAN ALDA Yeah.
DEAN POMERLEAU And so I will now hit the red button
- and now it is steering on its own.
ALAN ALDA How fast are we going?
DEAN POMERLEAU Fifty-five, fifty-seven
miles per hour.
ALAN ALDA Well this is not nearly as frightening as I thought it
would be. I'll tell you what's even more amazing is we've
been chatting about this and I've completely forgotten that
you have your hands off the wheel. I can't get over that!
There, it's going around a curve now, it's great to see that
wheel actually take the curve.
DEAN POMERLEAU Yeah.
ALAN ALDA (NARRATION) It's still up to the driver to apply the
brakes, but the system will be able to handle that, too.
ALAN ALDA OK, now look, this car just pulled into our lane. Did
we respond in any way to that?
DEAN POMERLEAU The radar located
it - you can see the uh.. pink line with the white dot right
in the middle of it - that's indicating it's tracking it and
it knows its in our lane.
ALAN ALDA (NARRATION) At the same time, the system excludes the
overtaking car from the red trapezoid, so the view of the
lane ahead is not disrupted. Ten years ago our cameras recorded
some of Chuck Thorpe's early experiments. The first NavLab,
as they called it, detected obstacles with a laser scanner.
It worked as well as the radar on the latest version, but
it was slow -- like everything on NavLab 1. Just like today's
system, it combined obstacle-detection with processed TV-camera
pictures. But its computers took ten seconds to process each
new frame of picture. So it found the road and stayed on it,
but at less than walking pace. Today the limit's no longer
the computers -- it's the law. There's not much doubt that
we'll be riding in robotic cars before long. But is that really
such a strange idea?
ALAN ALDA How much are we controlled by computers now - that we
just accept without giving it too much thought?
The interesting thing is that... that probably fifteen, even
ten years ago, the view of robotics was you'd have a... human-appearing
servant in your house, for example, that would uh... clean
your dishes, make your bed in the morning... uh... what's
really happened in robotics is that the technology has become
embedded and really invisible. So there are a lot of things...
uh, your microwave oven has a pretty sophisticated ah... computer
in it. So robotics is actually becoming less and less visible,
but more and more prevalent through society. And that's what
we see for this technology... um, just a few years ago, was
a camera about this big and needed a big platform to sit on,
but it's shrinking down and eventually we hope to put everything
in a box smaller than that. And... and just stick it on the
back of the rear-view mirror and the person won't even realize
it's there - until they begin to drift off the road and it
warns them and hopefully saves their life.
ALAN ALDA (NARRATION) It's almost 20 years since the Imperial Walkers
first lumbered into action in the movie "The Empire Strikes
Back." And if they look a bit tentative, it's not surprising.
Even movie magic can't disguise how hard it is to make a robot
walk. Looking remarkably like a model Imperial Walker, this
machine was actually built over 30 years ago by an engineer
named Ralph Mosher. Without the benefit of computers, Mosher's
walker translated his hand and foot motions into footsteps
powered by hydraulic motors. Mosher's machine was eventually
abandoned as impractical. But as well as inspiring the Imperial
Walker it also inspired this creation... A six-legged robot
built in the mid-1980s at Ohio State University. Sixteen onboard
computers made sure its center of gravity was kept safely
over its legs. Today, this four-ton monster is also in mothballs.
Needing always to be in balance made it simply too slow and
cumbersome. But also in the mid-1980s, an entirely different
type of walking machine was being pioneered at Carnegie Mellon
University. In these machines, balancing was an active rather
than passive process.
RAIBERT These machines balance the same
as me balancing this broom on my finger. If I keep my finger
under the body, it will stay stationary; if I move my finger
away it will tip towards me and if I move my finger towards
me it - the broom - will tip away. If I want to make it fall
towards me, I can put my finger away.
ALAN ALDA (NARRATION) Marc Raibert's hopper had to keep moving
to keep balanced -- but could perform some spectacular tricks.
Today, his two-legged hopper can bounce along without a boom
to steady it, and its gymnastic skills are even more impressive.
Unlike people, on the other hand, this biped can't simply
stand still, and it needs a lot of computing power to constantly
figure out what it needs to do to stay on its feet. This little
robot, being built at the University of New Hampshire, is
an attempt to much more closely mimic the way humans walk.
Its creator, Tom Miller, has not only modeled it's legs and
feet on our own, but also the way it acquires its skills.
MILLER Our approach is to try and learn from past experiences
and just remember what you've done in the past and how that
worked and then to try and make controlled decisions for future
ALAN ALDA (NARRATION) In other words, his robot learns. And the
first thing it learns, every time it has to stand on its own
two feet, is how not to topple over. Easing its top heavy
body into a crouch, the robot -- nicknamed Toddler -- not
only stands alone, but can even cope with some gentle nudging.
Toddler's brain is kept safely on a nearby desk, while its
knowledge about what's happening comes from force sensors
in its feet, as well as balance and movement sensors in its
MILLER The body of the biped has accelerometers on
the front and side of the robot and gyroscopes which measure
the rotation of the body of the robot. And that essentially
gives it a sense of what its orientation is relative to the
ALAN ALDA (NARRATION) Each time it's switched on, Toddler has to
start from scratch.
MILLER It's experimenting with how far
to lean right before it lifts its left leg, and you see the
knee flexes, it's trying to lift the left and if it doesn't
get off the ground it gets stable again and then leans a little
farther. Um, and now that time when it finally got off, you
see, it leaned a little too far.
ALAN ALDA (NARRATION) Miller hovers protectively as his robot experiments
first with swinging from side to side, while standing in place.
With every sway it's learning what works and what doesn't.
Then, cautiously, it takes its first, tiny steps.
has very simple intuitions about how to walk. It knows to
move side to side and that when it leans to one side it should
try and pick up the opposite foot. And then with the learning
it kinda... that's the problem, I kinda have to look at the
robot when its learning. This process is somewhat like a toddler
learning to walk. Just like a toddler, it needs an adult standing
right here to catch it when it makes mistakes. Let's try two
ALAN ALDA (NARRATION) Tom Miller's basic interest is in making
machines that can learn from experience, and a robot that
can walk on its own two feet is a perfect challenge for machine
MILLER We've programmed in procedures by which it
can evaluate its stability based on its sensors and so as
it does these motions it evaluates how stable it is and then
makes modifications to its primitive actions and if those
modifications improve the stability when it tries them and
it remembers that and um... tries it again and each time it
tries to just continually improves on its sense of stability.
ALAN ALDA (NARRATION) Like all toddlers, this one still has a lot
of learning to do before it can make its own way in the world.
As for what a walking robot might be useful for... well, what
did someone once say? What use is a baby?
ALAN ALDA IT -- wake up! Hello. Good morning.
ALAN ALDA (NARRATION) Meet IT -- a robot that makes human-looking
ALAN ALDA I'm sorry. Would you just say that again?
ALAN ALDA (NARRATION) IT is the creation of Rodney Brooks, a world-famous
robot-builder, and a professor at the Massachusetts Institute
RODNEY BROOKS There's a detector at the front,
ALAN ALDA (NARRATION) Get too close to one of ITs infra-red detectors,
and you trigger an annoyed-looking response.
ALAN ALDA It opens its mouth in utter surprise.
RODNEY BROOKS Yes,
and it raised its eyebrows. It's sort of got this reaction
of, Oh, what's happening here! Get away from me! Thanks to
microphones in the head and motion detectors in the eye, IT
seems to follow the back and forth of your conversation. And
IT smiles whenever there's a lot of activity going on. IT
tries to appear to be a human, so that we can interact with
it in a way that is like a human. And it's meant to be a robot
that you might interact with in an entertainment sort of thing,
in an amusement park or something like that.
ALAN ALDA (NARRATION) Of course, IT is far from human. But it does
exhibit a certain lifelike behavior, thanks to a programming
technique Rod invented. Instead of giving his robots complex
instructions, he uses layers of simple fast programs, which
all run at the same time and work like reflexes. In Rod's
famous insect robots, each leg independently senses, and lifts
over, obstacles. So these robots move quickly without doing
a lot of thinking. Rod used this same idea in developing small,
autonomous rovers for NASA to send to Mars. And now, he's
attempting to build the most ambitious robot ever -- a humanoid
called Cog. The goal is for Cog to one day have the dexterity,
intelligence and understanding of a real human being. When
I first met Cog two years ago, much of it was still in pieces
on the laboratory bench, waiting to be assembled. Cog is supposed
to learn about the world the way a child does. Early in life
children don't know much of anything -- not even how to coordinate
their own limbs. Yet out of the desire to explore and experience
the world, comes knowledge.
RODNEY BROOKS By being a human
shape and having the same arrangement of eyes, etc., it will
encourage people to interact with it as though it was a human.
And it will have the same sorts of experiences that a human
has when a human develops in a human society with other humans
interacting with it.
ALAN ALDA It's got two parts to each eye. Why's that?
Well that's much like humans have a wide angle view of the
world and a very foveal view. And if we come over here to
ALAN ALDA (NARRATION) The fovea is the high resolution, central
part of our eyes. In Cog's monitor bank I could see both a
close-up foveal view, and a wide-angle one -- corresponding
to the two cameras in each eye. Cog needs both views to follow
what's going on in the world.
RODNEY BROOKS Just like in your
eyes, you're able to see things without much detail off in
the distance. You know if you see a ball coming in, you'll
rapidly move your eyes over to see what's coming. And then
with your fovea track where it is and go out and catch it.
ALAN ALDA (NARRATION) Like humans, Cog must constantly shift its
eyes to see the details of interesting objects. Rod built
in the ability to analyze the wide angle view and pick out
motion -- which shows up on this display as white areas. But
Cog had to learn by trial and error how to actually move its
eyes. To make Cog's vision system fast, the components are
split up among separate computers.
ALAN ALDA Do you think that that imitates more accurately how people
RODNEY BROOKS That is how the brain is organized.
There is this very distributed system.
ALAN ALDA Distributed meaning?
RODNEY BROOKS There's lots of little
pieces. They all will have evolved over time separately. And
they're all running separately. And they don't even know about
the other pieces that are going on there. There're doing their
stuff, sending messages off to muscles or the cerebellum demanding
things to happen and maybe those things happen and maybe they
don't. Maybe some other part of the brain gets control at
that particular time.
ALAN ALDA (NARRATION) Cog's neck is another piece of the robot
that's controlled separately.
ALAN ALDA If the eyes look over there, and in your heart of hearts
you want the head to look where the eyes go, how do you get
the head to get that information from the eyes and use it?
RODNEY BROOKS Then we have another program that's sitting
running and it tries to always make the eyes be centered looking
straight ahead. But the only control it has is control of
the neck. So when the eyes are pointing over there, the only
way it can make the eyes be centered in the head is to turn
the neck. Because that's the only thing that particular program
has control of.
ALAN ALDA (NARRATION) Cog's neck will also be controlled by a second
program -- one that reproduces the human reflex to turn towards
the source of sound. Just like with babies, being able to
focus on where a sound comes from should help Cog to find
the interesting sounds around it, and eventually to understand
RODNEY BROOKS Let's come and look at the hand here.
ALAN ALDA (NARRATION) Cog will also be able to experience the world
RODNEY BROOKS This is a three-fingered and
one thumb hand. Here are the three fingers on the left hand.
And we've got just a little piece of skin attached to it so
it's touch sensitive and moves when we touch it. We just programmed
it to do that. Why don't you see if you can make it move?
Okay, it felt you and now it's going to grab the object it's
ALAN ALDA So, it's sort of programmed now to grab what it feel
RODNEY BROOKS Just with that one little finger.
Eventually, as we finish building this hand, we'll have the
fingers completely wrapped with this touch-sensitive material.
ALAN ALDA (NARRATION) Yet another independent computer will control
Cog's sense of touch. It will sit inside Cog's hand, so that
touch information can be processed right on the spot.
RODNEY BROOKS In the human, of course, our spine makes a lot of decisions
for us. It doesn't go all the way up our neck to our brain.
Because there's just not enough time to get your hand out
of a burning fire, or your foot out of a burning fire.
ALAN ALDA You haven't got time to decide to?
RODNEY BROOKS There's
not enough time for the message to travel up your leg, then
all the way up to your brain, then all the way back down.
So as soon as it gets to the base of the spine, makes the
decision to lift your leg up real quick. Here's the arm and
you can see it sort of matches the left arm of a human here...
ALAN ALDA (NARRATION) The more I learned about Cog, the more I
appreciated how closely it had been modeled after humans.
RODNEY BROOKS And it's got an elbow, and a shoulder, and a
ALAN ALDA (NARRATION) Of course, robot arms used in factories bend
and twist with apparently the same freedom as their human
counterparts, but using them on Cog would lead to a disaster
the minute people got involved.
RODNEY BROOKS A conventional
robot arm in a factory, if you tell the robot arm to straighten
out to be 85 degrees, it starts moving and it hits an obstacle
here and there's a feedback loop and it realizes it's not
getting to 85 degrees so it starts sending force to its motors
and it pushes harder and harder. It's made of metal and you're
made of flesh and it will just plow right through the middle
ALAN ALDA (NARRATION) By installing a metal strip between each
motor and the joint it moves, Rod has in effect put springs
all over the arm.
ALAN ALDA So it will give. As soon as it gets to my chest, it will
RODNEY BROOKS It will start giving. But then
we do something clever. Grab hold of the arm.
ALAN ALDA (NARRATION) As I twist the arm, I notice its resistance
ALAN ALDA OK, now there's more resistance.
ALAN ALDA (NARRATION) The arm's computer can adjust the resistance.
So it can be rigid like a factory robot for hammering a nail,
or springy for interacting with people. Two years after first
visiting Cog, we returned for a progress report. The arm was
now mounted -- with a temporary hand -- and Rodney set the
robot's memory back to zero to show us how Cog had been learning.
For this demonstration, he'll have Cog reach for a hockey
puck. First, Rodney checks to be sure Cog can see the puck.
With its wide-angle camera, Cog has the puck in view far to
the right. The motion detection system picks up the movement,
and commands the eyes to shift, putting the puck on the close
up screen. Cog is focusing on his target well, so now it's
time to try reaching for it. At first Cog doesn't even come
close. Because nobody programmed into Cog any information
about how its arm works or what the various motors do, it
has to learn by trial and error.
RODNEY BROOKS Right now it
doesn't know how to move its arm. It moves its arm out towards
what it saw, then sees where its arm ended up and sees that
it's wrong, and learns that it should move slightly differently.
And slowly over time, it's going to get better and better
at reaching towards the thing that it's seeing. Babies take
a few months to do this well. Cog takes a few hours to do
not quite as well as babies.
ALAN ALDA (NARRATION) Sure enough, Cog reaches for the puck perfectly
after practicing just a few hours. If Cog can keep learning
like this, as more parts are added, who knows how far it can
ALAN ALDA What's the picture in your head now? What do you think
of when you think of a mature Cog, a Cog that has reached
some point where you can say, that's our Cog?
What I really think of is a starfleet officer out there being
a lieutenant commander on a starfleet vessel.
ALAN ALDA First you've got to get these robots to build the vessel,
because that's already pretty advanced. So you see, does that
mean that you actually have a fantasy of a bunch of your robots
getting into a spaceship and traveling for a thousand years
out into the galaxies?
RODNEY BROOKS I have a fantasy of being
able to build a human-level equivalent robot which is able
to operate in the world in the way we operate. And maybe even
better than we operate , eventually.
ALAN ALDA (NARRATION) Dawn in Atlanta. And competitors-- many exhausted
from working all night-- are gathering at Georgia Tech for
a contest that has never been won. It's a contest between
flying robots. Here's the task. With no one at the controls,
each robot must take off from one corner of a 60 by 120 ft.
field, then find and fly to a ring containing 6 metal disks.
Still without human control it must pick up the discs one
at a time and carry them over a 3 ft. barrier to a second
ring, where the discs are deposited. In five years of competition,
no robot has even come close to completing the mission. First
on the field this year is a blimp from the Technical University
of Berlin, Germany.
ALAN ALDA Have you tried this out on a field back home?
MARION FINKE Yes, but inside.
ALAN ALDA Inside?
MARION FINKE Yeah, it's working very well inside.
We still have some problems with the...
ALAN ALDA (NARRATION) Like all the robots here, once it starts,
it's on its own.
STUDENT OK, and we are autonomous.
ALAN ALDA (NARRATION) And at once the Berlin blimp demonstrates
why the contest is so difficult-- especially for blimps. Its
six battery-powered propellers try hard to steer it toward
the ring - but the gentle dawn breeze wafts it away.
ALAN ALDA Has it lost control, do you think it's --
MICHELSON No, not completely.
ALAN ALDA (NARRATION) As Marion dashed off to help rescue her balloon,
I sought out the contest organizer, Rob Michelson.
ALAN ALDA What do you think happened there?
ROB MICHELSON They had the props going as hard as they could
down, and they were able to catch it before it went into the
power lines. But that's the problem with a blimp.
ALAN ALDA (NARRATION) Taking a radically different tack is the
next robot. This is the fifth consecutive year the University
of Texas at Arlington has entered what amounts to a flying
ALAN ALDA Just from your previous years' experience, what's it
like once you throw the switch and the thing starts to go
on its own, you can't do anything? All the work you put into
it matters, but you can't do anything at that point.
STUDENT 1 This, this year different from the previous years,
I'm really, really nervous because this is the closest we've
ALAN ALDA (NARRATION) The first year of the contest, the Texas
tailsitter was one of the few machines even to get off the
ground-- briefly. Last year-- after 3 years improving its
control systems-- the tailsitter flew beautifully. The problem
was still the landing.
First thing I need you to (inaudible) show me is how your
emergency set up --
ALAN ALDA (NARRATION) This year, the contests' judges are taking
I want to see the emergency set up first thing --
ALAN ALDA He said he wants to see the emergency procedures. I want
to see your emergency procedures too. I'll be over here.
ALAN ALDA (NARRATION) What had caught my attention was what looked
like a giant inflatable pig. It turned out to be another blimp,
this one from the University of British Columbia.
ALAN ALDA Are you about to take off for the first time now? CANADIAN
STUDENT We were flying earlier. What we're going to try is
a manual flight just to see that the system is working, which
ALAN ALDA Do you know what the problem is?
STUDENT No, I don't, engine trouble.
ALAN ALDA (NARRATION) The engine got fixed. But the navigation
system-- involving a video camera watching the size and shape
of the black spots-- got confused by a glint of sunlight,
and the Canadian robot waltzed off the field. Meanwhile, last
year's best performer-- a helicopter from the University of
Southern California-- was also having a bad morning.
MONTGOMERY We crashed last night about 4 in the morning. It
was flying great, things were looking good. We were, we crashed
and something mechanically is wrong with it. The craft is
not functioning right.
ALAN ALDA (NARRATION) Despite improvised repairs...
MONTGOMERY -- let's reset, start again.
ALAN ALDA (NARRATION) ...and last minute adjustments, the USC 'copter
simply couldn't get off the ground.
MONTGOMERY Aagh! Man! You look at this, you know, you think
"Gees". You go pick up a disk, you carry it across the barrier
and drop it off. That's so simple, what's the problem? But
they don't realize that, you know, what's very easy for humans
is much more difficult for robots. You have sensing problems,
you have to deal with variables such as wind and such and
it's not a very trivial problem at all. It just shows you
just how flexible and adaptive humans are.
ALAN ALDA (NARRATION) I really liked this little device, designed
to pick up the discs when dangling from its helicopter.
MEMBER Pick it up, and we have --
ALAN ALDA That works really smoothly.
ALAN ALDA (NARRATION) The problem was it wasn't quite so smooth
once its helicopter was carrying it. But now the Texas tailsitter
was taking off, guided by scanning laser beams. It seemed
to be flying very nicely, but made no attempt to go pick up
any discs. Everyone seemed very pleased when it landed without
toppling over. I was a bit puzzled by all the excitement.
ALAN ALDA What happened?
STUDENT 2 That was a completely autonomous flight and we took
off, hovered, and landed completely autonomously.
STUDENT 3 That's world history for this competition; that's
the first time that's ever been done in this competition and
I've been here for five years.
STUDENT 2 We're pumped now. The next step is a disk.
ALAN ALDA (NARRATION) But now Stanford University was taking the
field. These guys looked like pros. And they had a navigation
system to prove it, employing the Defense Department's Global
Positioning Satellite System.
ALAN ALDA What is that thing over there?
ROCK That is a GPS antenna. That is our ground station. That
GPS antenna sees satellites in the sky and we basically fly
to that reference station. We have four of those little antenna
on board the helicopter. And the thing that's neat about this
is that with this new GPS technology we're able to tell you
where each one of these little antenna is to within a centimeter.
ALAN ALDA A centimeter?
ROCK A centimeter.
ALAN ALDA (NARRATION) Once the helicopter locked on to the satellite...
CONWAY The flashing red lights on the back of the helicopter
means that everything is working.
ALAN ALDA (NARRATION) The Stanford robot took off without a hitch.
STEVE ROCK This is flying under complete computer control
ALAN ALDA (NARRATION) And unlike the other teams, didn't bother
ALAN ALDA Are you just going for autonomous flight now? Or are
you going to try to go over and get a disk?
ROCK We're going to try to pick up a disk right now. We're
flying the whole trajectory.
ALAN ALDA (NARRATION) To pick up a disc, the machine simply trawls
with a magnet.
ROCK Whoa! We're close. We're going to, he goes into a search
pattern so there'd be a little random motion here where we'll
try to drag it around, hoping that we'll bump onto a disk.
ALAN ALDA (NARRATION) It looked to me as if the helicopter was
flying too low to drag the magnet around the whole circle.
I found myself making an acute observation..
ALAN ALDA So the problem here is that you need to make the string
ROCK Maybe we need, yeah, high tech solution. (laughter) Make
the string a little bit shorter. Whoa, it got one.
ALAN ALDA (NARRATION) The robot had finally got a disk-- but then
it picked up another one-- and that's against the rules. But
the Stanford team still had plenty of time. For the Texas
team, though, time was running out-- and their tail-sitter
was heading the wrong way.
JUDGE Four minutes guys.
STUDENT 1 Think quickly.
ALAN ALDA What are you trying to come up with?
STUDENT 1 We're just trying to adjust the gyros as the instruments
ALAN ALDA (NARRATION) But the drift problem affecting the tailsitter
proved unfixable. At least this year, the Texas robot didn't
scatter itself across the field. The German team was also
coming down to the wire.
ALAN ALDA So you made it up.
MARION FINKE Yes.
ALAN ALDA You got up just barely. You have to be up off the ground
before you leave that square, huh?
MARION FINKE Yes.
ALAN ALDA (NARRATION) Sailing serenely-- and autonomously-- the
blimp headed in the right direction... Only to overshoot.
ALAN ALDA If it just gets a little calm air it might be able to
settle into that - oh no. Outside.
STUDENT OK. So we can put it again --
ALAN ALDA 45 seconds left. Watch out! Watch out!
ALAN ALDA (NARRATION) The blimp had begun its day battling the
ALAN ALDA 20 seconds. JUDGE You want to get it restarted. STUDENT
ALAN ALDA (NARRATION) And it was the breeze that finally defeated
MARION FINKE I think one could see that we could manage
with a little luck and with less wind.
ALAN ALDA Yeah. Congratulations.
MARION FINKE Thank you.
ALAN ALDA (NARRATION) Another blimp was having similar problems.
But this one-- with a distinctly home-made look-- turned out
to be the work not of a college team but of students from
the Thomas Wooten High School in Rockville, Maryland.
ALAN ALDA When you have school to worry about, how much of your
day can you spend on this?
STUDENT Oh, we would spend until midnight a lot. We would
spend 5, 6 hours a night on it. And, I mean that bag, everything
here is home made. We didn't buy anything, you know, bought.
2 Go that way.
STUDENT I'm going, I'm going.
ALAN ALDA The wind is too strong for you, huh?
STUDENT Yeah, too strong, too strong. Just a slight wind throws
everything off for the blimp at least. That's why a lot of
these college teams have helicopters.
ALAN ALDA Yeah.
STUDENT And you can see the other two blimps have the same
problems we do.
ALAN ALDA Yeah.
ALAN ALDA (NARRATION) Under direct radio control, the high school
blimp wasn't even trying to fly autonomously.
ALAN ALDA You're in, you're in.
WOOTEN STUDENT No, OK. Concentration people. It's not working
ALAN ALDA (NARRATION) But just by being here and flying, the Thomas
Wooten Students have led the contest organizers to consider
a high school version of the contest in the future.
ALAN ALDA What do you think you've learned here?
STUDENT The biggest thing I've learned is that, you know,
even though we're a high school team we can overcome whatever
challenge we need to overcome. It's amazing. I came here thinking,
"Wow, we're out of our league." And coming and see that they
have the same problems we do, it, it's, it's kind of a relief,
ALAN ALDA Yeah, yeah. It gives you confidence.
ALAN ALDA (NARRATION) The high schoolers had devised from a cylinder
and magnet a clever device for picking up and dropping the
discs -- a system that made Stanford's method for snagging
discs look primitive.
STUDENT How much do we want to shorten the string?
CONWAY Oh, about that much.
STUDENT About half, about half a meter?
ALAN ALDA (NARRATION) But now that they'd taken my advice and shortened
their string, Stanford took off again.
ROCK Hands off.
ALAN ALDA Oh you've got it, you've got it, you've got it. (applause)
ROCK OK, let's not get another one. If we can just move it
to the center fence we can get points.
ALAN ALDA (NARRATION) The helicopter, all on its own, did everything
it could to complete its mission.
STEVE ROCK All right, that's it whew!
ALAN ALDA (NARRATION) But the one thing its designers didn't have
time to include was a way to drop the disk once it had reached
its target. Ironically, what they'd planned to use was a device
using a magnet and a cylinder.
ALAN ALDA Using that cylinder seems to be the way that the high
school team solved that problem too.
ROCK It's almost an identical solution to the high school
team; I was looking at that earlier. And they've got a real
neat little system and it's the same kind of an idea that
ALAN ALDA Is that as well as that's ever been done?
MICHELSON That's the best it's ever been done in the history
of the competition. And if they just had an intelligent device
to let go of the dog gone thing, they would have totally completed
ALAN ALDA (NARRATION) In the final scores, the Texas tailsitter
took third place, the Berlin blimp was second, and the Stanford
helicopter-- so close to completing the mission-- came in