and welcome to Scientific American Frontiers.I'm
problem with most robots is that they tend to be, well, robotic.
They know nothing they aren't programmed to know, and can do nothing
they aren't programmed to do. But for many applications where robots
could be useful, they need to be more like humans, able to respond
as a cooperative partner rather than a mindless machine.
this program, we'll meet some robots that are learning to figure
out for themselves what their human companions have in mind.
is a robot that's scrupulously polite as it tries to understand
what you want
turns on the charm as he learns to become an active partner
Robonaut figures one day to be a fully-fledged member of an astronaut
team, helping with construction tasks in space.
all coming up in tonight's episode, Robot Pals.
ALDA Hand me the one on my right.
ALDA Hand me the one on my right.
ALDA (NARRATION) Believe it or not, this robot is called Ripley.
He somehow I'm assuming it's a he lives at MIT's famous
Media Lab, where right now he's trying to understand me.
ALDA Let go of it!
HSIAO Yank. Yank harder. You can do it. There you go. That's it.
ALDA Hand me the blue one. Hand me the blue one.
ALDA (NARRATION) Ripley's world is confined to this circular white
tabletop. But within that world he has a pretty good notion of what's
where though right now he seems to think the camera is me and
he apparently hates to give things up.
Good boy. Good boy. Let go, let go!
ALDA (NARRATION) Ripley is actually the test bed for an attempt
to make robots better understand human speech
ALDA You know, this is really a weird sight a hand with eyes.
not just by listening to words but by trying to
relate those words to what it sees. It's programmed to keep looking
around, especially for human faces. But to keep its motors cool
it also takes frequent rests. So in its quest to understand its
world Ripley has to balance native curiosity with occasional naps,
much like a human infant and that's no coincidence.
ROY Here's the basic idea. When you were an infant, when I was an
infant, the way we learned language was by connecting words we heard
to what we could see and feel and touch and taste, and those are
the earliest bits of meaning that we picked up. And then we built
on top of that levels of abstraction, where we could talk about
things that were far more abstract. If you look at the way we build
machines today that you talk to, we are trying to run before we
can walk. We haven't built in those more primitive levels of meaning.
ALDA (NARRATION) The creature sitting next to Deb a robot called
Toco was the first to try to connect words with what it could
see. Toco was not much more than a camera on an arm
ROY That's a yellow ball.
ALDA (NARRATION) But by looking at objects from different angles
it was able to learn both what things look like and what they are
ROY Look, it's a red cup.
Look, it's a red cup.
ROY So we used Toco as a way to develop some ideas. But then we
went to an infant lab and we actually had mothers and infants play,
we didn't tell them why, we jut gave them toys and said, "play with
your infant." We put wireless microphones on the moms and recorded
Look what we've got. Peekaboo. He's got a blue hat, doesn't he?
And what is it? It's a bear, it's a bear, he's a good hider.
ALDA (NARRATION) Before an infant can attach words to things she
first has to recognize what a word is in the stream of sounds coming
from her mother's mouth. Deb wanted Toco to so the same. So
ROY We took the speech recordings and we brought back the toys.
We let Toco look at the toys and listen to the mothers' speech.
ALDA What happenened? It wet itself!
ROY Toco learned.
ALDA (NARRATION) Once Toco learned what things are called it took
the next step recognizing objects and putting names to them
in other words it went from learning language to generating it.
ALDA (NARRATION) But Toco had reached its intellectual limit. It
was time for a robot that could learn not just the names of things,
but their context.
ROY One of my favorite examples is, you've got an everyday simple
use of words. We're sitting across the table from each other like
right now, and there's a cup of coffee between us, and I say to
you, "this coffee is cold." So what is the meaning of those words?
On the one hand, there's something like, just imagine almost the
words are floating between us as we've spoken them, you can think
almost of an arrow, pointing down at the thing between us, that
you can see and I can see
ALDA "This" is already a clue.
ROY So "this" is a clue, "is" is a clue, I'm talking about the coffee
now, not yesterday, not tomorrow. The words "coffee" and "cold"
have meaning for you because you've tasted the stuff probably and
so have I. You know what cold means, not because you've looked up
in a dictionary but because you can feel what things are like when
they're cold and hot. So there's an experiential background that
we're both relying on.
ALDA (NARRATION) Enter Ripley sleepily.
ALDA He looks like a dog resting on the floor.
HSIAO He's up, he's back.
ALDA Don't make any sudden movements.
HSIAO Well he hasn't killed me yet so you figure you've got a little
bit of leeway.
ALDA He's pretty heavy, you could really
HSIAO Oh yeah, totally, those are 300lb base motors down there.
ALDA Holy moly. Oh!
HSIAO You alright?
ALDA Yeah I'm OK now. What do you suppose that was?
HSIAO I reset the system
ALDA You reset the system
HSIAO And it just kind of dropped.
ALDA (NARRATION) Once both Ripley and I recovered it was time to
do a little work on context.
ALDA Hand me the heavy one.
ALDA (NARRATION) Now, Ripley can't tell which object is heavier
just by looking at them. So
HSIAO So what it's got to do it's got to try it. Just like if you
were holding an object like, it's pretty heavy, you know.
ALDA Yeah he's weighing it a little.
ALDA (NARRATION) This is just one example of what Deb Roy calls
grounding connecting words to the physical world, something we
do naturally but that most speech recognition machines that look
up a word's meaning in an internal dictionary are clueless about.
HSIAO ‘Cos he hasn't go two arms. If you had two arms you'd be like,
well, this one's heavy, this one's light, you know
ALDA (NARRATION) Which is why to really understand what things are,
Ripley has to have a body to experience them with.
HSIAO And with any luck he'll decide which one is the heavier one.
ALDA Ah, thank you, thank you.
ALDA It's funny how easy it is to anthropomorphize it. Sometimes
it looks like a puppy, sometimes it looks like a snake. And as these
things start to get more sophisticated that's probably going to
be a factor in
what is it, I'm talking, excuse me, talking to
your maker, do you mind close your mouth
the way it looks is
probably going to have an effect on how I relate to it and how it
relates to me.
ROY Absolutely. You can't completely separate function from form.
So it's not just here's a piece of software that's running on a
computer. The computer has a body, and so just like we're not just
brains in a vat, we're brains that are embodied and the particulars
of how you look and how you sense and how you interact with the
world, they don't just shape the way that you perceive the world,
they shape the way that people perceive you.
ALDA (NARRATION) This is a theme we'll pick up in our next story.
ALDA Da da da-da da.
ALDA You know, I tried that in the army and it didn't work.
ALDA Watch out, watch out, don't get excited.
ALDA Some day being a robot will mean never having to say you're
ALDA Not yet!
BREAZEAL So sad when I take away your toy.
ALDA (NARRATION) Kismet is a robot dedicated to being cute. Built
by Cynthia Breazeal, Kismet's deliberately cartoon-like responses
were designed to encourage us humans to play with it. A couple of
years ago, Cynthia joined forces with a Hollywood special effects
wizard to build Kismet's successor.
WINSTON It was probably the most exciting thought concept that had
ever come to me in my career. I mean, we have created the most organic
looking and acting robots ever in the films that we've done. But
they don't have any brains. No offense to the puppeteers. But the
robot itself doesn't have any brains. And this was a chance to do
something that was groundbreaking and historic.
ALDA Oh my God. Look. What?
ALDA (NARRATION) This is Leonardo, who made its first public appearance
exclusively for Scientific American Frontiers.
La la la. ALAN ALDA You can make it do that? Or are they making
it do that?
WINSTON They're doing it.
ALDA Yeah, right. Can you do "woo woo woo woo
ALDA (NARRATION) At this point, Leonardo was still brainless.
ALDA Wow. Look at that tongue action.
ALDA (NARRATION) His 61 different moving parts were controlled by
puppeteers. The goal was to replace them with a rack of computers.
WINSTON This is the birth of the first truly artificial intelligent
character that will have the ability to emote, that can reach every
range of expression. That can hear you and respond to you as an
artificial intelligence. It's pretty amazing.
BREAZEAL What we really are exploring here is social connectedness.
Social interaction, the human-robot connection.
WINSTON What it is right now is a brand new instrument that has
yet to be learned how to play.
we've created a grand piano but it's never been
particular character has never been built before. So all of what's
happening with him is a learning process. This is like the first
day anybody's seen it. Puppeteers that are dealing with it right
now—this is a new character. So it's like "what things make it do
this?" So we have to learn how to use this instrument and then once
we have learned as human beings how to create an expression, how
to create a phonetic, how to make it go "woo", how to make it go
"hello", whatever it is
we put it all together
And the computer will know how to do it?
ALDA (NARRATION) His skin and fur back in Hollywood for further
primping, Leonardo or rather the computers that control him
had to acquire for himself the skills of Stan Winston's puppeteers.
Cameras in his eyes help Leonardo pick out human faces and gestures,
while microphones in his ears recognize voices and translate the
words they are saying. But Leonardo's ears can do more than just
BREAZEAL You can do the backstroke, the crawl, it can wave, hello.
You can see all the technology that's gone into creating a robot
that is really optimized for expression. The ability of these robots
to interact with people in a way that's natural for them, and that
they accept the robot, is going to be really critical. And I think
that always Leonardo is pushing the idea of not making it look human,
but kind of pet-like but intriguingly anthropomorphic, like a pet
but perhaps even smarter than a pet.
ALDA (NARRATION) With his fur back on and even with only his mouth
able to move Leonardo was certainly hard to resist.
BREAZEAL When you were in that little interaction, notice how much
you were adapting to it.
ALDA Right. Yeah.
BREAZEAL That's what I'm getting at. That's the kernel of the learning
interaction I'm trying to capture. And you need something like this
to make you act that way.
WINSTON And how much fun are you having
.? He will learn ‘cause
you're so interested in playing with him
BREAZEAL That's right.
WINSTON That the machine, the computer, the brain will learn. And
because of your interest in his organic reality, you'll make him
more real. Imagine having to act with a character that doesn't exist,
who isn't there—and pretend it's not there. Or to actually have
that fantasy character sitting there and acting with you. It allows
you a better performance.
ALDA That's it.
WINSTON Therefore, you as an actor are going to be more believable
on screen, we're going to believe the story, we're going to believe
the fantasy character and the best performance is going to come
out of you.
ALDA You know, you just made me understand Cynthia's work better
because as an actor I understand how much more will come out of
me if the other actor in the scene with me is behaving like a real
person. Now I understand through that what you mean about how I'll
feel when I sit down or stand up with my computer and talk to it
and it talks back like a person. And responds to me with facial
expressions like a person, I think I'll be in a different relationship
to the computer or the robot than I've ever been before.
WINSTON The wonderful thing about this is that we're sort of seeing
its birth. The robot is there. The brains are now coming in and
the learning process is gonna go year after year. It'll be so interesting
I cannot wait for the deadline, which I kind of—because I have gray
hair—I said "In six months, those brains have got to be connected."
Then in the next six months we see how much we can do.
ALDA You're talking about in the robot
WINSTON I'm talking about in the robot, yes. ‘Cause mine are gone.
Press the red button.
ALDA (NARRATION) Since we first met Leonardo his brains have indeed
been connected, and he has begun to collaborate with his human designers
in learning a simple task.
Turn all the buttons on.
ALDA (NARRATION) Leonardo appears obviously baffled by this instruction
I can teach you how to turn the buttons on.
and it's here that his social skills are serving
him well. STUDENT Press the green button.
ALDA (NARRATION) Leonardo uses his gaze both to show what he's done
and to seek approval.
Now all the buttons are on.
ALDA (NARRATION) The relationship between a teacher and learner
is at its best a constant exchange of social signals, enabling each
to see into the mind of the other.
Can you turn all the buttons on?
ALDA (NARRATION) And that's one of the goals for Leonardo for
him to become a better learner by being able to keep checking in
with his human instructor for feedback, and by being able to signal
his own impressions of how well he's doing.
ALDA (NARRATION) Leonardo is gradually emerging as the first ever
robot to achieve the goal so beloved in science fiction
Not quite. Push the red button.
ALDA (NARRATION) Of becoming a friendly, smart and helpful mechanical
companion, socially adept and eager to learn. The collaboration
between Hollywood chutzpah and MIT know-how is unique and if Stan
Winston and Cynthia Breazeal can pull it off, may indeed result
in a robot that's not only useful but that you can care about
and that will at least give the impression of caring about you.
ALDA Want to go out for a cup of coffee later?
ALDA (NARRATION) We're at the Johnson Space Center in Houston, where
astronauts are trained. But Josh Mehling isn't planning to be an
astronaut. He's training to become a Robonaut or rather a Robonaut's
puppet master. The Robonaut itself has recently acquired a set of
wheels odd, you might think for a robot designed to help out in
space. We'll come back to those wheels in a moment. Right now Josh
is gradually bringing the rest of his robot counterpart on line.
When Josh moves his head, Robonaut moves his head. When Josh moves
his arms, Robonaut follows along. And as Josh flexes his fingers,
the exact motion is picked up by sensors in his gloves and sent
to the electric motors controlling Robonaut's fingers. In fact,
Robonaut's body is as much like a human's at least from his waist
on up as his chief creator, Rob Ambrose, can make him with exceptions.
AMBROSE So what you won't find is the appendix didn't copy that
from the human anatomy. There's no reason to copy things if they
don't have some function, so we really started with the functions.
And the functions are, two dextrous hands with a sweet zone right
here where two arms can work together, and it's not an accident
that that zone right here where the two arms are so nice, the head
sees that zone perfectly.
DIFTLER OK, Robonaut, reach out and take this handrail please.
ALDA (NARRATION) Josh shares Robonaut's view of the sweet zone through
DIFTLER That's a good grasp. Thank you.
ALDA (NARRATION) This tool in NASA-speak is an EVA Tether Hook
what astronauts use in space to attach themselves to the handrails
on the outside of their spacecraft.
DIFTLER Now Josh is going to do a more complicated task. He's going
to remove it from the handrail without me helping him. His is simulating
things floating in space. What's really nice about this is that
Josh only has about 12 or 14 hours experience working with Robonaut,
but because it's such a natural system to use, it was designed with
body parts similar to what a human has, it's a very intuitive system
to work with. All the skills we've developed since we were children
transfer very nicely to the robot.
ALDA (NARRATION) Robonaut started out as one arm and a head, suspended
in Rob Ambrose's lab. It was here that the team developed Robonaut's
stereo vision and began work on the most complex and sophisticated
of its body parts its hand. Dozens of electric motors control
Robonaut's fingers, allowing the thumb and first two fingers to
flex and rotate like human fingers. The ring finger and pinkie simply
open and close but because the palm can also flex, the hand can
grasp objects securely. The robot Josh is controlling is Robonaut
B, the younger brother of the original. And while the gloves covering
Robonaut B's hands are much more than cosmetic, they are not as
fancy as those now being tested on Robonaut A.
AMBROSE Younger brothers seem always to get the hand-me-down gloves,
but we try out the latest here on A. You can tell by the dirtiness
of the glove this robot's been put to some serious work. This is
the third generation glove that we've developed, it has tactile
sensors all along the underside of the fingers and on the palm,
kind of like callous points, and out in the palmar fascia. It's
able to sense where objects are after it's grasped them and pulled
them into its palm. And that's part of the most important signature
of what it means to have a good grip on an object.
ALDA (NARRATION) Getting a good grip matters more than usual when
working in space, and here Robonaut has an advantage over his human
co-workers by being designed with a single plug-in leg. In this
NASA animation, Robonaut is helping build the International Space
Station by setting up equipment before a human astronaut arrives
and dutifully putting it away when the astronaut's work is done.
Which brings us back to Robonaut B, which has no legs at all, but
wheels. And not just any wheels, but those of a self-balancing Segway
AMBROSE No, you're not going to see a Segway on the Moon, and it
makes no sense in zero-G, they'd just be fly wheels causing it to
pivot around. But what it allowed us to do very quickly was to learn
how to coordinate moving the torso and arms on the torso. When we
first built it we weren't even sure if one person would be enough
to drive it. We were pleased to find that a single person could
both coordinate moving the body and reaching out with the arms.
ALDA (NARRATION) But the ultimate goal is to eliminate Robonaut's
puppet master altogether. The NASA researchers have begun working
with the Media Lab team behind Leonardo, to give Robonaut the ability
to work autonomously as a member of a team, able to interpret it's
fellow astronaut's voice commands and even gestures.
AMBROSE Recently we've been working with a human paired with the
machine, where the human is more of the experienced master technician,
able to task the junior member of the team, the robot, as an apprentice.
"Robonaut, go torque that bolt, that one right there, I'll be back
in a few minutes, I'm going over there, call me if you have a problem."
We know when we've gotten there when the person standing next to
it, working, floating next to it, is able to just pretend it's another
ALDA (NARRATION) A robot astronaut would not only be helpful building
structures in space. It or more likely they could be sent ahead
on missions to the Moon or Mars, perhaps mounted on wheels like
a centaur, to prepare the way for humans.
AMBROSE What I would like, if I were going to be sent to Mars, is
I would like the robots already to have set up the habitat, have
already manufactured the fuel for the return vehicle, have tested
the rockets of the return vehicle to make sure that they're working
fine, the habitat is already filled with oxygen and at that point
I would be really excited leaving Earth to go live in this habitat,
knowing everything is working fine. That's what we'd like the robot
to do, long before the people get there, is to get everything set
ALDA (NARRATION) Already NASA has an active research program in
robotic vehicles, designed to accompany astronauts on lunar or Martian
expeditions. But the growing skill of robots especially robots
that can do jobs unsupervised brings up the obvious question.
If robots can do so much to explore other worlds, why send humans
AMBROSE There's a reason humans go to the beach. It's fun, it's
exciting, sometimes you discover things at the beach. I could send
the robot to the beach and it could do a much better job recording
the temperature there than me, do a much better job taking pictures
of the beach. But at some point, I want to go to the beach. Because
I'm a human.