FRIENDLY
CHARACTERS
ALAN ALDA (NARRATION) It was six years ago that I had
my first encounter with a sociable computer - or at
least one that tried to be sociable.
ALAN ALDA OK. KRIS THORRISON How does that feel?
ALAN ALDA Now these wires go from the gloves down my
back? KRIS THORRISON Down your back and into the computer.
And this eyetracker will allow it to see your eye movements.
ALAN ALDA (NARRATION) I was visiting MIT's Media Lab
- a visit which in those days often involved getting
rigged up in elaborate outfits designed to keep track
of what I was doing - and in this case, where I was
looking.
ALAN ALDA Oh, my god! Look at my eye on that monitor.
ALAN ALDA (NARRATION) The gloves were to allow the
machine to follow my gestures.
ALAN ALDA Oh, it speaks Italian
ALAN ALDA (NARRATION) The idea of all this was to allow
me to gesture to my computer.
ALAN ALDA I actually gesture at my computer quite a
bit, but the computer doesn't respond. In that regard
I can understand how it would be good to have gestures
understood by the computer, but why are you engaged
in this? Why do you want the computer to read my gestures?
JUSTINE CASSELL Because we do gesture. We gesture at
computers, we gesture at each other, like I'm doing
right now. We even gesture when we talk on the telephone.
So yeah, there you are, you're saying to your computer,
not that, the... I told you...
ALAN ALDA But mostly it's like this. I can go like
that.
JUSTINE CASSELL And it still doesn't respond.
ALAN ALDA No, it doesn't. It doesn't understand.
JUSTINE CASSELL A slow computer. Well, we'd like to
speed it up a little bit.
ALAN ALDA (NARRATION) The Media Lab is famous for its
"demos", demonstrations of technology that is still
far from perfected, but gives us a glimpse of the possible.
The idea here was for a character I could converse with.
GANDALF Hello, I am Gandalf, your guide to the galaxy.
ALAN ALDA Take me to Mars.
GANDALF Why not?
ALAN ALDA I love that eye action.
GANDALF Goodbye.
ALAN ALDA No wait, where are you going? Wait. Are you
going to leave me out here in space?
ALAN ALDA (NARRATION) Gandalf had trouble recognizing
my speech, let alone my gestures.
ALAN ALDA Tell me about Mars.
GANDALF Mars has two moons.
ALAN ALDA Tell me more.
GANDALF Mars is 6,000 kilometer in diameter.
ALAN ALDA (NARRATION) But he did have a cute personality.
GANDALF Mars is a cool planet.
ALAN ALDA You said it. Uh, what was the point of wearing
all this? What did that do? How did that change anything?
JUSTINE CASSELL So we talked earlier about interaction
of verbal and nonverbal behavior. You saw an example
of that. He knows, Gandalf knows when you're looking
at him, and he knows when you're looking at the screen.
And when you looked at the screen, he looked at the
screen too, just to be friendly. And when you looked
at him, he got ready to answer a question, or to interact
with you. When you looked away, he took a powder. You
can...
ALAN ALDA Did my making gestures change the way...
I didn't see a change in the way he answered questions
when I used my hands.
JUSTINE CASSELL In the future he'll understand the
content of your gestures. For the moment, what he understands
is that you're speaking.
ALAN ALDA The ways we've thought of computers up until
now has sort of kept computers in the box, and the way
you're thinking of computers is giving them a chance
to get out of the box by thinking of them more as people.
I mean, you're making them listen and feel and hear
and see like people, so then they can be all over the
place. They don't have to be only accessed by a keyboard
and a mouse.
JUSTINE CASSELL Absolutely. That's the point. They
should be able to be social actors. Actors in our social
world. They're not going to be people because they're
not people and we don't want to think that they're people
ultimately. I don't.
ALAN ALDA But they might be easier to use if they got
information from us the way other people do.
JUSTINE CASSELL Absolutely. That's the point.
ALAN ALDA (NARRATION) Today, Gandalf has retired, to
be replaced by REA, whose expertise is much more down
to earth.
TIMOTHY BICKMORE I'm starting up her brains. It takes
couple of minutes. She's in a deep sleep.
ALAN ALDA This is Gandalf's grand-daughter.
TIMOTHY BICKMORE That's right. We've made a number
of advances over the years. She does full gesture and
speech generation. So when she has something to say
she actually decides whether she should put it into
her speech or her hand gestures or both. So she's more
natural in that way in interacting with her. She recognizes
everything… well, not everything you say but everything
you say about real estate.
ALAN ALDA She's got a one-track mind.
TIMOTHY BICKMORE That's right, that's right.
ALAN ALDA Her grandfather needed me to get all duded
up in an outfit. Gloves on my hands and a helmet on.
She doesn't need that?
TIMOTHY BICKMORE She doesn't need that because she
has two cameras up here that recognize the position
of your head and hands in space. And from that she can
determine where you're standing so she can gaze as you
move around and look at you, she can recognize…
REA Hello.
TIMOTHY BICKMORE …if you're gesturing with your hands
or not.
ALAN ALDA Hello, hi.
REA How are you doing?
ALAN ALDA Pretty well. Pardon me?
REA Have you been in the Media Lab before?
ALAN ALDA Yes, I have.
REA Cool.
ALAN ALDA Cool.
REA They are doing some crazy things in here.
ALAN ALDA (NARRATION) Tim and Justine have found that
people tend to trust computer characters more if they
chit-chat before getting down to business.
ALAN ALDA What did she say?
ALAN ALDA (NARRATION) But Rea's patience with me quickly
runs out.
TIMOTHY BICKMORE She said goodbye.
ALAN ALDA She said goodbye?
JUSTINE CASSELL Yeah, I've never seen her do that before.
ALAN ALDA Look, I swear to God, I really think it's
cool.
ALAN ALDA (NARRATION) So Tim resets Rea's brain and
we try again. REA Hello
ALAN ALDA Hello
REA How do you like Boston?
ALAN ALDA Oh I like Boston. (Pause.) What? Don't go.
No, no. I want to buy a house. What, what?
REA How about this weather?
ALAN ALDA Yeah, it's cool. Uh, do you sell houses or
anything? Rea, real estate. Does that strike a chord?
This is the best example of soft sell I've ever seen.
REA She figures if you don't want to buy, she doesn't
want to push you.
ALAN ALDA I think she just sees in me that I'm not
really serious. REA I think so. Look at her, she looks
sad.
ALAN ALDA She really does.
REA I have a house.
ALAN ALDA (NARRATION) But finally Rea gets down to
business.
REA It has one bedroom, one bathroom, one living room,
one kitchen.
ALAN ALDA Could I see the kitchen?
ALAN ALDA (NARRATION) In theory, she's listening to
what I'm saying - with both my voice and my hands…
ALAN ALDA Where is the living room?
ALAN ALDA (NARRATION) …and then figuring out how to
respond - again with both voice and gestures.
REA The living room is downstairs across from the kitchen.
ALAN ALDA Uh, uh. Could I see the living room?
REA Were you talking to me?
ALAN ALDA Yes, I was.
REA I'm sorry, I did not quite catch that.
ALAN ALDA That's OK.
ALAN ALDA (NARRATION) Like Gandalf before her, Rea's
dependent on an off-the-shelf speech recognition system
- which is having trouble understanding me.
ALAN ALDA Show me the living room. Ah, here we go.
Ah, nice. Nice TV set there. Tell me about the living
room.
REA I'm sorry I did not quite catch that.
ALAN ALDA I bet your granddad was Gandalf.
JUSTINE CASSELL You have this remarkable effect on
her and I think it's actually because you have a low
pitched voice. I would suggest that you watch Tim interacting
with her.
ALAN ALDA Yeah, sure.
REA Let me see what I have available.
ALAN ALDA (NARRATION) Rea has had months of practice
recognizing Tim's speech, so their interaction goes
much more smoothly.
REA What can I tell you about those rooms?
TIMOTHY BICKMORE Tell me about the house.
JUSTINE CASSELL Once we get recognized speech from
this commercial recognition system, we're understanding
what the person is saying, and then Rea in real time,
as you're having this conversation, is using her intelligence
to decide what an appropriate response would be, to
formulate that response in English, and also to know
how to use her body. And that had never been done before.
TIMOTHY BICKMORE Where is the bedroom?
REA The bedroom is upstairs.
ALAN ALDA (NARRATION) Justine hopes that Rea will evolve
into virtual characters that could make all sorts of
interactions with the computer easier and more natural.
She's already developing on a fitness instructor that
could keep an eye on you as you work out at home, a
tutor for distance learning…
REA Take care.
ALAN ALDA (NARRATION) And Sam, a virtual playmate for
children.
JUSTINE CASSELL When Geneva starts to play, Sam is
all about eliciting from her the best of her language,
the best of her story-telling, and then giving her what
she needs to be encouraged, to be sophisticated in her
language use and to be actually focused on the kind
of language use that are going to help her as she goes
through school.
ALAN ALDA So Sam hears her speaking, not only understands
the story part of it but understands what she means
and picks out the best parts of her story telling and
encourages that? That sounds like an awful lot for Sam
to know.
JUSTINE CASSELL Right. And Sam doesn't do any of that.
ALAN ALDA Who does that?
JUSTINE CASSELL Tune back in in a couple of years when
we can do that.
ALAN ALDA (NARRATION) What Sam can do already is share
the Magic Tower with Geneva…
SAM Cool. OK, my turn…
ALAN ALDA (NARRATION) Even sharing toys across the
boundary that divides the real and virtual worlds.
ALAN ALDA How does Sam know what she just did?
JUSTINE CASSELL There are sensors in the bottom of
the toys, and sensors in the Magic Tower. And that's
what lets Sam know whether one toy is in there or two
toys.
SAM Long ago there was a fairy an old man and a wolf…
ALAN ALDA (NARRATION) Sam's background is generated
by a video camera in Geneva's world, so even the room
is shared between them.
SAM Help, I'm a fairy…
ALAN ALDA (NARRATION) Sam is happy to tell his own
stories, or to listen when it's Geneva's turn - prompting
her if she pauses.
SAM What happened next?
GENEVA So she took the watering can and she poured
it on each of her fishes…
ALAN ALDA She's just totally inspired by this. Would
she be that active in her story telling if she wasn't
prompted by Sam?
JUSTINE CASSELL That's exactly what we looked at. So
we ran a big long study in a local school, and we had
four conditions. One kid playing alone without Sam,
two kids playing with each other but without Sam, one
kid with Sam and two kids with Sam. And what we find
is that without Sam they stall, they hum, hey look up
at he ceiling and that's it, they don't tell. And one
kid with Sam, even with this minimal feedback, and the
fact that when she's finished Sam is going to take a
turn, seems to inspire children to be really creative.
GENEVA She gave the boy a radish and a lemon…
SAM Then what happened?
GENEVA And picked them up and put them down.
ALAN ALDA It's very interesting. She's moving these
figures around the house and every time she puts it
in a new room she looks up at Sam and gets a reaction.
JUSTINE CASSELL Right. One of the reasons that I launched
this project was in response to what I saw as a series
of games that talked at children. Computer games and
other games that just bombard the kid with images and
noise, and I wanted something that listened and that
evoked real physical play in the physical world, that
got kids to move around.
ALAN ALDA Geneva, how much fun is Sam as a playmate?
Do you like playing with Sam?
GENEVA Yeah.
ALAN ALDA What do you like about Sam?
GENEVA Because he listens and he says what happens
next and stuff.
ALAN ALDA (NARRATION) While Sam encourages his playmates
to make up stories, his own are all pre-recorded. But
another character in Justine's lab - a lot less wholesome-looking
than Sam - is able to put together sentences on the
fly.
BEAT If you want it to speak, you just have to type
something.
HANNES VILJHALMSSON In a way, Beat actually is sort
of like text-to-speech, except that it's not only producing
speech, it's producing the gesture that goes along with
the speech. So it takes an input, text, and outputs
an animation script that a character can execute. BEAT
This is both good news and bad news.
ALAN ALDA OK, well that's really interesting, because
this is both good news and bad news…
JUSTINE CASSELL Which is by the way exactly what you
did…
ALAN ALDA What I did, I know. But how did he…wait a
minute, he got it from "both," huh?
JUSTINE CASSELL No.
ALAN ALDA How did he know that there was this and this?
JUSTINE CASSELL Right. Because one of the things that
it knows about the world is that there are sets of things
in which one stands in contrast to the other. So we'll
let you type in a bunch of other sentences that have
contrasts in them, and you'll see that it will always
understand, if it has that knowledge in its data base.
BEAT Sometimes I like blue things and sometimes I like
red things.
ALAN ALDA Does this and that.
JUSTINE CASSELL Why did he point towards the truck?
ALAN ALDA Oh wait, wait. The truck is blue. But he
pointed to it when he said red. Let me see that again.
HANNES VILJHALMSSON He actually likes blue better.
BEAT Sometimes I like blue things and sometimes I like
red things.
JUSTINE CASSELL Now, why did he do it differently the
second time?
ALAN ALDA Don't ask me. I didn't invent it!
JUSTINE CASSELL No, I'm going to tell you. It's the
Socratic method. I'm seeing if it's in you somewhere.
But since it isn't I'll tell you. So in fact, not only
does this system have a model of the world, it also
has something else that we have, which is a model of
what we've said previously. And he knows that he already
said blue thing. So there's no need to gesture again.
You now share it.
ALAN ALDA (NARRATION) Beat also knows a little about
where he's standing.
HANNES VILJHALMSSON He actually knows what the scene
looks like.. BEAT I was just visiting the Media Lab.
HANNES VILJHALMSSON So he knows where the Media Lab
is and is able to point at it.
ALAN ALDA (NARRATION) Let's see how he says a new sentence
mentioning the lab.
BEAT Soon the Media Lab will have another building
attached to it.
ALAN ALDA Just a little half gesture toward it.
JUSTINE CASSELL Exactly, and that half gesture, he's
gesturing when he's saying the new thing, another building
attached to it, because that catches your interest and
pulls you to the contentful part of the sentence.
ALAN ALDA Let me see what he does if I let him say
it again. He probably won't make that same gesture,
will he?
BEAT Soon the Media Lab will have another building
attached to it.
ALAN ALDA It's almost in response to the unspoken question,
what was that you said, because he's just telling you
the words.
JUSTINE CASSELL Exactly, yeah.
ALAN ALDA If I do it again will he give me a look like,
I told you already.
JUSTINE CASSELL He'll start rolling his eyes.
BEAT I really like the "automated actor."
ALAN ALDA (NARRATION) Giving a cartoon character the
ability to interpret typed speech as if he understands
it may be useful to animators, but it isn't yet a threat
to the acting profession.
BEAT Now this is a lot easier than dealing with a real
actor.
JUSTINE CASSELL That's you he's pointing at. And what
we did to do this was, he already knows that in his
world is this building called the Media Lab, he's got
a blue truck, he knows that he's a so-called actor.
Hannes just went in and stuck another object into his
world, an object called Alan Alda equals actor, and
a real one.
ALAN ALDA Let's see…
ALAN ALDA (NARRATION) Well, if I'm going to be just
another object in Beat's little world, at least I want
better billing than his truck.
BEAT Alan Alda is a real actor, and a great one.
ALAN ALDA I think this is a wonderful program.
JUSTINE CASSELL Yeah, you want to take it home.
ALAN ALDA I think this gives you confidence.
ALPHA WOLF
ALAN ALDA (NARRATION) Bruce Blumberg loves dogs - especially
his silky terrier Sydney. And dogs have long inspired
Bruce's efforts to make computers more sociable.
ALAN ALDA Hello Silas.
ALAN ALDA (NARRATION) Silas, for instance, another
of Bruce's dogs, once lived happily in a room at the
Media Lab.
ALAN ALDA Where are you going? Hey, come here.
ALAN ALDA (NARRATION) Although only a virtual dog,
he'd been trained to respond to the gestures of the
people who entered his life.
ALAN ALDA Sit, sit. Oh wow, look at that.
BRUCE BLUMBERG He's like a real dog, that sometimes
he'll respond to your gestures and sometimes he won't.
ALAN ALDA Does he beg?
BRUCE BLUMBERG Yeah. Put both your hands over your
head.
ALAN ALDA Wow.
BRUCE BLUMBERG Now sometimes Silas will do what he
wants. So right now I think he's heading off to get
a ball, because he figures…
ALAN ALDA Here he is, here he is. Give me the ball,
give me the ball. OK, now I'll throw the ball for him.
ALAN ALDA (NARRATION) Several years -- and several
thousands of lines of computer code -- later, Silas
left his cramped little room and grew up to become Duncan,
a sheepdog with an entire Scottish estate to roam around
and a flock of virtual and rather flighty sheep to look
after.
BRUCE BLUMBERG Away.
ALAN ALDA (NARRATION) Bruce is trying to train Duncan
much as he would his real dog.
BRUCE BLUMBERG When I tell him "away" for example,
he understands that command, but then it's up to his
own intelligence in order to figure out how far away
from the sheep to go, how to avoid the obstacles that
he sees. Very much as if you had a puppy and the puppy
comes with some basic abilities and then you're training
it to use this action to get particular…
ALAN ALDA Now he's got them in kind of a lump over
there.
BRUCE BLUMBERG Yeah, that's good. Maybe I shouldn't
do anything. He's doing a great job of herding if I
don't do anything. Would you like to try it?
ALAN ALDA Yeah, sure.
BRUCE BLUMBERG Alright.
ALAN ALDA Away.
ALAN ALDA (NARRATION) Things start off promisingly…
ALAN ALDA Steady. I've lost a sheep here.
ALAN ALDA (NARRATION) But then Duncan and I seem to
go our different ways.
ALAN ALDA Bye. Pay attention. Bye. No… He's just separated
the whole herd of sheep here.
BRUCE BLUMBERG Someone's separated the whole herd of
sheep.
ALAN ALDA Well, if you don't mind I'll blame him. Now
this really calls to mind an interesting question. Let's
say I got really good at this…
BRUCE BLUMBERG Why.
ALAN ALDA Yeah, why? Why have you gone to all this
trouble?
BRUCE BLUMBERG Well, the fundamental thing I'm interested
in is sort of the nature of intelligence. What is it
that sort of is going on in our brains that allows us
to have the every day kind of intelligence that gets
us through our day? And for me, dogs are a perfect model
of that. Because dogs really have dinky little brains
-- their brains are the size of lemons if they're a
big dog -- and yet they have the kind of common sense
that underlies 90% of our behavior.
ALAN ALDA (NARRATION) Bruce uses a clicker to give
Duncan a virtual reward when he does the right thing.
Now I'm going to see if I can teach him something.
ALAN ALDA Jump.
ALAN ALDA (NARRATION) I first wait till he jumps, then
say the word jump as I reward him with a click.
ALAN ALDA Jump.
ALAN ALDA (NARRATION) After several training runs…
ALAN ALDA Jump.
ALAN ALDA (NARRATION) Will Duncan treat the word as
a command?
BRUCE BLUMBERG Cool. Cool. Do it again. Do it again.
ALAN ALDA Jump.
BRUCE BLUMBERG Awesome. Awesome. You know you are the…
this is a pretty fresh demo, meaning you are the first
person outside of the group that's actually trained
this dog.
ALAN ALDA Oh that must make you feel great that an
amateur can do it, that's great. (Wuff.)
ALAN ALDA (NARRATION) As a graduate sheepdog trainer,
my next assignment is tougher - raising a wolf pup.
BILL TOMLINSON OK, so now he's woken up.
ALAN ALDA I'm sorry, I'm laughing like a wolf. Do they
laugh? Now what?
BILL TOMLINSON Try howling for him.
ALAN ALDA (Howl.)
ALAN ALDA (NARRATION) This cute little black wolf pup
is mine to mold - to shape his social interactions with
two other pups, one being raised by Bill Tomlinson,
the other by Marc Downie. We can move our pups with
a mouse, but the only way to control how they behave
is through the sounds that wolves themselves make.
ALAN ALDA (Grr. Wuff. Wuff.)
BILL TOMLINSON Barking
means play so you're giving him a mixed signal.
ALAN ALDA I'm giving him mixed signals…
BILL TOMLINSON
You are. But just a nice grr, grr…
ALAN ALDA OK, I see. You know, you're threatening me
when you do that. No seriously, I'm a little frightened
of you.
ALAN ALDA (NARRATION) Just as Duncan the sheepdog models
animal intelligence, these wolf pups are intended to
model social behavior.
BILL TOMLINSON (Whine.)
ALAN ALDA That's better. (Grr, Grr.) Yeah, that's more
like it.
BRUCE BLUMBERG Wolves are really pretty amazing creatures.
A lot of a wolf's day is taken up sort of sorting out
the social relationship between it and the other members
of the pack. And what we're trying to do here is to
understand what sort of things might be going on in
the brain to be able to do that.
ALAN ALDA (NARRATION) Bruce and Bill got their introduction
to real wolf behavior through a visit to Wolf Park in
Battle Ground Indiana, where several packs of grey wolves
live in a semi-natural environment. Each pack has a
strict social hierarchy, with an alpha wolf at the top.
The Media Lab researchers are now trying to model the
way wolf puppies find their place in that hierarchy.
BILL TOMLINSON If you have your pup coming at me and
growling a lot, my pup will learn that you tend to be
a more dominant individual and it will in turn form
a submissive relationship with your pup.
ALAN ALDA Now, how is your pup going to learn that
if you keep giving him sounds to make that go against
what he's learning?
BILL TOMLINSON Well, it exhibits
that relationship by the emotional style in which it
performs the actions that you ask it to take. So if
you tell it to growl at somebody who is always dominant
to it, it will do so but it will growl kind of like
this, knowing that it's a bad move and that no good
can come from growling at this individual. Whereas if
you try to growl at somebody that you're traditionally
dominant to, it will run over and do it very confidently.
ALAN ALDA Well let me see what happens if I don't give
up and you don't give up.
BILL TOMLINSON OK.
ALAN ALDA I mean, I want to see, can get to a kind
of stand-off here. Let me see what happens. (Wuff.)
There, wake up (Howl). I'm going to go visit you (Howl).
There, that's the boy.
BILL TOMLINSON (Grr.)
ALAN ALDA (Grr.) Get up, get up.
BILL TOMLINSON I was
all over you on that one.
ALAN ALDA Alright, well I'll try that again. (Grr.)
BILL TOMLINSON (Grr.)
ALAN ALDA (Grr.) There he's getting all wimpy again.
What is this? Is it me?
BILL TOMLINSON No, I got a head
start, because I beat you down the last time and so
it's just easier this time. One of the problems too
is that human speech gets recognized as whining., which
is an interesting thing.
ALAN ALDA Human speech gets recognized as whining?
You know this has been my problem in life. I should
just keep growling.
BILL TOMLINSON If you want a dominant
wolf you have to make sure not to talk at all.
ALAN ALDA I'll talk like this.
BILL TOMLINSON There
you go.
ALAN ALDA Hey, turn around. (Grr.) I think I won that
one. I think I finally got the idea now. Now I know
how to dominate these guys.
ALAN ALDA Uh oh, now that's the adult. Now what if
I tried to get the adult to back down?
BILL TOMLINSON
Try growling at him.
ALAN ALDA (Grr.) It doesn't do any good. Look at this.
I'm totally cowed by this. So what they've learned as
puppies, that personality will carry on. For instance,
if I had been dominant as a puppy, will I be the dominant
adult?
BILL TOMLINSON Exactly. Your adult and my adult
will continue to manifest the same relationships that
we trained into them.
ALAN ALDA This is fascinating and fun. What do you
think it could possibly lead to?
BRUCE BLUMBERG Well, I think there are a couple of
different ways that I would love to see this work out.
Number one, as a tool ultimately for understanding more
about the social behavior of wolves. You know, all the
work that we do, I really want it to inform our understanding
of the natural world. Number two is that one of the
things that computational systems don't do is social
behavior. And social behavior is an important aspect
of natural systems, so I'm very interested in what would
be the benefits of incorporating social behavior into
computational systems.
ALAN ALDA (NARRATION) These wolf pups are of course
nothing more than complex software programs. But they're
programs that exhibit socially appropriate behavior,
emotionally charged memories, and the ability to learn
from their experiences - all features that would be
desirable in the computer now sitting on your desk.
Just as long as you don't have to growl at it - and
have it growl back.
BILL TOMLINSON People are often
worried about making computers that are sentient and
social, in that, you know, like the Terminator, they're
all going to come back and kill us all and it's going
to be this horrible thing. But people all around the
world are making entities that are stronger and smarter
and more interesting and more capable than us. We have
children. And if the process of making computational
systems can be more like making children that making
bombs, hopefully it will make computational systems
that don't turn out to be bombs but turn out to be more
like children in the way that they interact with us.
And just as I don't grow up to want to kill my parents,
hopefully the computational systems will want to have
wonderful interactions with us over the course of our
lives.
ALAN ALDA So we're going to have to deal with adolescent
computers?
BILL TOMLINSON I mean that is one of the
challenges. If you have a system that learns, at some
point, it hasn't learned yet. And that is a problem.
ALAN ALDA (NARRATION) Once our pups grew up, they sat
around and howled.
ALAN ALDA What are they all howling at?
BRUCE BLUMBERG Actually, real wolves do this, it's
called a rally howl, particularly in the early morning
and late afternoon.
ALAN ALDA (NARRATION) And since my wolf is one of them,
it seems appropriate to join in.
ALAN ALDA It's a fade out.
GETTING TO KNOW US
DOOR Welcome to Ted Selker's office. What is your name?
TED SELKER Ted Selker. DOOR Hi Ted Selker. Welcome to
your office.
TED SELKER I'll go into my office here.
Why don't you try to be Ted Selker?
ALAN ALDA OK, I'll try to be Ted Selker.
ALAN ALDA (NARRATION) Ted Selker invented that little
button that acts like a mouse in the middle of the keyboard
of IBM laptops. Now he's here at the Media Lab, where
his mission is to make computers aware of where we are
and what we're doing. The door knows there's someone
here - and it's not about to be fooled.
DOOR Hi Ted Selker. Wait, Ted Selker is already in
his office.
TED SELKER Let's see. I guess I'll let you in.
DOOR Ted Selker is ready to see you. Please come in.
ALAN ALDA Thanks, my heels are really cool by now.
Thank you.
TED SELKER Now, if somebody else comes up,
it's going to do something a little different. Why don't
you try to go in again?
ALAN ALDA I'll be somebody else. I get to play two
parts in this, it's great.
DOOR Welcome to Ted Selker's office. What is your name?
ALAN ALDA Ralph.
DOOR Sorry, I do not understand you. Please type your
name here.
ALAN ALDA (NARRATION) As usual, I'm having trouble
getting the voice recognition system to understand me.
DOOR Hi. You don't have an appointment with Ted Selker,
and he is busy right now. He has just started a meeting.
If this is important, please knock. Otherwise please
arrange an appointment or leave a message.
ALAN ALDA This is the knocker?
DOOR I will inform him that you are here. Please wait.
Sorry, it seems that Ted Selker is very busy.
ALAN ALDA Does he know it's about insurance?
ALAN ALDA (NARRATION) Well, I'll try my luck at making
an appointment.
DOOR No appointment set. Goodbye.
ALAN ALDA Do you know if he has insurance?
ALAN ALDA (NARRATION) Even after you've made it into
Ted Selker's lab, your every move is tracked - first
by a swarm of virtual bees, then by a clown-face stepping
on your foot.
COMPUTER Welcome.
TED SELKER We can measure
how heavy a person is and where they're standing and
it's neat because it's only about 20 cents a square
foot. So we're able to know these things and we use
them to tell, you know, social things too…
ALAN ALDA So in other words if the floor knows how
much I weigh then when I walk around does it track me
by how much I weigh?
TED SELKER Yeah.
ALAN ALDA So it knows where I am all the time?
TED SELKER So if you lose your keys, we can just tell it,
hey, where's that guy been?
ALAN ALDA Every floor could be made like this.
TED SELKER That's a good idea.
ALAN ALDA Then you really could track everything that
you do around the house.
TED SELKER Well, you could,
like, when you pour some milk into a recipe, you could
know when you've poured a cup in.
ALAN ALDA By the change in weight?
TED SELKER Yeah.
ALAN ALDA It's that sensitive?
TED SELKER Oh yeah…
ALAN ALDA (NARRATION) Intelligent doors, intelligent
floors…
ALAN ALDA This is an intelligent bed.
TED SELKER That's
right. So you're going to put your head underneath this
Luxor lamp over here. I bet you've got an eye. There
it is.
ALAN ALDA (NARRATION) They are all part of Ted Selker's
goal of using simple, cheap sensors everywhere around
us to keep track of what we're doing and anticipate
our wants.
TED SELKER If you kind of blink rapidly it
changed the music. And the reason it changed the music
is because it decided you weren't happy. People blink
when they're not happy. Now if you just close your eyes
for a moment, the system will think you're asleep and
now it puts z's across the screen, Now if you open your
eyes, the sun rises and you wake up kind of thing. Here
we'll try to have you do a couple of other things. Why
don't you look at this newspaper for a moment, stare
at it.
ALAN ALDA (NARRATION) By picking up my stares, the
sensor in the lamp over the bed allows me to browse
my email, read a book, even control a video game.
ALAN ALDA I imagine that eventually you're going to
find a way to find my eye no matter where I am in the
room, I don't have to be lying on a bed.
TED SELKER
Yeah, yeah, yeah.
ALAN ALDA (NARRATION) The lab has already developed
a simple pair of eyeglasses that are able to tell if
I'm staring at something - or even somebody. Selker
sees the day when people could exchange much more than
a glance when they gaze at each other.
TED SELKER You
being interested in me gives me your business card.
In a party, the person that's paying attention to someone
is giving all these people their cards.
ALAN ALDA I can just imagine that certain people would
go home with an awful lot of business cards!
ALAN ALDA (NARRATION) But first the glasses would have
to be much less obtrusive.
RESEARCHER It can be reduced to a very very small size.
We use it just to demonstrate that we have something,
otherwise it could be too small.
TED SELKER There are lots and lots of things that are
coming along that are going to make it very easy, almost
for nothing, recognize where you are, what you're looking
at, how you're looking at it, without even having to
wear anything, without you're having to be in any particular
place.
JENNIFER HEALY Clench your jaw for a second for me
so I can find the muscle.
ALAN ALDA (NARRATION) Six years ago when I was here
at the Media Lab, they were making a start not only
on letting computers know what you are doing but even
how you are feeling.
JENNIFER HEALY Go ahead and clench your jaw. See, so
that's how I know it's on a good place. Yeah, great.
Good job.
ALAN ALDA Wow, look at that.
ALAN ALDA (NARRATION) As well as the tension in my
jaw, Jennifer Healy measured the clamminess of my hands.
JENNIFER HEALY Yes, if you've ever noticed when you're
nervous, your palms get all sweaty, and that sweat helps
the electrical conductivity.
ALAN ALDA I see.
ALAN ALDA (NARRATION) Another sensor picked up my heart
rate. The idea of all this was to let the computer know
my emotional state - which unknown to me was about to
be aroused.
JENNIFER HEALY Look at that, isn't that cool?
ROZ PICARD He went off the scale there again.
ALAN ALDA (NARRATION) Now as anyone who knows me can
tell you, the secret to manipulating my emotions is
food.
ALAN ALDA All right, all right. I'm just going to think
of a saltine. I bet you a saltine gets me right back...
it went down, see?
ROZ PICARD Now you went down. You just dropped all
the way down.
ALAN ALDA Saltines are not very exciting to me. Now
watch this. Pasta. Oh, red sauce, a little ricotta on
the side. Some hot red peppers.
ROZ PICARD The peppers went up.
ALAN ALDA Did they? I could have told you that.
ROZ PICARD I don't know about the ricotta though.
ALAN ALDA (NARRATION) The point of having my computer
know my emotions is that it could adjust itself to my
mood, running faster when I'm bored, trying another
tactic if I'm frustrated. But back then in 1996, reading
my emotions was a cumbersome and intrusive business.
ALAN ALDA Does this meant that at the next stage of
this, somebody will have to be hooked-up like this in
order for the computer to recognize what they're going
through?
ROZ PICARD No, eventually we hope that these sensors
will disappear into your clothing or into the twiddler
itself, into the devices that you're naturally in physical
contact with. In fact, people are physically in contact
with computers more than with other human beings, frequently,
so there's a lot more opportunity than you realize for
these sensors to be collecting information.
ROZ PICARD Today you're going to see some of the progress
we've made along these lines. One of the things we did,
for example, was to take the blood volume pressure sensor
that before was this big old clunky thing on your finger,
and now you can wear it on your ear. Now I see that
you love to wear earrings, and maybe I can just attach
this to your ear, OK?
ALAN ALDA And this is shining a light through my ear
to see what the blood flow is in there?
ROZ PICARD Yeah, it shines an infrared light, it's
called a photoplethysmyograph and it picks up the amount
of light going through and the amount changes with your
heart rate and with stress information.
ALAN ALDA (NARRATION) And now the clamminess of my
hands can be measured with no wires at all - just a
glove equipped with a light-emitting diode.
ALAN ALDA But it's not lighting up yet.
ROZ PICARD No it's not lighting up. Well, we can debug
this together. First what you can do is try taking some
really quick deep breaths. It may take a second to get
you to glow. I'm seeing a little teeny bit of glimmer
there, very slight.
ALAN ALDA I'm going to pass out from hyperventilation.
ROZ PICARD Don't do that. The other thing you can do
is tell me about something significant that's happened
to you lately.
ALAN ALDA Something significant?
ROZ PICARD Or something that arouses you.
ALAN ALDA Um, I'm trying to think of something I can
tell you on camera.
ROZ PICARD That made him glow, look at that.
ALAN ALDA Look at that!
ROZ PICARD You didn't actually have to say anything.
ALAN ALDA This is the hot pepper phenomenon. Whoa!
ROZ PICARD This signal doesn't tell people what you're
feeling. It just tells what are the dimensions of the
emotion. So emotions such as anger, fear, excitement,
joy, strong interest, those map to a heightened arousal.
So this tells us something about the height of that
arousal level but it doesn't tell us whether it's a
positive or negative response, and that's a key piece
of information that we don't seem to get from the surface
of the skin right now. There we have to look at the
face.
ALAN ALDA Now this takes a picture of my face and it
finds my eyes right away.
ALAN ALDA (NARRATION) Right now the system can only
tell when I nod, shake my head or blink…
ALAN ALDA So how would this be used?
ROZ PICARD Well, obviously you wouldn't be looking
at a picture of your face, you'd be looking at some
other task that you're doing. And it's going to be watching
you as you engage with it. It will be looking at you
to see if you're looking at it or not, because if you're
not even looking at the screen, why should it continue
to present you with new information. It could be watching
to see if you are looking at it intently, if you're
raising your eyebrows, or if you're frowning by pushing
them together, lowering like you're concentrating. These
kinds of changes in the muscles around the eyes are
part of the information we want to use to try to determine
if you're interested, if you're on the task or not.
ALAN ALDA (NARRATION) Roz Picard's lab has also developed
a mouse that can tell how hard I'm gripping it - a measure,
perhaps, of how annoyed I am with my computer. These
images may also give hints at how I'm feeling about
my computer -- or the programs it's running.
ROZ PICARD Right now you're sitting in a special chair.
This chair contains a pad at the bottom and the back
that contains an array of pressure sensitive sensors
and it senses how… the points of pressure and the intensity
of those. So there's a rather unusual portrait of you
on the screen right now…
ALAN ALDA This is what I'm sitting on?
ROZ PICARD That's what you're sitting on.
ALAN ALDA And this is my back?
ROZ PICARD That's your back. So lean back and we should
see more blue lighting up there, you see the weight
shifting to the back of the chair. And as you slump
down the pattern shifts that way. The dynamics of this
pattern we think will give us some clues to things like
are you starting to get fidgety, you know, and antsy
or are you on the edge of your seat, literally.
ALAN ALDA (NARRATION) Sitting on the edge of his seat
is 6-year-old Adam, testing out the sensor chair as
it analyzes his shifting weight and decides on his most
likely posture. Graduate student Selene Modo has had
scores of children sit in the chair while working at
a computer. It's one more way Roz Picard's group is
trying to make our computers sensitive to our mood and
interest…
ADAM Ooh, look, a twenty. Cool, cool.
ALAN ALDA (NARRATION) So that by the time Adam is an
adult, his computer will know when it's losing him -
and try harder not to put him to sleep.
LEONARDO THE LOVABLE
ALAN ALDA (NARRATION) For almost a decade now we've
been following the evolution of an idea that began with
"It."
ALAN ALDA I'm sorry. Could you just say that again.
ALAN ALDA (NARRATION) "It" was the creation of MIT
researcher and famous robot builder Rodney Brooks. It
could react if you got too close with caricatured shock.
ALAN ALDA Heh, heh, it opens its mouth in utter surprise.
RODNEY BROOKS Yeah, and it raises its eyebrows. It's
sort of got this reaction of, "Oh, what's happening
here, get away from me."
ALAN ALDA (NARRATION) "It" was on of the first attempts
to put a human face on a computer with the express purpose
of making it easier for us to relate to it.
RODNEY BROOKS It tries to appear to be human, so that
we can interact with it in a way that's human.
ALAN ALDA (NARRATION) It soon evolved into a robot
head called Kismet, built by a student of Rodney Brooks,
Cynthia Breazeal. We first met Kismet three years ago.
ALAN ALDA These robots, like babies, are going to seem
especially appealing to us. We spend a lot of time with
babies and we can't resist them.
ALAN ALDA (NARRATION) Kismet's expressions -- ranging
from boredom… to happiness… to sadness… to interested
-- were deliberately over-the-top.
CYNTHIA BREAZEAL It's very much a caricature to make
it that much easier for you to read the robot. So that
when it looks happy, it's obviously happy; when it looks
sad, it's obviously sad. You kind of go, "Oh, I did
something to upset it, I should do something to make
it happy." So it's really trying to get you involved
at this kind of unconscious emotional level.
ALAN ALDA (NARRATION) Cynthia spent long hours in front
of Kismet, working on giving it the ability to recognize
objects and react with its cartoon-like responses.
CYNTHIA BREAZEAL Ooh, you don't like it when I take
away your toy…
ALAN ALDA (NARRATION) And her willingness to invest
the effort is itself a demonstration of the goal of
the research -- to make people want to help their robots,
so that robots can more easily learn the ways of humans.
CYNTHIA BREAZEAL Ooh, you're so sad. So sad when I
take away your toy…
ALAN ALDA (NARRATION) Robots learning the ways of humans
also happened to be a central theme of a movie then
in production. MOTHER This belonged to Martin, my son.
ALAN ALDA (NARRATION) In Steven Spielberg's A.I. Artificial
Intelligence, the teddy bear is a robot.
TEDDY Ahh.
ALAN ALDA (NARRATION) In the movie, the boy is also
a robot. But while he's played by an actor, the teddy
bear was created at Stan Winston Studio - which also
made the dinosaurs for the Jurassic Park movies.
MOTHER His name is Teddy. Teddy, this is David.
DAVID Hello Teddy
TEDDY Hello David.
ALAN ALDA (NARRATION) Teddy was one of the most lifelike
puppets ever made in Hollywood…
TEDDY I am not a toy.
ALAN ALDA (NARRATION) While Kismet was certainly the
most expressive robot ever built by computer scientists.
So when Cynthia Breazeal was hired as a consultant to
A.I. and met Stan Winston, they hatched a plan - to
combine their skills to create the most lifelike and
expressive robot ever built.
STAN WINSTON It was probably the most exciting thought
concept that had ever come to me in my career. I mean,
we have created the most organic looking and acting
robots ever in the films that we've done. But they don't
have any brains. No offense to the puppeteers. But the
robot itself doesn't have any brains. And this was a
chance to do something that was groundbreaking and historic.
ALAN ALDA Oh my God. Look. What?
ALAN ALDA (NARRATION) Meet Leonardo, making its first
public appearance, exclusively for Scientific American
Frontiers.
LEONARDO La la la.
ALAN ALDA You can make it do that? Or are they making
it do that?
STAN WINSTON They're doing it.
ALAN ALDA Yeah, right. Can you do "woo woo woo woo…"?
ALAN ALDA (NARRATION) Today, Leonardo is still brainless.
ALAN ALDA Wow. Look at that tongue action.
ALAN ALDA (NARRATION) His 61 different moving parts
are controlled by puppeteers. The goal is to replace
them with a rack of computers.
STAN WINSTON This is the birth of the first truly artificial
intelligent character that will have the ability to
emote, that can reach every range of expression. That
can hear you and respond to you as an artificial intelligence.
It's pretty amazing.
CYNTHIA BREAZEAL What we really are exploring here
is social connectedness. Social interaction, the human-robot
connection.
STAN WINSTON What it is right now is a brand new instrument
that has yet to be learned how to play.
ALAN ALDA Right.
STAN WINSTON It's…we've created a grand piano but it's
never been… This particular character has never been
built before. So all of what's happening with him is
a learning process. This is like the first day anybody's
seen it. Puppeteers that are dealing with it right now-this
is a new character. So it's like "what things make it
do this?" So we have to learn how to use this instrument
and then once we have learned as human beings how to
create an expression, how to create a phonetic, how
to make it go "woo", how to make it go "hello", whatever
it is…we put it all together…
ALAN ALDA …And the computer will know how to do it?
STAN WINSTON Exactly.
ALAN ALDA (NARRATION) It's a few weeks later. His skin
and fur back in Hollywood for further primping, Leonardo
sits patiently in the Media Lab, waiting for Cynthia
Breazeal to plug him into his control system. It's the
start of another day of Cynthia's translating the skills
of Stan Winston's puppeteers into computer code. Of
the 61 different moveable parts in Leonardo's body,
twelve are in his arms alone.
CYNTHIA BREAZEAL It gives us a very intuitive physical
sense of, you know, as I'm moving Leonardo's arms, to
what kind of motion the robot can do well to what's
kind of pushing its limits and so forth. So I think
it will give us additional insights as we write the
code to make sure that Leonardo moves in a way that
doesn't damage its motors, it doesn't overheat them
too fast.
ALAN ALDA (NARRATION) But making Leonardo move like
a living creature rather than a robot isn't enough.
Leonardo isn't just a robot. It's a character.
CYNTHIA BREAZEAL Character is expressed through the
entire body. Often characters have their own little
idiosyncrasies, so probably elements of those will be
put into Leonardo. We don't know what they are yet,
that's kind of Stan Winston's domain I think. But anything
regarding, I mean what is the character of Leonardo
is really going to be a joint effort between us as well
as Stan Winston Studios to try to define, what does
this robot sound like when it talks? How does it express?
How does it react to things? What is the character of
Leonardo, how do you encode that? And how do you encode
it in a way that the robot can learn and extend its
repertoire over time?
ALAN ALDA (NARRATION) For Leonardo to know what's going
on around it, cameras in its eyes will be able to recognize
faces and facial expressions, while microphones in its
ears will recognize individual voices and the sounds
they are making. And its body will be covered with touch
sensors.
CYNTHIA BREAZEAL You can imagine people are going to
be petting Leonardo, it's a very lovable little guy,
and it should be able to know that and respond to that,
know when it's being touched in a way that it likes,
that's good for it. And the way when people are touching
it that's not good for it, it should recoil, I don't
like the way you're touching me.
ALAN ALDA (NARRATION) One of the robot's most expressive
features are its ears - which can also perform a few
tricks not often seen in nature.
CYNTHIA BREAZEAL You can do the backstroke, the crawl,
it can wave, hello. You can see all the technology that's
gone into creating a robot that is really optimized
for expression. The ability of these robots to interact
with people in a way that's natural for them, and that
they accept the robot, is going to be really critical.
And I think that always Leonardo is pushing the idea
of not making it look human, but kind of pet-like but
intriguingly anthropomorphic, like a pet but perhaps
even smarter than a pet.
ALAN ALDA (NARRATION) With his fur back on - and even
with only his mouth able to move - Leonardo is certainly
hard to resist.
CYNTHIA BREAZEAL When you were in that little interaction,
notice how much you were adapting to it.
ALAN ALDA Right. Yeah.
CYNTHIA BREAZEAL That's what I'm getting at. That's
the kernel of the learning interaction I'm trying to
capture. And you need something like this to make you
act that way.
ALAN ALDA Yeah.
STAN WINSTON And how much fun are you having….? He
will learn 'cause you're so interested in playing with
him…
CYNTHIA BREAZEAL That's right.
STAN WINSTON That the machine, the computer, the brain
will learn. And because of your interest in his organic
reality, you'll make him more real. Imagine having to
act with a character that doesn't exist, who isn't there-and
pretend it's not there. Or to actually have that fantasy
character sitting there and acting with you. It allows
you a better performance.
ALAN ALDA That's it.
STAN WINSTON Therefore, you as an actor are going to
be more believable on screen, we're going to believe
the story, we're going to believe the fantasy character
and the best performance is going to come out of you.
ALAN ALDA You know, you just made me understand Cynthia's
work better because as an actor I understand how much
more will come out of me if the other actor in the scene
with me is behaving like a real person. Now I understand
through that what you mean about how I'll feel when
I sit down or stand up with my computer and talk to
it and it talks back like a person. And responds to
me with facial expressions like a person, I think I'll
be in a different relationship to the computer or the
robot than I've ever been before.
STAN WINSTON Absolutely.
STAN WINSTON The wonderful thing about this is that
we're sort of seeing its birth. The robot is there.
The brains are now coming in and the learning process
is gonna go year after year. It'll be so interesting
I cannot wait for the deadline, which I kind of-because
I have gray hair-I said "In six months, those brains
have got to be connected." Then in the next six months
we see how much we can do.
ALAN ALDA You're talking about in the robot…?
STAN WINSTON I'm talking about in the robot, yes. 'Cause
mine are gone.
ALAN ALDA (NARRATION) The collaboration between Hollywood
chutzpah and MIT know-how is unique - and if Stan and
Cynthia can pull it off, may indeed result in a computer
- or a computer-controlled creature - you can not only
talk to but care about - and that will at least give
the impression of caring about you.
STAN WINSTON Leonardo will actually be able to see,
be able to hear, and be able to touch and feel. It will
respond to you. It's amazing. And I'm going to take
credit for all of it.
CYNTHIA BREAZEAL Oh no you're not. Oh no you're not.
ALAN ALDA Want to go out for a cup of coffee later?
