Visit Your Local PBS Station PBS Home PBS Home Programs A-Z TV Schedules Watch Video Donate Shop PBS Search PBS
SAF Archives  search ask the scientists in the classroom cool science
scientists from previous shows
cool careers in science
ask the scientists

Photo of Prof. Cassell Body Talk

This story shows how Justine Cassell and other MIT researchers are designing computers that will "come out of the box" to see and hear more like people and to respond to our gestures. Following the premier of the Scientific American Frontiers Special Inventing the Future, Justine answered viewers' questions as part of an Ask the Scientists panel. Here are viewers' questions and her answers. Click Justine's picture or name for a brief bio.

QWith the advent of newer and faster technology and information, do you feel that scientists are responsible for educating the public about the hazards of misapplied technology? I get this sneaky feeling that "big brother" is watching more and more. Is this paranoia or do you think that technology has not been the panacea that everyone thinks it is? Don't get me wrong, I'm an engineer and love high technology. I also think new technologies have solved a lot of problems. But, they have also created a lot of new problems. Where do the scientists fit in to this equation of technology application?

A Technology itself is neither good nor bad. But, like everything else, technology can be put to good ethical uses, or exploitative bad uses. And that depends on the people who have access to the technology, and the way in which society views technology. I believe that scientists are responsible for creating systems and technology that can be used for the good of the people of the world. And I believe that we have a responsibility to demonstrate good and ethical uses for technology. And, if possible, I believe we should try to make technology accessible to people who have good ethical uses in mind.

QMy wife feels that having computers interact with us, say in our home to control normal everyday tasks, such as heat control, food preparation and television, will cause society to become more lazy and hence more out of shape. I feel, it will be great to simply enter a room and issue voice commands to an appliance-controlling computer, such as "I want to watch PBS" and the TV will turn on, with the channel already selected.

A Every time a new technology comes into existence, people get worried about the negative effects that it may have on their lives. For example, when radio was invented, there was public uproar: what if the family unit is destroyed, and the possibility for conversation is replaced by sitting around the radio? The same thing was said about television: what if children never leave their houses any more because they just want to sit in front of the tube?

Well, actually, I believe that for each negative effect of radio, television, and computers, there is also (at least) one positive effect. Radio, television and computers have made people aware of people in other countries, of the diversity of cultures, and the problems around the world that we all need to face.

But when I build computer technology, I try to ensure that there are benefits. It's possible that interactive computer house controls will make us lazy (not that walking to the light switch is a lot of exercise!) but they can also make us smart; we can learn about what computers can do, and we can learn about the workings of our home. To maximize these benefits, I try to design systems where the "guts are hanging out" so that users get to see how the thing works, and that way the technology does a task for them, but also teaches them something.

QHuman gestures seem very complex and, by nature, some people are more demonstrative than others. For instance, some people may show anger or frustration by withdrawing and making few gestures while others gesticulate wildly. I'm curious about how you will allow for these variations in human behavior as you continue your work to program responsiveness to human gestures into a computer.

A It is true that some people gesture more than others, but in fact the commonalities in gesture are more striking than the differences: everybody gestures, and everybody uses the same kinds of gestures. In our work to date we have concentrated on these commonalities, but we do want to be able to "personalize" the responsiveness of our computer systems. For example, one system that we are currently working on tries to match its conversational style to the conversational style of the human -- in the future perhaps the system will match gestural style as well.

QWhat are you going to teach Gandalf to do next? And why did you call him Gandalf?

A Next we want to teach Gandalf to pick up aspects of meaning from people's gestures that they haven't conveyed in their speech. So, if you illustrate something with your hands, but you can't find the right word, Gandalf should be able to say "oh yes, I know, you mean the carburetor" or whatever. Likewise, we'd like Gandalf to pick up aspects of meaning from people's intonation -- the melody of their speech.

We call him Gandalf after a Norse dwarf. Actually, the name of the computer software is Ymir, who was a Norse Giant who was evil, and when he died his body become the fertile ground out of which the rest of the world was built. Gandalf was the first being to grow out of Ymir. That's why we named the first humanoid agent to come out of the Ymir system Gandalf.

QAs someone who works at a computer constantly, I love the idea of a computer that responds to our speech. The physiological advantages of not being physically "tied" to a computer would be enormous. Any ideas about when this type of technology might be available and affordable to the public?

A Hope it will be soon! We need to shrink the number of computers needed to run Gandalf (currently 8!), and we are depending on better speech recognition becoming available.

QYou explain how computers are coming out of the box and are behaving more like people. Are you aiming for a day when we can talk to and treat computers like another person? If so, how long do you think it might take to develop such a computer?

A I hope that it won't take much longer. We are already developing computers that we can treat MORE like humans. The question now is how to make them MORE natural, and MORE like us. We have to decide what areas are the most important to concentrate on.

QWe are wondering how you got interested in doing such future-oriented research. How did you get started? It looks like fun -- do you enjoy it?

A I've had a pretty varied professional career! I studied Comparative Literature as an undergrad, and Linguistics and Psychology as a graduate student. My dissertation looked at how children used gestures in storytelling. Once I had started looking at people's gestures (adults and children), and realizing how important they were to our communication, it was a short jump (well, a medium-sized jump) to wanting to build computers that gestured, and computers that would take our gestures into account. It is fun -- I love my work!

QIn the future do you think that interactive beings like Gandalf will be used as a learning tool for students?

A I think that Gandalf will make a great learning tool because he's a good listener: he has nowhere to go, infinite patience, and he says "uh huh" a lot.


Scientific American Frontiers
Fall 1990 to Spring 2000
Sponsored by GTE Corporation,
now a part of Verizon Communications Inc.