ALDA Hello and welcome to Scientific American Frontiers. I'm Alan
used to the idea of there being computers in our cars, invisibly
monitoring and controlling everything from the engine to the airbags.
In fact, there are over 50 microprocessors tucked away in a typical
the day is fast approaching when you won't be driving a car with
computers so much as a computer with wheels. And while computers
that crash are all too familiar, tonight we'll be looking at computer-based
devices that are designed to prevent crashes of the deadly
going to Germany to drive some cars that watch out for dangers you
might be missing...
take a spin in a car that goes nowhere but that's helping identify
those potentially deadly moments when our attention wanders ...
we'll chat to a car that likes to talk back.
all coming up in tonight's episode, Cars that Think.
ALDA (NARRATION) We're starting with a quick trip into the past,
to a moment I remember vividly even though it was ten years ago.
I was driving a van on a German autobahn or rather, and this was
the point, not driving.
We're now speeding up.
ALDA (NARRATION) The road was the test track for what was then Daimler
Benz, and the van was a prototype of what Daimler researchers hoped
would become a vehicle that could drive itself. In the windshield
were cameras watching the lane markings in the road...
So back here is the control area for the system.
ALDA (NARRATION) While in the back was a huge rack of computers
tracking those markings and steering the van.
With these ten windows here we can actually follow the road, we
know the radius of the road, where we are in the lane and that lets
us drive automatically.
ALDA (NARRATION) A self-driving car would obviously have to do more
than just track the road...
I'll activate the system now by hitting the green button here.
ALDA (NARRATION) It would also have to keep an eye on all the other
vehicles sharing it.
Once we get up to speed I'll have Christof pass us. OK, Christof...
He will enter the road ahead of us here, the lane, we'll detect
him, if he slows down we will slow down, if he speeds up, we'll
ALDA What if there was an accident, if we turn this corner and suddenly
came up on an accident, there was a truck across both lanes?
At the moment we probably would not stop. We have work going on
now ...take control, the yellow...
ALDA Yellow, OK.
We have work going on where we look for more obstacles, not only
the back of vehicles, but also sides...
ALDA So you're really going step by step with this now aren't you,
and as you get, as you solve these problems, so you get on to more
Exactly, we're working our way up in complexity here, and we really
have to be able to find all the obstacles on the road and there's
an awful lot that could be out there, so it will take a while.
ALDA (NARRATION) Jumping forward ten years until today and the company
now DaimlerChrysler is still working on trying to identify all
those things out there on the road, from errant balls...
today, the goal of the research program has shifted radically. No
longer do company engineers dream of cars that drive themselves:
just imagine the law suits if you could blame your car instead of
yourself for an accident. Now, with the knowledge that some 90 per
cent of accidents could be avoided if the driver had been warned
in time of danger, DaimlerChrysler has turned its technology to
spotting things the driver shouldn't miss like stop signs.
ALDA Do you think the car will recognize that stop sign up there?
BREUEL Yeah, yeah, I'm sure, I'm sure.
ALDA Yeah, well, let's go for it.
ALDA (NARRATION) Driving the car on an empty airfield near Stuttgart
in Germany is DaimlerChrysler engineer Gabi Breuel.
BREUEL Look here, see.
BREUEL Now it's got it. The system itself brakes for a short time...
ALDA And you didn't...
BREUEL I didn't...
ALDA You kept your foot on the accelerator, right?
ALDA Well, we were sort of well into the intersection by the time
the car stopped.
BREUEL No, it's just a tactile warning for the driver that she has
ALDA I see. So in other words, if you feel the car stopping by itself,
it's a good idea to take your foot off the accelerator and put your
foot on the brake.
BREUEL That's right, that's right.
ALDA How does it see the stop sign?
BREUEL How does it see? You have stereo cameras because they allow
us to determine the distance and the size of objects in front.
ALDA How does the car recognize that what it sees is a stop sign?
BREUEL The shape. And after it's got the shape of the sign, then
the system is searching for the word stop.
ALDA What if it didn't have the word stop on there?
BREUEL Nothing happens.
ALDA If you turn the stop sign around?
BREUEL Nothing will happen.
ALDA Nothing will happen. We'll just go straight through.
BREUEL Yeah. It's not a gimmick!
ALDA It could see the shape. But it couldn't, it didn't see the
word. It's got to get both the word and the shape?
BREUEL Both are necessary for...
ALDA So if you're coming up to a billboard that said "stop smoking,"
it wouldn't stop the car?
ALDA (NARRATION) In the first demonstration, the car had warned
Gabi she'd missed the sign by braking briefly. But the system could
also be set to do the stopping all by itself.
ALDA OK. So at a certain distance from the stop sign if you're going
BREUEL Too fast...
ALDA And it says, if this person hasn't stopped yet, they're not
going to stop, that's when the car automatically stops.
ALDA (NARRATION) But again, DaimlerChrysler doesn't want to be sued
if your car and not you could be blamed for missing a stop sign.
So cars you may one day be able to buy will alert you to your errors,
not try to correct them.
BREUEL This will not be a system for the street, because our philosophy
is that the ultimate responsibility for operating the car remains
with the driver.
ALDA (NARRATION) Stop signs aren't the only things DaimlerChrysler's
engineers don't want you to miss. Aided by the sort of digital map
you can download into your car's navigation system today, this car
knows where traffic lights are located, and keeps its eyes out for
You want to know the status of the traffic light at all times, because
you don' t want to run a red light, it's expensive and dangerous.
So we're able actually to issue a warning to a driver when he's
about to run a red light.
ALDA So these cameras are turning now to see the light. Now that,
it picks up a picture of the light from some distance, so they point
straight ahead, and then it just keeps tracking the lights.
Very similar to a human head basically. With a head you look to
the right. When there's some stuff coming out, you look to the left,
then you look up. It makes sure you actually have green when you're
going through that intersection.
ALDA Now what about if you're turning right? Do these cameras
can they look over there and see what you're turning into?
You can use prior knowledge from a digital map again, and you know
there's an intersection coming up, the guys from the right side
have the right of way so you can turn your camera ahead of time,
because that's where you're expecting the danger from.
ALDA Now this is kind of... these cameras are kind of big and clumsy.
I mean. It's kind of hard to see over them. What will you do when
you're in a real car?
In a real car, we would actually take a much smaller active camera
and you would put it up here right next to the rear view mirror
and you would be able to hide it.
ALDA (NARRATION) So, ten years after I cavalierly took my hands
off the wheel, a car that will drive itself remains an engineer's
dream, unlikely to become reality. But at least one of the technologies
being pioneered back then has met the road the lane tracking assistant,
already installed on many trucks in Germany to warn drivers that
their rigs are straying, and now becoming available on some luxury
car models in the United States. What makes drivers stray out of
their lanes and what a thinking car might do about it is the
subject of our next story.
GREENBERG All right, come on in.
ALDA (NARRATION) I'm about to take a drive on a busy highway a
highway that's notorious, I'm told, for drivers straying out of
GREENBERG Come on around. And we have here our model T flat black
Taurus, and you can go ahead and have a seat. It's set up as close
to a normal Taurus as we can make it, so it has normal seat controls
for your seat and adjustable pedals right here. So get yourself
settled, buckle your seat belt and close your door when you're ready.
ALDA Shall I put on my glasses?
GREENBERG And put on your glasses, just as you would normally drive.
ALDA This feels OK.
ALDA (NARRATION) My passenger on the drive is Jeff Greenberg, a
researcher for the Ford Motor Company, who'll be passing judgment
on my driving skills.
GREENBERG OK, now the next thing we're going to have you do is we're
going to have you put on the eye-tracker. This is basically a baseball
cap; it's got just a Velcro adjustment here for your head. Slip
that on, it should slip on right over your glasses. And what we'd
like you to do is try to adjust it so that it's comfortable and
that your left eye is looking directly through that tilted piece
ALDA (NARRATION) The eye-tracking device will allow Jeff to see
where I'm looking as I drive along not on an actual highway,
of course, but on a virtual highway inside Ford's Virtexx driving
simulator. Cameras monitor my every move as my virtual car speeds
on its virtual way inside an enormous dome on stilts.
GREENBERG Just do a normal lane change, just a normal gentle lane
change into your left lane.
ALDA (NARRATION) As I steer to the left I feel the car move to the
left because while the road is virtual, the movement isn't.
GREENBERG And then when you're ready just go ahead and do a lane
change back to the right.
ALDA (NARRATION) Added to the realistic noise and vibration, the
movement quickly convinces my brain that I'm driving for real. Which
is just what Jeff wants me to feel.
GREENBERG Go ahead and look in your rear view mirror for a moment
and see if you can see what that car does.
ALDA Yeah, it's erratic.
GREENBERG That's right. Well, when you see him stray out of his
lane to the right, flash your turn signal to the right, and if you
see him go off to the left, flash your turn signal to the left.
ALDA I would speed up to get out of this guy's way...
GREENBERG Yes, you would. We're going to ask you to, you know, bear
with him and stay with him because there's more that's going to
happen, OK? Look at the Explorer in front of you and you'll see
that there's an erratic driver in front of the Explorer.
ALDA Right, you want me to...
GREENBERG Yeah, same thing. When you're not doing anything else,
you're just driving like this, you get used to it, when we measure
adults we find that 97% of the time they identify these events,
they're just driving. Three per cent of the time they don't see
them. These numbers change if they become involved in doing other
ALDA (NARRATION) No prize for guessing which other thing is the
most likely to make the numbers change...
GREENBERG Go ahead and pick up your phone.
ALDA And make a call?
ALDA (NARRATION) I make the call while doing my best not to misdial
while at the same time keeping an eye on what's happening in front
... as well as behind.
GREENBERG That kind of task takes a lot of visual attention.
ALDA OK, it sure does.
GREENBERG Now, you did see quite a few of the events that were in
front of you that were happening. But there were also things that
were happening behind you that were very, very difficult to track.
When we do that, for adults for example, instead of missing 3% of
the things that happen directly in front of them, they start to
miss about 12 to 13% of the things in front of them, about a factor
of four increase.
ALDA (NARRATION) It sounds bad enough that adults missed four times
as many potentially dangerous events when dialing a cell phone.
But the real shocker came when the Ford researchers tested teenage
GREENBERG Some people have speculated that because of video games
and familiarity with computers and technology, that young people
are just better at multitasking than adults are.
ALDA Is that what you found?
GREENBERG Well, in fact no. What we found was that when we did that
same task that you just did, where on average the miss rate for
adults rises from 3% to 13%, for teenagers it rose to 53%.
ALDA Fifty-three per cent?
GREENBERG Half the time that that happened they didn't see it. It
comes down, as you might expect, to experience. If you look at the
way that adults will do the task, they do it much as you did
they take the phone, and they brought it to the place where they
minimized the amount of time they had to look away from it. They
also break the task up into short bits, no more than a second long,
so they're not looking away from the road. The teenagers dialed
the phone much more quickly, sometimes almost twice as fast as this
group of adults, but they tend to do it without looking back at
the road, and they do it because they seem to have an implicit assumption
that the cars around them are going to behave themselves while they
are off doing something else.
ALDA Nothing will happen, don't worry. That's the phrase that guides
GREENBERG That's right.
ALDA Whoa, the guy behind me's getting crazy.
GREENBERG Yeah, he is. You know, if you've ever been in traffic
and you've had something like that happen in front of you, and I
don't know if you've ever been in a crash, you get the feeling that,
I don't think I'm going to be able to stop the car, and you get
that kind of knot in your stomach. You don't have to have that experience
very many times before you change the way that you behave in vehicles.
And new drivers haven't had that experience, and they are very,
very trusting of other traffic.
ALDA (NARRATION) American drivers spend something like a billion
minutes a year talking on their cell phones that's about
40% of all cell phone use. Research like Jeff Greenberg's has led
several states to require drivers to use headsets or other hands-free
devices in an attempt to minimize the driver distraction cell phones
cause. But the Ford experiments, as well as several other studies,
suggest just taking the phone out of the driver's hand doesn't make
the problem go away.
GREENBERG When people used the hands-free systems, they were generally
better in being able to pay attention to what was happening, but
there was an important exception. And that was incoming calls. When
a call comes in, whether it comes in on a hand-free phone or a hand-held
phone, in our experiments, the drivers were just not able to answer
the phone, find out who was calling them and why and still pay attention
to the things that were happening around them. ALAN ALDA Just before
the call started, or as the call was starting, they didn't see the
car veering out of the lane?
GREENBERG The place where they seemed to be the most vulnerable
was actually the point where they were conversing with the person
to find out who it was that called. So the first thing you might
say, and the best thing that you might say to drivers right now
is, turn your cell phone off while you're in the car. But in the
future, it might be that cars are smart enough to know when the
situation that you're driving in is demanding. We may know, for
example, from sensors that are in the car, when you're in heavy
traffic. We may know when you're approaching an intersection, or
in an intersection, or when you're in a curve. And if a call comes
in during a time like that, we may decide, this is really a time
when it's most appropriate to send that call to voice mail and alert
the driver later.
ALDA (NARRATION) The Ford researchers have also used their Virtexx
simulator to see how tiredness affects drivers. Volunteers were
deprived of sleep and stimulants, then asked to drive on the virtual
highway at night for three long, boring hours. Meanwhile, the eye-tracker
watched out for droopy eyelids.
GREENBERG Many of the drivers experienced long micro-sleeps. They
averaged about two and a half seconds, which is a long way: at 70
miles and hour on the highway, that's close to the length of a football
field with your eyes closed.
ALDA (NARRATION) If the driver strayed out of his lane, the simulator
was rigged to give him a warning the sound of a rumble strip...
or a horn and flashing lights... or a vibrating steering wheel.
GREENBERG We found a lot of feedback from our test participants
that if they're not drowsy, they don't want a warning, it's not
helpful to them, it's really not going to provide them with any
benefit at all. On the other hand, when they were drowsy and it
was really helping them, they were enormously grateful it was there.
So the key for us is to develop technology that's smart, that adapts
to the driver, that provides you the information when you need it
and doesn't bother you when you don't.
ALDA (NARRATION) This question of how best to keep a tired driver
alert is one of the biggest challenges facing designers of cars
that think as we'll see in our next story. Meanwhile...
ALDA I'm glad to be back on earth.
ALDA (NARRATION) We're about to take a drive with Mahesh Viswanathan
and his frequent traveling companion, Sally. This being an IBM parking
lot, it should come as no surprise that Sally isn't a person but
a computer or to use a favorite engineer's term a
VISWANATHAN Today, most of the systems that you have in the car
are what we call command and control. You give an instruction, it
performs an action. What we are trying to do is to build the next
generation, the next to next generation, where the interaction is
more pleasant and the system is much more helpful than it is today.
ALDA So when I talk to the car now, the car will know what I mean,
what I want.
VISWANATHAN When you talk to the car, it will not only know what
you want, it will also try to anticipate your needs. Good morning
How are you today, Mahesh?
VISWANATHAN I'm fine... She identified me as I said "Good morning
Sally" and we use that not only to greet me, which is nice, by name.
In addition to that, she sets all my personal profiles, so she knows
the address book that I have of interest, she knows my favorite
restaurants, she knows my favorite routes.
VISWANATHAN I need to go to JFK for a 3:30 flight.
Would you like me to find the quickest route to the airport?
VISWANATHAN Yes please
It should take about 90 minutes to get there, allowing you 30 minutes
ALDA You don't have to speak politely to Sally. You don't have to
VISWANATHAN No I don't.
ALDA I think I'd feed uncomfortable begging my car.
ALDA (NARRATION) Sally is actually listening in all the time through
a microphone above Mahesh's head, but she responds only to words
she's been programmed to recognize... like traffic.
VISWANATHAN Sally, is there any traffic on the route?
Currently all looks clear, but I will continue to update you. Turn
right onto Pines Bridge Rd.
ALDA What about if you were getting drowsy now?
ALDA (NARRATION) Sally's not only listening; the idea is that eventually
the system will be watching, too. A camera will look for signs of
drowsiness, like drooping eyelids. Then, like a real passenger,
she'll try to wake you up.
Would you like to play a game to fill the time?
VISWANATHAN What game?
How about "Name that tune?" Here's the first one.
ALDA You put these songs in, so you know what they are, right?
VISWANATHAN That's how I know what they are, I've lived in NY a
long time, what do I know about country music? I don't even know
if we have a country music station here. My guess is "Out of your
VISWANATHAN You notice that slight massage where she says "good
job?" To keep me occupied in the conversation.
ALDA Yes, to help keep you alert.
VISWANATHAN Instead of just playing the songs back to back and have
me answered, where I could get bored, it's nice to give a little
feedback to the user, a pat on the back, and I'm there.
ALDA When you think of what games you want to put in there, do you
go to the psychologist and say what do you think we should do, or
do you come up with ideas yourselves?
VISWANATHAN At this point, we just come up with ideas ourselves,
because what we want to show is the power of interaction in keeping
you alert. SALLY Now try this one...
ALDA (NARRATION) Sally will eventually be wirelessly plugged in
to real time information about weather, traffic... and your schedule.
Sorry to interrupt, but I just heard, that your flight has been
delayed by one hour.
VISWANATHAN Maybe I can stop for a bit to eat.
I know you prefer German restaurants, but there aren't any within
3 miles. How about Italian?
ALDA (NARRATION) Sally is currently only a demonstration model:
Mahesh's questions and remarks are triggering prepared responses.
VISWANATHAN Let's make it Italian.
ALDA (NARRATION) That is, if she hears them.
VISWANATHAN Italian sounds good.
ALDA (NARRATION) Computer speech recognition remains a thorny problem
especially in the noisy environment of a car.
ALDA Voliamo mangiare pasta.
ALDA (NARRATION) Which is why something called the McGurk Effect
is so fascinating to IBM researcher Stephen Chu.
Ba, ba, ba
CHU What do you hear?
ALDA I hear da, da, clear as a bell. And that's not what you're
saying, is it? S
CHU Right, so try to close your eyes and listen to the same thing
Ba, ba, ba
ALDA I hear ba, ba really clearly, just as clearly as I heard da,
da. It's amazing, it's the same recording.
CHU It's the same thing.
ALDA And what are you doing with your lips?
CHU Ga, ga, actually. So visually I'm ga, ga, but acoustically it's
ba, ba, but your mind is playing a trick on you, so you're hearing
neither ga, ga, nor ba, ba, but rather da, da.
ALDA This is wonderful, this is so great.
ALDA (NARRATION) Try it for yourself. Listen while you watch...
Ba, ba, ba
ALDA (NARRATION) Now listen again as we cover Stephen's mouth...
Ba, ba, ba
ALDA (NARRATION) The McGurk Effect beautifully demonstrates how
much we rely on what we see as well as what we hear in interpreting
speech. ALAN ALDA And McGurk figured this out? Who's McGurk? Is
his name really McGurk or is his name McDurk? 1178889...
ALDA (NARRATION) IBM is now developing a speech recognition system
that reads lips as well as listens in to what we say. When I tested
it, accuracy improved from about 80 per cent with audio alone, to
close to 100 percent. So the plan is that the camera in the car
that watches for droopy eyelids will also watch your mouth, making
Sally a better listener. Back in her car, Sally has one more trick
up her virtual sleeve.
You just received an urgent email. Shall I read it to you now?
VISWANATHAN Let me hear it
VOICE Mahesh, this is Peter. I know your flight is delayed. Let's
meet in the terminal lounge. See you then.
VISWANATHAN Sally, take an email for Peter.
VISWANATHAN Peter, I received your message. I'll see you at 3. End
Your email reads: Peter, I received your message. I'll see you at
3. Shall I send it?
VISWANATHAN Send it.
ALDA OK, right now I'll give you 50 dollars for this, you don't
have to make it any better. I'll take it.
VISWANATHAN 50 dollars?