[motivating music] - This is quite exciting to be kind of working these things that we think that you are changing the future.

We start developing and then implementing and see the immediate output resource.

- [Jose] So we set the new destination and we see the same thing, a new route.

And then the system is waiting for a new input.

So we push autonomous mode button.

- [Host] Engage as they say.

- [Jose] Engage.

[car whirs] - I'm trying not to say keep your hands on the steering wheel.

A massive amount of technology as well as faith in that technology rides along with the passengers in an autonomous vehicle.

This is the two mile rural test track for autonomous vehicles at North Carolina Agricultural and Technical State University.

Are you at the point where you're not nervous being in this anymore and you're trusting the technology?

Are you still nervous yet?

- [Jose] This is a project under development.

So after several rounds, you get more comfortable of course, because you know that the system has previously worked, it will work again and every time it works better.

But they still at this point, the driver needs to be alert.

- [Host] You're no doubt wondering how it all works.

[peaceful music] Well first think what's required for a person to drive a car.

- We first use our eyes to understand and the environment.

So we first perceive the environment around the car, then we make decision to whether we should accelerate or whether we should take a break or turn or whatever.

And then we use our hands or foot as a sort of actuator to accelerate or turn.

- [Host] An autonomous vehicle system works essentially the same way.

The vehicle must note where it is in space and what is around it.

That's where an array of sensors comes in.

There are two cameras above the windshield.

- One is looking in front of the car to understand the road, how the road is changing and whether it's turning or whatever.

The other one is looking upward to look at the traffic lights and whether it is red or green or whatever.

- [Host] Radar on the front bumper judges the speed of the vehicle in front of the car.

It also checks for any objects in front of the car.

There are two GPS receivers to tell the vehicle where it is.

Then there's the Lidar system on the roof.

- It's a sort of laser-based sensor that is sending laser signals and then receive them back and that's the way that you can very accurately understand the environment around the car.

- [Host] All of that information is provided to a powerful computer system that establishes the vehicle's location and speed, perceives the area around the car in real time including the locations and speeds of other vehicles and avoids collision with a planning technique.

- [Jose] Well, we are going now at 15 miles per hour.

- Right.

- because that is the speed limit that we set it up.

That map is telling you where it is located a crosswalk, where it is located the stop line.

And then there is a decision making system that is giving the information to other system that is a control system to reduce the speed here, increase it again or maybe if there is an object.

So it should, it says stop.

- I mean this is really a very active constant flow of information.

- Yeah, yeah, yeah, yeah.

It's a lot of things are currently going on to give us this right.

This a very, very complex system.

It's very important, the processing time of the information.

- [Host] All of that information is compared to a map of the area that is pre-programmed into the computer.

A drive by wire system adjusts the vehicle's speeds and turns the wheel.

- Yeah, so right now we are reducing the speed because there is a stop.

It does it like a human driving behavior.

It stops, waits a certain amount of time and then continues.

- You're not gonna do rolling stop signs or rolling stops in this.

Okay.

- Exactly.

- [Host] The result of all that data crunching is projected on a screen inside the car but that's also where challenges are visible.

You can see the vehicle and that's a building next to it but notice all that white in the distance.

That's because the vehicle isn't certain what's around it.

The maps the system is using were made in the summer when the trees were filled with leaves.

- Yeah.

It's like, oh, this kind of looks sort about the same, but not really.

The density of our trees are getting smaller, so the returns from Lidar is less.

And so most of the time they're just going through it and like, oh well this doesn't look exactly how it was a month ago when you made this map.

And so we've been having to battle that problem.

- [Host] That's why the world test track is so important.

- [Ali] Rural transportation has its own challenges.

Lack of GPS, lack of proper communication, low maintenance of the roads, narrowed the roads, sharp turns.

- [Host] The technology is rolling along but it hasn't arrived just yet.

- The major challenge they have to be able to deal with unseen scenarios like human.

There are so many problems that you are dealing and then you never saw it before and then you have to make decision about it.

So those that we know and we plan for it and we train them so they are good.

But then beyond that, that's the major research problem.

How we can generalize and how we can deal with unseen driving scenarios.

- [Host] Like squirrels.

- Yes, like squirrels.