Rise of the Robots

How will human-like robots change our future and how we see ourselves? Airing December 6, 2017 at 9 pm on PBS Aired December 6, 2017 on PBS

  • Originally aired 02.24.16

Program Description

Machines are everywhere. They run our factory assembly lines and make our coffee. But humanoid robots—machines with human-like capabilities—have long been the stuff of science fiction. Until now. Fueled by an ambitious DARPA challenge, the race is on to design a robot that can replace humans in disaster relief situations. Follow the robots and the engineers that program them as they strive to make their way out of the lab and into the real world. But how capable are they, really? How close are we to a future where humanoid robots are part of our everyday lives? And what will the future look like with robots that can do a human’s job? NOVA investigates the cutting-edge technologies that are advancing robotics—and the enormous challenges that robots still face.


The Rise of the Robots

PBS Airdate: February 24, 2016

NARRATOR: It's one of the most ambitious challenges in the history of technology:…

GILL PRATT (Defense Advanced Research Projects Agency [DARPA]): Ladies and gentlemen, start your robots.

NARRATOR: …the race to build robots that can save lives in a disaster, that can even rescue us. But to do that, they will need to climb ladders, walk over rubble, turn valves, and no one knows if it's even possible.

RUSS TEDRAKE (Roboticist, Massachusetts Institute of Technology): There's so many things that could go wrong. It's scary.

VIJAY KUMAR (Roboticist, Penn Engineering): When you try to instill human-like intelligence, human-like manipulation skills into machines. It's just very, very hard to do.

NARRATOR: What will it take for robots to have the right stuff? And are we ready for the consequences?

SHERRY TURKLE (Roboethicist, Massachusetts Institute of Technology): It's too easy to look at them and say, "Oh, they're not there yet. They will get to something very powerful, and then you have to say, "Well, where will we have gotten to?"

AYANNA HOWARD (Roboticist, Georgia Institute of Technology): They might actually have the capacity to be smarter than us.

NARRATOR: What impact will robots have on our future? On how we work, how we interact, even how we see ourselves? Rise of the Robots, right now, on NOVA.

For decades, roboticists have wowed us with their creations,…

iCub (Humanoid robot): Do you have any questions?

NARRATOR: …especially in Japan, where bots are starting to cook for us, do the laundry, even sing and dance. Some look so human it's hard to tell us apart. But looks can be deceiving.

DENNIS HONG (Roboticist, UCLA): People see these humanoid robots developed in Japan, all these fancy things, and they had the expectation that, "Oh, we have these robots, and they're going to, you know, save the world." But that didn't happen.

NARRATOR: March 2011: a devastating earthquake and tsunami rocked Japan. Explosions and fires at the Fukushima Daiichi nuclear power plant left the area dangerously radioactive. Going inside, even for a few minutes, was life-threatening. It was the perfect time for Japan's cutting-edge robots to come to the rescue. But none did.

PAUL OH (University of Nevada, Las Vegas): The real tragedy was if, simply, some valves could have been turned or some switches could have been flicked or hoses attached, a lot of the meltdown could have been prevented. The question was, "When robots were needed the most, just how come they weren't effective?" And so I think that led to a lot of self, you know, reflection.

NARRATOR: After decades of research where were our robot heroes? For generations, science fiction has portrayed robots as our loyal servants.

MARVIN MILLER (As Robbie the Robot in Forbidden Planet/Film Clip): Welcome to Altair IV gentlemen. I am to transport you to the residence.

NARRATOR: But it turns out, our imagination has taken us much further than our technology. It's hard to create robots that can function in our world.

TONY STENTZ (Carnegie Mellon University): One of the challenges with deploying robots in the real world is that the real world is like the Wild, Wild West. It's very rich and complex and very unforgiving.

NARRATOR: What will it take for robots to make their way out of the lab and into the real world? We're about to find out,…

GILL PRATT: Ladies and gentlemen, start your robots!

NARRATOR: …at the most ambitious robotics competition in history.

GILL PRATT: We welcome you to the DARPA robotics challenge. This is where imagination meets innovation.

RUSS TEDRAKE: You have completed the course! Wooooh!

NARRATOR: In 2013, at a racetrack on the outskirts of Miami, DARPA, the research arm of the U.S. Department of Defense, challenged 16 teams from around the world to build rescue robots that can help save lives in a disaster.

DENNIS HONG: Very exciting, very exhausted, very nervous and frustrated, at the same time. Dealing with robots is always like that.

NARRATOR: It is the kind of challenge DARPA is known for, cutting-edge and high-risk.

DARPA was formed back in the 1950s in response to the launch of Sputnik, the world's first artificial satellite, built by the Soviet Union. Since then, the agency has spent billions developing military technology, advanced weaponry like stealth technology, drones and night vision. But the spin-offs from this research have gone far beyond the military. The internet, G.P.S., bionic arms, even Siri, were all fueled with DARPA funding.

SAMUEL L. JACKSON (Actor in Television Commercial for Apple/Film Clip): Cancel golf today.

SIRI (Voice of Apple Operating System in Television Commercial for Apple/Film Clip): It's off your calendar.

SAMUEL L. JACKSON (Film Clip): Good.

NARRATOR: If these researchers succeed in creating rescue robots, the same technology could be used to develop robots that take care of the elderly, babysit our kids, clean up after us, robots in almost every facet of life.

But first, can robots really help out in a disaster?

To find out, DARPA has set up an obstacle course for bots. With human operators controlling their every move, the robots must perform basic tasks, like opening a door.

STUDENT ROBOTICIST #1: All right go, go, go!

NARRATOR: Turning a valve, drilling a hole in a wall, walking over rubble, even driving a car. Bot after bot takes their first steps into the real world, and there's nothing easy about it.

NICOLAUS RADFORD (Former NASA Johnson Space Center Valkyrie Team Leader): It's a monumental challenge. I mean, DARPA calls it "DARPA-hard." I mean, I almost said it was "DARPA-impossible."

NARRATOR: Robots fall off ladders. Some barely move an inch. They even struggle to open doors.

JERRY PRATT (Institute for Human & Machine Cognition): The things that a human can do instinctively, easily, you don't realize how hard something like walking is until you try to reproduce it in a machine.

PAUL OH: When my baby daughter took her first step, did she walk? No, she took a step, and she fell down. Over time, she started to walk, right? And it's kind of like what we're seeing here, there are a lot of baby steps.

NARRATOR: After two days of competition, the results are not impressive. But this is just the start of DARPA's grand experiment. The agency is giving the roboticists another chance to get it right, and they're putting their money where their mouth is. The top-scoring teams in this first challenge are receiving a million dollars each to continue their work. In 18 months they will return for the final challenge and the chance to win a grand prize of $2,000,000.

GILL PRATT: It's not clear to me how well the teams are going to do. It can't be too hard, because then everybody fails. It can't be too easy, because then it's not worth doing at all. But if you make it just right, like Goldilocks…right? I have tried to hit the sweet spot of difficulty, but I think risking failure is the DARPA way.

NARRATOR: Will the challenge push the technology a giant step forward? Will robots ever make their way through our world the way we do?


RODNEY BROOKS (Rethink Robotics): Back in the early '50s, Alan Turing, one of the founders of artificial intelligence, said that the best thing we could do was build a robot with TV cameras for its eyes and rotors to drive its legs and have it romp around the countryside and learn from the real world. But he decided that was technologically too hard back in the '50s, which it certainly was. So he said, let's leave that physical interaction until later, and let's work on more abstract problems, the intelligence abstract problems.

NARRATOR: The field of artificial intelligence, or A.I., has already built machines that beat us at chess, trade stocks with lightning speed, and search for anything we want, in an instant. Yet, when it comes to robotics, progress has been painfully slow.

Some of the biggest problems robots face are things we humans usually take for granted, like mobility, manual dexterity and the ability to see and understand our environment. These are the challenges the robotics teams will tackle in the DARPA competition, beginning with mobility.

What's the best way to make a machine that can move through our world? Does it need to walk on two feet, like us? Some roboticists think the answer is yes.

DENNIS HONG: The shape of the robot is dictated by what it needs to do. There's a reason why the step size in your home is this big. There's a reason why your door handle's this high, because it's designed for the humans to move around. So unless the robot is the shape and size of a human, it won't be able to navigate and move around in the environment designed for humans.

NARRATOR: But getting around on two feet isn't easy, even for us. It takes your average infant almost a year to go from crawling to toddling to walking. What does it take to give a robot this intrinsic human ability?

It's one of the biggest problems facing roboticists today. In Pensacola, Florida, at the Institute for Human & Machine Cognition, or I.H.M.C., one of the teams competing in the DARPA robotics challenge is hard at work, developing its own software to run this massive bot, named Atlas, a 385-pound powerhouse.

DARPA funded the design and development of this rescue robot.

Several teams competing in the challenge are using this hardware but writing unique software to guide their robot.

You might wonder why Atlas is so top-heavy. Most of its oversized head, is packed with cameras and sensors. And while its feet may look small, they're designed to fit in the kind of places we walk.

JERRY PRATT: With bipedal walking robots, there's still a lot of strategies we have to determine, and there's no known textbook solution yet. It's more art than science, still.

NARRATOR: For team leader Jerry Pratt, finding the best way for a bipedal robot to walk has not been easy or quick.

JERRY PRATT: Well, I've been working on bipedal walking robots since 1994, so, 21 years now.

NARRATOR: He started back in graduate school at the M.I.T. Leg Lab, where some of the most bizarre-looking bots hopped, jumped and flipped.

But the robots Jerry built were different, because they walked on two feet. Using nature as his guide, Jerry gave his bipeds hips, knees and ankles that mimicked how animals move.

This one, called "spring flamingo," had specially designed motors that worked a lot like muscles, varying the amount of power each joint used. Its legs and feet worked a bit like shock absorbers. They had a little give.

JERRY PRATT: That robot was a really good workhorse. We had it working for about three years, probably walked about 20 miles or so.

NARRATOR: When it came to designing a humanoid, he focused on developing the perfect combination of hardware and software that would enable his biped to stay upright as it calculates the best way to shift its center of mass, lift its leg, swing it forward, and put its foot back down in the right place and with just the right amount of pressure. To make these elaborate calculations, the robot needs instructions, lines of code that tell it how to move.

These lines of code are written in a programming language that resembles English, but for Atlas to use them, its onboard computer has a program that translates them into a language a machine can understand: zeros and ones.

It takes more than 2,000,000 lines of code to run Atlas; 500,000 just to put one foot in front of the other. It's the interplay of hardware and software that keeps this bot on its feet, an ability that took hundreds of thousands of years of human evolution to perfect.

JERRY PRATT: We'll often look at, "what strategies does a human use in order to walk?" Because the physics in the world that a human operates in is the same as the robot.

NARRATOR: But we wouldn't be able to take a single step without having some pretty amazing senses. When you walk, your eyes detect the position of your body relative to the world around it. At the very same time a series of fluid-filled canals in the inner ear, tells your brain the position and motion of your head, so you know which way is up. And a kind of sixth sense, called "proprioception" uses your muscles and nerves to detect where your arms and legs are, in relation to each other. All this information comes together in a part of the brain called the cerebellum.

What kind of senses does Atlas use to walk on two feet? For eyes, the bot has a stereo camera and a couple of fisheye lenses sticking out the side of its head, along with a spinning laser, called LIDAR, that scans everything in the world around it and creates a 3D model of its environment.

Atlas doesn't have an inner ear to tell it which way is up. Instead, it uses a small cylinder in its butt that contains gyros and accelerometers that tell it where it is and how it's moving. In addition, sensors on each joint tell Atlas where its limbs are in relation to each other. The result is an extraordinary sense of balance, allowing Atlas to stand on one foot, like a ballerina, at least in the lab.

But in the real world, Atlas's gyros and sensors aren't always enough.

JERRY PRATT: We only have a limited number of sensors on the robot, whereas with a human or an animal you have thousands of little force sensors on every square inch of your body.

NARRATOR: Over 100,000, just on the soles of your feet.

JERRY PRATT: You can step on something and detect what it is. Whereas, the robots, they don't know that. They just know they stepped on something.

NARRATOR: At the final challenge, if Atlas should step on something the wrong way, if the hardware and software don't work perfectly, it will fall.

JERRY PRATT: We cannot get up from a fall, currently, and we probably won't survive a fall. If we fall during the finals, that may be the end of our whole weekend.

NARRATOR: Atlas isn't the only biped that has trouble staying upright. Meet Hubo, designed and programmed by cousins Paul Oh and Junho Oh.

At the first challenge, Hubo clearly wasn't ready for the big, bad world.

PAUL OH: It's with fondness, it's also with some pain that we think about it. We went into the challenge with quite a lot of enthusiasm. But when we got there, it was just like one thing after another. There were a lot of these real world instances that we did not experience in the lab. That was a real awakening.

NARRATOR: Across the ocean, at the Korea Advanced Institute of Science and Technology, Junho Oh is also doing some soul-searching.

JUNHO OH (Roboticist, Korea Advanced Institute of Science and Technology): Our expectation was very high, but result was very catastrophic, so gap was too big. It took at least a couple of months for the recovery that I get back to my normal mental stage.

NARRATOR: After the shock wore off, the cousins agreed to make a radical change in the design of their biped.

PAUL OH: My cousin has come up with this idea of adding wheels to Hubo's knees, as well as casters on its toes. And I call it the "kneeling prayer" mode.

JUNHO OH: Okay, so we are so happy. So we decide short distance movement just walk okay. It's a risk, but it will be fine. And far distance, we move wheel. It's much more stable.

PAUL OH: Humans don't have wheels on the knees, but there's no reason why we can't add that to a robot.

NARRATOR: When the cousins return for the final challenge, they'll debut their humanoid, transformed with the help of some clever engineering.

But for a robot to help out in a disaster, getting there is just the first of many challenges. To assist rescue workers in the real world, it needs hands with the kind of strength and dexterity it takes to lift heavy hoses, drill into walls, solder and saw. It needs a pair of these.

OLIVER BROCH (Roboticist, University of Berlin): Our experience of the human hand is that we can do everything with it. We can dig ourselves through rubble, we can make really nice paintings, we can knit. The human hand really distinguishes our species from all other species.

SID SRINIVASA (Roboticist, Carnegie Mellon University): If you look at the way humans manipulate objects, we have this instinct for tactile sensing, for feeling the world. Understandably so. We've had over 200,000 years of human evolution to get to this particular point.

NARRATOR: Is it possible to translate a masterpiece of evolution into motors, cables, sensors and thousands of lines of code?

On the outskirts of London, behind an unassuming storefront, a small group of robot enthusiasts are building a robotic hand that they hope could one day rival our own.

ARMANDO DE LA ROSA (Engineer, Shadow Robotics): When we first started building a hand, we actually bought anatomy books and tried to understand how the hand works. We thought it would be good if we could copy, as best we can, the human hand.

RICHARD WALKER (Engineer, Shadow Robotics): The Shadow hand was built with the idea of trying to get as close as possible to the human hand but as engineers.

NARRATOR: Like the human hand, this robotic version has four fingers and a thumb. It's about the same size as the original and can even shake like one, but that's where the similarities end.

RICH WALKER: We pack a huge amount of sensing and actuation. We have 25 joint position sensors, nine analog digital converters; we have two tendons coming from each joint to a motor in the forearm; 20 motors in the forearm. Each motor has a temperature sensor, a current sensor and two full sensors, so we can tell how hard the motor is working and how hard it's driving the tendons. We put contact sensing in the fingertips, so we can tell that we touched something. So, there is a wealth of computing power just to get something that has the same kind of movements as your hand.

NARRATOR: And with this level of dexterity, it can do a lot more than card tricks.

ARMANDO DE LA ROSA: We have this hand handling pipettes and lab equipment, removing the human from the risk, for example, people that work with very, very nasty bacteria and viruses.

NARRATOR: The Shadow hand was designed for delicate tasks, not for dirty jobs like this.

RICH WALKER: When you look at where robot hands get used, you find people who want to do something delicate and precise and people who want something big and rugged and solid.

NARRATOR: For the robots in the DARPA challenge, strength and precision are a must. Finding the balance between them has been a struggle for team leader Brett Kennedy, who developed this four-legged rescue robot, named Robosimian, here at NASA's Jet Propulsion Laboratory.

BRETT KENNEDY (NASA Jet Propulsion Laboratory): My first intuition is that this needed to be a very robust hand. And I was thinking, "Well, robust means big. Why don't we find out how big Wilt Chamberlain's hand is, and make a hand that big?"

So, if anybody's curious, Wilt Chamberlain's hand is very, very big, and not only is it very, very big, when you make a robotic version of it, it actually cannot deal with normal human-scale tools.

NARRATOR: So they modify their design, making a smaller hand with just three fingers.

BRETT KENNEDY: This does everything we needed it to in the competition. It grabbed everything—the human tools—so we thought we were in pretty good shape. 

NARRATOR: But then, at the first challenge, Robosimian had to open a door, and things got messy.

BRETT KENNEDY: So, here's an individual finger, and in that individual finger there is an artificial tendon, it's a synthetic fiber, when excessive forces happen, it snaps.

NARRATOR: When Kennedy realized Robosimian's fingers couldn't provide the strength he was looking for, he came up with a radically different idea.

BRETT KENNEDY: So this is the "cam hand." To do most of the work that we need to, a simple hook works just fine, so that we can actually get a grasp, and we can hold onto most everything we have to.

The fact of the matter is this is a very simple, dumb system. It closes until it encounters something and it will hold it, and it'll hold it securely.

NARRATOR: In their quest to grasp victory, the teams competing in the DARPA challenge will rely on all kinds of hands. Many are using this gripper, designed with three fingers that can wrap around a variety of objects and even pick up something small.

CLÉMENT GOSSELIN (Roboticist, Laval University): If we look at tasks like, for instance, opening a door or using a drill, using hand tools and things, pretty much all these tasks you can perform using three fingers.

NARRATOR: Robotic grippers have already made their way onto the factory floor, attached to massive robotic arms that are as robust as they are precise. They build cars, lift heavy boxes, pack beer, sort through anything and everything from batteries to pancakes, doing the kind of jobs many people consider repetitive and downright boring.

But some experts fear, as robots move beyond the factory into the real world, they'll take on a lot more.

GARY MARCUS (Geometric Intelligence): I think that we will lose at least 20 or 30 percent of our jobs, over the next 20 years, and probably more. Fast-food workers are mostly going to be replaced. The cashiers at Wal-Mart, their jobs aren't going to last that much longer. The first to go are going to be drivers. Uber is spending a lot of money on driverless cars. Google is spending a lot of money. Now Toyota is. Forty years from now, there are not going to be a lot of jobs.

NARRATOR: But not everyone agrees the future looks quite so grim. Over the past century, while robots and automation took over many manufacturing jobs, other kinds of jobs have increased. Some think that trend will continue.

TERESA GHILARDUCCI (Economist, New School): Machines in all of modern economic history have helped create jobs, not taken them away. Machines are complements to workers, not substitutes. They come together and enhance the productivity of each. They need each other.

NARRATOR: Today, more and more jobs require humans to work side by side with robots on the assembly line, programming them, and repairing them. Technology creates new jobs, but only for those who have the skills to adapt.

What happens to the workers left behind?

TERESA GHILARDUCCI: The worker who was displaced has to be compensated through retraining or through a pension. So, that's a social problem, not a problem rooted in technology.

GARY MARCUS: People should already be thinking about what kind of society we would want if not everybody can have jobs.

NARRATOR: Many ethicists say we should also be thinking about which jobs we want to hand over to machines.

RON ARKIN (Georgia Institute of Technology): Much of the things that we are creating can be used for a whole broad range of potential applications, ranging from eldercare and childcare robots, to healthcare robotic platforms, even surrogate sex objects. What is acceptable?

NARRATOR: What impact these high-tech machines will have in the workplace and our homes remains unclear, but one thing's for sure, robots are starting to take their first baby steps out of the lab and into the real world, learning how to manipulate objects we use every day.

But for a rescue robot to be truly useful, there's another hurdle researchers face, the toughest one of all: the challenge of giving a machine the ability to understand its environment, to give it a brain.

MATT JOHNSON (Roboticist, Institute for Human & Machine Cognition): Our robot doesn't really have what you'd call a brain. He's not seeing the world, he's not perceiving the world. It's not thinking, it's not reasoning. You know, it's pretty dumb.

NARRATOR: In fact, today's rescue robots are so dumb DARPA permits human operators to guide them, step by step. The bot waits for instructions that tell it what to do: those zeros and ones that computers understand. The instructions travel through an elaborate pipeline that connects every piece of hardware, every motor and sensor to a computer controlled by a team of human operators. The operators communicate with their robots via a wireless connection, just like they would be at a real disaster and at the final challenge.

These operators, at Carnegie Mellon University, are about to guide their rescue robot, named CHIMP, through one of the skills it needs to master for the final DARPA challenge, turning a valve. Using six different cameras, located on the front and back of its head, and a spinning sensor, called LIDAR, the bot sends data about its environment to the operators.

A 3D image of CHIMP's world appears on their monitors. CHIMP has no idea what it's looking at, so, with the click of a mouse, the humans tell it exactly where the valve is.

CLARK HAYES (Roboticist, Carnegie Mellon): Recognizing objects, that's a very hard problem in robotics, so we let the human tell the difference between a cat and a dog, a valve and a door.

NARRATOR: Next, they show CHIMP where to grab the valve, which way and how far to turn it. But it would be impossibly slow and impractical for the operators to tell CHIMP how to move every joint, every sensor, every motor of its complex arm. This job must be done by the robot, all on its own. Using a process called "motion planning," CHIMP determines the best path for its arm to travel.

ERIC MEYERHOFF (Engineer, Carnegie Mellon): It shows the operator what its intentions are, what its path will be, how it's going to get there. There's a plan. In the end, it's going to be, you say, "Okay, I approve your plan, CHIMP. Go ahead and do your thing." And off it goes.

CLARK HAYES: As we move towards developing this technology further, we really want to push on the robot autonomy and let CHIMP do more things on its own, but we always want to keep this as a tool for a human.

NARRATOR: In other words, keep the human in the loop and in control, but that's not every team's goal.

At the Massachusetts Institute of Technology, Russ Tedrake is taking a different approach. He is determined to give his Atlas robot as much autonomy as he can, to make it a whole lot smarter.

RUSS TEDRAKE: Our goal as researchers, especially in this artificial intelligence lab, we want to solve the long-term research questions about how to make autonomous robots. This is the closest we've ever come to building an artificial intelligence machine, where this humanoid robot is moving through the world, solving real problems.

NARRATOR: At a disaster, where you can't always count on a wireless connection, the smarter the bot gets, the more it can do on its own. To create a more autonomous robot, Tedrake is developing software that helps it find and identify objects with a bit more independence.

RUSS TEDRAKE: So, if the robot is just looking at my kitchen at home, there are dishes everywhere, and you ask the robot to find a spoon, that's a really hard question. If the human just says "there's a spoon, roughly, over here, click," and he just has to look in a little patch of space for something that's roughly the same shape as a spoon, that takes an extremely hard problem of object recognition in a complicated environment, and turns it into a very simple problem of, "Okay, I want to look for spoon-shaped things in this small region of space."

NARRATOR: While CHIMP needs its human operators to tell it exactly where a valve is and where to grab it, M.I.T.'s Atlas is programmed to recognize a valve on its own, figure out the steps it needs to take as it approaches, then grab and turn it, with very little human help.

RUSS TEDRAKE: We can put it in an autonomous mode and, basically, just watch the robot execute. It shows us what it's about to do. We could always stop it if it looked like it was going to do something wrong. But, when things are going well, the robot's operating almost completely autonomously.

STUDENT ROBOTICIST #3: There's no real input coming to us from the robot, at this point. It's just going on with its task.

NARRATOR: A more self-sufficient robot could, potentially, help save more lives in a disaster, but just how independent do we want rescue robots to get? What if an autonomous robot enters a disaster and has to decide who lives and who dies?

It's the kind of moral dilemma that gave Will Smith nightmares in the movie I, Robot,…

ALAN TUDYK (As Sonny the Robot in I, Robot/Film Clip): You are in danger.

NARRATOR: …when a bot chose to save his life over the life of an eleven-year-old girl.

WILL SMITH (As Del Spooner in I, Robot/Film Clip): Save the girl. Save her.

I was the logical choice. It calculated I had a 45 percent chance of survival. Sarah only had an 11 percent chance. Eleven percent is more than enough. A human being would have known that.

NARRATOR: While today's cutting-edge machines, like the thousands of drones used by the U.S. military, still have a human in the loop, what happens in the future, if they don't?

AYANNA HOWARD: This conversation is going on right now, conversations about the ethical laws of robots. How far should we really push this technology?

PETER SINGER (New America Foundation): How much autonomy do we want to give this new technology? Even the semi-autonomous ones raise certain interesting issues, like, "Who's to blame when something goes wrong?"

RON ARKIN: These systems will not be foolproof. They will not be perfect. It's important to remember that.

NARRATOR: How these robotic technologies, autonomous or not, will be used on the battlefield of the future, remains an open question. But now, just a week before the final challenge, the roboticists are starting to feel the mounting pressure. Brett Kennedy, who leads the Robosimian team, is no exception.

BRETT KENNEDY: I really have no idea how we're going to place within the overall field at the robotics challenge. All the researchers that are bringing their teams are top flight, so where we end up in that, I don't know.

NARRATOR: Will Jerry Pratt's years of research with bipedal robots finally pay off? Will Paul Oh and Junho Oh redeem themselves after Hubo's poor performance at the first challenge?

JUNHO OH: My students, they're very nervous, because it's the last chance, and we believe that we can do it.

NARRATOR: Is CHIMP's combination of hardware and software just what its human operators need? Or will more autonomy help or hinder the team from M.I.T.?

RUSS TEDRAKE: There're so many things that could go wrong or,…geez, the chance that the robot could break, or something that worked 99 out of 100 times, we get unlucky. It's scary.

NARRATOR: June 5th, 2015, our roboticists meet once again, this time at the Los Angeles County Fairgrounds. Over the next two days, they'll face-off with finalists from around the world. Several teams come from Korea and Japan. This little robot comes from Germany.

Most are funded through government or corporate sponsorship, but some bots, like this odd looking one, called Cogburn, are funded by the teams themselves.

JUNHO OH: You want to be a first place in this competition, you too?

PAUL OH: Well, yeah, of course.

JERRY PRATT: Very stiff competition, looks like there're a lot of good teams, probably be a lot that make it through the course, so it will come down to the top speed.

NARRATOR: To win, each robot must perform a series of tasks, a lot like the ones they faced in the first challenge, from driving a car to drilling a hole in a wall, to walking over debris, to tackling a flight of stairs. The robot that completes the course in the shortest time wins.

That puts a lot of pressure on the operators to keep their bots moving. And DARPA has thrown another wrench into the mix, just like at a real disaster. The power of the wireless connection between the operators and their robots fluctuates. Sometimes the robots will receive a degraded signal, just bits and pieces of the data that tells them what to do, other times, no data, no instructions at all.

DARPA is hoping this will push the roboticists to develop more autonomous systems, bots that can finish a job without our help.

But what makes the final challenge downright nerve-wracking is that, unlike the first competition, where the bots were tethered, this time around there are no safety lines allowed, putting these multi-million dollar machines in real jeopardy.

GILL PRATT: What I was worried about is that almost none of the teams had tested their robots off the safety tether. We had no idea, when these robots fell, how badly they would break, so, many of the teams were really scared.

NARRATOR: Finally, the competition begins. Each robot has two chances to run the course. I.H.M.C. starts their first run. The robot drives without a hitch,…

JOHN KARP (Main Operator, Institute for Human & Machine Cognition): Perfect.

NARRATOR: …walks up to the door with ease,…

JOHN KARP: Go through the door.

NARRATOR: All is looking good. Just two more tasks to go.



NARRATOR: It's Jerry's worst nightmare. After years of perfecting his walking software, a tiny misstep in the real world brought one of the most advanced robots on Earth to its knees.

JERRY PRATT: Nobody was really sure if the Atlas robot would survive the fall. You know, we've never had Atlas fall before today.

NARRATOR: The team worries the bot can't be repaired in time for its final run.

For Carnegie Mellon's robot, CHIMP, the day is also filled with ups and downs.

ERIC MEYERHOFF: When he fell down through the door, you know, he's laying there, and we were just, "Oh no."

NARRATOR: Back in the garage, the operators scramble to find a way to get the red bot back up on its treads again. But the bigger a bot, the harder it falls. Smaller robots fall without breaking, and have truly bizarre ways of getting back up again. But they're too small to do the kind of heavy lifting rescue robots need to do at a disaster.

DENNIS HONG: Once you make it heavier and larger, you need stronger motors, and stronger motor means heavier motors. And heavier motors increase the overall weight.

NARRATOR: CHIMP weighs over four hundred pounds. That's an awful lot of robot to lift. The crowd begins to wonder if its run is done.

JOHN MARKOFF (New York Times): I found it fascinating to watch the crowd watching the robots. This was really an emotional moment. They're collections of wires and gears and motors, and we were sympathizing with them at a very basic level.

NARRATOR: Then CHIMP's leg begins to move.

ERIC MEYERHOFF: This little kid down in the front screams, "He's getting up!"

NARRATOR: The bot not only gets up, it completes all eight tasks, becoming the first robot to finish the course.

But not every team is so lucky. As M.I.T. starts its run, hoping to show off its robot's autonomy, things run amok in the control room.

RUSS TEDRAKE: We made a simple operator error. When we went to get out of the car, we forgot to turn off the driving controller, so the foot tried to push the throttle, when it was getting out of the car.

No, no, no…

NARRATOR: In an effort to drive and get out of the car at the same time, the robot keels over and breaks its right arm.

RUSS TEDRAKE: But the operators were able to recover and do, basically, the entire course left-handed.

NARRATOR: The robot is still able to do most of the tasks, but it faces one insurmountable problem: the team has programmed it to use two hands to pick up the drill and turn it on. It can't do that when one of its hands is broken. Without completing all the tasks, team M.I.T. has no chance of winning.

A weary team I.H.M.C. has worked through the night trying to fix its robot.

JERRY PRATT: Somehow, the robot's still working, but the fall we took did something with the robot where he's kind of all messed up a little bit.

NARRATOR: There's something wrong with its LIDAR, the system that scans the robot's environment.

JERRY PRATT: The operator is going to have to adjust on the fly and compensate in his head for the errors in our sensors.

JOHN KARP: Are we really leaning that much?

JERRY PRATT: Our strategy is still what it was yesterday: eight points or bust.

NARRATOR: The time has come for their final run.

JOHN KARP: All right!

NARRATOR: They make it through all the tasks, and approach the dreaded debris, where they fell on their first run.

JERRY PRATT: Right first…

NARRATOR: But this time, they succeed. The operator guides their robot all the way up the stairs, finishing the course in record time.

JERRY PRATT: You know, having a human in the loop, you've got a supercomputer up here. Humans can adapt to just about anything.

Eight points confirmed.

NARRATOR: The team is now in the running for first place. But a one-of-a-kind bot could still give them a serious run for their money. Cousins Paul and Junho are each competing with their own robot, doubling the chances that one of their humanoids on wheels will win the day.

Paul's team makes it as far as the drilling task, until the drill gets stuck, overheats and shuts down. But cousin Junho's team moves on. Their robot completes all the manual tasks by rolling on its knees or standing on its feet. But when it approaches its final task, the stairs, it suddenly stops. Back in the garage, its human handlers are double- and triple-checking the instructions they're about to send their robot. A mistake here will cost them the competition.

JUNHO OH: You have to understand that robot is staying. Stay means that operator room very busy. If the robot is moving, they don't do anything.

NARRATOR: Their job is done. All they can do now is watch and wait. Finally, Hubo starts to climb the stairs, unlike any human on earth, with its head facing forward and its feet facing back.

The humanoid on wheels finishes all eight tasks faster than any other robot. Junho Oh's robot Hubo wins the day; I.H.M.C. comes in second; Team CHIMP, from Carnegie Mellon, comes in third.

These bots may have moved slowly, but, just like a toddler, they're taking baby steps into our world.

RODNEY BROOKS: We tend to think the world today is what is going to be like in 10 years and 20 years. But if we look back 10 years what we have, look back 20 years what we have, the world has been totally transformed. It's really hard for us to imagine how different it's going to be.

VIJAY KUMAR: In some ways, the change in the field has been very incremental, but, in other ways, it's been completely unpredictable.

SHERRY TURKLE: It's too easy to look at them and say, "Oh, they're not there yet. They're…" Well, they will get to something very powerful, and then you have to say, "Well, where will we have gotten to?"

NARRATOR: In fact, a few months after the finals, Boston Dynamics, the makers of the Atlas robot, took this updated version for a stroll in a snow-covered woods. While it stumbles, it quickly recovers its balance, just as we would.

SHERRY TURKLE: So, I just think that we just need to approach this with our eyes open and understand our vulnerability to a technology that we are on the cusp of creating.

NARRATOR: There's no question that developing rescue robots, with the potential to save lives, makes a lot of sense. But the potential for other applications remains unclear. The time is now to think about their role in our lives, as we face the Rise of the Robots.

Broadcast Credits

Terri Randall
Jedd Ehrmann
Joseph Friedman
Steve Golden
Dale West
Eric Anderson
Jason Longo
Mike Coles
Joseph Brunette
Peggah Navab
William Howell
Lillie Paquette
Donika Jordan
Seungjoon Lee
Timothy Kitz
Dan Seiler
Keith Rodgerson
Frank DeAngelis
Dino DiStefano
Juan Fernandez
Pierre Bouchard
Eduardo Bejarano
Broatch Berry
Rick Prell
Craig Sechler
Derek Breuer
Joshua Shawn Zimmerman
David Margolies
Frame Runner
Evan Anthony
Sync Sound
Tony Pipitone
Leora Randall-Tavori
Zak Prell
100 Robi, Tomotaka Takahashi, K.K. DeAGOSTINI Japan, Vstone Co., Ltd, GEN ink.
AP Archive
Army Video
BAE Systems
BMW Manufacturing Co.
Carnegie Mellon University
Department of Defense
EBS (Korea Educational Broadcasting System)
FANUC America Corp
Ford of Europe
Fraunhofer IPA
Georgia Tech
Getty Images
Gregg Webb/IAEA
HYFLAM and the University of Hamburg
Hutchinson Community College, Visual Media Design – Animation and Game Development
Robin Kelly
Kia Motors Corporation
Kings College Library, Cambridge
Lawrence Journal- World, The University Archives
miSci: Museum of Innovation and Science
New Tang Dynasty Television
News Market, Inc.
Jerry Pratt
Prelinger Archives
Rethink Robotics
Riken RTC
Shadow Robot Co.
Toyota USA
TranTek Automation Corporation
Twendy-One, Waseda University
UK Aviation Movies
University of Bonn
U.S. Air Force
U.S. Navy
Joshua Shawn Zimmerman
Johanna Spangenberg Jones
Timothy Kilbride
Erico Guizzo
Will Knight
Samuel Bouchard
Kathleen Cullen
McGill University
Mark A. Vasquez
Byron Spice
Elena Meja
IEEE Robotics and Automation Society
John Daniels
Keil Vedrode
Kiwon Sohn
Lan Yoon
Megan Downs
Peter Ford Dominey
Pierre-Yves Oudeyer
Sisir Karumanchi
Steve Wilson
yU + co.
Walter Werzowa
John Luker
Musikvergnuegen, Inc.
Ray Loring
Rob Morsberger
The Caption Center
Spencer Gentry
Jennifer Welsh
Eileen Campion
Eddie Ward
Diane Toomey
Caitlin Rotman Saks
Anne Barleon
Linda Callahan
Sarah Erlandson
Janice Flood
Susan Rosen
Kristine Allington
Tim De Chant
Lauren Aguirre
Lauren Miller
Ariam McCrary
Ezra Wolfinger
Brittany Flynn
Kevin Young
Michael H. Amundson
Nathan Gunner
Elizabeth Benjes
David Condon
Pamela Rosenstein
Laurie Cahalane
Evan Hadingham
Chris Schmidt
Melanie Wallace
Julia Cort
Alan Ritsko
Paula S. Apsell

A NOVA Production by Terri Randall Productions for WGBH Boston.

© 2016 WGBH Educational Foundation

All rights reserved

This program was produced by WGBH, which is solely responsible for its content.

Original funding for this program was provided by Google, Cancer Treatment Centers of America, the David H. Koch Fund for Science, the George D. Smith Fund and the Corporation for Public Broadcasting.


Image credit: (robot)


Ronald Arkin
Georgia Tech
Rodney Brooks
Rethink Robotics
Armando De La Rosa T.
Shadow Robot Company
Teresa Ghilarducci
Clément Gosselin
Laval University
Clark Haynes
Carnegie Mellon University
Dennis Hong
Ayanna Howard
Georgia Tech
Matt Johnson
Brett Kennedy
NASA Jet Propulsion Laboratory
Vijay Kumar
University of Pennsylvania
Gary Marcus
Geometric Intelligence
John Markoff
The New York Times
Eric Meyhofer
Carnegie Mellon University
Junho Oh
Paul Oh
University of Nevada, Las Vegas
Gill Pratt
Jerry Pratt
Nicolaus Radford
Former JSC Valkyrie Team Leader
Peter Singer
New America Foundation
Siddhartha Srinivasa
Carnegie Mellon University
Tony Stentz
Carnegie Mellon University
Russ Tedrake
Sherry Turkle
Rich Walker
Shadow Robot Company


Full Program | 52:55

Full program available for streaming through

Watch Online
Full program available