NOVA Online | Special Effects: Titanic and Beyond

Virtual Humans
by Kelly Tyler

After millions of years of natural selection, humans beings have some serious competition for their lofty perch on the evolutionary ladder—and the challenger has only been evolving for less than a decade. Some computer artists contend that anything we can do, "virtual humans" can do better, and they're poised to revolutionize moviemaking with a new species that doesn't require an astronomical salary, works around the clock without complaint, and lives quietly on a hard drive between death-defying stunts.

A generation of computer-generated (CG) characters, called "synthespians" or "vactors," are attracting notice in Hollywood. Some insiders envision a future when digital stars compete for roles with the flesh-and-blood variety. While a photoreal digital actor has yet to carry a major motion picture, synthespians have captured supporting roles for some time now, whenever the going gets too tough or too expensive. Synthespians serve as doubles for breathtaking stunts too dangerous for mortal stars: a girl leaping from a skyscraper in "The Fifth Element," Sylvester Stallone chasing through the skies on an airborne motorcycle in "Judge Dredd," and a luckless attorney becoming tyrannosaur fodder in "Jurassic Park." And producers cut costs on the "cast of thousands" by using digital extras to stand in for the legions of troops in "Hamlet," mobs of Washington demonstrators in "Forrest Gump," and passengers aboard the doomed Titanic.

Fooling the eye

The leap from extra to starring role for synthespians is a big one, since it invites heightened scrutiny from the viewer. Human beings have a finely tuned ability to recognize their kind, an ability that is thought to be both innate and learned, and that ups the ante for filmmakers seeking to fool them with a synthetic stand-in. Creating convincing movement is particularly difficult. Animators can take the perceptual challenge head-on and painstakingly create movement for their characters frame by frame from scratch, or they can use the real thing. A technique called motion capture allows actual movement to be recorded and applied to digital characters. An actor wears reflective markers at key body joints, and surrounding cameras record the motion of reflected infrared light in the computer. Later, this motion data is transferred to the digital character.

The human face presents an even more daunting challenge. Ed Catmull, a computer graphics pioneer since the late 1970s and a founder of Pixar ("Toy Story," "A Bug's Life"), regards it as a central issue in character animation.

"The human face is a unique problem," he says. "We are genetically programmed to recognize human faces. We're so good that most people aren't even aware of it while they think about it. It turns out, for instance, that if we make a perfectly symmetrical face, we see it as being wrong. So we want things to be not quite perfect, have a lot of subtlety, but if they're too imperfect, then we think that they're strange."

For Scott Ross, President of Digital Domain, the problem is more intangible: "One of the things that I'm mostly concerned about in terms of virtual actors is that there's been millions of years of experience in our genetic code. And I'm concerned that when you create a close-up of a virtual actor and look into its eyes, that it will take real skill to be able to give that virtual actor soul. And I've not yet seen that."

The silicon rush

Ivan Sutherland demonstrates Sketchpad.

The quest to create virtual actors is comparatively recent; the first interactive computer graphics program was only developed in 1961. Designed by Ivan Sutherland at the Massachusetts Institute of Technology, Sketchpad generated simple geometrical line drawings for design and engineering applications. These simple operations required a state-of-the-art TX-2 defense computer to run.

The silicon rush in Hollywood began in 1985, when a knight sprang from a stained glass window and handily dispatched a human opponent in "Young Sherlock Holmes," courtesy of computer animation. A virtual stampede of digital characters followed: the water creature of "The Abyss," the quicksilver T-1000 of "Terminator 2: Judgment Day," the menagerie of "Jumanji," and the dinosaurs of "Jurassic Park." In 1995, "Toy Story" was released, the first CG film in history, populated entirely by digital characters in a world made of bits and bytes.<

Voyager fly-by animation

The reason for this explosion? Jim Blinn, an early computer graphics innovator who created the well-known Voyager fly-by animations for the Jet Propulsion Laboratory, credits the decreasing cost of computer memory. "In my lifetime, the cost of the basic tools of my trade—of making images with a computer—has gone from about $500,000 to about $2,000 dollars," he says. "It's a factor of 200 or 300 to one." A corresponding inverse growth in computer power and memory has equipped CG Pygmalions to cope with the high degree of complexity and detail inherent in living creatures. Today's microcomputers have roughly 400 times more memory and operate about 5,000 times faster than the TX-2 used by Sutherland.

Scott Ross projects continuing growth in hardware capability, spurring increasingly sophisticated animation. "The concept of Moore's law states that the processing power of computing doubles every year," he says. "We're seeing that in terms of what we're doing today in the film industry."

Building a better human

With this enhanced technology, animators have turned from fantasy characters and extinct animals to a new digital grail: a photorealistic Homo sapiens. Ellen Poon, a visual effects supervisor for Industrial Light and Magic who has been involved with such films as "Jumanji" and "Men in Black," is optimistic, but sees technical challenges ahead.

"I think we are very close to creating a realistic-looking virtual person," she says. "There are a few things that have to be right, and we're still in the process of researching them. And those elements are hair, skin, clothing, movement, and facial expressions."

The problem of hair has bedeviled animators for years. There are thousands of hairs on the human head, which vary in color, light reflectance, and texture and can move either singly or together. The lion in "Jumanji" required the modeling of one million individual hairs for the mane alone. Nadia Mangenat Thalmann of Miralab, a computer research center at the University of Geneva, tackled similar complexity as she developed clothing for digital characters.

"Fabric is very difficult," says Thalmann. "The computer has to know every moment where the wrinkles are that are created by the movement of the fabric. We used two or three hours of calculation for one single frame of animation to get it right—and there are 24 frames in one second of film."

Questioning the digital grail

Miralab's virtual Marilyn Monroe

View clip of demonstration:
RealVideo: dialup | broadband
Quicktime: 2.2MB
AVI: 2.2MB

You'll need either the free RealPlayer software, or free QuickTime software to be able to view this clip. If you already have the software, choose the appropriate connection speed/file size to view a clip.

Hurdles notwithstanding, the advantages of a virtual actor are appealing. A child synthespian won't have temper tantrums or work hours mandated by labor laws. A vactor will never be busy when it's time for re-shoots after a film has wrapped. A virtual human never grows old. And, to the delight of producers, a digital superstar won't require a $20 million salary, a deluxe trailer and a coterie of bodyguards, masseuses and aromatherapists to get the job done.

For the animators at Miralab, it's only a matter of time and computing power before icons of the past take the limelight again. Thalmann has developed a virtual Marilyn Monroe. She is uncannily realistic, but the illusion loses its photorealistic quality in close-up. Continuing technical advances raise the possibility that there may someday be stars who truly will live forever.

Even so, many in the special effects industry question whether silicon actors will ever pose a real threat to the carbon-based variety. Dennis Muren, visual effects supervisor for Industrial Light and Magic and nine-time Oscar winner, is skeptical of the creative benefits.

"What's the point?" he wonders. "If you want to put Marilyn Monroe in a movie, you could get a terrific actress, give her a great make-up artist, six months of studying and voice training, and she could do a better Marilyn Monroe than we could ever do."

Jim Blinn questions whether synthespians make economic sense: "A dinosaur doesn't exist, so it's practical to simulate it. With human beings, however, having a staff of 20 people all working on the lighting, the modeling, and the motion might not be a great trade-off, because you can replace that whole team with one human actor who can do what the director wants."

To the future...and beyond

Thalmann contemplates taking virtual humans to a new level. "I'm not so much interested to see pictures, which you watch passively," she says. "My ultimate goal is to be able to live in the virtual worlds, and to meet virtual humans that are collaborators." says Thalmann.

Thalmann is not alone. Computer game developers have begun experimenting with artificial intelligence (AI), endowing game characters with the capacity to learn and interact with their environment and the game player.

John Lasseter, director of "Toy Story," is charting a different course. "I'm interested in creating a film with characters that people obviously know don't exist," he says. "But then they look at it and say, `It seems so real. I know it doesn't - but wait. I know those toys aren't alive but it looks so real. No, they can't be alive, no. Are they?' So I think that's one of the really exciting things that computer animation can give you: a combination of fantasy and the photorealistic which has never been seen before."

With the latest generation of talent and tools at their disposal, visual effects filmmakers see a wide range of possibilities in the future, with or without virtual humans. "I think we can do just about anything right now if we had the time and money," says Dennis Muren. "It's really like your imagination has no boundaries at the moment...I'd rather put the work into something that's unique and new, and you haven't seen anywhere before, and there's no way a person could do it."

Kelly Tyler is co-producer of the NOVA program, "Special Effects Titanic and Beyond." She is also associate producer for NOVA's large format film unit, working on the films "Stormchasers," "Special Effects" and the upcoming "Island of the Sharks."

Photos/footage: (1) Reprinted with permission of MIT Lincoln Laboratory, Lexington, Mass.; (2) NASA/JPL; (3) Miralab.

Reel Timeline | 50-Foot Chicken | Wizards of Ahs | Be in Pixels
Titanic: What's Real | Virtual Humans | Resources | Teacher's Guide
Transcript | Site Map | Special Effects Home