Watch Visual Effects artist Joe Letteri explain how the science of light diffusion made digital characters more realistic.
It was near midnight in the winter of 1998 when computer graphics researcher Henrik Jensen made a discovery about light. While playing with a laser pointer, he noticed the way light seemed to diffuse through a sample of marble rock. It got him thinking about skin. Until then, digitally created characters in film had been lacking a certain human quality. And Jensen realized that graphic artists were failing to reproduce the way light rays pass through transparent objects, such as skin.
The phenomenon, known as subsurface scattering, explains why your hand has an orange glow when you hold it up to candlelight. The light passes through the skin and gets reflected out the other side.
Jensen teamed up with Joe Letteri, visual effects supervisor at Weta Digital, and they created the first digital character to have light-diffused skin: Gollum, the depraved human-like character in Peter Jackson’s “Lord of the Rings: the Two Towers.”
Gollum in Peter Jackson’s Lord of the Rings: The Two Towers. Photo courtesy Weta Digital.
“It became obvious that one of the qualities in skin we recognize intuitively, and we don’t think about articulating is the translucency.” Letteri said.
This breakthrough won Letteri, Jensen and colleague Stephen Marschner the Sci-Tech Academy Award in 2004 for “their pioneering research in simulating subsurface scattering of light in translucent materials.”
“The real excitement came when everyone got an eye full of Gollum and saw how convincing the skin texture was, even in close ups,” said Bill Taylor, a member of the steering committee for the Sci-Tech Academy Awards.
The technique would later bring alive characters like the Na’vis, the 10-foot blue alien characters in “Avatar”, King Kong in Peter Jackson’s 2005 remake and Benjamin as an old man in “The Curious Case of Benjamin Button.”
Since Weta Digital pioneered subsurface scattering, nearly every digital character has been aided by this technique, Letteri said. But despite its achievements, subsurface scattering is still evolving. Visual effects engineers and artists are working to master the close-up shot and find a so-called “unified model” for light movement — a model that applies not only to skin, but to other translucent objects like hair, clouds and water.
Close-ups on digitally-created characters that reveal the flaws of the skin in fine wrinkles and pores also reveal the flaws of the technique. As the light bleeds through the skin, it also blurs the fine details.
“If you were really close up, you’d have to dial back to hold detail, Letteri said. “If you were far away, you would allow it to bleed a bit more so you got more of the translucency effect.”
Neytiri, the Na’vi princess of the Omaticaya clan in James Cameron’s Avatar. Photo courtesy Weta Digital.
Light diffuses differently through hair, clouds and water. And lacking a unified model, visual effects engineers have to create a new mathematical model for every single light source on every variation of material when digitally creating a scene.
“Skin has a complex substructure, so a unified model for light diffusion would capture the scattering that occurs at different scales within the skin, rather than treating it all as a homogenous medium,” Letteri said.
A unified model, engineers say, will standardize the practice, simplify the math and improve the realism of the characters.
Letteri has improved this technique in subsequent films such as “King Kong”, where subsurface scattering allowed for the softness around King Kong’s eyes and in his hair, and in Avatar, where blue-skinned aliens had warm, human-like skin.
“We understand how to make things look good,” Letteri said, “but we’re really trying to understand the science behind it.”
The first mathematical model for the technique was published by Jensen and Marschner in 2001. The path that light rays take through the flesh is not practically replicable for visual effects because it take a long time. So Jensen and Marshner came up with an approximation, finding a series of nearby points where the light will travel and affect the surface. Visual effects artists and engineers plot these likely points of travel for light rays at all points of the skin and for every frame. Then, the information is rendered digitally to create the effect.
“Once you present it, it’s obvious, and you wonder how no one has thought of it before,” Jensen said. “but it just wasn’t on anyone’s radar.”
Technology is sometimes slow to catch up with the imagination of the screenwriter. James Cameron, for example, held onto the Avatar script for 10 years, because the technology wasn’t advanced enough to do justice to his aliens. Letteri said, adding that he finally pulled it back out after seeing Gollum.
“It was the first time anyone simulated the real behavior of skin,” Taylor said. “Rather than having an artist simulate skin texture and transparency through layers and layers of paint, they’ve recreated transparency. We can now make extraordinary characters believable.”
Photo credit: Gollum from Peter Jackson’s Lord of the Rings: The Two Towers. Photo courtesy Weta Digital.