Virtual Reality Pioneer Jaron Lanier, Part 1

The virtual reality pioneer discusses his new memoir Dawn of the New Everything and the way our minds are affected by digital technology.

Jaron Lanier is a scientist, musician, and writer best known for his work in virtual reality and his advocacy of humanism and sustainable economics in a digital context. His 1980s startup VPL Research created the first commercial VR products and introduced avatars, multi-person virtual world experiences, and prototypes of major VR applications such as surgical simulation. Both of his previous books, Who Owns the Future? and You Are Not a Gadget, have been international bestsellers. His latest text is Dawn of the New Everything: A Journey Through Virtual Reality.


[Walmart Sponsor Ad]

Announcer: And by contributions to your PBS station from viewers like you. Thank you.

Tavis: So pleased to welcome Jaron Lanier to this program. More than 30 years ago, he founded the first virtual reality company, VPL Research, and is widely credited with popularizing the term.

His new memoir, “Dawn of the New Everything: Encounters With Reality and Virtual Reality”, gives us a glimpse into his unusual childhood and how it led in part to his lifelong relationship with technology. It is an honor, sir, to have you on this program.

Jaron Lanier: Thanks for having me.

Tavis: I saw the piece that Maureen Dowd did with you in The New York Times, a wonderful piece. Did you like it?

Lanier: Oh, yeah, it was charming, sure, yeah.

Tavis: It was a great piece. It opened my eyes to things about you and about the book about your life that I didn’t know, so I was honored to have you on the program. I promised you that if you — so first of all, yeah, thank you, Jonathan. So explain why you have no shoes on.

Lanier: This is the fourth time in my whole life that I’ve worn a jacket. They have a dress coat here, so I said I’ll wear the jacket if I don’t have to wear shoes.

Tavis: Okay.

Lanier: So that was my bargain.

Tavis: And do you typically walk around barefoot?

Lanier: I like walking around barefoot if I can just because, you know, we have a lot…

Tavis: I know you don’t normally wear a jacket, so for me, you did. I thank you. Are you uncomfortable in the jacket? Do you want to take it off?

Lanier: My only problem is that you’re wearing shoes.

Tavis: So if I take my shoes off, you’ll be more comfortable?

Lanier: I’ll be more comfortable [laugh].

Tavis: Okay. Just take my shoes off. I want you to be comfortable. I’m just glad you asked me to take my shoes off [laugh]. That could be a problem if you asked for more than that.

Lanier: Hey, listen…

Tavis: No, no, no. We’re not gonna go there.

Lanier: Hey, we got to think about those ratings, man.

Tavis: There are my shoes. Take off my socks. I know my mama’s having a heart attack in Indiana right now.

Lanier: Oh, no!

Tavis: She’s like, “Is Tavis taking off his…?”

Lanier: He’s a good boy. You have nothing to worry about.

Tavis: Are you comfortable now?

Lanier: I am so comfortable.

Tavis: All right. Let the conversation begin [laugh]. Tell me about when you first became interested in, enamored by the notion of technology.

Lanier: Oh, God. You know, for me it had a really emotional start. What happened is my mom died when I was little and she had had a tremendous weight on her when she was alive. She was a concentration camp survivor.

And somehow just the world seemed so dark and especially other people seemed like so distant like these planets far away, these like fleshy planets with these brains inside that I could never know. I mean, I felt so isolated. I had this encounter actually with a painting, with Hieronymus Bosch’s “Garden of Earthly Delights” with this surreal painting, a medieval, old one, you know.

Just this idea that you could do things that brought what was inside your head out so they could be shared with other people. And you can call that art, you can call it technology, but that idea that we could actually do things to bridge that gap to these distant, distant others, it just seemed like the only hope.

Then I started discovering technology and I thought this is the way I can do it. I did everything I could — I mean, it’s just kind of crazy, but when I was a little kid, I would like build these electronic haunting houses to try to create what I thought was an experience from inside my head that might be meaningful to other kids.

So for me, it’s really, I guess, ultimately it’s about reconnecting with my mother, if you can believe it. I know that might sound farfetched, but that’s really what it is.

Tavis: It doesn’t sound farfetched at all. It actually roots it and grounds it for me. Because it’s hard for me, not being the techy that you are, it’s hard for me to imagine how your brain conceptualized this notion of virtual reality.

So the story you’ve just told now gives me some understanding of what you were aiming for. Tell me more about the practical process of what you were doing with your research company vis-à-vis virtual reality.

Lanier: Well, before I get to the company, let me go back a little bit. It’s the 70s, I’m a teenager, I’m like about 15. I discover that there’s a guy named Ivan Sutherland. He invented computer graphics, he invented interaction on the screen, all that. And he actually made in the late 60s the first thing that we’d call a virtual reality goggle. You know, a very simple one, obviously, because it was the first one.

But when I read that, I got so excited because I thought this is my path to creating this kind of dream-sharing thing. I would just run up to strangers on the street with like this computer science, “You gotta look at this! We’re all gonna share dreams!” They’d be like, “Who’s this weird kid?”

Remember, in those days, there wasn’t an internet, so that’s the only way you could sort of reach people. You just would run up to strangers. I know it sounds incredible. But later on, what I thought is I want to take what Ivan had done and turn it into this social thing with multiple people.

I was really fortunate. In the early 80s, I had a hit videogame, so I had royalty checks coming in and, instead of doing anything sensible like buying a house or whatever one’s supposed to do, my friends and I just started building these machines in garages.

And we didn’t know if we were starting a company. We didn’t know what we were doing. Eventually, one of the Silicon Valley founding venture capitalists came up to me and said, “Young man, you need venture capital.” I’m like, okay [laugh].

Tavis: Whatever that is, yeah.

Lanier: Yeah. Then all of a sudden, we had like a company. We actually built a little factory just with local people working. You couldn’t go to China, you know. It was a whole different world. We started building these things. So virtual reality means — Ivan Sutherland’s original one was just virtual world, one person. Virtual reality meant multiple people shared. That’s reality.

It meant you’d see each other so you’d have what we call avatars. You could turn into different creatures. You might say, “Who would buy this weird dream machine?”, especially since they cost over $2 million dollars a person back then So it was all like people designing cars, designing surgical procedures, all kinds of, you know, fancy customers. But really what it was about for us was this dream of dreams, you know.

Tavis: And what do you make then, Jaron, what do you make then of what has become of the efforts that you all put forth then?

Lanier: I have really mixed feelings. I have some incredibly positive and warm feelings and then I’m really creeped out by some other things and really disappointed. So I have like a really wide spectrum.

Tavis: Give me the range of the joy and the creeped out on the other end.

Lanier: You want the good first or the bad first?

Tavis: Whatever you want to do. I got my shoes off, so I’m with you, man [laugh].

Lanier: Okay [laugh].

Tavis: Wherever you go, I’m following you. So it’s too late to ask me now what I think. I’m in already, so go ahead, yeah.

Lanier: That happens, I guess.

Tavis: It happens in life, yeah, yeah.

Lanier: All right. So the joy is, you know, there’s this whole new generation of young people now who are building their own virtual reality stuff. I think some of them are creating works of real beauty. I think virtual reality has the potential to be maybe the greatest artform.

The way I like to think of it, it’s a combination of jazz because you can improvise virtual reality. You can make stuff up. It’s like anything’s possible. So you have this freedom that jazz brought to music. It’s that plus obviously computer stuff. There’s programming. And then it also has cinema because it’s vivid.

So you have jazz, programming and cinema all rolled into this thing and there are kids who are making that. I sometimes cry really when I see like somebody who’s in their 20s and, you know, they build this beautiful thing and I’m thinking, wow, you know, this is amazing to see. It’s just fantastic.

Now the creepy side [laugh], this is not very pleasant. But there’s this other side to virtual reality. So at the same time when I was a kid and I got so excited about the possibilities, I also read an early computer book called “The Human Use of Human Beings” by Norbert Weiner.

He talks about how if you have a computer that’s continuously interacting with somebody and calculating the feedback that person gets for everything they do, you can turn it into a behavior modification device that could be the most powerful possible one. It’s like putting a little rat in a cage where you’re constantly monitoring the rat and giving it different stimulus.

And he says in the book, “In order to do that, you’d need like a global computer system where everybody’s connected wirelessly all the time and they have devices wherever they go”, and that’s impossible. But, of course, that’s exactly what’s happened. So we have now these huge businesses like social media companies that are effectively giant behavior modification empires and…

Tavis: That’s a damning indictment. It might be right, but it’s…

Lanier: It’s right. It breaks my heart and it terrifies me. Of course, you know, Facebook owns one of the big VR companies. If VR goes down that same path in what we’ve seen so far with election meddling and all these things, it’s going to be so tiny compared to what could happen.

It could be the creepiest invention ever. It could be the most beautiful way of connecting with people and the most beautiful platform for art, or it could be the way we lose our free will forever and just become, you know, like the vision of the Matrix movies. And this is both are very real, both are very present.

And I have to say something else. Some of your viewers might say, “Well, then turn back for a minute. Why do you even go there?” I’ll tell you why. In the 80s, my proudest moment was co-creating the first surgical simulator with some of the folks from my group and with a surgeon from Stanford named Joe Rosen.

The last couple of years, my wife was battling cancer and she had an operation by a surgeon who was trained by somebody who was trained by Joe, my original collaborator, using a procedure that was designed in virtual reality that he trained for in virtual reality and now she’s cancer-free, okay?

So we cannot turn back. Technology is life. We cannot turn back to the helplessness and the suffering of the past which we so easily forget. So what we have to do is we have to move forward, but we have to move forward with our eyes open and not confuse ourselves to death. You know, we have to choose the good side of it, not the bad side of it.

Tavis: When you said rhetorically, for those who say “Why not turn back?”, what I thought you were about to say was there is no turning back. It’s not just not turning back because there’s so much more good to be done.

In my mind, there’s no turning back because I don’t know if that’s possible. It’s like, pardon this old word phrase, but the genie’s out of the bottle. How do you, in any aspect of technology, is it possible in any field or form of technology to turn back at this point?

Lanier: Once in a while, it is. And sometimes we should, but overall, we can’t turn back. Like I’d personally like to see a total ban on nuclear weapons, for instance. They do no good. Like why, you know? And there used to be an argument they create peace. Well, now they’re just creating horrible regimes. Why?

Tavis: Mutual deterrents? That’s the argument. Mutual deterrents.

Lanier: Yeah, yeah, but what it’s mutually deterring between the most awful regimes. You know, I mean, it’s not actually — you know, if it was mutually deterring stable democracies, that’d be one thing, but enabling North Korea is not a viable long-term solution.

Tavis: The other thing you said, Jaron, that got me thinking was, as you were talking, I was trying to do two things. I was trying to multitask, listen to you and, at the same time, I’m trying to think of another artform that does what you think VR has the capacity to do, or technology, more broadly, which is as you suggested, to be the arbiter of everything artistic and good and beautiful about the world and, at the same time, to be the worst invention ever created.

So to your point about the nuclear weapon, that’s just bad all the way around to my mind. Is there anything else that rivals VR for the capacity to be both good and evil at that level?

Lanier: Yeah. To me, the language.

Tavis: Language?

Lanier: Yeah, language.

Tavis: Okay. I can see that.

Lanier: So I used to call virtual reality post-symbolic communication, meaning language is made of symbols that refer to things, words and so forth. And virtual reality would be actually make the stuff. Now, believe me, you can spend all year arguing with philosophers about those terms.

But the basic idea is like when you’re a little tiny baby, you don’t know what’s real and what’s just in your head. So that means it makes you like a god. And suddenly you discover you’re this horribly like weak little pink thing that wets itself.

It’s like this huge demotion and then you start letting words and you discover that at least there’s this little part of your body, the tongue and the mouth, that can at least reflect and refer to all the other things you don’t have the immediate powers to affect or control. Then by communicating with words and by doing things with your hands, you start to gain not perfect god-like control, but gradual control of the world.

Essentially with virtual reality, what you can get potentially is a very fluid or virtuosic way of making the stuff directly even though it’s not real, even though it’s only virtual. Now we don’t have that ability yet, but we could with something like a musical instrument for creating realities. And I believe such a thing is possible.

And then you’d have this other option not just to use symbols to refer to things, which is to directly make the stuff of reality. So I think of it as being sort of like another step in the same direction that language took us.

Tavis: I’d never process it that way, but now you really got me thinking [laugh].

Lanier: This goes on and on [laugh].

Tavis: I see. I need more time to process that, but I think you’re right about the language thing. Let me shift. While I’m thinking about that on this side of my brain, let me shift to this side. Net neutrality. Big debate now.

Lanier: Yeah.

Tavis: 180 degrees, the Trump administration for what Obama said and attempted to do. Where are you at in this debate?

Lanier: Oh, yeah. This is a tricky one. Okay, so look. If the choice is between Facebook is overlord or Comcast is overlord, choose Facebook [laugh]. And so…

Tavis: Those are not good options either way, yeah.

Lanier: No. What I believe is that there should be a third option. What bothers me about the Net Neutrality Act is we’re sort of being asked to choose who the natural monopolist will be, all right? And like what we should be asking is how can we avoid natural monopolists?

You know, like how can we have a more vibrant kind of market, a more vibrant kind of society without this kind of power and wealth concentration and total control of communication that will accrue to one side or the other?

Tavis: That would be unAmerican, you realize [laugh]. That’s how things are done in America.

Lanier: I mean, in theory, America is supposed to work on in some combination of capitalism and democracy…

Tavis: In theory, yeah.

Lanier: And absolute monopoly on information transmission is not really in line with either of those. So, you know, for the moment, I mean, I think we have to demand that neutrality, I think, it’s the better option. In the longer term, I think there are other options that are better still because, you know, just if you make everything — this is a case where a very beautiful and very pure-hearted angelic project from the left backfired terribly.

So what happened is, in the early days of the internet, there was this tremendous feeling which remains for a lot of people that everything should be free, that free music, free journalism, free TV, free news, free email, free social media, everything’s free, but we still adore and mythologize our entrepreneurs.

We love our Steve Jobs and our Bill Gates and all that. So if you like both of those things and you want to combine them, what’s left? And there’s only one business model left which is advertising. So everything’s free, but there’s ads.

Now that would be fine except that when you have computers watching what everybody does every second and then providing feedback every second and advertisers are influencing that feedback, you’ve totally left behind advertising as we’ve ever known it and you’ve entered into this other realm of algorithmic behavior modification, okay?

And so essentially this project of the left turned into this massive behavior modification scheme. So the only way out is either to make everything free or everything paid.

If we try to do this combination of free stuff in a market society, you end up with behavior modification and that’s not survivable. So, you know, we have to find our way to some kind of a solution where there can be millions of little entrepreneurs, incredible diversity and no giant overlord who’s controlling information.

Tavis: How do you explain to people that a guy who is the godfather of VR can end up growing to become — can morph into someone who is such a staunch critic of the internet, or certainly activities that the internet allows for?

Lanier: Look, to me it all goes together. I want to say something. I love Silicon Valley. I love the big companies. My friends and I sold a company to Google. I’m currently working with Microsoft, total disclosure. It’s my world. It’s my world. But the thing is, as much as I love my world, I think it’s not only our duty, but just our truth, our joy, our center to just be honest with ourselves.

Like if you’re going to engineer something, you have to see how it functions. You have to be honest and look at the results. Otherwise, you’re just a superstitious con man. Like it’s the results that make engineering into engineering. So to me, I love what we do. I just feel that we have to be self-critical. We have to look at what we do critically.

You know, it used to be kind of lonely [laugh]. There weren’t too many people saying that, but it isn’t anymore. Like I think more and more, a lot of folks in Silicon Valley are recognizing that it’s really time to change our orientation a little bit and think a bit more self-critically. It only makes us better engineers, yeah.

Tavis: To the extent that you can, can you give me some evidence of that? Like give me more specifics on what you’re seeing that makes you believe that Silicon Valley’s becoming more self-critical.

Lanier: Well, for one thing, there’s just a lot of people from Silicon Valley who are saying, “Hey, there’s something wrong now.” And, I mean, just the whole — the election had a lot to do with it. You know, I think of people were really — that was a real wakeup call, you know. I think we have a ways to go, but I really kind of feel optimistic we’re gonna get there. I really do.

Tavis: Let me ask a question that has nothing to do with your book or nothing to do with anything except it’s one of my issues. And I’ve raised it so many times, countless times, over the course of my career, which is — and I’m only raising it now, Jaron, because if Silicon Valley is becoming more self-critical, then maybe they’ll finally get around to this, which is the democratization of Silicon Valley.

Put another way, why there is still so precious little color in Silicon Valley? If they’re starting to be self-critical, are they gonna get around to that?

Lanier: We have to. I mean, like…

Tavis: It’s like people of color are good enough to be consumers, but not producers.

Lanier: So there’s a lot of ways to address this question. I mean, I could go on for hours about just this one thing. Let me mention a couple of ways. This is not a pleasant thing to talk about because the current situation is crappy, right? Let’s just be blunt about it.

Okay, so the first thing to say is that technology and especially information technology is a very human endeavor. We like to pretend that we’re like in lab coats and that we’re doing this thing that’s very objective, but it isn’t. Like when we make algorithms, it reflects our assumptions and our culture.

And to the degree that we can’t diversify our own teams, we’re actually limiting our stuff and making our stuff worse. There’s so many examples of that. Like I saw some research that indicated that virtual reality worked better for men than for women and the researchers claimed this is intrinsic.

And I said, well, look at the teams who made the particular tests you’re using. Oh, guess what? There’s no diversity on the team. Try it with stuff from diverse teams. Oh, all of a sudden, it works. So this shouldn’t be a surprise, right?

So there’s a way that an initial bias or initial exclusion compounds itself over and over again. So it’s absolutely critical not to let it get started. There’s another level of this which is pretty dark. I’ll do my best to explain this really quickly.

The way the algorithms work on social media and in general with what we call advertising, the behavior modification loop business plan, is you have to give people stimulus from whatever they have, whether it’s a social media feed or whatever, that keeps them engaged.

This is engagement, right? So what keeps you engaged? Unfortunately, negative emotions, fear, anxiety, anger, these things are more engaging, more immediately and more persistently engaging, okay?

So I will ask you a question. Why is it that there have been so many phenomenon where people use social media and it seems to be creating positive social change and then, just like a year later, there’s this backlash that’s worse than anybody could have imagined?

I can mention a few examples. The Arab Spring was the first prominent example, but I also want to mention Black Lives Matter. So what’s going on is that the people — there’s sort of two levels to what’s going on.

On the surface level, which is what people see, these things are incredibly beautiful. Like I personally found Black Lives Matter to be incredibly moving and I think like Black Twitter is like major literature, like it’s literature for the ages. Really, it’s astonishingly beautiful.

But the thing is, behind the scenes, there’s a completely different game going on that has nothing to do with any of that. What’s happening is that all the content, the energy, the fuel that’s coming in from movements of this kind has to be processed in such a way as to generate engagement and profits for the machine.

So there’s no like evil genius doing this. It’s just algorithmic, so it gets processed to be turned into negative emotions for somebody because that’s the most efficient way to use the fuel.

So then what you have is this primed thing where it’s somehow packaged in order to irritate as many people as possible and, because the negative emotions are more powerful, the backlash that arises which would probably not have been stimulated otherwise is even greater than the initial thing.

So the reaction online from something like Black Lives Matter will always be more intense than Black Lives Matter…

Tavis: Stop [laugh]. I need more time and I’m out of time right now [laugh]. The new book — hold on. Come back tomorrow night. I promise. I got a few more questions. This stuff’s getting real good now. I’ll still have my shoes off if you come back tomorrow night [laugh].

The book is called “Dawn of the New Everything: Encounters With Reality and Virtual Reality”. Mr. Lanier will be back with us again tomorrow night. That’s our show for tonight. Thanks for watching [cough]. Come back tomorrow night. I’ll all choked up. And keep the faith until then.

Announcer: For more information on today’s show, visit Tavis Smiley at

[Walmart Sponsor Ad]

Announcer: And by contributions to your PBS station from viewers like you. Thank you.

Last modified: December 4, 2017 at 1:17 pm