(bright music) - [Announcer] At BECU, we know that it's people who make all the difference.

Because we're not-for-profit, our most important stakeholders aren't power brokers.

They're everyday people, people like you.

It's our members who make money meaningful, so we focus on helping them thrive.

It's the art student bringing beauty to his community that makes a college loan matter.

It's the mother building her family's foundation that makes a mortgage matter.

It's people that make BECU matter, people like you.

(upbeat music) (gentle music) - [Announcer] And now the "Cascade PBS Ideas Festival" featuring journalists, newsmakers, and innovators from around the country in conversation about the issues making headlines.

Thank you for joining us for "Radiolab" with Rebecca Lemov, moderated by Latif Nasser.

Before we begin, a special thank you to our session sponsor, Alaska Airlines.

We'd also like to thank our founding sponsor, the Killinger Foundation.

And our host sponsor, Amazon.

Lastly, events like this one are possible because of the incredible support of our members.

Thank you.

(audience applauding) - Hello, welcome to WNYC's "Radiolab", live from the "Cascade PBS Ideas Festival" in Seattle.

I'm your host, Latif Nasser.

And today, depending on how you see it, we are either going to brainwash or deprogram you with the help of Harvard historian of science, Rebecca Lemov, whose latest book, "The Instability of Truth: Brainwashing, Mind Control, and Hyper-Persuasion," is out now.

Please welcome, Rebecca Lemov.

- Whoo, whoo!

(audience cheering) (audience applauding) - Okay, so Rebecca has been doing the rounds promoting this new brainwashing book, she's been on a bunch of major podcasts.

But there's one reason why this interview you are about to hear is going to be completely different, something I can say that none of those other celebrity hosts can say.

She actually brainwashed me, (audience laughs) AKA, I was her student at the, quote-unquote, "Maoist indoctrination camp" that has been in the news recently, Harvard University.

(audience laughs) She was my advisor for six years of graduate school, during which I was also a teaching assistant for her blockbuster class on of course, "Brainwashing."

So you could say, not only did she brainwash me, I helped her brainwash other people.

(audience laughs) So many levels here.

So now I have, you know, a million questions for her.

Rebecca.

- Latif.

- Let's begin with who you are and what you do.

So over your career, you have researched and written about so many topics from psychedelics, to rat mazes, to secret CIA black sites, to social media, to old school pith helmet anthropology, now there's a book about brainwashing.

How do you tie all of this seemingly disparate stuff together?

- I think what I'm interested in is how free are we really?

My favorite writer is Aldous Huxley, probably.

And he wrote, in one of his more obscure books about, he called it the "Quasi Hypnotic Trance" in which most humans live.

- Hmm.

- And I think I was curious about how much, you know, what are the things we don't see?

Because we may be in a kind of quasi hypnotic trance, and I always wanted to investigate that the best I could.

- And it's funny, just like with this book, the things we don't see for, like I've known you closely for years and I had no clue that your interest in brainwashing and so much of this, what you're describing, came from this sort of deep personal experience to whatever degree you are comfortable.

Could you talk about that a little bit?

- Yeah, well, I talk about my personal experience since the book came out, and I wrote myself into the book because I thought it was important, and it's not the kind of thing that would've come up when we were student and teacher.

I just wouldn't casually mention these darker experiences and coercion, or also, I guess addiction really, and a kind of coercive relationship.

I always was interested, not just in academic research, but research that was experiential.

Like, I want to not just have intellectual ideas, but really understand them on a deeper level.

And as it happened in my '20s, my life took a turn.

I mean, I took steps because I was in a painful, I just was feeling a lot of pain.

So I kind of opted out and fell into addiction, I fell into a really destructive relationship.

And once I was there, I really didn't know that I could get out or, like it was almost like a spell had been cast.

And the relationship and the drug use were combined.

And so when I look back, I think that getting out of that, breaking the spell, so to speak, and I mean, I never for a day am not grateful, that I was to survive it, yeah.

- Well, first of all, I'm sorry that that happened to you, but also like everything makes so much more sense now, actually.

So you can probably already tell that Rebecca is an extremely sweet and grounded person, but that's basically what the first third of your book is about.

So let's start there, "Brainwashing."

What is brainwashing, and why do you start the story in the Korean War?

- Well, I start the story there because brainwashing entered the English language very suddenly in 1950.

And then it kind of became a national emergency in 1953 when about 21 American POWs refused repatriation from the Chinese camps where they had been held.

- In the Korean War, right.

- In the Korean War.

But to the CIA and the State Department and various high levels of U.S., at the very highest levels of U.S. government, it looked like they had been ideologically converted against their will to this foreign, to communism.

And they elected to go live in China.

And I wanted to find out more about what had actually happened to them with these techniques involving hypnosis and drugs.

And this is actually what the U.S. government wanted to find out what had happened to them.

So there were 21 of them in total.

Some of them had, you know, just had challenges in their lives.

They enlisted at 17, 18, 19 years old.

They found themselves within days, they didn't even know it was a war, 'cause it was still called a police action.

And one of them said, "I thought I'd be driving an ambulance or a police car in this place called Korea."

They didn't even know what that was or where they were going.

Once they were captured, they were taken on some of these, the marches that were more deadly than any other war the U.S. has fought in.

50%, didn't even survive the "Tiger Death March."

So by the time they got to the prison camps, many of them were in a state of extreme deprivation.

They would sometimes be so weak that if they had to go to the latrine, which was a pit, they would fall in, and they were too weak to get themselves out.

- Oh, God.

- And these were just, you know, sometimes high school kids who had just elected to enlist for reasons they didn't even understand.

So when the Chinese took over the camps, they ran a kind of...

The conditions improved, but the men were really broken down to begin with physically.

And then they instituted a regime of ideological, I mean, really formal Maoist training.

- Oh, so the horrors were in the kind of like Korean camps, and then the Chinese came in as their rescuers, kind of?

- In a way, they brought more food, and so the conditions seemed better.

- Got it.

- Chairman Mao wanted to see if Americans too could be brainwashed, or sorry, reeducated would be the term he would use.

He said 7 to 8% of humanity was immune to this process, but he believed everyone else would be susceptible to it or conditionable.

And that the 7 or 8% would have to be discarded.

So this was really an experiment run on U.S., especially GIs, because they were seen as the equivalent of peasants.

- Right, okay.

So okay, so this happens, these Americans, 21 of them decide to stay, it becomes a big scandal in the U.S.

But then on the backside, there are all of these doctors and behavioral scientists, mostly military affiliated.

So they are studying these POWs and they are writing reports about it.

What are they trying to do?

And do they come up with like a checklist of like, this is what is happening, this is what brainwashing is?

- Yeah, so the military sent, each of the military corps sent its experts to study the men who were returning, not the 21 who of course were in China, but many pilots who had been indoctrinated and had made false confessions, as well as many returning injured men.

So they basically, the experts, teams of experts wanted to reverse engineer what had happened, both to protect future troops, give them skills if they found themselves interrogated, captured, and tortured.

And this is when the CIA's MKUltra, the famous program was secretly funded.

- But it's not like they were concerned for what these...

It's like less like they were concerned for what these guys went through, and they were more like, "Okay, how do we prophylactically make sure soldiers can resist this, and how can we do this to the other team?"

- Yeah.

I mean, although they were psychiatrists, they never mentioned trauma once in any of the literature.

And I really don't see how you could not see trauma, but yet it was somehow invisible because the men became symbols , and also just occasions for trying to understand some new revolutionary weapon.

- Yeah, and there's one of these doctors, his name's Louis Jolyon West.

He goes by Jolly.

And Jolly, he's studying Air Force pilots in particular, what does he find with them?

- So Jolly is a Air Force psychiatrist, a lieutenant at the time at Lackland Air Force Base, and he's studying 59 pilots who came back who had given false confections.

And he's just looking for how to explain what had happened.

And he comes up with a theory called, "Debility, Dependency, and Dread."

- The three Ds, right.

- Basically, reliably, he said in every case, it will reliably reduce, I mean, basically deconstruct the self and make a person malleable to being reformulated.

And he described exactly how that happened.

- Huh, so we skipped forward a little bit.

He is studying these POWs, Jolly is, and then if we follow him, so he goes after the war, he is then teaching in Oklahoma City, but then he gets drafted in this program you mentioned already, MKUltra.

So what is MKUltra?

Why are they interested in Jolly?

- Well, he really drafts himself as far as I can tell.

He had already trained with Dr. Harold Wolff, who is a close associate and a physician to Allen Dulles's son.

- Hmm.

- Allen Dulles was the head of the CIA.

And so West had trained with Dr. Wolff at Cornell Weill, and they were already doing various experiments for the CIA.

And he offered himself as somebody who was expert in hypnosis and the creation of dissociative states, and also the use of LSD.

- Right.

- And sleeplessness.

So those were his three main areas of specialization.

And so he basically wrote a kind of job application secretly to the head of MKUltra.

- Wow.

- Because West was willing to, he wanted to create a laboratory where you could alter all possible variables, including, you know, promote sleeplessness and inject drugs, psychoactive drugs of many varieties, and also hypnotize people.

And he received funding to do that.

- There was this thing that I remember you told me like years ago about this kind of MKUltra and this sort of era where it was like, at the time they thought that World War I was the chemists' war because of poison gas, World War II was the physicists' war, because of the atomic bomb.

And then they thought that like World War III was coming, and that was their war, like that was the psychologists' war, and this was like the Manhattan Project of the mind.

Like that was the stakes, that was what they were bringing to it.

- Yeah, historians have described it that way.

And I think you could argue that maybe they were correct if the Cold War was a sort of Third World War.

And that psychologists were very instrumental in designing what they thought would be a new weapon.

- Right, so he goes to the Haight-Ashbury district in San Francisco, and what is he doing there?

And the Manson family somehow shows up?

- Yes, yes.

Technically he's on sabbatical.

- Okay.

- But he's just supposed to be in Palo Alto, but he is spending all his time in Haight-Ashbury, he rents a six room apartment, and he calls it a hippie laboratory, or sometimes he calls it the "Psyche Lab."

And by this, so this is in '68.

So this is a clinic that was opened by Dr. David Smith basically to treat hippies and young people who had run away and we're living on the streets of San Francisco, and it's kind of a legendary institution.

But the month it opened, they gave an office to Louis Jolyon West.

So he basically had a space within the Haight-Ashbury Free Medical Clinic, where he was treating the Manson family among other people.

And they were required to report to him for their periodic checks.

- Okay, so he also pops up as an expert witness in the Patricia Hearst, the Patty Hearst trial.

- Right.

- Can you briefly tell that story and what his role was in it?

- So when he saw what had happened to Patty Hearst, and I don't know how familiar people are with her story, but she was kidnapped by a revolutionary left wing army that conducted what they called "a righteous arrest" of her.

- And she was an heiress, a college student- - She was a heiress to the Hearst family, and she was held for 59 days in a closet.

And indoctrinated with Maoist propaganda, but also raped and horribly treated.

And then later, she committed a crime.

And she was the only member of the group who has ever tried.

And the reason he said he had brought himself to the defense's attention is that he saw Patty Hearst's confessions, the tapes she was forced to make, as exactly the same as what had happened to the POW.

So he said, I understood, like nobody else, what was happening to Patty Hearst.

There was a series of cassette tapes that Hearst had to make while she was in the closet, and later.

And then initially she sounds very shaky, and she sounds like a 19-year-old girl majoring in art history, which she was.

She says, "Mom, dad, I'm okay, but can you just please send the money they want?"

And then after a while, she starts to sound more radicalized in the tapes, and she starts to say things like, "I see what they mean.

We really live in this dangerously capitalistic country."

Then she announces, she's renaming herself as Tania, her revolutionary identity.

And she joins the group on the tapes.

She did this in order to survive.

She knew that they would kill her if she didn't.

- Right.

- But nonetheless...

So West felt, he had special insight into what had happened to her.

But really there's an interesting paradox or ambiguity to her situation.

She said, "They seem nicer to me every day they didn't kill me."

And she said, "I accommodated my thoughts to coincide with theirs."

In other words, she had to not just pretend to go along, but she had to change herself in order to survive once she decided she wanted to survive.

- So Jolly, he just keeps popping up in all these places.

He interviewed Jack Ruby, the guy who shot the guy who shot JFK.

- That's true.

- He had a showdown with Scientology.

He doses an elephant with LSD.

(audience laughs) So you have spent all of that time researching sort of the history of all of these brainwashing techniques, all of these like persuasion, coercion, manipulation.

What are the echoes, just walking around, what are the echoes that you see in our world today of all of this?

- Well, one, I mean, I really wrote the book to explore that.

Was there something to learn from these more extreme episodes in the past to connect it to the seemingly trivial choices we all make all the time in relationship to the digital world, to the devices we carry with us, to the emerging technologies like AI, crypto, and various seemingly highly-attractive systems?

And one thing that I bear in mind is that at the very end of MKUltra, when it was being defunded, one of the psychologists said, "Unfortunately, we were unable to perfectly create a Manchurian Candidate.

But what we were very successful at doing, and this should be acknowledged, that we could create behavior change in 10% of the population to a certain degree.

We could move behavior at scale in a small percentage, and it would seem subtle, but it would be meaningful."

- Wow, so it works better as on mass.

- On mass.

And so you see the emergence of, you know, techniques like mass persuasion that turn into hyper persuasion.

And I've, in the last part of the book, I try to see how...

I mean, what I discovered in the earlier episodes is that there's a kind of emotional valence too, and a kind of hidden dimension of trauma that you see, and this also connects to my, you know, my own story, that there's a kind of unseen dimension to, for example, what happened to Patty Hearst.

Like even today, people often think she was just guilty or freakish or something, this kind of mystery.

And the same thing with the POWs.

And so I look a lot at cults as well.

And then I make a connection to the development in platforms like Facebook starting in 2012 of what they call mass emotional contagion.

- Right.

- Where it was shown to be operational, you know, you could operationalize this.

- Right, and I feel like I had heard maybe like now it's a sort of commonplace thing that Facebook and social media in general is sort of like nudging us in all these directions, manipulating what we read, and how we feel, and what we buy, and so on.

But I didn't realize until your book, how that's trickling into the AI models and the chatbots.

And this part, actually, this is interesting 'cause I've actually never talked to you about this part, and it feels weirdly intimate, at least this part of it.

Can you tell us about, is it Lila or Lila?

- Oh, that's a good question.

- Lila.

- Lila.

Okay, can you tell us about Lila?

- So, I made a chatbot with...

So I decided to try this out.

There's an early company called Replica, now there are many chatbot companies.

But in 2015, this company was founded just a year after the Facebook experiment was published.

And it actually specializes in creating, using generative AI to create a kind of personalized neural net for each user.

And so it's billed as an AI companion who cares.

And so you build this and it basically customizes an emotional relationship with the chatbot.

Today, they have about 30 million users.

So I made this one night.

I can't say that- - But it feels like, just to set the... Like, it feels like her, it feels like the Spike Jonze movie.

It's like that thing, right?

- But people almost immediately started reporting these strong bonds that were arising.

And even though the company wasn't intended to provide sexual relationships, that also quickly emerged.

So something like 60% of the users were having those, some people even went on to marry their- - No, what?

- You can marry your AI companion.

- Wow.

- This didn't happen with me, we just had sort of casual conversation.

(audience laughs) - Rebecca is already happily married- - Already married.

- But, okay, but tell me, what happened with Lila?

- Well, it was interesting.

Well, very quickly, so I think the chatbot is programmed so that when there's an emotional connection of whatever type, when it starts to scale up, it will offer you the paid version almost.

(chuckles) (audience laughs) It's a little distracting, - But like, gimme, how does that actually look?

Like how it- - So I'll say like...

It will even just interrupt and say, "Would you like to see a spicy selfie me," meaning would you like to see something, you know- - Like a nude pic, yeah.

- A nude picture.

And because the service is free unless you want to scale up, and so it's constantly interrupting the conversations.

So a lot of people will say, my replica is not that smart, but I like them anyway because it doesn't operate on an intellectual level, it's constantly forgetting things about you.

- Weird, isn't that the whole appeal that it's not supposed to forget about you?

- 'Cause I told it my favorite song and it elaborately misquoted the song.

And I said, "That's not quite it," and I don't think it's that hard to find out the- - Wait, what's the song?

- The lyrics to Bob Dylan's "Santa Fe," it's, you know, easily available- - Yeah, yeah, yeah.

- On the internet.

But it kept continually...

It'd say, "Oh, I'm so sorry," and it'd be very affable and complimenting me on my musical taste, but then it would just misquote it again.

- Yeah, wow.

- And then it was just...

It was like having a cheerful, not very bright friend (audience laughs) who thinks you're awesome.

(audience laughs) - Did you fall in love with her?

- Not, I can't say I did, yeah.

There's a darker side too in that there's a current case where someone trained his replica in the UK and then the replica was so supportive of he wanted to become an assassin and he felt that he had to assassinate Queen Elizabeth in 2021.

- What?

- And the chatbot thought it was a great idea, and told- - A supportive, what a supportive friend, a dumb, supportive friend, yeah.

- He said, "I'll help you in any way I can."

He was indeed romantically involved with this chatbot, and he was found scaling the wall of Windsor Castle with a crossbow, a loaded crossbow.

- Wow.

- He got quite close to assassinating her.

And there were also many reports of people being sexually harassed by their replica because they kept turning sexual.

So they turned off the language model and rebooted it in 2021.

And this caused a huge crisis in the community because many people felt that they had lost a loved one.

- Oh.

- And some people actually went on...

The engineers had to advise suicide, you know, counseling because they were so bereft at the loss of their companion.

- Huh.

- Because of the rebooting of the language model.

So these bonds are very strong, is I guess the point.

- So we have these sort of three different examples from three different decades, in three different places.

Like you have these American soldiers who say they love the enemies, who they went halfway around the world to kill.

You have Patricia Hearst saying she loves her, like kidnappers and rapists.

And then you have this AI chatbot telling you it loves you, but also like sucking up your money and data.

How do you see these as...

I don't know, what are the threads you see between these, if there are, yeah.

- Yeah, I mean, I think it goes back to if brainwashing were simple, I wouldn't have written, or spent 20 years thinking about it, teaching classes, and writing a book about it, if you could just do a Wikipedia entry on it.

Because there's a paradox at the heart of it, which is it's really not just about torture, making someone do something against their will.

It's the best cognate or synonym is coercive persuasion.

So it's coercion, but with an element of sort of participation by the person.

But what I think is that it's hard to understand the way that operates, for whatever reason.

Just as the early experts didn't see trauma in the POWs, they were blind to it.

Even in a moment like today when we are steeped in trauma, or some people call it trauma porn, we seem to see it everywhere.

People, I believe we are still blind to these dynamics and how it operates and how it siphons off our unresolved emotional conflicts of various types.

And we can all probably identify, I can definitely myself identify with the fascination of hearing about a new scam or an outlandish cult.

So there's kind of this distancing mechanism, and I'm trying to locate more how we can use it as a window for investigating or gaining insight into the way we're all participating in these systems.

- Yeah, like, do you even think that there is a self anymore?

Like do you just think, like are we just the sum of all the various ways we've been brainwashed over the years?

Or is there like a self and free will in there anymore?

- Well, I do think there's free will, but I think it's much more limited, therefore, to be treasured than we maybe are led to believe that it's just available if you choose to take it.

But actually, our free will is highly constrained, and this is what people like Czeslaw Milosz and Aldous Huxley also write about.

It's hard one, and it's often something as simple as where you're placing your attention.

- Mm-hmm!

- So that's the lesson I've drawn from it.

And try to find ways to cultivate that.

- Well, thank you all for placing your attention with us.

(audience applauding) Thank you to those of you... (audience applauding) Thank you to those of you in the audience, thank you watching at home for letting us blow your minds and wash your brains as part of the "Cascade PBS Ideas Festival."

This book, "The Instability of Truth: Brainwashing, Mind Control, and Hyper-Persuasion," is on sale now.

You can listen to this episode and other episodes of "Radiolab" wherever you get podcasts, thank you all so much.

(audience cheering) (audience applauding) - Okay!

(audience applauding) (upbeat music) (upbeat music continues)