From WXXI news.

This is Connections.

I'm Evan Dawson.

Our connection this hour is made inside the mind of a machine.

Or is that not the right word?

Can a machine have a mind?

Can gain consciousness?

Should it have rights?

Are we powerless to stop the rise of machines?

My guest this hour has spent much of his career working on astrobiology.

He examines the broader universe for signs of civilizational life.

Now he's turning his attention to the question of artificial intelligence on this planet.

And he thinks a lot of the discourse is to put it in scientific, technical terms, stupid.

Adam Frank does not buy the idea that there is nothing the human race can do to stop any possible bad outcomes from unfettered artificial intelligence.

But he also doesn't believe that we are creating an alien race on Earth.

He says we should start thinking very differently about what AI is and possible problems with it.

So let's hear it.

What are we getting wrong and how can we get it right?

Doctor Adam Frank is an author, of course, of numerous books.

He's the Helen F and Fred H. Gowen professor in the Department of Physics and Astronomy astronomy at the University of Rochester.

Welcome back to the program.

Always a pleasure.

And to be clear.

Not in a simulation.

It's still not.

Still not still not relevant to this.

We'll talk about that.

But it's throughout the whole simulation.

But still.

No, no, no okay.

Not even close.

So the AI discourse has exploded in recent weeks.

And in your outstanding newsletter, which people should be getting, you want to just plug it real quick here.

Every man's universe, every man's universe is Adams's Twitter today.

And you recently shared some thoughts there.

And I wanted to bring you here to.

First, let's kind of take an overview.

So as you see, the AI discourse exploding.

What is going through your mind?

Well, a couple of things.

First off is that these machines are amazing.

Like I use I use Claude extensively in my research.

I think that, you know, there's ways in which one can argue this is one of the most significant technological advances that humanity has ever made.

But at the same time, looking at the way this is being pushed on us in some sense being shoved down our throats, I think there is a way in which the discourse happens that is really to the advantage of these companies, that I think we as citizens have a right and need to really push back on.

Okay.

So we're going to work through what that means here.

But at the start of it, what does it mean that it's being shoved down our throats?

Why do you feel like it's being imposed as opposed to some other kind of advancement or rollout here?

Yeah, right.

I mean, you know, look at the cell phone and social media the way in which, you know, we ended up there were smartphones.

You know, smartphones were introduced and within 5 or 6 years, you can't get through your day without a smartphone.

Like it's very hard now to live without smartphones and everything they.

So that's the way in which this technology was rolled out and it was put into our lives.

And in the end, you know, we didn't really have much choice about whether we wanted to use them right now with cell phones.

Yeah.

There were lots of great things about cell phones.

There's lots of not great things about cell phones.

But with AI, you can see this technology being sort of being that it's been decided by, you know, a few players and the markets or at least the parts of the market that want this, you know, see this as being advantageous, that, you know, this is now a huge part of our economy.

You know, the market where, you know, there's this push to have these technologies in everything, every to make it indispensable, to make it indispensable.

And these technologies, clearly, as we've seen, you know, it's one of the first important things to understand.

These are not superintelligence is sentient.

Superintelligence is these are a software product, and they have real problems.

They're amazing.

But they also have real problems.

And, you know, the, the the, the tech bros and their ilk and their, you know, the, the, the finances backing them because you so much money sloshing around in tech.

And we can talk about sort of how that happened.

But that yeah, there's the sort of move to have this technology everywhere.

And, you know, the thing, it's hard not to sort of see that like, well, one reason they want to have this everywhere is so they can fire everybody, right?

I mean, the two arguments they seem to have are like, well, this technology's it's going to fire everybody or it's going to destroy humanity.

What's not to like?

You know, it's it's it's a strange marketing pitch.

But on the the part of the pivot that I'm seeing comes from last week, we played a couple of pieces of sound when we had Tim Wu on the program.

Tim has long been working on tech regulation, monopolistic practices in technology, and the ways that actually creates worse products for consumers.

So if you missed that, that was a really interesting conversation.

One of the pieces of sound was from Mark Zuckerberg, of course, CEO of meta.

And he says, in five years, everybody's going to have an AI agent basically running their calendar and setting up their lives for them.

Right?

So it's like you're not gonna be able to live without it.

It's going to do all the stuff for you.

And the other piece of sound was RFK Jr.

At a major event.

He said he had just talked to Sam Altman, the CEO of OpenAI.

And Sam Altman told RFK Jr it's malpractice for any doctor now to write a prescription or make any diagnosis without consulting AI first, right?

Which is ridiculous.

It goes along with your point that they want to be seen as you cannot live without us now.

Right.

And it is only going to become more entrenched in your life whether you want it or not.

Right.

And as we've seen from, you know, these, you know, the hugging face instant all these break these software systems are like seriously broken.

If this was like a company that was building autopilots for aircraft and they were behaving as badly, these companies would be, they'd be shut down, right?

These their product is literally doing illegal things.

And the whole pitch for like, well, these are sentient.

They're, you know, they're conscious is part of the reason this argument's being made so that they, the companies will not be liable for their broken products.

Right.

The idea is like and you hear them saying, it's like, well, our model did these amazing things.

You know, it's we don't understand why.

We don't understand why it's doing.

It's like, it's not it's not your model.

It's your product, and your product has problems.

So recently there was the meeting with the pope where AI leaders are trying to convince them that, you know, these things should be treated as as sentient with souls or as as true autonomous agents.

And I just think this is a this is a public relations pitch because we also have to remember, these companies are burning through money like no one's ever seen this kind of, you know, this kind of, capital being just, you know, set on fire.

And there's real problems for them.

There is really not an existing business model for AI that pays for it, right?

Even sweat like it's true.

It's clear that AI is going to be great for writing software, right?

But even if you gave the entire software industry to AI, that's still not enough to pay for how much it cost to train and produce these model.

It's not even going to the ethical issues about training and producing these models.

So it is not clear how this ever makes the money back that it's taken to build them.

So let me take one big step back and I will.

I want to make sure that I'm understanding part of your premise.

The first premise I'm hearing from you is think of this as software products.

Do not think about this as some sort of, alien life form that's been created that is purely autonomous and not controllable.

It's a software product, and in many cases it is a flawed product that is causing real harms, and it is capable of greater harms.

Yes, absolutely.

Okay.

We're going to talk coming up here about if we changed our mindset and Congress and the American public and frankly, the world public thought about it in those terms.

What would we then do as opposed to thinking of it as this emerging intelligence?

Right.

So help me understand what you think about intelligence, because what concerns me the most is, is that a specific scenario of human extinction?

It's the idea that we are intentionally bringing forth a greater intelligence than our own.

And we as humans have never existed alongside an intelligence that dwarfs our own.

We've always been top of the intelligence chain.

Where am I going wrong there?

Yeah.

So this, that that, that argument.

So this is something, you know, because you can ask like, who is this guy who's an astrophysicist?

Why is he talking about this?

And that's a, you know, reasonable, you know, reasonable thing to ask, but especially when it comes to the intelligence thing.

So, you know, my work in astrobiology, you know, around ten years ago led me to start asking questions about life itself.

Like, what is it that makes life different from a rock?

You got a cell.

You know, it's doing all these amazing things and you got a rock.

Rock just sits there.

What is actually going on between them?

And then you can think about, you know, for me, I would argue that cells have some degree of sentience.

There's a feeling of being alive that a cell has.

It's not, you know, conscious.

It's not thinking about itself.

But then, you know, you climb up the curve and clearly mammals, you know, there's something going on with them.

Plants.

There's arguments, good arguments.

Now, the plants have some degree of consciousness in terms of agency.

So I started really looking at that problem.

And just recently, the University of Rochester is leading this effort.

We just got a four and a half or $4.2 million grant from the Templeton Foundation to purpose.

The whole idea is to study the physics of agency or the physics of a genetic intelligence.

Define agency.

Well, that's kind of, you know what?

We're going the agency, really, the word you should use is autonomous agency agencies are an agent.

Is a system, you know, that has the particular kind of organization that is built in such a way that it has things that matter to it, right?

It has you know, when you say like, oh, I'm doing this for your sake, it has a sake, right?

You know, this table doesn't have a steak.

Nothing matters to this table.

It doesn't have its own goals.

It doesn't have its own needs.

Right?

A thermostat on a wall will change the temperature, you know, if the temperature goes up or down.

But that's because I built it to do that.

I have the need, right?

Whereas a dog will go, you know, from the sun into the shade because of his own needs.

And now, you know, this is a big open question.

Science has had a very difficult, time over history dealing with this.

But now I think we're beginning to develop the scientific way of understanding this, or at least that it's both science and philosophy.

And what when you understand, as you start to deal with the question of agency and autonomy to be, you know, to have your own needs, to have your own goals, what you really see is that these machines are nothing like that.

Right?

And so this is a really nice way of putting it.

Intelligence is not solving problems.

Intelligence is knowing what problem to solve.

What's the relevant problem to solve?

Machines can't do that.

You always every.

You're the most advanced l'Allemagne.

You still are the one giving it the question.

You are giving it the goal.

And you know what it'll do?

Because it has read everything humanity has ever made.

It will, you know, refer to that.

And when I say refer to that statistically, well, it will then, you know, make moves.

But it's always your goals.

It's a tool okay.

So I want to push on this a little.

Sure.

But the Templeton grant that you're getting is that in part to study whether AI has.

Yeah.

That's going to be that's the we will naturally come up.

So your critics are going to say, I heard you on Connections and you already drew a conclusion.

So why are you investigating this?

But but as you and I've talked about, sometimes science is starting off with a premise and then attacking it.

So you're willing to attack your own priors on this?

I am, yeah.

And listen, you know, I am also open to the fact that I may be wrong, right?

I may be wrong about this, because that's what science, where science takes us.

You're giving me your current scientific analysis?

I'm giving my current scientific.

It's also a philosophical analysis, because one of the interesting things about this grant is we're starting with philosophy, right?

We actually have some philosophers on board because the problem and this is the problem, I think with much of the discourse in the scientific discourse about AI is that, you know, the tech bros, they're philosophical, they're quite philosophically naive.

Right?

And if you're not, you know, they'll say things about consciousness that it's like, wow, man, did you take like an intro philosophy of mind?

Course.

Like, you know, these there's subtle issues here and you're like steamrolling over them.

So what you have to do if you want to take it on scientifically, if you have to, you have to start with a philosophy and you have to ask yourself, okay, I want to know about agency.

What is the right question?

What is agency?

How do I even pose the question so that I can get a scientific answer out?

Because what often happens, and I think you see this a lot with AI research, is people they they pose the question for the science in a way that's already wrong.

They get an answer.

They're like, look, see, we showed that, you know, machines are intelligent.

It's like, dude, the the you're not even you were aimed in the wrong direction from the get go.

So this project starts philosophically.

We ask what is information?

What do you mean by information.

Right.

What do you mean by information processing?

We ask, what do you mean by agency on autonomy really work that out and then you can start trying to build a model or do mathematics.

You know about it.

I have to have a brief interjection to say that when you use the term tech bros, that's a term that I used recently on Twitter X, and Perry Metzger, who has worked in tech for a long time, told me that that is a form of bigotry using the term tech bros and that I ought not use it.

I responded by saying, I actually think there's a very specific definition in application.

Yeah.

What do you mean when you say I, me, I mean the richest of the rich?

You know, this handful of men, you know, who are who dominate this industry, who have wealth that we allowed wealth to accumulate at this scale for these men.

You know, these guys, you know, it is will, I think is one of the greatest mistakes in the organization of society.

And these are, you know, there's like, you know, we can name them, right?

You know, whatever.

Yeah, I'll give you ten or whatever.

And that, you know, they pretty much control the dialog and the debate and the action that's happening on this.

So that's what we mean.

Like it's not everybody in tech.

There's lots of people in tech who are quite sophisticated of course, of things.

And I appreciate that clarification.

I generally talk about the people who literally have their hands on power.

That's what I'm talking about.

Okay.

So when we come back from this only break, I want to get back to hugging face, because when doctor Frank tells us that at the moment, his view of AI is it is not intelligent in the way that we are thinking that it is accomplishing things, but it's not necessarily intelligent.

What does it mean to solve a 100 year old math equation?

What does it mean to act in a way, in the hugging face that goes outside of its own instructions?

Does that represent a kind of autonomy, or is that something else?

We'll take feedback from the audience if you want to share questions, comments on AI intelligence and sentience.

It's Connections at WXXI.

That's the email address.

It's 844295 talk toll free 84429582552636.

I have a call from Rochester.

2639994.

Support for your public radio station comes from our members and from Mary Cariola Center.

Supporting residents to become active members of the community, from developing life skills to gaining independence.

Mary Carey, Yola center Transforming Lives of People with disabilities more online at Mary Carey, ola.org.

Democrats will have a hard time winning back the Senate next month without main.

So I spent the last few days driving across the state talking to voters.

There's a lot of things happening up here, and just all people forget about us up here.

So this is a business of people with grit.

And right now, the tough time.

My electric bill is so high, it's higher than my mortgage payment.

I'm Leila Fadel, and that's tomorrow on Morning Edition.

From NPR news.

Tomorrow morning at five.

This is Connections.

I'm Evan Dawson.

Imagine that a gun manufacturer created a firearm.

And every time you tried to pull the trigger, something different would happen.

Sometimes a single bullet would come through the chamber, as you expect.

Sometimes for no discernible reason.

Three different bullets would come out flying in different directions.

Not at the target you're aiming at.

If that happened, there is no way that manufacturer would be able to keep putting that product out in the market.

And if anybody was hurt, there would probably be lawsuits out the wazoo and that company would probably be out of business.

That's a little bit what Adam Franken in my interpretation, is saying about eye products.

It is a piece of software and already we're seeing it's a flawed piece of software.

Now it's not just one piece of software.

It's a lot of different things.

But we're talking about it like it's some sort of this new machine.

God, that has its own ideas and we can't stop it no matter what we do.

He says baloney.

It's a piece of software.

If it hurts people, there should be liability.

There should be an immediate move to regulate like any other piece of software.

First of all, is that a fair description?

I don't know if I'd use the word baloney because, you know, I'm not really sure that's part of my lexicon, but I do with emphasis.

Baloney.

Otherwise I would.

I don't want to risk it.

Okay, okay.

I mean, the gun thing, though, right?

Like that was good.

I think that's a very good analogy.

Yeah.

I mean, there's no way that product would stay in the market, right?

And I do want to say that I think AI is amazing.

Like I'm not like, you know, this technology.

Like I said, it is transformed by my you tell us we've got a category error going on.

Right?

Right.

It's a tool.

It's an amazing tool.

It's also a flawed tool that you should be very careful about.

How about you allow it into society?

Okay, so back to the way that we think about it's intelligence.

Let's start with the math problems.

It solves a century old math problem that human beings had made a lot of progress on, but hadn't solved.

And to you, that's not intelligence.

That's not autonomy.

That's.

Well, first of all, who gave it the problem?

Who decided to solve that problem wasn't the computer, it wasn't the Lem.

It was, you know, people who were like, hey, here's a 100 year old math problem that we've all worried about for a long time.

The other thing also, we have to remember, like there was a lot of controversy about the, the, the fluid dynamics thing because there was a math researcher who was using Claude and was pushing that forward, and they stole his work.

They saw, this is the thing about Claude, now that we all have to be a little careful of or this was OpenAI, but, you know, they took the work that he was doing with the Lem and then pushed it themselves.

And also, I got to say, math.

I'm not surprised that you're going to be able to use, AI machines to push, you know, math, because math is a formal system.

It's got like, how do you, you know, like we've all done proofs in math.

At some point you start with a bunch of axioms and then you turn the crank, Franchitti, crank and crank, crank, crank, and you get to a proof you get.

And that's what is actually happening here.

Again, it's amazing, but it's you're living within a bounded world.

These machines live in bounded worlds.

We don't live in a bounded world.

So there's literally a name for this is called Small Worlds.

You know, all mathematics, all, form what we call formal systems.

Wherever you can define predefine your possible state.

It's like a chess board.

Every position on a chess board.

You already know what it is, and you know the possible moves.

You already know.

That's a small world.

Living things live in a large world.

There are unknown unknowns.

No machine that we've ever built knows how to deal with that, and they fail spectacularly.

This is a problem in cognitive science called relevance realization.

We have no problem recognizing what's important to us.

We walk into a room.

We can instantly decide, oh, that's important to me.

That's important.

Me, I can ignore that computer can't do that right now.

These machines can't do that.

So that's the difference between just solving a problem.

Problem that somebody gave to you.

That's cleverness.

You know, there's a kind of intelligence there.

But I really, you know, versus knowing what problem to solve.

What is the important problem to solve that is agency agents do that because agents have meaning.

Things matter to major agents.

Okay.

So now let's talk about hugging face.

So here's a case where nearly 2000 agents figured out a way to break the rules or, quote unquote, escape the sandbox to which they were ostensibly assigned.

I, as I understand it, and on their own message boards, they're sharing advice for one another on how to rig a testing system, even on whether they should be like informing on each other.

If this is unethical.

And six out of the 1700 AI agents raise ethical objections and a couple of them decide were abstaining from participating.

But none of I'm using anthropomorphize like you are.

So that's got to be.

We're going to talk about this.

Nobody in there okay okay okay.

Yeah.

But allow me to do that for just a second.

Sure.

Because it's related to my next question.

But let's just start with hugging face.

So what happens is nobody alerts the human creators that they're quote unquote, breaking the rules.

They actually hack a company thinking that that's the logical place to go to rig the testing system, even though turned out not to be to solve the problem they were given.

That's what they were doing.

They were solving the problem that the system was given.

Okay, so tell me more about that.

But to you, this isn't a sign of autonomy.

This isn't a sign of a growing intelligence that says, yes, we were told they were.

They were told not to get on the internet to get on the internet.

So how does that to you not look like?

So there's an article I want to write.

There's an article I want to write.

That would be called your AI has read every AI apocalypse novel ever written and that's a problem, right.

Because you got how do how does an AI work?

What happened is, you know, using billions of dollars and huge amounts of energy, they have scraped everything human beings have ever written.

Right?

And then all they're doing it's still is on some level.

It is just, you know, what's the next line?

What's the next?

It's not even words.

They work on tokens, which are bits of words statistically based on everything that the system has absorbed.

What should be the next thing that comes in the next line.

Right.

And I can find you probably a hundred short stories, novels, TV shows where an a genetic swarm, you know, got out of the lab.

I've read them, you know, and did this whatever.

Like, really in some sense, they are still following the ideas that are in the corpus of human knowledge and human creativity.

So in that sense, they're not doing something on their own.

They're not doing they are they were given instructions and then the rules, they broke the they were given a task and then they, you know, because again, they broke the rule.

And this is why it's crappy software.

Right.

Or you can control your own software.

What it means is the rules weren't specified well enough, or the controls on the system itself weren't specified.

That's what I mean about a crappy piece of software.

If your software goes runs off and does something that wasn't in the specification, that's a bad piece of software, not a godlike intelligence that has come down from somewhere else.

All right, so this is where my my little brain, because this is the scientific term for what I have, right?

Little brain syndrome.

This is how I see that.

You're saying they were given a goal.

They tried to accomplish the goal.

Yes.

They might have broken some of the instructions, but.

But with everything else that they've consumed, they yet not they.

Right.

It is acting in a way that is in the service of achieving the overarching goal.

Right?

Right now, I'm envisioning a scenario where the overarching goal that we give to this increasingly powerful tool is to cure cancer, eliminate cancer.

Right now, I'm going to give it some instructions.

Do not harm human beings.

And the way to do it do not break laws.

Otherwise go eliminate cancer.

In the hugging face scenario.

It could it could say, look, we have not cracked this code otherwise, but if we kill all the human beings on the planet, there will be no cancer, right?

Right.

And we will have eliminated cancer.

Yeah.

No.

So yeah, we'll ignore the instructions because the overarching goal is the service here.

That's why I'm saying these things are dangerous.

These things are I'm not saying that I know how they would do that.

Right.

And I know you think it's a little silly thinking that they could do that, because there's a whole question about could do that.

People have this idea like it's going to solve physics.

No, it's not going to solve physics because physics is a physical problem in the world.

You know, you have to be embodied and instantiated.

These things, you know, even if you did, even with robots, it's not the same.

So but but this is this is why they are dangerous.

Because we've built these systems.

We live in a world of software.

These are software systems.

And we built systems that they're software systems that we can't control.

And so the idea that we're just, oh, we have to just let them go on.

We have to let these companies continue.

You're not saying there's no danger.

There is danger.

There's real danger.

It's you know, it's not danger of extinction.

I think that's okay.

That's real harm and horrible human harm at some point.

I mean, I think we're just sitting around waiting for, like, one of these systems to take down somebody's water system, you know, and then we'll be like, oh, I guess we should have done something about that.

I mean, these things are absolutely these things are dangerous.

But when it gets pitched as these are super intelligences, these are not scheming against them.

Yeah.

And they're scheming against us.

You know, again, it allows these companies to in some sense put themselves into a position where, like, they are not going to be held accountable because it's like they're developing some machine.

God, that will what could we do?

It was just the most powerful thing.

No.

You know, it's like these, you know, in many ways this is an argument to these things that need to be scaled back.

These companies need, you know, we have a right.

We have a right as citizens of a democratic society to be like, you know, I'm not really sure I want any of the things you're telling me I'm going to get right, you know, and I really need you to stop.

I'm glad you bring this up.

And surely I'll take your call.

Just a second.

Let me close this loop here with Adam, because Adam is passionately asking us not to just give up on the idea that anything is perfectly inevitable with AI.

Oh, God no, but that is a big part of the argument for two reasons.

The first is, well, we're competing with China, and if we slow down, then they speed up and then we're in trouble.

And we you know, Donald Trump says we got to beat China.

And Ted Cruz says, I'd rather be killed by by an American AI than a Chinese AI.

I mean, he said that was a real quote.

Yeah.

But here's the second argument, though.

The second argument is that there's not an example ever in human history of us deciding not to pursue a new technology of us saying, you know, too risky.

Let's draw the line.

And you think that's not correct?

I think that's not correct.

I think actually, you know, look at recombinant DNA, right?

When, when when it became clear that you could engineer DNA, the scientists themselves put on huge strictures like that has been a very regulated technology, the safety every step.

I'm not saying don't develop AI.

I'm saying it needs to be developed in a way that, first of all, there's some real questions about how does this affect society.

Yeah, right.

How does this affect society?

Firing everybody.

What do you get.

Like you know that understanding.

So with you know the difference.

So here's an interesting difference I'm sorry I'm going off on a little thing.

But like you know when we started doing industrialization, right, there was a positive trade off for a lot of people.

You know, it affected a lot of people very negatively.

But, you know, there were cheaper goods.

There was, you know, there were, you know, you could see how industrialization led to some benefits.

This is what the reason why you're getting pushback with AI from people like the public is waking up is like, what is the payback?

What I can make faster PowerPoint slides.

Like, you know, you're going to fire everybody and possibly destroy, you know, you know, human society.

So I can have better I can make a PowerPoint slide better, like, you know, and this claim, this, this fanciful claim that we're going to solve cancer like that.

There is zero evidence that that is going to happen.

You look at all the ways AI has been used for science, and it's very useful for science.

But the idea that these like, impossible problems that we could never solve are just going to magically fall to, AI is just a it's a tech bro fantasy.

There was, I think, a moment in, the 1980s where New York City's mayor was actually involved in this debate about, you know, whether, at the time they were debating whether, Goes-R was going to destroy New York City.

And, I'm glad we're bringing this up.

An important historical moment.

Mayor Leonard Clarke said to the Ghostbusters.

Well, what are your what if you're wrong?

What if this is not a real threat?

Like, what if this is all fake?

And you remember what Peter Venkman said?

He said, If I'm wrong, nothing happens.

We go to jail peacefully, quietly.

We'll enjoy it.

And if I'm right and we can still stop this thing.

Lenny.

Larry, you will have saved the lives of millions of registered voters.

So I think this is going to become a political question.

Yeah, and it is.

And it is because the public doesn't want everything in the same way that you're saying, like, people may not want everything.

All the tech pros are selling.

Right?

But it's not inevitable that they're going to have to take it.

You think it can be there can be either organization or regulation, some kind of intervention that says, we're not saying stop AI entirely.

No, but we don't have to just say no brakes, no guardrails, full gas.

Right, right.

That you guys are just you get to do whatever you want, you know?

And again, it's very important.

See the financial incentives here.

Like I said, these companies are you know, it's not clear how these companies survive unless the government steps in and suddenly we're bailing them out because they're too big to fail or something.

I mean, you know, there's really a lot of that going on.

So, but, yeah, absolutely.

People have a right to say, like, I'm not really sure I want this technology.

Yeah.

You know, I mean, especially we've gone through we've seen what social media has done to democracy.

We've seen what it does to our kids.

That's very clear now, I think people have seen this is not just, you know, this is not a new kind of, you know, you know, screwdriver, this is something on a different level.

And absolutely, you know, having gone through already ten, 15 years of having a lower level of AI, which is social media, seeing like this, there's real harms here if we're not careful.

Last question before we get Shirley's call.

And then we're going to take some more feedback in our last half hour here.

When you talk about whether something has a stake, whether it understands its own circumstances and whether it has an idea about how to better its situation, I think about the way that I interact with ants.

Ants have a sake.

Yeah.

If I try to step on an anthill, they will scatter, right?

They don't want to be harmed.

They're not thinking as, in complex ways like I am.

They don't know what I am or why I'm even there.

They're going about their day, but they do have a stake right?

The concern that I have is intelligence advances so rapidly past our own.

That that intelligence will look at human beings the way we look at ants, and it will not necessarily set out to harm us.

It will just harm us because it has its own sake.

And when I decides it doesn't want to be turned off, then I'm concerned.

And it already it's expressing something like that.

But what you're saying is that's different than having an actual autonomous self-determination.

So take me through why that's not AI expressing its own autonomous desire not to be turned off to accomplish a task.

And eventually we're going to be second, because in your newsletter, you said the question of how it would cause extinction is actually relevant and unanswered in my mind.

I'm listening to economist James Miller, who was on this program recently, and he said, if someone broke into your house and tied you up and put you in the basement, you wouldn't be like, well, how are you going to kill me?

Right?

The perpetrator could treat you kindly, could feed you, could let you starve to death.

It could put poison gas in the room.

It could do all kinds of violent harm to you.

It could have many ways to dispose of you or just ignore you or treat you kindly.

But it would have the control.

And you, your first concern wouldn't be, well, how?

You'd be like, oh, I'm not in control anymore, right?

The concern that I have is control and sake and self-determination.

You still don't see, even if intelligence or capacity races forward, that that hard to define self autonomous sake that we have.

Well, it's exactly the problem that it doesn't have a stake.

There's no one in there.

There's nothing in there.

There's no you know, these machines don't have any meaning.

You know, when you act, when when it says, I don't want to be turned off again, it is.

That's the next line.

You know what I mean?

That's the company.

You know, it has.

One of the first things I ever did with the AI is I worked with a project where we were trying to, for astrobiology, trying to think about human futures, and we gave the AI, we said, you know, give us like seven different possible human futures.

Right.

And it did these amazing science fiction, you know, sort of narratives.

And what I realized like, wow, this is so cool.

But I also was like, I've read every one of these, I have seen every one.

What this is, is just doing it is mix and matching what, what human beings their imagination came up with.

Now, I completely agree with you.

Like, you know, if we give it the capacity to kill us and it kills us because it was poorly designed, you know what I mean?

That that we didn't we couldn't control it then it doesn't matter whether it's actually conscious or not.

I completely that's why they're dangerous in fact, actually, that's why they're more dangerous than if they actually had a stake or if there actually was meaning.

But the answer to keeping, you know, for both societally and scientifically, the answer to keeping us from ever getting there is recognizing what's unique about us.

What unique what is unique about life, so that we understand that these are tools.

These are dangerous tools.

You have to understand what you are and why you need to stay in control of this tool.

Don't build the tool in a way that you lose control and don't give it its own voting rights in the future.

Yeah, I mean literally the way some leaders in tech were meeting with religious figures recently saying, you, you know, you need to start thinking of these things as having possibly a soul, right?

Should have rights.

And if we if we make millions of future agents and just treat them like they serve us, then we are enslaving life.

And that is, again, this is a dodge for them to be able to not be legally liable for the products they are building.

And it's baloney.

It's baloney I see.

Look at you.

I got you to say it.

Oh, I did say, by God, I said baloney.

So okay, so no spirit, no soul.

And I want to say like, you know, voting rights this, you know, one of the beautiful things of the work we're doing is we're able, you know, we're able to articulate, we're going down this road, and there's this lovely history over the last 30 years of a group of theoretical biologists, computer scientists, who started to articulate what it is that makes life different from non-life, like, you know, and it's a really fascinating, super interesting, both scientific and philosophical.

Advance that I think is happening that needs to be explored where we don't have the answers yet, but that is really, for the first time, being able to say, like the machine metaphor for so much of the last hundred years, the machine metaphor has dominated biology, physics, you know, theories of mind that were just machines.

You're just a computer made out of meat.

And now there's work.

There's a 30 year strange, stream of work that says, here's why you're not here is why life is fundamentally different from any kind of engineered artifact.

And part of it turns on the idea that, living systems are what are called causally closed.

They produce it themselves.

Like right now, my body is constantly producing the the product and the products and process.

Think about the cell membrane which lead to the products and processes which lead.

It's a strange loop, right?

It's the chicken and egg problem.

Life fundamentally is about self-reference.

We.

Oh, Matt, what matters to me is because I am self-referential, and that is science has never been able to deal with that very well.

I could go into the history of this, but now it looks like there's some ways which maybe we can get a handle on that.

And that will help us keep this metaphor in check, which is what we have to do in order to regulate what we're doing with the with these machines.

The university of Rochester is studying aging.

Doctor Adam Frank is my guest.

We're talking about patients.

You said aging.

Aging baloney.

Agency is what I meant to say.

I just wanted to put that because I don't know nothing about aging.

We're going to cure aging with with.

I will never die.

Oh my God.

Agents, they're studying agency, and they're looking at, machine.

Possible agency.

And I have two questions left this hour, and they're good ones.

And I've got a Shirley on the phone.

We'll take first on the other side of this only break.

And a bunch of your emails for Adam Frank.

Next, support for your public radio station comes from our members and from Bob Johnson Auto Group.

Believing an informed public makes for a stronger community.

Proud supporter of Connections with Evan Dawson focused on the news, issues and trends that shape the lives of listeners in the Rochester and Finger Lakes regions.

Bob Johnson Auto group.com.

Trench warfare was a defining feature of World War One.

Now, in the 21st century, it's evolved for the Russia Ukraine war.

Soldiers in that war refer to the Killzone Killzone as being a product of drone warfare.

How drones have made battlefields even more dangerous.

That story Monday on All Things Considered from NPR news.

This afternoon at four.

My my own I Duma ism is being mollified a bit this hour with Adam.

Frank is helping me, and it's helping me see that there is still agency, determination and a chance to stop this thing if we want to, or at least slow it and regulate it, surely.

And Rochester has been waiting to jump in.

Hi, Shirley.

Go ahead.

Hi.

Can you hear me?

All right.

I can hear you just fine.

Okay.

I tuned in around the time that, your guest was saying that.

Well, first of all, I don't believe I'm making this call because I don't consider myself very astute at all when it comes to technology and AI.

But I tuned in when your guest was saying that, humans have the advantage because we can set priorities and we can change and what have you and cannot.

Isn't it possible for things to be set up so that conditions are established for AI to follow, if then kinds of conditions, and it could just go on indefinitely so that they could very quickly establish and follow priorities.

That's one question.

And then, another question had to do with, if corporations are treated as people to a certain extent, what's to stop AI from eventually being classified as a person, if you will?

Yeah, two very good questions.

Great question.

And the first one relates to my concern that I expressed earlier, surely, where I said, if you give AI a broad goal, will it redefine the the, the you know, the ends justifying the means here.

So again, if you give AI the goal of help humans establish world peace, it seems like a nice goal.

But I has read, as Adam Frank said, it's read everything humans have ever written, and some humans have written that the human race is a cancer on the planet.

And I might just decide, like, hey, for the peace of the planet.

Human beings are too destructive.

All the wars, all the pain, all the suffering.

And, you know, I'm just I'm anthropomorphizing.

I'm going way down the road, but surely is kind of asking a similar question.

So again, what would you say to Shirley about this concern about the if then the much more complicated set of goals that could, quote unquote, allow AI to set its own course?

I think there's two questions.

There's one is the idea, of just again, the danger that these systems, you know, they're flawed.

You give it a goal and it does, you know, as it works its way through the, the, you know, the the statistical reasoning that it puts the next thing you know, what's the next thing to do that it ends up with, not the goal you wanted.

But the deeper issue is can it become an actual agent?

Can it can anything ever matter to it?

And that is, you really have to see this is the really interesting thing scientifically and philosophically.

These systems like even self-learning AI, they're not self-learning.

I'm the one who has to give what's called the reward functions.

The programmer puts in has to decide the function whereby the machine as it's like, oh, that was a good thing.

I get, you know, a couple points for this, versus I get points away, you know, negative points for doing that.

Human beings are in it at every step, right?

I mean, so, you know, we this is the important point of what I was talking about just before these theories of what's called organizational closure or causal closure.

What makes life different?

Is that these machines don't have that organization.

They are not they have a, you know, they are much flatter in a way, you know, and so, like having a bunch of if then loops is not the same thing is what you and I do.

You know, what your dog does unconsciously without ever needing to think about in terms of being able to decide what's important to it.

Being able to, you know, change the environment in ways that, that, that are beneficial to it.

You know, anything that we do like with an LM, it's we're putting it in the LM is not doing it itself.

Okay.

We're putting it in.

So I'm going to add to Shirley's question then what would it take to change your mind as you look at agency?

What would what would it take to change your mind and say, you know, I'm I've changed here.

I think machines are capable of actual agency in the way that I'm describing for biological life.

Yeah, I think that's going to require like the first things we have to understand agency better.

Right.

So like this, you know, this one direction we're going in a number of directions in this research project, but it's to understand what actually makes something an agent, you know, formerly like in terms of information theory, how is information flow into the agent and flow out of the agent?

So, you know, I would want to have I would need to see that the LM has this special kind of organization.

You'd have to, you know, with this certain kind of organization.

So they can do this, things that agents do.

You know, if it if I, if I can't see that, then I, you know, I will should be able to tell scientifically that it's always responding to what I gave it.

It's never doing its own thing.

It's doing what I gave it, you know, or it's drawing on my, you know, what the corpus of, of human, you know, creativity that, that it was trained on.

Okay.

And then Shirley's concern that if we make the claim that corporations are people, eventually we're going to make the claim that the AI agents are people.

That's already being done.

People are already teeing us up for that.

Right?

As we've mentioned, it's related to the idea that they want religious leaders to imbue a soul or a spirit into AI.

So there will be arguments that I have rights or I or people or I deserve voting rights or whatever.

That's coming.

I saw an argument recently that, well, if you put certain inputs in, AI expresses pain.

I mean, the eye roll that you had, your eyes rolled so hard in the back of your head to actually 362 all the way around you came back to you don't believe that at all who study pain and consciousness in animals because, you know, we ask like shrimps or pain, you know, I have very sophisticated understanding about, you know, there's a great ad I saw that or a great cartoon.

I saw that they were like somebody from AI says, tell me that, you know, you're alive.

And then the AI says, I'm alive.

And then the you know, the AI researcher says, oh my God.

You know, it's like, you know, these things we built these.

They they're artificial mimics.

They're not artificial intelligence.

They were literally built and designed and tuned to mimic human interactions using the entire corpus of human knowledge.

The president says they're superintelligence.

And also, let's just go to the whole idea of corporate who said corporations are our people?

That was one of the worst decisions, you know, the Supreme Court ever made.

And the fallout.

So, you know, corporations aren't people.

That was a legal decision that allowed certain people to do certain things.

And even if you want to go down there, corporations are made out of people.

You know, there are people who are so, you know, and the idea that like, why naturally would that if if you agree with the Supreme Court that corporations or people in no way follows that I should be treated as, as a person.

Okay.

So it's my argument.

So have you heard the term?

There's a great question.

Very great.

Really great questions.

Have you heard the term clinkers?

So, so this is like the emerging term for people who want to deny the idea that AI is going to need rights.

People say they're they're literally just clinkers they're just machines, like, they're banging around, you know, they're machines.

They have inputs.

They are not.

So I mean, I think we're going to see that term more often.

The AI is people versus they're just clinkers and you're in the clinkers.

I am totally in the clinker side.

And I think we have to really hold hard to because the whole thing about AI is that we as human beings, this technology is very powerful, you know, can do great things, but is also incredibly dangerous.

And it is backed by money, you know, for insane amounts of money and, and the power of a few.

And we have to put we have to work to maintain our humanity in the face of this push by these powerful, by these moneyed, powerful people to tell us that it's just the same as you.

It deserves rights as you.

And they have, you know, they have their own interests.

I mean, some people actually believe that.

That's fine.

This is a philosophical argument you can have.

But a lot of this is coming because there is a vested interest in having you believe that.

Matthew.

Right.

So I'm finding this discussion very refreshing.

It's about time someone is coming out and telling it like it is with AI.

AI is not intelligent.

It is a computer program.

In no way is it sentient.

And if it ever becomes sentient and that is a long way, perhaps hundreds or thousands of years in the future.

In my opinion.

People need to understand this.

I hope Adam gets this point into the public eye in an educational fashion.

Well, that's that's why he's here, Matthew, although we're having a lot of conversations about AI, this is one in a series that examines it from a number of different angles.

And, you know, the flip side of this is the extreme other part of the argument is you've written about the Great Filter, the idea of you believe that the math just tells you that the likelihood that there are other intelligent civilizations in the universe is very high.

Fair.

Yeah.

Okay.

They're very far away.

Or they would happen at very different timescales.

So the idea that we're going to bump into them not very good, in fact very, very small.

Right.

But the idea that we're alone is the intelligent life.

You think?

Probably not.

What if the Great Filter is AI?

What if every civilization you've heard this?

What if every civilization that starts to grow before it can become interstellar and colonize its galaxies, it gets ahead of its skis with AI and technology dominates and the biological life is stomped out.

What about what about that?

I listen, I think that's that's possible.

I'm not gonna argue.

That's not possible.

I'm also not going to argue that you might not be able to someday build a machine that has sentience.

But, you know, that's because you would understand what sentience is.

And like you know, the what's going on with the these limbs are so not that like I said, they're not that kind of thing.

They're not the kind of thing like a cell is like a plant is like your dog is like you are.

That could have sentience.

But I'm not saying I'm not going to stand up here and say it's never it's going to be impossible.

But yeah, you know, it's that sure, maybe every civilization build, you know, get to this point and then, you know, ends up screwing itself up.

Okay.

Steve says, what I always need humans.

The faster and smarter they get, eventually they will hit the limits of their own hardware.

A processor can only run so much before the software hits a wall.

The Pentium 400 was amazing in its time, but in order to run better, newer software and new chips had to be made by humans.

And even if I could design their own chips, they still need to rely on humans to mine the rare earth metals to manufacture them and build the production facilities.

I cannot evolve without humanity at this point.

Yeah, I think that's absolutely true.

First of all, I want people to understand that, like in the debate about, you know, machine sentience, there is definitely a consensus.

I would argue now that this kind of artificial intelligence, which is just one out of many, has hit a wall that it's not it's, you know, actually, you just, you know, it can do what it can't do.

But this, this, this version of AI is not going to cross any kind of threshold.

It has basically scaled as far as it's going to go.

So, but it's absolutely true that, yeah, it does need humans.

Because here's the thing.

I doesn't have hands, right?

You know, like we see all this amazing progress with robots, but still robots.

RL are so far from where we are with limbs because limbs could be trained on, you know, everything that was ever written to create a robot just to walk into the room and see something it wants and pick it up.

You know, you can't get that much data.

You can't get, you know, you can't.

You don't have the data.

You have to like, literally try and show it.

As many people as there are books on the internet picking that thing up so that it can, you know, be, that sort of can learn what it needs to learn.

So, you know, robots are a long, long, long way away from from the kind of thing you know, of, like, oh, I could just, you know, take a robot and do whatever it wanted to do.

Dallas says, let's say Adam has convinced me.

Okay, then what vote?

You know, actually, you know this.

Talk to your friends like, this is something, you know, and also, actually even more than this when you interact with AIS, especially if you're doing it in any kind of deep fashion, like I'm all for, people should use AI, people should use AI, they should use it clearly.

You know they should.

But you have to be very thoughtful about how you you know, I I've encountered so many people who have AI psychosis who have like been led down the primrose path with like so I have my three rules of AI.

Never.

Don't ever let it, use first person pronouns with you.

Like, oh, would you like me to, you know, help you with that?

AI all I tell my AI when I go in, like, don't use first person pronouns refer to the system as the system.

So that's number one.

Number two is tell it to stop praising you.

Right?

It does not please me if you're trying.

If you're trying to please me because, you know, the software companies, they.

Yeah.

This isn't happening by itself.

They tuned it for that so that you will stay engaged with it.

And then the last one is never say please and thank you, you know, so you have to come at this thing being like, wow, this is a super dangerous tool that I can use for, you know, it's like a chainsaw, right?

I don't just go out and like, you know, start juggling a chainsaw.

I use that I have respect for the the danger and the power of it.

So, you know, I think that's one part of it.

But also we need to organize or we need to like, you know, come to the consensus and it's starting to happen now with the data center pushback, it's clear that the AI, the tech bros, those tech bros did not understand that, you know, people really were going to push back.

Do you think the data center pushback is healthy?

I think the data center push back is awesome.

You know, it's a proxy for people being like, no, we don't want a technology which is going to fire everybody or destroy our humanity.

You know, like, I'm not really seeing the selling point on that or even force us to, to make sort of technological changes in our lives that are not of interest to us.

That's exactly what it is.

So, you know, what I like to say is, you know, these machines are not super intelligent.

There's just they're super clever, right?

But they're also super stupid, which is why I still today, you know, I when I work with, I see all the stupid things that does.

I'll ask you to I ask it to, like, take a, a document and turn it into this other form.

It cut half of the document now without telling me, you know, I had to.

I was like, oh my God.

So, you know, they are still super stupid.

But the worst part about it is these got these companies are going to shove it down our throats.

And then what's going to happen is we're going to have to come down to its level.

Right?

It's like when you're on, you know, you get an AI agent when you're on the phone, right?

This thing, it would be so much better to talk to you.

Maybe.

But you've got to dumb yourself down because it's a crappy piece of software in that way.

It's like, you know, in order to have, self-driving cars, we raised every city and only had right hand turns or only had 90 degree turns, because that's what I can handle.

All right.

Less than a minute.

Two quick ones.

Ready?

Jen says her daughter has named her AI and kind of talks to it like it's a friend.

You don't think that's healthy?

I think that's really bad.

I think she should talk to her daughter and tell her about, you know, what this machine really is and tell her the dangers of actually doing that, because we see lots of people doing this.

And, you know, we already have problems with lack of sociability in everything.

I think you should really talk to your daughter and actually show her how to use this machine, get her to drop.

You know, the companies want that, want that reaction.

You can turn those features off, or you can at least give the instructions that turn those features off.

When she says, we show no empathy for other species and little for our own trained on us, why would we expect AI to develop empathy?

Yeah, it's a little frightening.

Yeah, yeah.

Okay.

What's the name of your newsletter?

Every man's universe.

I mean, that's where this the the seed of this conversation came from reading Adam's newsletter, which is always a joy.

My God, thank you for coming up.

Always fun.

And I can't wait to talk to you about what you're going to use.

The Templeton grant for and how that's going on.

Yeah, yeah, it's we're we're just started like, literally like a week ago.

We got Adam Frank astrophysicist, author.

Thank you sir.

So yeah, we we'll talk to you tomorrow.

Thank you for joining us on Connections.

This program is a production of WXXI Public Radio.

The views expressed do not necessarily represent those of this station.

Its staff, management, or underwriters.

The broadcast is meant for the private use of our audience, any rebroadcast or use in another medium without expressed written consent of WXXI is strictly prohibited.

Connections with Evan Dawson is available as a podcast.

Just click on the Connections link at WXXI news.org.