Self-regulation 'not enough' for AI safety, Gary Marcus says

Amna Nawaz:

Top executives from the leading tech companies in the U.S. met with President Donald Trump today to discuss the future of artificial intelligence amidst growing concerns over AI models losing control.

Geoff Bennett:

The meeting came after the president unveiled a new A.I.-powered Web site designed to help Americans navigate government services. The president is also pushing a new name for artificial intelligence, superintelligence, as he argues the technology is transforming the economy and that the U.S. cannot afford to lose its lead over China.

Donald Trump:

This is one more example of why I will never stifle the growth of technology that will be bigger than the Industrial Revolution. President Xi knows that. We're leading by a lot. And we don't want to mix it up too much either.

Somebody said, oh, let's do a joint venture. Let's work together with China. But if we do that, we're going to be giving a lot of secrets that -- that we have that they just would want to have or anybody else. And, frankly, there is nobody in third place. I mean, it's all between us and China.

Geoff Bennett:

That push for innovation comes amid growing calls from researchers and lawmakers for stronger safeguards and growing concern about whether those protections can keep pace with the technology.

For more on the future of AI, we're joined now by Gary Marcus, professor emeritus at New York University. He's also the author of "Taming Silicon Valley" and the Substack Marcus on AI.

Thanks for being with us.

Gary Marcus, A.I. Researcher:

My pleasure to be here.

Geoff Bennett:

So we have heard on this program from AI researchers and experts who have warned about the potential for catastrophic, even existential risks from AI. You have pushed back on some of those warnings. What do you see differently?

Gary Marcus:

So I don't think extinction is a real likely risk for humanity. All of the scenarios, the so-called doom scenarios where AI just wipes us out like in "The Terminator," all assume that we won't fight back, and we will.

They all kind of miss some details about how we are geographically diverse, genetically diverse. They're just fantasies and not realistic. So extinction is not likely, but catastrophe might be likely. So we already had an incident where an AI reported something wrong and almost led to a war.

And we will see more of those, because these systems make errors and hallucinations. We're seeing lots of cyber hacking. And that might lead -- excuse me -- to, for example, the temporary takedown of an electric grid that leads to the temporary takedown of a hospital. We saw something like that with the WannaCry virus in the U.K. a few years ago.

So, there are lots of vulnerabilities. There may be serious damage. I just don't -- I don't think that extinction is actually one of the things that should keep us up at night.

Geoff Bennett:

And it raises the question of what to do about it. The president, as he met with those tech leaders at the White House today, he seems focused on self-regulation. In fact, he says the industry is capable of what he calls tremendous self-regulation, and they signed a voluntary agreement focused on AI safety reviews.

Is self-regulation enough?

Gary Marcus:

It's certainly not enough.

And trusting these companies is insane. The New York Times just broke a story saying that OpenAI was aware of the problems that gave rise to the Hugging Face incident that everybody is talking about months before it even happened, and they didn't do anything because they wanted to rush their products to the market.

That's exactly why we can't rely on self-regulation and why we need outside regulation, just like we have in aviation and food and drugs and everything else. It's absurd to exempt the AI companies, when they have shown time and again that they are not responsible.

Geoff Bennett:

How do you regulate these companies without simply giving China an advantage?

Gary Marcus:

I think the race against China is a little bit overblown.

China actually does have regulation, has licensing regime around its own software products. And we race in industries all the time, but still have some kind of regulation around them. We don't make terrible cars just because we want to beat China on cars.

Right now, the U.S. AI industry is building deeply problematic software, and we need to make sure that the companies make sure that it is safe. And this kind of race against China is not a reason not to do that. Moreover, the race against China is really going to end in a tie. It's going to be like Coke and Pepsi.

Both nations are going to have their own so-called sovereign AI, but neither is going to completely wipe out the other. And so the whole race aspect makes it zero sum, and that's really not what we should be doing.

Geoff Bennett:

You have been calling for a dedicated federal AI agency for years. What power should the government have that it doesn't currently have?

Gary Marcus:

What we really need most of all is independent evaluation of new software products, the same way we have for new drugs. You should have to, if you're the maker, show that the benefits outweigh the risks.

This should be like 101 and obvious. Right now, the White House is actually doing a lot of that, but without independent oversight with a secret framework that they haven't revealed to the public. So they did actually preclude one system, which was Mythos by Anthropic. They delayed it.

They didn't delay OpenAI's latest system, which actually has, I think, more problems in some way. And so this is happening, but in an idiosyncratic way, which means customers can't trust U.S. AI right now. They don't really know what's going to be released, what's not. It's not transparent.

We need a public, clear set of criteria like we have for drugs and so forth. That's the number one thing that we need. The other thing we need is auditing once systems are released. We need full disclosure of incidents and things like that.

Geoff Bennett:

Do you think Washington is capable of imposing meaningful rules on this industry before there's a major crisis?

Gary Marcus:

I mean, that's the, you know, what is it, $3 trillion question is whether this is going to happen before or after there's a major incident.

The Hugging Face hack -- and it turns out there were tens of thousands of similar incidents -- is really a wake-up call. And one of those incidents -- most of them won't be harmful, but one of them could be very harmful. And we really are racing the clock here to see whether the government can do anything about it before something really bad happens.

Geoff Bennett:

Gary Marcus, great to speak with you. Thanks for joining us.

Gary Marcus:

Thanks for having me.