[Scott] Next on "Energy Switch," something you're probably using right now... Data Centers.

- Our primary source of energy in this country is still coal and natural gas, right?

So-- - Yeah, yeah.

- In the near term, the data center's energy use is gonna be primarily those utilities.

- For electricity?

- For electricity until we can find these bridge solutions.

- Yeah.

- But it's impactful.

- I mean-- - Yeah.

- If we're talking about scaling 10X-- - Yeah.

- For AI and putting an AI layer on top of the cloud, these are significant energy needs.

- The hyperscaler data centers, which are these big consolidated facilities that Josh was mentioning.

- Yeah.

- Could consume hundreds of megawatts.

So, that's equivalent to, one data center equivalent to a small city.

- Yeah.

- So, that's a lot of power consumption.

- You might say.

[all laugh] [Scott] Coming up, the massive energy required to serve our ever-larger data needs.

[Narrator] Major funding for this program was provided by Arizona State University.

Shaping global leaders, driving innovation, and transforming the future.

Arizona State, The New American University.

[upbeat music] - I'm Scott Tinker, and I'm an energy scientist.

I work in the field, lead research, speak around the world, write articles, and make films about energy.

This show brings together leading experts on vital topics in energy and climate.

They may have different perspectives, but my goal is to learn, and illuminate, and bring diverging views together towards solutions.

Welcome to the "Energy Switch."

Every time we look at our phones, every text, email, social media post, every web search, every digital photo, every credit card transaction, every online purchase, every streaming show, like this one, nearly everything in our lives involves data coming in and out of data centers.

They're becoming as necessary as electricity did a hundred years ago with incredibly huge electricity demands.

We'll talk about what that means for our power system with Ayse Coskun.

She's a Professor of Electrical and Computer Engineering at Boston University, and Director of their Center for Information and Systems Engineering.

Josh Fluecke is the co-Director of Syska Hennessy Group's Critical Facility Practice with over 25 years experience designing, building, testing, and managing data centers.

On this episode of "Energy Switch," the big benefits and big energy demands of data centers.

- Ayse and Josh, we're glad to have you here today.

Really high level.

What are we using 'em for?

We hear data centers.

So, what is it?

- So, we use data centers for all kinds of computing applications.

That include web hosting, anything on the internet essentially, storing our data, AI, and virtually all businesses use data centers today.

- Yeah.

- So, it's becoming part of critical infrastructure.

- Yeah, so every time I take a picture on my phone, that's a data center?

- Yes, yes, yes.

- Potentially storing that-- - You're storing it on the cloud.

- In the cloud.

- That's right.

- So, in order for you to be able to pull up that photo instantly, it's actually stored in three different data centers.

- Oh, is that right?

So, it's accessing whatever is the most efficient for me to?

- Exactly.

Right.

So, it's pretty incredible.

Data centers are touching just about every aspect of our lives.

Average home in America has 21 devices that are connected to the internet, and these are collecting, exchanging information.

And there's a lot of things that folks don't even realize.

Common ordinary devices like a doorbell or a thermostat are now connected.

So, data centers are this engine that is really driving all this technology that we've come accustomed to and began to rely on for our basic needs and functioning in society.

- Internet of things.

- Correct.

- Yeah.

And even if we don't want to be connected, are all those things connected, is that the reality now?

- I mean, you can turn off your phone.

[Ayse and Josh chuckling] Turn off all the electronics in your home.

But at this point, everything uses computing and internet.

- Yeah.

- I can't really think about many applications that can live without them.

- So, which countries are using the most now in terms of just data centers themselves, which?

- United States is number one in terms of the number-- - We're number one.

- Of data centers.

- Right.

- It's the U.S.

is also the leader in AI right now.

And then, China and European countries in the overall power consumption as a result.

[Scott] Okay.

- And just the United States has more than half of the data centers in the world.

- Oh, world.

- Right now.

Yes.

- We're really number one?

[Ayse] Yes.

- Oh, by a long stretch.

- Okay.

- Right.

Yeah.

Yeah.

- If you notice, a lot of data centers are being located closer to population centers.

So, a lot of the world is getting built up to kinda match the connectivity we have here.

- Yes.

- In the United States.

- So, the big cities are where the data centers are kind of clustering towards?

- Currently.

Yeah.

- Okay.

- That's been the trend.

And that makes, it's more efficient that way, Josh?

Or what?

- It's the latency.

The speed at which the information can exchange between whatever device you're using.

- Okay.

- And so, they've tended to be closer to the population center.

That's where that actual demand is.

- Got it.

- Yeah.

- Let me add something to that actually.

Because a variety of factors determine where data centers are built.

Incentives and land real estate.

Incentives and taxes.

Good reliable electricity.

- Right.

- And good electricity rates.

However, data center growth hasn't been homogeneous at all.

In some states, Virginia, more than 20% of Virginia's electricity is going to data centers today.

- Really?

A 5th.

- This is before.

Yes.

Yes.

- Wow.

- And there's a lot of government agencies, government organizations.

So, close proximity to that was attractive to a lot of companies building data centers.

So, as a result, there has been tremendous growth there.

- Right.

If I were to ride my electric scooter outside and pass a data center building.

- Yeah.

- I mean, what would it look like?

- From the outside, it looks like a warehouse without many people around.

- And not many bays.

- Very nondescript.

I mean, unless you knew what you were looking for, you may just say it's just another storage warehouse.

- Yeah.

- Or highly secured areas too.

So, it's not easy to get around.

But generally, they don't wanna advertise that they're there.

- For security reasons?

- For security reasons.

Right.

I mean, there are-- - Okay.

- Billions of dollars worth of equipment.

The equipment that's-- - Yeah.

- Sitting right behind you.

- I was gonna say it.

So, do they bring benefits to the local communities?

- Absolutely.

So, there's gonna be construction jobs for the building of the data center.

There's also the tax benefits that are gonna be gained by the community for the operation and the construction of the data center too.

- Okay.

So, once it's up and running?

- Once it's up and running, you have a 2-300,000 square foot facility.

You have 30 to 50 people.

- How big?

2 to 300,000?

- 2 and 300,000 square feet.

Right.

So significant.

- Like my house.

- This is just one building.

- Yeah.

[Josh laughs] [Ayse] Yeah.

- You get a lot in Texas, right?

- Yeah, that's right.

It's big here.

- Yeah.

Right, right.

But it's a 24/7 facility, so you'll have 30 to 50 people there.

But what's interesting is because of the equipment in there, you do need an army of folks to maintain this.

So, PricewaterhouseCoopers just released a study that said for every job that's created within a data center, six additional jobs are created in the service economy outside of there.

- Okay.

- So, you know, there's an incentive for communities to welcome in data centers because it can be a net benefit in terms of job creation.

- Interesting.

So, if I walked into one.

Like, is that one?

Is this what they look like?

- Data centers look like that pretty much.

- Yeah.

- There are racks.

Yes.

And then, this is in a big building.

And in addition to the racks, there could be power supplies, some other network devices and the like.

But essentially it looks like this, yes.

- I would think technology just continues to advance so rapidly.

How often would I replace the guts of this thing?

- There are small updates every three to six months.

Like, some new hardware comes in and data centers need to be up to speed in terms of getting the latest technology to serve their customers.

- Right.

- But the servers inside the-- - These black stacks, the hardware.

- Yes.

The hardware.

- Yeah.

- The computer hardware gets replaced every few years, essentially.

- Oh, it does?

And what happens to it?

The old stuff?

- Electronics recycling is a big problem.

And so, actually, part of the carbon footprint in data centers comes from manufacturing.

- Ah.

- These devices.

- I mean, I know we do a lot of stuff, we dump in landfill.

Is that where this is going or?

- Well, a lot of it's proprietary.

So, it's destroyed and then recycled.

But you know, it is a concern because there is a ton of embodied carbon that goes into- - You bet.

- A server.

There's a ton of, you know, very, very expensive metals that go inside these things.

- Yeah.

- That took a lot to get out of the ground.

- It's tough.

- Yeah, so.

- Two to three years.

That's more rapid than I would've thought.

- Well, you've seen it as low as 18 months.

You have to understand that these are not static facilities.

- Right.

- Right?

The customers are gonna drive the technology that's within there.

- Gotcha.

- Massive amounts of heat, I imagine.

[Ayse] That's right.

- From all the electronics.

- That's right.

- So now, I've gotta have air conditioning.

[Ayse] Yes.

[Scott] Always on?

- Always on.

- Yeah.

- You've gotta get rid of the heat somehow.

And so, as the demands of the power increase, the cooling demands increase commensurately.

- Okay.

- And so, there's still gonna be traditional data centers, cloud data centers, which are lower rack kilowatt density.

And we have the AI data centers that have a higher power density just because of the GPU, the different type of chip they're using.

- Yeah.

- Now that chip, one chip is a thousand watts.

- One chip?

- One chip.

And that's equivalent of a hair dryer.

And that's just one.

- One chip.

- And these are scaled over hundreds of thousands of feet of servers in these facilities.

[Scott] Yeah.

- And so-- - Incredible.

- All that heat needs to go somewhere.

- Yeah.

- Right?

Traditionally data centers have been using air for quite some time, and that worked up to a point.

But now, we're exceeding the power requirements.

When we get to the scale, the AI, a lot of it is liquid cooling, right?

So, they need to get liquid directly to the server.

It's a more efficient way to transfer the heat away from, from the server.

- Liquid being some sort of a?

- Yeah.

- Coolant or something, right?

- Right, right.

Close to the cooling system, yeah.

- Okay.

All right.

- So, we've seen a lot of improvement in cooling, data center cooling efficiency over the past decade or two.

However, this comes with cost in water use.

Right now, liquid cooling is mainly done through chilled water.

And this is great for efficiency, but not every place that's hosting data centers have ample water.

- Right.

- So, that's increasingly becoming a concern.

So, I think what I'm going to predict essentially is to see a mix again, of data centers.

If water is abundant, we'll see more data centers using chilled water.

These closed loop systems that Josh mentioned, there's a ton of new research coming into developing them, how to make it more efficient, also reliable because there's concerns about bringing liquids near electronics.

Like, we don't like water near our computers, right?

- Yeah.

- And I believe in the next decade we'll see a lot of new technologies making their way into data centers.

- Right.

Okay.

- And then, some of the new data construction due to convenience and costs and sometimes, lack of abundant water resources are still being done in the usual traditional way of air conditioning.

- Right.

- So, that's still happening too.

- Right.

- So, we'll see a mix of these things.

- Got it.

- But cooling is a big deal.

- Big deal.

And that's for the center.

That says nothing of the water used in the generation facility?

Cooling of-- - That's right.

- Nuclear, natural gas, coal.

- That's true.

- Lot of cooling water there.

So, there's kind of another impact to water-- - Definitely.

- Depending on where that's done.

Yeah.

Interesting.

So, we've talked a bit about it, but moving towards cities, is part of that the access to the reliable grid?

In other words, are these happy with a down grid?

- No.

No.

- Like you said.

- They're absolutely not.

I mean, that is the major selling point for a data center is its reliability and resiliency, right?

[Scott] Right.

- There's lots of service level agreements that are in place with these Fortune 500 companies that are expecting that the data center is there when they need it.

[Scott] Yes.

- And there's penalties to pay if you're not able to comply.

So, a lot is built into these, in these facilities to ensure that that power does not go down.

They'll have multiple primary utility services, each capable of picking up the entire facility.

They'll have an array of UPS systems to cover any momentary- [Scott] Uninterrupted power supply.

- Correct.

- Okay.

- And then, in the event of an extended utility outage, you have just a yard full of diesel engine generators to bridge an extended utility outage.

- You know, we were talking earlier about the redundancy.

Like, if you put a photo on the cloud, it's not in one location.

- Right.

- It's typically in multiple locations for latency issues.

And also in potential scenarios where one of the data centers is not accessible, you can still access your data through another data center.

- Yeah, and a lot of the expansion you're seeing currently in the U.S., I equate it to when we've originally started building the electrical grid to electrify the country and bring power to manufacturing, and all sorts of other uses we found at the time.

We're doing the same thing with data now.

- Yeah.

- We're creating the data grid for the country to operate because it's so important for our economy.

- Yeah.

- And this is gonna have to translate to all the major economies in the world and-- - Incredible.

- Other countries that are up and coming.

- So, let's talk about power.

Is it the highest consuming industry for electricity out there?

- It's not the highest consuming industry, but globally it's over a few percent of global electricity right now.

- Yeah.

- Which is huge.

In the U.S., the current estimate is about four percent of the electricity went into data centers.

- Okay.

- And this is expected to double in the next five years because of the expansion of AI, artificial intelligence into all kinds of applications.

- Right, right.

AI being much more intensive in its demands for energy than even a so-called Google search.

- That's right.

Compared to a normal Google search, an AI query like ChatGPT-- - Right.

- Consumes 10X or more.

- Right.

- Energy.

So, and then we use it more in all kinds of applications.

So, there's a cumulative effect of the usage as well.

- Interesting.

Three hundred thousand square foot data center.

It's a big footprint.

How much electricity does that use?

- Yeah, a good size data center would consume typically about 10 to 100 megawatts.

And 100 megawatts is roughly equivalent to about 80,000 homes.

The hyperscaler data centers, which are these big consolidated facilities that Josh was mentioning.

- Yeah.

- Could consume hundreds of megawatts.

So, that's equivalent to, one data center equivalent to a small city.

[Scott] Yeah.

- So, that's a lot of power consumption, - You might say.

[all laughing] Is there enough grid electricity?

And I think I've heard the answer, but let's just get it clear today for this growth.

- In the aggregate, yes, right?

I think last summer, I think, we came within six percent of our peak capacity, but I think if we're looking seven years out, 2030, we're gonna start, it's gonna become an increasing concern.

- Okay.

- And of course, when we say these megawatts, that's not always what a data center's using.

- Right.

- A lot of 'em are 70% utilized.

Could be several years before that place is fully filled up with all these AI servers that would need that power demand.

- Right.

- Is that enough time to catch up?

Sure.

- Yeah.

- I mean, if there is a demand, there's gonna be utility providers that are gonna be anxious.

- Right.

- To come out and say, "You know what?

We'll support."

So, we've seen the country go through the cycle before.

- So, you asked is there enough power in the grid?

- Yes.

- So, the answer is, I think, yes and no.

So no, because we have all these tens of gigawatts of demand per state coming and data centers are going to the power utility, and utility tells them, "Sure, I'll connect you in seven years."

And that's not acceptable for business.

And so, that means that we don't have enough power.

But then yes, because we are not necessarily using the power we have in the most efficient way.

- Right.

Are we gonna be starting old power plants?

- It's already happening.

It's already happened.

- What kind?

Natural gas?

- Yeah.

Yeah.

Well, natural, we've seen, yeah, we've seen some old coal fire plants.

- Oh.

- Potentially restart.

And a lot of it's due to the news that AI is coming.

We have these massive power demands, like, who's gonna support it?

- Right.

- There has been a lot of hyperscale interest in restarting nuclear power plants.

And actually, it's a fairly smart play because-- [Scott] Great play.

- We're putting in a carbon-free source that we can use.

We're restarting an old power plant, which is gonna have a lower carbon footprint, plus it locks in predictable energy availability for their growing needs.

- Yeah.

- There may be some issues with restarting nuclear power plants.

A lot of, you know, it's a highly regulated industry for that to happen in.

There's already been pushback for data centers within a community.

And you say, "Hey.

Nuclear."

You add that on top of it and you know, you're gonna hear some complaints and some pushback.

We're just trying to understand what that means.

- Yeah.

- So, in terms of nuclear for data centers, I also expect that some of it will be there because, you know, there's a lot of demand.

And nuclear is a technology that can provide that.

However, I don't see it as a scalable solution.

So, how many nuclear power plants?

Will we have one per town?

They also cost billions of dollars to build.

So, what I expect is to see a mix.

A lot more renewable generation because it's going to be easier to deploy in a scalable way probably.

- Yeah.

What backs up the solar and wind?

Natural gas?

- Right now, it's typically natural gas.

So nuclear, I think, will be in the mix for sure.

- Right.

- But if it's also costing billions of dollars, can we build hundreds of them the next 10 years?

Probably not.

- The small modulars, the promise of those.

One of the promises is you can put them where you want them.

So, that helps with some of it.

- Some of it.

Exactly.

But, I mean, if it's, for instance, I live in Boston.

Boston is also seeing some growth in data centers.

- Yeah.

- Will they put one of these in the middle of Boston?

Probably not.

- The issue is, is that it's probably maybe five to seven years away from getting regulatory approval.

- Yeah.

- For somebody to try out.

And then, it's gonna have to be in a spot where the community's gonna accept it, and you'd have to create a whole level of service and supply chain and whatnot just to support that new technology.

And that's gonna take some time.

- So, let's talk about carbon footprint of all of this.

The fleet today of data centers, what's its carbon footprint?

- Operational carbon?

Well, if you're looking at, you know, globally, I've seen estimates about one percent of global emissions are, you know, the greenhouse gas emissions are directly tied to data centers.

So, it's pretty significant.

If you look at the U.S.

having about half of the data centers, we're probably contributing about half of that minus the renewable sources where you use the nuclear power.

- We're half of the global emissions in the U.S.?

- Correct.

- But if you just come to the U.S., then it's gonna be a bigger number for U.S.-- - Correct.

- Emissions.

- That's true.

- And that's growing.

- Yeah.

It's gonna grow and-- - You mentioned coal, I mean, I was kinda surprised.

- Right.

Coal, right?

Yeah.

[Scott laughs] There's like, our primary source of energy in this country is still coal and natural gas.

Right?

So-- - Yeah, yeah.

- In the near term, the data center's energy use is gonna be primarily those utilities.

- For electricity?

- For electricity.

Until we can find these bridge solutions.

- Oh yeah.

- But it's impactful.

I mean, if we're talking about scaling 10X for AI and putting an AI layer on top of the cloud, these are significant energy needs.

And what we're seeing right now is, you know, where a data center is cited, you know, that location, availability of power is the primary driver for that location.

- Hmm.

- Right?

And they're irrespective of that energy source where it's getting, where it's developing energy.

- So, how do we meet decarbonization goals and [laughs] the new data center and AI center demands for power?

How do we do that?

- It's true that traditional data centers want uptime 24/7, right?

- Yeah.

- So, they want to be online.

However, a lot of the things that we run on data centers are not that urgent.

So sure, like, when you are doing a bank transaction, you want that response now.

You don't want it five minutes later.

But if you are training an AI model, that's going to take thousands of GPUs for, like, three months.

- Right.

- You may not mind if it's a little slower for the next four hours.

- Right.

- Or similarly, there's a lot of long-running jobs on data centers.

- Right.

- That can be slowed down.

- Okay.

Interesting.

- Paused or moved elsewhere temporarily.

And we have a lot of knobs and hardware and software where we can tune things really rapidly, unlike many other use of electricity in the society.

So, if we think about future data center design with a little bit of flexibility.

- Right.

- That would mean a more resilient grid, more immediate growth in AI data centers, more renewable deployment because renewables come with volatility like you mentioned.

So, this volatility can, to some extent, to some of it can be absorbed by batteries.

But we don't have large batteries at scale either, so this kind of flexibility-- - That's a whole other challenge.

- Through programs like demand response and growing them more could bring a lot of benefits today.

- Could AI save AI?

In other words, can AI help us optimize its own demands grid wide?

- That's right.

Yeah.

Yeah.

- To do this?

- There is research going on in that space.

AI for the grid.

Also AI for efficiency.

- Yeah.

Yeah.

- So, AI could be useful in doing this optimization for sure.

- To save itself.

Yeah.

Interesting.

It's been an incredibly fascinating dialogue.

Couple final thoughts.

- Sure.

So, we have these two infrastructures that are kind of colliding.

One is the power grid where we get our electricity and then the data infrastructure.

The data centers, AI, and the internet.

Instead of thinking about these two as separate things, how about we design the new AI data centers in a way that recognizes what's going on on the grid?

I think to enable this, we need more efficiency, sure.

We need energy being a primary consideration for designing anything related to AI.

But also we need to think about this flexibility.

We need to think about how to manage the grid and resilience of the grid along with the AI data centers.

[Scott] Okay.

And so you think it's doable?

- I think it's doable.

- And we have the two rules.

- I'm working on it.

- Yeah.

[Ayse laughs] That's quite hopeful.

Josh, how about you?

- Well, I truly believe with AI, we're at the advent of the next industrial revolution, right?

I think we're gonna reap enormous benefits in terms of medical breakthroughs, becoming more efficient at things, solving bigger problems with AI than before.

So, we're gonna need data centers and constant energy sources to support that growth.

And to me, you know, it's amazing time to be involved in the design of these facilities and seeing this growth and how we're gonna manage it, and how we're gonna intelligently approach these problems and challenges.

But, you know, we have some of the smartest engineers in the world in the United States, and some of the best technology companies in the world currently focused on data centers.

So, my hope is that we're gonna be able to tackle a lot of these issues head on.

They're not unaware of these by any respect.

- Right.

Well, look, I've really enjoyed our visit.

Ayse, thank you so much for being with us here.

- Thank you so much.

- Josh.

- Thank you.

- Thanks for your expertise and knowledge.

Scott Tinker, "Energy Switch."

Our lives and nearly all our electronics are connected to data centers.

Basically, they're huge warehouses full of computer servers, which are scrapped and replaced every two or three years.

Data centers are located near cities where their customers are, but increasingly they're placed wherever they can satisfy their enormous power demands-- Tens to hundreds of megawatts, equivalent to a small city.

It's consumed by the servers, but also their cooling systems.

Twenty-four/seven always-on electricity is essential.

Data centers have fleets of backup diesel generators, enough to operate off grid if necessary.

To meet their needs, utilities are building new natural gas plants, are running existing ones more frequently, restarting old coal plants, and considering restarting or building new nuclear plants.

The key to building the next generation of data centers, especially the huge hyperscalers required for AI, may be to design them with flexible power demands, so they can operate more harmoniously with the many other demands on the electric grid.

[upbeat music] ♪ ♪ ♪ ♪ [Narrator] Major funding provided by Arizona State University.

Home to the Thunderbird School of Global Management, redefining management education to empower transdisciplinary leaders.

Arizona State, The New American University.