Visit Your Local PBS Station PBS Home PBS Home Programs A-Z TV Schedules Watch Video Donate Shop PBS Search PBS
I, Cringely - The Survival of the Nerdiest with Robert X. Cringely
Search I,Cringely:

The Pulpit
The Pulpit

<< [ Seeing Is Believing ]   |  Energy Crisis  |   [ Changing the Guard ] >>

Weekly Column

Energy Crisis: It's Hard to Make a Buck When You Are Storing Everyone's Stuff for Free

Status: [CLOSED]
By Robert X. Cringely

Combining themes of several recent columns, we can see that a lot of money is being bet on a future user computing experience based on web services. Ubiquitous broadband along with hefty processing capability in your desktop, notebook, and coat pocket will bring entertainment, information, and even classic office services to us wherever we are, finally making real Sun's old motto that the network is the computer. But this future brings with it a number of expenses and vulnerabilities. For users, there's the dilemma of trusting our data to whomever. For service providers, there's the alternate dilemma of having to hold that data, because the cost of keeping that data online all the time will be huge. It's an energy crisis in the making.

The best bet in terms of who will be providing these future web services is simple: everyone. Microsoft will build hooks into Windows Vista, making it work seamlessly with a variety of services that will clearly be coming from a revitalized MSN. What Microsoft does, Yahoo will do and Google will do -- entertainment suites, search suites, communication suites, Office suites. As Dave Winer said last week on NerdTV, developers will give users whatever they want. The big boys will be competing furiously for market share, too, as you can see from the recent public wrestling over buying a piece of AOL. And remember, that's the FREE part of AOL they are fighting over.

Wherever the majors let us down, the startups will take over with yet more individual services and suites. AJAX porno, anyone?

But every one of these service providers, if they really do intend to be there when we need them with all our pictures, videos, love letters, and construction blueprints, are going to have to keep all that data available online 24/7, which is an unprecedented storage challenge, and one that the storage industry has NOT been working on.

As my Mama would say, this is a problem of having a big old back end.

Let's imagine some typical numbers. In the U.S. alone, according to Nielsen/Netratings, we have approximately 202 million Internet users, each of whom is eligible for a free Gmail account with two gigabytes of storage. Since my mother uses less than two gigs and I use more, let's do our rule-of-thumb estimate with that number, making the potential Gmail storage obligation 404 million gigabytes or about 400 petabytes. That's 400 times the current capacity of the Internet Archive, but it is also probably a tenth or less the total capacity of our PC and DVR hard drives today, so I think it is a very fair number to play with.

Of course, all that storage won't be required just for Gmail, unless Microsoft decides to create phantom users and take down its competitors through overwork. (Would that be legal? Maybe.) Rather, the 400 gigs will be shared among many competitors. But for this exercise it doesn't really matter because the issue is TOTAL cost, not who is bearing that cost.

Probably 80 percent of this capacity will be borne by the major players, with each of those taking a roughly equal share. That's MSN, Yahoo and Google, assuming that AOL will be somehow distributed between them, with each having about 100 petabytes of storage.

How much storage IS that, really? Well, the biggest enterprise hard drives available today hold 400 gigabytes each, which means each of these companies is going to need AT LEAST 250,000 drives, making Seagate, Hitachi, Maxtor, and Western Digital all very happy. Though with volume discounts that's really only about $25 million in disk drives -- far less than Microsoft's legal bills.

Now let's build a data center using those 250,000 drives. A disk array can hold about 32 drives in a 3U space. In a typical cabinet you can store about 12 arrays or a total of 384 drives. That cabinet sits on a 2' x 2' floor tile, plus some aisle space, or about 10 square feet of floor space for planning purposes. 250,000/384=651 cabinets or about 6,500 square feet. Heck, that's nothing when you read about all the hosting companies, with their 20,000 square foot data centers containing 20,000 servers each.

But just how many of those 20,000 square foot data centers are there, really? Do a little investigating and you'll find many hosting companies share the same building and claim the same 20,000 square feet.

The problem comes when you start to think about power consumption. It's not that disk drives consume so much power or that they haven't come down in consumption over the years, but each of those cabinets will require using modern drives about 3,300 watts to run while the full 100 petabytes will require 2.148 MEGAwatts. And all that heat has to go somewhere, so the building will typically use three to four times as much power for air conditioning as it does to run the drives, taking our total power consumption up to just under 10 megawatts, which at typical U.S. industrial power rates will cost about $5 million per year.

NOW we know why Google bought those 30 acres on the Columbia River in Oregon right next to a generating station from the Bonneville Power Administration. It's a source of cheap, uninterruptible power.

Of course nobody would build such a data center today because it would require 330 watts per square foot, and even the most modern facilities are provisioned only with 200 watts per square foot. Most are designed for 100 watts. So chances are any of these companies would spread their storage over two to three facilities.

This is the kind of planning and provisioning required to support FREE services. Add pictures and especially video and the total data storage requirements go up by another two orders of magnitude, much of that supposedly still supported by ads.

That's a heck of a lot of ads.

My point here is that we're entering another period of Internet exuberance. Yes, a lot has changed since 1999, but it's amazing how many of the ideas being pushed are the SAME ideas, just empowered now by dark fiber, cheap broadband, and six years of Moore's Law. And this time I think it will actually work and the Internet will change even more than it has the ways we live and work. But it isn't going to come easy and it isn't going to come cheap.

Comments from the Tribe

Status: [CLOSED] read all comments (0)