The Cloud Factories: Data Centers Waste Vast Amounts of Energy, Belying Industry Image

The company had been packing a 40-by-60-foot rental space here with racks of computer servers that were needed to store and process information from members’ accounts. The electricity pouring into the computers was overheating Ethernet sockets and other crucial components.

Thinking fast, Mr. Rothschild, the company’s engineering chief, took some employees on an expedition to buy every fan they could find — “We cleaned out all of the Walgreens in the area,” he said — to blast cool air at the equipment and prevent the Web site from going down.

That was in early 2006, when Facebook had a quaint 10 million or so users and the one main server site. Today, the information generated by nearly one billion people requires outsize versions of these facilities, called data centers, with rows and rows of servers spread over hundreds of thousands of square feet, and all with industrial cooling systems.

They are a mere fraction of the tens of thousands of data centers that now exist to support the overall explosion of digital information. Stupendous amounts of data are set in motion each day as, with an innocuous click or tap, people download movies on iTunes, check credit card balances through Visa’s Web site, send Yahoo e-mail with files attached, buy products on Amazon, post on Twitter or read newspapers online.

A yearlong examination by The New York Times has revealed that this foundation of the information industry is sharply at odds with its image of sleek efficiency and environmental friendliness.

Most data centers, by design, consume vast amounts of energy in an incongruously wasteful manner, interviews and documents show. Online companies typically run their facilities at maximum capacity around the clock, whatever the demand. As a result, data centers can waste 90 percent or more of the electricity they pull off the grid, The Times found.

To guard against a power failure, they further rely on banks of generators that emit diesel exhaust. The pollution from data centers has increasingly been cited by the authorities for violating clean air regulations, documents show. In Silicon Valley, many data centers appear on the state government’s Toxic Air Contaminant Inventory, a roster of the area’s top stationary diesel polluters.

Worldwide, the digital warehouses use about 30 billion watts of electricity, roughly equivalent to the output of 30 nuclear power plants, according to estimates industry experts compiled for The Times. Data centers in the United States account for one-quarter to one-third of that load, the estimates show.

“It’s staggering for most people, even people in the industry, to understand the numbers, the sheer size of these systems,” said Peter Gross, who helped design hundreds of data centers. “A single data center can take more power than a medium-size town.”

Energy efficiency varies widely from company to company. But at the request of The Times, the consulting firm McKinsey & Company analyzed energy use by data centers and found that, on average, they were using only 6 percent to 12 percent of the electricity powering their servers to perform computations. The rest was essentially used to keep servers idling and ready in case of a surge in activity that could slow or crash their operations.

A server is a sort of bulked-up desktop computer, minus a screen and keyboard, that contains chips to process data. The study sampled about 20,000 servers in about 70 large data centers spanning the commercial gamut: drug companies, military contractors, banks, media companies and government agencies.

“This is an industry dirty secret, and no one wants to be the first to say mea culpa,” said a senior industry executive who asked not to be identified to protect his company’s reputation. “If we were a manufacturing industry, we’d be out of business straightaway.”

These physical realities of data are far from the mythology of the Internet: where lives are lived in the “virtual” world and all manner of memory is stored in “the cloud.”

The inefficient use of power is largely driven by a symbiotic relationship between users who demand an instantaneous response to the click of a mouse and companies that put their business at risk if they fail to meet that expectation.

Even running electricity at full throttle has not been enough to satisfy the industry. In addition to generators, most large data centers contain banks of huge, spinning flywheels or thousands of lead-acid batteries — many of them similar to automobile batteries — to power the computers in case of a grid failure as brief as a few hundredths of a second, an interruption that could crash the servers.

“It’s a waste,” said Dennis P. Symanski, a senior researcher at the Electric Power Research Institute, a nonprofit industry group. “It’s too many insurance policies.”

The Cloud Factories: Data Centers Waste Vast Amounts of Energy, Belying Industry Image

The company had been packing a 40-by-60-foot rental space here with racks of computer servers that were needed to store and process information from members’ accounts. The electricity pouring into the computers was overheating Ethernet sockets and other crucial components.

Thinking fast, Mr. Rothschild, the company’s engineering chief, took some employees on an expedition to buy every fan they could find — “We cleaned out all of the Walgreens in the area,” he said — to blast cool air at the equipment and prevent the Web site from going down.

That was in early 2006, when Facebook had a quaint 10 million or so users and the one main server site. Today, the information generated by nearly one billion people requires outsize versions of these facilities, called data centers, with rows and rows of servers spread over hundreds of thousands of square feet, and all with industrial cooling systems.

They are a mere fraction of the tens of thousands of data centers that now exist to support the overall explosion of digital information. Stupendous amounts of data are set in motion each day as, with an innocuous click or tap, people download movies on iTunes, check credit card balances through Visa’s Web site, send Yahoo e-mail with files attached, buy products on Amazon, post on Twitter or read newspapers online.

A yearlong examination by The New York Times has revealed that this foundation of the information industry is sharply at odds with its image of sleek efficiency and environmental friendliness.

Most data centers, by design, consume vast amounts of energy in an incongruously wasteful manner, interviews and documents show. Online companies typically run their facilities at maximum capacity around the clock, whatever the demand. As a result, data centers can waste 90 percent or more of the electricity they pull off the grid, The Times found.

To guard against a power failure, they further rely on banks of generators that emit diesel exhaust. The pollution from data centers has increasingly been cited by the authorities for violating clean air regulations, documents show. In Silicon Valley, many data centers appear on the state government’s Toxic Air Contaminant Inventory, a roster of the area’s top stationary diesel polluters.

Worldwide, the digital warehouses use about 30 billion watts of electricity, roughly equivalent to the output of 30 nuclear power plants, according to estimates industry experts compiled for The Times. Data centers in the United States account for one-quarter to one-third of that load, the estimates show.

“It’s staggering for most people, even people in the industry, to understand the numbers, the sheer size of these systems,” said Peter Gross, who helped design hundreds of data centers. “A single data center can take more power than a medium-size town.”

Energy efficiency varies widely from company to company. But at the request of The Times, the consulting firm McKinsey & Company analyzed energy use by data centers and found that, on average, they were using only 6 percent to 12 percent of the electricity powering their servers to perform computations. The rest was essentially used to keep servers idling and ready in case of a surge in activity that could slow or crash their operations.

A server is a sort of bulked-up desktop computer, minus a screen and keyboard, that contains chips to process data. The study sampled about 20,000 servers in about 70 large data centers spanning the commercial gamut: drug companies, military contractors, banks, media companies and government agencies.

“This is an industry dirty secret, and no one wants to be the first to say mea culpa,” said a senior industry executive who asked not to be identified to protect his company’s reputation. “If we were a manufacturing industry, we’d be out of business straightaway.”

These physical realities of data are far from the mythology of the Internet: where lives are lived in the “virtual” world and all manner of memory is stored in “the cloud.”

The inefficient use of power is largely driven by a symbiotic relationship between users who demand an instantaneous response to the click of a mouse and companies that put their business at risk if they fail to meet that expectation.

Even running electricity at full throttle has not been enough to satisfy the industry. In addition to generators, most large data centers contain banks of huge, spinning flywheels or thousands of lead-acid batteries — many of them similar to automobile batteries — to power the computers in case of a grid failure as brief as a few hundredths of a second, an interruption that could crash the servers.

“It’s a waste,” said Dennis P. Symanski, a senior researcher at the Electric Power Research Institute, a nonprofit industry group. “It’s too many insurance policies.”

Bits Blog: Mobile App Developers Scoop Up Vast Amounts of Data, Reports Say

They can stream music, make restaurant reservations, and offer distractions for ornery children. But few things in life are free – least of all applications.

Two new reports this week reveal how app developers are scooping up enormous amounts of data from mobile devices and serving up ads in unlikely places. Advertising revenues have long been the oil that greases the Internet. Mobile apps are its new frontier. Both reports nudge application developers to adopt standards for data collection and advertising.

The Future of Privacy Forum, an advocacy group, on Wednesday, reported first the good news: Application developers are getting better at writing up privacy policies on both Google and Apple operating systems, the group concluded. But some still don’t take that minimum first step, even when collecting sensitive information, like location; they included a game, Fruit Ninja Plus, and a photo app, Camera+.

A private firm, Lookout Security, found far more invasive practices by the companies that serve up advertisements on behalf of app  developers. Some pushed advertisements to the device notification bar, well beyond the confines of the actual application. Others inserted new icons or shortcuts on the mobile desktop and tweaked bookmarks settings.

The proliferation of applications is staggering. Apple reported earlier this year that 25 billion applications had been downloaded on its iOS platform to date; Google estimated 20 billion had been downloaded on the Android platform.

“Given the pace at which the mobile ecosystem is moving, it’s important that standards are developed to ensure that private user data is accessed and managed appropriately,” Lookout Security said in its report.

It urged app developers to provide opt-out controls for when ads appear outside the walls of the application and provide explicit notification if the ad network intends to collect personal data including a user’s name and phone number.

The Future of Privacy Forum urged app developers to provide “enhanced notice” if the application is collecting information that users might not expect and to give users choice and control over what they share.

Bits Blog: Mobile App Developers Scoop Up Vast Amounts of Data, Reports Say

They can stream music, make restaurant reservations, and offer distractions for ornery children. But few things in life are free – least of all applications.

Two new reports this week reveal how app developers are scooping up enormous amounts of data from mobile devices and serving up ads in unlikely places. Advertising revenues have long been the oil that greases the Internet. Mobile apps are its new frontier. Both reports nudge application developers to adopt standards for data collection and advertising.

The Future of Privacy Forum, an advocacy group, on Wednesday, reported first the good news: Application developers are getting better at writing up privacy policies on both Google and Apple operating systems, the group concluded. But some still don’t take that minimum first step, even when collecting sensitive information, like location; they included a game, Fruit Ninja Plus, and a photo app, Camera+.

A private firm, Lookout Security, found far more invasive practices by the companies that serve up advertisements on behalf of app  developers. Some pushed advertisements to the device notification bar, well beyond the confines of the actual application. Others inserted new icons or shortcuts on the mobile desktop and tweaked bookmarks settings.

The proliferation of applications is staggering. Apple reported earlier this year that 25 billion applications had been downloaded on its iOS platform to date; Google estimated 20 billion had been downloaded on the Android platform.

“Given the pace at which the mobile ecosystem is moving, it’s important that standards are developed to ensure that private user data is accessed and managed appropriately,” Lookout Security said in its report.

It urged app developers to provide opt-out controls for when ads appear outside the walls of the application and provide explicit notification if the ad network intends to collect personal data including a user’s name and phone number.

The Future of Privacy Forum urged app developers to provide “enhanced notice” if the application is collecting information that users might not expect and to give users choice and control over what they share.