How the Internet Kept Running During The Hurricanes

October 16, 2017      

At one node of the industrial backbone that keeps the internet running, employees sheltered from the worst of Hurricane Irma in a stairwell of a seven-story building in downtown Miami. When the power had gone out, diesel generators instantly kicked in to keep the lights on and prevent the internet from going down.

In Houston, at another of these nodes, called data centers, Hurricane Harvey pushed waters so high that a live fish flopped in a loading dock, but the physical defenses held.

Yet another data center, west of Houston, was so well prepared for the storm — with backup generators, bunks and showers — that employees’ displaced family members took up residence and United States marshals used it as a headquarters until the weather passed.

“It wasn’t Noah’s ark, but it was darn close,” said Rob Morris, managing partner and co-founder of Skybox, the company that runs the center.

For all their seeming immateriality, the internet and the cloud rely on a vast industrial infrastructure consisting of data centers linked through a sprawling network of fiber optics. The facilities are stacked with servers — boxlike computers that crunch the data for everything from hospitals, law enforcement agencies and banks to news websites, email and weather reports — that cannot be without electricity and cooling for even a fraction of a second.

Yet even as millions of people lost power across Florida, and thousands of homes and businesses were flooded out in Miami and Texas, the heavy digital machinery at the heart of the internet and the cloud held firm.

Though the storm disabled some cellphone towers and local connections, the center never stopped processing and transmitting data. It lost utility power around 8:30 a.m. on Sunday, Sept. 10, but supplied its own electricity with the generators.

When the storm eased and he walked outside, Mr. Eassey said, he immediately saw the effect that Digital Realty and other data centers had by keeping the servers going. “Everyone was talking on their phones, searching on their phones, and commerce that uses the internet to do their business was up and running,” he said.

A list of Digital Realty’s top customers shows why those operations are so important: Clients include IBM, Facebook, CenturyLink, Oracle, Yahoo, Morgan Stanley, AT&T and JPMorgan Chase, among others.

Inside the centers, the drill during a natural disaster can be overwhelming. One company, EdgeConneX, lost utility power for three days at a Miami data center, two days at a center in Jacksonville, Fla., and two days at an Atlanta location. But Rich Werner, director of operations at the company, said that backup generators turned on and service was never interrupted.

“Data center operations, to me, is 362 days of boredom,” Mr. Werner said. “And then you get these hurricanes coming through, and it’s three days of pulling your hair out.”

There are no legal standards for data centers unless they house servers for clients like government agencies, which require special protections.

But a professional organization, the Seattle-based Uptime Institute, rates the facilities on four tiers of resiliency for events like storms, earthquakes and run-of-the-mill power failures — possibly one reason the internet performed creditably during the hurricanes. The institute’s engineers “go to the site and pull the plug,” said Mark Harris, senior vice president of marketing, “and to be certified, it has to keep running.”

So when a hurricane or a tornado or an ice storm happens, the entire data center has been designed to withstand these external factors.

During Harvey and Irma, those measures seemed to work, said Doug Madory, director of internet analysis at Dyn, a web performance company with servers at a Miami data center. “I’m not aware of any core internet services going down,” he said.

Practically every single large oil and gas company in the world” operates servers in its data centers — an indication of how commerce of all kind relies on the facilities.

Depending on local conditions, some centers got off easier than others. In Houston facilities run by Data Foundry, automated systems worked so well that employees pooled resources and watched the Floyd Mayweather-Conor McGregor fight on pay-per-view.

Everyone in the business says that extensive preparation is critical.

The peak of the storm passed over the weekend, and when the power went out, the center was ready with diesel generators and a 10,500-gallon diesel tank on the site.

In Houston’s so-called energy corridor, the Skybox campus stayed above water. Utility power never went down. But the site was stocked with thousands of gallons of diesel fuel, as well as food and water, emergency medical kits, showers, bunk rooms and flares.

The site, which is a “modern-day fortress,” began hosting not only employees but several of their families, whose houses were flooded. Then the building doubled as an emergency response center for the United States marshals. Roughly 50 people used or stayed on the campus at one point or another during the storm.

If you want to keep your business up and running during the worst of weather, contact South Shore Generator in Wareham, MA.

Source: NY Times

---

< Go Back to Blog