Five ways to build resilient data centres

Waste heat utilisation

Across Europe, tech com­pa­nies are exper­i­ment­ing with waste heat recov­ery in their data cen­tres. Meta has been reusing heat from data cen­tres to warm 6,900 homes in Den­mark. Microsoft pow­ered a data cen­tre in Fin­land with car­bon-free ener­gy and recy­cled the waste heat to near­by homes and busi­ness­es. Ener­gy effi­cien­cy agency Code­ma part­nered with an Ama­zon data cen­tre in Ire­land to cap­ture the waste heat for use in homes and coun­cil build­ings. And in Swe­den, a project called Stock­holm Data Parks is run­ning in part­ner­ship with the city’s gov­ern­ment, the local heat­ing and cool­ing agency, and mul­ti­ple data cen­tres with the goal of heat­ing 10% of the city by 2035.

“Data cen­tres in Ger­many have evolved from being ene­mies of the state to now becom­ing one of the heat sources,” says Ste­fan Mink, head of TechOps host­ing at IONOS, who has been in charge of the plan­ning, build­ing and run­ning of 20 data cen­tres in Europe and the US. “It becomes a cir­cu­lar econ­o­my where the data cen­tres are using the ener­gy, but then also pro­vid­ing ener­gy in terms of heat use.” A 2017 white paper from the Alliance for the Strength­en­ing of Dig­i­tal Infra­struc­tures in Ger­many not­ed that the more than 13 bil­lion kilo­­watt-hours of elec­tric­i­ty that was con­vert­ed into heat in Germany’s data cen­tres would if reused, have met the ener­gy needs of Berlin. 

In 2019, invest­ment analy­sis of waste heat from data cen­tres showed that the process of reusing waste heat was a finan­cial­ly viable option and could pro­vide a pos­i­tive return on invest­ment for com­pa­nies. Fur­ther, by help­ing take pres­sure off the main grid, the process would even­tu­al­ly come back around and help make the data cen­tres them­selves less prone to out­ages. 

There is still a way to go before waste heat util­i­sa­tion can go main­stream, how­ev­er. Most data cen­tres still use air-based cool­ing but because air isn’t an effi­cient medi­um to trans­port heat, the con­sumer of the cap­tured heat needs to be locat­ed close to the data cen­tre. Added to this, the exist­ing infra­struc­ture would need an upgrade. “Cap­tur­ing and reusing heat does require a full over­haul of your entire facil­i­ties while [oth­er options] may be less inva­sive to your hard­ware set­up,” says Daan Terp­stra, exec­u­tive chair­man of the Sus­tain­able Dig­i­tal Infra­struc­ture Alliance. “But a typ­i­cal hard­ware refresh­ment cycle of all data cen­tres being some­where between five to sev­en years, I think this is an ide­al moment to start plot­ting the chart and plac­ing this at the top of the list.” 

Liquid-cooling technology

With the demand for data ris­ing expo­nen­tial­ly, data cen­tres need a large amount of ener­gy and elec­tric­i­ty to stay run­ning – and cool. Spe­cialised com­put­ing equip­ment can emit large amounts of heat and it’s impor­tant to reg­u­late this heat to keep the data cen­tre func­tion­ing.

Tra­di­tion­al­ly, this was done by cre­at­ing almost sub-zero, freez­er-like con­di­tions for the equip­ment. But in recent years, it’s been under­stood that data cen­tres are most effi­cient at ambi­ent tem­per­a­tures of 18–27C. As recent­ly as five years ago, 40% of the total ener­gy con­sumed by data cen­tres was going towards cool­ing the IT equip­ment, although that cur­rent­ly stands at approx­i­mate­ly 10%. 

While we’re cur­rent­ly in the era of air-based cool­ing, experts agree that liq­uid cool­ing – where heat from the equip­ment is trans­ferred to a liq­uid and siphoned away – is far more ener­­gy-effi­­cient. “Air-based cool­ing push­es the hot air out, you exhaust it,” says Lawrence. “And that’s wast­ed ener­gy.”

With the oppor­tu­ni­ty to trans­fer waste heat from data cen­tres and use it as an ener­gy source, liq­uid cool­ing becomes an impor­tant tech­nol­o­gy. Even in insu­lat­ed pipes, the hot air can’t trav­el very far before becom­ing use­less. Hot liq­uid, on the oth­er hand, is much more trans­portable. “The oth­er thing about direct liq­uid cool­ing is it uses very lit­tle water. It’s quite inno­­va­­tion-proof in the sense that how­ev­er hot it gets over the next two decades, it will be easy to use it.”

Artificial intelligence

Large tech­nol­o­gy com­pa­nies such as Microsoft, Google and Meta have set aggres­sive renew­able ener­gy tar­gets for their data cen­tres. Meta, which has more than 20 data cen­tres in oper­a­tion around the world, com­mit­ted to 100% renew­able ener­gy in 2011, fol­lowed by Apple, Google and Ama­zon.

Microsoft has pledged to be car­bon neg­a­tive by 2030 and has com­mit­ted to remov­ing all the car­bon the com­pa­ny has emit­ted, either direct­ly or by elec­tri­cal con­sump­tion, since it was found­ed in 1975, by 2050. A blog post on the company’s web­site states: “To reach this, data cen­tres must be part of the solu­tion for broad decar­bon­i­sa­tion.” Microsoft cur­rent­ly man­ages 200 data cen­tres and is on pace to build between 50 and 100 new data cen­tres each year.

Still, the path to net zero is by buy­ing off­sets, which means that effec­tive­ly many tech giants are still using fos­sil fuels. That sit­u­a­tion may change, and quick­ly, believe many experts, in part due to soci­etal pres­sure and upcom­ing leg­is­la­tion. “Based on the cur­rent social and eco­nom­ic cli­mate in con­ti­nen­tal Europe and the UK, sus­tain­abil­i­ty will become a licence to oper­ate,” says Terp­stra.

AI is one of the most cost-effec­­tive and scal­able ways to improve ener­gy effi­cien­cy in data cen­tres. In 2018, Google and Deep­Mind joint­ly devel­oped an AI-pow­ered rec­om­men­da­tion sys­tem to direct­ly con­trol data cen­tre cool­ing, result­ing in con­sis­tent ener­gy sav­ings of 30% on aver­age, the com­pa­ny claimed. 

AI can bring much more than ener­gy sav­ings – and there­fore, cost-effec­­tive­­ness. There’s also resilien­cy. Aliba­ba Cloud deployed machine learn­ing-based tem­per­a­ture alert sys­tems in its glob­al data cen­tre. “We took hun­dreds of tem­per­a­ture sen­sors’ time series mon­i­tor­ing data, using an ensem­ble graph mod­el to quick­ly and pre­cise­ly iden­ti­fy a tem­per­a­ture event due to cool­ing facil­i­ty faults,” a com­pa­ny engi­neer told Dat­a­cen­ter­Dy­nam­ics. “It gen­er­at­ed alerts much fur­ther in advance and pro­vid­ed the data cen­tre oper­a­tion team pre­cious time to respond to the fault, reduc­ing fail­ure impact.” 

Cool­ing and pre­dic­tive main­te­nance are the most cit­ed use cas­es for data cen­tre AI. Microsoft is devel­op­ing an AI sys­tem to analyse data and gen­er­ate alerts to “pre­vent or mit­i­gate the impact of safe­ty inci­dents” and Meta is inves­ti­gat­ing ways that AI can antic­i­pate how its data cen­tres will oper­ate under “extreme envi­ron­men­tal con­di­tions”.

Microgrids

Most data cen­tres have mul­ti­ple sources of pow­er so that if one source fails or goes down, anoth­er can keep it func­tion­ing. Resilien­cy has always been a pri­ma­ry con­cern for data cen­tre oper­a­tors and while the threats and the solu­tions might be evolv­ing, the abil­i­ty of a data cen­tre to with­stand fail­ures cost-effec­­tive­­ly remains para­mount. 

Micro­grids are increas­ing­ly seen as an excel­lent back­up solu­tion for data cen­tres. A micro­grid is an autonomous local ener­gy grid that allows you to gen­er­ate your elec­tric­i­ty, which means it isn’t depen­dent on the tra­di­tion­al grid. They can not only keep the data centre’s pow­er on dur­ing grid out­ages but store elec­tric­i­ty and sell it back to the grid dur­ing peak demand. “So many out­ages are hap­pen­ing that any crit­i­cal facil­i­ty – whether it’s a hos­pi­tal or a data cen­tre – is think­ing about how to make sure they’re able to run if the grid goes down, not just for an hour or two but poten­tial­ly for days or weeks,” says Jayesh Goy­al, chief rev­enue offi­cer at Enchant­ed Rock, a com­pa­ny con­tract­ed by Microsoft to devel­op California’s largest micro­grid. The micro­grid will use renew­able nat­ur­al gas and pro­vide Microsoft’s San Jose data cen­tre with back­up pow­er.

What makes micro­grids espe­cial­ly note­wor­thy, observes Goya, is that you can choose how you want to pow­er them. Renew­able, nat­ur­al gas or fuel cells, the choice is yours and con­strained only by cost and space. Nat­ur­al gas is a pop­u­lar choice for micro­grids because of its acces­si­bil­i­ty and low­er envi­ron­men­tal foot­print. But what’s excit­ing to many experts is the oppor­tu­ni­ty to bring hydro­gen fuel cells into the pic­ture. 

In 2020, Microsoft and Pow­er Inno­va­tions pow­ered a row of data cen­tre servers for 48 con­sec­u­tive hours using hydro­gen fuel cells with a first-of-its-kind hydro­gen gen­er­a­tor. Hydro­gen is described as a clean fuel because it only pro­duces water as a by-prod­uct. But hydro­gen only occurs nat­u­ral­ly in com­pound form and the cost and tech­nol­o­gy of sep­a­rat­ing it from oth­er ele­ments have been pro­hib­i­tive. This sta­tus has begun to change, though, and as it does so, gen­er­a­tors and micro­grids fuelled by hydro­gen start to look like a real pos­si­bil­i­ty. 

In Terpstra’s view, for hydro­gen fuel cells to be scal­able they need to be used for more than back­up gen­er­a­tors and micro­grids. “The busi­­ness-case cal­cu­la­tions I’ve seen mean that the invest­ment cost of set­ting up hydro­gen back­ups ver­sus the num­ber of times you would need it are com­plete­ly out of bal­ance. The run­time on back­ups is too lit­tle, com­pared to the invest­ment costs.” He judges that the only way it becomes cost-effec­­tive with a pos­i­tive return on invest­ment is to build a data cen­tre that is ful­ly pow­ered by hydro­gen fuel cells.

Underwater data centres

In 2018, Microsoft ran Project Nat­ick, drop­ping a data cen­tre con­tain­ing 855 servers 35m below the sea near the Orkney Islands. The aim was to insu­late it from tem­per­a­ture fluc­tu­a­tions and test whether under­wa­ter data cen­tres could be reli­able and prac­ti­cal while using ener­gy sus­tain­ably. 

Two years lat­er, in 2020, the data cen­tre was retrieved, and only eight servers were down. The com­pa­ny said the aver­age on land in the same time frame would have been 64. Sub­sea Cloud, which plans to start oper­at­ing an under­wa­ter data cen­tre off the US West Coast before the end of this year, said under­wa­ter data cen­tre con­struc­tion is cheap­er and would reduce car­bon emis­sions by 40%. 

In an attempt to meet their stat­ed tar­gets, Microsoft and oth­er hyper­scale com­pa­nies are exper­i­ment­ing with ways to make data cen­tres more sus­tain­able. While this is to be laud­ed, many of these exper­i­ments are also imprac­ti­cal in terms of cost and scal­a­bil­i­ty, says Terp­stra.

“We’re run­ning out of time. It’s super cool to have an under­wa­ter data cen­tre, yes, but there are so many oth­er, less invest­­ment-heavy, less engi­neer­ing-heavy solu­tions pos­si­ble that would result in the same effect by look­ing at the reli­a­bil­i­ty and cli­mate impact from a holis­tic design.”

Heat recov­ery from exist­ing data cen­tres, more clean ener­gy sources and eas­i­ly imple­ment­ed AI. Things that move the nee­dle now – and con­tin­ue to cre­ate an impact as the infra­struc­ture improves.