Adoption of RackCDU D2C Liquid Cooling Flows On

Recent activities on both the OEM and the end user side point to the accelerated adoption of, not only liquid cooling in general, but in particular Asetek’s RackCDU D2C™ (Direct-to-Chip).

RackCDU is Asetek’s innovative hot water, direct-to-chip, data center liquid cooling technology. In the RackCDU D2C system Direct-to-Chip server, coolers replace the CPU/GPU heat sinks to remove heat in the server and to provide distributed pumping. The heat is then transferred to facilities water via a CDU extension on the back of each rack. A video detailing how RackCDU D2C works can be seen at http://bit.ly/1HqmLCv.

The increased adoption of RackCDU D2C is evident with both OEM and end users. At the International Supercomputing Conference (ISC15) on July 13th, Fujitsu announced Cool-Central™ Liquid Cooling for PRIMERGY™ Scale-out Servers. The CX400 M1 and its HPC cluster nodes use Asetek’s D2C. Fujitsu Cool-Central liquid cooling reduces cooling costs by up to 50 percent. The PRIMERGY CX400 M1 and its cluster nodes can increase data center density by up to five times to a maximum of 160 Intel Xeon processors and 1280 memory modules per standard rack. This enables wattages of over 30 kW per rack today with room to grow.

Additionally for HPC sites, Cray™ has announced a second generation of liquid cooled cluster supercomputers using Asetek’s D2C, the Cray CS-400LC, which like the CS-300LC allows for hot water liquid cooling of CPUs, memory and GPUs.

As seen in figure 1, there are many potential drivers for the increased adoption of liquid cooling. They can be bundled into the categories of relieving operating constraints, relieving capital constraints, addressing sustainability goals and enabling maximum performance and compute density.

Of these drivers, density and wattage trends are of high interest to HPC sites. The demand for higher density servers and racks incorporating higher wattage CPUs and GPUs put increased pressure on the OEMs to provide cooling for 1U and even 2U nodes. For example, Fujitsu anticipates that high density servers will reach more than 50 kW per rack by 2016 and is putting in place the modular and expandable liquid cooling technology with D2C to address this need.

Asetek is targeting the need for higher wattage nodes for OEM designs that will use next generation CPUs & GPUs. At ISC15, Asetek showed publically for the first time two forward looking liquid coolers. First, a liquid cooler for Intel™’s not yet released next generation Xeon Phi (Knights Landing). This bootable MIC includes its own CPU in a single package. Also on display was a liquid cooler for Power processors designed for use in overclocking OpenPOWER™ systems. (Asetek is a member of OpenPOWER).

In terms of HPC D2C end user adoption and installations, one of the most significant new sites is at the Sandia National Laboratories. As publically reported in Sandia’s “Sandia Lab News”, the liquid cooled 600-teraflop Sky Bridge supercomputer is a 1848 node Cray CS300-LC cluster supercomputer.

Expanding footprints at existing sites are, of course, of particular interest as they reflect end user satisfaction and validate reliability. The University of Tromso in Norway has been running production HPC workloads since January 2014 using D2C and this success has led to build out of the Stallo system to over 6,500 liquid cooled cores. In addition, Mississippi State University (MSU) installed a Cray-300LC cluster in February 2014 and expanded to a second cluster in December.   Meanwhile National Renewable Energy Lab (NREL), the world’s most efficient data center, has been operating with 100% uptime with D2C since June 2013 and is ordering new RackCDU D2C cooled equipment for installation in 2H 2015.

Perhaps the greatest affirmation of the increased adoption of Asetek’s data center liquid cooling technology comes from those who are running the data centers:

“We have moved from counting how many flops from the supercomputer to how many watts we recycle.” – Svenn Hanssen, UiT

“We’d rather pay for cycles than chillers.” – Roger Smith, Senior Computer Specialist, MSU.

“The facilities cost for a hybrid liquid/air-cooled system was 50 percent of the cost of a completely air-cooled system.” – John Noe, Manager of Scientific Computing (Sandia Labnews, December 2014).

“Asetek liquid cooling is easy to deploy and just plain works.” – Steve Hammond, Director, Computational Science Center.

As the use of liquid cooling in data center evolves, results will continue to speak for themselves. Increased adoption amongst OEMs and repeat orders from end users and HPC sites speaks to RackCDU’s performance and reliability. Looking forward, Asetek will can be expected to continue to address the key needs felt by both data centers and OEMs that will increasingly require liquid cooling.