supercomputer

data center now super energy efficient

 

 

 

 

Shyh Wang Hall. Photo: Department of Energy

computing water & energy savings

This data center houses supercomputers that require A LOT of cooling and was designed to be 90% more efficient than typical data centers, which we help achieve.

payback

<3 years

annual energy savings (kWh)

2.3M

annual water savings (gallons)

200,000

IT load

5 MW

PUE

< 1.1

cutting cooling for sustainable supercomputing

The National Energy Research Scientific Computing Center (NERSC) is a high performance computing center owned by Lawrence Berkeley National Laboratory (LBNL) and housed in Shyh Wang Hall. 

 

Data center energy efficiency is often characterized using a ratio called power utilization effectiveness (PUE).

 

An ideal PUE is 1.0, and the NERSC data center was designed to achieve a PUE of less than 1.1, about 90% more efficient than typical data centers.

 

We began working with LBNL after much of NERSC’s IT infrastructure was built out to help ensure that PUE goal was met.

 

You can read more about the project in the press release from LBNL.

data center energy efficiency consulting lbl nersc cray supercomputer

Cray Supercomputers. Photo: Creative Commons

summary

To ensure that PUE goal is met and produce results quickly, we work closely with the LBNL sustainability team and facilities staff to brainstorm ideas and start implementation as soon as we agree on the approach.

 

We use trends from the building automation system (BAS) to both identify trends as well as optimize operation after the measures are implemented.

efficient, high energy use design

The data center was designed to be highly efficient, employing both a chiller-less cooling water system and air side economizers.

 

Most of the IT load in the data center consists of two Cray supercomputers consuming roughly 5,000 kW, cooled by integrated cooling water coils and fans.

 

It also houses approximately 500 kW of IT equipment in hot/cold aisle configuration.

 

There are also floors of office space served by rooftop units which receive condenser water from the data center cooling towers.

computing savings

We implement measures to optimize the operation of fans within the computers by working directly with Cray, which is the first project in the world that has been done with Cray supercomputers.

 

This allows us to drive the building-level cooling systems harder to reduce the supercomputer energy, optimizing the facility’s energy consumption.

finding savings in efficient design

Despite the highly efficient baseline, we identified over 2M kWh in potential energy savings.

 

In the first 6 months, we helped implement, optimize and verify energy savings over 700,000 kWh. Since then, we’ve helped implement an additional 300,000 kWh of energy savings and additional efficiency measures are in currently progress.

hydronic cooling system optimization

One energy saving project is the optimization of NERSC’s hydronic cooling system, which involves several phases.

 

The graph below shows the progression of savings after the first phase. The purple line shows the baseline energy consumption and the green line shows the consumption after optimizing the condenser water reset based on outside air wet bulb temperature – nice and smooth.

 

This is only the first step in optimizing the system, so we didn’t dial it in perfectly. Next steps include:

  • Reduce bypass flow in closed loop.
  • Turn off redundant tower water pump.
  • Install booster pump to serve RTUs and allow closed loop pumps to operate at reduced dP all year.
  • Reset the closed loop supply water temperature to reduce energy consumption of supercomputers.
hydronic cooling system data center graph energy savings

hydronic cooling system use following first phase of optimization

ongoing commissioning

Over the 2018 fiscal year, we continued measure implementation and improved our ongoing commissioning process by using SkySpark.

 

SkySpark allows us to import building data from multiple sources into one platform to automate a significant amount of data analysis which helps us efficiently detect operation problems and for optimize controls sequences.

 

This project was presented in 2017 at the Better Buildings Summit in Washington, DC. You can view the slides from that presentation here.

Scroll to Top