Chapter 12: Green Data Centers

1
Chapter 12 Green Data Centers
HANDBOOK ON GREEN INFORMATION AND COMMUNICATION
SYSTEMS

• Yan Zhang and Nirwan Ansari
• Advanced Networking Laboratory
• New Jersey Institute of Technology
• Newark, NJ 07102

2
Power Consumption of Data Centers

• In 2007, Environmental Protection Agency (EPA)
  Report to Congress on Server and Data Center
  Energy Efficiency assessed trends in the energy
  usage and energy costs of data centers and
  servers in U.S.
• Based on the power consumption of data centers in
  U.S. from year 2000 to 2006, the power
  consumption of data centers is predicted under
  five different scenarios
• Two baseline prediction scenarios historical
  trend scenario, current efficiency trend
  scenario.
• Three energy-efficiency scenarios improved
  operation scenario, best practice scenario, and
  state-of-the-art scenario.

3
Power Consumption of Data Centers

• This prediction was performed with the total
  power consumption of the installed base of
  servers, external disk drivers, and network ports
  in data centers multiplied by a power overhead
  factor caused by the power usage of power
  distribution and cooling infrastructure in data
  centers.

Improved operation trend utilizes any
essentially operational technologies requiring
little or no capital investment to improve energy
efficiency beyond current efficiency trends.
Best practice trend adopts more widespread
technologies and practices in the most
energy-efficient facilities in operation.
State-of-the-art trend maximizes the energy
efficiency of data centers using the most
energy-efficient technologies and best management
practices available today.
Historical trend simply estimates the power
consumption trends based on the observed power
usage from year 2000 to 2006.
Data centers and servers in U.S. consumed about
61 billion kWh in 2006 for a total electricity
cost of about 4.5 billion.
The energy use of data centers and servers in
2006 was more than doubled the electricity that
was consumed by data centers in 2000.
It is estimated the energy usage of data centers
could nearly double again in 2011 to more than
100 billion kWh with historical and current
efficiency trends.
Current efficiency trend estimates the power
usage trajectory of U.S. servers and data centers
by considering the observed efficiency trends for
IT equipment and site infrastructure systems.
4
Energy Efficiency of Data Centers

• APC White Paper 6 (Ref 2) investigated the
  total cost of ownership (TCO) of physical data
  center infrastructure, and found that the cost of
  electrical power consumption contributed to about
  20 of the total cost.
• Numerous studies have shown (Ref 34)
• Data center servers rarely operate at full
  utilization.
• The average server utilization is often below 30
  percent of the maximum utilization in data
  centers.
• At low levels of workload, servers are highly
  energy-inefficient.

5
Energy Efficiency of Data Centers
Typical servers consume about half of its full
power at the idle state.
Power proportional servers consume about half of
its full power at the idle state.
Server power usage and energy efficiency at
varying utilization levels, from idle to peak
performance (L. A. Barroso and U. Hölzle, The
Case for Energy-Proportional Computing,
Computer, 40(12)3337, Dec. 2007).
6
Energy Efficiency of Data Centers
A typical data center power usage (adapted from
M. Ton, B. Fortenbery, and W. Tschudi, DC Power
for Improved Data Center Efficiency,
http//hightech.lbl.gov/documents/DATA_CENTERS/DCD
emoFinalReport.pdf, Mar. 2008).
7
Energy Efficiency Metrics for Data Centers

• In order to quantify the energy efficiency of
  data centers, several energy efficiency metrics
  have been proposed to help data center operators
  to improve the energy efficiency and reduce
  operation costs of data centers
• Power usage effectiveness (PUE) and data center
  infrastructure efficiency (DCiE)
• Data Center energy Productivity (DCeP)
• Datacenter Performance Per Energy (DPPE)
• Green Grid Productivity Indicator

8
Power Usage Effectiveness (PUE)

• The most commonly used metric to indicate the
  energy efficiency of a data PUE and its
  reciprocal DCiE.
• PUE definition
• PUE1.0 implies there is no power overhead and
  all power consumption of the data center goes to
  the IT equipment.
• PUE measures the total power consumption overhead
  caused by the data center facility support
  equipment, including the cooling systems, power
  delivery, and other facility infrastructure like
  lighting.

9
Power Usage Effectiveness (PUE)

• The average data center PUE in the US in 2006 is
  2.0, implying that one Watt of overhead power is
  used to cool and deliver every Watt to IT
  equipment (Ref1).
• It also predicts that state-of-the-art" data
  center energy efficiency could reach a PUE of 1.2
  (Ref 66).
• Google publishes quarterly the PUE results from
  data centers with an IT load of at least 5MW and
  time-in-operation of at least 6 months (Ref
  67).
• The twelve-month, energy-weighted average PUE
  result obtained in the first quarter of 2011 is
  1.16, which exceeds the EPA's goal for
  state-of-the-art data center efficiency.

10
Data Center energy Productivity (DCeP)

• Energy efficiency and energy productivity are
  closely related to each other.
• Energy efficiency focuses on reducing unnecessary
  power consumption to produce a work output.
• Energy productivity of a data center measures the
  quantity of useful work done relative to the
  amount of power consumption of a data center in
  producing this work.
• DCeP allows the continuous monitoring of the
  productivity of a data center as a function of
  power consumed by a data center.
• DCeP metric tracks the overall work product of a
  data center per unit of power consumption
  expended to produce this work.

11
Datacenter Performance Per Energy (DPPE)

• DPPE evaluates the energy efficiency of data
  centers as a whole. The DPPE metric indicates
  data center productivity per unit energy.
• DPPE defines four sub-metrics
• IT Equipment Utilization (ITEU)
• ITEE (IT Equipment Energy Efficiency)
• PUE
• GEC (Green Energy Coefficient)
• These four sub-metrics reflect four kinds of
  independent energy-saving efforts, and are
  designed to prevent one kind of energy-saving
  effort from affecting others.

12
Datacenter Performance Per Energy (DPPE)

• ITEU
• It measures the degree of energy saving by
  efficient operation of IT equipment through
  virtual techniques and other operational
  techniques.
• ITEE
• It is defined as the ratio of the total capacity
  of IT equipment to the total rated power of IT
  equipment.
• This metric aims to encourage the installation of
  equipment with high processing capacity per unit
  electric power in data centers to promote energy
  savings.

13
Datacenter Performance Per Energy (DPPE)

• PUE
• It indicates the power saving for data center
  facilities.
• The less power consumption of facility
  infrastructure, the smaller the value of PUE.
• GEC
• It is defined as the ratio of the Green Energy
  produced and used in a data center to its total
  power consumption.
• The value of GEC becomes larger if the production
  of non-CO2 energy is increased in a data center.
• DPPE
• Considering the definitions of the above four
  sub-metrics, DPPE incorporates these four
  sub-metrics and can be expressed as a function of
  them as follows

14
Green Grid Productivity Indicator

• Green Grid Productivity Indicator is a
  multi-parameter framework to evaluate overall
  data center efficiency.
• Through the use of a radial graph, relevant
  indicators such as DCiE, data center utilization,
  server utilization, storage utilization, and
  network utilization can be quickly, concisely and
  flexibly emerged to provide organizational
  awareness.
• How it works
• Set up the target value for each indicator.
• Plotting the peak and average values of each
  indicator during the period of monitoring,
  together with their target and theoretical
  maximum values on a radial graph.
• Assess how well the data center resources are
  utilized,
• Check if the business targets are achieved
  visually and quickly,
• Figure out how to spend their efforts to maximize
  the benefits.

15
Green Grid Productivity Indicator
Examples of using the Green Grid indicator tool.
(adopted from The Green Grid. The Green Grid
Productivity Indicator, http//www.thegreengrid.o
rg//media/WhitePapers/White_Paper_15_-TGG_Product
ivity_Indicator_063008.pdf?langen).
16
Techniques to Improve Energy Efficiency of Data
Centers

• IT Infrastructure Improvements
• Servers and Storages
• Network Equipment
• Power Distribution
• Smart Cooling and Thermal Management
• Power Management Techniques
• Provisioning
• Consolidation
• Virtualization
• Others

17
IT Infrastructure Improvements

• Approximately 40 - 60 power consumption of a
  data center is devoted to IT infrastructure,
  which consists of servers, storage, and network
  equipment
• Servers and Storages
• CPU
• Dynamic Voltage/Frequency Scale (DVFS) 23 - 36
  energy savings.
• memory and disk
• power shifting (Ref 14) re-budget the
  available power between processor and memory.
  Power shifting is a threshold-based throttling
  scheme to limit the number of operations
  performed by each subsystem during an interval of
  time, but power budget violations and unnecessary
  performance degradation may be caused by improper
  interval length.
• Mini-rank (Ref 16) an adaptive DRAM
  architecture to limit power consumption of DRAM
  by breaking a conventional DRAM rank into
  multiple smaller mini-ranks with a small bridge
  chip.
• Dynamic Rotations per Minute (DRPM) (Ref 17) a
  low-level hardware-based technique to dynamically
  modulate disk speed to save power in disk drives
  since the slower the disk drive spins the less
  power it consumes.

18
IT Infrastructure Improvements

• Energy proportional systems
• PowerNap (Ref 19)
• Attune the server power consumptions to server
  utilization patterns.
• Transit rapidly between a high-performance active
  state and a minimal-power nap state in response
  to instantaneous load.
• PowerNap can be modeled as an M/G/1 queuing
  system.

19
IT Infrastructure Improvements

• Network Equipment switches, routers, wireless
  access points.
• Sleeping mode
• Transit into the low-power sleep mode when no
  transmission is needed, and return back to the
  active mode when transmission is requested.
• The transition time overhead of putting a device
  into and out of the sleep mode may reduce energy
  efficiency significantly.
• Rate-adapting
• The lower the line-speed is, the less power the
  devices consume.
• Adapt the transmission rate of network operation
  to the offered workload.
• Speed negotiation is required in the
  rate-adaption scheme for both of the transmission
  ends.

20
Power Distribution

• Current typical power delivery systems for data
  centers still use alternating current (AC) power
• Distributed from utility to the facility, and is
  then stepped down via transformers and delivered
  to uninterruptible power supplies (UPS).
• Several levels of power conversion exist in both
  data center facilities and within IT equipment
  that results in significant electrical power
  losses, including power losses in UPS,
  transformers, and power line losses.
• DC power distribution system has been
  demonstrated and evaluated for data centers (Ref
  32).

21
Smart Cooling and Thermal Management

• Most data centers use liquid cooling for computer
  room air conditioning (CRAC).
• Rack-level liquid-cooling solutions bring chilled
  water or liquid refrigerant closer to the
  servers.
• Rear-door liquid cooling
• Sealed rack liquid cooling
• In-row liquid-coolers
• Overhead liquid-coolers
• Data center liquid cooling techniques tend to use
  naturally-cooled water, like lake or sea water
• Electrical savings by eliminating or reducing the
  need for water chillers in data centers.

22
Smart Cooling and Thermal Management

• The predominant air cooling scheme for current
  data centers is to use the CRAC units and an
  under-floor cool air distribution system.

23
Power Management - Provisioning

• Provisioning is an effective solution to reduce
  the power consumption by turning off the idle
  servers, storages, and network equipment, or by
  putting them into a lower power mode.
• An adaptive dynamic server provisioning technique
  (Ref 49)
• Effective to dynamically turn on a minimum number
  of servers required to satisfy application
  specific quality of service and load dispatching.
• Tailored for long-lived connection-intensive
  Internet services.
• A power-proportional cluster (Ref 50)
• Consists of a power-aware cluster manager and a
  set of heterogeneous machines.
• Uses currently available energy-efficient
  hardware, mechanisms for transiting in and out of
  low-power sleep states, and dynamic provisioning
  and scheduling to minimize power consumption.
• Especially tailored for short lived
  request-response type of workloads.

24
Power Management - Provisioning

• Sierra (Ref 53)
• A power-proportional, distributed storage system.
• Turning off a fraction of storage servers during
  trough traffic period.
• Utilizing a set of techniques power-aware
  layout, predictive gear scheduling, and a
  replicated short-term store, to maintain data
  consistency and fault-tolerance as well as system
  performance.
• Rabbit (Ref 54)
• A power-proportional distributed file system
• Provides ideal power-proportionality for
  large-scale cluster-based storage and
  data-intensive computing systems by using a new
  cluster-based storage data layout.
• Rabbit can maintain near ideal power
  proportionality even with node failures.

25
Power Management - Provisioning

• ElasticTree (Ref 55)
• Dynamically adjust the set of active network
  elements, links and switches, to satisfy changing
  data center traffic loads.
• Given the data center network topology, the
  traffic demand matrix, and the power consumption
  of each link and node, ElasticTree minimizes the
  total power consumption of a data center by
  solving a capacitated multi-commodity cost flow
  (CMCF) optimization problem.
• Urja (Ref 56)
• A network wide energy monitoring tool
• Integrated with network management operations to
  collect configuration and traffic information
  from live network switches and to accurately
  predict their power consumption.

26
Power Management - Consolidation

• Power savings with application consolidation
• Application consolidation in cloud computing (Ref
  57)
• The energy performance trade-offs for
  consolidation.
• The application consolidation problem can be
  modeled as a modified bin-packing problem, and
  the optimal points exist.
• Generic application-layer energy optimization
  (Ref 58)
• Guides the design choices by using energy
  profiles of various resource components of an
  application.
• Intelligent data placement and/or data migration
  can be used to save energy in storage systems.
• Hibernator (Ref 59)
• A disk array energy management system.
• Several techniques to reduce power consumption
  while maintaining performance goals, including
  disk drives that rotate at different speeds and
  migration of data to an appropriate-speed disk
  drive.

27
Power Management - Virtualization

• Effective to enhance server utilization,
  consolidate servers and reduce the total number
  of physical servers.
• Power consumption caused by servers is reduced.
• Cooling requirement should also be reduced
  comparably.
• Effective to build energy proportional storage
  systems.
• Sample-Replicate-Consolidate Mapping (SRCMap)
  (Ref 61)
• A storage virtualization solution for
  energy-proportional storage.
• Consolidating the cumulative workload on a
  minimal subset of physical volumes proportional
  to the I/O workload intensity.
• GreenCloud (Ref 62)
• Enabling comprehensive online monitoring, live
  virtual machine migration, and virtual machine
  placement optimization to reduce data center
  power consumption while guaranteeing the
  performance goals.

28
Others

• Energy-aware routing (Ref63)
• An energy-aware routing optimization model.
• The objective is to find a route for a given
  traffic matrix that minimizes the total number of
  switches.
• The proposed energy-aware routing model is
  NP-hard, and a heuristic algorithm was required
  to solve the energy-aware routing problem.
• Energy proportional datacenter network
  architecture (Ref64)
• A flattened butterfly data center topology is
  inherently more power efficient than the other
  commonly proposed topology for high-performance
  data centers.

29
Conclusions

• Power consumption is a central critical issue for
  data centers.
• As reported in 2005, the electricity usage of
  data centers has been almost doubled from 2000 to
  2005.
• The electricity cost accounts for about 20 of
  the total cost of data centers.
• Numerous studies have shown that the average
  server utilization is often below 30 of the
  maximum utilization in data centers.
• To quantify the energy efficiency of data
  centers, several energy efficiency metrics have
  been proposed
• PUE, DCiE, DCeP, DPPE, and Green Grid
  Productivity Indicator.
• Techniques to improve energy efficiency of data
  centers.

30
Thanks for your attention!