Electronic Cooling Solutions Inc.

1
(No Transcript)
2
Electronic Cooling Solutions Inc.

• Thermal management consulting company
• Located in the heart of Silicon Valley
• Provide solutions for thermal design problems
• Use of experience, modeling experimental
  methods in the design process
• Clients include over 60 companies

3
Outline

• Introduction
• Need for Innovative Cooling Solutions
• Objective
• Scope of this Presentation
• Cooling Solutions Air-based Cooling Systems
  
• Water-based Cooling Systems
• Refrigerant-based Cooling Systems
• Comparative Analysis
• Conclusion
• Vendor Contacts

4
Introduction - Power Density and Heat Load Trends

• Electrical Power Waste Heat
• Increase in power densities from the CPU-level
  to the System-level

System-level
Rack-level
Board-level
Component level
Room-level

• Current power density at the rack level 1 to
  3 kilowatts
• (up to 30 kilowatts per rack in two to four
  years - Hannemann and Chu 07)

5
Need for Innovative Cooling Solutions at
Rack-Level

• Datacenter TCO is characterized on a per rack
  basis
• Addition of newer and higher powered equipment in
  existing datacenters
• Hotspots in datacenter resulting from
  high-density servers
• Hotspots resulting from unavailability of cooling
  air from CRAC units
• Design shortcomings within the rack result in
  inefficient cooling

6
Objective

• Cooling of equipment in rack using airflow and
  impact of minor design changes for better cooling
  
• Cooling of high density equipment using
  water-based cooling techniques
• Cooling of high density equipment using
  refrigeration-based cooling techniques
• Provide vendor data for the above mentioned
  products

7
Scope of this Presentation

• Limited to discussion of cooling solutions only
  at the rack level
• Closed racks and commercially available products
• Discussion on Component-level, Board-level and
  Room-level cooling can be personally consulted

8
Airflow-based Cooling Techniques
9
Cooling of Racks with Conditioned Air

• Most easy to implement and maintain
• Limitation based on cooling capacity, acoustics
  and power consumption
• Detailed analysis could improve efficiency of air
  cooling

Application of Computational Fluid Dynamics (CFD)

• Pressure drops and airflow patterns
• Determine by-pass air
• Determine areas of re-circulation
• Determine failure modes of cabinet fans
• Requires testing to develop confidence in models

10
Cooling of Racks with Conditioned Air

• Airflow Enhancement in Racks
• Based on CFD Analysis
• Sample case study 1
• Sample case study 2
• Cabinet powered fans airflow layouts
• Best practices
• Airflow enhancing products

11
Sample Case Study 1

• Study done by Electronic Cooling Solutions Inc.,
• 42U Cabinet fitted with twenty one 2U units
• 15.75 kilowatts per rack
• Inlet temperature of 40 deg C
• Airflow 1932 CFM
• 10,000 ft Altitude Conditions
• Simplified cabinet shown here
• Objective was to optimize the cabinet for better
  cooling/use of higher powered equipment

12
Sample Case Study 1 (Contd )
Higher temperatures at the inlet sides
Inlet Temperature
Re-circulation of Flow
13
Sample Case Study 1 (Contd )

• Added vertical blockages

Open space between the rack rails and cabinet
sides
Vertical blockages between the rack rails and
cabinet sides
14
Sample Case Study 1 (Contd )

• Added blockage above the topmost unit of the rack

Top of 2U server
Area above the topmost 2U rack is blocked
Top of rack cover meant for passing cables from
front to rear
15
Sample Case Study 1 (Contd )
16
Sample Case Study 1 (Contd )
No blockages
Blocked passages
17
Sample Case Study 1 (Contd )

• Current study shows 17 to 25 increase in power
  dissipation based on exhaust temperature.
• By blocking re-circulating flow, it is possible
  to use higher powered equipment in the rack.
• Blocking can be done by employing Brush Strips.
• Avoid using larger racks with rails set to lower
  rack width settings (Using 23 rack with rails
  set to hold 19 equipment)

18
Sample Case Study 2
Container wall

• Study done by Electronic Cooling Solutions Inc.,
• Racks placed in containers
• Create airflow model of blowers
• Evaluate alternate designs for blower module

Heat Exchanger
2U Servers
2U Servers
Blower Module
Rack Model
Symmetry Walls
19
Sample Case Study 2
Testing
Module 1 860 CFM
Module 2 1024 CFM

• Apprx 20 increase in airflow with the
  re-designed baffles and perforated casing

Testing to collect data for modeling
20
Cooling of Racks with Conditioned Air

• Airflow Enhancement in Racks
• Based on CFD Analysis
• Sample case study 1
• Sample case study 2
• Cabinet powered fans airflow layouts
• Best practices
• Airflow enhancing products

21
Airflow Enhancers Cabinet Powered Fans
Front or Footprint Inlet
Supply Air
Raised Floor
Front In Rear Out
22
Airflow Enhancers (Cabinet Best Practices)
Raised Floor
Supply Air
23
Airflow Enhancers (Cabinet Best Practices
(Contd))
Dropped Ceiling
Dropped Ceiling
Raised Floor
Raised Floor
Supply Air
Supply Air
24
Airflow Enhancing Products

• APC 2U Rack Air Distribution
• Delivers air directly from the raised floor into
  the rack inlet
• Minimizes top-bottom inlet temperature
  distribution
• Allows rack loads up to 3.5 kilowatts per rack

Bottom-Top Rack Air Distribution Unit
Side Rack Air Distribution Unit
Roof Air Removal Unit
Application View
Images APC (www.apc.com)
25
Airflow Enhancing Products (Contd)
Rittal Side Breathing Air Baffle System
Rittal Enclosure Blower
Images Rittal (www.rittal-corp.com)

• Higher density rear door rack air removal unit
• Allows rack loads up to 16.5 kilowatts /14
  kilowatts per rack
• Challenges in obtaining flow through tiles in the
  datacenter

Liebert XDA
APC
Images www.apc.com and www.liebert.com
26
Water-based Cooling Techniques
27
Water-based Cooling

• Basis QLOAD mCp DT rVCp DT
• (Water has 3000 times higher heat carrying
  capacity than air)
• Chilled water from building supply
• Cooling high density servers up to 70 kilowatts
  per rack
• Lower energy cost as some of the CRAC units can
  be removed
• Avoid hotspots due to high power-density
  equipment
• Possible to have redundant systems (Chillers,
  pumps, piping, and power supply) to avoid
  downtime
• Importance of CDU
• Electrically conductive, corrosiveness and high
  flow rates

28
Water-based Cooling
Pure Water-based Cooling
Combination of Air and Water Cooling
Heat Exchanger
Heat Exchanger
Raised Floor
Raised Floor
29
Water-based Cooling

• Cooling system design by Naissus Thermal
  Management Solutions
• Heat removal of 20 kilowatts
• Closed liquid loop with bottom mounted fin and
  tube heat exchanger
• Thermal test done with 5 blade servers

Water from Chiller
Water sent to Chiller
30
Water-based Cooling
Temperature Distribution inside the Rack
31
Water-based Cooling

• Cooling system design by Vette Corp.
• Heat removal of up to 30 kilowatts
• Rear door closed loop liquid heat exchanger
  designed by IBM
• Currently available only for IBM Enterprise Rack
• Available from Rittal for retro-fit designs

IBM Rear Door Heat Exchanger
Images www.vette-corp.com
Pressure drop across the heat exchanger for a
typical 1U fan setup
32
Water-based Cooling

• Cooling system design by APC.
• Heat removal of up to 70 kilowatts
• Controlled in-row cooling
• Row air containment
• Modularity
• Similar designs from HP (35 kilowatts)
• Similar concepts available from Rittal (30
  kilowatts)
• Similar concepts available from Liebert (8
  kilowatts and 17 kilowatts)

Front View
Rear View
Heat Exchanger and Fan Assembly
Images www.apc.com
33
Refrigerant-based Cooling Techniques
34
Refrigerant-based Cooling

• Phase change (latent heat transfer)
• Electronics-safe
• Low flow rates and non-corrosive
• Some systems are stand-alone and hence flexible
• CRAC units are the most common ones
• Chilled water from building supply may be used
  for supplemental cooling
• Expensive ( comparable to water additives)

35
Refrigerant-based Cooling

• Cooling system design by Liebert
• XDF- Cooling capacity of 14 kilowatts
• Stand-alone unit

Liebert XDF Self Contained Unit
Images www.liebert.com
36
Refrigerant-based Cooling

• Cooling system design by APC.
• Heat removal of up to 43 kilowatts
• Modularity
• Rack air containment

APC In-Row Cooling
37
Refrigerant-based Cooling

• Cooling system design by Liebert
• XDV- Rack mount air conditioners (10 kilowatts) -
  Almost no floor space required
• XDH-Rack Cooling capacity up to 30 kilowatts
• Also available from Rittal

Liebert Roof Mount Cooling
Liebert In-Row Cooling
Rittal Rear Door Hx
Images www.liebert.com
38
Liquid Based Touch Cooling
39
Water/Refrigerant-based Touch Cooling

• Direct contact cooling combined with chip cooling
• Remove heat at the source
• Available from
• Clustered Systems
• Rittal (Power electronics)
• SprayCool (20 to 30KW)

Cold plate with Liquid Cooling
Liquid Cooling of Boards
Images www.rittal-corp.com
Images www.ibm.com
Spray Cooling
Images www.spraycool.com
40
Refrigerant-based Touch Cooling

• Cooling system design by Thermal Form and
  Function
• Pumped liquid multiphase cooling
• Heat removal of up to 10 kilowatts
• per evaporator (Modular)
• Designed for retro-fit applications
• Air/Water cooled condenser unit can be used

Thermal Form and Function Refrigeration Unit
Images http//www.thermalformandfunction.com/
Two Phase Flow
41
Comparison of Cooling Techniques
42
Comparative Analysis

• Study by Hannemann and Chu Interpack 07
• Comparative study of cooling technologies with a
  model datacenter

Capital Expenditure of Cooling Equipment
Area required for Cooling Equipment
Power Consumption of Cooling Equipment
43
Conclusion

• Reviewed innovative and commercially available
  technologies for cooling racks
• Discussed design approaches with use of CFD to
  maximize performance of air cooling
• Reviewed products and techniques for enhancing
  airflow within a rack
• Reviewed cooling of high density equipment using
  chilled water and refrigerant
• Selection of cooling strategy will depend on the
  specific requirements of the client

44
Vendor Contacts

• APC Morrison, Harold Wells Associates
  (925-355-9900)
• Rittal Sales (800-477-4000)
• Liebert Frank Stone (925-734-8660)
• Spray Cool Sales (866-993-2665)
• Clustered Systems Phil Hughes (415-613-9264)
• Trox AITCS - Thomas Hudgens (347-325-4347)
• Thermal Form and Function Joe Marsala
  (978-526-9672)
• Vette Corp - Skye Emerson (508-203-4694)
• Naissus Thermal Management Solutions - Mirko
  Stevanovic (416-892-4071)

45
References

• Product websites and communication with vendors
• Hannemann, R and Chu, H., (2007), Analysis of
  Alternative Data Center Cooling Approaches, ASME
  Interpack 1176, Vancouver, BC.

46
Acknowledgement

• Speck Design
• Our colleagues at Electronic Cooling Solutions
• Khyati Varma
• Ceferino Sanchez
• Adriana Romero
• Sridevi Iyengar

47
(No Transcript)