QUESTnet

1
QUESTnet Data Centre Workshop

• Mike Andrea
• Director
• National Data Centre Practice Manager
• May 2007

ICT master planners and strategists
ICT management services Data centre design
and development Telecommunications and
networking Project office services Contact
centre design and development
2
Topics

• Strategic Directions Overview
• AFCOM Five Bold Predictions
• Standards and Best Practice
• Design Considerations
• Build versus Lease
• Polaris Data Centre Overview
• QLD Model for discussion

3
The Strategic Directions Group

• Five Practice Areas
• ICT management services
• Data centre design and development
• Telecommunications and networking
• Project office services
• Contact centre design and development
• Vendor independent
• Design Authority on the 200m Tier III Polaris
  Data Centre at Springfield (14,000m2 N2,
  high-availability, high-security co-location
  facility)
• External CIO for Springfield, including
  responsibility for the ICT Master Plan for the
  new city of Springfield under development to
  provide
• 85,000 residents
• Education City (USQ Campus)
• Health City (public and private hospitals)
• 1 million square metres of commercial space

4
Strategic Directions Data Centre Practice

• Focus on business ICT service delivery through
  effective data centre facilities
• We act as the glue
• Between IT and Facilities
• Between specialist engineering disciplines
• Between the business owner and the delivery team
  (financials, design, efficiency and value for
  money)

5
Strategic Directions Data Centre Practice

• Recent international investigations include
• 2006
• China
• APC InRow Cooling Manufacturing Plant (Shanghai)
• 2007
• UK
• Reuters (media) and JP Morgan Chase Data Centres
  (London)
• Spain
• Santander Bank Data Centre (Madrid)
• Germany
• CommerzBank Data Centre (Frankfurt)
• Fiducia Data Centre (Karlsruhe) under
  construction
• Osram Plant (Regensburg)
• Piller Power Systems (Osterode)
• MTU Diesel Engines (Friedrichshafen)

6
AFCOM - Predictions
Five Bold Predictions for the Data Center that
will Change Your Future AFCOM Data Center
Institute, March 2006

• AFCOM is a leading association supporting the
  educational and business development needs of
  data center management, executives and vendors
  around the globe.
• Report
• Five Bold Predictions for the Data Center that
  will Change Your Future, March 2006
• Source www.afcom.com

7
AFCOM Predictions
Five Bold Predictions for the Data Center that
will Change Your Future AFCOM Data Center
Institute, March 2006
8
AFCOM Predictions
Five Bold Predictions for the Data Center that
will Change Your Future AFCOM Data Center
Institute, March 2006

• AFCOM discusses Mainframe skills
• In terms of large-scale design and construction
  (not retrofit) of data centres in Queensland, the
  market is very small, and the pool of experienced
  / skilled design teams is even smaller
• Most businesses are structured with
• IT groups targeted with business based ICT
  delivery not design and build of data centres
• Facilities teams focused on building maintenance,
  contracts and change management not design and
  build of data centres
• Data centre design and build projects dont occur
  that often, therefore in most cases, theres been
  no need to invest in in-house skills
• Selective out-tasking of skills?

9
AFCOM Predictions
Five Bold Predictions for the Data Center that
will Change Your Future AFCOM Data Center
Institute, March 2006

• If this is correct its only 3 years away
• Options include
• Upgrade or extend existing facilities
• Retrofit an existing building with a new data
  centre facility
• Relocate higher density servers / platforms to a
  3rd party facility
• Relocate the entire facility to a 3rd party
  facility
• Others?
• What does the budget (CAPEX OPEX) allow?
• Has anyone forward planned budget increases?
• Power charges are likely to increase
• Power and cooling density is increasing
• Power consumption has increased 7-fold in the
  last 7 years

10
AFCOM Predictions
Five Bold Predictions for the Data Center that
will Change Your Future AFCOM Data Center
Institute, March 2006

• The water crisis has recently seen power
  generation output of some Queensland power
  stations curtailed
• Queensland is growing not just in the South East
• The investment required in the energy network to
  continue to meet residential growth and business
  demand is huge
• Public holidays are some of the largest peak
  demand periods for energy providers
• Some commercial building owners are refusing, or
  are unable, to upgrade mains power and water
  feeders in order to support tenants data centres

11
AFCOM Predictions
Five Bold Predictions for the Data Center that
will Change Your Future AFCOM Data Center
Institute, March 2006

• Virtualisation consolidation
• Internet delivery of services and applications
• AFCOM notes IPv6
• Mobility both devices and users
• Increase in broadband and wireless internet use
• Virtual office working
• Working from home, vehicle or remote office
• The data centre is at the heart

12
AFCOM Predictions
Five Bold Predictions for the Data Center that
will Change Your Future AFCOM Data Center
Institute, March 2006

• Business disruption could be caused through
• Loss of power, cooling or communications
• Fire or water damage
• Natural disaster
• Bomb threat / terrorism
• Staff error / sabotage
• Data loss
• Security breach
• System availability of some server hardware can
  be less than the data centre itself

13
Standards and Best Practice

• There are plenty of Standards
• But what is best practice?

14
Standards and Best Practice

• There are plenty of Standards
• But what is best practice?
• Is it the best way to meet local standards?

15
Standards and Best Practice

• ANSI/TIA-942-2005 (approved 12 April 2005)
• Telecommunications Industry Association
  Telecommunications Infrastructure Standard for
  Data Centers
• Uptime Institute (Tier I to IV)
• Tier I (99.67) to Tier IV (99.99) availability
• Requires consistent application of Tier concepts
  to 16 critical subsystems
• ASHRAE (American Society of Heating,
  Refrigerating and Air-conditioning Engineers)
• DSM (Defense Services Manual Australian Federal
  Government)
• EPA (Environmental Protection Agency) Qld State
  Government
• BCA (Building Codes Australia) and Local Council
  building regulations and approvals
• Includes a mix of US, European and Australian
  standards
• Different voltages, codes, and interpretations

16
Standards and Best Practice

• Formal due diligence reviews can get tied up
  comparing US standards against Australian
  building codes
• EPO is a prime example (not desired or required
  on data centre floors for fire management)
• EPO in plant areas for electrical safety
• Fire separation of infrastructure and fire zones
• Fire ratings of walls and doors (circular
  mantraps are a problem)
• Size of zones impacts fire suppression system
  selection
• Security has significant implications in the
  design of the physical building
• Distance between redundant access points
• Loading bay access control management
• Fire escape doors (e.g. quad locking, steel
  encased)

17
Standards and Best Practice

• Availability of a facility
• How is it measured?
• Mixed levels of redundancy across a facility
• What if the ICT infrastructure is single corded?
• Does it account for all components of the service
  delivery chain?
• Electrical, mechanical, security, fire
  suppression, hydraulics, telecommunications
• People, support and maintenance
• Where are the single points of failure?
• Have you looked at the building riser lately?
• Mains power and water service connections
• Lifts and other access points

18
Design Considerations

• Just a few items of note

19
Design Considerations

• Most obvious is Power and Cooling
• Mains supply to the building (and/or suburb) for
  both power and water
• Weight - floor slabs (if not on ground) typically
  constructed in the range of 100kg/m2 to 400kg/m2
• Some 19 racks known to weigh over 1000kg (approx
  1600kg/m2, with point load of 400kg)
• Batteries for UPS systems
• Floor to ceiling height, including height of
  raised floor (if you have one)
• Fire separation (diesel generator, UPS,
  batteries, transformers, switchboards) and
  associated fire suppression

20
Design Considerations

• Lead times, local support/maintenance and supply
• Not just the obvious, big items gas fire
  suppression, switch boards, distribution boards
• Skills shortage its real there is a massive
  amount of infrastructure being built in
  Queensland
• It is difficult to secure consultants and trades
  with experience in data centres
• Ensure your design team understands the concept
  of a data centre, high-availability and
  high-density computing
• Services co-ordination, it can be a stretch to
  fit everything in the same riser or wall/floor
  penetration
• 415v or 400v ?

21
Design Considerations

• Location and climate dictates some solutions
• How are Acceptance Tests conducted?
• Initial build, and then during normal maintenance
  routines?
• Pay as you grow
• UPS, diesel generators, chillers, cooling towers,
  transformers, switchboards, racks
• Cabling (power and comms), cable trays
• Some keys to success
• Ensure the building itself doesnt cause future
  fit-out problems risers, diesel tank sizes and
  location
• Mains services (water and power) should be sized
  for ultimate load (cost of change later is too
  high and may not be possible)
• Design with concurrent maintainability and live
  upgrades in mind
• Have the phase 1 project team document the
  process for future fit-out

22
Build vs Lease

• Whats your preference

23
Build vs Lease Business Metrics

• Does an investment deliver and/or reinforce core
  business what is the return on the investment?
• Efficiency
• Floor Space Ratio ratio of plant room floor
  area vs raised floor area a direct result of
  density and appropriate infrastructure sizing
• Selection of redundancy level, UPS, cooling
  systems, fire suppression solution, and diesel
  generators all impact efficiency
• Ride-through capacity to limit false diesel
  starts and loss of cooling capacity through
  brown-outs

24
Build vs Lease Business Metrics

• Financials CAPEX and OPEX
• Lifecycle replacement costs and timeframes
• Rent of the area (raised floor and plant) and/or
  return (income) on the construction cost
• Sustainability Availability
• Ability to keep operating during mains power
  and/or water loss
• Total site availability across all components
• Risk
• Who bears the risk of repairs, replacements,
  capacity upgrades and/or non-performance

25
Build vs Lease build (or retrofit)

• Considerations regarding the investment
• Is it, or will it reinforce, core business?
• Could capital (short term investment) provide a
  higher return spent elsewhere?
• Are the ongoing operational expenses budgeted and
  committed for the next 15 years?
• Getting the right team together is key
• Electrical, mechanical, hydraulics,
  architectural, civil, construction, security,
  ICT/BMS, other
• Complying with formal/traditional procurement
  processes can create issues
• Cost of an experienced design team can be high
• Long lead times (from order to delivery)
• Fixed priced contracts may have exclusions
  relating to fluctuating copper, stainless steel,
  concrete and water prices
• Is the project big enough to ensure the right
  companies respond?
• Can the building cope?
• Full operational costs are yours and cant be
  shared

26
Build vs Lease lease

• Considerations regarding the investment
• All costs are OPEX, capital free for use
  elsewhere
• Are the ongoing operational expenses budgeted and
  committed for the next 15 years?
• Possible additional carriage costs
• Reliant on a 3rd party to ensure operational
  performance, management and maintenance of the
  facility
• The 3rd party isnt subject to formal procurement
  processes
• Service levels available, allowing IT group to
  focus on IT service delivery
• In a co-location facility, the operational costs
  are spread across the total leased raised floor
  area
• Support staff, maintenance, security and
  operations
• Consumables including diesel, water and gas
  suppression storage
• Upgrades, repairs and replacements are the 3rd
  partys responsibility
• Benefits may result through larger scale
• Mains services redundancy
• Carrier diversity
• Higher redundancy and/or longer sustainability
• Concurrent maintainability
• Pay for what you use

27
Polaris Data Centre

• Overview

28
Polaris Data Centre
A 200m private investment in Queenslands ICT
industry Lead by Queensland companies

• Springfield Land Corporation
• Strategic Directions
• Suncorp
• Thiess
• Deicke Richards
• Multitech Solutions
• BRW Enterprises

29
Polaris Data Centre
7000m2 of raised floor in a 14000m2 purpose
designed and built facility

• Tier III (N2)
• 800W/m2 to 1500W/m2
• 3 data centre levels
• 2 plant levels
• Design nominal 99.999 availability
• NN power distribution to the rack
• N1 Inert gas fire suppression store
• 48 hours sustainability

30
Polaris Data Centre

• Concurrent maintainability
• High security (24x7 security, DSM compliant
  intruder resistant perimeter, full CCTV coverage,
  mantraps and biometric access)
• Scalable, flexible, pay-as-you-grow modular
  system from 800W/m2 to 1500W/m2
• Slab loading to 1600kg/m2, with raised-floor to
  1500kg/m2
• Piller Rotary UPS solution (UBDT) with coupled
  MTU diesel engine, and additional Rotary UPS
  powerbridge (UBT) for chillers

31
Polaris Data Centre
32
Polaris Data Centre Key Learnings

• Everyone you talk to is a data centre design
  expert
• The IT and construction industries are poles
  apart
• A data centre is designed to fit within the
  physical boundaries presented hence no data
  centre is the same
• Diesel engines require a lot of air for cooling
  and combustion
• External smoke entry via fresh-air intake

33
Polaris Data Centre Key Learnings

• Investigate the power grid with Energex
• What appears simple can turn out very complex and
  take a long time
• Network upgrades and investment requires long
  term planning (transformers, feeders etc)
• The supply of water is critical (potable
  recycled)
• Some equipment lead times can be up to 2 years
• Meeting EPA requirements can be expensive
• Acoustics, emissions, diesel storage, waste
  disposal
• Size does create efficiency up to a point
• 3000W/m2 maybe in Polaris 2

34
Polaris Data Centre Core Design Team
COMPANY ROLE FIRST INVOLVED
Springfield Land Corporation Project Owner 2003
The Strategic Directions Group Design Authority ( ICT Systems) 2004
Deicke Richards Architects 2004
Thiess Design Construction Management 2006
Webb Australia Electrical, Communications and Security ( ICT Systems) 2006
MultiTech Solutions Mechanical, Dry Fire BMS 2006
Robert Bird Group Civil 2005
BRW Enterprises Hydraulics Wet Fire 2006
Clifton Coney Group Project Management 2005
APC Strategic Alliance Partner, Industry and RD 2005

• Queensland Headquartered Companies

35
QLD model for consideration

• Universities are
• undertaking reviews
• planning and/or performing upgrades
• Considering the build of new facilities
  (retrofit)
• Why not pool the resources to build something
  specific to University needs?
• Dedicated, purpose-built data centre facility as
  DR, secondary or primary site
• Collaborative model

36
QLD Model issues discussion

• Who
• Pays what?
• Owns the facility?
• Controls and manages the facility?
• How
• Are telecommunications charges considered?
• Are consumption charges considered?
• Are different Uni requirements met?
• Big should it be what Tier?

37
QLD Model issues discussion

• Does the university funding / budget model
  support and/or allow for leasing data centre
  space?
• Private investment in developing such a facility
  is based on financial models outside the
  traditional ICT business case
• Springfield as a location
• Energex Strategic Alliance with dual zonal
  substations (one online, second planned)
• Water connections potable and recycled
• Telecommunications multi-carrier, with
  dark-fibre to Brisbane
• Access to developed land with public transport,
  shopping centres and easy access to Brisbane
  airport
• Natural events are low risk
• Close to experienced consulting, design and
  construction team

38
European Data Centres
39
Summary

• Strategic Directions Overview
• AFCOM Five Bold Predictions
• Standards and Best Practice
• Design Considerations
• Build versus Lease
• Polaris Data Centre Overview
• QLD Model

40
Data Centres at the very heart of IT

• Thank You
• Mike Andrea
• Email mandrea_at_strategicdirections.com.au
• Phone [phone]
• Mobile [phone]
• Web www.strategicdirections.com.au
• Polaris Data Centre www.strategicdirections.com.a
  u/polaris