SCU Warts

1
SCU Warts All Server And Storage Consolidation
And Virtualisation experience
2

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Topics

• About SCU

3

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Topics

• About SCU
• Technical environment

4

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Topics

• About SCU
• Technical environment
• Server Infrastructure

5

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Topics

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure

6

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Topics

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization

7

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Topics

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps

8

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Topics

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

9

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

About SCU
10

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

About SCU

• Campuses located at Tweed Heads, Lismore, Coffs
  Harbour,
• Satellites located at Sydney (Hotel School)
  Melbourne (School of Business)
• Approx. 14K students
• Approx. 800 staff

11

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Technical Environment
12

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Technical Environment

• Intercampus connectivity 1GB via AARNET for major
  campuses with 100MB for Sydney and 512 KB ADSL at
  Melbourne,
• Networking infrastructure CISCO from edge to
  core, mix of 6500s and 3750s,
• Server Operating systems include, Novell Netware,
  Win 2003, Linux Red-hat, Sun Solaris and some
  Tru64,
• Server platforms Rack mount, HP DLs, DELL Power
  Edge 1650 1850, HP bl20 Blades, various Sun
  Ultra-Spark platforms, and some DEC Alpha

13

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Technical Environment

• Storage platforms HP EVA 3000, 4000, 56 spool
  capacity currently operating 146FC (7TB), MSA
  1500s currently operating 300GB Ultra SCSI, and
  500GB SATA,
• Enterprise backup HP Data Protector 6.0
  streaming to a MSL6060 LTO2 library,
• Virtualisation platform ESX 2SNP currently
  operating 8 licenses, environments are a mixture
  of Novell, Win 2000 2003, Linux Redhat.

14

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Server Infrastructure
15

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Server Infrastructure

• 2002 the Data Centre was fast becoming a
  bottleneck for growth as a result of space
  constraints due to the mishmash of various server
  configurations and footprints housed by
  in-locking space hungry shelving.
• End of 2002 SCU entered into a lease agreement
  opening the way for replacement of the entire
  server fleet equating to approx. 40 physical
  platforms at the time.
• From 2003 2006 server growth increased by
  around 50 to approx 95 physical platforms today,

16

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Server Infrastructure

• A significant portion of this growth can be
  attributed to a normalization of platforms
  required to deliver services reliably, i.e.
  impact of deploying NX tier architectures for
  high availability etc.
• 2003 Server platforms were standardized around 19
  inch rack mount systems with a combination of
  DELL and HP platforms,
• 2005 2006 saw the introduction of blade
  servers, and VMWARE,

17

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Server Infrastructure

• 2002 2005 saw a turn around in data centre
  constraints from space to power and cooling
  requiring additional investment. Power and Air
  are still not as good as they could be, seem to
  be playing catch-up,
• Server explosion rather than consolidation, there
  is some light on the horizon with VMWARE,
• Status today of the server fleet is services are
  reliable and asset management is in place.

18

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Storage Infrastructure
19

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Storage Infrastructure

• 2002 state of play All services operate with
  localized or direct attach disk, some raided and
  some not, services were constrained as drives
  reached capacity, costs increased as more array
  devices were brought in to pluck the gaps,
• The writing was definitely on the wall,
• Total storage under management 1.2TB including
  system disks,

20

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Storage Infrastructure

• We tendered for a Storage Area Network for 3.5
  TB with plan for 30 growth/annum and a
  replacement cycle of 5 years for the SAN.
• As a result of the Tender we ended up with a 6TB
  HP EVA 3000 dual controllers with redundant
  fabric,
• We also had to be back this thing up, so a part
  of the tender we include a requirement for an
  enterprise level backup and recovery solution HP
  Data protector, MSL6060 LTO2 library,

21

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Storage Infrastructure

• Within 6 months and with assistance of the vender
  we had consolidated all service storage and
  backup requirements onto the SAN, brilliant
  with capacity to burn so we thought!
• 2003 went past, then 2004, then half way through
  2005 we were at capacity, remember we only
  started with 1.2 TB in 2003 now we were at 6TB,

22

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Storage Infrastructure

• Infrastructure rollover was in 2008 so we needed
  to buy time, we added the remaining capacity of
  650GB usable this got us to 2006.
• 2006 Crunch time, the numbers on replacing a
  shelf of our 146GB FC disks with 300GB FC did not
  add up in-terms of value.
• For the approx. same cost we could obtained a
  tier 2 HP MSA1500 running 56 Ultra SCSI 3 300GB
  drives approx 12TB of usable space.

23

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Storage Infrastructure

• We are now in the process of migrating around 4TB
  off our SAN onto the MSA, services will include
  File Print, development environments, and other
  unstructured data, plus providing a disk to disk
  backup capacity,
• We should now easily make the rollover timeframe
  of the current SAN,
• We believe that a large component of this storage
  up take was a normalisation of usage as the
  Organisation had been constrained for so long
  with legacy infrastructures,

24

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Storage Infrastructure

• Apart from our capacity issues we do love our SAN

25

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Virtualisation
26

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Virtualisation

• Why? Primarily it has allowed us to provision
  cost effectively development/testing and less
  crucial production environments. Covering the
  shortfall in small project budgets that struggle
  to fund the necessary infrastructure,
• Bigger Picture! As we become more experienced
  with VMWARE we plan to assess all critical
  services inline with asset replacement schedules
  for the virtualisation option,

27

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Virtualisation

• Story to date In 2005 we acquired 3 chassis of
  HP BL20 blades, and 3 ESX 2 way licenses with
  virtual centre manager,
• End 2006 we have grown to 8 ESX 2 way licenses
  operating approx. 40 mixed environments including
  Novell, Win 2003 and Linux Redhat. (These are
  predominately development/test systems),
• Early 2007 will see implementation of our CRM
  system completely in VMWARE, this will also lift
  the VM profile for all the doubters out there,

28

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Virtualisation

• As a result of VMWARE we have seen an explosion
  of environments however this has can also be
  attributed to a normalisation of actually
  environmental necessities,
• Provisioning controls As VMWARE does lend itself
  well to on demand provisioning controls to
  regulate growth are crucial, as ideas and whims
  could easily be acted upon,
• Initial challenges breaking the stakeholder
  mindsets of physical infrastructure ownership vs
  virtualisation, expelling internal fear
  campaigns around performance and capacity, when
  90 of the server infrastructure is only
  operating at 20 to 30.

29

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Virtualisation

• Learning Curve Administrator learning curve is
  steep, getting the VM farm architecture right at
  the start is crucial, understanding the best way
  to provision storage and networking is essential,
  (Port, VLAN and LUN provisioning and structures,
  Firewall nightmares)
• Complexity combination of VM and Blades can add
  orders of magnitude in complexity when
  considering blades operate 4 Nics, Dual HBAs,
  and VM trunking can become a trifle complex,
  (Documentation Change Management crucial)

30

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Virtualisation

• Server sizing crucial to maximizing your license
  and service deployments, we currently have
  operate std 2 way 9GB VM environments this has
  mainly been due to funding constraints,
• Licensing we have found it cumbersome to manage
  yearly maintenance contacts for support as they
  all fall due at different times. We have arrange
  co-termination with VM but we have to do this
  with each license purchased is there a better
  way?

31

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Virtualisation

• Today, VMWARE is a core component in our
  architecture plan moving forward with expected
  infiltration into more crucial architectures over
  the next couple of years.

32

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Next Steps
33

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Next Step

• Determine consolidation and virtualisation
  targets,
• Leverage asset replacement schedules to fund the
  re-engineering of targeted services with the VM
  option,
• Integrate VMWARE into the design and development
  of the University IT DRP-BCP planning,
• Develop a framework for managing, determining and
  deploying virtual environments,
• Continue with the expansion of SAN architecture,
• Investigate ILM options for unstructured data,

34

• About SCU
• Technical environment
• Server Infrastructure
• Storage Infrastructure
• Virtualization
• Next steps
• ?

Questions