The 30,000

1
The 30,000 View of Research Storage
Infrastructure

• Dick Deason
• Assistant Director
• Systems Administration
• rdeason_at_ufl.edu

2
Why am I here today?

• Much of your valued research data is sitting on
  local drives exposed to potential loses due to
  lack of Disaster Recovery options.
• The ICBR is pumping out large quantities of raw
  image data that require some level of
  preservation to extend usefulness in the event
  new algorithms are developed. In addition, many
  of you have local lab instrumentation that have
  similar data retention requirements
• Generation of final data sets from raw ICBR Gene
  Sequencing Data and other research venues such as
  BLAST results will require more storage and
  better protection than currently afforded.
  Offsite protection should be considered to ensure
  survivability and later accessibility.
• Access to your research data is critical.

3
Who are we?

• UF Health Science Center (HSC) is the countrys
  only academic health center with six
  health-related colleges located on a single,
  contiguous campus. The colleges teach the full
  continuum of higher education from undergraduates
  to professional students to advanced
  post-doctoral students, enrolling over 6,000
  students each year.

• The HSC is a leader in interdisciplinary
  research, generating 65 of UFs total research
  awards (422M). Included are McKnight Brain
  Institute, UF Genetics Institute, UF Shands
  Cancer Center, and the Institute on Aging.
• Total HSC employees is over 5000. Our Systems
  Team managing commodity services for the HSC is
  currently staffed with 5 FTE.
• We support every HSC mission to include
  Administration, Education, Research and Clinical
  activities all who have varied sources of
  funding.

4
What do we do?

• Two Secure Data Centers (Gainesville and
  Tallahassee)
• 49 Physical Hosts / 32 Virtualized Hosts
• 140 Mile geographical separation
• Two Tivoli Storage Manager instances
• Connected via High Speed Florida Lamda Rail

• Hardware HP and IBM
• Storage Two Xiotech Mag3D3000s
• OS - Windows 2003 and Redhat Enterprise
• Virtualization Virtual Infrastructure 3.0.2 (2
  three node clusters)
• DR IBM 3584 LTO Libraries, Tivoli and CommVault

• Commodity Services Clustered Storage, Print,
  Exchange 2k3/2k7, SQL2005, Oracle 9i/10i, IIS,
  Apache, Cisco VoIP, and Virtualization Hosting.

5
Storage Topology (GNV)
HSC Gainesville Data Center
Redundant AC / Redundant Ethernet 1GB / Redundant
208v Power Emergency PWR upgrade May
2008 Monitored Card key Access for physical
security 28 Racks of Equipment
Cisco Director Class Fabric (9513)
Provides Fiber Channel path / host-storage zoning
RESEARCH
1 1 5 4 1
1 0 7 1 0
FY2007
FY2005
FY2008
IBM 3584 LTO3/4 Libraries
6
Storage Topology (TLH)
HSC Tallahassee Data Center
Redundant AC / Redundant Ethernet 1GB / Redundant
208v Power Northwest Regional Data Center 1Gb
Data path across Florida Lambda Rail Research
Network
Brocade Fabric
Provides Fiber Channel path / host-storage zoning
TLH SAN
FY2009
IBM 3584 LTO3/4 Libraries
7
Storage and HSM Data Flow
Fiber Channel Disk Storage (GNV)
Research Host(s)
Based on retention qualities
For offsite data security
LTO4 Tape Storage (GNV)
LTO4 Tape Storage (TLH)
LTO4 Tape Storage (GNV)
ICBR Gene Sequencing
8
Storage Capacities by unit
XIOTECH MAGNITUDE 3000s (10710) Capacity
68TB File Commodity Storage Email Databases
XIOTECH MAGNITUDE 3000s (11541) Capacity
84.8TB Virtualization Hosting Exchange 2007 MS
SQL2005 Transactional Data
XIOTECH MAGNITUDE 4000e (TBD) Capacity 156TB VA
Research HSC Research UFGI UFSCC UFBI
IBM3584 LTO3/4 LIB Capacity 408TB-816TB Commodit
y DR VA Research HSC Research UFGI UFSCC UFBI
IBM3584 LTO3/4 LIB Capacity 408TB-816TB Commodit
y DR VA Research HSC Research UFGI UFSCC UFBI
Xiotech SAN 10710
Xiotech SAN 11541
Xiotech SAN Research
IBM 3584 GNV
IBM 3584 TLH
9
Who has paid for this?
VPHA ongoing support for RESEARCH MISSION Infrastructure (FY2007-FY2013) VPHA ongoing support for RESEARCH MISSION Infrastructure (FY2007-FY2013) VPHA ongoing support for RESEARCH MISSION Infrastructure (FY2007-FY2013)
  Annual Contribution One Time Startup Costs
Mag3D400s Infrastructure 25,066.00 NONE
Manpower (.25) 21,250.00  
Maintenance Support Contract 3,816.00  
ICON Management Licensing NC  
     
Cisco Fiber Channel Network 32,678.28 11,530.00
Manpower (0.10) 8,500.00  
SunTrust lease 18,936.95  
Maintenance Support Contract 5,241.33  
     
TSM DR/Archive Services 64,304.00 4,844.00
Manpower (0.40) 34,000.00  
SunTrust lease(s) 23,485.00  
Maintenance Support Contract 4,844.00  
TLH NWRDC Floorspace/Contract 1,975.00  
     
UFGI-Stats (SAS) / COP-Research (SAS) 4,250.00 NONE
Manpower (0.05) 4,250.00  
GNV DataCenter NC  
     
TOTALS 126,298.28 16,374.00
5 YEAR Commitment 631,491.39 16,374.00
10
Existing Request to Research Office
Funding Request to Sponsored Research (FY2008-FY2013) Funding Request to Sponsored Research (FY2008-FY2013)  
  Annual Contribution One Time Startup Costs
Mag3D400s Infrastructure 32,485.95 25,984.00
Cisco Fiber Channel Network 4,879.72  
TSM DR/Archive Services 15,901.77 2,990.40
     
TOTALS 53,267.44 28,974.40
5 YEAR Commitment 266,337.21 28,974.40
UFGI Contribution for Storage (FY2008)    
  Annual Contribution One Time Startup Costs
Mag3D400s Infrastructure - 45,000.00
Storage Bay/Disks/Licensing -  
     
TOTALS   45,000.00
5 YEAR Commitment - 45,000.00
This will allow Primary Researchers throughout
HSC to add Storage Costs related to their
specific research requirements incrementally, and
as required.
11
Why a SAN and HSM Solution?

• Because of our varied missions and funding
  sources, we were continually ask to provide
  services that had no boundary definitions in
  place. SAN and HSM (Hierarchical Storage
  management) Architectures allowed sustained as
  well as unplanned growth.
• We needed to align costs of services to mission
  groups, and budget long term consistent funding
  to ensure consistent service. SAN and HSM
  Architectures allowed storage to be commoditized.

• Information Technology roles in Health Care and
  Research Missions had grown to more than just
  convenience. Information Technology was now an
  infrastructure necessity and required strong
  continuity of services.

12
Where we were before Xiotech and IBM

• In 1999, we were a small IT shop with four
  systems administrators, six physical hosts, and
  only 80Gb of Direct Attached Storage for
  File/Email.
• Every service was affected by lack of flexibility
  in storage. We simply provisioned new hosts to
  get new storage.
• We supported about 550 users mainly
  administrative and educational missions. We
  couldnt facilitate extensive research or clinic
  operations because we couldnt react to the tempo
  of their operations.
• We were under constant pressure to accommodate
  growing needs, react quicker to storage
  requirements, and to accomplish more than we had
  physical resources to handle.
• Budget did not match service requirements at
  all.

13
How Storage Vendors catalyzed our successes

• Xiotech and IBM storage has allowed us to adapt
  quickly to changing conditions, physically and
  logically while maintaining sound fiscal
  management.
• Xiotech and IBM storage has allowed us to gain
  precision in our infrastructure costs and align
  them tightly with budget/service requirements.
• Xiotech and IBM has allowed us to grow into an
  enterprise organization supporting commodity
  services 24/7/365 across all Health Care Missions
  because of our consistent and easily managed
  storage architecture.

14
Leveraging SAN/HSM Storage for the Future

• Purchase 3th Magnitude unit to host Secure VA
  Research Data and valued Computational Biology
  Data.
• Expand IBM TSM and HSM solutions to migrate aged
  data to cheaper media.
• Finalize FISMA and ISO 27001 Data Center and
  Storage Certification for hosting of VA
  restricted Data and valued Computational Biology
  Data.
• Initiate eDiscovery processes to determine
  Archiving and Retention qualities across the
  Enterprise.

15
Questions?

• Dick Deason
• Assistant Director
• Systems Administration
• rdeason_at_ufl.edu