What is HPSS

1
What is HPSS?

• High Performance Storage System (HPSS) is
  software that provides hierarchical storage
  management (HSM) and archive functions for
  high-performance computing environments
• Manages hierarchy of disk arrays, disks, tapes
• Automatic data migration between levels

2
What is HPSS?

• The product of a collaboration between IBM and
  the Department of Energy. Developers/Owners are
• IBM Global Services - Federal
• Lawrence Livermore National Laboratory
• Los Alamos National Laboratory
• Sandia National Laboratory
• Oak Ridge National Laboratory
• Lawrence Berkeley National Laboratory
• NASA Langley Research Center
• An IBM commercial offering
• Open system with source code available to
  licensees

3
What isHPSS?

• Network-centered design
• Very fast for large files
• Scaleable architecture

4
What isHPSS?

• Network-centered design
• Very fast for large files
• Scaleable architecture
• Remote movers

5
What isHPSS?

• Network-centered design
• Very fast for large files
• Scaleable architecture
• Remote movers
• Subsystems

6
What isHPSS?

• Network-centered design
• Very fast for large files
• Scaleable architecture
• Remote movers
• Subsystems
• Linked remote HPSS

7

HPSS A Multi-Vendor Mass Storage Solution
8
(No Transcript)
9
Enhanced Scalability

• Why worry with Scalability?
• Petabyte systems are coming
• The DOEs Accellerated Strategic Computing
  Initiative (ASCI) has aggressive scalability
  requirements
• Our goal is to be the premier high-end storage
  system, period.
• Scalability and HPSS

• Huge file systems
• Parallel I/O
• Distributed movers

• Release 4.2
• Storage subsystems
• Collaborative HPSSs

10
Subsystems andLinked Remote HPSS

• Federated Name Space Diagram

Cooperative Storage Environment
HPSS Site 1
HPSS Site 2
HPSS Site 3
Name Server
Name Server
Name Server
WAN
Local Site Network
Local Site Network
WAN
Local Site Network
11

• HPSS serves as the deep, scalable store for the
  distributed multi-campus file system
• Serves the entire Indiana University community
  (gt100,000 users)
• Plans to extend the architecture to
  collaborating institutions
• Will link multiple HPSS locations with a common
  name server and name space

12
Standard vs. Parallel FTP Data Paths
13
Control and Data Flow
Application Client API
1
Name Server
get
Control
Data
2
read(IOD) (srcfile, sinknet addr)
1
open
Bitfile Server
read(IOD) (srcsseg, sinknet addr)
3
Physical Volume Repository
Physical Volume Library
5 mount
4 mount
Storage Server
6
6
read(IOD) (srcdevice, sinknet addr)
7
Network
Mover
7
Mover

• Simple Parallel Read Flow Diagram

14
Control and Data Flow (contd)
read(IOD) (srcsseg, sinknet addr)
Bitfile Server
6
1
write(IOD) (srcnull, sinksseg)
Storage Server
Physical Volume Repository
3 mount
Physical Volume Library
7
7
7
7
read(IOD) (srcdevice, sinknet addr)
Storage Server
2 mount
4
4
write(IOD) (srcnull, sinkdevice)
Mover
9
5
8,10
Mover
Mover
Network
9
IOR (net addr) 5
11
9
Mover
11
9
8,10
Mover
Mover
8 - connect to listen port send transfer
info. 9 - send data port addrs/info
10 - connect to data port 11 - send data
Control
Data

• Mover to Mover Protocol Flow Diagram

15
HPSS Metrics

• 400,000 executable lines of code in current
  release
• 200 person years of development and test since
  inception - IBM, US Department of Energy
• 30 people currently working HPSS development and
  test - IBM, US Department of Energy
• 41 Installations in US, Europe, Japan
• Quality
• SEI Level 3 assessment
• RD 100 award (RD Magazine)
• ISO 9001 certification expected by mid 2000
• Formal development and test procedures based on
  NASA and DoD standards
• HPSS is sold and serviced by
• IBM Global Services - Federal
• For more information
• www.clearlake.ibm.com/hpss
• Harry Hulen hulen_at_us.ibm.com