Lower%20Level%20of%20Data%20Movement

1
Lower Level of Data Movement
Enstore Project Status A. Moibenko
2
Current Hardware Summary
3
Data in enstore

• Data stored 1.3 PB (passed 1 PB on August 20)
• data rates 20TB/day

4
New Hardware

• DB server host for each system
• external RAID arrays
• increase reliability
• more flexible and convenient service
• decrease downtime
• Admin group needs to review spare servers, RAIDS

5
Software

• Accounting, drivestat, file, volume, pnfs DB
• enstore servers
• encp
• enstore administrative features
• data migration

6
Data Bases

• Deployed Postgres DB
• Accounting DB is finished
• File and volume clerks are ready to be switched
  from libdb to Postgres
• Drivestat DB is finished
• New version of PNFS server in production

7
System

• Better error handling and diagnostics
• Handling drive firmware errors

8
Encp

• V3_1 released.
• First major release in a year
• optimized for large RAID arrays (dcache)
• direct i/o support on XFS (CDF dcache pool nodes)
• multithreaded
• Better error handling
• more effective load balancing on multihomed nodes
• Improved diagnostics
• dcache features added

9
Misc.

• Administrative components
• better system start / stop
• volume audit
• user tools
• information about file in enstore for SAM -
  file_info
• entv

10
CDF Tapes Migration

• Migration of data from 9940A to 9940B tapes
• needs special attention.
• Pnfs and file DB integrity.
• Automation is needed.

11
Security

• Enstore security assessed
• "Current State of Enstore Mass Storage System and
  Risk Reduction Plan" document written
• Flipping write tabs. (March)
• tab flip procedure needs updates
• tab flip audit runs on General production system
• restriction of physical and functional access to
  critical hosts and enstore servers
• backup of all metadata and code

12
Configuration management

• Use rgang (rsh) and CVS for installing and
  upgrading SW.
• Admin group needs to implement better
  configuration management to load/boot/upgrade
  nodes

13
Upcoming work

• federation of all systems into one
• configuration management (admin. Group)
• auto migration
• system support and features addition
• dispersal of the system