Storage Systems as Permanent Archives

1
Storage Systems as Permanent Archives

• D. Petravick
• Fermilab

2
FNAL Permanent Archive

• 1.5 PB.
• Almost 100 TB ingest/month.
• Read-dominated archive.
• CRC part of meta-data

3
Experience W/ Disk

• In the context of buffering
• Experience here is a guide to thinking about
  permanent store.
• Capacious gt 300 TB deployed _at_ CDF
• Two deployment cased
• Stand-alone server nodes
• Storage system muxed onto compute farm

4
The Non-issue of Performance w.r.t permanent
stores.

• Permanent storage and buffering are two
  relatively independent uses of storage devices.
• Very high performance is merely an enhanced goal
  for a disk-based permanent store.

5
Qualitative Issues

• Tautology Permanence is the main quality of a
  permanent storage system.
• A Test -- is this a permanent store
• If the system loses information do we act as if
  the information may likely be truly lost?
• Send media for last-chance low level recovery.
• Begin a frantic search for replicas outside the
  system
• Stewardship efforts
• Systematic response to loss - what should be
  done differently?

6
Issues discovered by our thinking

• Commissioning problem
• User accident
• Stewardship.

7
Comissioning(1)

• Ability to add capacity to a permanent store
• Expects to have no faults when doing this.
• Tape- typical strategy
• Monocrop the technology.
• First level of expansion is simple passive item
• Disk systems
• TBD.

8
Commissioning (2)

• Disk based archives
• Many active components.
• Permanent archive not primary purpose.
• Relatively more variation in details of component
  implementation.
• Quality variation (lot-wise, revision-wise,
  model-wise)
• Type of physical support for unit of expansion
• tape shelf v.s.
• powered cooled floor space.
• Monocropping is not an obvious solution

9
User-Accident problem

• Ambitious software above the storage systems is
  being constantly updated.
• Permanent stores need to protect against loss
  induced errors in upper levels software and
  system.
• Common solution is a trash-can or journaling type
  solution.
• Authority and authorization to empty trash needs
  careful design
• At FNAL, this is a separate role. (human in
  loop)

10
Stewardship

• Active features of system to measure
  permanence-related properties.
• Examples
• Uniform sampling of archive
• Life-time of media.
• Security
• Write protect tab

11
Stewardship - File Corruption

• All files have CRC meta-data.
• Corrupt replica deleted. (replicas exist)
• Files placed on disk from tape are CRCed
• read back from disk, not fs cache
• Jan 04 2 errors, 100 TB read from tape
• Feb 04 (up to Feb 12th) 21 errors.
• Files residing on disk are read in the background
  and CRCed
• 2 errors/100 TB, 100 TB/month

12
Resiliancy and Availability through Replication

• Currently developing experience with availablity
  v.s. replica count.
• Experience with corruption is relevant to
  thinking about permanence v.s replication count.
• HEP envisions distributed stores, there will be
  other factors
• Security
• Varying level of other qualities from site to
  site.

13
Retirement

• Hardware will be retired from time to time.
• State must be moved as hardware is
  de-commissioned.
• State may have to be removed in presence of
  commissioning difficulties.

14
Summary

• Permanent storage systems differ from storage
  systems whose function is to buffer
• Some distinguishing characteristics are
• Layers of protection from loss.
• Integrated with work (have to be driven).
• Some analogs from tape storage systems can inform
  the development of permanent disk storage systems