AFS at CERN and other High Energy Physics Sites

1
AFS at CERN and other High Energy Physics Sites

• Rainer Többicke / CERN

2
AFS Software

• Servers OpenAFS 1.2.6
• Focus stability
• Additional patches
• Local debugging
• Clients OpenAFS 1.2.7
• Linux RPMs
• Historical accumulation wide spread of partly
  ancient versions
• and system architectures

3
Service

• Total disk space
• 10 TB installed
• 6 TB usable
• 15 Sun Servers, 10 PC servers
• Sun A1000 RAID (-5) Systems
• Clients
• 4000-6000
• 2000 off-site

4
Usage

• Private user files - home directories
• 14000 users, 5000 active, 2TB disk space
• Scratch
• low quality space at descretion of major user
  groups, 1 TB disk space
• Software development
• Production environment
• Software
• Controlled by major user groups

5
Evolution of users
6
Evolution User space
7
Evolution Project space
8
The Future of AFS

• History
• Characteristics
• What do we require?
• Distributed file systems

9
History

• IBM sponsored development at CMU in the late 80s
• Transarc Corporation, later owned by IBM
• Strong academic (CMU, MIT, Umich) user community
• AFS 3.2 Introduced _at_ CERN in 1993
• Now almost ubiquitous in HEP
• OpenAFS - open source as of 2000
• Active development 4-5 releases/year
• Alternative implementations Arla, Red Hat
• Prime time quality software

10
AFS characteristics

• Global file system
• Security, flexibilty
• NAS approach
• No special infrastructure
• Moderate performance single path to data
• Performance (in 2002) for big files
• Slightly below NFS with caching 20-25 MB/s
• approaching RFIO without caching 30 MB/s

11
AFS characteristics contd

• Scaling
• Number of files in a directory 62000
• 2 GB file size limit
• No single directory bigger than a disk
• AFS cache size?
• Scaling non-issues
• Number of servers
• Performance limits
• No file striping file access limited by single
  server
• Client disk caching
• Flat client/server topology, no proxies or
  cascades

12
Requirements ?

• Global access
• Cascading / proxies?
• Security / access control
• Manageability
• Performance
• infrastructure files
• data files
• Scalability
• Cost

• Standard in AFS
• Kerberos, ACLs
• AFS volumes
• OK
• problematic
• Looks OK
• Free

13
Distributed File Systems

• Production, IP (NAS)
• NFS V3, V4
• Microsoft Dfs
• SAN based
• GFS, CXFS
• GPFS
• Recent Developments
• Lustre, Intermezzo , StorageTank
• ? PASTA III (2002) Distributed File Systems

14
Summary

• From a shaky interesting idea in 1993 AFS has
  grown to a stable system
• There is a need for a distributed file system for
  LCG
• Characteristics?