File Processing : Storage Media

1
File Processing Storage Media

• 2008, Spring
• Pusan National University
• Ki-Joune Li

2
Major Functions of Computer

• Computation
• Storage
• Communication
• Presentation

3
Storage of Data

• Major Challenges
• How to store and manage a large amount of data
• Example more than 100 peta bytes for EOS
  Project
• How to represent sophisticated data

4
Modeling and Representation of Real World

• Example
• Building DB about Korean History
• Very complicated and Depending on viewpoint
• Database Course 2008 Fall semester

Real World
Computer World
5
Managing Large Volume of Data

• Large Volume of Data
• Cost for Storage Media
• Not very important and negligible
• Processing Time
• Comparison between main memory and disk access
  time
• RAM several nanoseconds (10-9 sec)
• Disk several milliseconds (10-3 sec)
• Time is the most valuable resource
• Example
• Retrieving a piece of data from 100 peta bytes DB

6
Managing Large Volume of Data

• Management of Data
• Secure Management
• From hacking
• From any kinds of disasters
• Consistency of Data
• Example
• Failure during a flight reservation transaction
• Concurrent transaction

7
Goals of File Systems

• To provide with
• 1. efficient Data Structures for storing large
  and complex data
• 2. Access Methods for rapid search
• 3. Query Processing Methods
• 4. Robust Management of Transactions

8
Memory Hierarchy

• Large Data Volume
• Not be stored in main memory
• But in secondary memory
• Memory Hierarchy

9
Flash Memory

• Non-Volatile
• Data survives power failure, but
• Data can be written at a location only once, but
  location can be erased and written to again
• Can support only a limited number of write/erase
  cycles.
• Erasing of memory has to be done to an entire
  bank of memory
• Speed
• Reads are roughly as fast as main memory
• But writes are slow (few microseconds), erase is
  slower
• Cost per unit of storage roughly similar to main
  memory
• Widely used in embedded devices such as digital
  cameras

10
Optical Storage

• Non-volatile
• data is read optically from a spinning disk using
  a laser
• CD-ROM (800 MB), DVD (4.7 to 17 GB), CD-R, DVD-R
• CD-RW, DVD-RW, and DVD-RAM
• Speed
• Reads and writes are slower than with magnetic
  disk
• Juke-box systems
• Large numbers of removable disks,
• Few drives, and
• Mechanism for automatic loading/unloading of
  disks
• For storing large volumes of data

11
Tape

• Non-volatile
• Primarily Used for backup
• Speed
• Sequential access much slower than disk
• Cost
• Very high capacity (40 to 300 GB tapes available)
• Tape can be removed from drive
• Drives are expensive
• Tape jukeboxes
• hundreds of terabytes to even a petabyte

12
Data Access with Secondary Memory
Access Request
How to increase hit ratio ?
If in main memory
MainMemory
If not in main memory
Disk
13
Why Hit Ratio is so important ?

• Example
• for(int i0ilt1000i)
• Nbytesread(fd,buf,100)

1000 10-2 sec 10 sec
1000 10-8 sec 10-5 sec
14
Physical Structure of Disk
15
Disk Access Time

• Disk Access Time
• t tS tR tT , where
• tS Seek Time
• Time to reposition the head over the correct
  track
• Average seek time is 1/2 the worst case seek time
• 4 to 10 milliseconds on typical disks
• tR Rotational Latency
• Time to reposition the head over the correct
  sector
• Average rotational latency ½ r (to find index
  point) ½ r r
• In case of 15000 rpm r 160sec/15000 4 msec
• tT Transfer Time
• Time to transfer data from disk to main memory
  via channel
• Proportional to the number of sectors to read
• Real transfer time is negligible

16
Block-Oriented Disk Access

• Example
• for(int i0ilt1000i)
• Nbytesread(fd,buf,10)

17
Disk Block

• Unit of Disk Access
• Block Size
• Normally multiple of sectors
• 1K, 4K, 16K or 64K bytes depending on
  configuration
• Why not large block ?
• Limited by the size of available main memory
• Too large unnecessary accesses of sectors
• e.g. only 100 bytes, when block size is given as
  64K
• 1 block 128 sectors (about ½ track, ½ rotation,
  2 msec)
• Too wasteful