Data, File, and Storage Concepts

1
Data, File, and Storage Concepts
2
Topics

• What is data?
• What is information?
• What is a file?
• What is a database?
• What is a database management system?
• Basic data structures
• User-defined data types
• Files and file structures
• Database technology
• Database management systems
• Physical processing of data

3
Data versus Information

• Data a collection of facts
• Information data organized and presented in a
  format such that it can be used for analysis
  and/or decision making

Data and information are different!
4
What is a file?

• Physical structure
• Size
• Definition (type)
• Resident on secondary storage
• Magnetic
• Optical
• Managed by the operating system
• FMS File Management Subsystem

5
What is a database?

• Physical structure that contains data
• Organized template to facilitate operations on
  data
• Connotations of structure and access methods
• Static capability of information systems
• Exists within a file (set of files) defined to
  the operating system

6
What is a database management system?

• Set of software that
• Defines the structure of a database
• Provides tools for access to stored data
• Provides tools to store data
• Defines accessibility for utilization of stored
  data
• Provides a record of processing operations
  utilizing a database
• Provides a set of utilities for supporting
  operations
• Executes in memory as an executive software
  application at a very high priority
• Service capability utilized and shared by
  applications (problem programs)

7
Basic Data Structures

• Array two-dimensional construct consisting of
  rows and columns (a/k/a table)
• Homogeneous all entries of same type
• Heterogeneous entries of different types
• List a collection of entries arranged
  sequentially
• Queue a list arranged in such a way that the
  first item placed in the list is the first item
  to be removed (FIFO algorithm)
• Stack a list arranged in such a way that the
  last item placed on the list is the first item to
  be removed (LIFO algorithm)
• Tree a collection of entries arranged in such a
  way as to have a clearly defined hierarchy
• Ring a collection of entries arranged in such a
  way as to continuously point to the next entry

8
Data Structures Representation

Queue
Stack
List
Array
Hierarchy
Ring
9
Storage based on data structures - arrays
x
A one-dimensional array. Each cell has an (x,1)
address
y
A two-dimensional array.
(x,y)
Expansion in the y-direction asnew rows are
added.
Each cell has an (x,y) address
10
Arrays in memory
x
An array data structureexists in memory as
y
(x,y)
Row 1 Row 2 Row 3 etc
. . .
11
Storage based on data structures queues and
stacks
Item is removed from the top

• Queue list structure based on a first in,
  first out algorithm
• Stack list structure based on a last in, first
  out algorithm

Aged items move to topof list for service
Aged items move to topof list for service
New item is added after the last entry
Item is removed from the top

Aged items move to bottom of list for service
New item is added above the first entry
12
Linked lists as a data structure concept

• Generalized name applicable to
• Hierarchies
• Rings
• Utilizes a pointer to establish the order of the
  list
• Pointer address
• Identifies next in sequence

13
Storage based on data structures - hierarchies

• Hierarchy unidirectional data structure based
  on higher-order precedence
• Characterized by the concept of persistence
• Attributes of higher level elements are retained
  by lower level dependents
• Explicit dependency

Root
Leaves
subordinate addresses
data
addr
14
Storage based on data structures - rings
Head (if headed)

• Ring elliptical data structure representation
• May or may not include concepts of precedence and
  persistence
• May be either unidirectional or bidirectional

present only if bidirectional
present only if headed
always present
data
addr
nextaddr
prioraddr
headaddr
15
Storage with linked lists
CASE 1 Insert a new record between the head
and first record
CASE 2 Insert a new record between the head
and first record with back pointers
16
Storage with linked lists
CASE 1 Insert a new record between the root
and first record
17
Custom data types

• Specific items of data
• Support problem solution
• User-defined
• Represents a processing variable
• Programmatic structures aligned with data
  structure types

18
Files and File Structures

• Sequential
• ISAM Indexed Sequential Access Method
• IDAM Indexed Direct Access Method
• HISAM Hierarchical Indexed Sequential Access
  Method
• HIDAM Hierarchical Indexed Direct Access Method

19
Database Technology

• Mechanism for storing data based on inherent
  relationships
• Historical significance
• Where it began
• The grand vision
• Todays reality
• The subject approach data as a reflection of
  the enterprise

20
Basic Concepts

• Data - set/collection of facts
• Information data organized and presented in a
  way that supports analysis and/or decision making
• Database logical structure for organizing the
  storage of data
• Data
• Storage structures
• Database Management System software application
  that permits the definition of and access to
  stored data
• Data definition
• Data manipulation
• Data control
• File physical structure for storing data

21
Files and Databases
Databases exist within files
22
Static and Dynamic Elements

• Static elements incapable of self-change
• Dynamic elements cause changes
• Files static element
• Database static element
• Software dynamic element
• Database management system
• Applications

23
Static and Dynamic Elements
FMS File Management System DBMS
Database Management System For the
Database Management System For the
Application
24
Spreadsheets and Databases

• Spreadsheet set of rows and columns expressed
  as a one-dimensional table
• Database set of data expressed as entities and
  attributes that contains included relationships
• Can be thought of as multi-dimensional

25
Spreadsheets and Databases
Spreadsheets
Databases
26
Database, Not Spreadsheet

• Accuracy
• Completeness
• Consistency

27
Data Essentials

• Attribute an atomic element of data
• Entity set of related attributes
• Relationships intersections among entities

Data Model
Element
28
Data Model

• Representation of (an enterprises) data and the
  relationships among them
• Conceptual schema
• External schema
• Constraints
• Operations on data
• Storage Definition Language (SDL)
• Influence physical definitions

Data Definition Language (DDL)
Data Manipulation Language (DML)
29
Database Design Process
ERD

• Computer Analysts and User Community
• Describe physical characteristics of data
• Identify database and DBMS considerations
• Update data representations
• Modify entity-relationship diagram

• User Community
• Identifies business problem
• Identifies business functions
• Identifies participating data

• Computer Analysts
• Understands business problem
• Understands business functions
• Understands participating data
• Prepares entity-relationship diagram

LDBD

• Computer Analysts
• Reviews application requirements
• Reviews workload projections
• Models systems performance (simulations)
• Modifies logical database design
• Reviews security requirements
• Creates physical database design
• Implements database schema

• Computer Analysts and User Community
• Refines schema f (processing requirements)
• Adapts design to meet workload and capacity
  requirements
• Integrates database design with applications
• Develops security plan
• Develops test plans identifies test data
  requirements
• Prepares test data

PDBD
Logical Database Design Physical Database
Design
30
Levels of Abstraction in a DBMS
External Model f (User View)
External Schema 1
External Schema 3
External Schema 2
Conceptual Schema

• Data independence - Logical - Physical

Conceptual Model Describes data model f
(DBMS)
Physical Schema
Physical Model Describes data model
f (physical characteristics)
Database
31
Database in the scheme of things
Database
Users Application
Database Management System

• Coded logic to solve users need
• includes DML to instruct DBMS access
  database

32
Structure of a DBMS
33
The Physical Environment
34
DBMS Internals
Physical I/O performed by OS
I/O
Operating System
DBMS Executive
DBMS Work Area
Database
Database Schema
Schema provides view of enterprise data
Application Program Work Area
35
Memory Map
Operating System
Network Manager
Database Manager
Application Programs
36
Retrieving a Logical Record from Disk Storage
37
Storing a New Logical Record
38
Changing Data in an Existing Logical Record
39
Database Types

• Inverted List
• Internal data dictionary
• Course index
• Fine index bit map
• Network
• Ring structure
• Multi-directional
• Physical addressing
• Sets
• Master
• Detail
• Hierarchy
• Hierarchical structure
• Uni-directional
• Physical addressing
• Roots and leaves

• Relational
• Table (array) structure
• Logical addressing
• Ability to model network or ring structures
• Object
• Based on business item
• Logical addressing
• Access based on interface to object
• Object-Relational
• Hybrid
• Characteristics
• Object
• Relational

40
Database Components
User data
Metadata
Indexes Overhead data
Application Metadata
41
Database Models

• Network
• Hierarchical
• Inverted List
• Relational
• Object
• Object Relational

42
Network Model

• Characterized by establish a network of data
  components
• Master
• Detail
• Physical addresses and pointers
• Next
• Prior
• Master
• Linkages
• Forward
• Backward
• Retrieval
• Random based on primary key
• Direct based on a known value

43
Network Model basic structure
44
Hierarchical Model

• Characterized by establish a network of data
  components
• Root
• Leaf
• Physical addresses and pointers
• Linkages
• Downward from root
• Retrieval
• Sequential search from root
• Random based on primary key value
• Direct based on a known value

45
Hierarchical Model
46
Inverted List Model

• Identifies records with specified key values
• Few records with a given value

Coarse Index

• Data
• sets of files containing records

Runtime Data Dictionary

• Identifies records with specified key values
• Many records with a given value - Bit map

Fine Index

• Describes data items
• Identifies location - Implied relationships

47
Relational Model (1)

• Tabular format
• Columns
• Rows
• Cell
• Intersection of a row and column
• Relationships based on data values
• Points to specific row(s)

48
Relational Model (2)
Index
Index
Identifies row with specific key value(s)
Identifies row with specific key value(s)
Table 1
Table 2
Relationship
49
Relational Model
Databases/DBA
Programs
Presentation of data
Preparation and processing of data
Enforcement of business and integrity rules
Management of access to data
50
Object Model
Databases/DBA
Programs
Presentation of data
Preparation and processing of data
Enforcement of business and integrity rules
Management of access to data
51
Relational versus Object
Purchase Order
Employee
Materials
Purchase Order
Contract
In an object database, the data isstored as the
business object itself,encapsulated from the
software so thataccess is via an interface
language
In a relational database, the datais stored in
several structures andrelated to the business
object via software
52
Object-Relational

• Hybrid
• Relational model
• Object model
• Evolved from competing camps
• Technology state-of-the-practice
• Marketplace realities

53
Language of Database

• Data Definition Language (DDL) used to define
  the structure and characteristics of the database
• Data Manipulation Language (DML) used to
  operate on the data stored in the data base
• Data Control Language (DCL) used to manage
  access to and use of the database and its data

54
SQL

• Lingua franca of relational databases
• Incorporates data management languages as
  sub-elements
• DDL
• DML
• SDL
• DCL