Bill Pierce

1
Emerging Practices and Standards for Information
Lifecycle Management

• Bill Pierce
• SNIA, TeraCloud Corp.
• September 14, 2005

2
Outline

• What is ILM?
• SNIAs ILM Modeling and Standards
• Implementing ILM at a Small Software Company
• Reactions?

3
SNIA Definition of ILM
The policies, processes, practices, services and
tools used to align the business value of
information with the most appropriate and
cost-effective infrastructure from the time
information is created through its final
disposition. Information is aligned with business
requirements through management policies and
service levels associated with applications,
metadata and data.
4
The Problem

• Information stewardship falls mostly on IT
• Data placed for convenience
• Too many configurations to support
• Nonstandard, unreliable results
• Fails to take advantage of economies of scale

5
Business Goals of ILM

• Reduce the cost of managing storage services
• Obtain an objective best fit between data
  requirements and storage services
• Ensure compliance with corporate governance and
  government regulation

6
SNIAs role in ILM evolution

• This is work in progress!
• Technical Working Groups
• Architecture
• Standards
• Product interoperability
• Marketing Forums
• Requirements analysis
• Use cases
• Best practices
• Education and certification

7
Data Management Forum Use Cases

• Primary Archetypes
• Business Process Analyst (business requirements)
• IT Architect (makes it happen)
• Records Information Manager (regulatory
  requirements)

8
ILM in Systems Administration

• Just as systems administration is the process of
  applying design and planning to meet computing
  application requirements at least cost, ILM is a
  similar process focused on information
  requirements.
• ILM provides a language for business, IT and
  records management to share the responsibility of
  information stewardship.

9
ILM TWG Modeling (work in progress!)

• Data Classification
• Data Lifecycle Management
• Composite Storage Services
• Storage Service Level Management
• Data Placement Services

10
Data Service Resource Mgmt
Business Applications
Data Service Resource Mgr
Mgmt
Data Services
Data Center Management
Mgmt
Composite Storage Set
Mgmt
Mgmt
Mgmt
DataPlacement Service
DataProtection Service
DataSecurity Service
Mgmt
Storage Storage Services
11
Information Lifecycle
Information Classification Collaboration
Key Performance Indicators (KPIs)
Business
Data Lifecycle
Data Center
SNIA will define standards for - Data SLOs -
Offered Data Svc Levels - Data Lifecycle Policy
Management - Composite storage configs - Mgmt
of underlying storage services
Data Service Level Objectives
Data Classification Lifecycleautomation
policies
Standard Configurations - Storage - Data
Protection - Data Security
12
Data Classification
Data ClassificationPolicies that group data so
as to determine what Service Level Objectives
must be applied.

• Examples
• Files in /usr/foo/bar
• Files in /user/fred older than 3 months
• Files named .mp3
• Files bigger than 1GB
• Files originated by a corporate officer
• Files in the engineering group
• Files with keywords in content
• Files about oil exploration

13
Data Classification Model
14
Standard Metadata org.snia.fsmtwg.
15
Standard attributes org.snia.ilmtwg.




16
Data Lifecycle Management
Data Lifecycle ManagementPolicies that define
Service Level Objectives for data over its
lifetime.
Data
Data SLO Policy
Data SLO Policy
Data SLO Policy
17
Data Service Level Objectives (examples)

• Budget - MaxBudget
• Accessibility
• ReadWriteRatio, AvgDataThroughput, SpaceUsage,
  SizeGrowth, Location
• Availability
• PlannedDowntime, MaxUnplannedDowntime, Aggregate,
  MaxUnplannedDowntimePerInstance
• Data Restore RPO, RTO
• Security Expiration, DestructionType

18
Composite Storage Service
CSS
Service
Configuration template
19
Composite Storage Set Attributes (examples)

• Storage
• Workload, AvgThroughput, InitialAccessTime
• Protection
• RPO,DataRestoreTime,MaxDataRetentionCapability,Dat
  aCopyLocale
• Security
• Accountability, Integrity, Physical, Destruction

20
CSS, ODSL, and SLOs Service Level Mgmt
Data
Data Service Level ObjectivesDerived from
requirements defined inInformation
Classification processes.Includes requirements
specific to thedata and an applications use of
data.
Data SLOs
Map to best fit
Service Classification
ODSL Offered Data Service LevelPart of the
service catalogue. Each serviceclassification
describes the service levelthat can be provided
to data.
Properties
Map per data center policies
CSS Composite Storage SetAn abstract storage
configuration templatedescribed by its overall
technical capabilities such as storage
performance, availability, data protection and
data security.
Capabilities
21
CSS, ODSL, and SLOs why 3 abstractions?
Data
Data Service Level ObjectivesRequirements for
data may or may not change independent of
service catalogue.
Data SLOs
Map to best fit
Service Classification
ODSL Offered Data Service LevelService catalog
may remain stable while supporting
configurations evolve.
Properties
Map per data center policies
CSS Composite Storage SetSupported
configurations evolve with technology towards
improving costs while meeting existing service
requirements.
Capabilities
22
SMIS 1.3 Expanded Data Services
Data Classification
Data Lifecycle Management
Data Service Level Management
23
Implementing ILM at a Small Software Company

1. Classify by Business Process Application
2. Define Lifecycles and SLOs
3. Define Lifecycles for backup copies
4. Classify storage or define ODSLs (lt10)
5. Match Phase SLOGs to ODSLs
6. Configure CSSs
7. Match ODSLs to CSSs
8. Implement Lifecycles
9. Operational Adjustments

24
Classify by Business Process Application

• Project documents
• Software archive
• Windows Updates
• Financial spreadsheets
• Customer contracts
• Home directories
• Web site
• Released build archive
• Accounting database
• File transfer

• OS Software
• OS Images
• Wiki Site
• Corporate Documents
• Recent Builds
• Source code files
• Source Control Database
• Exchange Email
• Temp Files
• Backups

25
Try to identify common superclasses

• Working Documents
• Project documents, Recent builds, Web site
• Archives
• Software archive, OS images, Released build
  archive
• Database Managed Data
• Exchange Email, Source Control Database
• Sensitive Data
• Customer contracts, Accounting database
• Temporary
• Temp files

26
Define Lifecycles and SLOs
27
Define Lifecycles and SLOs
28
Define Lifecycles for Backup Copies

• Will not discuss (much)
• These copies have their own lifecycles and
  requirements in each phase
• Includes need for technology refresh
• Physical (worn out media)
• Logical (antiquated data formats)

29
Define ODSLs
30
Best Practices for ODSLs

• You are trying to simplify and reduce the number
  of ODSLs you provide.
• There is a strong tendency to over-optimize each
  storage solution. Resist it.
• In the name of simplicity, you will have to
  over-provide for some classes of data

31
Match Phase SLOGs to ODSLs
Class Phase ODSL
Working Docs Tier 1 Active
Working Docs Tier 2 Archive
Archives Tier 1 Archive
Database Tier 1 Active
Sensitive Data Tier 1 Sensitive
Sensitive Data Tier 2 Sensitive
Temporary Tier1 Archive


32
Configure Composite Storage Sets

• This is where a lot of practical considerations
  come in.
• Space and throughput limitations
• Budget
• Technology (old and new)
• Physical and logical access
• Vendors
• SRM tools can help

33
Match ODSLs to CSSs

• Probably a manual process
• With standardization, more automation

34
Implement Lifecycles

• Work in progress
• Includes initial data placement
• Technologies available
• Scripting and Symbolic Links
• HSM and SAN Filesystems
• DMAPI

35
Operational Adjustments

• Further economies may be achieved by splitting or
  combining classes and ODSLs
• New applications come online
• New technologies become available
• With ILM you have the infrastructure to deal with
  these changes in a coherent way

36
Summary

• ILM brings information storage in line with the
  requirements of the data
• Almost any IT organization can benefit from some
  aspects of ILM practice
• SNIA is laying the foundation of ILM through the
  development of Best Practices and Standards

37
For more information

• www.snia-dmf.org
• www.snia.org

38
Questions? Reactions?

• Was this
• Useful?
• Obvious?
• Impractical?
• Something you are already doing?
• In need of standardization?

39
Abstract
Information Lifecycle Management (ILM) is an
effort to formally align the business
requirements of data with the storage services
that support it. It is not a product, but a
process that could benefit most organizations.
Failure to do this can result in lost data,
stolen data, unavailable data, overly expensive
data and too much useless data. The Storage
Networking Industry Association (SNIA) is working
to define Best Practices and standards to support
ILM. This talk will describe ongoing work in the
SNIA and its application to the data and storage
resources at a small software company.
40
Bio
Dr. Bill Pierce started his career as an
experimental physicist at Cornell University, the
University of Washington and Northwest Research
Associates, Inc. In 1995, he turned to systems
administration and software engineering,
supporting scientific projects during the early
growth of the Internet. From 1998-2002 Bill
worked on one of the first SAN Management
applications at Vixel Corp. Since 2002 he has
been actively involved in the Storage Networking
Industry Association and the development of
Information Lifecycle Management at TeraCloud
Corp. Bill is the author of the Open Source
storage utilities fcping and io_profile and
numerous scientific and technical articles. He is
a member of the Seattle Area System
Administrators Guild and a founding member of the
League of Professional System Administrators.