Active Data Object based on Intelligent Storage Concept

1
Active Data Object based on Intelligent Storage
Concept

• Intelligent Storage Consortium
• July 20, 2002
• David H.C. Du

2
What is happening?

• Computing devices with large storage capacity
  becoming pervasive
• Wireless and mobile devices becoming popular
• Storage systems becoming cheaper and larger
• The volume of available data becoming extremely
  large and hard to manage

3
Future Computing Environment

• Global Internet reach everywhere
• Pervasive Computing include many appliances with
  wireless ad hoc networks
• Intermittent Connectivity
• Large storage capacity in each device
• Data duplication is a must
• Dynamically changed user demand

4
Our Initial Focus

• Develop and extend the OSD (Object Storage
  Device) Standards
• Apply the OSD concept to future computing
  environment
• Investigate applications and environment that can
  benefit directly

5
Object Storage Model
Introduction

• An object is a logical unit of storage
• Lives in flat name space with an ID
• Contains data and metadata (similar to an inode)
• File-like methods open, close, read, write
• An OSD stores objects and could be any of
• Disk drive, storage appliance, storage
  controller,
• OSDs enables high performance and cross platform
• Use the higher level abstraction we needed
• Offload read write from the storage server

Objects enable self-managed storage.
6
OSD System Architecture
Introduction ? Object Storage Model
I/O Application
Storage System
Manager
OPEN/CLOSE
Storage Device
READ/WRITE
MANAGEMENT
The Manager is not in the data path.
7
Implications of OSD
Introduction ? Object Storage Model

• Intelligence at the device
• Requires silicon and software
• Compute power scales with number of devices
• Opportunity for hints, QoS, active disks
• No server to validate each I/O request
• Need new security model
• Manager grants clients access
• OSD performs the validation

Are objects worth all of this effort?
8
The Next Generation File Server
Introduction ? Object Storage Model
Control Operations
Application Servers
Data

File System Management
Yes!
9
Objects, Methods Sessions
Architecture

• Objects are logical units of storage
• Comprise attributes and data
• Inherent default attributes from parent object
• Three types of objects
• Root Object one per device
• Group Object a directory of objects
• User Object for user data
• Objects are accessed via methods

10
The Root Object
Architecture ? Objects, Methods Sessions

• Each device has exactly one root object
• Attributes contain at least
• Object and group ID of zero
• Total capacity
• Maximum number of objects
• Characteristics such as QoS features , etc.
• Data contains
• List of valid group object IDs

11
The Group Object
Architecture ? Objects, Methods Sessions

• Group objects are created in the root object
• Attributes contain at least
• Unique group object ID
• Current maximum number of user objects
• Data contains
• List of user objects

Group objects contain user objects with similar
attributes.
12
The User Object
Architecture ? Objects, Methods Sessions

• User objects contain user data
• Attributes contain at least
• User object ID
• Logical size of user data
• Characteristics such as QoS requirements
• Data contains
• User data

13
OSD Methods
Architecture ? Objects, Methods Sessions

• Create Remove
• For both objects and groups.
• Open Close.
• Read, Write, Append
• Get Set Attributes
• For both objects and groups

Objects use file system semantics.
14
Sessions
Architecture ? Objects, Methods Sessions

• State maintained on OSD for data transfer
• Specifies set of QoS agreements
• Optionally created by Open method
• Bounded by Close method
• Default session used if none specified at Open
• Not persistent across resets

Sessions are still controversial. Good for QoS.
But more work for the OSD.
15
OSD Security
Architecture

• Security is necessary to guarantee
• Authentication you are who you say
• Authorization you have permission
• Integrity data is not corrupted/modified
• Privacy data is not be seen
• Authentication handled by OSD transport

16
OSD Security Illustrated
OPEN
DATA

SECRET KEY

• Recalculate MAC using READ Permissions
• Compare to READ MAC

SECRET KEY
17
Proposed Extension to OSD

• Data Ownership
• Data Access Rights
• Data Encryption Information
• Current Version Number
• Metadata Manager Location
• Data Access Log
• Potential Data Processing Methods

18
Potentials of OSD in A Distributed Environment

• Global data identity and not associated with a
  computer system
• Data can be easily duplicated and re-synchronized
• Can contain ownership, access right, security and
  required performance information
• Can be easily distributed to satisfy larger
  demand

19
Potentials (Cont.)

• Can be cross-platform
• Can be active without file systems
• Can be efficient without servers and file
  systems as bottleneck
• Can be mobile

20
Proposed Work

• Apply OSD to the following environment and
  applications
• Environments Personal, Enterprise, and Internet
• Applications Database and Data Mining
• We will initially focus on enterprise market to
  make OSD a main stream product
• Build prototype devices based on OSD
• Management Module over Internet
• Meta data management
• User Account Information
• Concurrency Control
• Universal Location Identification
• Security

21
Storage Security

• Strong security needed for enterprise customers
• Support organization structure and projects
• Security during transmission
• Security at rest
• Key management and distribution
• Secure data deletion

22
Concurrency Control Issues

• Meta Data Management in Distributed Environment
• How to locate the near-by copy?
• How to manage multiple duplications?
• How to manage version control (data resyn)?
• Meta Data within objects/storage device

23
Data Distribution

• How to distribute data?
• How to distribute metadata?

24
Fault Tolerance

• Can you always access data?
• Are metadata always available?

25
Security

• Can OSD provide strong data security for
  Enterprise market?

26
OSD Issues

• How to handle complex objects (object of
  objects)?
• How to carry out methods on data objects?

27
Performance Management of Storage

• Performance
• Cached and duplicated for performance improvement
• Dynamically adjusted within a storage device
  (e.g., different stripping)
• Management
• Monitoring user demand
• Monitoring device performance
• Virtualization

28
Enterprise Market

• How to support organization structure and
  dynamically changed projects?
• How to charge (accounting issues)?

29
Internet Market

• How to implement pay-per-use?

30
Scalability and Automatic Configuration

• Scale up with user demand
• No server and file system overhead?

31
OSD Applications

• Relational Database
• Data mining?

32
(No Transcript)