Dimple Kaul

1

• Dimple Kaul
• dimple.kaul_at_vanderbilt.edu
• Vanderbilt University
• Nashville, Tennessee
• http//www.dre.vanderbilt.edu/dkaul/

2
Motivation

• Optimize/improve various features of L-store
  application using Aspect Oriented Programming
• Object Oriented Programming provide good
  modularity for system core functionality i.e.,
  the main business logic
• Adding system level functionality with core
  functionality result into a system which is
  difficult to understand and evolve and result
  into code tangling scattering

3
A typical distributed storage scenario on campus
at Vanderbilt
Fermilab
Workstations
L-Store Client Application
L-Store Client Application
L-Store Client Application
Analysis data from labs can be stored using
L-Store

• Storage and access of data can be accomplished
  using
• L-Store
• From remote labs
• Between data centers
• Between Labs

Firewall
L-Store at Vanderbilt
L-Server and Depot
Storage Media
4
What is L-Store?

• L-Store provides a distributed file system for
  storing arbitrary sized data objects.
• Provides a flexible, distributed and scalable
  storage solution
• Base functionality provides a file system
  interface to the data.
• Upload, Download, Mkdir, Deldir, ls, stat etc
• Base version of L-Store has a single metadata
  server
• Next generation of this system (to be showcased
  at supercomputing 06) will be scalable in both
  Metadata and storage.

5
(No Transcript)
6
Global scenarios for data storage

• Real time data transfer for applications such as
  tele-robotic surgery
• Redundancy in data storage across geographically
  distributed locations
• Remote data mining applications, for areas such
  as weather simulation
• High energy physics pile up samples and Image
  processing applications

7
Logistical Computing and Internetworking (LoCI)
tools
L-Store version uses an unchanged IBP layer from
the LoCI toolset and a modified version of the
ExNode layer

• L-Store uses LoCI tool set
• For managing and using remote storage
• Supports multiple threads/depot
• User configurable block size
• Designed to support large scale, distributed
  systems.

LoCI Tool sets
8

• Functionality Available
• Upload file
• Download file
• List file
• Stat file
• Make Directory
• Remove Directory
• Distributed storage
• Single metadata server

9
Meta Data management using AspectJ

• Transaction control (Persistence)
• Exception safety Logging
• Database connection pooling
• Authentication Authorization

10
Aspect-Oriented Development
11
Transaction control Persistence

• Correct transaction control is vital for
  maintaining the CONSISTENCY of a database
• Using AspectJ transaction control/persistence was
  introduced to L-store

Example
aspect DatabaseAspect pointcut
transactionalMethods () execution (/ pointcut
for transactional methods /) before()
transactionalMethods () initialiseDatabase(
) after() returning transactionalMethods(
) commitTransaction() after()
throwing transactionalMethods()
rollbackTransaction() //Can do exception
handling here
12
Database connection pooling

• Database connection pooling using AspectJ
• Upfront decision making
• Plug and play
• When and how to obtain connection from pool and
  how to put them back
• Conventional db connection pooling methods
  reduces time to obtain new db connections but
• Extra memory resources are used
• Enable connection pooling for the modules where
  the benefits of improved speed outweighs the cost
  of extra space
• Driver-supported connection pooling will act as
  secondary pooling because AspectJ will override
  the default connection pooling (JDBC 2.0)

http//www.dre.vanderbilt.edu/dkaul/presentations
/Database Connection Pool management.ppt
13
Exception Handling

• Catch any exception in all the public methods
• aspect CatchException
• void around() execution(public (..))
• try
• proceed()
• catch(Exception e)
• System.out.println("Printing exception")
• //Handle exception
• Catch SQL exceptions that need to be logged
• after() throwing (SQLException ex)
  inDataLayer()
• logException(ex)

14
Authentication Authorization(LACS)

• Provided basic authentication mechanism
• Implemented SSL protocol for secure communication
  between client server
• Designed and implemented basic authorization
  model

http//www.dre.vanderbilt.edu/dkaul/presentations
/Database Connection Pool management.ppt
15
Data explosion

• Focus on increasing bandwidth and raw storage
• Assume metadata growth is minimal

Metadata
Raw storage
16
Future work on Data explosion

• For collections of small files the metadata
  becomes the bottleneck
• Need ability to scale metadata
• Moving to Chord (distributed hash table) design

Metadata
Raw storage
17
Chord RingDistributed Hash Table

• Key (K) -hash(name)
• Nodes (N) are distributed around the ring and
  are responsible for the keys behind them.

18
Locating keys
N14
19
Current Development Status for single metadata
server

• Transaction Control (Done)
• Database connection pooling (Done)
• Basic Authentication (Done)
• Basic Authorization (In progress)
• Exception handling Logging (In progress)

20
Thank you