Dimple Kaul

1

• Dimple Kaul
• dimple.kaul_at_vanderbilt.edu
• Vanderbilt University
• Nashville, Tennessee
• http//www.dre.vanderbilt.edu/dkaul/

2
Motivation

• Optimize/improve and add various features to
  L-store application using Aspect Oriented
  Programming (AOP)
• OO programming provides good modularity for
  system core functionality i.e., the main business
  logic
• Adding system level functionality with core
  functionality results into a system which is
  difficult to understand and evolve resulting into
  code tangling scattering

• Storage and access of data can be accomplished
  using
• L-Store Application
• From remote labs
• Between data centers
• Between Labs

3
What is L-Store?

• L-Store provides a distributed file system for
  storing arbitrary sized data objects.
• Provides a flexible, distributed and scalable
  storage solution
• Base functionality provides a file system
  interface to the data.
• Upload, Download, Mkdir, Deldir, ls, stat etc
• Base version of L-Store has a single metadata
  server
• Next generation of this system will be scalable
  in both Metadata and storage.

4
Large Scale Installation
5
Logistical Computing and Internetworking (LoCI)
tools
L-Store version uses an unchanged IBP layer from
the LoCI toolset and a modified version of the
ExNode layer

• L-Store uses LoCI tool set
• For managing and using remote storage
• Supports multiple threads/depot
• User configurable block size
• Designed to support large scale, distributed
  systems.

6
Global scenarios for data storage

• Real time data transfer for applications such as
  tele-robotic surgery
• Redundancy in data storage across geographically
  distributed locations
• Remote data mining applications, for areas such
  as weather simulation
• High energy physics pile up samples and Image
  processing applications

7

• Functionality Available
• Upload file
• Download file
• List file
• Stat file
• Make Directory
• Remove Directory
• Distributed storage
• Single metadata server

8
Meta Data management using AspectJ

• Following are the features added using AOP
• Transaction control (Persistence)
• Exception safety Logging
• Database connection pooling
• Authentication Authorization

9
Transaction control Persistence

• Correct transaction control is vital for
  maintaining the CONSISTENCY of a database
• Using AspectJ transaction control/persistence was
  introduced to L-store

Example
aspect DatabaseAspect pointcut
transactionalMethods () execution (/ pointcut
for transactional methods /) before()
transactionalMethods () initialiseDatabase(
) after() returning transactionalMethods(
) commitTransaction() after()
throwing transactionalMethods()
rollbackTransaction() //Can do exception
handling here
10
Database connection pooling

• Database connection pooling using AspectJ
• Plug and play
• When and how to obtain connection from pool and
  how to put them back
• Conventional db connection pooling methods
  reduces time to obtain new db connections but
• Extra memory resources are used
• Enable connection pooling for the modules where
  the benefits of improved speed outweighs the cost
  of extra space
• Driver-supported connection pooling will act as
  secondary pooling because AspectJ will override
  the default connection pooling (JDBC 2.0)

http//www.dre.vanderbilt.edu/dkaul/presentations
/Database Connection Pool management.ppt
11
Exception Handling

• Catch any exception in all the public methods
• aspect CatchException
• void around() execution(public (..))
• try
• proceed()
• catch(Exception e)
• System.out.println("Printing exception")
• //Handle exception
• Catch SQL exceptions that need to be logged
• after() throwing (SQLException ex)
  inDataLayer()
• logException(ex)

12
Authentication Authorization (LACS)

• Provided basic authentication mechanism
• Implemented SSL protocol for secure communication
  between client server
• Designed and partially implemented basic
  authorization model

Security
SSL
External Authentication Server
http//www.dre.vanderbilt.edu/dkaul/presentations
/Database Connection Pool management.ppt
13
Data explosion

• Focus on increasing bandwidth and raw storage
• Assume metadata growth is minimal

Metadata
Raw storage
14
Future work on Data explosion

• For collections of small files the metadata
  becomes the bottleneck
• Need ability to scale metadata
• Moving to Chord (distributed hash table) design

Metadata
Raw storage
15
Chord RingDistributed Hash Table

• Key (K) -hash(name)
• Nodes (N) are distributed around the ring and
  are responsible for the keys behind them.

16
Locating keys
N14
17
Current Development Status for single metadata
server

• Transaction Control (Done)
• Database connection pooling (Done)
• Basic Authentication (Done)
• Basic Authorization (In progress)
• Exception handling Logging (In progress)

18
Thank you