SPITFIREAMI

1
SPITFIRE/AMI
Jerôme Fulachier Atlas Software Week Database
session
2
What is Spitfire ?

• Spitfire is a project of Work Package 2 within
  the EDG Project.
• Spitfire provides a means to access relational
  databases from the Grid. The Spitfire Task has
  the mandate to provide middleware to provide
  transparent, secure access to metadata for GRID
  middleware and applications. Currently it
  consists of the Spitfire Server module and the
  Spitfire Client libraries, a Browser and command
  line executables.
• (firsts slides are from wp2 web site)

3
What is Spitfire ? (2)

• Spitfire is a very thin layer on top of an
  RDBMS (by default MySQL) that provides a JDBC
  driver. It uses Web Service technology (Jakarta
  Tomcat) to provide SOAP-based RPC (through Apache
  Axis) to a few user-definable database
  operations. The Spitfire layer gives a quick and
  easy access to data, but is not supposed to be
  used as a fully-fledged Data Management System
• Spitfire is designed to give quick and easy
  access to (meta)data where the access patterns
  are simple single- or few-row lookups and
  inserts, updates and deletes. Information on the
  tables is also provided.
• Spitfire also has a 'browser' module that helps
  the developers to create JSP pages to expose the
  data through a secure web interface.
• Spitfire makes use of Security modules to
  enable full GSI authentication and additional
  authorization.

4
Why would I want to use it ?

•  Grid middleware and Grid application
  software often need access to persistent data or
  need to write data into a persistent store. For
  massive amounts of application data the
  applications will continue to use their own
  optimized data stores. But for short lived, small
  amounts of data and metadata that needs to be
  highly accessible to many users and applications
  throughout the Grid there is a need for an
  abstract high-level Grid database interface.
  Without such a service applications and Grid
  middleware services will continue to use dozens
  of varying and incompatible approaches
  necessitating complex and expensive translation
  and conversion steps.

5
How does Spitfire address this problem ?

• Spitfire provides a uniform way to access
  many RDBMS systems through standard protocols and
  well-published interfaces. Grid users are mapped
  to DB roles in a configurable way to support many
  different policies. Three SOAP services are
  defined a Base service for standard operations,
  an Admin service for administrative access and an
  Info service for information on the database and
  its tables. There is also the possibility to use
  a web browser to access the database through HTTP
  and canonical XML through a simple
  SQLDatabaseService.

6
Spitfire Architecture

• The Local Layer service connects through a
  RDBMS through JDBC (black solid line) using
  predefined roles. The client can connect directly
  to the local layer through any published
  interface, and perform database operations. In
  the future Global Layer services will be provided
  for more sophisticated distributed database
  methodologies. The Database Services are
  implemented as a Java servlet through AXIS. SOAP
  is used for remote messaging to ensure
  interoperability. We will apply the Open Grid
  Services Architecture guidelines for our
  interfaces as soon as its spec stabilizes. WSDL
• Depending on the requirements on the quality of
  the service, it can be deployed in freely
  available servlet containers (like Tomcat) or
  commercial servlet containers. The client
  libraries provide the necessary Grid
  authentication mechanisms to connect to a secure
  session through HTTPS. For normal unsecured
  sessions any HTTP client can be used (wget, curl
  et al).

7
How does it work ?
It is a web services base application composed
of three main components
Servlet Engine Tomcat with security modules
(Java)
Databases
HTTPS SOAP
xsql
Client-side APIs are provided for java and c
for the SOAP-enabled interfaces, there are also
some client-side tools for the SQLDatabaseService.
The Spitfire services are abstracting the access
to a local RDBMS.
User
8
It gives acces to Grid securty
CA
cert
CA
key
cert
HTTPS (SOAP)
User
key
Globus grid proxy
Grid proxy
DB Cluster
Grid users mapped to DB roles
Grid Security DB
9
How AMI uses it ?

• Before, the AMI client was using a JDBC
  connection to access AMI Databases (With a
  configuration file with a password and a user
  name).
• What Spitfire provides, is access to AMI
  Databases using secure connections with
  certificate authentification to various RDBMS .
  The user sends the same commands, with a grid
  enabled switch. This switch triggers a Spitfire
  connection in the client software instead of a
  JDBC connection.

10
AMI use Spitfire as a plugin
JAVA
AMI DBs
AMI / BKKJDBC
1
1
LOGIN PASSWORD
local cluster
3306
1
1
CERTIFICATE
HTTPS (SOAP)
8443
11
The AMI Architecture
AMI
project, process
references
1

BkkJDBC
12
AMI Status

• The core AMI package has been updated to deal
  with plugins. That makes AMI non dependent of a
  specific database connexion layer.
• The standard JDBC layer has been turned into a
  plugin. The interface of the AMI package has not
  changed, so applications using AMI are still
  compatible with the new system.
• A Spitfire server has been installed on a
  development database server, and a Spitfire
  plugin is in development.
• -gt will introduce before DC2
• The Mysql production server has been updated
  from 3.23 to 4.0.11. Replication will soon
  restart, but we need a system to switch in case
  of a crash (web interfaces (php/java).

13
AMI Status (2)

• At the moment AMI web interfaces provides
  generic query submission, but a specific
  interface is in developement for ATLAS production
  managers.
• -gt will introduce the 6/6/03
• An administration module for AMI is in
  developement too.
• -gt will come during the summer
• The TagCollector which is at the moment in PHP (
  but uses an AMI compliant database), will be
  rewritten in Java using AMI API.
• PHP was a cool stuff for XP developement with
  changing specifications, but now requirements are
  quite stable, we must introduce more modularity
  in its implementation for easier integration of
  new functionalities.
• -gt will start in September

14
Some questions remain about Spitfire

• At the moment Spitfire has a static management
  of databases and AMI needs in the long term a
  dynamic one.
• The client must be largely distributed.
• Server installation has to be standardized and
  made much easier.
• Advanced tools for user management are needeed.

15
Documentation / Installation

• You can find all information about Spitfire here
• EDG Work Package 2 http//edg-wp2.web.cern.ch/ed
  g-wp2/
• AMI documentation and API (web will be updated
  during this month)
• http//atlasbkk2.in2p3.fr/AMI/

16
Questions

• Is it efficient? Effort has been invested into
  enabling high performance and low latency. Java
  servlets are highly efficient and scalable
  solutions, and known to be superior to other
  approaches such as CGI scripts, modperl, PHP. To
  minimize the number of connection setup and
  tear-downs, persistent HTTP(S) 1.1 connections
  can be used (but not required). For similar
  reasons, the services use thread and JDBC
  connection pooling and advanced caching. The net
  effect is that requests and responses are passed
  through layers with little delay.
• Problems may arise for queries returning a very
  large result set, long-running queries and other
  complex operations like multi-way joins that need
  a large amount of system memory etc. The Spitfire
  module itself provides an example of a simple set
  of very common database operations.

17
Questions (2)

• Is it secure? The HTTPS configuration is fully
  GSI compatible. Spitfire uses the
  edg-java-security module of WP2, see the Security
  Task's webpages.
• Ok, I want to use it. What do I need to know?
• All of the Spitfire documentation is available
  from the documents page. Spitfire chooses a set
  of standard technologies to enable GRID access to
  metadata. These components are
•  Relational Database Management Systems (click
  here for an introduction)
•  JDBC - the Java Database Connectivity
•  Java Servlets
•  Web Service Technologies SOAP and WSDL
•  GSI - X.509 certificate based authentication
  (see Security Task)
•  Local Authorization - controls access to
  Spitfire resources (see Security Task)

18
Glossary

• AMI Atlas Metadata Interface
• AXIS Apache AXIS is an implementation of the
  SOAP submission to W3C.
• EDG European Data Grid
• GSI Grid Security Infrastructure
• JDBC Java Database Connectivity
• JMX Java Management Extensions
• PoolMan The PoolMan library and JDBC2.0 Driver
  and DataSource provide a JMX-based,
  XML-configurable means of pooling and caching
  Java objects, as well as extensions for caching
  SQL queries and results across multiple
  databases.
• RPC Remote Procedure Call
• RDBMS Relational Database Management System
• SOAP Simple Object Access Protocol
• WSDL Web Service Definition Language
• XML Extensible Markup Language