Component Frameworks and Programming Approaches for Application Web Services

1
Component Frameworks and Programming Approaches
for Application Web Services

• Rainer Schmidt,
• Austrian Research Centers ARC
• rainer.schmidt_at_univie.ac.at

VGE-CCA
2
Outline

• The VGE-CCA Component Framework
• VGE and VGE-CCA extensions
• CCA libraries, Infrastructure Elements, Client
  Environment
• Application Component Libs (clibs)
• Plugged into the Web Service Container
• Distributed Application Coordination
• Programming Model
• Component Creation and Composition, Handling
  State
• Data Staging, Coupling, Infrastructure Services
• The Planets Interoperability Framework
• A Service-oriented Architecture for Digital
  Preservation
• Component and Programming Model
• Configuration and Execution Environment

3
Common Component Architecture

• CCA is designed for high-performance application
• Facilitates import of scientific code into
  frameworks.
• Provides a plug-and-play environment
• 3 party composition capabilities, based on
  Interactions
• CCA Fundamental Concepts
• Components Units of software that can be
  composed
• Frameworks Environment for components to run and
  interact
• CCA Specification defines behavior for component
  and framework interaction
• Platform and language neutral.
• does not prescribe implementation details
• Applicable to parallel and distributed/Grid
  environments.

4
VGE - On-Demand Supercomputing

• HPC Service provision framework
• Encapsulates native HPC applications on parallel
  hardware
• Offers common set of services (conversational)
• job execution, monitoring, file transfer, QoS,
  error recovery
• Negotiable QoS support for time-critical
  applications
• Advance Reservation, Perf. Models
• Dynamic Configuration, WSLA
• Standard Web service technology
• AXIS, SOAP attachments, WS-Addressing
• Security TLS, WS-Security, X.509 certificates,
  End-to-End
• Application services, Registries, High-level
  client API
• EU Projects GEMSS, Aneurist

5
VGE-CCA

• Distributed CCA framework
• atop a Web service based Grid environment
• Models VGE application services as distributed
  CCA components.
• Adds port-based composition capabilities to Web
  services.
• Plugin mechanism for application-specific
  libraries
• Model for Distributed Grid Programming
• Actively interacting Grid application components
• sequentially, concurrently
• Co-scheduling concurrently running Grid
  components.
• Client composition and computational steering.

6
The VGECCA System

• Service-side CCA Libraries
• No modification of VGE core functionality (still
  a valid VGE service)
• Adds Builder service (creation and connection)
• Provide CCA framework to the application service.
• Transparently utilize remote framework services
  (registry, proxy)
• Component Libraries (clibs) Plugin Mechanism
• Extend VGE comp. with app. specific. logic,
  ports, dependencies
• Method for distributed application coordination.
• Framework Web Services
• Component Registry, Proxy Registry, Negotiation
  Service
• Component-based Client API
• Distributed programming by interlinking parallel
  app. components.

7
Anatomy of a VGE-CCA Component
binaries, files
OS-levelinteraction
VGE Ports (App, QoS, Recovery)
App.Handler

VGE libs
Creation Connection Port
service specific comp. code
connected Services
App. Specific Provides Ports
Port Conn. Table
CCA Types
CCA Services
CCA lib
Fw. Web Service (component, proxy repository)
8
Creation and Composition

• Component Creation by Contextualization
• Conversational Identifier, WS - Addressing
• Different Types of Connections
• stateful stateless interactions
• sequential concurrent running applications
• Port Connections to Infrastructure Services
• Connect Grid component to Infrastructure Web
  service
• E.g. CRL-Service, Logging Service
• Connect using port mechanism
• visible dependencies
• Configured descriptively or dynamically found

9
Application Coupling

• Coupling Application Parts
• Co-scheduled application components.
• VGE Advance Reservation
• Selection QoS Negotiation Service
• Port-based Interactions of concurrently running
• Components interact as needed
• Application Specific Component Plugins

Ant Colony Optimization - Component signals
current best solution
10
Example Ant Colony Optimization

• ACO algorithm for solving a VRP.
• MPI-based application withmixed parallelization
  strategy
• Several colonies of ants cooperate in
• finding good solutions.
• Cooperating colonies periodically exchange single
  solutions.

• Basic Decomposition Approach
• Partition each colony of ants into n (
  processors) sub-colonies (fine grained)
• Distributed coarsely grained families.
• Provide Ant Colony as service.
• Exchange information between colonies

11
Distributed Coordination
Ant Application Components exchange solutions as
required. Client Application remains unaware of
application specific interaction logic Client is
not involved into computation, may go
offline. Only required for application steering
(start, stop, )
Idea Aggregate Distributed Resources
12
Application Specific Plugins

• Pluggable Component Libraries (clibs)
• hooks needed to extend an application service
• intercept port invocations
• add custom ports and dependencies
• application specific component interactionusing
  the service-side CCA Framework
• "Signal" Interface added for Ant components
• Provides and uses Port
• Implemented as one-to-many connection
• Daemon keeps track of local optimum andsignals
  if required.

13
Programming Model

• VGE-CCA Programming Model
• Abstract Specification (ports, apps, quality)
• registry and negotiation
• Spatial Composition
• port-based connections, dynamic proxy
• Control-Flow (Computational Steering, Monitoring,
  Data Staging)
• Client workflow orchestration
• Separates applications composition from runtime
  control flow steering
• VGE-CCA provides an extensible API for basic CCA
  constructs and specialized components.

14
ACO Client Application
//component description cmpDsc.addProperty(CConst.
APP, "ACO") qosDsc.setBeginTime(beginTime) qosDs
c.setWallTime(wallTime) cmpDsc.addProperty(CConst
.qosDsc, qosDsc) //creation and application
assembling VgeComponent ant1 builder.createInsta
nce(cmpDsc) ... AntComponentGroup antComposite
new AntComponentGroup(ant1, ant2,ant3) //client
workflow antComposite.upload(vrp_infile) antCompo
site.start() while(antComposite.getStatus !
...)
15
Planets Project

• Integrated Services for Digital Preservation
• IST FP6 Project - IP 14m
• Consortium of Libraries, Archives, Universities,
  Tech. Companies
• http//www.planets-project.eu
• OGF Preservation Environments RG
• Focused on Data Grids
• Sub-Project Interoperability Framework
• Defines a SOA for Digital Preservation
• builds atop Application Web Service Model
• Interoperability, Virtualization

16
Distributed Digital Preservation System

• Integration of heterogeneous environment
• Existing Applications, 3rd Party Services,
  Repositories
• Massive Data Amounts, Complex Object Types
• Results in Complex Grid Applications and
  Workflows
• Strong focus on data provenance
• Develop a Common Data Model
• Ensure repository consistency
• Provide Transaction model
• Component-based Programming Model
• Abstract Complexity
• Configuration and Execution
• Plug-and-Play Environment

17
Planets Interoperability Framework

• Generic, distributed portal framework
• Ties together the involved components
• Migration, Emulation, Characterization Services
• Preservation Planning, Format Registries,
  Repositories
• Common Services User Management, Service
  Registry, Workflow Enactment, Security
• Provide Programming and Execution Environment
• Currently based on EJB/Web Service Model
• Add Independent DP Component Model
• Programming and Workflow features

18
IF Architecture
19
App. Service - Abstraction Levels
Level 0 Application
Level 1 Web Services
Level 2 Planets Components
A
B
C
Workflow - Level
Planets IF - Instance
Planets IF - Instance
20
Components Model

• Abstraction based on layered approach
• Level 1 Any network accessible applications
  (services)
• Level 2 Compostable HL Abstractions (components)
• API/Type System for digital preservation
  components
• High-level with explicit context dependencies
• Defined interfaces and composition
• Exchange/manipulate a common data model.
• Life cycle, exception handling,
• Abstracts data model on component level
• Implicitly maintain information about provenance
  of data
• Automate Repository Interaction
• Consistency, Transactions, Abstraction

21
Programming and Execution

• Levels of application abstraction and
  construction
• Different user groups with different expertise
  PA -gt Component developers PP -gt Workflow
  developers Archivists, Libarians -gt User.
• Workflow Templates
• Assembled from high-level preservation components
• Independent from underlying services/middleware
• Rely on provider concept
• Workflow Configuration and Instantiation
• Automatically by Preservation Planning Services
  OR
• Based on Graphical Interface (Portal)
• Process archive consists of template and
  associated view.

22
Conclusions

• Light-weight App. Service Containers
• No changes to underlying application (legacy
  apps.)
• Advanced Grid Services (SLA, interactions)
• Provide High-Level Programming Model
• Req. Extensibility Based on Common Independent
  Concepts
• Abstract and Independent from underlying
  Middleware
• Allow Encapsulate Application Complexity
• Distributed Coordination
• Implicitly interact with Infrastructure Services
• Potential for Cloud Computing
• Bridge SOA and Component Technologies
• Composition, Dynamic Resource Selection

23
(No Transcript)