ProActive Integration with HP Software in the AGOS project Towards the integration of business services within a grid environment supported by a virtualized infrastructure

1
ProActive Integration with HP Software in the
AGOS projectTowards the integration of business
services within a grid environment supported by a
virtualized infrastructure

• Grids_at_Work 2008
• Sophia Antipolis October 21st, 2008

2
Agenda

• Project context
• Partner use cases
• Draft architecture
• Products mapping
• HP integration
• ProActive features

2
25 January 2014
3

Project context
Putting together experts from grids, services,
resources and data to define a simple, coherent
and realistic solution
4
Project context

• Selected by the Pôle de compétitivité Solutions
  Communicantes Sécurisées
• List of partners HP, Oracle, Inria, Activeeon
• Defined scope integration project
• Definition of an integration methodology
• Definition of new software components
• Integration of business and service dashboards
• Integration of services with virtualized
  resources
• Contribution to standards
• Objective Integrate GRID technology into a
  business oriented SOA stack running over a
  virtualized infrastructure

5
Proposed solution

• A reference architecture putting together
• business intelligence
• service oriented components
• grid environment
• virtualized systems
• A library of process and service templates based
  on standards
• A set of tools to easily build such applications
• A real-time monitoring of business processes,
  services and resources
• A methodology to migrate to the architecture

6
Applied onto water operator use case

• Simulation application to better handle water
  distribution network and provision adequate
  hydraulic meshing
• Focus is on software which allows the study, the
  planning and the control of irrigation
• hydraulic simulation software for meshed networks
  for distribution of raw water or drinking water
  under pressure
• full network is too large to be simulated at
  once split into sub-network parts that can be
  studied independently
• Main interest
• Parallelization of the application (SPMD
  approach)
• Overlap multiple runs (parameters sweeping) on
  subnetworks
• Better integrate with other applications
• Better use of resources to speed up computation
• Measured in terms of speed increasing of digital
  processing provided by the grid architecture

7
Applied onto global distribution systems usage
use case

• Parsing and analysis of web session logs from
  target applications
• Focus is on providing massive data analysis
• Customer session graph analysis or other data
  mining services
• Know how groups of customers behave the usage /
  navigation in websites
• Main interest
• Handle the size and the volume of data generated
  by core applications
• Parsing of logs done massively in parallel
• Results need to be stored in a distributed
  database
• Better understand how the products are used
• Help to improve the products continuously through
  the feedback loop
• Expectations
• Enable an advanced workflow beyond current
  standards to support processes and sub processes
  and/or to compose hybrid processes with services.
• Enable streaming results from any parallel
  activity on a grid to feed a distributed database
• Support operational aspects like monitoring,
  clustering, scalability, security, load balancing
  and failover thus enabling high availability SLA
  compliance agreements to be respected.
• Enables one button push start, stop and resume on
  any running grid process

8

Draft architecture
From building blocks to functional architecture
proposal
9
Base line

• Staged pyramid
• Adaptive infrastructure of heterogeneous
  resources
• Service delivery with quality added-value
• Business intelligence based on standardized
  processes
• Based on standards
• As defined in the related standards documents
• Existing implementations (proprietary /
  open-source)
• With grid middleware
• Grid environment parallelism, distribution
• Web interfaces thin web clients access
• Modeling facilities composition, adaptive,
  configurable
• Efficient execution design execution patterns

9
25 January 2014
10
of a Service Grid
Functional
Standards
BusinessIntelligence
BI
Grid Grid Grid Grid
ServiceArchitecture
SA
AdaptiveInfrastructure
AI
11
Main building blocks

• Systems Virtualization
• Grid Utility Interface
• Resource Manager
• Tasks Services Scheduler
• Enterprise Service Bus
• Distributed Shared Memory
• Database Interface
• Application Server Interface
• Web Service Interface

• SOA Registry / Repository
• SOA Governance
• Component Architecture
• Load Balancer
• Fault Tolerance
• Reconfiguration Process
• Business Process Execution
• Real-time Monitoring Tools
• Service Level Management
• Business Intelligence
• Reporting Tools

12
Define the business processes

• Processes defined in terms of business items
• Cope with business processes workflow
• Focus on business interactions between human and
  applications
• How are used the applications
• How are they chained in any given scenario
• What is expected as an output
• Not a programmatic task flow
• Do not cope with service implementation details
• Far from resource allocation concerns
• Handle specific concepts
• Long transactions (suspended time, recurring
  patterns)
• Requests approval (decision making, validation)
• Human interactions (subjective perception, biased
  usage)
• Business ecosystem (management, billing,
  visualization)

13
Business Intelligence
Reporting
BusinessIntelligence
Service Level Management
Business Process
- Delivered results analysis - Scenario
validation - Service Service Level monitoring -
Reporting on key aspects by category - Validation
of performance gain, result precision, time to
deliver - Comparison with reference benchmarks
ServiceArchitecture
AdaptiveInfrastructure
14
Calling value-added services

• Services stored in a registry with governance
  applied
• Manipulating proxies to available web services
• Services as composite components
• With predefined characteristics and features
• Interface contract
• Policy enforcement
• Delivered quality of service
• Mapped onto
• Existing web services (black-box)
• Composite web services (functional composition)
• Skeleton web services (parallel pattern)
• Subordinate processes (time composition)
• Simple sequential services

15
Implemented as grid services

• Components executed on a set of resources
• Skeletons providing parallel patterns for
  efficient execution on the grid
• Transformed as jobs delegated to the grid
  scheduler
• Optimized active objects exposed as web services
  on the grid
• Jobs scheduled on the grid with direct access to
  the grid scheduler
• Specialized grid schedulers dedicated for
  specific usage

16
Service Architecture
BusinessIntelligence
SOA Environment Repository, Registry,
Orchestration
Service Composition Service Component
ArchitectureParallel Programming Distributed
shared memory
ServiceArchitecture
- Distributed parallel services accessing data in
distributed shared memory - Functional
composition of service components with dynamic
reconfiguration - Exposed as web services in
registry with a policy enforcement contract -
Executed through proxies orchestrated by a
workflow engine
AdaptiveInfrastructure
17
Executing on heterogeneous resources

• Code execution on well-known resources or over a
  set of dynamically discovered resources
• As tasks through a scheduler managing
  load-balancing, fault tolerance, high
  availability
• As active objects with associated features such
  as migration, file transfer
• As uncontrolled legacy application
• As tasks delegated to another grid utility
  environment
• Evolving set of resources (join / leave support)
• With specific or adaptive characteristics
  (virtualized or clustered)
• On a set of various operating systems
• Protected or not, intrusive or not

18
Adaptive Infrastructure
BusinessIntelligence
- Virtual resources managed dynamically- Tasks
executed on the set of evolving resources-
Requests of tasks execution received via the bus
ServiceArchitecture
Task Scheduling
Resource Manager
AdaptiveInfrastructure
Grid Utility interface
Resource Virtualization
19
Proposed architecture
20
Products integration mapping

• Global contribution
• ActiveEon products from the ProActive Suite
• Oracle products from the Business Services Suite
• HP products from the Virtualization, Quality and
  SOA Centers
• Alternatives from the open-source among others
• ActiveBPEL, Cockpit
• Sequoia
• Tuscany
• Xen VM

21
Proposal
Reporting
BusinessIntelligence
Business Process
SLA Mgt
BPEL Engine
SOA Environment Repository, Registry,
Orchestration
Service Composition Service Component
ArchitectureParallel Programming Distributed
shared memory
ServiceArchitecture
Real-Time Monitoring
Quality Testing
Enterprise Service Bus
Task Scheduling
Resource Manager
AdaptiveInfrastructure
Grid Utility interface
OS Virtualization
OS, HW
22
Open Source
BusinessIntelligence
BPEL Engine
SOA Environment Repository, Registry,
Orchestration
Service Composition Service Component
ArchitectureParallel Programming Distributed
shared memory
ServiceArchitecture
Real-Time Monitoring
Quality Testing
Enterprise Service Bus
Task Scheduling
Resource Manager
AdaptiveInfrastructure
OS Virtualization
Grid Utility interface
OS, HW
23
HP Integration Points

• Products Methodology

24
HP contribution

• Adaptive Infrastructure lab for the project
• Architecture blueprint draft
• Consulting Integration methodology
• HP Software products from
• SOA Center (Service Oriented Architecture)
• BSM Center (Business Service Management)
• VSE Center (Virtual Server Environment)
• Quality Center (Quality Test Framework)
• Objective Virtualization of services and
  resources of the parallel / distributed grid in a
  business context

25
AGOS blueprint draft
26
Products from HP SOA Center

• HP Systinet
• UDDI Registry
• As the de-facto SOA reference repository
• HP Policy Enforcement
• Policy Enforcement
• As the smart SOA governance runtime
• HP BAC for SOA
• Services Monitoring
• As the integrated SOA services monitoring
• Combining business, service and resource
  monitoring
• HP Quality for SOA
• Services Testing
• As the best-in-class testing environment

27
Integrated specifically within AGOS

• Provide a library of business processes templates
  including both functional aspects (gather input
  data, run simulation, perform pre-processing,
  analyze output data) and business aspects (warn
  user, check execution time, request for more
  resources)
• Define the library of predefined SOA services
• Declare all the SOA services to be used by the
  business processes in the SOA registry
• Define and declare all SOA policy enforcements
  (as proxy of services) on top of existing SOA
  services

28
Products from HP BSM Center

• HP Business Availability Center
• Integrated business dashboard
• Reporting for business metrics at every layer
• Business processes
• Services execution
• Resource consumption
• HP System Insight Manager
• Collect raw metrics from adaptive infrastructure
• Feeder of HP BAC systems for real or virtualized
  hardware problems

29
Integrated specifically within AGOS

• Define or discover the business service models to
  get monitored
• Get in SOA registry all the services to get
  metrics on the architecture components
• Push the indicator values into the BAC monitoring
  system (source, metrics name value)
• Define the SLA based on customers expected
  results
• Define the reports to deliver to the business
  managers

30
Products from HP VSE Center

• HP VM
• Virtualization of resources and operating systems
• Adaptive deployment of resources with hypervisors
• Support enterprise hardware and configurations
• Storage Area Network
• Blade Servers (Itanium, Pa-Risc, x86)
• Hardware Software High Availability
• HP RDP
• Resource deployment protocol
• Manage VM and installed software transparently
• Integration expertise with Xen environment
• Integration expertise with VMWare environment
• Thursday 23rd 1030 HP VSE by Francois Donze

31
Integrated specifically within AGOS

• Deploy hardware (cpu, ram, disk) on demand
• Deploy operating system / virtual machine or
  dedicated application through web services
• Integrate with ProActive resource manager
• Add, get or list available nodes
• Define constraints of usage or quality of service
• Deploy within the hour
• Schedule the deployment of x VM
• Get node with x ram y cpu

32
ProActive used features

• Framework in action

33
ProActive Suite
34
Integration with ProActive
Business Intelligence
BPEL Engine
SOA Environment Repository, Registry,
Orchestration
Service Composition Service Component
ArchitectureParallel Programming Distributed
shared memory
Service Architecture
Real-Time Monitoring
Quality Testing
Enterprise Service Bus
Task Scheduling
Resource Manager
AdaptiveInfrastructure
Grid Utility interface
OS Virtualization
OS, HW
35
Integration points with ProActive

1. Generate SCA component as a GCM component
2. Expose grid components in SOA Repository
3. Provide distributed shared memorys data to tasks
   components
4. Publish metrics from components, skeletons, tasks
   and resources
5. Make tasks scheduler accessible as a service
6. Expose virtual machines as resource nodes
7. Expose external grid services (like scheduler as
   a node resource hosting a tasks scheduler)
8. Provide IC2D aggregated metrics towards
   monitoring system
9. Provide introspection agent to test a particular
   service, component, task or object

36
Thank youQuestions / Answers