Automating Provisioning of a Complete Software Stack in Grid Environment

1
Automating Provisioning of a Complete Software
Stack in Grid Environment

• Bikash Agarwalla (Georgia Tech)
• Vanish Talwar (HP Labs)
• Sujoy Basu (HP Labs)
• Raj Kumar (HP Labs)

2
Outline

• Motivation
• Problem Statement and Research Issues
• System Architecture
• Installation and Deployment System
• Implementation
• OS installation
• Application installation and Deployment
• Experiments
• Related Work
• Conclusion and Future Work

3
Motivation
RedHat 9.0 JBoss
RedHat 7.0 Apache
RedHat 8.0 JBoss Apache PostgreSQL
4
Motivation
RedHat 9.0 JBoss
Software Installation Version Upgrade Software
Configuration Deployment Life Cycle Management
Termination Software Uninstall
RedHat 9.0 Apache PostgreSQL
5
Problem Statement

• How to Install, upgrade, configure and deploy the
  OS and application software on large number of
  machines
• With
• Automation
• When
• Resources are geographically distributed and
  heterogeneous
• Resources can dynamically join and leave
• Users may request for resources, with specific OS
  and application software, dynamically

6
Research Issues

• Handling
• Scale
• Heterogeneity
• Dynamism
• On demand provisioning of software stack
• Solution Requirements
• Automation
• Validation
• Access to Resources across Administrative
  Boundaries
• Generic Architecture in the Grid Framework

7
Outline

• Motivation
• Problem Statement and Research Issues
• System Architecture
• Installation and Deployment System
• Implementation
• OS installation
• Application installation and Deployment
• Experiments
• Related Work
• Conclusion and Future Work

8
System Architecture for Enabling Applications on
Grid
9
Installation and Deployment System

• Automating the OS installation
• Scaling it to thousands of machines
• Remote installation
• Installation without the need for a floppy or
  CD-ROM
• Automating 3-tier application deployment
• Dependencies
• Integration
• Download
• Install
• Configuration
• Deployment
• Lifecycle management
• Termination
• Uninstall

10
Installation and Deployment System Architecture
11
Interface within the Grid

• OS installation engine
• Sends
• Completion message
• Progress information
• Installs
• OS
• specific application (SmartFrog)
• Application Installation Engine
• Sends
• Completion message
• Progress information
• Installs
• 3-tier application
• Application Deployment Engine
• Sends
• Progress information
• Status information to other dependent
  applications
• Installs
• Life cycle management component

12
Outline

• Motivation
• Problem Statement and Research Issues
• System Architecture
• Installation and Deployment System
• Implementation
• OS installation
• Application installation and Deployment
• Experiments
• Related Work
• Conclusion and Future Work

13
Implementation

• OS installation (RedHat Linux)
• Using Kickstart with Pre-boot execution
  environment (PXE)
• 3-tier application installation and deployment
• Application Server (JBoss)
• Database Server (Postgres)
• Web Server (Apache)
• Used SmartFrog open-source tool from HP

14
OS Installation (Kickstart and PXE)
15
Application Installation
16
3-tier application Installation

• Used open-source tool from HP (SmartFrog)
• Language based framework
• Component model
• Life cycle management framework
• Developed SmartFrog components
• 3-tier application installation
• 3-tier application deployment

17
Smart Frog
18
Application Installation

• Extended SmartFrog in-built components to
  download and install JBoss and Postgres
• Application specific configuration by the
  specific component
• Special configuration to integrate JBoss and
  Postgres
• Deployment in a particular order

19
Experiments

• Benefits of Automation
• Performing multiple instances of application
  installation
• Total installation time as a function of number
  of machines
• Results Installation time increases at an
  acceptable rate of 30

20
Related Work

• OS installation
• NPACI Rocks
• Kickstart
• Jumpstart
• Configuration Management
• Smart Frog
• LCFG
• cfEngine
• GridWeaver
• Virtual Data Toolkit (VDT)
• Our Solution integrated Smart Frog and Kickstart
  with intelligence in the middleware!

21
Conclusion

• Automatic installation and deployment of software
  stack is challenging!
• We presented a generic system architecture that
  automates installation, configuration and
  deployment of software stack using
• separate system components and clearly defined
  interfaces
• policies for decision making
• multiple servers for load sharing
• We implemented a prototype and initial experiment
  results are encouraging

22
Future Work

• Further experimental studies evaluating the
  scalability of the system
• Guidelines for performance related policies
  through experimental evaluation
• Algorithms for load-sharing of various servers
• Tools for remote monitoring
• Security

23
Questions?

• Thank You !
• Bikash Agarwalla
• bikash_at_cc.gatech.edu
• http//www.cc.gatech.edu/bikash