Apache Airavata

1
What Is Apache Airavata ?

• A framework that supports execution and
  management
• Of computational scientific applications and
  workflows
• Mainly for grid based systems but will support
• Remote clusters, cloud-based systems
• Can be extended via implementation
• Can be a middleware for scientific developers
• Offers an extensible architecture
• Open source Apache 2.0 license
• Written in Java, C

2
Airavata Architecture
3
Airavata Architecture

• Airavata API
• Users / gateway developers can communicate with
  Airavata
• Orchestrator
• Manages application AMQP worker queue
• Workflow interpreter
• Manages submitted work flows
• Application Factory (Gfac)
• Communicates with remote resources

4
Airavata Architecture

• Registry
• The Airavata data store
• Messaging
• Publishes notifications for application statuses
• API written using Apache Thrift
• Supports many languages

5
Airavata AppCatalog Data Model
6
Airavata AppCatalog Data Model

• Used for application registration and contains
  ...
• ComputeResourceModel
• Abstraction of remote compute resource
• Contains information about resource queues
• Host addresses, job submission protocols
• Useful information related to compute resource
• ApplicationModule
• Remote resources contains modules / applications
• Already installed in the systems
• User defined applications possible

7
Airavata AppCatalog Data Model

• Used for application registrationand contains
  ...
• ApplicationInterfaceModel
• Interface which defines application I/O
• Can have multiple interfaces defined
• ApplicationDeploymentModel
• Bridges the compute resource and the module
• GatewayProfileModel
• Abstraction of the science gateway
• GatewayPreferenceModel
• Model to specify gateway compute resources
  preferences
• Includes information like login usernames,
  allocation ids etc

8
Airavata ExperimentCatalog Data Model
9
Airavata ExperimentCatalog Data Model

• Used for application execution
• Application might use single node or work flow
• Airavata internally creates models to represent
  process
• Process model, task model, job model
• Work flows contain a list processes
• Tasks can have many types
• Input staging, job submission
• Job monitoring, output staging etc
• Users can develop their own tasks for the task
  chain

10
Airavata UI

• Has a Java based application called XBaya
• Create workflows,
• Submit and manage multiple applications
• Works best with Airavata 0.14
• Also has a web-based interface (PHP Gateway)
• Written using Airavata PHP client library
• Users can deploy it in their own systems
• Use it to register applications
• Run and monitor applications

11
Available Books

• See Big Data Made Easy
• Apress Jan 2015
• See Mastering Apache Spark
• Packt Oct 2015
• See Complete Guide to Open Source Big Data
  Stack
• Apress Jan 2018
• Find the author on Amazon
• www.amazon.com/Michael-Frampton/e/B00NIQDOOM/
• Connect on LinkedIn
• www.linkedin.com/in/mike-frampton-38563020

12
Connect

• Feel free to connect on LinkedIn
• www.linkedin.com/in/mike-frampton-38563020
• See my open source blog at
• open-source-systems.blogspot.com/
• I am always interested in
• New technology
• Opportunities
• Technology based issues
• Big data integration