Real-World Distributed Computing with Ibis

1
Real-World Distributed Computing with Ibis Henri
Balbal_at_cs.vu.nlVrije Universiteit Amsterdam
2
Outline

• Distributed systems
• Distributed applications
• The Ibis distributed programming system
• Ibis in the mobile networked world

3
Distributed Systems 1980s

• Networks of Workstations (NOWs)
• Collections of Workstations (COWs)
• Processor pools (Amoeba)
• Condor pools
• (Beowulf) clusters

4
Distributed Systems 1990s

• Metacomputing (Smarr Catlett, CACM)
• Flocking Condor (Epema)
• DAS (Distributed ASCI Supercomputer)
• Grid Blueprint (Foster Kesselman)
• Desktop grids, SETI_at_home

DAS-3
5
Distributed Systems 2000s

• Cloud computing
• Infrastructure as a service
• Virtualization
• Mobile computing
• Sensor networks
• Smart phones
• The Networked World

6
Problem

• How to write high-performance applications for
  real-world distributed systems?
• How to integrate many different resources?

7
Our approach

• Study fundamental underlying problems
• hand-in-hand with realistic applications
• integrate solutions in one system Ibis

!
Distributed Systems
User
8
Fundamental problems

• Performance efficiency on wide-area systems
• Heterogeneity different systems APIs
• Malleability resources come and go
• Fault tolerance - crashes
• Connectivity firewalls, NAT, etc.

9
Applications

• Scientific applications
• Imaging (VU Medical Center, AMOLF)
• Bioinformatics (sequence analysis, cell modeling)
• Astronomy (data analysis challenge)
• Multimedia content analysis
• Games and model checking
• Semantic web (distributed reasoning)

10
Awards
Astronomy
DACH 2008 BS
DACH 2008 - FT
(Cluster/Grid08)
SCALE 2008 (CCGrid08)
ISWC 2008
Multimedia Computing
Semantic Web (van Harmelen et al.)
AAAI-VC 2007
11
Multimedia content analysis

• Automatically extract information from images
  video
• Extract feature vectors from images
• Describe properties (color, shape)
• Data-parallel task on a cluster
• Compute on consecutive images
• Task-parallelism on a grid

12
MMCA
13
Games and Model Checking

• Can solve entire Awari game on wide-area DAS-3
• Needs 10G private optical network (StarPlane)
• Distributed model checking has very similar
  communication pattern
• Search huge state spaces, random work
  distribution, bulk asynchronous transfers
• Can efficiently run DeVinE model checker on
  wide-area DAS-3, use up to 1 TB memory
• See IPDPS09 (May 2009)

14
Distributed reasoning

• MaRVIN (Frank van Harmelen et al, VU)
• a distributed platform for massive RDF
  inferencing (deductive closure)
• a brain the size of a planet
• Uses Ibis to run on heterogeneous systems
  (clusters, desktop grids)
• 3rd prize at Billion Triple trackof Semantic Web
  Challenge 2008

15
European users

• D-Grid Workflow engine for astronomy
• U. Erlangen grid file system
• INRIA ProActive on Ibis RMI
• U. Patras Jylab scientific computing
• UPC Barcelona Grid Superscalar
• HITACHI Peta-scale data management

16
Outline

• Distributed systems
• Distributed applications
• The Ibis distributed programming system
• Ibis in the mobile networked world

17
Ibis Philosophy

• Real-world distributed applications should be
  developed and compiled on a local workstation,
  and simply be launched from there

18
Ibis Approach

• Virtual Machines (Java) deal with heterogeneity
• Provide range of programming abstractions
• Designed for dynamic/faulty environments
• Easy deployment through middleware-independent
  programming interfaces
• Modular and flexible can replace Ibis components
  by external ones

19
Ibis Design

• Functionality from programming languages
• High-Performance Application Programming System
• Functionality fromoperating systems
• Distributed Application Deployment System

20
Ibis System
21
Programming system

• Programming models
• Message passing (RMI, MPJ)
• Divide-and-conquer (Satin)
• Jorus (multimedia applications)
• Dataflow framework (Maestro) ? see HPDC09 Sat
  14.30
• IPL (Ibis Portability Layer)
• Java-centric run-anywhere library
• Point-to-point, multicast, streaming, .
• Simple model (Join-Elect-Leave ) for tracking
  resources, supports malleability
  fault-tolerance

22
SmartSockets library

• Detects connectivity problems
• Tries to solve them automatically
• With as little help from the user as possible
• Integrates existing and several new solutions
• Reverse connection setup, STUN, TCP splicing, SSH
  tunneling, smart addressing, etc.
• Uses network of hubs as a side channel

23
Example
24
Example
25
Deployment system

• IbisDeploy GUI
• JavaGAT
• Java Grid Application Toolkit
• Make applications independent of underlying
  middleware
• Zorilla P2P system
• Job management, gossiping,clustering, flood
  scheduling

26
JavaGAT
Grid Application
submitJob(...)?
File.copy(...)?
fork pbs condor unicore globus
cp ftp gridftp scp http

• For simple grid operations there are many ways of
  implementing them
• different middleware available on different sites

27
JavaGAT
Grid Application
submitJob(...)?
File.copy(...)?
?
fork pbs condor unicore globus
cp ftp gridftp scp http
?

• Which should you use ?
• Some may not be available on all sites
• Many combinations

28
JavaGAT
Grid Application
submitJob(...)?
File.copy(...)?
GAT
Remote Files
Monitoring
Info service
Resource Management
GAT Engine
GridLab
Globus
Unicore
SSH
P2P
Local
gridftp
globus
29
Multimedia Content Analysis
Ibis (Java)
Client
Servers
Broker

• Runs simultaneously on clusters (DAS-3, Japan,
  Australia), Desktop Grid, Amazon EC2 Cloud
• Connectivity problems solved automatically by
  Ibis SmartSockets

30
Connection management
Standard sockets only local VU machines can be
reached due to firewalls problems
With SmartSockets run everywhere
31
Ibis movie (part 1)
32
Performance on 1 DAS-3 cluster

• Relative speedups of Java/Ibis and C/MPI
• Using TCP or Myricoms MX protocol
• Sequential performance Java 80 of C

33
Speedup (wide-area)

• Homogeneous wide-area systems (DAS-3)
• Frame rate increases linearly with clusters
• World-wide experiment
• 24 frames per second
• Speed limited by camera,not computing
  infrastructure

34
Outline

• Distributed systems
• Distributed applications
• The Ibis distributed programming system
• Ibis in the mobile networked world

35
Smart Phones

• GSM PC GPS camera networks .
• Location-aware
• What if everyone always carries a smart phone
  (like a GSM now)?
• Next wave in computing?

36
Ibis on Smart Phones

• Our focus distributed smart phone applications
• Applications running on multiple phones
• Integration with distributed computing backbone
• Use Android for development
• Googles open-source platform
• Java-based

37
Distributed applications

• Disaster management (Katrina)
• Use ad-hoc Wifi network when GSM network fails
• Finding nearby people with certain skills
• Bus drivers, CPR
• Distributed decision support
• Moving people to shelters (logistics)
• Social networks
• Similar issues
• Find nearby friends, decide on restaurant

38
Wild example

• Track position gt automatic diary of your life
• Cross-comparisons between diaries

39
eyeDentify

• Object recognition on a G1 smartphone
• Smartphone is a limited device
• Can run only 64 x 48 pixels (memory bound)
• 1024 x 768 pixels would take 5 minutes
• Distributed Ibis version

2.0 seconds
1024 x 768 pixels
40
Ibis movie (part 2)
41
Interdroid
Novel Mobile Distributed Applications
Data Management
Context Sensitive Programming Models
Distributed Communication
42
Current work

• Raven API for Viable Episodic Networking
• Decentralized synchronization API
• Fine grained control over data sharing
• Bluetooth support for ad-hoc communication
• Discovery of devices using multiple networks
• Context Aware Programming Models
• Supporting distributed decision making
• Representing and using context (location etc.)
• Exploiting social relationships (Hyves, Facebook)

43
Summary

• Ibis provides integrated solutions for many hard
  problems
• performance, heterogeneity, malleability, fault
  tolerance, connectivity
• It combines functionality from programming
  languages and operating systems
• Used for many applications on real-world
  distributed systems
• Download from http//www.cs.vu.nl/ibis/

44
Acknowledgements

• Niels Drost
• Ceriel Jacobs
• Roelof Kemp
• Timo van Kessel
• Thilo Kielmann
• Jason Maassen
• Rob van Nieuwpoort
• Nick Palmer
• Kees van Reeuwijk
• Frank J. Seinstra
• Kees Verstoep
• Gosia Wrzesinska

45
Questions?
46
DAS-3