ALEAE : Handling Uncertainties in LargeScale Distributed Systems

1
ALEAE Handling Uncertainties in Large-Scale
Distributed Systems

• Emmanuel Jeannot
• LORIA - INRIA - CNRS
• ALEAE Kick-off
• April 1st 2009

2
Introduction

• What is a grid?
• An infrastructure
• Distributed
• Heterogeneous
• But also
• Dynamic
• Shared
• Lot of uncertainties

3
Uncertainties

• Uncertainties
• unpredictable behavior
• Behavior not as expected
• Where does it come from?
• Infrastructure (hardware)
• Application (software)
• Users

4
Uncertainty at the infrastructure level

• The hardware that compose a grid can
• Fail
• Be volatile (be removed or added)
• Have performance degradation (due to a shared
  usage)

5
Uncertainty at the application level

• It is often assumed that
• One know the duration of the composing part of an
  application
• Its resource usage is known
• It does not fail.
• However, this is not always the case

6
Uncertainties due to the users

• Users
• Submit jobs/requests randomly
• May behave with some malignity (voluntarily or
  not)
• DOS attack
• Desktop grid give wrong answer

7
Rationale

• As resource management algorithms
• cope with heterogeneity or distribution,
• they also must cope with uncertainty

8
Ways to cope with uncertainty

• Proactive methods (static)
• Redundancy
• Duplication
• Reactive methods (dynamic)
• Check-point restart
• migration
• Mixed (provide a static solution and adapt it
  dynamically)

9
Functional Goals

• Different kinds of uncertainties lead to
  different desired behavior
• Reliability, fault-tolerance
• Hardware failure
• Software failure
• Robustness
• Hardware perf. degradation
• Software unpredictability
• Correctness
• Bad usage
• Etc

10
Multi-criteria approach

• The old good metrics are still valid
• Makespan
• Load-balance
• Response time
• Lateness
• Etc.
• Most of the time these metrics are contradictory
  with the other one.
• Need of a multi-criteria approach (ex
  makespan/reliability).

11
Open issue (1)

• Gather traces
• - What is the behavior of users/programs/infrastr
  ucture?
• - Ease the extraction of useful information
• - Ensure generality

12
Open issues (2)

• Model the uncertainty
• Trace the behavior
• Analyze
• Provide modeling

13
Carefully define metrics

• Mapping a goal into a metric is not trivial
• Ex robustness
• Intuitive notion
• Many metrics (one per paper)
• Question relation between these metrics.

14
Open issues (4)

• Provide resource management (scheduling)
  algorithms
• Mono-criteria/Multi-criteria
• Static/dynamic/mixed
• Works well in the worst case/on the avarage
• Etc.

15
Open issues (5)

• Static vs. Dynamic?
• Each approach advantages and drawback.
• Dynamic (ex. check-point-restart) time costly,
  but handle almost every cases
• Static (ex. duplication) resource costly, can
  provide some guarantee.
• What is the best approach depends on the problem.
• Is the mixed approach always possible/profitable?

16
Open issues real scale experimentation

• Provide detection mechanisms
• Failure
• Malignity
• Resource usage
• Correctness
• Etc.
• Program and test solutions
• Real-scale (grid5000, DAS-3)
• Simulation
• Emulation?
• Validation of the models.

17
Big picture
18
Today

• Kick-off
• ALEAE a two year INRIA funded project (20
  k/year)
• Presentation on each item
• Technical presentation on sub-item
• Work plan
• Other/next meetings
• Visit/exchange
• Mission
• Post-doc
• Synergies between teams
• Important I am moving to INRIA Bordeaux.

19
Conclusion

• Grid environments are full of uncertainties
• These uncertainties come from different factors
• Handling them is difficult (especially with the
  traditional criteria)
• What is the best way to tackle this problem
  (dynamic/static/mixed), is of crucial interest.
• The goal of ALEAE is to tackle such issues.