An Adaptive Multi-Objective Scheduling Selection Framework For Continuous Query Processing

1
An Adaptive Multi-Objective Scheduling Selection
Framework For Continuous Query Processing
A Presentation _at_ IDEAS, Montreal, Canada, July
27, 2005

• Timothy M. Sutherland
• Bradford Pielech
• Yali Zhu
• Luping Ding
• and Elke Rundensteiner
• Worcester Polytechnic Institute
• Worcester, MA, USA

2
Continuous Query (CQ) Processing
Register Continuous Queries
May have different QoS Requirements.
Streaming Data
Streaming Result
Stream Query Engine
May have time-varying rates and data distribution.
Available resources for executing each operator
may vary over time.
Run-time adaptations are required for stream
query engine.
3
Runtime Adaptation Techniques

• Operator Scheduling
• Query Optimization
• Distribution
• Load Shedding
• Others

4
Operator Scheduling for CQ

• Operator Scheduling
• Determines the order to execute operators
• Allocates resources to query operators
• Existing Scheduling Algorithms
• Round Robin (RR)
• First In First Out (FIFO)
• Most Tuples In Queue (MTIQ)
• Chain BBM03
• Others

4
scheduler
3
1, 2, 3, 4
1
2
stream B
stream A
5
Properties of Existing Scheduling Algorithms

• Uni-Objective
• Designed for a single performance objective
• Increase throughput
• Reduce memory
• Reduce tuple delay
• Fixed Objective
• Cannot change objective during query run
• May be insufficient for CQ processing

6
Performance Requirements in CQ

• May be multi-objective
• Example
• Run time-critical queries on memory-limited
  machine
• Two performance goals less result delay and less
  memory
• May vary over time
• Example
• System resource availability may change during
  query run
• Under light workload faster throughput
• Under heavy workload less memory
• Existing scheduling algorithms
• Not designed for multiple changing objectives
• As a result, each has its strengths and weaknesses

7
Scheduling Example FIFO
FIFO Start at leaf and process the newest tuple
until completion.
End User
s 1 T 0.75
3
Time
s .1 T 0.25
2

• FIFOs queue size grows quickly.
• FIFO has fast first outputs

s 0.9 T 1
1
Stream
8
Scheduling Example MTIQ

• MTIQ
• Schedule the operator with the largest input
  queue.

End User
s 1 T 0.75
3
Time
s .1 T 0.25
2

• MTIQs queue size grows at a slower rate
• Tuples remain queued for longer time

s 0.9 T 1
1
Stream
9
Performance Comparison
10
Summary of Problem

• What does CQ need
• Runtime Adaptation
• Multiple prioritized optimization goals
• Dynamically changing optimization goals
• Our solution -- AMoS
• Adaptive Multi-Objective Scheduling Selection
  Framework

11
Outline

• Introduction and Motivation
• The AMoS Framework
• Experimental Evaluation
• Conclusion

12
General Idea of AMoS Framework

• Meta-scheduler
• User specify multiple performance objectives
• Rank scheduling algorithms based on performances
• Dynamically select the current best algorithm
• Design logic
• Simple, low overhead and effective

Performance Requirements
Algorithm Evaluator
Statistics
Scheduling Algorithm Library
Algorithm Selector
Selecting Strategy
Decision Request
Scheduler Decision
13
Specifying Performance Objectives

• Metric Any statistic calculated by the system.
• Quantifier Maximize or Minimize
• Weight The relative weight / importance of this
  metric
• The sum of all weights is exactly 1.

Metric Quantifier Weight
Output Rate Maximize 0.60
Memory Minimize 0.25
Delay Minimize 0.15
14
Adapting Scheduler

• Step 1 Scoring Statistics Periodically
• Step 2 Scoring Scheduling Algorithms
• Step 3 Selecting Scheduling Algorithm

15
Step 1 Scoring Statistics
Performance Objectives
Stats Score Matrix
Metric Quantifier Weight
Output Rate Maximize 0.60
Memory Minimize 0.25
Delay Minimize 0.15
Output rate Memory Delay
MTIQ 0.10 0.12 0.46
FIFO 0.20 -0.31 -0.16
Chain 0.15 -0.50 0.21
Output rate Memory Delay
MTIQ
FIFO
Chain
quantifier Zi_new meaning
Maximize gt 0 Perform above average
Maximize lt 0 Perform below average
Minimize gt 0 Perform below average
Minimize lt 0 Perform above average

• Update stats scores of the current scheduling
  algorithm Acurrent
• Zi_new -- score of stat i for Acurrent
• µiH, maxiH, miniH -- mean, max and min history
  value of stat i.
• µiC -- most recent collected stat i.
• decay -- decay factor in (0, 0.5). Exponentially
  decay out-of-date data.
• Zi_new ? (-1, 1).

16
Step 2 Scoring Scheduling Algorithms
Stats Score Matrix
Performance Objectives
Output rate Memory Delay
MTIQ 0.10 0.12 0.46
FIFO 0.20 -0.31 -0.16
Chain 0.15 -0.50 0.21
Output rate Memory Delay Score
MTIQ 0.10 0.12 0.46 0.96
FIFO 0.20 -0.31 -0.16 1.22
Chain 0.17 -0.50 -0.21 1.65
Metric Quantifier Weight
Output Rate Maximize 0.60
Memory Minimize 0.25
Delay Minimize 0.15
quantifier Zi meaning qizi effect on ScoreA
Maximize gt 0 above average gt 0 Increase score
Maximize lt 0 below average lt 0 Decrease score
Minimize gt 0 below average lt 0 Decrease score
Minimize lt 0 above average gt 0 Increase score

• Update score of scheduling algorithm A.
• ScoreA -- score for A.
• zi score of stat i for A.
• qi -1 for minimize, 1 for maximize
• wi Weight in table of objectives.
• Add 1 to shift from -1, 1 to 0, 2.
• ScoreA ? (0, 2).

17
Issues In Scheduler Selection Process

• The framework need to learn each algorithm
• Solution All algorithms are initially run for
  once
• Algorithm did poorly earlier may be good now
• Solution Periodically explore other algorithms
• Reason for adopting the Roulette Wheel Strategy

18
Step 3 Selecting Next Algorithm

• Roulette Wheel MIT99
• Chooses next algorithm with a probability
  equivalent to its score
• Favors the better scoring algorithms, but will
  still pick others.

• Well performed ones have better chances
• Others also have chances to be explored
• Lightweight so overhead is very low
• Proven to be effective in experimental study

19
Overall Flow of the Adapting Process
Input performance objectives candidate
scheduling algorithms
Initially run all algorithms once
Periodically Scoring statistics
change requested?
N
Repeat until query is done
Y
Rank candidate algorithms
Select next algorithm to run
20
Summary of the AMoS Framework

• Light-weight
• Use runtime statistics collected by the system
• Ranking formula are simple yet effective
• Self learning
• No apriori information needed
• Learn behaviors of scheduling algorithms on the
  fly
• Easily extendable
• Add more scheduling algorithms
• Add more performance objectives
• Add more selecting strategies

21
Outline

• Introduction and Motivation
• The AMoS Framework
• Experimental Evaluation
• Conclusion

22
Experimental Setup

• Evaluated in CAPE system cape04
• A prototype continuous query system
• Query plans
• Consists of join and select operators
• Input streams
• Simulated with Poisson arrival pattern
• Performance objectives
• Different number of objectives
• Different weight of objectives

23
One Performance Objective
24
Two Performance Objectives
50 focus on output rate, 50 focus on tuple delay
25
Two Performance Objectives (cont.)
70 focus on tuple delay, 30 focus on output rate
30 focus on tuple delay, 70 focus on output rate
26
Three Performance Objectives
Equal focus (33) on output rate, memory and
tuple delay
27
Conclusions

• Identified the lack of support for
  multi-objective adaptation
• Existing approaches only focus on single
  objective
• Cannot change objective during query run
• Proposed a novel scheduling framework
• Allows applications to control performance
  objectives
• Alters scheduling algorithm based on run-time
  performances
• Independent of scheduling algorithms or
  performance objectives
• AMoS strategy shows very promising experimental
  results.
• Developed and evaluated in the CAPE system
• W/ single objective, performs as well as the best
  algorithm
• W/ multiple objectives, overall better than any
  algorithm

28
Thank You!

• For more information, please visit
• davis.wpi.edu/dsrg