Divisible Load Scheduling

1
Divisible Load Scheduling

• A Tutorial

Thomas Robertazzi University at Stony Brook
2
What is a Divisible Load?

• A computational networkable load that is
  arbitrarily partitionable (divisible) amongst
  processors and links.
• There are no precedence relations.

3
Simple Application Example

• Problem Sum 1000 trillion numbers
• Approach Partition the numbers among 100
  processors
• But how?

4
Simple Application Example

• To optimize solution time (maximize speedup) one
  needs to take into account heterogeneous link and
  processor speeds, computation and communication
  intensities, interconnection topology and
  scheduling policy.
• Divisible Load Scheduling Theory Can Do This!

5
Applications (Generic)

• Grid Computing/Meta-computing
• Data Intensive Computing
• Sensor Processing
• Image Processing
• Scientific/Engineering Computing
• Financial Computing

6
Applications (Specific)

• Pattern Searching
• Database Computation
• Matrix-Vector Computation
• EM Field Calculation (CAD)
• Edge Detection

7
DLT Modeling Advantages

• Linear and Deterministic Modeling
• Tractable Recursive/Linear Equation Solution
• Schematic Language
• Equivalent Elements
• Many Applications

8
Interconnection Topologies

• Linear Daisy Chain
• Bus
• Single Level and Multilevel Trees
• Mesh
• Hypercube

9
Directions Scalability
1
1
3
1
2
1
Simultaneous Distribution (Scalable)
Sequential Distribution (Saturation)
Hung Robertazzi
Cheng Robertazzi
10
An Example

• Model Specifications
• A star network( single level tree network), and
  multi-level tree.
• Computation and transmission time is a linear
  function of the size of load.
• Level to Level Store and Forward Switching
• Same Level Concurrent Load Distribution.

11

• Children without Front End
• After receiving the assigned data, each child
  proceeds to process the data.

12

• Timing Diagram (single level tree)
• Children without Front End

13
m1 unknows vs. m1 Eqs.

• Recursive equations
• Normalization equation

14

• Distribution Solution

15

• The load distribution solution is similar to the
  solution of the state-dependent M/M/1 queuing
  system.

16
Similarities to Queueing Theory

• Linear model and tractable solutions
• Schematic Language
• Equivalent Elelements
• Infinite Size Networks

17

• Speedup Analysis

18

• Speedup Analysis (continued)

19

• Tree Network
• (Children without Front Ends)

20
Collapsing single level trees
21
Bandwidth of Fat Tree

• Definition The bandwidth of level j in a fat
  tree can be defined as pj-1z.

22
Directions Sequencing and Installments

• Daisy Chain Surprise

• Efficiency Rule

Ghose, Mani Bharadwaj
23
Directions Sequencing and Installments

• Multi-installment for Sequential Distribution

6
4
5
3
1
2
Ghose, Mani Bharadwaj
24
Directions Sequencing and Installments
Diminishing returns in using multi-installment
distribution.
Ghose, Mani Bharadwaj
25
Directions Sequencing and Installments
Drozdowski
26
Directions Time Varying Modeling
Sohn Robertazzi
Can be solved with integral calculus.
27
Directions Monetary Cost Optimization
N

• Min CTotal S ancnwnTcp

n1
Bus
Processors
Optimal Sequential Distribution if
cn-1wn-1 less than cnwn for all n
Sohn, Luryi Robertazzi
28
Directions Monetary Cost Optimization

• 2 US Patents
• Patent 5,889,989 (1999) Processor Cost
• Patent 6,370,560 (2001) Processor and
• 
  Link Cost
• Enabling technology for an open e-commerce
• market in leased proprietary computing.

Sohn, Charcranoon, Luryi Robertazzi
29
Directions Database Modeling
Expected time to find multiple records in flat
file database
Ko Robertazzi
30
Directions Realism

• Finite Buffers (Bharadwaj)
• Job Granularity (Bharadwaj)
• Queueing Model Integration

31
Directions Experimental Work
Database Join (Drozdowski)
32
Directions Future Research

• Operating Systems
• Incorporate divisible load scheduling
  
• into (distributed) operating systems
• Measurement Process Modeling
• Integrate measurement process
• modeling into divisible scheduling

33
Directions Future Research

• Pipelining (Dutot)
• Concept Distribute load to
• further processors first
  for
• speedup improvement
• Improvement reported for daisy chains

34
Directions Future Research

• System Parameter Estimation (Ghose)
• Concept Send small probing loads across
• links and to processors
  to estimate
• available effort
• Challenge Rapid change in link processor
• state

35
DLT has a Good Future

• Many Applications including
• wireless sensor networks
• Tractable (Modeling Computation)
• Rich Theoretical Basis

36
Thank you!

• Questions???