EconomyBased Optimisation of File Access and Replication on a Data Grid

1
Economy-Based Optimisation of File Access and
Replication on a Data Grid

Floriano Zini ITC-irst, zini_at_itc.it MRG - SRA
2K2 Symposium Levanto, June 2002
2
The project

3
Our task in WP2 (Data Management)
We focus on Grid Query Optimisation, the process
of optimising the geographic distribution and
replication of data, in order to minimise the
overall cost of file access by users jobs The
QO-Team
4
Replica optimisation
2-Phase Optimisation Phase 1 Find suitable CE
for job execution (WP1 task) Phase 2 Re-optimise
file access during job execution
Replica Catalogue
5
The proposed solution for phase 2

• Optimisation of data access andreplication based
  on an economicmodel

• View files as digital assets which can be bought
  and sold for profit
• Let nodes on the grid interact according to an
  economic model and thus optimise the system
  implicitly rather than explicitly
• Local optimisation is computationally feasible

6
Replica optimiser architecture

• Access Mediator (AM) - contacts replica
  optimisers to locate the cheapest copies of files
  and makes them locally available
• Storage Broker (SB) - manages files stored in
  storage element, trying to maximise profit for
  the finite amount of storage space available
• P2P Mediator (P2PM) - establishes and maintains
  P2P communication between grid sites

7
Replica Optimiser Interaction Protocol

• We need a mechanism to fix the price of a file
  sold by a SB to an AM (or another SB) that
  guarantees
• Low price for purchaser
• Trading fairness
• Minimal messaging / fast as possible
• We use a Vickery auction
• Every potential seller makes an offer (lower or
  equal than the proposed price)
• The seller that made the lowest offer is chosen,
  and it is paid the second-lowest offer

8
Vickery auction call for bids
CFB(F,MaxPrice)
CFB(F,MaxPrice)
9
Vickery auction bid collection
10
Vickery auction bid selection and payment
B3 lt B1 lt B4 lt B2
11
Nested auction
CFB(F,MaxPrice)
CFB(F,MaxPrice)
CFB(F,MaxPrice)
CFB(F,MaxPrice)
12
Nested auction
?F
F
F
F
F
F
F
13
Storage Broker economic reasoning (1)

• A SB rationally decides to replicate file f (and
  possibly to delete another file f in storage) if
  this increases its cumulative profit over time
• We need to attribute a value function to a file
• The value of a file at a given point in time is
  the sum of the payments received for that file
  over a certain time period into the future
• Example value of file f2

14
Storage Broker Economic Reasoning (2)

• We define the change in profit which will result
  from the purchase of f (and the deletion of f)
  as the difference between the value functions of
  the two files less the cost of buying the new
  file
• Example change in profit obtained substituting
  f1 by f2

15
Predicting file values

• We have defined a 2 future revenue prediction
  functions that uses historical data about file
  requests
• For the moment we assume
• Sequential correlation among file requests (files
  with similar content are requested sequentiallly)
• Unitary prices
• Binomial or normal distribution of the next file
  request in each point of the file request history
• The prediction functions returns the most
  probable number of times a file will be requested
  within a time window W in the future based on
  the requests (for that or similar files) within a
  time window W in the past

16
Testing the prediction functions (1)

• Real data not available yet DataGRID doesnt
  yet exist
• Simulate data requests coming from a particular
  site (running a particular job) as a random walk
  with random step size, and some distribution

17
Testing the Prediction Functions (2)

• We have compared predicted values with real
  ones, calculated from some random walks
• Example

18
Current Future Work

• OptorSim is a grid simulator written in JAVA
  created to model the behaviour of replica
  optimiser algorithms.
• We are currently using OptorSim to compare some
  replication strategies (e.g., always
  replication or never replication) with a
  strategy based on the economic model we have
  presented
• First experiment (no auction) are encouraging
• The best strategies will be included into the
  first release of a real optimisation service
  for DataGRID