Performance Evaluation of Distributed SWAPGA Models with GridRPC

1
Performance Evaluation of Distributed SWAP-GA
Models with GridRPC
Shamim Akhter, Kiyoshi Osawa, Kento Aida

• Presented By
• Shamim Akhter
• PhD Student
• AIDA Lab
• Department of Information Processing
• Tokyo Institute of Technology, Japan

2
Introduction

• Agricultural Activities Monitoring
• Crop monitoring- Sowing Date, Cropping intensity,
  Growth, Water stress etc.
• Production of food security
• Water management in Irrigation Activity
• Remote Sensing (RS)
• Satellite Image Processing
• Plays a vital role for providing useful
  information over large areas (Country level)
• Some Crop monitoring data (Crop parameters) such
  as sowing date, cropping intensity, growth,
  stress etc. are not visible through RS Images.

3
Introduction (Cond)

• Crop Growth Model
• Continuously Monitoring in various aspects of
  Crop fields
• Useful for prediction
• Use Real fields experiment data
• Data Assimilation Technique
• To estimate parameters which can not be observed
  by RS images

4
SWAP-The Crop Model

• SWAP is abbreviation of Soil Water Atmosphere
  and Plant equipped with Crop models and water
  Management Module.
• The growth and development of a crop can be
  simulated under different climatic and
  environmental conditions.
• ETa (Evapotranspiration Evaporation from soil
  and water with the sum of Transpiration from
  plant) Value
• ETa and LAI (Leaf Area Index value) both are the
  indicators of crop productivity and can be
  estimated from satellite remote sensing.
• SWAP can produce both ETa and LAI value by using
  crop field and weather data.

5
SWAP-GA Model

• SWAP-GA is a combined model of SWAP and Genetic
  Algorithm (GA)
• Developed by Ines et al. 2002
• SWAP-GA proposed a way to identify the SWAP
  unknown parameters with RS data
• SWAP parameters can be estimated by assimilating
  SWAP model output data (swapData) with RS data
  (satData).
• In our experiment, we are working with ETa data.
• Open Source Software GRASS has been used to
  process RS images.
  

6
SWAP Model Parameter identification - Data
Assimilation using RS and GA -
SWAP Input Parameters sowing date, soil property,
Water management etc.
RS Observation
SWAP Crop Growth Model
LAI, Evapotranspiration
LAI, Evapotranspiration
4
.
00
4
.
00
Fitting
3
.
00
3
.
00
Assimilation by finding Optimized parameters By GA

Eavpotranspiration LAI
Evapotranspiration LAI
2
.
00
2
.
00
1
.
00
1
.
00
0
.
00
0
.
00
0
45
90
135
180
225
270
315
360
0
45
90
135
180
225
270
315
360
Day Of Year
Day Of Year
RS
Model
7
SWAP-GA Working Modules
8
A Practical Problem Arises

• Population 1 chromosome
• 1 set of parameters,
• 1 SWAP approx 2 sec.
• Pixel needs 1 to several hundred thousands
  evaluation
• 30 minutes by 1 CPU

RS image of 100 x 100 square kilometer of 100x100
pixels will take more than 0.5 years (30 min.
100 100) running with sequential SWAP-GA.
9
Objective

• High demand of distributing behavior is appealed
  inside the SWAP-GA module.
• Grid computing can be a solution for this
  circumstance.
• Evaluate different strategies (Distributed
  Models) to implement SWAP-GA model in Grid
  architecture to optimize the total run time
• Implementation with GridRPC (Ninf-G)

10
Implementation Schemes

• Three different strategies are applied to run the
  SWAP-GA in distributed manner and their
  performance will be discussed.
• All schemes used GridRPC as programming framework
  but the ways of task distribution are different.
• The Implementation strategies are
• Population Distribution (Ninf-G)
• Pixel Distribution (Ninf-G)
• Hierarchical Distribution (Ninf-G MPI)

11
Overview of Ninf-G System
12
1. Population Distribution Implementation
Methodology ( as Master-Slave paradigm )
Master distributes populations among Slave nodes
Slaves/Computing nodes perform the evaluation,
generate fitness and send back the populations
(with fitness) to Master. This same procedure
will continue for all pixels.
13
1. Population Distribution with Ninf-G
14
2. Pixel DistributionImplementation Methodology
( as Master-Slave paradigm )
Master distributes pixels among Slave nodes
Slaves/Computing nodes perform whole SWAP-GA
serially, and generate the best unknown
parameters set for the assigned pixel (s) and
send back the parameters list to Master.
15
2. Pixel Distributionwith Ninf-G
16
3. Hierarchical Distribution Model
Implementation Methodology ( as Master-Slave
paradigm )
Pixel Distribution and Population Distribution
model are Combined Together.
In Grid, Pixel will be distributed among PC
Clusters trough Ninf-G. Populations will
be distributed inside PC Clusters through MPI
17
3. Hierarchical Modelwith Ninf-G MPI
18
Clusters Specification
Workload is 15 Pixel,10 Generation and 60
population
19
Experiments on Single Site
Population Distribution (Blade Cluster)
Increasing Computing Nodes reduces the workload
time. Ninf-G calls happened regularly at each
population execution Estimation The highest
performance can be estimated when computing nodes
equal to the population numbers
Performance is not in desirable state 10
computing nodes performance 4 times than
1. Sol Reduce Ninf-G calls
20
Experiments on Single Site
Hierarchical Distribution Model with Ninf-G and
MPI (Blade Cluster)
Increasing Computing Nodes reduces the workload
time. Estimation The highest performance will
gain when cluster numbers equal to the pixel
number and slaves in each Cluster is equal to the
population number
Performance is improved than Population
Distribution Model However MPI communication cost
is still available.
21
Experiments on Single Site
Pixel Distribution (Blade Cluster)
Increasing Computing Nodes reduces the workload
time. Communication overhead is hidden through
Calculation workload. Estimation The highest
performance will gain when computing nodes equal
to the pixel number
Performance is in desirable state 10 computing
nodes performance 9 times than 1.
22
Time Chart of SWAP-GA Models (Blade Cluster)
Experiments on Single Site
23
Hierarchical Distribution and Pixel Distribution
Models Time Performance in Real Grid Testbed
The Performance of Hierarchical
Distribution Model is improved by Including More
CPU powers. Which Highlights the main Drawback
of Pixel Distribution Model that additional
CPU powers will not improve performance when CPUs
are more than Pixel No.
24
Discussion

• Hierarchical Distribution model performance
  highly depend on the number of Clusters.
• In this research, we just experiment on two
  Clusters and the performance of Hierarchical
  Distribution model is better than pixel
  distribution model.
• So, number of Clusters increment will increase
  the Hierarchical model performance more.
• In F-32 required some waiting time in the Queue.

25
Conclusion

• Three different implementation strategies for the
  SWAP-GA model were successfully developed and
  implemented on GridRPC framework.
• Increasing the computing nodes number improves
  the performance of the SWAP-GA model.
• The Pixel Distribution model and the Hierarchical
  Distribution model performances improve in the
  Grid testbed by providing more computational
  power with respect to pixel number and population
  number respectively.
• Inside one PC Cluster, Pixel Model is good
• Inside multi PC Clusters System (Grid),
  Hierarchical Model is good.

26
Future Works

• The web based portal for SWAP-GA on distributed
  platform is required and it will be developed in
  near future.
• Furthermore, GA can also be replaced by any
  probabilistic calculative model such as MCMC
  (Markov Chain Monte Carlo).