Effective gradient-free methods for inverse problems

1
Effective gradient-free methods for inverse
problems

• Jyri Leskinen
• FiDiPro DESIGN project

2
Introduction

• Current research
• Evolutionary algorithms
• Inverse problems
• Case study Electrical Impedance Tomography (EIT)
• Future

3
Current research

• Inverse problems
• Shape reconstruction
• Electrical Impedance Tomography (EIT)
• Methods
• Evolutionary algorithms (GA, DE)
• Memetic algorithms
• Parallel EAs
• Implementation of the Game Theory
• Nash GAs, MAs, DEs

4
Evolutionary algorithms

• Based on the idea of natural selection (Darwin)
• Operate a population of solution candidates
  (individuals)
• New solutions by variation (crossover, mutation)
• Convergence by selection (parent selection,
  survival selection)

5
Evolutionary algorithms

• Several methods
• Genetic algorithms (Holland, 1960s Goldberg,
  1989)
• Evolutionary strategies (Rechenberg, 1960s)
• Differential evolution (Price Storn, 1995)

6
Evolutionary algorithms

• Simple EA
• Generate initial population
• Until termination criteria met,
• Select parents
• Produce new individuals by crossing over the
  parents
• Mutate some of the offspring
• Select fittest individuals for the next generation

7
Evolutionary algorithms

• Pros
• Global search methods
• Easy to implement
• Allows difficult objective functions
• Cons
• Slow convergence rate
• Many objective function evaluations needed

8
Local search methods

• Operate on neighborhoods using certain moves
• Pros
• Fast convergence rate
• Less resource-intensive
• Cons
• Converges to the nearest optimum
• Gradient methods need nice objective function

9
Memetic algorithms

• Hybridization of EAs and LSs
• Global method
• Improved convergence rate
• Memetic algorithms
• A class of hybrid EAs
• Based on the idea of memes (Dawkins)
• LS applied during the evolutionary process

10
Memetic algorithms

• Simple MA
• Generate initial population
• Until termination criteria met,
• Select parents
• Produce new individuals by crossing over the
  parents
• Mutate some of the offspring
• Improve offspring by local search
• Select fittest individuals for the next generation

11
Memetic algorithms

• Typically Lamarckian
• Acquired properties inherited
• Unnatural
• MAs not limited to that!
• Parameter tuning
• Local search operators as memes
• Parameters encoded in chromosomes
• Meme populations
• etc.

12
Inverse problems

• Inverse problem
• Data from a physical system
• Construct the original model using available data
  and simulations
• Typical IPs
• Image reconstruction
• Electromagnetic scattering
• Shape reconstruction

13
Inverse problems

• Objective function for example a sum of squares
• min F(x) ? x(i) x(i)2
• x the vector of values from a simulated solution
  (forward problem)
• x the vector of target values

14
Inverse problems

• Often difficult to solve because of
  ill-posedness the acquired data is not
  sufficient ? the solution is not unique!
• Extra information needed regularization

15
Electrical Impedance Tomography

• Used in
• Medicine (experimental)
• Geophysics
• Industrial process imaging
• Simple, robust, cost-effective
• Poor spatial, good temporal resolution

16
Electrical Impedance Tomography

• Data from electrodes on the surface of the object
• Inject small current using two of the electrodes
• Measure voltages using the other electrodes
• Reconstruct internal resistivity distribution
  from voltage patterns

17
Electrical Impedance Tomography
Source Margaret Cheney et al. (1999)
18
Electrical Impedance Tomography
Source Margaret Cheney et al. (1999)
19
Electrical Impedance Tomography
Source The Open Prosthetics Project
(http//openprosthetics.org)
20
Electrical Impedance Tomography

• PDE Complete Electrode Model
• Forward problem calculate voltage values Ul
  using FEM

21
Electrical Impedance Tomography

• Inverse problem minimize F(sh) by varying the
  piecewise constant conductivity distribution sh

22
Electrical Impedance Tomography

• Mathematically hard, non-linear ill-posed problem
• Typically solved using Newton-Gauss method
  regularization (Tikhonov, )
• Resulting image smoothed, image artifacts

23
Electrical Impedance Tomography
24
Electrical Impedance Tomography

• Solution Reconstruct the image using discrete
  shapes?
• Resulting objective function multimodal,
  non-smooth
• Solution Use global methods

25
Electrical Impedance Tomography

• Simple test case Recover circular homogeneity (6
  control parameters)
• Two different memetic algorithms proposed
• Lifetime Learning Local Search (LLLSDE)
• Variation Operator Local Search (VOLSDE)

26
Electrical Impedance Tomography

• Evolutionary framework based on the self-adaptive
  control parameter differential evolution (SACPDE)
• LLLSDE
• Lamarckian MA
• Local search operator Nelder-Mead simplex method
• VOLSDE
• Weighting factor F improved by one-dimensional
  local search

27
Electrical Impedance Tomography

• Five algorithms tested (GA, DE, SACPDE, LLLSDE,
  VOLSDE)
• Result
• GA performed poorly
• DE better, some failures
• LLLSDE best, but the difference to other adaptive
  methods minimal

28
Electrical Impedance Tomography
29
Now future

• Improve diversity using multiple populations
  (island model)
• EAs can be used to find Nash equilibria
• Improve convergence rate with virtual Nash games?
• Can competitive games sometimes produce better
  solutions than cooperative games in
  multi-objective optimization?

30

• Thank you for your attention!
• Questions?