Two-Stage Optimisation in the Design of Boolean Functions

1
Two-Stage Optimisation in the Design of Boolean
Functions

• John A Clark and Jeremy L JacobDept. of Computer
  Science
• University of York, UK
• jac_at_cs.york.ac.ukjeremy_at_cs.york.ac.uk

2
Overview

• Optimisation
• general introduction
• hill-climbing
• simulated annealing.
• Boolean function design (reprise)
• Experimental approach and results.
• Conclusions and future work.

3
Optimisation

• Subject of huge practical importance. An
  optimisation problem may be stated as
  follows
• Find the value x that maximises the function z(y)
  over D.

Given a domain D and a function z D ? ? find x
in D such that z(x)supz(y) y in D
4
Optimisation

• Traditional optimisation techniques include
• calculus (e.g. solve differential equations for
  extrema)
• f(x) -3 x26x solve f '(x)-6x60 to
  obtain x1 with maximum f(x)3
• hill-climbing inspired by notion of calculus
• gradient ascent etc.
• (quasi-) enumerative
• brute force (a crypto-favourite)
• linear programming
• branch and bound
• dynamic programming

5
Optimisation Problems

• Traditional techniques not without their problems
• assumptions may simply not hold
• e.g. non-differentiable discontinuous functions
• non-linear functions
• problem may suffer from curse (joy?) of
  dimensionality - the problem is simply too big
  to handle exactly (e.g. by brute force or dynamic
  programming). NP hard problems.
• Some techniques may tend to get stuck in local
  optima for non-linear problems (see later)
• The various difficulties have led researchers to
  investigate the use of heuristic techniques
  typically inspired by natural processes that
  typically give good solutions to optimisation
  problems (but forego guarantees).

6
Heuristic Optimisation

• A variety of techniques have been developed to
  deal with non-linear and discontinuous problems
• highest profile one is probably genetic
  algorithms
• works with a population of solutions and breeds
  new solutions by aping the processes of natural
  reproduction
• Darwinian survival of the fittest
• proven very robust across a huge range of
  problems
• can be very efficient
• Simulated annealing - a local search technique
  based on cooling processes of molten metals (used
  in this paper)
• Will illustrate problems with non-linearity and
  then describe simulated annealing.

7
Local Optimisation - Hill Climbing

• Let the current solution be x.
• Define the neighbourhood N(x) to be the set of
  solutions that are close to x
• If possible, move to a neighbouring solution that
  improves the value of z(x), otherwise stop.
• Choose any y as next solution provided z(y) gt
  z(x)
• loose hill-climbing
• Choose y as next solution such that
  z(y)supz(v) v in N(x)
• steepest gradient ascent

8
Local Optimisation - Hill Climbing
z(x)
Really want toobtain xopt
Neighbourhood of a point x might be
N(x)x1,x-1Hill-climb goes x0 ? x1 ? x2
since f(x0)ltf(x1)ltf(x2) gt f(x3) and gets
stuck at x2 (local optimum)
xopt
9
Simulated Annealing
Allows non-improving moves so that it is possible
to go down
z(x)
in order to rise again
to reach global optimum
x
10
Simulated Annealing

• Allows non-improving moves to be taken in the
  hope of escaping from local optimum.
• Previous slide gives idea. In practice the size
  of the neighbourhood may be very large and a
  candidate neighbour is typically selected at
  random.
• Quite possible to accept a worsening move when an
  improving move exists.

11
Simulated Annealing

• Improving moves always accepted
• Non-improving moves may be accepted
  probabilistically and in a manner depending on
  the temperature parameter Temp. Loosely
• the worse the move the less likely it is to be
  accepted
• a worsening move is less likely to be accepted
  the cooler the temperature
• The temperature T starts high and is gradually
  cooled as the search progresses.
• Initially virtually anything is accepted, at the
  end only improving moves are allowed (and the
  search effectively reduces to hill-climbing)

12
Simulated Annealing

• Current candidate x.

At each temperature consider 400 moves
Always accept improving moves
Temperature cycle
Accept worsening moves probabilistically. Gets
harder to do this the worse the move. Gets
harder as Temp decreases.
13
Crypto and Heuristic Optimisation

• Most work on cryptanalysis attacking variety of
  simple ciphers - simple substitution and
  transposition through poly-alphabetic ciphers
  etc.
• more recent work in attacking NP hard problems
• But perhaps most successful work has been in
  design of cryptographic elements.
• Most work is rather direct in its application.

14
Boolean Function Design

• A Boolean function

f(x)
f(x)
x
For present purposes we shall use the polar
representation
Will talk only about balanced functions where
there are equal numbers of 1s and -1s.
15
Preliminary Definitions

• Definitions relating to a Boolean function f of n
  variables

Linear function
Lw(x)w1x1? ? wnxn
(polar form)
Walsh Hadamard
16
Preliminary Definitions

• Non-linearity
• Auto-correlation
• For present purposes we need simply note that
  these can be easily evaluated given a function f.
  They can therefore be used as the functions to be
  optimised. Traditionally they are.

17
Using Parsevals Theorem

• Parsevals Theorem
• Loosely, push down on F(w)2 for some particular w
  and it appears elsewhere.
• Suggests that arranging for uniform values of
  F(w)2 will lead to good non-linearity. This is
  the initial motivation for our new cost function.

NEW FUNCTION!
18
Moves Preserving Balance

• Start with balanced (but otherwise random)
  solution. Move strategy preserves balance

f(x)
f(x)
x
g(x)
Neighbourhood of a particular function f to be
the set of all functions obtained byexchanging
(flipping) any two dissimilar values. Here we
have swapped f(2) and f(4)
1
-1
-1
0
1
1
0
1
-1
0
1
1
1
-1
1
0
1
1
1
-1
-1
1
-1
-1
19
Getting in the Right Area

• Previous work (QUT) has shown strongly
• Heuristic techniques can be very effective for
  cryptographic design synthesis
• Boolean function, S-box design etc
• Hill-climbing works far better than random search
• Combining heuristic search and hill-climbing
  generally gives best results
• Aside notion applies more generally too - has
  led to development of memetic algorithms in GA
  work.
• GAs known to be robust but not suited for fine
  tuning.
• We will adopt this strategy too use simulated
  annealing to get in the right area then
  hill-climb.
• But we will adopt the new cost function for the
  first stage.

20
Hill-climbing With Traditional CF (n8)
21
Varying the Technique (n8)
Non-linearity
Non-linearity
Non-linearity
Autocorrelation
Simulated AnnealingWith Traditional CF
Simulated AnnealingWith New CF
Simulated AnnealingWith New CFHill Climbing
With Traditional CF
22
Tuning the Technique

• Experience has shown that experimentation is par
  for the course with optimisation.
• Initial cost function motivated by theory but the
  real issue is how the cost function and search
  technique interact.
• Have generalised the initial cost function to
  give a parametrised family of new cost functions

Cost(f)S F(w)-(2 n/2K) R
23
Tuning the Technique (n8)
Non-linearity
Autocorrelation
Illustration of how results change as K is
varied400 runs
24
Tuning the Technique (n8)
Non-linearity
Autocorrelation
Further illustration of how results change as K
is varied. 100 Runs
25
Comparison of Results
26
Summary and Conclusions

• Have shown that local search can be used
  effectively for a cryptographic non-linear
  optimisation problem - Boolean Function Design.
• Direct cost functions not necessarily best.
• Cost function is a means to an end.
• Whatever works will do.
• Cost function efficacy depends on problem,
  problem parameters, and the search technique
  used.
• You can take short cuts with annealing parameters
  (and computationally there may be little choice)
• Experimentation is highly beneficial
• should look to engaging theory more?

27
Future Work

• Opportunities for expansion
• detailed variation of parameters
• use of more efficient annealing processes (e.g.
  thermostatistical annealing).
• evolution of artefacts with hidden properties
  (you do not need to be honest - e.g. develop
  S-Boxes with hidden trapdoors)
• experiment with different cost function families
• multiple criteria etc.
• evolve sets of Boolean functions
• other local techniques (e.g. tabu search, TS)
• more generally, when do GAs, SA, TS work best?
• investigate non-balanced functions.