ART Artificial Reasoning Toolkit Evolving a complex system

1
ART Artificial Reasoning ToolkitEvolving a
complex system
Marco Lamieri lamieri_at_econ.unito.it
2

• Agenda
• Overview on genetic algorithm
• Create a mind for agents learning classifier
  system
• Some improvement to the methods, the ART project
• Our theoretical applications with ART
• Move to real world with an industry application
  the Penelope project
• References

3
The idea of genetic algorithm

• Starting from Survival of the fittest Darwin,
  1859
• Genetic Algorithms (GA) are evolutionary programs
  that manipulate a population of individuals
  represented by fixed-format strings of
  information.
• The background theory is the adaptation in
  artificial system discussed by Holland 1975.
• GA are used to solve real-world optimization
  problems within a very large solution space and
  ill-defined problems.

4
How does a GA work

• An initial population of individuals (solutions)
  is generated
• individuals represent potential solutions to the
  given problem and are described as binary
  strings
• each character in the individuals data string is
  called a gene and each possible value that the
  gene can take on is called an allele.
• parents and individuals that are going to survive
  to the next generation are selected using a
  fitness proportional mechanism
• The selected individuals are evolved by means of
  reproduction using two operators
• crossover,
• mutation.
• Process go on untill the population almost
  converge to a specific individual.

5
Example (square root of 2)

• The solution space is bounded between 0 and 1.
• We use a binary representation on 10 digits.
• There are 1024 numbers 210, starting from 0
  and ending at 1023 210 -1.

6
Generate random population

• A population of solutions is generated randomly.
• For the square root problem, a fixed number of 10
  character binary strings are generated randomly.

7
Define the fitness function

• Darwinian evolution of a population implies that
  the strongest individuals will probably survive.
• The fitness of an individual is a numerical
  assessment of that individuals ability to solve
  the problem - it is the ability of the individual
  to satisfy the requirements of the environment.
• In terms of the square root problem, the perfect
  individual is the numerical value approximated by
  1.414213562373.
• In economic problems,the profit can be used to
  generate a fitness function

8
(No Transcript)
9
Selection process (roulette wheel)

• To select individuals is used the roulette wheel
  technique.
• The roulette wheel implementation implicitly
  forces fitness-proportionate reproduction.
• Selection is divided in 2 steps
• Individuals that are going to survive to the next
  generation are selected
• Individuals that are going to reproduce are
  selected.

10
Crossover

• Crossover swaps some of the genetic material of
  two individuals, creating two new individuals
  (children), who are possibly better than their
  parents.

11
Mutation

• In order to recover from this loss of genetic
  material, the individuals are allowed to change
  their genes randomly.

12
Convergence

• John Hollands Schema Theorem Holland, 1992 is
  widely accepted as mathematical proof that the
  genetic algorithm, due to its fitness-proportionat
  e reproduction, converges to better solutions.
• Via the convergence method is possible to solve
  non well-defined problems where the best
  solution is not known a priori.

13
Remarks

• There is no ultimate goal or problem that must be
  solved by natural evolution.
• Evolution itself does not guarantee the creation
  of fitter individuals.
• The GA use a fuzzy logic that not always lead to
  the best solution but to a good one.
• The algorithm is problem independent.

14
ART Some improvement to the method

• ART, starting from John Holland's work,
  introduces some extensions and innovations
• extended alphabet each gene can be represented
  by up to 32000 values. In a standard
  representation the genes have a binary alphabet
  and can become meaningless. With the extended
  alphabet each allele can be a meaningful part of
  the solution and the translation process is
  easier.
• multi genome the multi genome schema give a
  high degree of freedom to the user in formalizing
  problems in which coexist different binded
  aspects.
• rescale fitness operatorthe natural selection
  process has been modified in order to improve
  efficiency and manage negative fitness values.
• univocal genome using this option each value of
  the alphabet is unique within the genome.

15
Classifier systems

• Classifier system is an induction self-learning
  system based on a set of simple logical rules
  called classifiers.
• Each rule has the following structure
• if condition then action.
• During learning process rules priorities
  (strengths) are changed. In case of success
  current and previous activated rules are
  encouraged. Evolutionary methods are used for new
  rules searching.
• CS consist of four principal components
• List of classifiers (population of classifiers).
• List of messages that plays the role of a
  message board for communications and short term
  memory.
• Input interface (detector) that represents the
  environment state.
• Output interface (effector) that ensures
  interaction with the environ- ment or its change.

16

• The computations of a CS are the follows
• add messages obtained with the help of the input
  interface to the input message list
• compare all the messages on the message list with
  the condition parts of all classifiers and
  remember all the classifiers for which
  coincidence has been observed
• select by a auction some classifier actions
• pass the messages obtained through the output
  interface
• replace the contents of the message list with new
  messages from the environment
• if a classifier chain is implemented add to
  message list also the classifiers action
• auctions bid is paied using the bucket brigade
  algorithm.

17
ART CS Some improvement to the method

• Rewards Memory
• the metaphor is that in most difficult situation
  a sub optimal response can be acceptable (if a
  trading strategy was successful in the past it
  could earn again money in the future) while the
  opposite is true for a easy situation (if it is
  raining bring umbrella) and a sub optimal action
  is not acceptable
• the reward given to a specific action is
  internally computed in respect to the average
  past reward.
• If a strategy is harder to grasp than a reward
  higher than the average of the previous rewards
  has to be considered successful and at the
  opposite a non perfect outcome of a very easy
  strategy has to be considered a bed one.
• the memory is limited by a parameter in order to
  emulate the cognitive limit of agents

18
Our applications

• Genetic algorithm
• Find the weight of a artificial neural network
  in order to solve the XOR problem
• Find the Cournot equilibrium in a simplified
  simulated economic
• Penelope

• Classifier system
• Suggest diagnosis for chest pain where symptoms
  are conditions and diagnosis are actions.
• Evolve a good strategy to play Minority Game
  introducing communication among agents
• Evolve a good strategy for the iterated prisoner
  dilemma where the mind of the agents is a
  classifier system.
• Evolve a good strategy for the Minority Game

19
References

• ART project http//eco83.econ.unito.it/golem
• Penelope project http//www.penelopeproject.org
• This presentation is available at
  http//eco83.econ.unito.it/dottorato/marco_lamieri
  /20050114-isiWinterSchool-art.ppt
• For any further information
• lamieri_at_econ.unito.it
• morini_at_econ.unito.it