CAP6938 Neuroevolution and Developmental Encoding Working with NEAT

1
CAP6938Neuroevolution and Developmental
EncodingWorking with NEAT

• Dr. Kenneth Stanley
• September 27, 2006

2
Exploiting Speciation

• Keep a list of species with increasing IDs
• Species elitism duplicate the species champs in
  the next generation
• Only if species is above size 4 or 5
• Irrelevant in rtNEAT
• Stagnation Detection
• Track generations since last improvement for each
  species
• Drop fitnesses of stagnant species to near 0

3
Crossover Issue

• Two different connections can end up connecting
  the same nodes
• Maybe disallow

5
9
4
Look Out for Bugs

• Gene pointing to nodes that arent in Gemome
• Gene innovation numbers out of order
• Node innovation numbers out of order
• Duplicate genes
• May want to check for these situations after
  major operations during debugging
• See Genomeverify() in NEAT C
• Can check whole population in a loop

5
Network Activation

• Use standard arbitrary-topology activation
• (Lecture 3)
• Do not activate in layers!

6
Real-time Evolution

• If all your agents are evaluated simultaneously,
  rtNEAT may be more appropriate
• rtNEAT produces and replaces one individual at a
  time (no generations)
• See REAL-TIME NEUROEVOLUTION IN THE NERO VIDEO
  GAME on my homepage
• Also see NERO
• rtNEAT source available

7
Your Genes May Have Extra Fields

• Time constants in nodes
• Activation function in nodes
• Learning parameters in connections
• You can have whatever you want, whatever you are
  evolving
• Should extra params affect compatibility?
• Remember, more fields means more possibilities,
  but also more dimensions

8
Important Methods

• Crossover mate_multipoint and mate_multipoint_avg
  
• Mutation mutate_add_connection,
  mutate_link_weights, mutate_enable_bits, other?
• Special Genomes Operations duplicate, compare
  (getting the delta for compatibility distance)
• reproduce (inside Species)
• epoch (inside Population)
• Remember http//www.freshraisins.com/neatdoc/file
  s/genetics-cpp.html

9
Developmental Encoding and Non-neural NEAT

• Your genes may not map directly to pieces of
  structure
• However, genes can still be marked with
  historical markings and complexification can
  still occur
• Need a standard starting structure

10
Sensors Must Be Chosen Wisely
11
Noisy Evaluation

• Randomness in an environment can cause evaluation
  to be inconsistent
• For example, in a game of chance
• Same individual can receive different fitness on
  different trials
• Causes confusion for evolution
• Not always fatal, but merits attention
• Maybe give all individuals in same generation
  same evaluation sequence

12
Saving and Loading
/ Species 3 (Size 5) (AF 6.03094) (Age 29)
/ / Organism 0 Fitness 8.77021 Error
1.03855 / genomestart 0 trait 1 0.1 0 0 0 0 0 0
0 trait 2 0.2 0 0 0 1 0 0.21903 0 trait 3 0.65
0.110145 0 0 0 0 0 0 node 1 3 1 3 node 2 3 1
1 node 3 3 1 1 node 4 1 0 2 node 9 1 0 0 node 86
1 0 0 gene 3 1 4 -0.798127 0 1 -0.798127 1 gene 3
2 4 3 0 2 3 1 gene 2 3 4 -3 0 3 -3 1 gene 2 3 9
0.505339 0 12 0.505339 1 gene 1 9 4 1.02135 0 13
1.02135 1 gene 1 1 9 -3 0 95 -3 1 gene 1 2 9
2.28649 0 266 2.28649 0 gene 1 2 86 -1.22499 0
281 -1.22499 1 gene 1 86 9 1.72795 0 282 1.72795
1 genomeend 0 / Organism 1 Fitness 5.37326
Error 1.68197 / genomestart 1 trait 1 0.1 0 0 0
0 0 0 0 trait 2 0.2 0 0 0 1 0 0.21903 0 trait 3
0.65 0.110145 0 0 0 0 0 0 node 1 3 1 3 node 2 3 1
1 node 3 3 1 1 node 4 1 0 2 node 9 1 0 0

• Genomes/Populations should be savable and
  loadable
• Important for debugging
• Very disappointing to lose a good thing forever
• Some people use standards like XML

13
Modularity Means Future Expansion (and maybe
other users)

• Think about what you might want to do someday
• Add new parameters to nodes?
• Add new parameters to links?
• Change the mapping?
• Change platforms?
• Completely different experiments?
• More visualization?
• Change phenotype?
• Change genetyoe?

14
Too Much Scope Too Much Rope

• Go for proof of concept
• What feature matters least?
• If nothing works what could be a prerequisite
  task?
• Incremental evolution strategies

15
Next Class Developmental Encoding

• The power of reuse
• Prior work
• Biological underpinnings
• The future

A Taxonomy for Artificial Embryogeny by Kenneth
O. Stanley and Risto Miikkulainen (2003)