AN INFORMATION-THEORETIC PRIMER ON COMPLEXITY, SELF-ORGANISATION

1
AN INFORMATION-THEORETIC PRIMER ONCOMPLEXITY,
SELF-ORGANISATION EMERGENCEMikhail
ProkopenkoCSIROFabio BoschettiCSIROAlex
RyanDSTO
2
a Complex System story

• many, but not too many, components interact in a
  non trivial fashion
• the system is open (receives energy/information/m
  atter from the environment)
• interactions ? symmetry breaking ? coordinated
  behaviour arises
• no central director/template ? the system
  self-organises
• coordination as patterns detectable by an
  external observer or structures convey new
  properties to the systems itself
• new behaviours emerge from the system
• coordination and emergence may arise from
  response to environment ? adaptation
• when adaptation occurs across generations at a
  population level we say that the system evolved
• now, at new scale, the system can be identified
  as a novel unit
• this becomes the building block ? new cycle at a
  new scale

3
Aim

• A framework in which concepts like
• complexity,
• emergence,
• self-organisation,
• adaptation and evolution
• can be
• described,
• distinguished
• defined consistently

4
Framework

• We chose an information-theoretic framework
• we borrow from work pioneered by the Santa Fe
  Institute
• it provides a well developed theory
• definitions can be formulated mathematically
• some computational tools are readily available.

5
Complexity

• Predictive Information diversity (signal) -
  non-assortativeness (diversity between past and
  future)
• Excess Entropy Richness of structure
• Statistical Complexity minimum Memory for
  optimal predictions

6
Complexity

• Predictive Information Excess Entropy
  Statistical Complexity
• Predictive information amount of structure
  memory for optimal predictions
• The memory needed for optimal prediction cannot
  be lower than the structure contained in the
  data, i.e., the mutual information between the
  past and future.

7
Self-Organisation
Organisation increase in complexity
The more a system organises ? the more behaviours
it can display ? the more effort is needed to
describe its dynamics.
8
Self-Organisation
Self spontaneous the amount of information
flowing from the outside is strictly less than
the change in statistical complexity IOutside lt
C(t?t)- C(t) Increases in organisation gt
Information received. Or Energystimulus lt
Energyresponse
9
Emergence
There are classes of phenomena, which when
observed at different levels, display behaviours
which appear fundamentally different. What level
should we choose for our analysis? The level at
which it is easier or more efficient to construct
a workable model. Efficiency of Prediction
Excess Entropy / Statistical Complexity (Shalizi)
How much can be predicted / How difficult it
is to predict
10
Adaptation and Evolution
Adaptation increase in the mutual information
between the system and the environment. Evolutio
n increases the amount of information a
population harbors about its niche" (Adami) I
(Environment, Population) Entropy (Population)
Entropy (Population Environment) entropy
in the absence of selection (Max Population
Entropy) - diversity tolerated by selection in
the given environment how much data can be
stored in the population - how much data
irrelevant to environment is stored
11
Concept Plain English Information Theory
Complexity The amount of information needed to describe a process / system / object. PI diversity - non-assortativeness PI E C
Self-Organisation Spontaneous increase in complexity InfOutside lt C(t?t)- C(t)
Emergence Presence of behaviours which appear fundamentally different when observed at different levels. Efficiency of Prediction eE/C
Adaptation / Evolution Increase in the mutual information between the system and its environment. I (Pop, Env) H (Pop) H (Pop Env)
12
a Complex System story

• many, but not too many, components interact in a
  non trivial fashion
• the system is open (receives energy/information/m
  atter from the environment)
• interactions ? symmetry breaking ? coordinated
  behaviour arises
• no central director/template ? the system
  self-organises
• coordination as patterns detectable by an
  external observer or structures convey new
  properties to the systems itself
• new behaviours emerge from the system
• coordination and emergence may arise from
  response to environment ? adaptation
• when adaptation occurs across generations at a
  population level we say that the system evolved
• now, at new scale, the system can be identified
  as a novel unit
• this becomes the building block ? new cycle at a
  new scale

13
(No Transcript)
14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
(No Transcript)