Josh Bongard

1
July 9, 2007
Automated Reverse Engineering of Nonlinear
Dynamical Systems
Josh Bongard Hod Lipson Computational
Synthesis Laboratory Cornell University hod.lipson
_at_cornell.edu Current Address Department of
Computer Science University of Vermont josh.bongar
d_at_uvm.edu
2
Model d?/dt ? d?/dt ?
3
Best evolved model d?/dt 1.008?
0.0028 d?/dt -19.43sin(1.0009?0/-1.575/-2.673)
f(?,?) f friction term
Best human-created model d?/dt ? d?/dt
-9.8Lsin(?) f(?,?) L length of arm
First reported algebraic model describing pendula
published by H. Kamerlingh Onnes in 1879. Form
of these equations still presented to this day in
engineering textbooks as the mathematical
description of a damped pendulum. Meets criteria
(B) Our model matches the best-known model
form for a pendulum. Our models contain
explicit terms for the friction term f
friction terms are unknown for physical, damped
pendula dependent on mechanical coupling,
shape of arm, wind resistance, etc. Our
model automatically captures in symbolic
form, changes to the system (rotation of the
main box).
Rotated -1.57rad
Box is flat
Rotated -2.67rad
4
Best evolved model dG/dt ? dA/dt ? dL/dt ?
5
Best evolved model dG/dt 0.96A2/(0.96A21) dA/d
t G( L/(L1) A/(A1) ) dL/dt -GL/(L1)
dG/dt A2/(A21) 0.01G 0.001 dA/dt G(
L/(L1) A/(A1) ) dL/dt -GL/(L1)
6
Best evolved model dG/dt 0.96A2/(0.96A21) dA/d
t G( L/(L1) A/(A1) ) dL/dt -GL/(L1)
Lac operon discovered in 1961 Jacob F, Monod J
(1961). "Genetic regulatory mechanisms in the
synthesis of proteins". J Mol Biol. 3
318-56. As of 1998, the best human-created model
was from Keener J, Sneyd J (1998)
Mathematical Physiology (Springer). Criteria
(B) The result is equal to or better than a
result that was accepted as a new scientific
result at the time when it was published in a
peer-reviewed scientific journal. Took humans 37
years to create this model from
observations Took our algorithm 5 minutes
running on a desktop PC to create a very similar
model from time series data.
dG/dt A2/(A21) 0.01G 0.001 dA/dt G(
L/(L1) A/(A1) ) dL/dt -GL/(L1)
7
Best evolved model dH/dt ? dL/dt ?
8
Best evolved model dH/dt 3.42x106 - 67.82H -
10.97L dL/dt 3.10x105 32.66H - 63.16L
9
Best evolved model dH/dt 3.42x106 - 67.82H -
10.97L dL/dt 3.10x105 32.66H - 63.16L
Best human-created model dH/dt aH
ßHL dL/dt -?L dHL
Proposed independently by American biophysicist
Alfred Lotka and Italian mathematician Vito
Volterra in 1925. (Volterra, V. (1926) Jour. du
conseil intl. pour lexp. de la mer) Still forms
the basis of most population dynamics models
today. Meets criteria (B) Our model is similar,
but simpler No nonlinear terms, yet still
conveys which species in the prey, and which
the predator prey decreases proportionally to
amount of predator predator increases
proportionally in response to prey when
integrated, produces coupled oscillations predat
or peaks follow prey peaks.
10
Criteria (G) The result solves a problem of
indisputable difficulty in its field.
Algorithm presented with time series data from
nonlinear, coupled systems of increasing
size. Algorithm correctly inferred the underlying
form in 5 minutes.
System 1 System 2 System 3
2 variables dx1/dt -3x1x1 -3x1x2 2x2x2 dx2/dt -x1x1 -3x1x2 -2x2x2 dx1/dt-3x1x1 3x1x2 3x2x2 dx2/dt-3x1x1-2x1x2 2x2x2 dx1/dt3x1x1 -x1x2 -x2x2 dx2/dtx1x1 3x1y2 -x2x2
Success rate 20 (no partitioning) 100 (partitioning) 0 (no partitioning) 96.7 (partitioning) 0 (no partitioning) 100 (partitioning)
3 variables dx1/dt-3x1x3 -2x2x3 -3x3x3 dx2/dt-3x1x2 x1x3 -3x2x3 dx3/dt3x1x2 3x1x3 x2x3 dx1/dt-x1x2 x1x3 -x2x3 dx2/dtx1x1 2x1x2 2x2x3 dx3/dt-2x1x1 x1x2 -3x2x3 dx1/dt-3x1x2 x1x3 -x3x3 dx2/dt-2x1x3 3x2x3 3x3x3 dx3/dt2x1x2 -2x1x3 -2x2x3
Success rate 0 (no partitioning) 63.3 (partitioning) 0 (no partitioning) 100 (partitioning) 0 (no partitioning) 96.7 (partitioning)
4 variables dx1/dt-x1x12x2x32x3x3 dx2/dtx1x2 -3x1x3 -3x2x3 dx3/dt-x1x1 -x2x4 3x4x4 dx4/dt-3x1x2 -3x1x4 3x3x4 dx1/dtx1x4x2x4x4x4 dx2/dt-3x1x2-2x2x3-3x3x4 dx3/dt2x1x2-x1x32x2x2 dx4/dtx1x33x2x3-x3x4 dx1/dt-3x1x13x1x23x2x4 dx2/dt-x1x1-2x1x3-3x4x4 dx3/dt-2x1x4x2x2-3x3x4 dx4/dt-x1x22x1x4-3x3x4
Success rate 0 (no partitioning) 90 (partitioning) 0 (no partitioning) 83.3 (partitioning) 0 (no partitioning) 90 (partitioning)
5 variables dx1/dt-3x1x5 3x2x3 -3x2x5 dx2/dt-3x1x3 -2x3x4 -x4x5 dx3/dtx1x1 -3x1x4 x2x4 dx4/dt3x1x3 -3x1x4 2x2x2 dx5/dt3x1x4 3x3x3 3x3x4 dx1/dt-2x2x23x3x52x4x5 dx2/dt3x1x2 x1x5 -2x2x5 dx3/dtx1x2 2x2x5 2x4x5 dx4/dt2x1x2 3x1x5 -x4x5 dx5/dt2x1x5 -x2x5 -2x5x5 dx1/dt2x1x4 2x2x3 -x2x4 dx2/dtx1x3 3x1x4 x2x4 dx3/dt-2x1x1 2x1x2 -3x1x3 dx4/dt-3x2x5 3x3x4 -x3x5 dx5/dtx1x1 x1x5 x2x3
Success rate 0 (no partitioning) 76.7 (partitioning) 0 (no partitioning) 76.7 (partitioning) 0 (no partitioning) 76.7 (partitioning)
6 variables dx1/dt-2x1x6x2x4-2x2x6 dx2/dtx1x4-x1x5-2x4x4 dx3/dt2x2x5-x3x4x5x5 dx4/dt-3x4x5-2x4x62x5x5 dx5/dtx3x6-2x4x4-3x4x5 dx6/dtx3x4-x3x62x4x6 dx1/dt-2x1x3-3x2x42x3x6 dx2/dt-3x2x4x3x4-x3x6 dx3/dt-x1x2-x1x3x4x6 dx4/dt-x1x4x3x5-2x4x6 dx5/dt3x1x2-3x1x6-x5x5 dx6/dt-3x1x3-2x1x6-3x4x6 dx1/dtx1x5x1x6x4x5 dx2/dt-2x2x5-2x2x62x3x6 dx3/dt-x1x5-2x3x4x4x4 dx4/dt3x1x23x2x3-2x4x5 dx5/dt-3x1x5x2x23x2x6 dx6/dt-x2x5-2x3x5-3x5x6
Success rate 0 (no partitioning) 80 (partitioning) 0 (no partitioning) 70 (partitioning) 0 (no partitioning) 93 (partitioning)
2 variables
Best-known predator-prey model single pendulum
model
3 variables
Best-known lac operon model of E. coli as of 1998
5 variables
Best-known lac operon model of E. coli as of
2003 (Yildirim Mackey, 2003)
11
Why should this entry be considered best? Work
appeared in one of the worlds most prestigious
scientific journals, and not in an evolutionary
computation journal Was favorably reviewed by
four renowned scientists outside of the EC
community. One reviewer remarked that this
approach has a chance of changing the way some
disciplines conduct science.
CitationBongard J. and Lipson H.(2007).
Automated reverse engineering of nonlinear
dynamical systems. Proceedings of the National
Academy of Sciences, 104(24) 9943-9948.