Model Reduction for Parameter Estimation

1
Model Reduction for Parameter Estimation

• Eric Mjolsness
• Scientific Inference Systems Laboratory (SISL)
• University of California, Irvine
• www.ics.uci/edu/emj
• and Caltech Biological Network Modeling Center
  (BNMC)
• in collaboration with Rebecca Castaño, Dasha
  Chudova, Michael Duff, Victoria Gor, Henrik
  Jönsson, Tobias Mann, George Marnellos, Elliot
  Meyerowitz, John Reinitz, Bruce Shapiro, David
  Sharp, Padhraic Smyth, Yuanfeng Wang, Barbara
  Wold, Guy Yosiphon, Li Zhang
• Parameter Estimation In Systems Biology (PESB)
• Pascal Workshop, Manchester, UK
• March 28, 2007

2
Topics

• A long-running thread in parameter estimation
• Biological applications
• transcriptional regulation
• development
• Perspectives
• a near-universal bio. modeling language and
  semantics
• and its implications for
• parameter estimation and model reduction

3
Transcriptional Gene Regulation Networks

• Gene Regulation Network MSR91 model

E.g. Drosophila A-P axis
Drosophila gap gene expression patterns. Reinitz,
Mjolsness, Sharp, Journal Experimental Zoology
271(47-56) 1995. Fitting method demonstrated in
Mittenthal and Baskin, The Principles of
Organization of Organisms, Addison Wesley 1992.
Mjolsness et al. J. Theor. Biol. 152 429-453,
1991
4
GRN Parameter Optimization

• Simulated Annealing 1990, 92
• Lam/Delosme SA for real-valued params
• Gap genes JEZ 271(47-56) 1995
• 33 real-valued parameters
• Genetic Algorithm
• Distributed over islands with migration, for
  diversity
• SA, GA compared in G. Marnellos thesis 1997
• GA won on evolution (life history) problems
• SA won on development problems
• Other apps to GRNs and signaling Gor, Zhang
• Then many others. Recently
• Kozlov BGRS 2006 differential evolution
• Tomlin 2006 Adjoint method BP/cont. time

5
GRN ANN Equations 91

• Model statement and its derivation from stat
  mech
• Mjolsness Sharp and Reinitz, J. Theor. Biol.
  152 429-453, 1991

Key properties (1) additivity, (2) saturation
above and below, (3) monotonicity.
6
Model Reduction Example Gene Regulation
NetworkDerived from Stat Mech
J. Theor. Biol. 152 429-453, 1991

• MSR91 equations are no longer just
  phenomenological.

J.Bioinformatics Comp. Biology, in press 2007
7
Dynamical Model Reduction via Clustering

• Core/Halo Models
• From Coexpression to Coregulation NIPS 1999
  p.928-34
• Identifiability by Gibbs sampling
• Duff et al., ICSB 2005
• Functional Mixture Models Chudova et al. NIPS
  2003

8
Core/Leaf Model Inference

• 3-node oscillator leaves
• Modeled by SDE
• topologies
• Identifiability
• x25 time points identifiable
• x10 points not identifiable
• x10 points x2 genotypes identifiable (ranked
  3)
• Duff et al. ICSB2005

9
SDE Advantages

• Intermediate cost for stochastic simulation
• Relationship to stochastic optimization
• Derivation from Fokker-Planck equation
• Eg. for GRN, HCA JBCB in press 2007

10
Hierarchical Cooperative Activation Alternative
diagram notations

• Bio-like
• Machine learning

11
Hierarchical Cooperative Activation Model (HCA)
In Computational Methods in Molecular Biology,
eds. J. M. Bower and H. Bolouri, MIT Press 2001
12
How to model transcriptional regulation?
E.g. Drosophila D-V axis

• Robert P. Zinzen, Kate Senger, Mike Levine, and
  Dmitri Papatsenko. Current Biology 16, 18, July
  11, 2006

13
Hard vs. Soft Logic
Hierarchical Cooperative Activation (HCA)
Zinzen et al. modification
Experiment Yuanfeng Wang, UCI Physics
14
HCA- Z and ANN-like Equations
A model reduction

• Assume many binding sites per module
• Assume extreme (usually low) occupancy per site

where
15
GRSN Gene Regulation Signal Transduction
Network

Marnellos, Mjolsness, Shapiro
L
Drosophila neurogenesis Marnellos, Mjolsness PSB
98 Xenopus ciliated cells PSB 00
cell
ligands
nucleus
Arabidopsis SAM Gor, Mjolsness,Meyerowitz, NASA
Evolvable Hardware 99
receptors
T
transcriptional regulation targets
16
Arabidopsis Shoot Apical Meristem (SAM)
17
WUS
Fletcher et al., Science v. 283, 1999
Brand et. al., Science 289, 617-619, (2000)
18
SAM growth imageryH2B cell nuclei
V. Reddy, Caltech
19
CLV3/WUS networks
V. Agrawal, B. Shapiro, Caltech
20
CLV/WUS model behavior
Activation domains in Cellerator model WUS
(yellow), CLV3I1 (green), CLV3 (blue and purple),
CLV1 (red and purple).
B. Shapiro, JPL/Caltech
21
CLV/WUS Parameter Optimization by SA
14 parameters
Courtesy H. Jönsson 2007 cf. ICSB 2006
22
Biological scale hierarchies
Perspective
mutant

• Biology, networks, models

Noun and verb hierarchies
wild type
23
Dynamical Grammar Aims

• Biology Model complex systems
• developmental biology (fly embryo, plant
  shoot/root)
• molecular complexes
• multiple-scale, heterogeneous, variable-structure
  systems
• Mathematics Capture, unify, extend techniques
• Generalized reactions cover all processes
• Operator algebra, perturbation theory,

Annals of Math. and A. I., 47(3-4), January 2007
24
Elementary Reactions

• A? B C with rate kf
• B C? A with rate kr
• Effective conservation laws
• E.g. NA NB, NA NC

25
Elementary Processes

• A(x) ? B(y) C(z) with rf (x, y, z)
• B(y) C(z) ? A(x) with rr (y, z, x)
• Examples
• Chemical reaction networks w/o params
• .
• XXX from paper
• Effective conservation laws
• E.g. ? NA(x) dx ? NB(y) dy ,
• ? NA(x) dx ? NC(z) dz

26
Elementary process models

• Composition is by independent parallelism
• Create elementary processes from yet more
  elementary Basis operators
• Term creation/annihilation operators for each
  parm value,
• Obeying Heisenberg algebra
• Yet classical, not quantum, probabilities

27
SPG Modeling Language Semantics Semantic map Y
G?H from Grammar to Stochastic Process

• Commutative diagrams for composition operations
• Translation of a Rule

G
G
?
?
?
S
H, dp/dt
H, dp/dt
28
Time Ordered Product Expansion (TOPE)

• Time Ordered Product Expansion (TOPE) formula
• H0 the easy part (if only recursively)
• Feynman diagrams result (QFT Perturbation
  theory, Wicks theorem)
• Gillespie stochastic simulation algorithm
• H0 diag( 1 H) H1 H
• Mixed (heterogeneous) ODE/SSA algorithm (novel)
• Implemented in Plenum (Yosiphon)

Annals of Math. and A. I., 47(3-4), January 2007
29
Model Reduction for Dynamical Systems

• Diagram
• Objectives
• Thus, parameter estimation can aid model
  reduction
• Uses of diagram

UCI ICS TR 05-09
30
Composition vs. Specializationin a Lattice of
Models

• Orthogonal kinds of model reduction/expansion
  (PartOfInA, IsA)
• Commutative diagram for model lattice
• Specialization eg. discretized (DBN) vs.
  continuous (ODE) vs. quantized (stochastic) vbls,
  time, space - heterogeneous dynamics
• Initialize param search in specialized model
• high-level vision app NIPS 1990
• Thus, model reduction can aid parameter estimation

31
A Parameter Estimation Future

• Parameter estimation ? model reduction
• Multiscale, heterogeneous, variable-structure,
  models all incorporated in a lattice
• Common (operator algebra) semantics
• Perpetual data assimilation
• Continual influx of data
• Perpetual fitting to an expanding lattice of
  models
• Specialize to the limit of identifiability
• Model analyses to explain the hits

32
Conclusions

• Model reductions for transcriptional regulation
• GRN91, HCA
• Model reduction for large-scale data
• Core/Halo, Functional Mixture, models
• Common framework generalized reactions
• Dynamical grammars ? operator algebra
• Parameter estimation ? model reduction
• Mutually enhancing interaction

33

• For further information
• www.ics.uci.edu/emj
• www.computableplant.org
• Funding US National Science Foundation FIBR
  program,
• NIH BISTI program
• Invitation

34
(No Transcript)