Using Novel Hybrid Neural Networks for In Silico Metabolic Modeling

1
Using Novel Hybrid Neural Networks for In Silico
Metabolic Modeling

• By Heather Richardson
• And Loretta Macklem
• Supervised by Dr. Zohdy

2
What are Neural Networks?

• Bio-inspired networks of neuron-like systems that
  work together to compute objects.
• Inspired by the brain in two ways
• Knowledge is acquired by a learning process
  (trial and error)
• Synaptic weights are used to store knowledge

3
Advantages of using NN

• Learning
• Robustness
• Adaptability

4
How do NN Work

• Train the network
• Present data to the network
• Network computes output
• Output compared to target output
• Weights are modified to reduce error
• Use Network
• Present new data to the network
• Network computes output based on training

5
Types of NN

• FF
• A set of input nodes, hidden nodes and output
  nodes.
• Simple functions calculated on each node
• Elman and Hopfield
• Start with input and network adjusts weights and
  feeds back into the network until the test data
  fits the target data

6
Function Approximation

• Process modeling, process control, data modeling,
  machine diagnostics
• Y f(X)
• X and Y are a set on inputs and outputs
• F() is an unknown functional relationship between
  the two.
• NN approximates f() to find the appropriate
  output for the input

Function Approximation Create
continuous Input-output map
Numerical inputs
Numerical outputs
7
Classification

• Types of classification
• Target recognition
• Character Recognition
• Speech recognition
• Uses discrete values
• Evaluates by thresholding

Numerical outputs
Function Approximation except with On/off outputs
Interpret output As classes/decisions
On/Off Outputs
Numerical inputs
8
Time Series Prediction

• Very similar to function approximation except
  time plays an important role
• In static function approximation, all
  information needed to create output is determined
  in the current input (E.g. image processing)
• In time series prediction (dynamic function
  approximation), information from the past is
  needed to determine the output. (stock price
  predictor)

Function Approximation With future values as
output
Past and Present values of input parameters
Future values to be predicted
Desired Prediction
Delay time series
Function Approximation With future values As
output
Network Prediction
Time Series
Delay
9
Other Applications

• Learn flaws in products
• Flash flood predictor
• Road Navigation

10
What is Metabolism?

• Metabolism describes the process of breaking down
  or creating substances in living things
• Metabolic Pathways describes the order of
  processes involved with metabolism
• Like a miniature Chemical Factory

11
Why

• With the aid of modern biological knowledge of
  the Genome, using genetics to model processes is
  the new field for Biology
• Start with ECOLI, then yeast, then fruit fly,
  then.. Humans!!!

12
Our Goals

• Design a NN based suite of programs that are
  able to predict stable metabolite and biomass
  concentrations given the initial concentrations
  and environmental inputs
• Optimize the accuracy of these programs by adding
  context and hint features to it

13
Escherichia coli K12 MG1655
14
Definitions

• Protoheme(C34H32N4O4Fe)
• An essential iron compound
• Siroheme (C34H32N4O4Fe)
• Another iron compound
• Tryptophan(C11H12N2O2,)
• An essential amino acid dietary component
• Glutamine(C5H10N2O3,)
• An amino acid used to make proteins

15
More Definitions

• Phenylalanine(C9H11NO2) and Tyrosine(C9H11NO3)
• Amino acids which leads to the formation of
  adrenaline which causes alertness, suppresses
  appetite, and elevates mood

16
Our Biosynthesis Pathways

• Protoheme and Siroheme production
• Tyrosine production
• Tryptophan production
• Phenylalanine production

17
Our degradation and Signal Transduction Pathways

• Degradation
• Glycolysis
• process by which glucose is broken down
• Signal Transduction
• Nitrogen signal transduction
• Varies production of glutamate and glutamine
  depending on nitrogen levels
• Low nitrogen more glutamine gets produced

18
Our Functional blocks
They displayed in their hierarchy from top to
bottom!
19
Half of Glycolysis pathway
20
Four alternative uses of NN

• Single NN, feature and context free
• 3 types, Elman, FF, and Elman with time
• Single NN with one of the key features
• key features are Inheritance Encapsulation,
  Context, and Hints
• Many NN(polymorphic) program with all features
  except hints
• An all feature(fully loaded) polymorphic NN
  program

21
PIECH

• Polymorphic
• Many networks
• Inheritance
• Large networks that inherit from smaller
• Encapsulation
• Using a smaller network for a simpler version of
  the problem
• Context
• Providing type of pathway
• Hints
• Providing known knowledge about the pathways

22
Our first NN(NO PIECH)

• We have the Elman and a FF network
• Initial/final
• 1 Elman with time
• Each tries to model all of the Pathways
• Neither were accurate
• However, Elman showed less error

23
Results
These are graphs of the error shown by our Elman
and FF initial/final
24
Time Results
Our network produced this ouput!
25
More results
Network output
26
Intermediate Performance Networks

• We take the Elman with time and added one of the
  features to it
• One with Inheritance
• One with Encapsulation
• One with Context
• One with Hints

27
Inheritance

• We created a copy of our original Elman network
• Added 5-15 neurons to the first layer
• This creates a larger network
• By adding more neurons, our accuracy improved,
  without have to train on new data sets

28
Results with I(added 5 neurons)
29
Results with I(added 5 neurons)
30
Results with I(added 15 neurons)
31
Results with I(added 15 neurons)
32
Encapsulation

• Give more general data to a simpler network
• Have it try to make the same decision as the more
  specific and bigger network

33
Results with E
34
More Results with E
35
Context

• Give it information about what type of reaction
  we are activating narrowing its field
• Biosynthesis
• Degradation
• Signal Transduction
• Or a mix of any of the three
• This information was conveyed in a three column
  vector with each row having 1 for being active or
  1 for inactive following the order above

36
Results with C
37
Results with C
38
Hints

• We tell it the time scale of each reaction from
  known data
• This allows units to be applied to the time scale
• This makes the information more specific

39
Results with hints
40
More Results with hints
41
Error comparison for IEC
42
Our Better Program

• A polymorphic NN program with a dynamic and
  static version
• Uses multiple and different types of networks to
  capitalize on strengths of each network
• Also uses Context, Inheritance, and Encapsulation

43
Polymorphism

• FF
• Signal transduction
• Reaction object, coenzyme
• Reaction object, enzyme
• Elman
• Translation/transcription
• Hopfield
• Context

44
Better Program, initial/final
45
Better Program, with time
46
More result Comparison
47
Error Comparison
To calculate error we took the average error of
the metabolite over time
48
Our Best Program

• Uses all of the features
• PIECH
• This network produced the highest accuracy while
  conveying the most in depth information
• The accuracy was /- 0.02 for final concentrations

49
Best Program
50
More Best Program Results
51
How does bacteria respond to Carbon and Nitrogen!
52
Programs Used

• Matlab
• WinScamp

53
Bio-informatics Toolbox

• Cluster proteins and metabolites while
  identifying chemical properties such as molecular
  weight
• Time scale

54
Protein Plot(Trp)
55
Real data

• We obtained our real data for testing and
  verification of concentrations by using a
  metabolic simulation program called WinScamp
• We obtained other real data from various papers
• nBolouri, Hamid and Eric Davidson.
  Transcriptional regulatory cascades in
  development Initial rates, not steady state,
  determine network kinetics
• nKremling, A, Katja Bettenbrock, Sophia Fisher,
  Martin Ginkel, Thomas Sauter, and Ernst Dieter
  Gilles. Towards whole cell in silico models
  for cellular systems model set-up and model
  validation. nPapin, J.A., Stelling, J., Price,
  N.D., Klamt, S., Schuster, S., and Palsson, B.Ø.,
  "Comparison of Network-Based Pathway Analysis
  Methods

56
Sample WinScamp output
57
Typical values for minimum reaction energy and
enzyme efficiency(KV)

• Glycolysis
• V1, K9
• Tyrosine/Tryptophan
• V2, K2
• Heme production and Nitrogen signal transduction
• V2, K4

58
Changing Parameters

• When I change the V and K values in WinScamp, the
  results change
• By changing the glycolysis V to 2 and K to 8 we
  ran a test to see how well our program could
  predict them

59
Results
60
It Been an Interesting Trip!
61
Possible Future REU research

• Explore new ways to make program adapt to
• Temperature(ambient and otherwise)
• Pressure(External/Internal)
• Amount of Light at Various wavelengths

62
Future Research

• Discovery of new metabolic pathways
• Take more than one organism and compare their
  pathways to see similarities and
  differences(genetic tree)
• Create metabolic Tool box for Matlab
• Current software doesnt handle all processes
• Exploit Bacteria to modify engineering material
  e.g. plastics
• Using Bacteria to make new more flexible
  materials

63
References

• Neural Network Papers
•  
•  
• nGuez, Allon. On the Stability, Storage
  Capacity, and Design of Nonlinear
• Continuous Neural Networks. IEEE Transactions on
  System, Man, and Cybernetics 18.1 (1988) 80-86.
•  
• nHopfield, J. Neurons with graded response have
  collective computational
• properties.Proceedings of the National Academy
  of Sciences of the United States of America. 81
  (1984) 3088-3092.
•  
• nMazzara, Maria and Matthew Sadiku. Computing
  with Neural Networks. IEEE Potentials. 12.3
  (1993) 14-16.
•  
• nMoore, Kevin Artificial Neural Networks. IEEE
  Potentials 11.1 (1992) 23-28.

64
References

• Metabolic Network Papers
•  
• nBolouri, Hamid and Eric Davidson.
  Transcriptional regulatory cascades in
  development Initial rates, not steady state,
  determine network kinetics Proceedings of the
  National Academy of Sciences of the United States
  of America. 100 (2003) 9373.
•  
• nKremling,A.,Jahreis,K., Lengeler, J.W., Giles,
  E.D. (2000). The Organization of Metabolic
  Reaction Networks A Signal-Oriented Approach to
  Cellular Models. Metabolic Engineering. 2.
  190-200.
•  
• nKremling, A, Katja Bettenbrock, Sophia Fisher,
  Martin Ginkel, Thomas Sauter, and Ernst Dieter
  Gilles. Towards whole cell in silico models
  for cellular systems model set-up and model
  validation. Max-Planck-Institute. 21 July 2004.
•  
• nPapin, J.A., Stelling, J., Price, N.D., Klamt,
  S., Schuster, S., and Palsson, B.Ø., "Comparison
  of Network-Based Pathway Analysis Methods,
  Trends in Biotechnology, in press (2004).
•  
• nSchilling, Christophe H., Letscher, David,
  Pallsso, Bernhard. (2000). Theory for the
  Systemic Definition of Metabolic Pathways and
  their use in Interpreting Metabolic Function from
  a Pathway-Oriented Perspective. Journal of
  Theoretical Biology. 203. 229-248.

65
References

• Tools
•  
• nKarp, Peter D, Julio Collado-Vides, John
  Ingraham, Ian Paulsen, Milton
• Saier. Ecocyc. SRI International, Marine
  Biological Laboratory, DoubleTwist Inc., The
  Institute for Genomic Research, University of
  California at San Diego. 21 July 2004.
• n The MathWorks, Inc. Matlab Neural Network and
  Bioinformatics Toolboxes 22
• July 2004.
• nSuaro, Herbert. WinSCAMP Metabolic Simulation
  and Analysis. 22 July 2004.

66
Acknowledgments

• Dr. Zohdy
• Dr. Mili, Dr. Sethi, Dr. Elhajj, Dr. Li
• Anne (for being so nice to us when ordering
  things)
• Nick and Tera (our favorite grad students with
  Dr. Zohdy)
• Dr. Herbert Sauro of WinScamp
• And all the REU students for going out to lunch
  with us on
• Fridays/Thursdays

67
Any Questions???