Protein Fold Recognition with Relevance Vector Machines

1
Protein Fold Recognition with Relevance Vector
Machines

• Patrick Fernie
• COMS 6772
• Advanced Machine Learning
• 12/05/2005

2
Relevance Vector Machine

• A Bayesian treatment of a generalized linear
  model
• Yields a formulation similar to that of a Support
  Vector Machine
• Hyperparameters Instead of Margin/Costs

3
Relevance Vector Machine
SVM RVM
Hard Binary Outputs or Point Estimates Probabilistic Outputs
Requires a Mercer Kernel Can Use Arbitrary Kernel
Must Determine Suitable Cost and Insensitivity Values Nuisance Values Automatically Determined
Sparse (USPS 2500) Sparser USPS (316!)
4
Relevance Vector Machine

• Cant Use qp()
• Must solve iteratively (Sequential Minimization
  Optimization)
• As we iterate, many hyperparameters (ai) values
  become arbitrarily large allows pruning.

5
Relevance Vector Machine

• Faster Algorithm (Still not SVM fast)
• Minimizes Number of Active Kernel Functions to
  Reduce Computation Time
• Analytic Approach to Pruning/Adding Basis
  Functions

6
Protein Fold Recognition

• Protein Structure Families
• Many Fold Families
• Not Necessarily Directly Related by Protein
  Sequence

7
Protein Fold Recognition

• Prime Situation for Machine Learning Techniques!
• NN, SVM, etc.
• Large Number of Classes

8
Protein Fold Recognition

• 27 Fold Families
• Train Many 2-Class Classifiers
• One vs. Others False Positives
• Unique One vs. Others Like One vs. Others, with
  Another Round of Training
• All vs. All Requires a Lot of Classifiers!

9
RVMs Protein Folds

• Why RVMs?
• Probabilistic Outputs
• Sparsity (useful only in assessment)
• True Multiclass Prediction
• No Need to Find Nuisance Parameters

10
Issues/Future Work

• Optimize RVM Classification
• Implement True Multiclass
• Reduced Greediness and Sequential Convergence
  Optimization
• Novel Kernels?

11
References

• M. Tipping, The Relevance Vector Machine,
  http//www.relevancevector.com
• M. Tipping, Sparse Bayesian Learning and the
  Relevance Vector Machine, JMLR, 2001 1211-244.
• M. Tipping and A. Faul, Fast Marginal Likelihood
  Maximisation for Sparse Bayesian Models,
  http//www.relevancevector.com
• C. Ding and I. Dubchak, Multi-class Protein Fold
  Recognition Using Support Vector Machines,
  http//www.kernel-machines.org