Overview

1
Overview

• PSYC 575
• Computational Modeling

2
Goals

• Introduce enough concepts so that you will
  recognize these building blocks in articles that
  you might read that use modeling
• Expose you to a variety of modeling approaches so
  that you know what tools are available
• The goal is not to teach you how to use all of
  these tools or the entire palette of possible
  tools
• There are simply too many
• The goal is also not to teach you how to program
• You will learn how the models behave and why
• Programmers will learn enough in the course to
  program models themselves

3
General approach in class

• Start with a historical overview
• Discuss the linear statistical approaches as
  baseline of comparison
• Then progress through families of
  architectures/algorithms
• Move from simplest to most complex
• We will often look at a specific application of
  the technique.

4
Modeling is all about the formal specification of
relationships

• These relationships are either
• Observed - your model summarizes historical
  relationships,
• Inferred - your model goes beyond historical
  relationships to posit unobserved ones.
• Model used to generate predictions.

5
Approaches discussed in class

• Modeling linear relationships to predict y
• Regression (GLM)
• Modeling nonlinear relationships to predict y
• Pattern association (inferring one set of
  variables from another)
• Linear, nonlinear, backprop, nearest neighbor,
  RBF.
• Modeling time series (using nn approaches)
• Finding patterns - Unsupervised learning.

6
Connectionist models vs. statistical techniques

• In neural network terminology, statistical
  inference is learning to generalize from noisy
  data.
• Many neural networks that can learn to generalize
  effectively from noisy data are similar or
  identical to statistical methods.

7
Finding a solution

• Connectionist models provide an iterative method
  to converge on a solution data is given a bit
  at a time and solutions are approached, not
  solved for.
• Take a guess at right answer and then adjust ad
  infinitum
• Statistical methods are sometimes iterative, but
  often can find a solution (e.g., LSM) in a single
  step given all of the data

8
Classes of relationships to be learned - I

• Predicting continuous variables from others
• Regression
• Associative learning
• Problem most people know how to do simple and
  multiple regression, but few have the tools and
  knowledge to do nonlinear regression or
  multivariate regression.

9
Classes of relationships to be learned - II

• Predicting continuous variables from categorical
  ones
• ANOVA, MANOVA
• Associative learning
• Issue how many are comfortable with MANOVA?

10
Classes of relationships to be learned - III

• Predicting categorical from categorical
• Chi-squared, loglinear analysis
• Associative learning w/response mapping
• Issue do you even know what loglinear analysis
  is?

11
Classes of relationships to be learned - IV

• Cluster analysis
• Various cluster analysis techniques
• Competitive learning
• Dimensionality reduction
• Factor analysis principal component analysis
• Unsupervised Hebbian learning, autoassociative
  models

12
Specific mappings

• Feedforward nets with no hidden layer are
  basically generalized linear models.
• Feedforward nets with one hidden layer are
  closely related to projection pursuit regression.
  
• Kohonen nets for adaptive vector quantization are
  very similar to k-means cluster analysis.
• Hebbian learning is closely related to principal
  component analysis.
• Some neural network areas appear to have no close
  relatives in the existing statistical literature
• Reinforcement learning, backpropagation

13
Bottom line

• Most statistical techniques assume a particular
  functional relationship between the independent
  and dependent variables (but, spline fits and
  co.).
• Backprop is a universal function approximator
• If your problem conforms to a standard
  statistical technique that you are familiar with
  and it can answer your questions, use it!
• Dont just use connectionist or nonlinear models
  just because you can.
• Statistical models can tell us a lot.
• e.g., what is the relative weight of various
  factors
• e.g., does the variability across many behaviors
  boil down to a handful of underlying factors.

14
Modeling issues

• Many of the issues you face with statistical
  analyses also arise in modeling.
• Examples
• Too little data on which to base model.
• Too many predictors can create a model with too
  many dfs. Overfitting.
• Use of an intercept use of bias nodes.
• Multicollinearity in predictors produces unstable
  estimates of the relative importance of each
  predictor

15
Different modeling philosophies

• Models for theory testing
• Strong inference method comparing two
  models/theories quantitatively.
• Example Lots Young Wasserman, 2001
• Models of the brain
• Neurophysiology drives the architecture and
  computational methods used.
• Simplification is inevitable.
• Examples (next)

16
Atallah et al. (2004)
Levine
17
Modeling philosophies, cont

• Models as evidence proofs
• Common method of weak inference?
• Example Look! A simple associative system can
  do it without positing complex symbolic
  structures
• Rationale based on law of parsimony but doesnt
  disprove a more complex theory.
• Models as thought experiments
• Provides a different way of thinking about
  psychological processes and representation.
• Common in philosophy (Dennett, Churchlands,
  Clark).
• Models for data analysis
• Uses it like a statistical tool.
• Often to solve applied problems (e.g., diagnosis,
  prediction).

18
Your philosophy will drive the input and output
representations

• What are your goals?
• e.g., Veridical input vs. preprocessed input
• An example, text comprehension
• Must you solve the problem of converting bitmaps
  to letters and then to words before you can start
  worrying about the extraction of meaning?
• Evidence proof vs. theory testing
• Evidence proof sufficient to show that one type
  of network with one set of parameters will solve
  the task.
• Theory testing need to compare the best fitting
  parameters for two models that are being
  compared.

19
Next class

• An informal introduction to R
• Discussion of Platt article on strong inference
• The history of connectionist/neural network
  modeling