A Brief Introduction to Artificial Neural Networks and Educational Assessment

1
A Brief Introduction to Artificial Neural
Networks and Educational Assessment

• Robert J. Mislevy
• University of Maryland
• January 30, 2006
• Source on neurons and vision
• Anderson, J.R. (2000). Cognitive Psychology and
  its implications (fifth edition). New York
  Worth.

2
Why are we talking about neural networks? (1)

• Levels of analysis in psychology
• The social or interpersonal level refers to the
  social interactions that define and give meaning
  to information.
• The computational level refers to the nature and
  content of information that is involved, and ways
  it is used.
• The algorithmic level refers to procedures that
  people carry out, working with information
  described at the computational level.
• The implementation level refers to the physical
  structures and mechanisms that embody cognitive
  processes.

3
Why are we talking about neural networks? (2)

• In educational assessment we will usually work at
  the computational and social levels. Neural nets
  are algorithmic level.
• But we gain metaphors insights into learning,
  thus into assessment.
• Neural networks connect the social and
  computational levels with the implementation
  level. It is an active frontier in cognitive
  science today.
• Clarks (1997) Being there
• Hawkins Blakeslees (2004) On intelligence.
• There are practically important uses of neural
  networks in assessment, in evidence
  identification (i.e., task-level scoring).

4
A Riddle

• Q What do you get when you cross a cats eye
  with logistic regression?
• A An artificial neural network.

5
Real Neurons
Anderson, J.R. (2000).

• Electrical signals from environment or other
  neurons (at dendrites)
• Some increase electrical potential in the cell
  (excitatory)
• Some decrease electrical potential in the cell
  (inhibitory)
• When the cells threshold is reached, it sends a
  signal (through axon) to other neurons or organs
  (e.g., muscle cells)

6
Neurons that detect light spots and dark spots
Anderson, J.R. (2000).
Anderson, J.R. (2000).
7
Feature detectors
Anderson, J.R. (2000).
8
Generalizations of feature detectors

• Intensity
• Movement
• Bug detectors
• Lettvin, J.Y., Maturana, H.R., McCulloch, W.S.,
  Pitts, W.H. (1959). What the frog's eye tells
  the frog's brain. Proceedings of the IRE
  (Institute of Radio Engineers), 47, 1940-1951.
• Some higher-level activity same from actual
  stimulus and recall of stimulus
• Top-down as well as bottom-up processing
• Integration of fragments into perceived whole (V5
  cortical area)
• Saccades eye-tracking tracenext slide

9

• Schwank, I. (2001). Analysis of eye-movements
  during functional versus predicative problem
  solving. Paper presented at the 2nd Conference
  of the European Society for Research in
  Mathematics Education, February 24-27, 2001,
  Mariánské Lázn, Czech Republic.

10
An artificial neuron k (a neurode)
x1
a1
Sajkxj
Yk
a2
x2
. . .
xJ
an

• 0/1 signals in xjs. 0/1 output Yk
• Weights ajks (real). Threshold tk (real).
  
• Probability that Yk1, given inputs xj
• e.g., logit P(Yk1a) N(Sajkxj - tk , 1)

11
An artificial neural network (1)A feed-forward
net, re perception
x1
z1
w1
x2
y1
z2
w2
x3
z3
w3
x4
y2
z4
w4
x5
Output
Hidden layers
Input

• Lowest layer Feature detectors. Highest
  layer Criteria.
• Hidden layers allow nonlinear nonmonotonic
  combinations of information.
• Weights hidden nodes sometimes interpretable,
  sometimes not. E.g., in text-to-speech, cononant
  blends.

12
Learning in an ANN

• Supervised learning (vs. unsupervised)
• Many cases with known inputs and outputs
• Backpropagation
• Repeated cycles over training set,
• Each time increase by an amount the weights for
  inputs that were right and decrease those that
  were wrong, based on partial derivatives.
• Propagate back down through the layers.
• Do until no systematic change in weights, or
  outputs for a criterion set of inputs.
• See a tutorial on machine learning from Leeds
  University http//cbl.leeds.ac.uk/nikos/pail/intm
  l/subsection3.11.4.html

13
An artificial neural network (2)A Network of
Associations

• McClelland, J. L. (1981). Retrieving general and
  specific information from stored knowledge of
  specifics. Proceedings of the Third Annual
  Meeting of the Cognitive Science Society,
  170-172.
• Each of five persons has an age, occupation,
  education, marital status, and gang membership.
• This graphic by Bill Wilson http//www.cse.unsw.ed
  u.au/billw/cs9414/notes/ml/pdp/iac-2005.html

14
An artificial neural network (3)A Network of
Associations

• McClelland, J. L. (1981). Retrieving general and
  specific information from stored knowledge of
  specifics. Proceedings of the Third Annual
  Meeting of the Cognitive Science Society,
  170-172.
• Each of five persons has an age, occupation,
  education, marital status, and gang membership.
• This graphic by Bill Wilson http//www.cse.unsw.ed
  u.au/billw/cs9414/notes/ml/pdp/iac-2005.html
• Interactive example at http//srsc.ulb.ac.be/pdp/I
  AC/IAC.html

15
Characteristics of Knowledge in Neural Nets

• Knowledge in a network at a given point in time
  depends on both the initial structure of the
  network and the connections tuned by the
  particular examples that have been experienced.
• Capacity for generalization
• Knowledge is not a set of discrete propositions
  or localized relationships, but a distributed
  pattern of activation across many nodes.
• Meaning is situated i.e., conditional on all the
  other facets of a situation.
• At higher levels, patterns of patterns
• Key to language? 6 cortical layers in humans.

16
Insights into human learning (1)

• Similarities to human learning
• the power law of increasing accuracy
• the stage when children over-generalizing regular
  tense forms to irregular verbs (I goed to the
  store)
• Modular models (like local areas of the brain)
• Phased training sets
• Simpler cases first, increasing occurence of
  complex ones, e.g., verbs. Otherwise, local
  optimanot principled basis
• Analogy to instruction design, e.g., sequence of
  models

17
Insights into human learning (2)

• Short-term memory -- Increase short-term memory
  with same training set

x1
z1
w1
x2
y1
z2
w2
x3
z3
w3
x4
y2
z4
w4
x5
z1
Output
Input
z2
z3
z4
18
Patterns in human learning

• Patterns of many kinds
• Some perceptual, reflective
• Some conscious, structured, as in school learning
• Some are cultural models (Strauss Quinn)
• Patterns at many levels
• Jointly activated, as in language use
• In learning, knowledge is constructed from
  similarities across experiences
• In perception / understanding, meanings are
  constructed in every situation, and update
  knowledge every time, just as in ANNs
• Patterns are not necessary coherent or integrated
• Misconception research
• How to teach to promote transfer

19
Examples of ANN use in assessment for task scoring

• Automated essay rating
• ETSs e-rater site http//www.ets.org/research/er
  ater.html
• Ron Stevens IMMEX problem-solving example
• Exploring the Dynamics of Complex Problem-Solving
  With Artificial Neural Network-Based Assessment
  Systems
• Hurst, Casillas, Stevens (1997) CSE Technical
  Report 444 http//www.cse.ucla.edu/CRESST/Reports
  /TECH444.pdf
• Jared Bernsteins PhonePass (Now Ordinate)
• Test of spoken English
• Features are acoustic
• http//www.ordinate.com/
• Note the free demo you can try yourself.