Neural Nets

1
Neural Nets

• Neural network or connectionist network
• a set of connected cells or computational units,
  and one-way connections between them
• input cells input representations of behaviour
  examples
• output cells their outward arcs represent an
  activation number
• arcs within network have weights between -1
  (neg.correlation) and 1 (pos. correlation)
  (therefore 0 no correlation)
• 1.1 Gallant

2
Neural nets

• a network topology is first decided upon
• different network topologies have different
  reactive behaviours towards input, as well as
  differing efficiency
• 1.4, 1.5 Gallant

3
Neural nets

• Then the network is trained with respect to a
  set of training examples.
• Main learning strategies
• supervised a human teacher reinforces correct
  performance
• unsupervised patterns are discovered
  (clustering)
• real-time learning learning occurs during use
• 1.2 Gallant

4
Neural nets

• so long as the input examples can be labelled or
  numbered wrt input cells, and activation number
  can be mapped to discernable behaviours of those
  examples, the network can be used to learn
• ie. discover patterns of examples that result in
  desired output
• neural networks can have noise filtering rules
  (unlike rule-based approaches)
• formula are used to map connection weights into
  cells, to output activation weight of the cell
• 1.6

5
Neural nets Expert Systems

• neural nets have been increasingly successful in
  developing expert systems
• a 1991 conference vision system to sort apples
  medicine handwriting recognition commodity
  training ...
• mid 80s example MACIE - MAtric Controlled
  Inference Engine
• input( for a small subset of system)
• (i) symptoms (6 total) info about whether
  present, absent, or unknown (1, -1, 0)
• (ii) diseases (2 total) present, absent, unknown
• (iii) treatments (3 given)
• 14.5 Gallant

6
MACIE

• 14.4 Gallant

7
MACIE

• Training input file
• 14.6, 14.9, 14.10 Gallant

8
Neural nets comments

• techniques exist for extracing IF-THEN rules from
  network
• required for explanation
• many possible rules exist in net the ones of
  interest are
• (i) those that are valid for all values the other
  variables take
• IF u3 is true
• u5 is false
• THEN conclude u8 is true
• (ii) maximally general
• IF u3 is true
• u5 is false
• u7 is false
• THEN conclude u8 is true
• less general than rule (i), therefore not used

9
Neural nets comments

• neural nets can be thought of as automatons that
  automatically set certainty values
• differ from rule-based approaches knowledge is
  not structured
• however, it can be argued that experts do not
  necessarily think in a structured fashion
• also, noise can be absorbed
• whereas rule-based systems will crash given a
  noisy (bad) rule
• neural nets very good for expertise that uses
  sophisticated pattern-matching,
• eg. image analysis, identification, ...

10
Comparing NN with Rule-based KBS

• History
• NN and rule-based systems developed at the same
  time in 50s-60s
• NN fell out of favour in late 60s due to Marvin
  Minskys examples of weaknesses with NNs
  (inability of Perceptron to recognize XOR)
• rule-based systems were focus of AI for next
  decade, until NN developed interest in late 70s
  RBS systems didnt deliver high promises
• similarities
• can encode high-level knowledge
• can generalize, learn
• both are Turing powerful can simulate each other
  abstractly
• differences
• NN are low-level, bottom-up, data-driven systems
  RBS are top-down, rule-driven systems
• NN are highly parallel RBS are sequential
  (inference is seql)
• NN automatically account for uncertainty, noise
  more difficult with RBS
• NN knowledge takes form of weights in network
  RBS - knowledge is explicit
• NN training is faster than RBS
• NN behaviours are not well understood RBS is
  better understood

11

• NN better for
• pattern recognition visual, audio
• when fast training required
• when very noisy data set
• Rule-based systems better for
• systems whose conceptual organization is
  structured
• Which is better?
• ill-phrased question, as both are suitable to
  different problems
• many are working on incorporating them together
  into single systems