Neural Networks in Computer Science

1
Neural Networks in Computer Science

• CS/PY 231 Lab Presentation 1
• January 14, 2005
• Mount Union College

2
von Neumann Architecture (typical desktop
computer)

• one powerful, fast processor
• device capable of performing calculations and
  comparisons
• large memory that the processor may access
• serial processing model processor performs one
  operation at a time, in series

3
Traits of von Neumann Machines

• much better than humans at performing
  calculations and logical step-by-step procedures
• since computers were so much better at these
  kinds of tasks, it was assumed that they would
  outperform humans in all mental tasks
• as soon as we could determine how to program the
  machines for those tasks

4
Not So Fast, My Friend

• von Neumann architecture turns out to have
  serious limitations when confronted with certain
  problem types
• von Neumann bottleneck even with a fast
  processor, doing one thing at a time is too slow
  when facing a huge task
• idea have several processors working on the
  same problem

5
A Paradox

• Researchers have discovered that machines based
  on the von Neumann architecture are
• good at problems that are difficult for humans
  (e.g., calculations), and
• bad at problems that humans find to be easy
  (e.g., pattern recognition)
• This seems to be a limitation arising from the
  machines architecture

6
A Novel Idea

• Why not build computers that are structured in
  the way brains are?
• Problem brain structure is fabulously complex
  and consists of massive numbers of processors
  (neurons)
• We can start with small artificial brains and add
  complexity as we progress

7
Pattern Recognition Requires

• Massive Parallelism!
• many tasks are performed by the brain at the same
  instant of time (breathing control, digestion,
  visual processing, thinking, etc.)
• problem how to program such a complex
  combination of processing devices?
• solution look to nature!

8
Neural Networks

• term for computing devices based on brain
  structure
• idea connecting a huge number of simple (slow,
  inexpensive) processors into a large
  interconnected network
• typically organized in layers input, middle
  (hidden), output
• How does this network know what answers to
  produce for some input?

9
Training Neural Networks

• A neural network may be trained to produce a
  desired output when exposed to a given input
  pattern (this involves modifying connection
  weights)
• it turns out that networks are GREAT at solving
  pattern recognition tasks
• they may be trained to recognize any patterns
  (visual, audio, representational)
• as long as the input is encoded in a form that
  the network can receive)

10
Modeling Nature

• much of the complexity of a brain cell is present
  to support the living operation of the cell
• we can devise a simplified model of the neuron
  that focuses on the processing function of the
  cell
• we will then use these units to construct
  artificial neural networks (brains)

11
The Perceptron

• a simple model of a real neuron
• contains essential features
• a number of input connections (values received
  from other neurons)
• each input connection has an associated weight (
  the effect of the input signal)
• the perceptron has a threshold value, and
  produces
• an output value (the answer)

12
The Perceptron
I1
w1
I2
w2
Output
w3
I3
? (threshold)


wn
In
13
The Perceptrons Computation

• at any instant of time, the output generated by
  the perceptron is calculated as follows
• first, each input signal is multiplied by the
  weight associated with that connection
• all of these products are added together
• if the sum equals or exceeds the threshold, the
  perceptron fires

14
The Perceptrons Calculation

• if the sum is less than the threshold value, the
  perceptron does not produce a signal
• we use output values of 1 for firing and 0 for
  not firing
• simplification of what actual brain cells do
• but this simple concept is the basis for neural
  network computing!

15
Simplest Perceptron

• limited to two input connections
• can be used to classify input patterns into
  classes (accepted or rejected)
• Perceptrons can be used to solve a wide variety
  of decision problems
• but are too simple for many other problems
• for those, well have to combine perceptrons in a
  hierarchy or network

16
Neural Networks in Computer Science

• CS/PY 231 Lab Presentation 1
• January 14, 2005
• Mount Union College