Neural Networks in Computer Science

1
Neural Networks in Computer Science

• CS/PY 231 Lab Presentation 1
• January 14, 2005
• Mount Union College

2
Some Computing History

• First all-electronic computing device ENIAC
  (1946) Ekert Mauchly (Penn)
• application computing ballistic firing tables
  for US Navy
• special-purpose device hard-wired, like the
  computer in your cars engine
• to change the problem, device must be rewired
• all memory dedicated to data storage

3
Innovation Stored Program Concept

• First stored program computer EDSAC (1952) von
  Neumann (Princeton)
• reprogramming became a logical (not physical)
  task
• memory stores both software and data
• much easier to program therefore, use of
  computers explodes commercially

4
von Neumann Architecture

• one powerful, fast processor
• device capable of performing calculations and
  comparisons
• large memory that the processor may access
• serial processing model processor performs one
  operation at a time, in series

5
von Neumann Architecture
Memory
Processor
Program Instructions
Data
6
Traits of von Neumann Machines

• much better than humans at performing
  calculations and logical step-by-step procedures
• since computers were so much better at these
  kinds of tasks, it was assumed that they would
  outperform humans in all mental tasks
• as soon as we could determine how to program the
  machines for those tasks

7
Not So Fast, My Friend

• von Neumann architecture turns out to have
  serious limitations when confronted with certain
  problems
• von Neumann bottleneck even with a fast
  processor, doing one thing at a time is too slow
  when facing a huge task
• idea have several processors working on the
  same problem

8
Example Sorting a list of values

• data 45, 67, 12, 87, 19, 55, 32, 29
• method bubble sort
• single processor 27 comparisons, 16 switches
• two processors 12 comparisons, 5 switches, 8
  merge steps
• complications coordination of processors,
  communication, limited of processors, etc.

9
A Paradox

• Researchers discovered that machines based on the
  von Neumann architecture are
• good at problems that are difficult for humans
  (e.g., calculations), and
• bad at problems that humans find to be easy
  (e.g., pattern recognition)
• This seems to be a limitation of the machines
  architecture

10
A Novel Idea

• Why not build computers that are structured as
  brains are?
• Problem brain structure is fabulously complex
  and consists of massive numbers of processors
  (neurons)
• We can start with small artificial brains and add
  complexity as we progress

11
Processing Speeds

• Modern processors are almost a billion times
  faster than real neurons
• However, the brain contains hundreds of billions
  of neurons, with trillions of connections, each
  of which can process and store signals
• The brain ends up having much more raw processing
  power than a single processor

12
Essential for Pattern Recognition

• Massive Parallelism
• many tasks are performed by the brain at the same
  instant of time (breathing control, digestion,
  visual processing, thinking, etc.)
• problem how to program such a complex
  combination of processing devices?
• solution look to nature!

13
Neural Networks

• term for a computing device based on brain
  structure
• idea connecting a huge number of simple
  processors into a large interconnected network
• organized in layers input, middle (hidden),
  output
• How does this network know what answers to
  produce for some input?

14
Traditional Computer Programming

• we must develop a step-by-step sequence of
  operations that when carried out will produce the
  answers to the problem being considered
• someone must write such a program -- this is a
  difficult task
• writing software for parallel architectures is
  MUCH harder than this!!

15
Training Neural Networks

• A neural network may be trained to produce a
  desired output when exposed to a given input
  pattern
• it turns out that networks are GREAT at solving
  pattern recognition tasks
• they may be trained to recognize any patterns
  (visual, audio, representational)
• as long as the input is encoded in a form that
  the network can receive)

16
Modeling Nature

• much of the complexity of a brain cell is present
  to support the living operation of the cell
• we can devise a simplified model of the neuron
  that focuses on the processing function of the
  cell
• we will then use these units to construct
  artificial neural networks (brains)

17
The Perceptron

• a simple model of a real neuron
• contains essential features
• a number of input connections (values received
  from other neurons)
• each input connection has an associated weight
• the perceptron has a threshold value, and
  produces
• an output value (the answer)

18
The Perceptron
I1
w1
I2
w2
Output
w3
I3
? (threshold)


wn
In
19
The Perceptrons Calculation

• at any instant of time, the output generated by
  the perceptron is calculated as follows
• first, each input signal is multiplied by the
  weight associated with that connection
• all of these products are added together
• if the sum equals or exceeds the threshold, the
  perceptron fires

20
The Perceptrons Calculation

• if the sum is less than the threshold value, the
  perceptron does not produce a signal
• we use output values of 1 for firing and 0 for
  not firing
• simplification of what actual brain cells do
• but this simple concept is the basis for neural
  network computing!

21
Simplest Perceptron

• limited to two input connections
• can be used to classify input patterns into
  classes (accepted or rejected)
• Perceptrons can be used to solve a wide variety
  of decision problems
• but are too simple for many other problems
• for those, well have to combine perceptrons in a
  hierarchy or network

22
Neural Networks in Computer Science

• CS/PY 231 Lab Presentation 1
• January 14, 2005
• Mount Union College