Encoding Sensory Inputs in Numeric Binary Form

1
Encoding Sensory Inputs in Numeric / Binary Form

• CS/PY 231 Lab Presentation 7
• March 4, 2005
• Mount Union College

2
The Problem

• So far, we have seen networks that can learn to
  produce numeric output patterns in response to a
  sequence of numeric inputs
• What do organisms do?
• produce thoughts and behaviors in response to
  sensory inputs
• We need a way to represent real-world data in
  numeric form

3
Example Sequence of Letters

• What if we wanted to train a neural network to
  respond to a letter of the alphabet with the
  letter that follows the input letter?
• e.g., C gt D, U gt V
• Simple task, yet our TLearn networks need to have
  numeric data to process
• What prior art can we use?

4
Computer Hardware example

• Computer Memory is a collection of a large number
  of electronic circuits
• Each circuit has two states on or off
• number each state 1 or 0
• A single circuit can represent a 2-valued datum
• yes/no, true/false
• what to do for data items that need more than 2
  possible values?

5
Binary s ? Group of Circuits

• Consider two circuits as a single group
• 00 01 10 11
• 2 choices for first bit x 2 choices for second
  bit 4 possible values
• yes, no, maybe, huh?
• still not enough for letters of the alphabet,
  digits, etc.

6
Binary s ? Group of Circuits

• Consider three circuits as a single group
• 000 001 010 011
• 100 101 110 111
• Twice as many possible patterns as with 2 bits
• 2 x 4 8 23
• 4 bits 24 16 patterns 5 bits 25 32
  patterns
• n bits 2n patterns

7
Example from Lab 6

• We trained a network with 3 inputs and 2 outputs
• Example from Lab 6
• Answer output is the number of zeros in the
  input pattern, expressed as a binary number!
• But how could you know that? We also need to
  decode numeric output into a form that humans can
  use.

8
Question for Understanding

• How many binary input and output signals would be
  needed to train a network to discriminate thusly
  given an American League baseball team, ranking
  the Indians as great, the Yankees, White Sox, Red
  Sox and Orioles as lousy, and all others as
  mediocre?
• 14 teams, 3 possible rankings

9
Where does encoding/decoding happen?

• In our computational model, sense organs convert
  analog data into digital signals that can be
  operated on by a neural network
• To model this, we have two options
• Build hardware components that perform this
  conversion for us o (
• Convert data into digital form, then present this
  to the network o)

10
Where does encoding/decoding happen?

• Output of our model is a bunch of numeric
  signals.
• We need to convert this into understandable
  information
• Two approaches
• Build hardware to do the conversion (robot
  lab!!!)
• Convert data ourselves (either by writing a
  program, or doing it manually)

11
Example Diagnosing Illness

• Input parameters
• temperature numeric value
• pulse numeric value
• palpating abdomen hard, squishy, normal
• headache? yes or no
• how many fingers? too few, too many, right
  number
• Output diagnoses
• ebola, migrane, hypochondria, normal health

12
Example Diagnosing Illness

• Input parameters
• numeric value
• numeric value
• 1 hard, -1 squishy, 0 normal
• 1 headache, 0 no headache
• -1 too few fingers, 1 too many, 0 right
  number
• Output diagnoses
• some numeric code for each disease

13
Central Problem in Compuational Neural Networks

• Without a correct and proper encoding scheme, the
  network wont produce meaningful answers
• A large, important part of your term project will
  be selecting a data representation for the
  problem you choose

14
Encoding Sensory Inputs in Numeric / Binary Form

• CS/PY 231 Lab Presentation 7
• March 4, 2005
• Mount Union College