Kevin H. Knuth, Ph.D.

1
The Age of Robotics

• Kevin H. Knuth, Ph.D.
• Departments of Physics and Informatics
• University at Albany

2
If every tool, when ordered, or even of its own
accord, could do the work that befits it... If
the weavers' shuttles were to weave of
themselves, then there would be no need either of
apprentices for the master workers or of slaves
for the lords. Aristotle, De Republica
Atheniensium, 322 BC
3
Leonardo da VinciThe Father of Robotics
Leonardo da Vinci1452-1519
4
Leonardos Robot Based on da Vincis sketches
from 1495The robot is a knight in
German-Italian Medieval Armor
5
Leonardos AutomataA small wooden car, powered
by springscould be programmed by inserting
various cogs and gears.
(Image Institute and Museum of the History of
Science, Florence)
6
The Dream of Robotics has long been with us
Maria from Fritz Langes Metropolis (1927)
7
The Dream of Robotics has long been with us
The Tin ManFrank L. BaumsThe Wizard of
Oz (1900 Book, 1939 Film)
Robby the RobotForbidden Planet 1956
8
And has changed with us
BenderFuturama (1999-Present)
R2D2 and C3POStar Wars (1977)
9
Reality is always Stranger than Fiction
Opportunity currently on Mars
Cassini in orbit around Saturn
10
And creeps up on us Unsuspectingly
11
And creeps up on us Unsuspectingly
12
How do we make them INTELLIGENT?
13

• How Do We Function?

14
The Brain The Living State of Matter

• 1011-1012 Neurons
• 104-105 Connections per Neuron
• Maximum Firing Rate 1 ms
• 1kHz massively parallel computer
• Information Processed on order of 100s ms
• MUST use Prior Information

The Virtual Hospital, Ch 5, Williams,
Gluhbegovic, and Jew
15
A Powerful Computer

• Hree is an ecxlelnet eaxmlpe of how yuor
  wnodreful mnid can raed tihs txet eevn touhgh its
  all jmbuled.

16
Sounds and Prior Information

• Listen to these sounds

Sounds from Haskins Laboratories, Rubin, Remez,
Pardo
17
Sounds and Prior Information

• Now listen to this one

Sounds from Haskins Laboratories, Rubin, Remez,
Pardo
18
Sounds and Prior Information

• And now go back to this one

Sounds from Haskins Laboratories, Rubin, Remez,
Pardo
19
Sounds and Prior Information

• What about the rest?

Sounds from Haskins Laboratories, Rubin, Remez,
Pardo
20
Prior Information is Key

• Only 10 of the inputs into primary visual cortex
  come from the retina via the lateral geniculate
  nucleus. The rest come from higher visual and
  frontal areas.
• Perception can also be modified by attention.
• Thus the brain can actively focus on relevant
  information.

Human Brain basal view (front at top)The
Virtual Hospital, Ch 5, Williams, Gluhbegovic,
and Jew
21
The Brain Models its Environment

• The frontal regions of the braincreate models of
  the world basedon prior experience.
  Thesemodels affect perception and attention.
• In addition, the brain modelsitself.
• Experiments in multi-sensoryprocessing has shown
  thatthe information processing is consistent
  with Bayes Theorem

The Virtual Hospital, Ch 5, Williams,
Gluhbegovic, and Jew
22
Thinking Machines

• Your frontal lobes carry a model of yourself that
  is continually updated from data received from a
  dense sensor network. This implements both
  Instrument Health Monitoring and Calibration
• You learn from new data by updating your model of
  the world.
• You actively seek new data by asking relevant
  questions.

23
Body and Brain form a Symbiotic Unit
24

• Relevant Information

25
Relevance and Perception
A. L. Yarbus, Eye Movements and Vision, Plenum,
New York, 1967 (Originally published in Russian
1962)
26
Free Examination
Three minute recording
A. L. Yarbus, Eye Movements and Vision, Plenum,
New York, 1967 (Originally published in Russian
1962)
27
Relevance and Perception
A. L. Yarbus, Eye Movements and Vision, Plenum,
New York, 1967 (Originally published in Russian
1962)
28
Estimate Ages of the People
Three minute recording
A. L. Yarbus, Eye Movements and Vision, Plenum,
New York, 1967 (Originally published in Russian
1962)
29
Relevance and Perception
A. L. Yarbus, Eye Movements and Vision, Plenum,
New York, 1967 (Originally published in Russian
1962)
30
Remember their Clothes
Three minute recording
A. L. Yarbus, Eye Movements and Vision, Plenum,
New York, 1967 (Originally published in Russian
1962)
31
Relevance and Perception
A. L. Yarbus, Eye Movements and Vision, Plenum,
New York, 1967 (Originally published in Russian
1962)
32
Estimate Material Circumstances
Three minute recording
A. L. Yarbus, Eye Movements and Vision, Plenum,
New York, 1967 (Originally published in Russian
1962)
33
Relevance and Perception
A. L. Yarbus, Eye Movements and Vision, Plenum,
New York, 1967 (Originally published in Russian
1962)
34
How Long has the Visitor been away?
Three minute recording
A. L. Yarbus, Eye Movements and Vision, Plenum,
New York, 1967 (Originally published in Russian
1962)
35
Relevance and Perception
A. L. Yarbus, Eye Movements and Vision, Plenum,
New York, 1967 (Originally published in Russian
1962)
36

• Do We Analyze Everything?

37

• http//viscog.beckman.uiuc.edu/grafs/demos/15.html
  

38
Inattentional Blindness

• http//viscog.beckman.uiuc.edu/grafs/demos/15.html
  
• http//viscog.beckman.uiuc.edu/grafs/demos/10.html
  
• http//viscog.beckman.uiuc.edu/grafs/demos/12.html
  

39

• AUTOMATED INQUIRY

40
Spirit and Opportunity Remote Science
More and more are our instruments required to
perform science operations further from the
intervention of humans.
Dust devils whip across Gusev Crater on Mars
41
The Expansive Floor of Gusev Crater
42
Rock Outcrop (Methuselah)
43
Underwater Robotic Explorers

• At the Monterey Bay Aquarium Research Institute
  (MBARI), researchers are employing robotic
  submarines to explore the Deep Pacific Ocean.

44
Intelligent Autonomous Instruments

• Require
• Stability Control
• Instrument Health Monitoring
• Automated Calibration
• Accurate Onboard Data Analysis
• Adequate Data Coverage
• Ability to Actively Seek Data

45
Novel Instrument Design

• To accomplish these goals, these novel
  instruments must
• Monitor their own state (health and calibration)
• Infer their state from self-sensing
• Be equipped with dense sensor networks
• Infer calibration parameters
• Learn from data
• Make inferences from data
• Perform hypothesis testing
• Ask new questions
• Actively seek new data
• Select optimal experiments

46
The Basic Components
47
The LEGO Mindstorms NXT System
The NXT Brick is the brain of the system.
1
2
Touch Sensor
3
Microphone
4
Light Sensor
UltrasonicRangefinder
5
250!
Servo Motors
6
48
Lego teams with HiTecnic
NEW!
PrototypeBoard
Accelerometer
Color Sensor
Digital Compass
Sensor and Motor Multiplexers
49
Robotic Scientists
This robot is equipped with a light sensor. It is
to locate and characterize a white circle on a
black playing field with as few measurements as
possible. LANDMINE DETECTION!
50
The Robots Thoughts
Past Measurement LIGHT
Next Measurement
Past Measurement DARK
Set of Hypothesized Circles
Mean Circle
Area within Robots Reachcolored according to
ENTROPY
Robot Center
51
'Am I already in the shadow of the Coming Race?
and will the creatures who are to transcend and
finally supersede us be steely organisms, giving
out the effluvia of the laboratory, and
performing with infallible exactness more than
everything that we have performed with a slovenly
approximativeness and self-defeating inaccuracy?'
George Eliot (Mary Anne Evans), The
Impressions of Theophrastus Such, 1879.
52
Special Thanks to John SkillingAriel
CatichaJanos AczélKeith EarlePhilip
ErnerDeniz GencagaPhilip GoyalSteve
GullJeffrey JewellCarlos Rodriguez
And also to Emily KnuthRockne KnuthAnn
KnuthJoshua KnuthRoland DerouenBernadette
Derouen For their patience with the robotic
invasion that has been thrust upon them.