Some Thoughts on Machine Understanding

1
Some Thoughts on Machine Understanding

• Peter Clark
• Knowledge Systems
• Boeing Engineering and Information Technology

2
On Machine Understanding

• Understanding creating a situation-specific
  model (SSM), coherent with data background
  knowledge
• Data suggests model fragments which may be
  appropriate
• Models suggest ways of interpreting data

?
?
Garbled graph of relationships
Coherent Model (situation-specific)
3
On Machine Understanding

• Core theories of the world
• Ton of common-sense/ episodic/experiential
  knowledge (the way the world is)

• Only a tiny part of the target model
• Contains errors and ambiguity
• Not even a subset of the target model

Assembly of pieces, assessment of
coherence, inference
?
?
Garbled graph of relationships
Coherent Model (situation-specific)
4
What are some ingredients?

1. Elaboration (scene building)
2. Representing possibilities
3. Coherence assessment (matching?)
4. Viewpoints/context
5. Knowledge acquisition

5
ElaborationThe Parachute Sentences
Parachutes slow down a person falling through
the air. This means that he or she can land
safely when jumping out of a plane. When open, a
parachute creates lots of drag as air pushes
against its underside. This slows its fall.
6
The Parachute Sentences
Parachutes slow down a person falling through
the air. This means that he or she can land
safely when jumping out of a plane. When open, a
parachute creates lots of drag as air pushes
against its underside. This slows its fall.
7
1. Elaboration (cont)
John chopped down the tree.

• A vivid picture comes to mind
• John adult male, out in woods
• holding an axe (or chain saw?)
• Tree is 30ft high pine tree
• or a modification of that time I sawed a
  Christmas tree
• or that documentary on logging in Canada
• 81 ratio of prior to explicit knowledge
  (Graesser, 81)
• Episodic/experiential knowledge plays a key role
• Also core knowledge plays a key role
• Not a deductive process!

8
1. Elaboration Using WordNet

• Augment semantic structure with definitional
  knowledge.

The kid hit the ball hard.
9
1. Elaboration Using WordNet

• Augment semantic structure with definitional
  knowledge.

The kid hit the ball hard.
10
1. Elaboration Another example
The Global Positioning System is a satellite
navigation system designed to provide
instantaneous position, velocity and time
information almost anywhere on the globe.

• satellite orbit around earth receive/send radio
  messages
• navigation information about location
• system assembly of artifacts which together
  perform a task
• people often want to know where they are
• (after more sentences) entire model on how GPS
  systems work.

11
2. Representing Possibilities

• Went to encode a space of possibilities
• not what the model is, but constraints on what
  the actual models might be
• enable actual models to be built and assessed

where?
Most eucaryotic genes have their coding
sequences interrupted by noncoding sequences,
called introns. The scattered pieces of coding
sequence, called exons, are usually shorter than
the introns, and the coding portion of a gene is
often only a small fraction of the total length
of the gene. Most introns range in length from
about 80 nucleotides to 10,000 nucleotides,
although even longer introns exist.
p220, Alberts 1998.
12
2. Representing Possibilities
Got
Want
Possible (consistent)
More likely/ preferred
Impossible (inconsistent)
Less likely/preferred
Spaces of possible models
13
2. Representing Possibilities

• Are a few (feeble) methods in KM for this
• type restrictions
• (every Person has (spouse ((must-be-a Person)))))
• (sometimes ltxgt)
• (every Car has (parts ((sometimes (a
  Spare-Wheel)))
• Cardinality constraints
• (a Group with (min-cardinality ())
  (max-cardinality ()))
• Range of values
• size is between X and Y
• Still largely lacking in how to represent and
  reason with vague knowledge

14
3. What Makes a Representation (Model) Coherent?

• We dont just blindly accept new knowledge
• Minsky We proactively ask a set of pertinent
  questions about a scene, e.g., what is X for?
  What are the goals? etc.
• What makes a representation coherent?
• Simple consistency (The man fired the gnu.)
• Purposefulness (for artifacts)
• The engine contains a thrust reverser.
• vs. The engine contains an elephant.
• vs. The engine contains a book.
• Knowledge entry is a serious misnomer!
• Really talking about Knowledge Integration

15
3. What Makes a Representation (Model) Coherent?
TRANSPONDER A combination receiving and
transmitting antenna on a communications
satellite. TRANSPONDER A combination receiver,
frequency converter, and transmitter package,
physically part of a communications satellite.
Transponder parts antenna
purpose receive, transmit
Transponder parts receiver
frequency converter transmitter
part-of communications satellite
Relay/Mediate
16
3. A Catalog of Coherence Criterea

• Volitional actions
• Agents must be capable of an action
• legally, skill, fiscally, anatomically
• Action serves a broader purpose/goal
• Need equipment/resources/instruments, instruments
  must be adequate
• Non-volitional actions
• There is a cause (inc. randomness)
• Spatial
• statics objects must be close
• dynamics objects can move in the required way
• Temporal objects exist at the same time

17
3. A Catalog of Coherence Criterea (cont)

• Objects
• physically possible
• parts connected together at appropriate places
• materials are appropriate
• suspension/tension etc., gravity
• physically normal/expected/standard
• need to know normal shapes, sizes, etc.
• Artifacts
• Purposefulness
• all parts play some role wrt. one of its intended
  functions (or subtasks thereof). Expect design to
  be optimized.
• Could treat biological objects as artifacts

18
3. Coherence and KM

• KM unable to tolerate incoherence
• Current Error! Switching on the debugger
• Desired This representation is generally ok,
  except this bit looks weird, and that bit
  conflicts with this bit.
• Problem compounded by long inference chains
• (cf. Cyc dont think too hard ?)
• How could we change KM to be more tolerant?

19
4. Viewpoints and Context
Component theories/ ontologies(?)
vs.
Reason over Giant KB
Problem-specific KB, contains selected units

• Latter seems right, but
• can a big KB really be partitioned like this?
  (everything is connected!)
• Models may vary by
• ignoring detail
• making different approximations
• using different ontologies

20
4. Viewpoints and Context

• e.g., DNA sequence of different region types
• intron-exon-intron-exon
• promotor-gene-terminator
• nucleotide pair-nuc pair-nuc pair
• Makes a difference
• Given The polymerase attaches to the promotor,
  and then moves down the strand.
• then answer Where will the polymerase be?
• nucleotide? gene? intron?
• The point Its not simply a matter of having all
  viewpoints coexisting
• Another example A satellite sends
  signals/messages/position information.

21
5. Knowledge Acquisition

• Where do the core theories come from?
• Hand engineered?
• Where does all the mundane knowledge come from?
• Schubert-style?
• Dictionary/glossary definitions?

22
5. Knowledge AcquisitionCan all this be
Bootstrapped?
Collection of coherent scenes
Jungle of parse trees/ semantic graphs
KB
Text
Domain Ontology
List of compound nouns and verbs (entities and
actions)
Scene library