Cognitive Modeling Techniques for Reasoning from Diagrams

1
Cognitive Modeling Techniques for Reasoning from
Diagrams

• Ron Ferguson
• Intelligent Systems and Cognitive Science Groups
• College of Computing

2
Cognitive Modeling

• Using AI techniques to model or simulate aspects
  of cognition
• One approach Compare model with results of
  psychological experiments
• Bi-directional influences
• Can investigate complex (from a psychological
  point of view) cognitive phenomena

3
Motivation What makes a diagram understandable?

• Not simplicity
• Simple diagrams can still confuse
• Not realism or ability to recognize components
• A combination of
• Easily perceived visual relations, which link to
• Qualitative spatial relations, which support...
• Specialized spatial reasoning in the domain
• Also Regularity

4
Regularity in Diagrams
Regularity in Diagrams
Diagram Examples
Visual symmetry conveys functional symmetry
Use of repetition to convey state transitions
Visual repetition conveys functional repetition
Multiple repeating objects with critical
differences
5
Key Issues

• How is visual structure used to understand
  diagrams?
• What role does symmetry and repetition play?

6
GeoRep A Spatial Representation Engine
7
GeoReps Low-Level Relational Describer (LLRD)
8
Example representation from GeoReps LLRD
(mid-connect ltL2gt ltL13gt)(hort-interval-equal
ltL7gt ltL6gt ltreference-frame
1gt)(parallel-above ltL10gt ltL3gt
ltreference-frame 1gt)(above ltL13gt ltL11gt
ltreference-frame 1gt)(horizontal ltL11gt
ltreference-frame 1gt)(h-aligned (corner ltL10gt
ltL11gt) (corner ltL11gt ltL12gt)
ltreference-frame 1gt)(during ltL1gt ltL2gt)(before
ltL1gt ltL5gt)

• (polygon ltpolygon-1gt)(polygon-member ltpolygon-2gt
  ltL7gt)(number-of-sides ltpolygon-1gt 6)
• (obtuse (corner ltL11gt ltL12gt)
  ltpolygon-1gt)(corner ltL9gt ltL10gt)(acute (corner
  ltL9gt ltL14gt) ltpolygon-1gt)(concave
  (corner ltL9gt ltL10gt) ltpolygon-1gt)(conve
  x (corner ltL9gt ltL14gt) ltpolygon-1gt)(perpendicular-
  in (corner ltL9gt ltL10gt) ltpolygon-1gt)(indent
  ation ltpolygon-1gt (set (corner ltL9gt
  ltL10gt)))(protrusion ltpolygon-2gt (set (corner
  ltL7gt ltL8gt)
• (corner ltL3gt ltL8gt) (corner ltL3gt
  ltL4gt)
• (corner ltL4gt ltL5gt)
• (corner ltL5gt ltL6gt)))

9
GeoReps High-Level Rules
Antecedent clauses TEST TEST-NOT TEST TEST-NO
T PROXIMATE DIFFERENT DIFFERENT-ORDERED TEST-
EVAL RLET GENTEMP Consequent clauses Assert
(default), ASSUME, RETRACT

• Macro overlay of LTRE rules
• Special features
• proximity test
• visual operations test

  NAND Gate Template (grule ((attached ?line1
?arc) (attached ?line3 ?arc)
(different ?line1 ?line3) (members-of
(polyline ?polyline)
(line-grouping ?line1 ?line2 ?line3))
(abuts ?circle ?arc) (gentemp ?gate-name
"NANDGATE-")) (and (represents
(elements ?polyline ?arc ?circle)
(nand-gate ?gate-name))
(input-plate ?gate-name ?line2)
(output-circle ?gate-name ?circle)))
10
GeoRep Summary

• A 2-level architecture
• Low-level visual relations, domain-independent
• High-level place vocabulary, domain-dependent
• Domains
• Logic circuits
• Physical diagrams
• Military planning diagrams

11
Key Issues

• How is visual structure used to understand
  diagrams?
• What role does symmetry and repetition play?

12
Phenomenological Aspects of Regularity
Squares or diamonds?
Goldmeier, E. (1936/1972). Palmer, S. E.,
Butcher, N. M. (1981).
13
Which figures are regular?
Phenomenological Aspects of Regularity
14
Review of other symmetry detection techniques

• Transformational invariance (Weyl, 1952 many
  others)
• Point-to-point measurements (Jenkins, 1983
  Labonte, 1995 Wagemans, 1993)
• Brushfire methods (Blum, 1978 Brady, 1983
  Burbeck Pizer, 1995 many others)
• Measures of asymmetry (Zabrodsky, 1992 1995)

15
The Symmetry as Self-Similarity Hypothesis

• Instead of looking for transformational
  invariance, find relational self-similarity
• Key idea Symmetry detection utilizes same
  cognitive process as similarity comparison
• Use Structure Mapping Theory (Gentner, 1983) as
  basis

16
Visual symmetry detection using MAGI
INPUT Line drawing
17
Examples
21 entities 160 relations SES 14.361 9.9 sec.
13 entities 113 relations SES 4.924 4.2 sec.
14 entities 112 relations SES 12.245 10.3 sec.
18
Examples
Mapping Perceptual and Conceptual Repetition
19
The Preference for Vertical Symmetry

• Geometric symmetry doesnt depend on figure
  orientation
• Human symmetry detection does
• Vertical symmetry is detected most easily (Mach,
  1896 many others)
• Explanations
• Evolutionary explanations
• Retinocentric explanations
• But...

Harder
Easier
Symmetric?
20
Overview of the MAGI model
INPUT Line drawing
21
MAGIs simulation of orientation effects in
symmetry detection

• Depends on visual relations
• - above-below relations are directed and
  salient - left-right relations are
  commutative
• Relations that depend on the reference frame
  change with object orientation

22
Handling figures with good intrinsic axes
In some figures, the available visual relations
are sufficient for symmetry detection at any
orientation.
Figure from Wiser (1981).
23
Simulation of Orientation Effects

• Stimuli from Palmer Hemenway (1978)
• Set of 30 16-gons
• Divided evenly between 4x, 2x, single, near and
  rotational symmetry conditions
• Shown at 4 orientations tilted left, vertical,
  tilted right, horizontal
• Task Determine if figures mirror-symmetric
• Same figures given to MAGI. Systematicity rating
  (SES) used to estimate the strength of the
  symmetry in each figure

Score
Humans (Palmer and Hemenway, Exp 1).
MAGI.
24
Summary of MAGIs explanation for the preference
for vertical symmetry

• Determined by visual relations that are
  reference-frame dependent
• Figures with intrinsic axes attenuate the
  preference for vertical symmetry
• Visual structure in these figures are sufficient
  for symmetry detection, even without
  reference-frame dependent visual relations

25
Summary

• We can use cognitive modeling
• To inform existing AI and psychology research
• To explore more complicated cognitive phenomena
  (such as reasoning from diagrams)
• GeoRep A model of the creation of visual
  structure
• MAGI A model of symmetry detection

26
To find out more

• My web page www.cc.gatech.edu/faculty/Ron.Ferguso
  n
• Cognitive Science Brown Bag, Noon, November 2,
  Psychology. Focus on symmetry and cognitive
  modeling
• Intelligent Systems Brown Bag, 2 pm, Nov 13,
  Physics N210. Focus on the construction of
  large-scale diagrammatic reasoning systems
• Spring Course Cognitive Modeling
• Spring Seminar (tentative) Seminar on Visual
  Thinking