Information Foraging

1
Information Foraging

• From Information Visualization
• to Sensemaking
• Connecting the Minds Eye
• to the Minds Muscle
• Stuart Card
• User Interface Research Group
• Palo Alto Research Center
• 11 October 2004 - Austin, Texas

2
INFORMATION VISUALIZATIONThe Minds Eye
3
www.fundrace.org
4
SmartMoney U. Maryland
5
University of Groningen
6
Information Visualization The use of
computer-supported, visual representations of
abstract data to amplify cognition.
7
Statistical Graphics
Data Graphics Info Design
? Tukey
? Cleveland
Cognition Perception
? Eick
? Bertin
? Ware
?Tufte
Information Visualization
User Interfaces
? PARC
?De Fanti et al
Nat. Science Visualization
? Shneiderman
? Mackinlay
? Catarci
? Roth
Computer Graphics AI
Databases
8
U. Maryland
Börner Viswanath, U. Indiana
9
PARC
U. Maryland
CMU
Börner Viswanath, U. Indiana
10
PARC
U. Maryland
CMU
Börner Viswanath, U. Indiana
11
Georgia Tech
PARC
U. Wittenberg
U. Maryland
CMU
Bell Labs
Virginia Tech
Börner Viswanath, U. Indiana
12
Georgia Tech
U. Berkeley
PARC
U. Wittenberg
U Minnesota
U. Maryland
CMU
Bell Labs
Virginia Tech
Börner Viswanath, U. Indiana
13
Börner Viswanath, U. Indiana
14
Statistical Graphics
Analysis Pipeline
Graphical Semiotics
IVEE
Information Design
DB Dynamic Queries
Film Finder
Cone Trees
Life Lines
Table Lens
Algo- rithms
Zoomable UI
Tree Maps
WebBook
UI 3D Info Workspace
Hyperbolic Trees
Spacial Info Mgt
Rubber Sheet
Doc Lens
Magic Lens
Bi-Focal Lens
PWall
UI Fish-Eye
UI Controls
DB Multivariate
15
Visualization Reference Model
Card, Mackinlay, Shneiderman, 1999
16
Mapping Data Into Visual Form
Data Vocabulary N Norminal O Ordinal Q
Quantitative
Visual Vocabulary Space (Nominal, Ordinal,
Quantitative) Connection/Enclosure Marks
(Points, Lines, Areas, Volumes) Retinal (Size,
Orientation, Grayscale, Color,
Texture, Shape) Temporal
Bertin, Mackinlay
17
Taxonomy of Information Visualization Techniques
Taxonomies by Bertin, Cleveland, Wilkinson,
Shneiderman, Card and Mackinlay, Stolte and
Hanrahan, Amar and Stasko
Card, 2002
18
I. Simple Visual Structures Direct Reading
XYZ
(XYZ)R
(XY)Z
X
XY
1D
2D
3D
4D
XYR
Beyond Here Barrier of Perception
19
I. Simple Visual Structures Articulated
XYR2
Enclosure Trees
5D
4D
Link Trees
Barrier of Perception
20
II. Composed Visual Structures
(XY)5
XYR
XYZXYZ
21
III. Interactive
Dynamic Queries Magic Lens Overview
Detail Linking Brushing Extraction
Comparison Attribute Explorer
22
IV. Attention Reactive
Data-based Methods Filtering Selective
Aggregation View-based Methods Micro-macro
readings Highlighting Visual Transfer
Functions Perspective Distortion Alternate
Geometries
23
Pauling Point Place where results in a field
slow down (and Linus Pauling exits).
RESULTS
TIME
24
(No Transcript)
25
(No Transcript)
26
SENSEMAKINGThe Minds Muscle
27

• The purpose of information visualization is
  insight, not pictures.
• With apologies to Richard Hamming

28
Example

• Design a curriculum for printer repair
• Russell, Stefik, Pirolli, Card (1983)

29
Residue
Schema
30
Residue
Data Coverage
Schema
31
Schema
Residue
32
Schema
Residue
33
Sensemaking Learning Loop
Learning Loop Complex
Search for Good Representations
Generation Loop
RepresentationalShift Loop
Representations
Residue
Encode Data in Representation
Data CoverageLoop
ProcessingRequirements of Task
Encodons
Task Structure
task operations
Russell, Stefik, Pirolli, and Card (1983)
34
Sensemaking

• Process of
• searching for a representation and
• encoding data in that representation to
• answer task-specific questions.

35
Schemas for Business Intelligence Analyst
36
Get Data
Use in Task
Represent in Schema For Task
37
Sensemaking Loop
Operators
1 ---- a --- b --- 2 --- a ---
Use it in a task
World of Information
Analytical Visualization Representation
Get data
38
Research Problems for Sensemaking

• Reference model for the Sensemaking Loop.

39
Sensemaking
Applicable to many domains Education,
Business, Journalism Harbinger
domain (extreme environment) intelligence
analysis Intelligence Analysts are the Jet
Pilots of Sensemaking.
40
3.Search for Information
6.Search for Relations
9.Search for Evidence
12.Search for Support
15.Reevaluate
16.Presen-tation
15
Sensemaking Loop for Analysts
14
13.Hypotheses
12
DualSearch
9
10.Schema
11
Foraging Loop
7.Evidence File
8
6
STRUCTURE
Sensemaking Loop
3
4.Shoebox
5
1.External Data Sources
2
NIMD
2.Search Filter
5. Read Extract
8.Schematize
11.Build Case
14.Tell Story
EFFORT
41
Sensemaking Problems
Cycle Time
? Strategic Intelligence
? Preparing a Legal Brief
? Buying a Laptop
Data Density
? Tactical Intelligence
? Emergency Response
Individual
Workgroup
Organization
Radius of Collaboration
42
Five Step Model for Intelligence (CIA)
Scientific Experimentation (Lederer)
Customer Knowledge Sharing Model (Xerox)
Boyd OODA Loop (Air Force)
43
Research Problems for Sensemaking

• Reference model for the Sensemaking Loop.
• Pain points and interventions.

44
? Interactive Presentation
? Hypothesis Management Structured
Argumentation
? Schema Visualization
? Accelerated Extraction Linking
? Accelerated Search
? Accelerated Reading
45
Research Problems for Sensemaking

• Reference model for the Sensemaking Loop.
• Pain points and interventions.
• Conceptual schemas for a domain and analytic
  visualizations for them.

46
Analytic Visual Schemas

• Space / Location
• Time-Lines
• Organizations
• People Profiles
• Project Flows
• Networks
• Causation
• Cross-Products of Above

47
Research Problems for Sensemaking

• Reference model for the Sensemaking Loop.
• Pain points and interventions.
• Conceptual schemas for a domain and analytic
  visualizations for them.
• Understand the integrated perception-cognition of
  analytic visualizations.

48
(No Transcript)
49
(No Transcript)
50
Research Problems for Sensemaking

• Reference model for the Sensemaking Loop.
• Pain points and interventions.
• Conceptual schemas for a domain and analytic
  visualizations for them.
• Understand the integrated perception-cognition of
  analytic visualizations.
• Develop attention-reactive HII systems.

51
Human-Information Interaction
52
attention-reactive user interface
FRONT END Reactive Visualizations
basic architecture
BACK END Semantic Contextual Processing
53
PARC CONCEPTUALARCHICTURE
Analyst Layer
Workspace HI Layer
3Book
Entity Wall
Time Trees
ACH
CounterPoint
Perspective Book
Dynamic Relations
Facts v beliefsAssertions v. denials
Discourse Context Analysis
Content Processing Layer
Limited Inference
Ontology Use
Relations, Sentence Condensation
Deep Analysis of Sentence Structure
Entities, Names, Dates, Places, Pathogens,
Authority, Topics, Summaries, Relations
Statistical Text Analysis
Shallow Parsing
Document Base Layer
InfoScent Focus Crawler
Source Document Manager (UpLib)
54
5 characteristics of HII

• based on adaptation to large information
  landscapes
• maximize rate of gain of information relative to
  task
• content and context aware
• compute semantic analysis
• user aware
• track Degree-of-Interest
• externalizes cognition
• information visualization
• supervisory control system
• mixed initiative

55
A. Adaptation to Information Space
High profit relevant item
Explore (Monitor)
Enrich (Narrow)
Exploit (Read Analyze)
Space of all accessible relevant and irrelevant
items
56
information patches
57
example calculate patch residence
timerandomly-ordered prey
Cumulative gain g(tW)
R
R2
R1
t1
t
t2
tB
tW
Between-patch time
Within-patch time
58
Charnovs marginal value theorem
max gain when slope of within-path gain g
average gain R (tangent in diagram)
Gain
R
g(tW)
Within-patch time
Between-patch time
tB
t
59
between-patch enrichment
Gain
R2
R1
g(tW)
Within-patch time
Between-patch time
tB1
t1
tB2
t2
enrichment
Example arrange physical office efficiently
60
within-patch enrichment
Example Better filtering of search hits
enrichment
g1(tW)
Within-patch time
Between-patch time
61
B. Content Aware
SEMANTIC PIPELINE
Discourse Context Analysis
Formal semantics language processing
Facts v beliefsAssertions v. denials
?
Limited inference
?
Ontologies and Normalization
Ontology Use
RelationsSentence Condensation
?
Lexical Functional Grammar
Deep Analysis of Sentence Structure
AuthorityTopicsText Summaries Relations
Machine Learning PLSA ClassificationSpreadi
ng Activation
Statistical Text Analysis
?
Shallow Parsing
Finite State Parser
?
Entities
Raw documents
62
C. User Aware
63
D. Externalizes Cognition
www.fundrace.org
64
E. Supervisory control system
BACKEND
- degree of interest - initiative
FRONTEND
BACKDOOR
22 different mixed Initiative modes.
- visualization - attention-reactive display
SEMANTICPIPLINE
extract task-relevant properties
65
mixed initiative data flow between focus context
IMPLICIT RETRIEVAL AND INTERRUPTION
TASK SWITCH
1. User activity ? Implicit retrieval 2. Hi Info
Scent ? User Notices 3. User puts to use
66
four levels of information visualization UI
1. Infosphere 2. Information Workspace 3.
Visual Knowledge Tools 4. Visually-Enhanced
Objects
Retrieve
File
Manipulate
Perceive
67
reading as sensemaking
Adlers model

• ELEMENTARY READING
• Harder than you think in an electronic book.
• Navigation
• Legibility of large pages

68
reading as sensemaking

• INSPECTIONAL READING (What is it about?)
• Skim
• Superficial reading
• ANALYTICAL READING (What does it mean?)
• Classify
• Restate the point
• Outline
• Find authors goal
• interpret key words
• Grasp main propositions
• Find main argument
• Determine success of argument
• Understand before criticizing
• React
• SYNTOPTICAL READING (How does it compare with
  other books?)
• Find relevant passages
• Establish a common terminology
• Clarify the questions
• Define the issues

69
page mechanics
70
reading large format books
71
(No Transcript)
72
sensemakng 1
73
reading as sensemaking

• ELEMENTARY READING (What does it say?)
• Page turning
• Zoom to read
• Rocker pages
• Point and pan
• Click ahead reading
• Thumb to open
• INSPECTIONAL READING (What is it about?)
• Accelerated Finding
• Smart TOC
• Smart Index
• Accelerated Reading
• Summary in place highlight for skimming
• Degree of Interest highlighting
• ANALYTICAL READING (What does it mean?)
• Slide-out pages
• Page compare
• Highlighting
• Bookmarks

74
CONCLUSION

• Information Visualization is pretty well solved.
• Next problem Use Infovis for Sensemaking
  (visualization-centric part visual
  analytics).
• Open research directions

75
Research Problems for Sensemaking
Sensemaking
HII

• Sensemaking Loop.
• Pain points and interventions.
• Schema visualizations
• Understand perception-cognition of
  visualizations.
• Attention-reactive HII systems.

• Adaptive to information landscape
• Content and context aware
• User aware
• Externalizes cognition
• Supervisory control

76
(No Transcript)