Distributed Cognition in the Pacific Rim

1
Distributed Cognition in the Pacific Rim
PARVAC

• Prof. Tom Furness
• Human Interface Technology Lab
• University of Washington

2
before....
3
after...
4
The power of vision
5
(No Transcript)
6
(No Transcript)
7
adaptive system element...
8
Distribution of receptors in eye
Rod and cone densities as a function of
eccentricity along the horizontal meridian.
9
Super Cockpit revisited
10
(No Transcript)
11
(No Transcript)
12
egocentric vs. exocentric views
13
finger in gods eye display
14
(No Transcript)
15
What we learned

• We have more power than we think
• But the interface makes all the difference!
• Virtual interfaces really work
• Accelerated learning when immersed
• gtgt 60 degrees
• Situation awareness

16
But are we making progress?
17
Distribution of receptors in eye
Rod and cone densities as a function of
eccentricity along the horizontal meridian.
18
the most important interface

• the last few inches

• How to get bandwidth to and from the brain
• How to get bandwidth between brains

19
PARVAC
Pacific Rim Visualization and Analysis Center
20
PACIFIC RIM
21
WATER!!
22
FIRE!!
23
http//earthquake.usgs.gov/eqcenter/recenteqsww
24
Pacific Rim Threats

• Natural
• Earth
• Earthquakes
• Volcanoes
• Water
• Tsunami
• Waterways
• Birds fish
• Migration patterns
• Man-made
• Ports of entry
• Shipping
• people
• Lots of borders
• Agri-terrorism

Ring of Fire!
25
PARVAC Mission

• To advance distributed visual analytics for
  public safety and security around the Pacific
  Rim.

26
Threads

• Unlock intelligence
• Bandwidth to brain
• Link minds
• Bandwidth between brains
• Enlighten minds
• Courses and simulations
• Focus on threats to the Pacific Rim
• Provide tools to other RVACs

27
PARVAC Collaborators
28
Partners

• US
• UW (Human Interface Technology Laboratory)
• Tom Furness (IE, EE, ME, AA, TC)
• Suzanne Weghorst (HIT Lab)
• Bruce Campbell (HIT Lab-Oceanography)
• Don Parker (HIT Lab-Medical School)
• Bill Winn (Education)
• Zelda Zabinsky (IE)
• Univ. of Hawaii - Maui High Performance Computing
  Center Pacific Disaster Center
• Tak Sugimura
• Univ. of Alaska - Arctic Supercomputing Center
• TBD

29
Partners

• Canada
• Simon Fraser University (John Dill, Nick Hedley)
• Univ. of British Columbia (Brian Fisher)
• Australia
• Univ. of Western Australia (Karen Haines)
• Royal Melbourne Inst. Of Technology (Mark Burry)
• University of South Australia (Bruce Thomas)
• HIT Lab Australia - University of Tasmania (Young
  Choi)

30
Partners

• New Zealand
• HIT Lab NZ- University of Canterbury (Mark
  Billinghurst)
• Industrial Members
• The Boeing Company (David Kasik)
• Change Tools Inc. (Fred Ordway)
• ARToolworks (Alden Jones)

31
PARVAC TEAM
Human Photonics
Interface FLOW
Augmented Reality
Seibel
Billinghurst
Weghorst
Adv. Interfaces
Fear Pain
Human Interface Technology
Furness
Human Factors
Telecollaboration
Education
Winn
Campbell
Parker
32
PARVAC PARTNERS
existing partners
planned partners
33
Our expertise

• Super cockpit
• Super computing
• Cognitive modeling
• Visualization (Center for Environmental Viz)
• Representational constructs
• Virtual reality, augmented reality, haptics
• Control/display technology
• Distance collaboration (e.g. GreenSpace)
• Decision making optimization
• Vision
• Education

34
Distributed Cognition thread

• DC proposes that human knowledge and cognition
  are not confined to the individual but
  distributed by placing memories, facts, or
  knowledge on the objects, individuals, and tools
  in our environment.
• DC views a system as a set of representations,
  and models the interchange of information between
  these representations. These representations can
  be either in the mental space of the participants
  or external representations available in the
  environment.

from Wikipedia
35
Further definitions of DC

• Distributed Cognition -Type T (thematic)
• deconstruct VA tasks into parts and distribute
  for analysis
• Trends
• Anomalies
• Reintegrate parts for correlation and hypothesis
  formulation
• Team handoff
• Distributed Cognition - Type S (spatial)
• Location-specific perspectives
• Common threats
• Collaborative analysis of outcomes

36
PARVAC Project elements

• MELD Mind Expansion over Long Distances
• RimSim Pacific Rim simulation engine
• RimSem emulation engine
• JITC3 just-in-time mobile command and control
  environment for first response
• Bold Stroke blue sky journeys into threat space

37
MELD

• Mind Expansion across
• Long Distances

38
Vulcan mind meld
39
MELD

• Representational Constructs
• Type T Knowledge representation mental
  scaffolding
• For hypothesis formulation and testing
• Type Ssituational awareness (spatial and state)
• For decision making, preparedness and emergency
  response
• Interactive constructs (as related to the
  representational constructs)
• Individually within the VA process for individual
  themes
• Between individuals for reintegrating thematic
  space
• Between individuals to construct SA from
  different perspectives
• Mind portals
• Transporting contents of constructs from one mind
  to the another
• Asynchronously
• Synchronously

40
MELD approach

• Driven by cognitive science models
• Build upon super cockpit experiences
• Build upon Center for Environmental Visualization
  (CEV) ocean viz work
• Network PARVAC participants to walk the talk
• Develop metrics to assess goodness (e.g. flow,
  resource expenditure, etc.)

41
Center for Environmental Visualization -
Oceanography
42
JITC3

• just-in-time mobile command and control
  environment for first response

43
Emergency Response Network
MCP-Fire
FR-F1
MCP-Police
FR-P1
FR-P2
FR-P4
MCP-Medical
C3
FR-P3
MCP-Transportation
MCP-other
FRfield responder
MCPmobile command post
44
NZ VisionSpace
45
Mixed Reality
46
(No Transcript)
47

• NSF sponsored project at Boston Museum of Science
• ARToolworks Inc.
• HIT LAB US
• HIT LAB NZ
• Technical Director - Dr. Nick Hedley
• Opening Oct 27, 2005
• www.starwars.mos.org

48
JITC3

• Develop a flexible, portable command post for
  deployment with emergency response personnel
• Build upon augmented reality, MagicBook
  technology and virtual retinal display (use
  ARToolwork Inc. ARTK and ChangeTools ZENU
  technology
• Build network with field PDA units that
  communicate to the command post and in turn to
  the central command post.
• Integrate with C3 simulations in US, Canada,
  Australia and NZ (e.g. VisionSpace at HIT Lab NZ

49
RimSim

• Simulation Engine for the Pacific Rim

50
RimSim

• Adapt existing gaming engine to generate dynamics
  of
• Tsunami
• Dirty bomb in a west coast port SimCity-like
• Use to test MELD JITC3
• Use to test preparedness and first response
  dynamics
• Grow RimSim --------gt RimSEM (combination of
  simulation and emulation of the actual code and
  algorithms that can be deployed

51
BOLD STROKE!!

• Blue sky thinking about VA solutions

52
Bold Stroke Team
Greg Bear
Nick Hedley
Doug Trumbull
?
David Kasik
53
PARVAC Products

• Tools for modeling, simulating and visualizing
  dynamic effects of geophysical events (e.g.,
  earthquakes) and other geospatially distributed
  threats (e.g., pandemic) on local/regional
  critical infrastructure.
• Tools for optimizing supply cache distribution in
  preparation for most likely threats.
• Tools for parsing disaster response strategies
  into logistic functions (i.e., distributed
  cognition).
• Tools for optimizing analyst flow in detecting
  and monitoring emergent events.
• Tools for optimizing situation awareness and
  responder flow during an emergent event.
• TOOLS FOR LINKING MINDS ACROSS THE PACIFIC RIM.
• TOOLS FOR TEACHING THROUGH SIMULATION.

54
Current PARVAC Research
BOLD STROKE
55
PARVAC Near-Term Capabilities

• Software pipeline for geospatially distributed
  data, based on CEV ocean sciences visualization
  platform.
• Incorporation of Seattle/KingCo critical
  infrastructure data into CEV platform, for
  simulation of earthquake/tsunami effects.
• Distributed cognition task/role analysis for
  Seattle/KingCo medical disaster response.
• Simulation and optimization methods for regional
  distribution of medical response supplies.
• Human interface requirements analysis and
  laboratory testbed configuration for primary
  classes of incident response roles (C3, crew
  chief, field operatives, local public).
• RimSim game architecture for collaborative/competi
  tive resource partitioning for simulated Pacific
  Rim events.
• PARVAC team distributed collaboration platform,
  with support for both synchronouns and
  asynchronous collaboration.

56
Where there is no visionthe people perish.
Proverbs 2918