Eric Horvitz

1
Toward Models of Surprise

• Eric Horvitz
• Microsoft Research
• University of Washington

2
Surprise
Unexpected or umodeled future event or outcome of
significant consequence.

• Unexpected event We thought about it, but
  considered it would occur with probability lt x
• Unmodeled event We never even thought about
  that
• Significant consequence Significant positive or
  negative swing in utility of outcome for observer
  or group
• Events can be personal, local, national, world

3
Surprise
Unexpected or umodeled future event or outcome of
significant consequence.

• Surprise defined in terms of an observers
  expectations
• Deliberation and modeling can influence degree of
  or nature of surprise
• Moving from surprise to plausible expectation
• Expectation of classes or properties of surprise
  should given that a surprise occurs.

4
Forecasting Surprise
Qualitative and quantative approaches

• Qualitative brainstorming about likelihood and
  nature of surprises over future time periods from
  knowledge, trends, and historical data
• Opportunity Statistical models of surprise
• Learn from databases of past surprise to predict
  future surprises
• Opportunity Statistical models of predictive
  competency
• Context- and content-centric failure to predict
  with accuracy

5
SurpriseCasting Abstract and Project

• Example
• Identify top n most surprising things in prior 1
  year (world events or personal life)
• Repeat for 5 years of surprises
• Characterize nature and rate of significant
  surprises
• Consider top n over longer-term periods in same
  manner

6
SurpriseCasting Abstract and Project

• Turn to the future with statistics about types,
  examples.
• Consider triaging of attention and recommending
  actions.

Assume type occurs give top m examples per type
Recommended triage of focus of attention
additional study, info. gathering, monitoring per
type
Recommended actions to react, contain, disrupt
negative suprises and/or exploit positive
surprises
Top n surprise types, by class or properties
Surprises in technical sophistication of weaponry
available inexpensively and universally
Largescale simultaneous continuous disabling of
communication and sigint satellites available to
multiple states.
7
SurpriseCasting Abstract and Project

• Deliberation about potential surprises
  Backcasting
• Chaining from surprising outcome to explore
  feasibility

Assume type occurs give top m examples per type
Recommended triage of focus of attention
additional study, info. gathering, monitoring per
type
Recommended actions to react, contain, disrupt
negative suprises and/or exploit positive
surprises
Top n surprise types, by class or properties
Exotic transnational actors with access to
increasing technical prowess.
Terrorist organization establishes secret
alliance with seemingly friendly former adversary
state, intolerant of US empire gains access to
nuclear arms with deniability for former
adversary.
8
Expecting Surprises

• Technical developments
• International political terrain

9
Technical Surprises from Past

• Big surprises over 50 years
• Flight
• From likely unachievable dream, to major pillar
  of force projection and deterrence in just a few
  summers
• Quickly melted into background of obviousness
• In a few years From awe of a canvas flying
  contraption to concerns about low-salt meal while
  streaking across ocean in widebody jet.
• Nuclear fission and fusion
• Theory of Relativity and implications out of the
  blue
• Shifts in balance of power
• Risk to nation, human civilization

10
Classes of Future Technical Surprises?

• Computation
• Intelligent systems change world (consumer,
  defense, communications, etc.) in a fundamental
  way?
• Energy
• New surprise source of energy changes economics
  of world
• Changes nature of powerplants in our and
  adversaries weapon systems
• Shift in balance of power hinge on the new source
• Asymmetric force have new tool on their side for
  unleashing chaos and disruption
• Biology
• Assembly of Polio virus via internet and mail
  order in 2002 heralds era of bio-hacking
• Major disruptions via pandemics

11
(No Transcript)
12
Futurist Community Wildcards

• A wild card is a future development or event
  with a relatively low probability of occurrence
  but a likely high impact on the conduct of
  business
• BIPE Conseil, Copenhagen Institute for Futures
  Studies, Institute for the Future Wild Cards A
  Multinational Perspective, Institute for the
  Future 1992

13
Futurist Community Wildcards

• Futurist discussion of wildcards, futurequakes
• Are natural, human-caused, or combinations
• Are negative and positive
• Multiple wildcards can interact
• Breaks, discontinuities vs. trends
• Peterson (2000), Out of the Blue How to
  Anticipate Big Future Surprises
• Cornish (2003), The Wild Cards in Our Future
• Steinmüller (2003), The Future as Wild Card A
  Short Introduction to a New Concept

14
On Expected Surprises

• Earth Sky
• Biomedical Developments
• Geopolitical Sociological changes
• Technology Infrastructure Upheaval
• Surprise Attack
• Spiritual Paranormal
• J. Petersen, Out of the Blue How to Anticipate
  Big Future Surprises, 2000.

15
On Expected Surprises
We are now living in another period of
significant transition--a foreshortened span of
time, during which our surroundings and
experiences will change more than during any era
in history. Humanity has never lived through
the convergence--and, in some cases, the
collision--of global forces of such magnitude and
diversity. One foreseeable outcome might be
global instability another, a planetary
renaissance. J. Petersen, Out of the Blue How
to Anticipate Big Future Surprises, 2000.
16
On Expected Surprises
In any case, during the next two decades,
almost every aspect of life will be fundamentally
reshaped. . J. Petersen, Out of the Blue How
to Anticipate Big Future Surprises, 2000.
17
Technological Prowess and Wildcards
Rapidly growing technological prowess of humans
has, for the first time in history, produced new
classes of wild cards that could potentially
destroy the whole human race. The new kinds of
wild cards are simply too destructive to allow to
happen and therefore require active attempts to
anticipate them and be proactive in dealing with
their early indicators. "Peterson, Out of the
Blue How to Anticipate Big Future Surprises,
2000
18
Preparing for Wildcards
Wild cards can be anticipated and prepared for.
Thinking about a wild card before it happens
is important and valuable. The more that is
known about a potential event, the less
threatening it becomes and the more obvious the
solutions seem. "Peterson, Out of the Blue How
to Anticipate Big Future Surprises, 2000
19
Backcasting

• Cornish Multipronged Aerial Terrorist Attack
  was identified,
• e.g.,1987 article (B. Jenkins) 1994
  article (M. Cetron)
• "Targets such as the World Trade Center not
  only provide the requisite casualties but,
  because of their symbolic nature, provide more
  bang for the buck.
• "In order to maximize their odds
  for success, terrorist groups will likely
  consider mounting multiple, simultaneous
  operations with the aim of overtaxing a
  government's ability to respond, as well as
  demonstrating their professionalism and reach."
• Despite all this, terrorism will remain a
  back-burner issue for Western leaders as long as
  the violence strikes in distant lands and has
  little impact on their fortunes or those of their
  constituents.
• E. Cornish, The Wild Cards in Our Future. The
  Futurist. Washington Jul/Aug 2003.Vol. 37, Iss. 4
  .

20
Backcasting

• How might we proceed to evaluate this warning?
• Poll experts on likelihood and targets of aerial
  suicide attacks. Assume aerial attacks plausible.
• Backcasting Reason about the challenges and
  solutions.

21
Research on Statistical Models of Surprise

• Most reasoning about surprise qualitative
• Challenge Data sparcity
• Pushing on theoretical foundations of surprise
• Example Research traffic prediction service,
  Seattle

22
Streaming Intelligence

• Continual sensing, learning, and reasoning always
  available in stream of daily life

23
Predictions in Your Pocket
24
Traffic Prediction Challenge
25
From Incident Reports to Salience

• e.g., Accidents
• How many lanes?
• Emergency vehicles?

Operator ID Nick Heading INCIDENT
MessageINCIDENT INFORMATION Cleared 1637 I-405
SB JS I-90 ACC BLK RL CCTV 1623 WSP, FIR ON
SCENE
26
Data Set Data Abstraction
Lane Station Bottleneck
27
(No Transcript)
28
(No Transcript)
29
(No Transcript)
30
(No Transcript)
31
(No Transcript)
32
Building Predictive Models from Data

• System-wide
• status dynamics
• Incident reports
• Sporting events
• Weather
• Time of day
• Day of week
• Season
• Holiday status

33
Predict Future Surprises

• Surprise at time t in the future Infer surprise
  with model built from training set
• Evidence
• Time of day, day of week
• Bottlenecks, flows and their durations
• Incident reports
• Weather
• Games
• Accidents

34
Learning Context-Sensitive Competency

• Learn models that predict likelihood that
  base-level models will fail and annotate
  real-time inferences.

BN1 BN2 . BNn
35
Sharing Context-Sensitive Competency
Traffic forecasting service

• System-wide
• status dynamics
• Incident reports
• Sporting events
• Weather
• Time of day
• Day of week
• Season
• Holiday status

36
Building Models of Surprise

• Learning predictive model of surprise
• Model of user expectation and suprise
• Models of future surprises

Expectations
Events at t

• Major events
• Weather
• Time of day
• Day of week
• Holiday status

37
Building Models of Future Surprises

• Learning predictive model of surprise
• Model of user expectation and suprise
• Models of future surprises

Events at t
Events at t-T

• System-wide
• status dynamics
• Incident reports
• Sporting events
• Weather
• Time of day
• Day of week
• Season
• Holiday status

38
Adding Surprise Modeling
Traffic forecasting service

• System-wide
• status dynamics
• Incident reports
• Sporting events
• Weather
• Time of day
• Day of week
• Season
• Holiday status

Competency annotation
39
Forecasting Surprise

• Traffic surprise in e.g., 30 minutes

40
Where will there be surprises in time t?
41
Sample Models and Exploration

• Influence of Seattle sporting events

42
Sample Models and Exploration

• Influence of weather

43
Sample Models and Exploration

• Influence of an accident at Bottleneck 15

44
Examining Details
Acc 15 ? .5 surprise -low

• Influence of accidents at B15 and B3 on anomaly
  at B4 at 30 minutes

Acc 3 ? .5 surprise -.25 low -.25 high
45
(No Transcript)
46
Alerting on Device and Desktop

• Specify preferences and download to device
• Time- and route-specific preferences
• Routes composed from bottlenecks
• Multiple alert types

47
(No Transcript)
48
(No Transcript)
49
(No Transcript)
50
Models of Surprise

• Definitions of surprise
• Qualitative and quantitative
• Abstraction about past surprises to expect
  classes of surprise
• Wild card notion in world of futurists
• Directions Statistical models of surprise

51
SurpriseCast
Assume type occurs give top m examples per type
Recommended triage of focus of attention
additional study, info. gathering, monitoring per
type
Recommended actions to react, contain, disrupt
negative suprises and/or exploit positive
surprises
Top n surprise types, by class or properties
52
(No Transcript)