What Users Want

1
What Users Want

• Daniel Weld
• University of Washington

2
Two Interface Trends
Usage
3
Steelcase-Inspired Software
-- David Gelernter

• One Size Fits All

• Need Increased Adaptivity
• Beyond inconsistent defaulting
• Adapt to available devices, connectivity,
• Adapt to user location
• Adapt to user tasks goals
• Adapt to user calendar current time
• (Some overlap with Contextual Computing)

4
Adaptivity Customization

• Deep Deployment OS layer
• Consistent Across Applications
• Adaptation in every dialog
• Bridging Applications
• Data gathering
• Transformations

5
Outline

• Motivation
• Deliberative Software Agents
• Programming by Demonstration
• Adaptive Websites
• Adaptive User Interfaces

6
Genesis Internet Softbots
Etzioni Weld CACM 94

• Interface Principles
• Goal-Oriented
• Integrated
• Balanced
• Safe

User says what she wants Agent determines how
when to achieve it ? planning-based softbots
Single, expressive uniform interface E.g.,
printing files vs. FTPing them
DT-lite Softbot must balance the cost of
finding information on its own with the nuisance
of asking the user.
Asimovs first law Safety, tidiness
thriftiness constraints in planner
7
UW Softbot Family Tree

• Specifying goals is bottleneck
• Logical spec. language ?!_at_!
• Forms interface
• Limited to anticipated use
• Not scalable
• Often easier to just do task!
• Except data integration tasks

8
Outline

• Motivation
• Deliberative Software Agents
• Programming by Demonstration
• Adaptive Websites
• Adaptive User Interfaces

9
Programming by Demonstration
Lau Weld IUI-99
Wolfman et al. IUI-01
Lau et al. ICML-01

• If its too hard for users to specify goals
• Lets watch them
• And try to help
• Like plan recognition
• Initial domains
• Email
• Text editing
• Cross domain

10
Gentle-Slope Systems
C
ClicknCreate (Multimedia Fusion)
VBasic
Hypercard
MFC
C
C Plugins
kCmds
Difficulty of use
Hypertalk
Ideal
Basic
Sophistication of what can be created
Adapted from Myers et al.
11
Objectives
Demand on user
Expressiveness
Domain engr
Robustness
Macro Record
None
On / Off
Brittle
Straight Ln
12
SmartEdit 1
Lau et al. ICML-01
13
2
14
3
15
4
16
5
17
6
18
PBD as a learning problem

• Action is function input state output state
• Editor state text buffer, cursor position, etc.
• Actions move, select, delete, insert, cut,
  paste,
• Given a state sequence, infer actions
• Many actions may be consistent with one example
• Challenge Weak bias low sample complexity

19
Version Space Algebra

• Version space set of complex functions
• Define version space hierarchically
• Combine simpler version spaces with algebraic
  operators
• Union È analogous to set union
• Join cross product with consistency
  predicate
• Transform convert functions to different types
• Can factor a version space

20
SMARTedit's version space
21
Fast Consistency

• Test consistency of example against entire
  version space
• Quickly prune subtrees

• Innovations
• Independent join allows BSR representation

22
Preliminary User Study

• 6 undergrad CS majors
• 7 repetitive tasks with later w/out SMARTedit
• Tasks 4 to 27 iterations, 1-5 min to complete
• Evaluation metrics
• Time saved completing task with SMARTedit's help
• user actions (keyboard mouse) saved
• User feedback

23
Time saved using SMARTedit
Time (sec) Cost Saved
Six Users
X X
Task Number
24
Action Savings with SMARTedit
Six Users
Percent of Actions Cost Saved
XXXX
X X
Task Number
25
Observations from PBD

• Overhead of macro recorder UI is high
• Most repetitive tasks short
• How many shell scripts do you write / day?
• Focus on pure adaptivity
• E.g., automatic segmentation

26
Outline

• Motivation
• Deliberative Software Agents
• Programming by Demonstration
• Adaptive Websites
• Adaptive User Interfaces

27
Early Adaptation Mitchell,Maes

• Predict Email message priorities
• Meeting locations, durations

• Principle 1 Defaults minimize cost of errors
• Principle 2 Allow users to adjust thresholds

28
Adaptation in Lookout Horvitz
Adapted from Horvitz
29
Resulting Principles
Horvitz CHI-99

• Decision-Theoretic Framework
• Graceful degradation of service precision
• Use dialogs to disambiguate
• (Considering cost of user time, attention)

Adapted from Horvitz
30
Principles About Invocation

• Allow efficient invocation dismissal

• Timeouts minimize cost of prediction errors

31
Adapting to Small Screens
32
Web Site Adaptation in Proteus
Anderson et al. WWW-01

• Architecture
• Personalizing in two steps
• 1. Learn model of visitor from access logs
• Transform content per learned model
• Hill-climbing thru space of websites
• Transforms shortcuts elision
• Decision-theoretic guidance

33
Guiding the Search

• Expected utility based on model of visitor
• Model learned by mining server access logs
• Sum value of each screen of each page
• Discount by difficulty of reaching screen from p
• Depends on how manylinks followed and howmuch
  scrolling required

p
34
Proteus Empirical Study

• Observe real users on the desktop
• Info-seeking goals drawn from random distribution
• Personalize based on observations
• Measure performance on mobile device
• Number of links and scrolls, amount of time
• Compare unmodified and personalized sites
• Half users did unmodified first, others vice versa

35
Average number links followed
36
Analysis of Proteus

• Why Proteus worked well
• Suggested useful shortcuts
• Elided mostly unnecessary content
• Why Proteus worked poorly
• Sometimes elided useful content
• Users didnt find shortcut, tho it existed
• Flaws with implementation more than concept

37
Principles

• Saliency of new UI operations is crucial
• How name shortcuts?
• Eliminating features is dangerous
• Must partition dynamicity
• Maintain separate dynamic static areas
• Always allow previous navigational methods
• Duplicate functionality if necessary
• Accurate prediction also crucial

38
Partitioned Dynamism
39
Partitioned Dynamism
40
Partitioning Failure
41
Principles

• Must partition dynamicity
• Accurate prediction also crucial

42
Predicting User Behavior
Anderson et al. IJCAI-01

• Model as Sequential Process
• Markov Models
• Mixtures of Markov Models
• Second-Order
• Conditioning on Position in Trace
• Etc.

times s?d was followed
P(s?d)
Total visits to s
43
Weakness of Markov Models

• Each state is trained independently
• Abundant training data at one state cannot
  improve prediction at another state
• Large state models require vast training data

• Problematic since Web trace data is sparse
• A single visitor views 0 of any site
• New dynamic content not in training data

44
Reasoning about Uncertainty
PRM
Bayes Net
DBN
Structure
Relational
Sequence
MM
45
Relational Markov Models
Anderson et al. KDD02

• Domains often contain relational structure
• Each state is a tuple in relational DB sense
• Structure enables state generalization
• Which allows learning from sparse data

ProductPage
StockLevel
ProductName
in_stock
Apple_iMac
backorder
Palm_m505
46
Defn Markov Model
Relational

• Q set of states
• Pages in a web site

• Each state a relation
• ProductPage(Apple_iMac, in_stock)

• p init prob distribution

• A transition probability matrix

• R a set of relations

47
Domain Hierarchies

• Instance of relation
• with leaf values is a
• state, e.g.
• ProductPage(iMac, in_stock)

ProductName
48
Domain Hierarchies

• Instance of relation with
• non-leaf values is a set of
• states an abstraction, e.g.
• ProductPage(AllComputers, in_stock)

ProductName
49
E-commerce Site Markov Ver.
m505_backorder.html
main.html
checkout.html
iMac_instock.html
dell4100_instock.html
50
RMM generalization

• Want to estimate P(s ? d) but no data!
• Use shrinkage

• Can do this with abstractions of d and s
• Let ? be an abstraction of s and ? of d

?
?
?
51
Calculating Shrinkage Weights

• Intuitively, the lab should be large when
• Abstractions are more specific
• Training data is abundant
• Three methods for assigning weights
• Uniform
• Heuristic (Based on lattice depth and number of
  examples)
• EM (Data intensive)

52
Gazelle
53
Outline

• Motivation
• Deliberative Software Agents
• Programming by Demonstration
• Adaptive Websites
• Adaptive User Interfaces

54
The Google Generation

• Most WWW traces very short
• Cant beat trace 2
• Not true in desktop apps

55
Striving for Duplex
56
Still Striving for Duplex
57
Finally!
58
Confirm (Twice!)
59
State machine (partial)
Six clicks required!
60
Remember Partitioned Dynamicity

• Why Proteus worked poorly
• Users didnt find shortcut, tho it existed
• Saliency of new UI operations is crucial
• Must partition dynamicity
• Maintain separate dynamic static areas
• Duplicate functionality

61
With Controlled Adaptation
Maintain Stable Navigation
Optimize For User Behavior
62
Or Rather
Curry to Boolean
63
Future Work

• Conceptual user study
• What do users want?
• Interface description language
• Enhance Pebbles representation?
• Transformation algorithms
• Implementation experiments

64
Conclusion
High-level Customization

• Goal-oriented softbots
• Programming by demonstration
• Adaptive interfaces / websites

Pure Adaptation

• Principles
• Partitioned Dynamicity

• Techniques
• Version-Space Algebra
• Relational Markov Models

65
Acknowledgements

• Corin Anderson
• Oren Etzioni
• Pedro Domingos
• Keith Golden
• Cody Kwok
• Tessa Lau
• UW AI Group
• NSF, ONR, NASA, DARPA