User Interface Research

1
User Interface Researchat Microsoft Mary
Czerwinski
2
Research Strategy
Exploit Technology Discontinuities
Leverage Human Capabilities
Compelling Task Information Access
3
Engaging Human Abilities
communication
perceptual
motor
cognitive

• understand complexity
• new classes of tasks
• less effort

Helps User
4
Data Mountain
Strongest cue ... relative size
Subject Layout of 100 Pages
5
Motivation Task

• Large information spaces are hard to navigate
• Users develop Personal Information Spaces
• Familiarity and customization
• WWW ? Favorites/Bookmarks
• File System ? Desktop icons
• Average user

6
Motivation Technology

• Apply Information Visualization and 3D User
  Interface techniques
• Aim to leverage human abilities
• Spatial memory
• 3D Perception
• Pattern recognition
• Spatial Memory is the focus
• Does it work in Desktop 3D?

7
3D UI Hypotheses

• Advantages
• Easier to evoke spatial metaphor
• Easier to engage spatial memory
• Display more information
• Disadvantages
• Need for camera manipulation
• People get lost
• Need for high-DOF input device?

8
A Sweet Spot?

• Use 3D Visuals
• Apply 3D cues
• Occlusion, Relative Size, Shadows
• Spatialized Audio
• 2D Interaction technique
• No egocentric navigation
• Objects come to the user

9
Video
10
Whats New Here?

• Space is topologically 2D, allowing 2D
  Interaction technique
• Direct manipulation placement to tap spatial
  memory?
• Deeper Processing
• Physical Objects

11
Usability Study Design

• DM Pilot with 10 users
• 22 users in the test
• All experienced with IE4
• Half used IE, half the Data Mountain
• 100 web pages from Top50 roulette
• Phases Store, Re-organize, Retrieve
• Incorrect retrievals not discouraged
• Four cueing conditions

12
Cueing conditions
13
Pilot Results

• Pilot essential for design iteration
• Video showed post-pilot Data Mountain
• Fixes
• Prevent page loss due to occlusion
• Provide visual correspondence between title
  pop-up and thumbnail
• Spatialize audio

14
Usability Study 1 Results

• Organizing times comparable
• Reliably faster (26) retrieval with Data
  Mountain, averaged across all cues
• DM as fast as IE or faster in all conditions
• DM users had fewer incorrect retrievals
• IE users did better with Title than All
• DM users leveraged all information

15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
(No Transcript)
19
Informal Observations

• Oh, thats over there
• Layouts are very personal
• Users invent adjust categs as they go
• Categories used very similar across users
• Low cost of interrogation is important
• Titles, Thumbnails, Open Hierarchy
• Landmarks and 3D cues could be better

20
Grouping mechanism

• Users did fine
• without a grouping mechanism
• without category labels
• without multi-level hierarchy
• Users grouped anyway
• Some sub-grouping possible seen
• Some labeled with salient thumbnails

21
Open Questions

• Relative contribution of Visual Cues vs. Spatial
  Memory?
• Other ways to leverage deeper processing
• Scalability
• Manual placement means incremental use
• Light-weight deeper hierarchy?
• Multiple Data Mountains ? mental overlay?

22
Study 2

• Better determine the contribution of spatial
  location memory
• Examine longevity of memory for cued web pages in
  DM
• Track user satisfaction

23
Thumbless Data Mountain
24
Study 2 Methods

• Brought back old users after 6 months
• Reran through retrieval phase of original study
• Every other block of 10 trials, turned off the
  thumbnail images during retrieval
• DVs RT, incorrects, failures, subjective
  ratings, and cue rankings

25
(No Transcript)
26
(No Transcript)
27
(No Transcript)
28
(No Transcript)
29
Cue Ranking Results

• On average, thumbnail images ranked most helpful
  (avg. 1.8)
• Mouse-over text (avg. 2.0) next, then...
• Spatial location of the web page (avg. 2.2)
• Spatialized audio feedback ranked least helpful
  cue (avg. 4.0).
• These differences were significant as determined
  by a Kruskall-Wallis nonparametric test,
  chi-square(3) 24.02, p lt .001.

30
Subjective Measures
31
Study 3 Visualizing Implicit Queries

• 2 studies examined Implicit Queries effect
• on info management
• on info retrieval
• Visualized with Data Mountain
• 3D, spatial layout of web pages
• Pages similar to currently selected page
  highlighted

32
Implicit Queries (IQ) Performance Benefits

• IQ users (IQ1 2)
• built more categories
• took longer organizing
• faster finding pages
• Retrieval far more frequent implies IQ faster
  overall

IQ1,2 use different similarity metrics
33
Future Work

• Multi-item dragging
• Help users keep layout tidy
• Web pages live
• Continue researching deeper processing
• Computer generated spatial layouts
• Landscaped terrain--better landmarks

34
3D UI Conclusions

• Exploration of Desktop 3D sweet spot
• Prototype already outperforms existing GUI in
  effectiveness
• Usability test informs design evolution
• System low on policy lets users choose an
  organization
• Spatial memory does work in Desktop 3D

35
Camera-Based Head-Motion Parallax

• Motion parallax is one of strongest 3D depth cues

36
Glances

• Novel 3D navigation technique
• Lightweight, ephemeral
• Two-handed
• Non-dominant hand used for glance gesture
• Low-cost awareness of surrounding area

37
Toolspaces

• Objects attached to virtual body
• Glances used to get to toolspaces
• Multiple uses
• 3D widgets
• Information store
• Object transport

38
Video
39
3D UI Timetable

• 1/99 85 of new PCs ship with 3D
• 6/99 expect 100 will ship with 3D
• Turn over installed base
• 5 years if only driven by games
• 3 years if we have compelling non-game apps
• Target 2002 MS shipping 3D capable apps
• 3D Shell 3D Visualizations

40
Microsounds

• Experimentally verified heuristics for creating
  sounds for human computer interaction
• If you dont have much to say, dont take too
  long to say it.
• How short can a sound be and still be
  distinguishable and meaningful?
• The answer is lt50ms
• Sounds were distinguishable one from another
• Coded intuitive negative-neutral-positive
  distinctions

41
Microsounds

• Useful for
• Increasing user awareness of interface events and
  state without disrupting the users attention.
• Visually impaired.
• Next steps
• Experimentally test coding power of pitch
  relationships in lt80ms Microsounds
• Use Microsound techniques to sonify the state of
  Web transactions

42
Audio UI Timetable (?)

• Necessary hardware available now
• Virtualize audio devices system level mixing 6
  mo. to 1 yr
• OS events available for sonification 1 yr
• Consistent use of audio in MS apps 1 yr
• Authoring for 3rd party 2 yr

43
The TouchMouse

• Sense hand contact via capacitance
• Input modality for awareness
• New events Touch, Release
• For Palm area
• for button, wheel, thumb,
• Goal use sensors to enhance simplify UI
• Zero activation force Passive sensing
• Flexible form factor Its paint. Sense touch on
  curved surfaces, tight spaces, moving parts, ...

44
Whats a TouchMouse Good For?

• Example UI always up vs. maximum doc real estate
• Most widgets only useful if youre holding mouse
• So fade in / out portions of display via touch
• Just use mouse the same way you always have
• Test users loved it Easy just does the right
  thing

45
Sensors on WinCE Palm-sized PCs

• Touch-sensing PsPC w/ touchpad
• Help overcome limited screen real estate
• Lightweight actions for efficient access to cmds
• Make better use of both hands
• Past successes Sharing data, handedness, tilt
• Sensing enables new UIs with better awareness of
  context and thus can potentially both simplify
  enhance the users computing experience.

46
Summary

• Identify and engage human abilities
• Exploit technology discontinuities
• Result easier access to more complexity
• Result dramatic increase in user base