Direct Manipulation: Ideas, Benefits,

1
Direct Manipulation Ideas, Benefits,
Limitations

• ??????
• ??????????
• 3D??
• ??? Edward Shen
• May 28, 2005

2
Papers

• Direct Manipulation for Comprehensible,
  Predictable, and Controllable User Interfaces,
  Ben Shneiderman, IUI 1997
• The History and Future of Direct Manipulation,
  David Fronhlich, HP Lab, 1993
• Direct Manipulation vs Software Agents, ACM
  Interactions, 1997

3
My Motivation

• With our behavior authoring system, I would like
  to introduce to you this 20-year-old design
  methodology, which has greatly influenced todays
  software interfaces.
• And with this opportunity, I wish to bring you
  some ideas about different HCI design
  philosophies

4
Outline

• Preface - Behavior Authoring v.s. Synthetic
  Animals
• Direct Manipulation Definition, Benefits,
  Implementations
• Direct Manipulation v.s. Software Agents
• Direct Manipulation From a historic point of
  view
• Conclusion - Toward a newer philosophy

5
Preface

• Behavior Authoring

6
Our Behavior Authoring System
7
More Directness

• Programming with codes/graphs

Direct Manipulaiton
8
But Indeed, the Difficulties Are

• The more power users are endowed, the more
  complex the manipulative interface would become
• Variables, conditions, iterations are hard to
  achieve
• What if the system is smarter???
• Preferences, Habits, History, etc

9
Synthetic Characters

• By Bruce Blumberg, MIT, since 1995

Blumberg, B. et al Integrated learning for
interactive synthetic characters. SIGGRAPH 2002,
pp. 417426.
10
Which is the better interface design?

• Is it more direct to specify details from a
  programming point of view?
• Or to interact with the characters in an
  interaction/conversational pattern?
• Which is the better way for users?

11
Direct Manipulation
12
Origin

• Ben Shneiderman, Direct Manipulation, A Step
  Beyond Programming Languages, 1983
• One of the most significant development of the
  1980s in HCI as a theoretical concept and
  design practice (Frohlich93)

13
Motivation

• To create environments where
• users comprehend the display
• users feel in control
• the system is predictable
• both users and the systems are willing to be
  responsible for their actions
• user-friendly -gt time spent, error
  rate/distribution, retention over time, etc

14
Definition

• Direct Manipulation is characterized by
• 1. Continuous representation of the objects and
  actions of interests
• 2. Physical actions or presses of labeled buttons
  instead of complex syntax
• 3. Rapid incremental reversible operations whose
  effect on the object of interest is immediately
  visible.

15
Physically Manipulating the Graphical
Representations

• Benefits
• Ease of learning
• Ease of use
• Retention of learning
• Reduction and ease of error correction
• Reduction of anxiety and greater system
  comprehension

16
Examples

• Video games
• Spreadsheets
• Other office systems

17
Why Exciting

• Programming Language sort database,
  statistics, editing, systems, etc

18
FilmFinder (1994)
19
FilmFinder (contd)
20
FilmFinder (contd)
21
Visible Human Explorer (1996)
22
The Direction Towards DM

• Information Visualization
• Enabling users to navigate through 4000 or more
  icons (23 orders of magnitude of current use)
• The visual presentation gives users enormous
  bandwidth and feeling of being control and
  responsible for decisions they make

23
Debate (1997)Direct Manipulation v.s.
Software Agents
24
Pattie Maes

• MIT Media Lab (software agents -gt ambient
  intelligence)
• PostDoc in MIT AI Lab
• Software agents, machine learning, collaborative
  filtering

25
Software Agents

• Difference from conventional software
• Personalized habits, preference, interests
• Proactive take the initiatives
• More Long-lived
• Adaptive

26
Why Agents?

• Personalization is needed as programs goes
  complex and users become naïve
• Computers have become a window onto a vast
  dynamic network impossible to visualize
• Users are different too
• To delegate tasks and information to
  people/assistants
• Suggestion (web browsing), reminder (email),
  matchmaking (interests, marketing)

27
Kasbah
28
Complementary, Not Alternative?

• Pattie benefiting untrained users
• Suggestions, not decisions (e.g. movies)
• Direct manipulation interface still in need
• Ben full control, achievement, responsibility
• Human-to-human interaction is NOT a good model
  for the design of user interfaces
• Adaptive systems may be annoying, irresponsible,
  unsuitable for financial, military usages

29
Direct Manipulation From a historic point of
viewby David Frohlich, 1993
30
David Frohlich

• Senior Researcher _at_ HP Lab
• In fact, the entire debate about the relative
  advantages and disadvantages of language versus
  action based interfaces turns on an attempt to
  explicate the conditions under which each is most
  direct

31
Historical thinking on direct manipulation (I)

• Hutchins, Hollan, Norman (1986)
• Directness
• Distance between users goals and actions
• Engagement of feeling oneself in full control
  over sth
• Lost of the power of abstraction
• Unseen objects
• Retrieval would be much easier with conversation
• Agents The user should be able to have a
  conversation about the world with the agent, and
  both the user and the agent should be able to
  manipulate the shared world

32
Historical thinking on direct manipulation (II)

• Laurel (1990)
• Difficult to do with first-person mode
• Retrieving, sorting, organizing, programming,
  scheduling
• Needing computers to do proactively
• Information filtering, reminding, help, tutoring

33
Historical thinking on direct manipulation (III)

• Claassen et al (1990)
• Conversation mode
• Bad for maintaining a mental model, spatial
  structures, referring to entities
• Manipulation mode
• Bad for functional, causal properties,
  abstractions, referring to invisible objects

34
Major Tech. Development since 1983

• Virtual world systems
• Virtual partner systems
• Mixed mode systems

35
Virtual World Systems

• Traced back to Sutherlands Sketchpad, 1963
• VR DataGlove, Head-Mount Display used in
  military, entertainment, medical area
• Mark Weiser

36
Virtual Partner Systems

• Knowledge based or expert systems (e.g. MYCIN)
• Natural language information retrieval systems
  (e.g. INTELLECT)
• Handwritten input devices/systems
• Computer mediated communication such as email,
  videoconferencing

37
Mixed Mode Systems

• The separation between the conversational and
  manipulative components is less clear.
• Linguistic commands on menus are invoked in
  combination with manipulative metaphors (e.g.
  cut-n-paste) in desktop office systems

38
Mixed Mode Systems
39
Summary Constraints of Direct Manipulation

• Interaction constraints
• Informing and reminding
• Responding to interrogation
• Helping and advising
• Delegation and problem solving

40
Summary Constraints of Direct Manipulation

• Task constraints
• Referring to parts of the previous interaction
• Scheduling actions to take place in the future
• Identifying unseen objects
• Identifying groups of objects
• Performing repetitive actions
• Doing more than one thing at a time
• Specifying actions very precisely

41
Todays Technologies
42
Tangible User Interface
43
Commonsense Computing
44
Conclusion

• Pure direct manipulation will eventually
  impossible to empower novices over complex tasks
• I personally agree with the mixed mode, reserve
  the control to users while provide proper
  suggestions