Title: Lecture 7: New toolkit input models: Garnet
1Lecture 7New toolkit input modelsGarnet
Amulet
- Brad Myers
- 05-830Advanced User Interface Software
2Overview
- Try to provide more support so input handling
isn't so difficult - Make easy things simple and complex things
possible - Based on the "Model-View-Controller" architecture
from Smalltalk - True separation of graphics (view) and input
handling (controller) - Also uses idea from FoleyWallace of identifying
types of input handlers - move
- grow
- rotate
- text edit
- gesture
- select (pick)
3Innovations
- Identifying primitive "Interactor" objects and
correct parameterizations so most direct
manipulation UIs can be constructed by re-using
built-in objects. - Only a few kinds of behaviors, and standard
parameters - Real separation between input and output handling
- Handles all input
- insides of widgets
- and for application programs
- First successful separation of View from
Controller in Smalltalk MVC - Integration of gestures with conventional
interaction. - Easier to code because substantial re-use
- Built-in support for multi-window dragging
4General idea
- Attach interactor objects to a set of graphical
objects to handle their input. - Graphical objects don't handle input
- No "event methods" in objects
- Instead, define invisible "Interactor" objects
and attach them to graphics - Interactors can operate on multiple objects
- Strategy pick the right type of Interactor,
attach to the objects to be moved, fill in
necessary slots of interactor - Widgets use interactors internally
- Can have multiple interactors on an object (e.g.,
different mouse buttons) - Interactors directly set slots of objects using a
standard protocol - constraints can be used to map those slots into
behaviors - Details of input events and event processing is
hidden - Used first in Garnet, refined in Amulet.
5Full Documentation
- Full Amulet Manual
- http//www-2.cs.cmu.edu/afs/cs/project/amulet/amul
et3/manual/Amulet_ManualTOC.doc.html - Tutorial
- Interactors and Command Objects
6Types of Interactors
- Am_Choice_Interactor select one or more of a
set of objects - Am_One_Shot_Interactor - single action, like
Choice - Am_Move_Grow_Interactor move or grow objects
with the mouse - Am_New_Points_Interactor to create new objects
by entering points while getting feedback "rubber
band" objects - Am_Text_Edit_Interactor mouse and keyboard edit
of text - Am_Gesture_Interactor interpret freehand gestures
7Parameters for all Interactors
- Set of objects to operate on
- To be active, Interactor must be attached to an
object which is (recursively) attached to the
screen - Equivalent to visibility of graphical objects
- Unlike graphical objects which can only be added
as parts of windows or groups, interactors can be
added as parts of any object rect.Add_Part(my_in
ter) - Default operates on the object attached to
- But also common to operate on any member of a
group. - Controlled by the Am_Start_Where_Test slot, which
should contain a method
8Example Start_Where_Test
- SP_Ship_Mover Am_Move_Grow_Interactor.Create(
"SP_Ship_Mover") .Set (Am_START_WHEN,
"LEFT_DOWN") .Set (Am_START_WHERE_TEST,
Am_Inter_In_Part)ship_group.Add_Part(SP_Ship_Mov
er)
9More standard parameters
- Multiple groups
- interactor can span multiple windows
- start, stop and abort events
- single key, mousebutton, "any" mousebutton,
modifiers, (shift, meta...), double click, click
vs. drag, etc. - active?
- priority levels
10Parameters for specific types of Interactors
- For buttons (Choice Interactors)
- how many objects to select set, toggle,
list-toggle - For move-grow
- interim feedback object (while the mouse moves)
- if missing then object itself is modified
- gridding
- move or grow
- where-attach
- center, n, ne, nw, w ... , where-hit
- flip if change sides
- minimum size
11Example
- Am_Object feedback_circle moving_circle.Create()
.Set (Am_LINE_STYLE, Am_Dashed_Line) .Set
(Am_VISIBLE, false)my_win.Add_Part
(feedback_circle)// The definition of the
interactor Am_Object objs_grower
Am_Move_Grow_Interactor.Create() .Set
(Am_START_WHERE_TEST, Am_Inter_In_Part) .Set
(Am_GROWING, true) // grow instead of move
.Set (Am_FEEDBACK_OBJECT, feedback_circle)objs_g
roup.Add_Part (objs_grower)
12Parameters for New_Point
- interim feedback object (while the mouse moves)
- gridding
- minimum size
- abort if too small
13Parameters for Text_Interactor
- editing translation table (to map keystrokes and
mouse into editing functions)
14Parameters for Gesture_Interactor
- gesture recognizer table
- If missing, can use this to get freehand drawings
15Simple Example
- To make an object movable with the mouse
- Am_Object rect Am_Rectangle.Create()
.Set(Am_LEFT, 40) .Set(Am_TOP, 50)
.Set(Am_FILL_STYLE, Am_Red) .Add_Part(Am_Move_Grow
_Interactor.Create())
16Standard Behavior
17Operation
- Interactors do 3 things
- modify objects directly
- set Am_VALUE slot of their command
- call the command's Am_DO_METHOD
- Can just have a constraint from an object or
widget to the Am_VALUE of the widget - Method to override for application-specific
operations - Fill in the UNDO_METHOD to support undo
18Operation Picture
19Choice Interactor
- Three ways to get the result
- Access the Am_INTERIM_SELECTED and Am_SELECTED
slot of the object itself - By default, sets the Am_INTERIM_SELECTED and
Am_SELECTED slots of the affected objects - Constraints that depend on these slots
- Access the Am_VALUE slot of the interactor
- Write a Am_DO_METHOD for the command object, and
access the command object's Am_VALUE slot
20Example
- Am_Define_Style_Formula (line_from_selected)
if ((bool)self.Get(Am_INTERIM_SELECTED))return
Am_Red else if ((bool)self.Get(Am_SELECTED))re
turn Am_Black else return Am_BlueAm_Object
my_prototype Am_Line.Create().Set(Am_LINE_STYL
E, line_from_selected)my_group
Am_Group.Create().Add_Part(Am_Choice_Interactor.C
reate().Set(Am_START_WHERE_TEST,
Am_Inter_In_Part)
- Now add instances of my_prototype to my_group
- Also collects a list of the selected objects in
the Am_VALUE slot of the command object in the
interactor
21Values
- Value of Command in Choice Interactor set to
object selected - Value of Command in button-type widgets set to
Label of command, or Am_ID field of the command - Remember, label can be a string or a graphical
object
22Debugging Tracing
- Can turn on tracing and get print out of which
Interactors run and what they do - Options trace all, trace only setting of slots,
trace a particular interactor, short trace (only
which interactors run), etc. - Inspector just toggles inspecting all or none
- Printout to console window
23Example
- ltgtltgtltgtltgtltgt LEFT_DOWN x180 y289
time3114329169Enter GO for ltgrow_inter_in_handl
e_185gt, state0...Checking start event against
wanted LEFT_DOWN SUCCESSChecking start
where.. SUCCESSltAm_Rectangle_650gtMove_Grow
starting over ltAm_Rectangle_650gt translated
coordinates 169,268Calculated attach point for
non-line is Am_ATTACH_SObject
ltgrow_inter_in_handle_185gt setting Am_VISIBLE of
ltSel_Rect_Feedback_197gt to trueObject
ltgrow_inter_in_handle_185gt setting
objltSel_Rect_Feedback_197gt setting
objltSel_Rect_Feedback_197gt LEFT90 TOP142
WIDTH182 HEIGHT148ltgtltgtltgtltgtltgt LEFT_UP x179
y326 time3114329838 drawonableAmulet Test
Selection Widget(0x4015b848)Enter GO for
ltgrow_inter_in_handle_185gt, state1...Checking
abort event against wanted CONTROL_g
FAILEDChecking running where..
SUCCESSltwindowgtChecking stop event against
wanted ANY_MOUSE_UP SUCCESSMove_Grow
stopping over ltAm_Rectangle_650gtObject
ltgrow_inter_in_handle_185gt setting Am_VISIBLE of
ltSel_Rect_Feedback_197gt to falseObject
ltgrow_inter_in_handle_185gt setting
objltAm_Rectangle_650gt setting objltAm_Rectangle_6
50gt LEFT79 TOP121 WIDTH182 HEIGHT185
24Advanced Feature Priorities
- If two interactors want to run, priorities used
to determine which - Am_PRIORITY slot contains a number. Default 1
- When running, 100 added to it
- Inspector interactors use 300.0
- If multiple with same priority, runs the one
attached closer to the leaf
25Example
26Advanced Feature Using Slots of Interactors
- In addition to value set into Command, a number
of slots are set into the Interactor itself,
which might be useful. - Can get the interactor as the Owner of the
command passed to the DO_METHOD - Am_START_OBJECT, Am_START_CHAR, Am_FIRST_X,
Am_FIRST_Y, Am_WINDOW, Am_CURRENT_OBJECT
27Doing something with results Command Objects
- Each Interactor and Widget has a Command object
as a part in the Am_COMMAND slot - Instead of executing a callback function,
interactors and widgets create a "Command Object"
and execute its "Do" method - Slots of command object
- DO_METHOD
- UNDO_METHOD
- REDO_METHOD (undo the undo)
- SELECTIVE_UNDO
- SELECTIVE_REPEAT and SELECTIVE_REPEAT_On_NEW
- HELP for "balloon" or status line
- LONG_HELP
- ACTIVE (enabled) -- usually contains a constraint
- VALUE, OLD_VALUE
- etc.
- Like Command Objects in MacApp, but hierarchical
- Customize by overriding default methods
28Handling Undo
- Add an undo-handler to the window
- Each command will be registered with the undo
handler - Built-in types of undo handlers
- One undo then redo, like Mac
- Infinite undo, one redo
- Selective undo mechanism
- Each DO method saves necessary information for
undo in command object
29Selective Undo
- Implementing selective undo not much harder than
regular undo - Allocates a new command object and adds to top to
history list - Semantics is based on what the user would want
- Undo the operation in a new context means to set
the object back to its previous value - Enabled if object is still available
- Undo of create is delete
- Redo the operation means to set the value of the
object again create a new object - Also, redo on new object
30Original Interface
31Commands in Widgets
- Various kinds of button panels
- Take list of items in the Am_ITEMS slot as an
Am_Value_List - Contents of list can be strings, graphical
objects, or instances of Am_Command - If commands, then the Am_LABEL field is used, and
the DO_METHOD of that command is called - If not commands, then DO_METHOD of the top-level
widget is called - For menu_bar, Am_ITEMS slot of each top-level
command contains Am_Value_List of the sub-commands
32Innovations
- Better reuse
- Many commands are in the library and are usually
used without change - Cut, Copy, Paste, Create-Object, Delete-object,
To-Top, etc. - Usually can be used without change
- Easier to customize when necessary
- Better modularization
- Hierarchical means each level only has to deal
with its own actions - New form of selective undo, repeat
- Includes undo/repeat of selections and scrolling
- Single place for enabling, help, label, etc.
- Can be computed by constraints
- Makes scripting easier
- End-User programming by demonstration
- Parameterization possible without help from
application - May help with CSCW
- Executing the command on multiple machines
33Scripting
- Added for CHI'98
- Not in standard release
- Select set of commands and specify that in a
program - Can parameterize actions
- Moving selection handles
- Forwards, backwards, left, right, up, down, in,
out - Search for object of a particular type or value
- Little or no change to application if it supports
Selective Repeat
34Pictures for Scripting, 1
35Pictures for Scripting, 2
36Pictures for Scripting, 3