CSCW Tutorial

1
Paper Digital Applicationsfor mobility and
collaboration
Ron B Yeh 14 November 2006 UNIFOR
2
Augmenting Notebooks
123
3
Augmenting Walls and Tables
123
4
Designing BuildingPaper Interactions
123
5
BackgroundResearch
123
6
Fluid Interaction with Paper
7
Interviews and Observations
8
Los Tuxtlas Tropical Rainforest
9
We also observed that

• The process of capturing data is easy, but

the data are not.
10
Clear Difference in Advantages

• Paper Notebooks Robust, ? Battery,
• Computers Search, Storage, Sharing,

11
Digitizing Pen Technology
12
Digitizing Pen Technology
13
Augmenting Notebooks
123
14
(No Transcript)
15
Notes Photos associated by time
Automatic Association
notes _at_ 443pm
photo _at_ 444pm
16
Notes Photos associated by Inked Gesture
(Hotspot Gestures)
Manual Association
17
Video 0Hotspot Linking
18
ButterflyNet Browser
19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
Video 1Browser Spreadsheet
24
Browse using the Physical Notebook
Navigate by Pen
25
Longitudinal Evaluation

• Used By
• Field Biologists at JRBP/Stanford
• NASA Researchers(Death Valley, Alaska Toolik
  Field Station)
• Anthropologists at Intel Digital Home
• Design Students at Stanford

26
Augmenting Walls and Tables
123
27
Poster-Sized Displays
The Office of the Future
28
(No Transcript)
29
Download Photographs
Photo Wall
30
Video 2Photo Wall
31
(No Transcript)
32
Video 3Network Monitoring
33
Search for Photographs
Map-Based Queries
34
Video 4Twistr Game
35
Designing BuildingPaper Interactions
123
36
Reduce, Recycle, Reuse (R3)A Paper Applications
Toolkit
http//hci.stanford.edu/paper/
37
Goals for the Toolkit

• Lower the threshold for building this class of
  pen-based paper digital user interfaces.
• Provide a high enough ceiling that talented
  designers/developers can do interesting research
  and build useful systems.

38
Pipeline for building a Paper Digital
Application

• Design and Print the Paper User Interface
• Architect the Application Logic and Connect the
  Sheets to Pens and Other Devices
• Write custom GUIs and Event Handlers

39
Designing and Printing

• Create the Paper Interface in Illustrator (or
  some other tool) and Export to PDF
• Add Active Regions through Java Code or the
  Interactive Designer
• Export to Pattern-Augmented PDF
• Print to Laser Printer or Wide-format Inkjet

40
(No Transcript)
41
(No Transcript)
42
(No Transcript)
43
Event Handlers

• Add behavior to your Paper UI by attaching code
  that will be run whenever a person clicks on,
  drags on, writes on one of your active regions.
• toBeClickedRegion.addEventHandler(new
  ClickHandler()
• public void clicked(PenEvent e)
• lt do something gt

44
Building Blocks

• Sheets, Regions, Handlers
• Devices, Pens
• Applications
• Toolkit

45
Powerpoint Advancer Example
46
Turn Pen Events into Keystrokes
public static void main(String args)
Application app new Application("Advancer")
Sheet s new Sheet(new Inches(8.5), new
Inches(11)) Region r new Region("Button", 1,
1, 4, 2) r.addEventHandler(getMarkHandler()) s
.addRegion(r) app.addSheet(s, new
File("data/Grid/App.patternInfo.xml")) app.addPe
n(new Pen("Primary Pen")) PaperToolkit toolkit
new PaperToolkit() toolkit.startApplication(ap
p)
47
Create an Application
public static void main(String args)
Application app new Application("Advancer")
Sheet s new Sheet(new Inches(8.5), new
Inches(11)) Region r new Region("Button", 1,
1, 4, 2) r.addEventHandler(getMarkHandler()) s
.addRegion(r) app.addSheet(s, new
File("data/Grid/App.patternInfo.xml")) app.addPe
n(new Pen("Primary Pen")) PaperToolkit toolkit
new PaperToolkit() toolkit.startApplication(ap
p)
48
Add one Sheet
public static void main(String args)
Application app new Application("Advancer")
Sheet s new Sheet(new Inches(8.5), new
Inches(11)) Region r new Region("Button", 1,
1, 4, 2) r.addEventHandler(getMarkHandler()) s
.addRegion(r) app.addSheet(s, new
File("data/Grid/App.patternInfo.xml")) app.addPe
n(new Pen("Primary Pen")) PaperToolkit toolkit
new PaperToolkit() toolkit.startApplication(ap
p)
49
One Patterned Region
public static void main(String args)
Application app new Application("Advancer")
Sheet s new Sheet(new Inches(8.5), new
Inches(11)) Region r new Region("Button", 1,
1, 4, 2) r.addEventHandler(getMarkHandler()) s
.addRegion(r) app.addSheet(s, new
File("data/Grid/App.patternInfo.xml")) app.addPe
n(new Pen("Primary Pen")) PaperToolkit toolkit
new PaperToolkit() toolkit.startApplication(ap
p)
50
Use the Marking Gesture Handler
public static void main(String args)
Application app new Application("Advancer")
Sheet s new Sheet(new Inches(8.5), new
Inches(11)) Region r new Region("Button", 1,
1, 4, 2) r.addEventHandler(getMarkHandler()) s
.addRegion(r) app.addSheet(s, new
File("data/Grid/App.patternInfo.xml")) app.addPe
n(new Pen("Primary Pen")) PaperToolkit toolkit
new PaperToolkit() toolkit.startApplication(ap
p)
51
Add the Region Sheet
public static void main(String args)
Application app new Application("Advancer")
Sheet s new Sheet(new Inches(8.5), new
Inches(11)) Region r new Region("Button", 1,
1, 4, 2) r.addEventHandler(getMarkHandler()) s
.addRegion(r) app.addSheet(s, new
File("data/Grid/App.patternInfo.xml")) app.addPe
n(new Pen("Primary Pen")) PaperToolkit toolkit
new PaperToolkit() toolkit.startApplication(ap
p)
52
We need at least 1 Pen
public static void main(String args)
Application app new Application("Advancer")
Sheet s new Sheet(new Inches(8.5), new
Inches(11)) Region r new Region("Button", 1,
1, 4, 2) r.addEventHandler(getMarkHandler()) s
.addRegion(r) app.addSheet(s, new
File("data/Grid/App.patternInfo.xml")) app.addPe
n(new Pen("Primary Pen")) PaperToolkit toolkit
new PaperToolkit() toolkit.startApplication(ap
p)
53
Ask the Toolkit to Start the App
public static void main(String args)
Application app new Application("Advancer")
Sheet s new Sheet(new Inches(8.5), new
Inches(11)) Region r new Region("Button", 1,
1, 4, 2) r.addEventHandler(getMarkHandler()) s
.addRegion(r) app.addSheet(s, new
File("data/Grid/App.patternInfo.xml")) app.addPe
n(new Pen("Primary Pen")) PaperToolkit toolkit
new PaperToolkit() toolkit.startApplication(ap
p)
54
Marking Gesture Handler?
private static EventHandler getMarkHandler()
return new GestureHandler() public void
handleMark(PenEvent e, GestureDirection dir)
switch (dir) default RobotAction
robot new RobotAction() robot.keyType(KeyEv
ent.VK_RIGHT) robot.invoke() break

55
The Event includes the Marks Direction
private static EventHandler getMarkHandler()
return new GestureHandler() public void
handleMark(PenEvent e, GestureDirection dir)
switch (dir) default RobotAction
robot new RobotAction() robot.keyType(KeyEv
ent.VK_RIGHT) robot.invoke() break

56
To Key Stroke
private static EventHandler getMarkHandler()
return new GestureHandler() public void
handleMark(PenEvent e, GestureDirection dir)
switch (dir) default RobotAction
robot new RobotAction() robot.keyType(KeyEv
ent.VK_RIGHT) robot.invoke() break

57
Other Directions
private static EventHandler getMarkHandler()
return new GestureHandler() public void
handleMark(PenEvent e, GestureDirection dir)
switch (dir) case NE // do
something break case E // do something
different break default RobotAction
robot new RobotAction() robot.keyType(KeyEv
ent.VK_RIGHT) robot.invoke() break

58
Swing Example
59
R3 Example
60
Video 5Other Paper Applications
61
AudioGuide Example

• We started out with an idea of making a mobile
  audio interface for biologists, and decided that
  a tourist map would be a good first step.

62
The Map
63
Map ? Adobe Illustrator
64
Add Active Regions to the PDF

• In this case, we used the R3 Acrobat Designer.
• You may alternatively choose the pure
  programmatic route, by writing Java code to
  define the active regions.

public class AudioGuide extends Application lt
lots of stuff gt
65
Add Active Regions to the PDF
public class AudioGuide extends Application
public static void main(String args)
AudioGuide guide new AudioGuide() PaperTo
olkit p new PaperToolkit() p.useApplicationMa
nager(true) p.loadApplication(guide) lt
lots of stuff gt
66
Add Active Regions to the PDF
public class AudioGuide extends Application
public static void main(String args)
AudioGuide guide new AudioGuide() PaperTo
olkit p new PaperToolkit() p.useApplicationMa
nager(true) p.loadApplication(guide) lt
lots of stuff gt
67
(No Transcript)
68
Conclusions

• Audience for the R3 Paper Toolkit isHCI
  Practitioners like yourself!
• Open Source Software
• Can evolve based on your feedback

http//hci.stanford.edu/paper
69
Questions???
http//graphics.stanford.edu/ronyeh
ronyeh_at_cs.stanford.edu