Jeffrey Nichols and Brad A. Myers

1
Personal Universal ControllersControlling
Complex Appliances with GUIs and Speech
CHI 2003 Conference on Human Factors in
Computing Systems Ft. Lauderdale, Florida

• Jeffrey Nichols and Brad A. Myers
• Carnegie Mellon University
• April 8, 2003

2
The Problem

• Appliances are too complex

3
The Problem, cont.

• Each complex appliance has its own idiosyncratic
  interface!
• Home and Car Stereos
• VCRs Camcorders
• Car Navigation Systems
• Answering Machines
• Increasingly Computerized
• Low Usability

4
Our Solution

• Separate the interface from the appliance!
• Handheld becomes personal universal controller
  (PUC)
• Key Features
• User interface-independent appliance
  specification
• Automatic generation of GUI and speech interfaces

5
Automatic Generation of UIs

• Benefits
• All interfaces consistent for the user
• With conventions of handheld
• Other applications and UI guidelines
• Even from multiple manufacturers
• Addresses idiosyncracy problem!
• Multiple modalities (GUI Speech UI)
• Can take into account user preferences
• Will work on special purpose devices (for
  disabled)

6
Overview

• Appliances

• Controller Devices

7
Demonstration 1

• Sony Camcorder
• PocketPC, Smartphone, Speech, Tablet PC

8
Architecture
- Appliance Adaptors
- Comm. Protocol
- Interface Generators
XML-based
9
XML-based Specification Language

• Describes appliance with these features
• Functions of Device
• State Variables and Commands
• Labeling
• Multiple labels are necessary
• Grouping
• Hierarchical groups
• Dependency Information
• For enabling and structure

10
Demonstration 2

• Windows Media Player
• PocketPC, Smartphone

11
Demonstration 3

• GMC Information System / Climate Control
• PocketPC

12
Important Work By Others

• INCITS V2 Standardization Effort
• SIG at CHI 2003
• Toward a Unified Universal Remote Console
  Standard
• Thursday 900-1030, in Room 122
• Xweb (now ICE)
• Olsen Jr., UIST 2000
• Stanford iRoom, iCrafter
• Ponnekanti, Ubicomp 2001
• Speakeasy
• Newman, UIST 2002

13
Future Work

• Improve quality of generated user interfaces
• More and better rules
• Specification and generation of more appliances
• answering machine, navigation system
• Consistent interface generation
• New interfaces modeled on past interfaces
• Generating interfaces for the experience
• One interface generated for multiple connected
  appliances
• General purpose appliance adaptor for HAVi and
  UPnP protocols

14
Acknowledgements

• Funding
• National Science Foundation
• Microsoft
• General Motors
• Pittsburgh Digital Greenhouse
• Equipment Grants
• Mitsubishi (MERL)
• VividLogic
• Symbol Technologies
• Hewlett-Packard
• Lucent

• PUC Project Members
• Brad A. Myers
• Thomas K. Harris
• Roni Rosenfeld
• Michael Higgins
• Joseph Hughes
• Kevin Litwack
• Rajesh Seenichamy
• Mathilde Pignol
• Stefanie Shriver
• Jeffrey Stylos
• Peter Lucas

15
Thanks!
http//www.cs.cmu.edu/pebbles/puc/ http//www.cs.
cmu.edu/jeffreyn/

• Conference on Human Factors in Computing Systems
  (CHI)
• April 8, 2003

16
(No Transcript)
17
Rise of Mobile Devices

• Mobile devices are becoming cheaper and more
  common

18
Demonstration 4?

• Two-way X10 Lighting
• PocketPC, Smartphone

19
X10 Demo 1
20
X10 Demo 2
Back
21
DV Camera 1
22
DV Camera 2
Back
23
Windows Media Player
Back
24
GM Driver Information System
Back
25
GM Climate Control System
Back