Visions for Mobile Devices Beyond Their Current Role

1
Visions for Mobile Devices Beyond Their Current
Role

• Brad A. Myers
• Carnegie Mellon University
• Human Computer Interaction Institute
• bam_at_cs.cmu.edu
• http//www.bam.hcii.cmu.edu/

2
Our Research on Handhelds, 1

• Pebbles project
• Since 1997
• Presented at Faculty Summit 2002, 2003, 2004
• http//www.pebbles.hcii.cmu.edu
• How handhelds can be used together with desktop
  computers
• Remote Commander
• Shortcutter
• SlideShow Commander

3
Our Research on Handhelds, 2

• Pebbles Personal Universal Controller (PUC)
• Automatically create remote control panels on
  PocketPC and Phone for appliances
• Appliance description in XML
• Interfaces that are consistent for the user
• Automatically combine interfaces for multiple
  appliances

4
Thoughts on Research Topics for the Future of
Handhelds
5
Data Integration

• Multiple antennas in all devices
• Cellphone BlueTooth WiFi
• Ability to be constantly connected
• Not just once a day for synchronization
• Can re-think the data model
• Integration of all of the users data
• Small to large pieces of data
• Cut-and-paste across devices
• Making short notes and short annotations
  (meta-notes) a fundamental part of data that is
  stored
• Intelligent and automatic consistency of data
• Never re-enter data more than once anywhere!

6
Where is the Data?

• On desktop, in the network, on body,
  synchronized everywhere?
• Storage is basically free but still have the
  synchronization problem
• Could put on network, but then loose reliability,
  worry about security/theft
• Worry if lose mobile device with data
• Make control of what is shared with others easier
  than emailing

7
Application Integration

• Portable applications across differentform
  factors
• Run same application on handheld as on PC?
• Web pages
• Word ? Pocket Word
• Automatic adaptation to I/O capabilities
• Even dynamically as application runs
• Would help with innovating in hardware since
  might just work on the new device

8
Entering Data

• Text entry still hard
• Hand-helds have voice recorders, cameras,
  styluses for strokes and handwriting
• Easier on a phone to take a picture than to type
• Rich data types input and output
• Need better ability to handle all these types as
  easily as text
• Handwriting picture typed spoken
• Anywhere can use one can use another
• Pictures as the answer to questions
• Use context, location, etc. so enter less
  information
• Intelligent interfaces to fill in field with
  right values?
• Use history, etc. to learn from user

9
Handheld and PC Augment Each Other When Nearby

• Touch screen, camera, voice recorder, phone
  modem, as PCs I/O
• PCs keyboard, mouse, big screen for handhelds
  I/O
• Opportunistic annexing of anything in the
  environment (displays, keyboards, antennas,
  networks, etc.)
• Office, conference room, home, automobile
• E.g., Use cars speakers, microphones, heads-up
  or nav-system display
• http//www.cc.gatech.edu/projects/PIE/projects/opp
  annex.shtml
• UI research issues where best for
• Control across appliances/devices, input methods
• Output displays on TV vs. front panel vs.
  handheld

10
Form Factors For Handhelds

• Fundamental human sizes
• In Hand
• Phone, PDA - smaller, pocket
• On Arm
• Tablet
• On Wrist
• Watch
• On Tables
• Clamshell Windows CE ? not successful
• New Mini-pcs
• Laptop
• Wearables
• Sensors like BodyMedia
• Tradeoffs portability, battery life, screen size
  (readability), text entry

11
Special Purpose Devices vs. Convergence

• Mobile game device, PDA, phone, pager, still
  camera, video camera, music player, movie
  player, watch, flash memory,
• Star Trek Tricorder, Communicator, Phaser,
  Medical scanner, portable memories
• Or one device that does it all?
• Marketing issues will people create content
  (games, apps, movies) for your device?
• HCI issues
• Good game control UI ? good phone control
• Larger display ? smaller device
• Battery life

12
Convergence vs. Special Purpose, cont.

• If many devices, still shouldadhere to enter
  once principle
• Scattered around environment,vs. carry around in
  pocket/on body

13
Idea Disassemble the Parts

• Each function in separate device
• Speaker in ear, microphone, display on arm,
  processor in pocket, memory, antennas
• Integrate with watch display, input
• E.g., Intel personal server
• Addresses the one v. many issue?
• Specialized I/O mechanisms (sold separately?)
  attach to common core
• Game controller/game media reader
• Camera lens

14
Other Research Areas for Handhelds

• New UIs for Handhelds
• Not just shrunk desktops
• More uses for cameras
• Handicapped access
• Digital inclusion using handhelds
• Security in general

15
Summary

• Still lots of interesting and important research
  topics in the handheld / mobile area!
• Challenge How can handheld devices improve the
  user interfaces of everything else, and not just
  be another gadget to be learned
• 6.5 billion people in the world
• 1.5 billion cell phones worldwide, vs. 500
  million PCs (?)vs. only 46 million PDAsvs. 1
  million TabletPCs

16
Visions for Mobile Devices Beyond Their Current
Role
Thank You!

• Brad A. Myers
• Carnegie Mellon University
• Human Computer Interaction Institute
• bam_at_cs.cmu.edu
• http//www.bam.hcii.cmu.edu/

17
(No Transcript)
18
-- Extras / Not Used --
19
Handhelds as Platform When Cant Afford a PC

• Less-developed countries
• How use mobile phones for what we would use a PC
  for?
• What makes sense to try to do?
• Already bought a cell phone, computation is free,
  what can be done with it?
• Computers used for
• Communication, entertainment, business stuff,
• Collaboration through handheld as a way to bring
  experts to less-developed countries
• Applications in Health care, Education,
  Agriculture, etc.
• Specialized applications ? how create apps.
  cheaper? Easier to program/customize

20
Text Entry on Handhelds Still Hard

• Lots of research, still not solved

EdgeWrite
QuikWriting
Shark
Dasher
Xnav
21
More Uses for Camera

• Scanner for text, UPC reader, sign translation
• Sensor for movement,
• Mouse replacement
• Gesture recognition
• Where am I? (augment GPS)
• Games that use camera as a sensor
• Face recognition
• Plant recognition, disease recognition,
• Type on table with camera detecting virtual keys
• Collaboration support show the objects discussed
• But needs much better integration ? still too
  hard to send/receive pictures

22
New User Interface Metaphor?

• Completely new UI metaphor that is appropriate
  for phones/mobile devices
• Maybe not just applying the 25-year-old desktop
  UI to the phone
• What could be done?
• Haptic interfaces for mobile devices
• Besides just vibrate
• More appropriate than for desktop

23
Other Research Issues

• Handicapped access
• Blind Braille handhelds?, talking
• Muscular difficulties
• Cognitive difficulties
• Security in general
• Safety of data
• Trustedness of devices
• Transmission of data