Challenges in MultiModal and ContextAware UI

1
Challenges in Multi-Modal and Context-Aware UI

• (the 5 minute version)

2
Multi-Modal

• Im using multiple output media (audio, visual,
  gestural, stance, posture, eye gaze, etc.).
• Challenge apply this same fluidity, range, and
  artful redundancy/overlap to computer input
  techniques.

3
Context-Aware

• Where am I?
• What room?
• What noise level?
• What lighting level?
• What elevation?

4
Context-aware (2)

• Who am I?
• Biometrics
• best tailoring extreme interfaces
• Who else is nearby?

5
Context-aware (3)

• What is around me?
• Other devices (IR/RF communication networks)
• true plug-n-play (Handspring).

6
Context-aware (4)

• How do you know what I mean?
• Clutching problem
• How far back does context apply?
• Affects all the other issues (what, when, etc.)

7
2 modest proposals

• get 20 itsys (or similar technology), embrace
  and extend them.
• Lego Mindstorms
• Study sign language

8
Hmmm.

• When two fluent speakers of North American Indian
  Sign Language communicate
• 60 - redundant
• 30 - augment
• 10 - disjoint!