How can we speak Math?

1
How can we speak Math?

• Richard Fateman
• Computer Science
• Univ. Calif, Berkeley
• The Evolution of Mathematical Communication in
  the Age of Digital Libraries
• December, 2006

2
Humans speaking math into computers

• A primary input method
• Correction of errors (secondary)
• Simultaneous with handwriting, supportive of
  ambiguity resolution, error correction

3
The reverse computers speaking math aloud

• AsTeR (T.V. Raman) for blind readers
• Display-free communication (e.g. telephone)
• Simultaneous with handwriting, supportive of
  ambiguity resolution, error correction

4
Ambiguity and Expectation

• minus b plus or minus square root of b squared
  minus 4 a c divided by 2 a .
• Without prior knowledge of this formula how could
  you know if the 4ac or even the 2a belongs within
  the square-root?
• Could it be -b ?b 2 -4ac/2a ?
• In Lisp, ambiguity is removed with one mechanism,
  parentheses/prefix
• (/ ( (- b) (sqrt (- ( b 2) ( 4 a c))) ( 2 a))
• (/ ( (- b) (-(sqrt (- ( b 2) ( 4 a c)))) ( 2
  a))

5
Digression on math input/output

• Written Math Output
• TeX, other typesetting systems
• Interactive systems with selection
• MathML, other notations
• Not entirely solved but good enough for most work
• Written Math Input
• Traditional keyboard
• Menu selection
• Handwriting
• Mixture of above

6
Speaking math into computers

• Why bother?
• Speed. Compare
• bold italic gamma ?
• Convenience Write with hand or mouse, modify
  with voice. Correction. Markup.
• Keyboard-impaired users
• Note there is a trivial non-solution. Spell out
  G-A-M-M-A

7
Pick a subset of the problem. Numbers are easy,
right?

• 1/10, 9/10/ 10/11
• 14/100, 14/10000
• 3/100, 300
• 34/100, 30/400
• 1-3, 1 2 3.
• 7-4-6, 7-4-1776
• Pick a number from 1, 2, 10 .

8
OK, we can delimit the speakers flexibility on
numbers. Then what?

• Still problems
• Homophones (sine, sin, sign)
• Near Homophones (p,b,t)
• Learning discrimination of ambiguous math
• a(bc)
• f(bc)
• Bracketing (), The quantity end quantity

9
Looks tough

• We believe it is not as tough as handwriting.
  Grammar-based tools are part of advanced speech
  recognition.
• No special hardware (maybe cheap microphone,
  already used for Skype etc)
• Combined with selection, menu, keyboarding, looks
  promising.

10
What next

• Continued development of tools and interfaces
  (Lisp, .net)
• Relying on technology from other domains (fun,
  frustration)
• Integration with Computer Algebra Systems, Math
  Browsers, Education projects, Disabled-access