Human Computation

1
Human Computation
Steven Emory CS 575 Human Issues in Computing
2
Overview

• What is Human Computation?
• History of Human Computation
• Examples of Human Computation
• Challenges in Human Computation

3
Definition of Computation

• Normally we rely on computers to do all the work
• On input
• Step 1 (computer)
• Step 2 (computer)
• ... (computer)
• Step n (computer)
• Ouput

4
Definition of Human Computation

• Some or all steps solved by human(s)?
• Roles reversed Computer asks us to solve a
  problem
• On input
• Step 1
• Step 2 (ask a human)?
• ...
• Step n
• Output

5
History of Human Computation

• 1980's Interactive Genetic Algorithms
• 2000's Human-Based Genetic Algorithms
• 2000's Outsourced Human Spam
• 2000's Interactive Guessing Games

6
Notes on Human Computation

• While Interactive and Human-Based GAs can employ
  the use of a single person, the term Human
  Computation is typically only used when many
  humans are involved
• Difficult to get a lot of data out of just one
  person
• Results are statistically better when many people
  agree (if it's just one person are you sure he's
  telling the truth?)?
• Humans are typically networked (not
  communicating, collaborating, or cheating)?

7
Quicksort Example

• Bad Example Quicksort
• Input Unsorted array
• Select a pivot (human selection)?
• Swap last element with pivot element
• Partition array using pivot element
• Insert pivot element into correct position
• Repeat above steps for left/right side partitions
• Output Sorted array

8
Quicksort Problems

• Solved faster by computer alone
• Not rewarding
• Boring/painful
• Wouldn't do this even if I was paid to do it
• Using first row of students, would everyone pick
  the same pivots?

9
Photomosaic Demo

• To demonstrate that humans are better at some
  things than computers, consider a photomosaic
• Can be solved by computer alone.
• On input image gallery, source image1.) Tile
  source image.2.) From left-to-right,
  top-to-bottom, compare each image in the image
  gallery to each tile in the source image,
  inserting the gallery image with the lowest
  error.3.) Output photomosaic.

10
Photomosaic Problems

• Noticeable visual artifacts (no image reuse)?
• Could use a randomized algorithm instead
• Could optimize important features first
• However, no general select important features
  algorithm exists
• Must ask a human (the user) to select important
  features

11
Photomosaic Solutions

• Revised algorithm
• On input image gallery, source image1.) Tile
  source image.2.) Select important features (ask
  human).3.) Optimize (randomly) important
  features.4.) Optimize (randomly) unimportant
  features.5.) Output photomosaic.
• Unlike the Quicksort example
• this is rewarding and fun

12
When to Use

• When the problem
• is hard for a computer, but easy for a human
• is not boring to humans (music, art, games)?
• is rewarding (financially or emotionally)?

13
Human Computation Challenges

• User interface design
• Coordinating many human participants
• Analogous to distributed computing
• Honesty
• Prolonged computation
• Maintaining human interest and motivation

14
reCAPTCHA Example

• Used to digitize old books (make e-books)?
• OCR normally works 99 of the time
• OCR accuracy drops for older books
• Old paper
• Old printing techniques
• Solution Ask humans to determine words OCR fails
  to classify
• When enough humans agree, consider it solved

15
reCAPTCHA Example
16
Metadata Example

• Algorithm Assign metadata to images
• Useful for content/multimedia management systems
  (i.e. Istockphoto)?
• No algorithm exists for image labeling
• Luis Von Ahn's solution The ESP
  Gamehttp//www.espgame.org
• ESP Game Demo

17
Metadata Example

• Problem
• Getting humans to agree correctly

18
Electric Sheep Example

• Brief fractal explanation
• Iterative process based on chaos, dynamical
  systems
• Newton's Method FractalSolve az3 bz2 cz d
  0 for complex numbersa, b, c, d are fractal
  parameters
• Cubic equation has 3 rootsRed converges to
  root 1Green converges to root 2Blue
  converges to root 3Black fails to converge

19
Electric Sheep Example
20
Electric Sheep Example

• Animated fractal screensaver application
• Also an interactive genetic algorithm
• Humans (users of screensaver) evaluate fitness
• Animated fractal parameters are mutated
• Algorithm has been running for years
• http//www.electricsheep.org/

21
Electric Sheep Example
22
Outsourced Human Spam

• On input http//acm.calstatela.edu/forum
• Find Register link (spam bot)?
• Download registration form (spam bot)?
• Fill out registration form (spam bot)?
• Find CAPTCHA (spam bot)?
• Solve CAPTCHA (human)?
• Send registration form (spam bot)?
• Output SPAM!!!

23
Conclusions

• There many hard/impossible to solve problems
• Nothing shameful about using Human Computation
• Applications in art, music, computer vision,
  security, content management

24
References

• Luis Von Ahn
• Google Talk lecture on Human Computation
• http//www.cs.cmu.edu/biglou/
• http//www.espgame.org
• The Art of Artificial Evolution (2008)?
• http//www.springerlink.com/content/r68831/?pd04c
  460aa17749eb8153fec3d0507f68pi9