Butterfly Effect: An Augmented Reality Puzzle Game

1
Butterfly EffectAn Augmented Reality Puzzle
Game

• Marleigh Norton
• Blair MacIntyre
• Steven DowMaribeth Gandy

2
Game Overview

• Spatial puzzle where physical world is not
  modeled
• Virtual butterflies in physical environment
• Catch them!
• Butterflies are stationary
• Challenge comes from reaching the butterflies
• Bring butterflies into reach by rotating the
  virtual world
• Virtual world rotates in 90 degree chunks about a
  player defined axis.

3
Equipment
4
Head-Mounted Display
5
Tornado Stick
6
Rotations

• Tornado Stick controls Virtual Axis
• First Button Place / Remove Axis
• Second Button Pause / Unpause Axis
• Third Button Rotate Butterflies

7
Tornado Stick

• People had trouble understanding rotations
• Not everyone is used to 3D geometry
• Though 3D puzzles have worked before
• Change the metaphor, not the interaction
• Tornado Stick

8
Butterfly Effect
9
Butterfly Effect Video
10
Design Context

• Start with medium, then design game
• Mapping physical and virtual worlds is hard
• AR works best when user is moving slowly
• Tracking more accurate
• Safer for player Cognitive Tunneling

11
Design Context

• Home environment
• Run on next generation game consoles
• Must be easy to set up
• Minimize use of special equipment

12
1st Prototype 2D Paper
13
2nd Prototype 2D Interactive
14
3rd Prototype Video
15
Tuning

• Two Major Challenges
• 90 degree rotation constraint
• Understanding virtual space and physical space
• Possible Changes
• Loosening rotation constraint
• Adding more manipulations (translations)
• Adding better depth perception cues

16
Open Issues Depth Perception

• Static Depth Cues

17
Open Issues Depth Perception

• Remove Automatic Depth Cues

18
Open Issues Depth Perception

• Remove Unavailable Depth Cues

19
Open Issues Depth Perception

• Texture probably the best

20
Without Grid
21
With Grid
22
Is Occlusion Really Impossible?

• Ad hoc modeling
• Automatic modeling during calibration
• Fiducials
• Real time depth
• Depth from stereo
• Something to be explored

23
Virtual Obstacles - Bees

• No consequences for using poor strategy
• Bees behave like butterflies
• Sting if caught
• Penalty
• Lose life
• Butterflies escape

24
Questions?
25
Old Slides from Past Talks

• (In case people as a question I have slides for.)

26
Butterfly EffectAn Augmented Reality Puzzle
Game

• Marleigh Norton
• Technical Advisor Blair MacIntyre

27
Person Walks Through Game Area
28
Person Walks Through Game Area
29
Person Walks Through Game Area
30
Derive Walls from Path
31
Derive Walls from Path
32
Why this is Fun

• Fun in the same ways Rubiks Cubes are fun
• Cailloiss Four Categories of Games 1
• agôn, alea, mimicry, ilinx
• MacTavishs astonishment at visual and auditory
  technology 2

33
Open Issues

• Scoring
• Capture Interaction
• Depth Perception

34
Open Issues Scoring

• Score based on efficiency, not speed
• Penalized for moving too much
• Accrue score as you walk
• High score bad
• Problem High Score Traditionally Good

35
Open Issues Capture Interaction

• Location-Based Capture
• Capture butterfly by getting close enough
• Pros Technically simple. No additional cost.
• Cons Precise head tracking more important. Less
  thematically appropriate. Less fun.
• Butterfly Net Capture
• Capture butterfly in a net
• Pros More active, thematically appropriate
  interaction.
• Cons Requires an additional tracking system,
  likely expensive.

36
Open Issues Depth Perception

• Judging depth in virtual worlds is difficult
• Overcoming this difficulty is part of the game,
  yet it should not be impossible
• Problem What can we do to help the player
  understand the relative locations of the
  butterflies?
• Research on Depth Perception yields several
  techniques 4 5 6 7

37
Questions Feedback

• Scoring Efficiency
• Capture Interaction
• Location-based capture
• Butterfly Net Capture
• Depth Perception
• Which cues to use? More ideas?
• Other Feedback

38
References Game Theory

• 1 Caillois, Roger, Man, Play, and Games,
  trans. Meyer Barash, University of Illiniois
  Press, 1961 (orig.1958) ch 2.
• 2 Mactavish, Andrew, Technological Pleasure
  The Performance and Narrative of Technology in
  Half-Life and other High-Tech Computer Games,
  in Geoff King and Tanja Krzywinska (eds.)
  Screenplay Cinema/ Videogames/ Interfaces
  (London Wallflower Press, 2002) 33-50

39
References Depth Perception 1

• 4 Cutting, J. E., and P.M. Vishton, 1995.
  Perceiving layout and knowing distance The
  integration, relative potency and contextual use
  of different information about depth, in
  Perception of Space and Motion, W. Epstein and S.
  Rogers, Eds. Academic Press, New York, 69--117.
• 5 Furmanski, Chris, Ronald Azuma, and Mike
  Daily. Augmented-reality visualizations guided
  by cognition Perceptual heuristics for combining
  visible and obscured information, in Proc. IEEE
  and ACM Int'l Symp. on Mixed and Augmented
  Reality (ISMAR 2002) (Darmstadt, Germany, 30
  Sept. - 1 Oct. 2002), pp. 215-224.

40
References Depth Perception 2

• 6 Sinai, M.J., W. K. Krebs, R. P. Darken, J. H.
  Rowland, and J. S. McCarley, 1999. Egocentric
  distance perception in a virtual environment
  using a perceptual matching task, in Proceedings
  of the 43 rd Annual Meeting Human Factors and
  Ergonomics Society, 43, 1256-1260.
• 7 Surdick, R. T., E. T. Davis, R. A. King, and
  L. F. Hodges, 1997. The perception of distance
  in simulated visual displays A comparison of the
  effectiveness and accuracy of multiple depth cues
  across viewing distances, in Presence
  Teleoperators and Virtual Environments 6, 5
  (October), 513--531.

41
References DART

• 8 MacIntyre, Blair, Maribeth Gandy, Steven Dow,
  and Jay David Bolter. "DART A Toolkit for Rapid
  Design Exploration of Augmented Reality
  Experiences." To appear at conference on User
  Interface Software and Technology (UIST'04),
  October 24-27, 2004, Sante Fe, New Mexico.

42
Implementation Status

• Its playable!
• Populate the world with a pre-defined set of
  butterflies
• Butterflies can be captured
• Tallies of caught and remaining butterflies are
  displayed
• Butterflies can be rotated about the axis
• Physical axis interface

When all of the equipment is working.
43
The Screen
44
Feedback

• Rotation Interface
• Depth Perception
• Score
• Aesthetics

45
Rotation Interface

• Problem Rotating about an axis? Huh?
• Suggested alternatives
• Randomization
• Jar, Mini-version of gamespace
• (Rotation about a point)
• Tornado Stick
• Changes Tornado Stick
• Axis/Tornado Stick only interface in which all
  manipulations are valid moves
• Arrows pointing to butterflies final location

46
Depth Perception

• Problem Its hard to tell where the butterflies
  are, especially when theyre behind walls
• Complication cant change appearance of
  butterfly based on presence of wall(or can we?)
• Changes Add grid with drop shadows
• Also, its less of a problem when youre actually
  playing

47
Score

• Problem for Efficiency-based score, lower was
  better (confusing players)
• Suggestions
• Timer
• Subtract from High initial score
• No Score
• Changes No Score
• Score isnt really that important
• Was causing more problems than it solved

48
Aesthetics

• Problem What will the game look like?
• Collecting the butterflies should be compelling
• Changes Aesthetics more fully fleshed out
• Childlike graphic design with scanned crayon
  textures
• Player is frog, eating butterflies

49
Frog has Butterfly in Sights
50
Frog Catches Butterfly
51
Frog Eats Butterfly
52
Future Work Finish Implementation

• Sounds
• Graphics
• Animations
• Refine Stick
• Are those the right buttons?
• Buttons or Switches or ???
• InertiaCube Issues with Ferris Environments

53
Questions?

• Thanks to the Augmented Environments Lab,
  especially
• Blair MacIntyre
• Steven Dow
• Enylton Machado Coelho