Fun Meters for Games

1
Fun Meters for Games

• Nicole Lazzaro, President
• XEODesign
• Larry Mellon, System Architect
• Emergent Game Technologies

AGC Oct 28, 2005
2

• Measure
• Fun

3
Pulled Pork Sandwiches Yum! (But Why?)

• Structured observations of the people
• - Smiling? Savoring?
• - Structured questionnaires

?
Insertion of probes into the sandwich -
Temperature? Sauce balance? - Sandwich mods?
4

• Fun meter requirements
• Multiple data sources
• Repeatable validity
• Game player variance

• What could you use it for?
• Pre-ship evaluation
• Early evaluation of game play

5
Nicole Lazzaro
Larry Mellon

• President and Founder XEODesign
• 15 years of design and research player
  experiences
• Sony, EA, LeapFrog, Sega, Ubisoft, PlayFirst
• Cognitive Psychology Stanford
• Recent Play Tombstone Hold em

• CTO Emergent Game Technologies
• 20 years of tools and engines for distributed
  systems supercomputers
• Jade, EA, DARPA,
• Parallel simulation virtual worlds
• Automated testing metrics in games
• Recent Play Go

6
Next-gen I get the graphics, but wheres the
game?
Computational power
Visual effects
Innovative game play
7
Blocking IssueIn next-gen games, how much
innovation will be fundable?
Cost Risk
Innovation
8

• Next Gen Games
• Increased Complexity
• Increased Complexity of Analysis

9
(No Transcript)
10
Next Gen Games
11

• Mobile
• Off-Computer Play

12
Next-gen gambling roll dem bones!
Success
Failure
13
Blocking IssueBig games are hard to change at
the last minute!
Project Start
Gold
14
2. Fun Meters Today

• We can measure aspects that contribute to fun
  today

15
Probes showed us how players interacted with each
other socialization more important to players
than dancing
16
House Categories (Beta Test)
The house categories mini-game just wasnt
rewarding enough to players a game dead spot
.

17
Fun Meter

• "Where is the fun in that?" Bruce Shelly

18
Knowing in Time
19
The 4 Keys The Ride
4 Flavors of Fun
Easy Fun Filling Attention
Hard Fun Challenge Mastery
Altered States Change Internal Sensations
The People Factor Social Mechanism
www.xeodesign.com/whyweplaygames
20
The 4 Keys The Ride
4 Flavors of Fun
Easy Fun Filling Attention Surprise Curiosity
Hard Fun Challenge Mastery Frustration Fiero
Altered States Change Internal Sensations Excite
Relax
The People Factor Social Mechanism Shadenfreude
Naches
www.xeodesign.com/whyweplaygames
21
Hard Fun - Fiero

• Personal Triumph Over Adversity

22
My Fun Meter Read Out
23
Some Players
24
Other Players
25
LeapFrogs FLY Pentop
Player Prototypes and Field Studies
26
Section 3. Audience participation(5 min)
What would a unified fun meter have to measure in
your game?
27
Fun Meter

• "Pace is everything.  User attention is the
  game." Mark Terrano

28
3. Fun Meter Challenge

• More than 20 aspects to measure fun
• Larry Nicole have a bet
• Left and Right sides compete
• Winner gets a duck!
• List posted w/slides
• 260 ideas/hour 4.3/min (21.5 / 5 mins)

29
What to Measure

• Pace
• Exploits
• Rewards
• Power Ups
• Score Jumps

30
Fun Meter Challenge

• Take Away
• Lots of Aspects Effect Fun
• Each Needs a Meter

31
4. Sources of Data

• Every Meter Needs a Data Source
• Data Relates to the Question

32
Section 4. Data extraction is always tricky
Indirect (asking players)
Direct (observing players)
33
Fun Meters

• Are people around the company  (e.g.
  non-testers) playing? Steve Meretzky

34
Indirect Measures

• Gathering Opinions
• Co-Worker
• Focus Group
• Survey
• Beta/QA Testing

• Lots of noise
• Cheap to implement
• Requires player to interpret events

35
Internal
Indirect Measures
36
Internal
Indirect Measures
37
Beta/QA Testing

• Strengths
• Many People Playing Game
• Bugs and Exploits
• Easy to Set Up
• Forgive Artifacts
• Fill in Whats Missing
• Rapid Iteration

• Limits
• Professional TestersBug Focus vs. Fun
• Player Filters Data
• Not Target Market
• Seen Earlier Versions
• Group Think

38
The Vocal Minority

• Type of play they expect?

Silent Majority
Love It!
Hate It!

• On Line Surveys
• Post Play Surveys

• Beta/QA Testing
• Forums/Chat
• Focus Groups

39
Focus Group

• Strengths
• Brainstorming
• Packaging
• Generating Ideas

• Limits
• Ø Behavior
• Group Think
• Opinion Leader
• Fibbing
• Post Play
• Watch not Play

40
Meta Critic

• Automated
• Opinions of Play

• Limits
• Reviews Weighted
• Comes too Late

41
Online Surveys

• Strengths
• 35K/5yrs
• How Many Play w/Their Partner?

• Limits
• Ø Behavior
• Fibbing
• Question Bias

Daedalus Project, Nick Yee
42
On Line Surveys

• Strengths
• Nifty Charts
• Measure Existing Ideas with Accuracy
• Automated
• Low Cost per Respondent

• Limits
• Self Select
• Ø Behavior
• Fibbing
• Mostly Quantitative
• Find What Ask For
• Bias in Questions

43
Fun Meter

• We didn't have the techniques then that you are
  creating now to measure where the person was, and
  instead looked at progress vs. time or survey
  results after playing to try to determine where
  the person was at. Jeff Pobst

44
Direct Measures

• Observation
• Data Probes

• Behavior Based Data
• Requires Expert Interpreter
• Tell us things that the user cant tell us.

45
Observe Player

• XEOAnalysis
• Usability
• Un-fun
• Player Experiences
• Emotions Drivers
• 4 Flavors of Fun

46
SWG Fun Btn

Usability Remove Un-Fun (51UI)
47
Frustration w/o Fiero

• Fiero flowchart w/hard and easy fun

48
Observe Player Experience

• Strengths
• Behavior Data
• Directly Observe Player, Game, Context
• What and Why
• Explore New Ideas
• Less Up-Front Bias
• Paper or Electronic

• Limits
• More Effort to Collect Data and Analyze
• Time
• Automation
• Trained Observer

49
Context - Last Call Poker
50
Observe Players Early

• Paper Fun Meters
• New Roles for NPCs
• Wire Frame UIs
• Paper Prototype
• Play Test the Fun
• Automate Data Collection w/ Consumables or Tokens

51
Fun Meter

• There's a core rule, a way to interact, and then
  tuning, an adjustment to see if it is gripping
  and attractive or is it a drag on the
  experience? - Hal Barwood

52
Tuning imbalances can throw your entire economy
out of kilter (killing fun), but remember to
triangulate!
53
Economy Detailed View
More tuning imbalances visitor bonus did not
provide enough incentive to attract other players
to your home, lowering in-game social connections
54
Play styles vary wildly across player types
hardcore players will go to great lengths for a
minor cash advantage
55
Looking at what people spend their in-game money
on gives you clues into what entertains them
.
4 of top 5 windows??
56
Correlated data is much more valuable than
single-source data
Click rate
Time
Screenshots!
57
Measurement techniquesprobes embedded in the
game

• Strengths
• Concrete, accurate metrics
• Easy, fast, automated
• Limits
• Does not directly measure fun at all, only things
  that indirectly impact fun
• Limited insight into player motivations
• Best uses
• Hot spot / dead spot analysis, tuning feedback
• Understanding the true actions of your players

58
Future Fun Meters

• New Data Sources
• Biometrics
• Eye Tracking
• GSR
• Facial Tracking
• FMRI

Vittorio Gallese, 2004
59
Section 5 Conclusion (5 min)

• Section Outline
• Spectacle alone will not sustain next-gen gaming
  breakout games are based on new forms of
  gameplay, not just better graphics
• visual doorways hit titles the next gen
  door ? how are we going to get there?
• visual repeat iceberg/potOfGold measured
  iteration, early in the design cycle is the way
  to go
• Call to action the industry as a whole needs to
  be going this route of measured innovation
• were looking for partners in trying this theory
  exploring new types of fun meters

60
ConclusionsWhy do we care about fun meters?

• Next-gen spectacle alone cannot sustain us
• talking points
• Its an exciting time for the game industry
  radical new capabilities in the Internet and
  high-definition and a chance to push past movies
  TV. But were not going to get there with Doom
  I vs Doom XXV (same gameplay, just better
  visuals) you become habituated lose the impact,
  less impact after X releases,
• Will the power of all this take us past movies?
• And the cost will kill us who can afford 80
  artists per title? Not the innovative games
• Breakout games will need innovative forms of fun
• talking points for next slide
• If we can iteratively measure improve it, we
  can take risks, with great rewards
• Gameplay is more valuable than pixels new
  gameplay will broaden our audience base how do
  we get there?
• The Sims was cancelled multiple times who would
  play that?
• How many more versions of Harry Potter or Madden
  do we need?
• Innovation leads to more breakout hits

61
Conclusions
62
Knowing When

• Player Testing Reviews

• Data Probes

• Observation and Prototyping Play

63
We need more than better visuals in next-gen games

• Visual impact eventually tapers off
• And who can afford it?

64
Breakout games games that establish a new genre
have innovative gameplay

• Populous
• Pac Man
• Doom
• The Sims
• Habitat
• Grand Theft Auto
• Katamari Damacy

?
What new types of gameplay will open up new
genres in next-gen?
65
Measured innovation, early in the design cycle,
is a way to break out of the box
Complexity
Scale
Cost
Pac man visual
66
Fun meters of the future unite!
67
QA

• Slides are on the web
• www.maggotranch.com/MMP
• www.xeodesign.com/funmeter
• Our contact data
• nicole_at_xeodesign.com
• larry_at_maggotranch.com