The Effects of Loss and Latency on User Performance in Unreal Tournament 2003

1
The Effects of Loss and Latency on User
Performance in Unreal Tournament 2003

• Tom Beigbeder, Rory Coughlan, Corey Lusher, John
  Plunkett, Emmanuel Agu, Mark Claypool
• Computer Science Department
• Worcester Polytechnic Institute
• Worcester, MA, USA
• http//www.cs.wpi.edu/claypool/papers/ut2003/

2
General Motivation

• In 2003, despite economic downturn, games only
  entertainment industry to grow Essential
  Facts, ESA - 2004
• For frequent game players, 43 play online ESA -
  2004
• First-Person Shooters a top selling genre
• 11.5 Essential Facts, ESA, 2003
• Unreal Tournament 2003
• Popular (1700 servers, 4400 players)
  Gamespy Oct 2003
• Award winning Best of Show, E3 expo
• Studied less than Quake-derived FPS

3
Specific Motivation The Network and Network
Games

• Network games can be demanding in terms of
  network requirements
• Capacity? (Not usually)
• Latency? (Sometimes)
• What parts of a FPS game Moving? Shooting?
• Quantitative effect?
• Loss? (Unknown)
• Retransmission will increase latency
• Knowing game constraints useful for
• Building better network games
• Building better networks to support games (QoS)
• Our goal
• ? Quantitative effects of loss and latency on the
  performance of UT2003

4
Outline

• Introduction ?
• Experiments ?
• Analysis
• Conclusions

5
Methodology

• Characterize fundamental user interaction
  components of FPS games
• Design maps that exercise each type of
  interaction
• Construct test environment for measuring the
  effects of latency and loss
• Conduct user studies
• Analyze results

6
FPS User Interaction Components

• Movement
• Simple (Moving in a straight line)
• Complex (Running, jumping, turning)
• Shooting
• Precision Example Weapons
• High Shock Rifle, Link Gun, Lightning Gun
• Medium Assault Rifle, Mini-Gun, Bio Rifle
• Low Flak Cannon, Rocket Launcher
• Movement and Shooting
• Design maps for each component

7
Controlling Loss and Latency

• Ethereal for traces
• UT2003 is Client-Server, uses UDP, Server updates
  every 50 ms
• NIST Net for Router ? Control network
  characteristics to each client
• Control loss 0, 6
• Control latency 0 ms, 400 ms

8
Outline

• Introduction ?
• Experiments ?
• Analysis
• Application Level ?
• Network Level
• User Level
• Conclusions

9
Complex Movement

• Flux2 map with pre-defined obstacle course

• Movement not affected by loss or latency

10
Precision Shooting

• CTF-Face3 map with lightning gun versus dodging
  human opponent

• Precision shooting not affected by loss
• Precision shooting is affected by latency

11
Restricted Deathmatch

• Training Day map with only medium precision
  weapons versus bot

• Movement and shooting not affected by loss
• Movement and shooting affected by latency
• Less significantly than shooting alone

12
Full Deathmatch

• Training Day map with full precision weapons
  versus Bot

• Movement and shooting not affected by loss
• Movement and shooting barely affected by latency

13
Outline

• Introduction ?
• Experiments ?
• Analysis
• Application Level ?
• Network Level ?
• User Level
• Conclusions

14
Network Turbulence
5

• No visible effects of loss or latency
• ?Holds for packet sizes, inter packet times

15
Outline

• Introduction ?
• Experiments ?
• Analysis
• Application Level ?
• Network Level ?
• User Level ?
• Conclusions

16
User-Level Analysis

• 75 ms latency, users noticed sluggishness
• Felt play was worse
• True even for unrestricted tests
• 100 ms latency always noticeable
• Play less enjoyable
• Frustrating for precise shooting
• 0-3 loss rarely even noticed
• 3 loss sometimes noticed, but only because gun
  effects not always displayed

17
Conclusions

• FPS games not affected by typical Internet packet
  loss
• Loss up to 3 and not even noticed
• TCP fairness at higher bitrates?
• FPS games can be affected by latency
• Perceived impact as low as 75 ms
• Quantitative impact at 100 ms
• At the network level
• Small packets and low bitrates
• Turbulence consistent over all network conditions

18
Future Work

• Effects of variance in latency (jitter)
• User adaptation strategy
• Build model for FPS interactions so can apply to
  different FPS games (i.e. BF1942)

19
The Effects of Loss and Latency on User
Performance in Unreal Tournament 2003

• Tom Beigbeder, Rory Coughlan, Corey Lusher, John
  Plunkett, Emmanuel Agu, Mark Claypool
• Computer Science Department
• Worcester Polytechnic Institute
• Worcester, MA, USA
• http//www.cs.wpi.edu/claypool/papers/ut2003/