Coordinator: Andy Crabtree axccs.nott.ac.uk

1
Understanding Social Interaction in UbiComp
Environments

• Experiences from e-Social Science Research Node,
  DReSS (www.ncess.ac.uk/digitalrecord)
• Developing new forms of Digital Records for
  e-Social Science
• Understanding social interaction in UbiComp
  environments is a central focus of the research
• Seeks to combine traditional ethnographic
  resources (video, photos, fieldnotes, etc.) with
  resources internal to UbiComp environments
  (various kinds of system log) to understand
  social interaction
• Report on experiences to date and the ongoing
  development of tools to support this

2
The Challenge of Understanding Social Interaction
in UbiComp Environments

• The fragmented character of social interaction
• Interaction is distributed across a burgeoning
  array of small, mobile devices and online
  environments which exploit invisible sensing
  systems
• Not distribution that is the problem but that
  different devices and environments exploit
  different interaction mechanisms (e.g., audio
  recordings and text messaging)
• Furthermore, invisible sensing systems impact
  upon the use of those interaction mechanisms
  (e.g., GPS or WiFi blackspots)
• So interaction is fragmented in the sense that
  it is differentially distributed and mediated,
  and in the sense that it is, in various ways,
  interrupted by the underlying infrastructure
• The primary challenge to understanding social
  interaction in these environments is one of
  reconciling the fragments - of pulling the
  various pieces of interaction, and technical
  effects on interaction, together

3
Fragmented Interaction? A Perspicuous Setting

• Can You See Me Now?
• A mobile mixed reality game (MRL and Blast
  Theory)
• Runners located on city streets chase and catch
  online players
• Interaction mechanisms
• Runners have GPS-enabled PDAs exploiting a local
  WiFi infrastructure that provide a map view of
  the game zone and display players positions
  (indicators 5 metre catch zone)
• Players have a 3D virtual model of the game zone
  and avatars which they manipulate to move around
  and to avoid runners avatars
• Players communicate with one another and runners
  via text messages
• Runners communicate with one another via
  walkie-talkie, which also streamed to the players
  (runners also use the walkie-talkies to talk to
  technical support located in a nearby control
  room)
• Ethnographic studies done to understand social
  interaction

4
Studying Runners Work

• Straightforward ethnography
• Getting out on the streets with runners
• Shadowing them
• Videoing their work
• Discussing the work with them afterwards
• Complemented by GPS and WiFi log visualizations
  showing good and bad areas of connectivity
• Blue areas poor coverage and sites where
  technical interruptions are likely to occur

5
Studying Runners Work

• Ordinary interactional troubles handling
  interruptions
• Runner 2 is chasing a player
• Runner 2 on walkie-talkie. Runner 2. Ive just
  lost all players Ive lost all players!
• Runner 2 (Looking at PDA). Ive got
  disconnection here.
• (Runner 2 abandons the chase).
• Runner 2 on walkie-talkie Runner 2. Heading
  seawards on Otto. I am currently disconnected.
• (He turns around and starts walking back down
  the street to the last known point at which he
  had connectivity. He arrives at the car park
  where he last checked the PDA).
• Runner 2 on walkie-talkie Runner 2. Ive
  connectivity again. Im in Vern.

6
Studying Runners Work

• Distributing Awareness
• On encountering an interruption, runners make the
  kind of interruption they are experiencing (a
  disconnection in this case) public knowledge
• Interruptions are announced and broadcast to
  other runners
• The announcement makes others aware that a
  technical trouble is to hand and of the location
  or place where it has occurred
• The interruption is repaired by retracing steps
  and moving to a location where connectivity was
  previously available
• This practice trades on and exploits working
  knowledge of the technology - of knowing that
  disconnections are transient technical phenomena
  that may be resolved by moving to a better
  location
• It also exploits local knowledge of the physical
  environment in which the technology is situated -
  knowledge of where in the environment is a
  better location
• Both forms of knowledge are thoroughly
  intertwined in practice and developed through
  hands on experience and shared awareness
• Together they combine to shape a common stock of
  knowledge amongst runners as to the kinds of
  technical troubles that occur in the course of
  use and practices for the solution of those
  troubles

7
Studying Runners Work

• Ordinary interactional troubles handling
  interruptions
• Runner 2 on walkie-talkie Runner 2. Im heading
  seawards on Wilamena, waiting for a server
  update.
• Runner 2 on walkie-talkie My GPS is currently
  35 metres. My server position is about 50 metres
  out.
• Runner 2 on walkie-talkie This is Runner 2. Can
  Runner 1 and Runner 4 hear me, or Runner 3
  please? Come in.(Runner 2 switches to the
  technical channel).
• Runner 2 on walkie-talkie This is runner 2 on 4
  Zero. I cant get any response from anyone else
  on 238 (gameplay channel). Can you please confirm
  that the other runners are on 238?
• Runner 2 on walkie-talkie And who else is on 4
  Zero (technical channel) please?
• Runner 2 Runners 1 and 3 are having technical
  trouble. 4s in.
• (Runner 2 sees Runner 3 on the other side of the
  street and goes over to him).
• Runner 3 Are you on 238?
• Runner 2 Im on 238, yeah. I just switched
  back. Whats the problem?
• Runner 3 Just not moving.
• Runner 2 Yeah, Im having the same. Looks like
  we have a bit of a server screw up.
• Runner 3 All right.(Runner 2 walks away from
  Runner 3).
• Runner 2 on walkie-talkie This is runner 2.
  Ive had no GPS update in 2 or 3 minutes. (Runner
  2 walks towards the seafront, where there is
  usually good GPS coverage).

8
Studying Runners Work

• Diagnosing interruptions
• While interruptions are apparent - e.g., being
  stuck because of a lack of updates - the source
  and extent of interruptions is not clear
• Is a server problem to hand? Do we have poor
  satellite availability? Is my equipment bust? Are
  others experiencing the same troubles?
• Finding out is important, as answers inform
  decision-making - i.e., identifying an
  appropriate course of action to solve the problem
• Finding out is done through diagnostic work,
  which consists of
• Determining which channel others are on (gameplay
  or technical and done directly or through
  control)
• Collaborating with others face-to-face
• And, on occasion, by surreptitiously monitoring
  other runners

9
Studying Runners Work

• The purpose of diagnostic work
• These collaborative activities enable runners to
• Establish the gameplay status of their colleagues
• To see for themselves if others are encountering
  interruptions
• To see what sort of interruptions they are
  encountering
• They enable runners to establish a sense of the
  generality of interruptions, which allows them to
• Establish whether not interruptions are theirs
  alone and related to their personal circumstances
  and kit
• Or being experienced by others and related to the
  games technical infrastructure
• In turn, this knowledge suggests an appropriate
  next move in managing the interruption e.g.,
• Moving off to a better location and waiting for a
  GPS update
• Restarting the PDA
• Or getting control to restart the game

10
Studying Players Work

• Not straightforward
• Players were globally distributed
• A handful were local (some terminals at venue),
  but very noisy and difficult to study
• Anyway, terminal not where the interaction was at
  
• Needed access to online environment in some way
• So, we decided to consult system logs, which
  recorded text messages sent by players

11
Studying Players Work

• Text logs difficult to work with (very messy)
• But they did reveal various arrangements of
  collaboration whereby players
• Oriented new players to runners and the game
• What do they look like, what do I do?
• Helped each other avoid runners
• Be careful, runner 3 is at
• Took evasive action
• Look out, to your right. Get out of there!
• Organized collaborative play to disrupt the
  runners
• Lets meet the runners together, should confuse
  them.
• Searched for and found players
• Has anyone seen? Yeah, shes at
• Arranged meeting places in the virtual city
• Lets meet up at

12
Studying Players Work

• The logs also revealed ways in which players
  encountered and made sense of interruptions
• 1. MARCEL Attention. Runner 1 is cheating by
  using his invisible coat
• HBAB Whats an invisible coat?
• MARCEL Never mind what the coat is - he can pop
  out of nowhere
• 2. MARJOLEIN Anyone seen the runners?
• MELISSA I think they can turn off their signal
• HANNE I only see two runners - are the rest
  taking a coffee?
• BLASTER Runner 1 is just a lazy joke
• HANNE If they can turn off their signal thats
  pretty scary and not really fair
• MELISSA Tell me about it
• MARJOLEIN Well maybe the satellites dont work
  properly
• 3. JO Runner 1 needs a GPS update 3 maybe
  shes already on me
• HBAB Looks like runners without a red circle
  dont have GPS
• JO Too bad the GPS is so unreliable. I was
  supposedly seen with no runner in sight
• 4. Blaster Not only have we a scary looking
  dark building to hide behind but its also crap
  GPS. Pray hard to the anti satellite god
• Over time, players wove interruptions into
  gameplay, even exploiting them to tactical
  advantage

13
Reconciling Interaction and Effects

• Exploiting digital records initial experiences
• Extremely time consuming and laborious
• Valuable but of limited use
• Analysis fairly broad
• Some collaborative features
• But does not enable analysis of moment-by-moment
  interaction between participants (runners and
  players)
• Doesnt really reconcile the fragments then
• Largely because of what was logged
• Need a richer set of resources
• And better tools for working with them

14
Developing Digital Records

• e-Social Science and Uncle Roy
• Another mobile mixed reality game
• This time players online and on the streets
• Must collaborate to find Uncle Roys office
• Interaction mechanisms
• Audio messages from street players
• Text messages from online players
• Street players have a PDA and map view of game
  zone, and receive clues from Uncle Roy to guide
  them
• Online players have 3D virtual model and can
  track players avatars through it
• Street players must collaborate with online
  players to find a postcard somewhere in the city,
  which triggers the provision of information to
  online players with which to guide street players
  to Roys office

15
Logging Interactional Fragments

• Log records audio, text messages, clues, etc.

16
Working with the Log

• Synchronizing log content with external resources
  (video, etc.)
• Extracting sequences of interaction that run
  alongside videoed sequences
• Removing non-relevant content (including
  non-relevant conversational threads)
• Adding audio file content (utterances)
• Adding video content (practical actions and talk
  on the streets)

17
Reconciling Interactional Fragments

• Managing Logged Order and Interactional Order
• The two are not the same
• When a message is logged by the system is not
  necessarily when it enters interaction
• Need to consult video to represent interactional
  order

18
Tools to Support Work of Reconciliation

• Data Goggles

19
Tools to Support Work of Reconciliation

• Selecting relevant sequences

20
Tools to Support Work of Reconciliation

• Adding content and editing order

21
(No Transcript)