Subjectivity and Whole Body Interaction

1
Subjectivity and Whole Body Interaction
2
Subjectivity and Whole Body Interaction
Peter Wright Art and Design Research
Centre C3RI Sheffield Hallam University
3
Acknowledgements
EPSRC Leonardo Network Research at the
intersection of art and HCI EPSRC LTR
Project Theory and method in experience-centred
design Design 21 My Exhibition
Project Affective communication in product and
exhibition design Sheffield Health Trust 2nd
Life for the Third Age Hester Reeve, Ben
Heller, Jon Wheat, John McCarthy
4
Overview

• The interesting case of Installation Art
• Is it happening?
• Theories of Embodiment and experience
• Paul Dourish McCarthy and Wright
• Second Life for the Third Age
• agency and coupling in virtual space
• My Exhibition
• Personalisation and ambient interaction
• Towards a critical framework for whole body
  interaction

5
Disclaimer!!
6

• The interesting case of Claire Bishop on the
  History of Installation Art

7
The subject in the art work
Installation differs from traditional media
(sculpture, painting, photography and media) in
that it addresses the viewer directly as a
literal presence in the space. Rather than
imagining the viewer as pair of disembodied eyes
that survey the work from a distance,
installation art presupposes an embodied viewer
whose senses of touch, smell and sound are as
heightened as their sense of vision.
8
But more than this

• The viewer is not just in the art work but they
  complete it
• The spectator is in some way regarded as
  integral to the completion of the work. Reiss
• They are then at once experiencing the art work
  and part of it.
• The spectator is so integral that without
  having the experience of being in the piece,
  analysis is difficult. Reiss

9
The experiencing subject

• If installation art is at pains to offer 1st-hand
  experience what kind of experience does it offer?
  
• Four modalities of experience that installation
  art structures for the viewer, each with a
  different model of the subjective self
• The Freudian self
• The phenomenology of perception
• Lacan and Barthes beyond the pleasure principle
• The activated/politicised subject

10

• In general HCI has been slower than Installation
  Art to escape the gravitational pull of
  Cartesianism
• As a consequence it has under theorised
  experience and subjectivity
• The disembodied mind
• The dominance of the eye
• and hand
• Representation rather than participation
• But things are beginning to change

11

• Embodiment and Embodied Interaction
• Paul Dourish

12
Dourish Embodied interaction

• Embodiment is both a physical and a social
  phenomenon
• Meaning is constructed in practical acts of
  engagement with social and physical worlds
• Objects and people in the world can be understood
  as affording different forms of engagement
• Meaning is maintained by a coupling between the
  agent and the world

• Embodied phenomena are those which by their very
  definition occur in real time and real space
• Embodiment is the property of our engagement
  with the world that allows us to make in
  meaningful.
• Embodied interaction is the creation,
  manipulation and sharing of meaning through
  engaged interaction with artifacts

13
Dourish Space and place

• Space is often characterised in Cartesian terms
  as a container for our action (an immutable
  backdrop)
• But in fact space is constructed in response to
  human concerns
• Place is characterised in terms of human
  habitation, adaptation, culture but also personal
  histories and emotional meanings
• See also Fitzpatrick on Locales

14
Dourish Key Points

• the key things to take from Dourish then are
• embodiment as both a physical and social
• participation rather than representation
• meaning as constructed through physical/social
  interaction in place
• coupling between the embodied person and the
  environment
• The theory orients towards the situated nature of
  our interactions with technology but
  under-theorises the felt life of these
  interactions

15
But more than this

• With his concern for the social, Dourish
  under-theorises, the subject
• While he acknowledges both the physical and the
  social aspects of embodiment, he does not explore
  how these inter-relate
• We have tried to foreground felt life and the
  subjectivity of our experience with technology

The spectator is so integral that without
having the experience of being in the piece,
analysis is difficult. Reiss What kinds of
experience of experience does technology
structure?
16
Just three themes

• A holistic approach to Felt Life
• where embodied, narrative, emotional and sensual
  aspects of felt life are treated as equal,
  interdependent constituents of experience
• Continuous engagement and sense making
• Wherein the self is the centre of experience, is
  already engaged in experience and bring to a
  situation a history of personal, social and
  cultural meanings, and anticipated futures that
  structure experience through acts of sense making
• A dialogical approach to self
• Wherein self, other, object and setting are
  actively constructed as multiple centres of value
  (multiple voices) and are always an unfinalised
  project

17
A holistic approach
The threads in experience

• Sensual
• our sensory engagement with a concrete situation-
  palpable visceral
• Emotional
• the evaluative relations that unite needs and
  desires to the particular
• that was satisfying, Im proud of that
• Compositional
• the narrative structure of a situation, before
  during and beyond
• whats this about, what has happened, what
  should I do?
• Spatial temporal
• The quality of space and time,
• Place, pace, control,

18
Sense making

• anticipating
• expectation as a continuous process in experience
• connecting
• immediate pre-linguistic sense of a situation
• interpreting
• understanding the events in terms and how they
  fit with expectations
• reflecting
• making judgements about the experience
• appropriating
• making something our own
• recounting
• to self and others shapes our understanding and
  fuels anticipation

Making sense of the threads

19
Dourish, McCarthy and Wright Summary

• Our embodied interactions are shot through with
  sensations, emotions, values, ideals, and
  feelings that are subjectively felt and can be
  intersubjectively shared
• We experience our interactions from a unique
  position
• yet our sense of self is constituted in response
  to the voices (values) of others
• Experiences do not come ready made they are
  actively finished by the subject

20

• Second life for the Third Age
• Or
• Nelly and the avatars

21
Nelly and the Avatars

• The people
• Fall Pre-hab class, 6 people aged 86-91,
• No previous experience of computers, WIIs,
  Playstations or X- Boxes
• The technology
• Weight sensors on weighing scales controlling a
  Line Dancing avatar created though 3-D motion
  capture
• The aim
• to explore the feasibility of using whole body
  interaction with such an extreme user group
• To assess whether the technology would be
  experienced positively by both client and
  therapist

22
Video Excerpt
23
Nelly and the Avatars

• Initial impressions
• Fall Pre-hab class, 6 people aged 86-91,
• no previous experience of computers, WIIs,
  Playstations or X Boxes
• The technology
• Weight sensors on weighing scales controlling a
  Line Dancing avatar created though 3-D motion
  capture
• The aim
• to explore the feasibility of using whole body
  interaction with such an extreme user group
• To assess whether the technology would be
  experienced positively by both client and
  therapist
• The vision
• to have a group of people some of who cannot
  walk, some of whom cannot talk, some of whom
  cannot move their upper torso dancing together in
  virtual space.

24
Nelly and the Avatars Initial Impressions

• The emotional response
• What a remarkable bunch of elderly people!
• They loved it!
• They want to do it again
• They talked about dancing
• Coupling issues
• The mapping between user and avatar movements
  didnt appear to be self-evident
• Remembering the required movements was an issue
• The language of movement was confusing
• Multiple couplings between feet, weight and
  avatar movement were confusing

25

• My Exhibition

26
Background

• Project aims
• How can the visceral qualities of interactive
  images, sounds, lighting and other sensory
  factors, can be used to help people personalise
  experiences of exhibition content?
• We are designing interactive "zones" in the
  exhibition which will help people attend to
  narratives and forms of engagement that will suit
  them.
• Zones will recognise individual visitors, offer
  choices, record their decisions and actions to
  build up a record of the visit that will inform
  the next waypoint about that visitor's interests
  and guide them onward in their visit. The
  "ambient" communication, and unencumbered
  interaction

27
Background

• The Froissart Chronicles
• The exhibition is centred on a collection of
  illuminated manuscripts chronicling The Hundred
  Years War
• Each written by the same author - John Froissart,
  but for different Patrons- French and English.

28
Background

• Our Corridor
• Our installation will be in the approach corridor
  and will comprise three zones
• Each Zone will present a story
• Sir John Chandos
• Sieging the Castle
• The writing of the Chronicles
• When our part of the exhibition is open it will
  be the visitors first experience of the content

29
Background

• Our Corridor
• A physical mock-up of part of the corridor has
  been built in our interaction lab

30
Background

• Some themes
• The stories offer a wealth of personal and
  historical perspectives which we have
  provisionally organised around themes
• Passion, History, Politics, Forensics, The
  corpse, The chronicler, The curator
• These themes offer a way of organising and
  potentially personalising content
• The also offer the possibility of experiencing
  multiple points view and multiple voices that
  characterise the chronicles

31
Interaction Design

• We are in the middle of this, we have
• A digital infrastructure for controlling lights,
  video and sound
• A database to record personal details, develop
  personalisation trajectories and to track
  movement through the space and interaction in it.
  
• We have a interoperable toolkit for rapidly
  prototyping interactions including RFID, and
  gesture-based interaction
• The system knows who a person is where they are
  where they have been and what they have seen.
• The aim is personalisation of based on who a
  person is where they have been and what they have
  already seen

32

• Towards a Critical Framework

33
So what would I like to see in a critical
framework?

• A framework that
• Centres on the experience of interaction not the
  technology for interaction
• Takes as its starting point the richness of the
  embodied subject
• Focuses on the design space but connects to use
• Encompasses work, leisure, educational and
  cultural applications
• Can be made meaningful to user researchers,
  artists, performers, designers and engineers

34
A possible starting point Hornecker Buur 2006

• Many one-dimensional viewpoints already exist
• Data-centred viewpoint
• Focus on body as a source of data and coupling
  between body and digital representations
• Expressive-Movement Centred viewpoint
• Focus on sensory richness of expressive
  movement
• Space-Centred view
• Focus on the installation and the interaction
  space (whole body multiple people

35
A possible starting point Hornecker Buur 2006

• The framework themes
• Tangible Manipulation (sensual interaction,
  objects, tactile qualities)
• Can users grab, feel and manipulate the important
  elements
• How easy is it to grasp action-effect mappings?
• Expressive Representation (what is
  represented/manipulated,its legibility)
• Are representations meaningful and relevant?
• Are the representations self-evident in terms of
  use?
• Spatial Interaction (relations of people and
  objects in space)
• Do people and objects meet in meaningful ways?
• Is movement meaningful to system and others?
• Embodied Facilitation (relations in social space)
• Does the physical arrangement encourage/force
  collaboration?
• Does ensemble of representations build on users
  experiences?

36
Looking around

• Paul Dourish
• Where the action is (MIT Press 2001)
• McCarthy and Wright
• Technology as Experience (MIT Press 2004)
• Hornecker and Buur
• Getting a grip on tangible interaction (CHI 2006
  Proceedings)

37
(No Transcript)