COMP3501/7904 Lecture 1

1
COMP3501 Social Aspects of HCI Designing for
Collaboration and Communication. Affective HCI
and emotional design. Prof P M
Sanderson (Based on Chs 4 and 5, Preece et
al.,2002)
2
Overview

• Social aspects of human interaction
• Conversation
• Coordination
• Awareness
• Designing collaborative technologies to support
  the above.
• Computer-supported cooperative work (CSCW) and
  ethnography.
• Affective HCI and emotional design.

3
Social aspects of human interaction Conversation

• Formal
• People have specific roles which constrain
  turn-taking
• e.g., board meeting.
• Informal
• Take place in a wide variety of settings
• Facilitated by physical proximity, gesture and
  posture.

4
Social aspects of human interaction
Conversational rules

• Turn-taking to coordinate conversation
• A Shall we meet at 8?
• B Um, can we meet a bit later?
• ------------------
• A Shall we meet at 8?
• B Wow, look at him?
• A Yes what a funny hairdo!
• B Um, can we meet a bit later?
• Back channeling to signal to continue and
  following
• Uh-uh, umm, ahh
• Farewell rituals
• Bye then, see you, yeah bye, see you later.
• Implicit and explicit cues
• e.g. looking at watch, fidgeting with coat and
  bags
• Explicitly saying Oh dear, must go, look at the
  time, Im late

5
Social aspects of human interaction What happens
in technology-mediated conversations?

• Do same conversational rules apply?
• Are there more breakdowns?
• How do people repair them?
• Phone?
• Email?
• Instant messaging
• SMS texting?

6
Supporting conversations

• How to support conversations when people are at
  a distance from each other
• Many applications have been developed
• Email, videoconferencing, videophones, computer
  conferencing, instant messaging, chatrooms,
  collaborative virtual environments (CVEs), MUDs,
  MOOs, media spaces
• How effective are they?
• Do they mimic or extend existing ways of
  conversing?

7
Conversational coordinationHydra

• Designed to capture spatial cues of gaze, head
  turning, gaze awareness and turn taking that are
  found in face-to-face meetings.
• Preserve spatial relationships "around the
  table.

Designer John Arnott http//www.dgp.toronto.edu/OT
P/TPHomepage/Toys/16.jpg
HCI developer and tester Abi Sellen
8
Conversational coordinationHydra

• Sellen (1992) tested peoples ability to conduct
  conversations with the Hydra system compared with
  the conventional quad or four-quadrant display.
• Hydra system arrangement was the only arrangement
  of sound and images that supported parallel
  conversations.
• Hydra system preserved direction of eye gaze,
  making it possible to form subgroups within the
  larger group.

9
ConversationsWill video succeed on mobile
phones?

• Will judder, sudden jerks and shadowsdisappear?
• Can people establish eye contactand read lips on
  such a small image?
• Is it socially acceptable to talk to an image of
  someone in the palm of your hand?

10
New forms of interaction

• Move beyond trying to support face-to-face
  communication to new ways of interacting and
  talking
• Examples include
• SMS texting via mobile phones
• Online chatting in chatrooms
• Collaborative virtual environments
• Media spaces

11
ConversationsCharacters in collaborative
virtual environments
The rooftop garden in BowieWorld, a Collaborative
Virtual environment (CVE), supported by
Worlds.com. Users take part by dressing up as
an avatar. There are 100s of avatars to choose
from, including penguins and real people. Once an
avatar has entered a world they can explore it
and chat to other avatars.
12
Hypermirror(Morikawa and Maesako, 1998)

• Allows people to feel as if they are in the same
  virtual place even though in physically different
  spaces

People in different places are superimposed on
the same screen to make them appear as if in same
space.
Woman in white sweater is in a different room
from the other three
13
HypermirrorCreating personal space
They move apart to allow person in other real
space more virtual personal space.
Two people in this room seem to be invading the
virtual personal space of the other person by
appearing to be physically on top of them.
14
HypermirrorEveryone happy
15
Social aspects of human interaction Awareness

• Involves knowing who is around, what is
  happening, and who is talking with whom
• Peripheral awareness
• Keeping an eye on things happening in the
  periphery of vision
• Overhearing and overseeing - allows tracking of
  what others are doing without explicit cues

16
Social aspects of human interaction Awareness
Clearboard (Ishii et al, 1993)

• ClearBoard - transparent board that shows other
  persons facial expression on your board as you
  drawleads to better, swifter, coordination.
• Development of ClearBoard from early prototypes
  to current form discussed on http//web.media.mit.
  edu/ishii/CB.html. See further examples plus
  videoclips of ClearBoard in use at
  http//www.wtec.org/loyola/hci/c3_s1.htm

http//www.ntticc.or.jp/Calendar/2000/Tangible_Bit
s/Works/clear.html
http//www.hitl.washington.edu/publications/r-98-3
6/Image276.gif
17
Social aspects of human interaction Awareness
Clearboard (Ishii et al, 1993)

• How Clearboard works

http//www.wtec.org/loyola/hci/fh3_3b.gif
18
Social aspects of human interaction Awareness
Portholes (Xerox PARC)

• Regularly updated digitized images of people in
  their offices appeared on everyones desktop
  machines throughout day and night.
• One of many attempts in the 1980s and 1990s to
  provide media spaces that aided communication
  while preserving privacy.

19
Social aspects of human interaction Awareness
VideoWindow system (Bellcore, 1989)

• A shared space that allowed people 50 miles apart
  to carry on a conversation as if in same room
  drinking coffee together via 3 x 8 ft
  picture-window
• People did interact via the window but strange
  things happened (Kraut, 1990)
• Talked constantly about the system
• Spoke more to people in the same room
• When tried to get closer to someone in other
  place had opposite effect - went out of range of
  camera and microphone
• Between 2 and 3 times less communication than
  when co-located.

20
Social aspects of human interactionEntry into
personal space

• People approaching electronically do so via a
  monitor and speaker mounted above the door (inset
  on right image). 
• Same social conventions are used for both
  physical and electronic visitors. (Buxton
  website)
• Video Surrogate principle
• Don't think of the camera as a camera. Think of
  it as a surrogate eye. Likewise, don't think of
  the speaker as a speaker. Think of them as a
  surrogate mouth (Buxton website)

21
Social aspects of human interaction Coordination

• Coordination takes place when a group of people
  act or interact together to achieve something
• Coordination mechanisms
• Verbal and non-verbal communication
• Schedules, rules and conventions
• Shared external representations
• Pilots in cockpit
• Conventional cockpit cues to mental activity
  from location of physical activity
• Glass cockpit cues to mental activity less
  available from locationtends to all be in same
  place (CRT or MFD).
• Ground operations control rooms
• Monitor others activities
• Adjust own actions appropriately to achieve goals

22
EthnographyDefinition

• For some, the term ethnography is loosely applied
  to any qualitative research project whose purpose
  is rich description. A more precise definition,
  rooted in ethnography's disciplinary home of
  anthropology, is a qualitative research process
  and product whose aim is cultural interpretation.
  The ethnographer goes beyond reporting events and
  details of experience and works to explain how
  these represent the webs of meaning in which we
  live.
• http//labweb.education.wisc.edu/cni916/def_eth.
  htm

Lucy Suchman Cultural anthropologist and
ethnographer of the workplace http//sln.fi.edu/tf
i/exhibits/bower/02/ccscience.html
23
Ethnography and design

• Conducting an ethnographic study is one of the
  main ways to inform the design of collaborative
  technologies that take into account social
  concerns.
• An ethnographic study is a type of field study
  involving observation of people as they engage in
  work and/or collaborative practices.
• Home, public place, school, work, etc.
• Ethnography can be used to inform the design of.
• Specific products for specific workplaces
• New generic technologies
• Ethnography may be used to analyse problems with
  existing interactive technologies.

24
Affective computing and emotional designOverview

• Expressive interfaces
• how the appearance of an interface can elicit
  positive responses
• Negative aspects
• how computers frustrate users
• Anthropomorphism and interface agents
• The pros and cons
• Designing synthetic characters

25

Positive affect makes us smart Norman
(2003) http//www.jnd.org/dn.mss/Emotion-and-desig
n.html

• Affect makes us smart that's the lesson of my
  current research into the role of affect.
• Negative affect can make it harder to do even
  easy tasks positive affect can make it easier to
  do difficult tasks.
• This may seem strange, especially to people who
  have been trained in the cognitive sciences
  affect changes how well we do cognitive tasks?
  Yup.

26
Subtle effect of aesthetics and affectNorman
(2002) http//www.jnd.org/dn.mss/Emotion-and-desi
gn.html

• In the early days of the personal computer, all
  the display screens were black and white. When
  color screens were first introduced, I did not
  understand their popularity. ... From a cognitive
  point of view, color added no value that could
  not be provided with the appropriate use of
  shading. But businesses insisted on buying
  color monitors. Obviously, color was fulfilling
  some need, but one we could not measure.
• In order to understand this phenomenon, I
  borrowed a color display to use with my computer.
  After the allocated time, I was convinced that my
  assessment had been correct -- color added no
  discernible value for everyday work. However, I
  refused to give up the color display. Although my
  reasoning told me that color was unimportant, my
  emotional reaction told me otherwise.
• I can hear it now "Hey, Norman says it's OK to
  be pretty," and off people go, feeling free to
  ignore decades of work by the usability
  community. That's the wrong lesson to learn from
  this essay.

27
Affective aspects

• HCI has generally been about designing efficient
  and effective systems (efficient effective
  satisfying)
• Recently, move towards considering how to design
  interactive systems to make people respond in
  certain ways (satisfying fun enjoyable)
• Some evidence that affective aspects affect the
  perceived usability of an interface
• People are prepared to put up with certain
  aspects of an interface (e.g. slow download rate)
  if the end result is very appealing and aesthetic
• Some evidence that affective aspects do not
  affect consciously judged usability as much as
  previously thought.

28
Friendly interfaces

• Microsoft pioneered friendly interfaces for
  technophobes - At home with Bob software
• 3D metaphors based on familiar places (e.g.
  living rooms)
• Agents in the guise of pets (e.g. bunny, dog)
  included to talk to the user
• Supposedly make users feel more at ease and
  comfortable
• Users have created emoticons to compensate for
  lack of expressiveness in text communication
• -) -) -(

29
Negative affectUser frustration

• Many causes
• When an application doesnt work properly or
  crashes
• When a system doesnt do what the user wants it
  to do
• When a users expectations are not met
• When a system does not provide sufficient
  information to enable the user to know what to do
  
• When error messages pop up that are vague, obtuse
  or condemning (see Shneidermans guidelines in
  text)
• When the appearance of an interface is garish,
  noisy, gimmicky or patronizing.

http//www.usabilitypartners.se/news/2001/editoria
l11.shtml
30
Anthropomorphism

• Attributing human-like qualities to inanimate
  objects (e.g. cars, computers)
• Well known phenomenon in advertising
• Dancing butter, drinks, breakfast cereals
• Much exploited in human-computer interaction
• Make user experience more enjoyable, more
  motivating, make people feel at ease, reduce
  anxiety

31
AnthropomorphismShould computers say theyre
sorry?

• Reeves and Naas (1996) argue that computers
  should be made to apologizeviz, as per human
  etiquette
• BUT would users be as forgiving of computers
  saying sorry as people are of each other when
  saying sorry?
• How sincere would they think the computer was
  being? For example, after a system crash
• Im really sorry I crashed. Ill try not to do
  it again
• How else should computers communicate with users?

32
AnthropomorphismWhich do you prefer?

• 1. As a welcome message
• Hello Chris! Nice to see you again. Welcome
  back. Now what were we doing last time? Oh yes,
  exercise 5. Lets start again.
• User 24, commence exercise 5.
• 2. Feedback when get something wrong
• Now Chris, thats not right. You can do better
  than that.Try again.
• Incorrect. Try again.
• Is there a difference as to what you prefer
  depending on type of message? Why?

33
AnthropomorphismEvidence to support it as a
design goal

• Computers that flatter and praise users in
  education software programs -gt positive impact on
  them (Reeves and Naas,1996)
• Your question makes an important and useful
  distinction. Great job!
• Students were more willing to continue with
  exercises with this kind of feedback but they
  sometimes can make people feel anxious, inferior
  or stupid
• People tend not to like screen characters that
  say
• Now Chris, thats not right. You can do better
  than that.Try again.
• Many prefer the more impersonal
• Incorrect. Try again.
• Studies have shown that personalized feedback is
  considered to be less honest and makes users feel
  less responsible for their actions (e.g.
  Quintanar, 1982)

34
Virtual characters

• Increasingly appearing on our screens
• Web, characters in videogames, learning
  companions, wizards, newsreaders, popstars
• Advantages
• Provide a persona that is welcoming, has
  personality and makes user feel involved with
  them
• Disadvantages
• Lead people into false sense of belief, enticing
  them to confide personal secrets with chatterbots
  (e.g. Alice)
• Annoying and frustrating
• E.g. Clippy
• Not trustworthy
• virtual e-commerce assistants?

35
Virtual charactersSales agents

http//www.ncr.com/repository/case_studies/self-se
rvice/westpac.htm Bank customer interacts with
an ATM whose instructions are given by an
obliging female bank employee.
36
Virtual charactersSales agents

• What do virtual agents do?
• Do they elicit an emotional response?
• Are they trustworthy?
• Is the style of interaction different for men and
  womans clothes?
• Is Miss boo.com believable, pushy, helpful?
• Would it be different if she was a male figure?

37
Virtual characters

• Can be classified in terms of the degree of
  anthropomorphism they exhibit
• Synthetic characters
• Animated agents
• Emotional agents
• Embodied conversational agents

38
Virtual charactersSynthetic charactersSilas
the dog
Autonomous, with internal states and able to
respond to external events.
(Blumberg, 1996 - MIT)
39
Virtual characters Embodied conversational
agents

• Rea, real-estate agent, showing useran apartment
  
• Human-like body
• Uses gesture, non-verbal communication (facial
  expressions, winks) while talking
• Sophisticated AI techniques used to enable this
  form of interaction

http//gn.www.media.mit.edu/groups/gn/projects/hum
anoid/index.html
Cassell, 2000, MIT
40
Virtual characters Embodied conversational
agents conversation with Rea

• Mike approaches screen and Rea turns to face him
  and says
• Hello. How can I help you?
• Mike Im looking to buy a place near MIT.
• Rea nods, indicating she is following.
• Rea I have a house to show you. (picture of a
  house appears on the screen)
• Rea it is in Somerville.
• Mike Tell me about it.
• Rea looks up and away while she plans what to
  say.
• Rea Its big.
• Rea makes an expansive gesture with her hands.
• Mike brings his hands up as if to speak, so Rea
  does not continue, waiting for him to speak.
• Mike Tell me more about it.
• Rea Sure thing. It has a nice garden... etc

41
Affective computingWhich are the most
believable agents?

• Believability refers to the extent to which users
  come to believe an agents intentions and
  personality
• Appearance is very important
• Are simple cartoon-like characters or more
  realistic characters, resembling the human form,
  more believable?
• Behaviour is very important
• How an agent moves, gestures and refers to
  objects on the screen
• Exaggeration of facial expressions and gestures
  to show underlying emotions (cf animation
  industry)

42
Affective computingKey points

• Affective computing is are concerned with how
  interactive systems make people respond in
  emotional ways
• Well-designed interfaces can elicit good feelings
  in users
• Expressive interfaces can provide reassuring
  feedback
• Badly designed interfaces make people angry and
  frustrated
• Anthropomorphism is increasingly used at the
  interface, in the guise of agents and virtual
  screen characters

43
Affective computingInterviews, commentary

• Your textbook doesnt have an interview for Ch 5.
• Read the following online interview with Don
  Norman
• http//www.acm.org/ubiquity/interviews/d_norman_2.
  html
• Read obituaries for Clippy
• http//www.microsoft.com/office/clippy/
• http//www.cnn.com/2001/TECH/ptech/04/12/office.cl
  ippy/

44
END