Creating Chat Connections: Evaluating Virtual Reference Transcripts

1
Creating Chat Connections E-valuating Virtual
Reference Transcripts

• Marie L. Radford
• ACRL
• Delaware Valley Chapter
• November 2, 2007

2
Seeking Synchronicity Evaluating Virtual
Reference Services from User, Non-User, and
Librarian Perspectives

• Project duration 2 ½ Years (10/05-3/08)
• Four phases
• Focus group interviews
• Analysis of 850 QuestionPoint live chat
  transcripts
• 600 online surveys
• 300 telephone interviews

3
Phase II Transcript Analysis

• Random sample
• 7/04 to 11/06 (18 months)
• 500,000 pool of transcripts
• 30-50 per month 850 total sample
• 746 usable transcripts
• Excluding system tests technical problems
• 372 classified by age/educational level
• 146 Screenagers (Middle High School)
• 226 Others (College/Adult)

4
6 Analyses

• Geographical Distribution
• Originating library
• Librarian respondents
• Type of Library
• Wait Time Session Time
• Type of Questions
• Katz/Kaske Classification
• Subject of Questions
• Dewey Decimal Classification
• Interpersonal Communication
• Radford Classification

5
VRS Session Times

• Wait time
• Mean 1.87 Minutes
• Median 1 Minute
• Minimum 1 Second
• Maximum 67 Minutes
• Session time
• Mean 12.42 Minutes
• Median 12 Minutes
• Minimum 12 Seconds
• Maximum 71 Minutes

6
VRS Transactions by Library Type
7
VRS Questions by Location of Originating Library
8
VRS Questions by Location of Librarian
Respondents
9
Wait Time for VRS Users
10
VRS Mean Wait Time by Library Type
11
VRS Mean Session Times by Library Type
12
VRS Questions by Type
13
VRS Questions by Subject
14
Interpersonal Communication Analysis

• Theoretical Framework
• Watzlawick, Beavin Jackson (1967) Pragmatics of
  Human Communication
• All messages have both content relational
  dimension.
• Content Information (WHAT)
• Relational Relationship Aspects (HOW)

15
Method

• Qualitative Analysis of Transcripts
• Development of category scheme
• Careful reading/analysis
• Identification of patterns

16
Interpersonal Communication Research Questions

• What relational dimensions are present in chat
  transcripts?
• Are there differences in relational
  dimensions/patterns of chat users librarians?
  If so, what are they?

17
Results

• Relational Facilitators
• Interpersonal aspects of the chat conversation
  that have a positive impact on the
  librarian-client interaction and that enhance
  communication.
• Relational Barriers
• Interpersonal aspects of the chat conversation
  that have a negative impact on the
  librarian-client interaction and that impede
  communication.

18
Transcript Examples Relational Facilitators

• The Size of an Atom
• Question Type Subject Search
• Subject Type Life Sciences, Biology (DDC570)
• Duration 40 min.
• Diabetes
• Question Type Subject Search
• Subject Type Business
• Duration 43 min., 15 sec.

19
Transcript Example Relational Barriers

• Mesopotamian Government
• Question Type Subject Search
• Subject Type History of Ancient World (DDC930)
• Duration 27 min.
• Telekinetic Powers
• Question Type Subject Search
• Subject Type Parapsychology Occultism
• Duration 7 min., 29 sec.

20
Facilitators VRS Users Screenagers (n146)
vs. Others (n226)

• Lower numbers/percentages per transcript
• S O
• Thanks 21 (75) vs. 77 (175)
• Agreement to try what 32 (46) vs. 51 (116)
• is suggested
• Closing Ritual 32 (47) vs. 49 (111)
• Self Disclosure 42 (61) vs. 55 (125)
• Seeking Reassurance 39 (57) vs. 49 (111)
• Admit lack knowledge 19 (13) vs. 21 (47)

21
Facilitators VRS Users Screenagers (n146)
vs. Others (n226)

• Similar numbers/percentages per transcript
• S O
• Alternate Spelling/ 28 (41) vs. 27 (60)
• Abbreviated Words
• Informal Language 9 (13) vs. 9 (21)
• Offering Confirmation 8 (11) vs. 8 (13)
• Empathy 3 (4) vs. 4 (8)

22
Barriers VRS Users Screenagers (n146) vs.
Others (n226)

• Higher numbers/percentages per transcript
• S O
• Impatience 8 (12) vs. 6 (13)
• Rude or Insulting 6 (9) vs. 4 (9)

23
Facilitators - Librarians Screenagers (n146)
vs. Others (n226)

• Lower numbers/percentages per transcript
• L to S L to O
• Offering Opinion/Advice 29 (43) vs. 37 (83)
• Explaining Search Strategy 6 (9) vs. 14 (31)
• All Lower Case 11 (63) vs. 18 (43)
• Encouraging Remarks 12 (18) vs. 17 (39)

24
Facilitators - Librarians Screenagers (n146)
vs. Others (n226)

• Higher numbers/percentages per transcript
• L to S L to O
• Seeking Reassurance 61 (89) vs. 51 (115)
• Greeting Ritual 52 (76) vs. 48 (108)
• Asking for Patience 39 (57) vs. 35 (80)
• Explaining Signing off 5 (8) vs. 1 (2)
• Abruptly

25
Facilitators - Librarians Screenagers (n146)
vs. Others (n226)

• Similar numbers/percentages per transcript
• L to S L to O
• Polite Expressions 57 (83) vs. 56 (127)
• Inclusion 33 (48) vs. 34 (76)
• Thanks 22 (32) vs. 23 (51)
• Makes Sure User Has 18 (27) vs. 20 (45)
• No More Questions
• Interjections 8 (11) vs. 9 (20)

26
Barriers - Librarians Screenagers (n146) vs.
Others (n226)

• Higher numbers/percentages per transcript
• L to S L to O
• Abrupt Endings 16 (23) vs. 9 (20)
• Limits Time 6 (9) vs. 0 (1)
• Sends to Google 5 (8) vs. 0 (0)
• Reprimanding 4 (6) vs. 0
  (1)
• Failure/Refusal to 5 (7) vs. 2 (5)
• Provide Information

27
Strategies that Work!All Modes of Reference

• Basic interpersonal skills
• Recognizing that user may need reassurance
• Providing reassurance
• Awareness of appropriate self-disclosure
• When to disclose
• Acknowledgment of users self-disclosure
• Humor importance of acknowledgment

28
More Strategies

• Greetings Closings.
• Beware negative closure!
• Beware robotic scripts!
• Inclusion (use of we, lets, etc.).
• Mirror relational strategies.
• Dont b afraid 2 use informal language,
  abbreviations emoticons as appropriate )

29
Boost Satisfaction

• Collaborate across generations
• End encounter on a positive note.
• Ask Have I answered your question completely?
• Avoid Negative Closure
• Invite to return to desk or e-service if further
  help needed.

30
Bottom Line

• Communication critically important!
• Difficult process
• Generational differences add to complexity!!
• Use your experience intuition as guides.

31
Questions?

• Marie L. Radford, Ph.D.
• Email mradford_at_scils.rutgers.edu
• www.scils.rutgers.edu/mradford

32
End Notes

• This is one of the outcomes from the project
• Seeking Synchronicity Evaluating Virtual
  Reference Services from User, Non-User, and
  Librarian Perspectives
• Funded by IMLS, Rutgers University, OCLC Online
  Computer Library Center, Inc.
• Special thanks to Lynn Silipigni Connaway,
  Patrick Confer, Timothy Dickey, Jocelyn DeAngelis
  Williams, Julie Strange, Janet Torsney, Susanna
  Sabolski-Boros.
• Slides available at project web site
  http//www.oclc.org/research/projects/synchronicit
  y/