If You Build It, Will They Learn

1
If You Build It, Will They Learn?

• Lara Cummings
• Instruction Librarian
• Washington State University
• Online NW, Corvallis, OR
• February 10, 2006

2
Introduction and Background

• Tools we already had
• Web-based tours and tutorials
• Interactive readings, Qarbon viewlets, and
  HTML-modules
• Quizzes, crossword puzzles etc
• Purpose of these tools
• Just in time need
• Supplement to Library Instruction
• Designed to teach and promote Information
  Literacy
• Tied with ACRL standards
• DDP Students

3
Introduction and Background

• What we received
• Internal Grant - 5,000
• What we needed to do
• Assess our tools
• Align tools with state mandate for IL
• DDP Population
• Usability

4
Getting Started

• Two-pronged approach
• Measure student success AND tool navigability and
  utilization

Learning VS Functionality How are the students
using the tools?
5
Getting Started

• Key Objectives
• Evaluate usefulness and effectiveness of existing
  tutorials
• Gather and analyze assessment data
• Address findings by
• editing existing tutorials
• developing new tutorials
• ensuring the ACRL IL standards are addressed and
  communicated

6
Methodology

• Assessment Activity Goals
• Gather student data about their
• Use of resources
• Attitudes about the libraries
• Perceptions about online tutorials
• Additionally, each assessment module would
  require the completion of tasks, to verify if
  student learning was a success

7
Methodology

• Four tools selected
• Griffin Tutorial Outline and use of WSU
  Libraries' Griffin catalog
• NetLibrary Tutorial Learn how to find electronic
  books from NetLibrary
• Online Library Tour for Undergraduates Overview
  of WSU-Pullman library resources and policies
• ProQuest Direct Viewlet Use ProQuest Direct to
  locate scholarly articles

8
Methodology The Modules

• Four parts to each assessment
• Introductory information about students and their
  library use
• Opportunity to complete a tutorial or tour in a
• hands-on activity
• Complete tasks based on what the tutorial or tour
  taught them
• Post-activity information based on perceptions
  and attitudes

9
Methodology Participation

• Monetary reward for participation (!)
• Students could complete more than one
  assessment
• Marketing was done
• In library
• To student groups
• To distance degree students
• Scheduling the time
• One full day in the Online Computer Lab

Turnout 98 on-campus and DDP participants
10
Methodology Procedures

• Student consent forms
• Payment vouchers
• Assigned logins for tracking
• Response Database
• Participant responses recorded in an SQL database
  using PHP scripting

11
Assessment Griffin Tutorial
WSUs online catalog Viewlet completed in April
2003 Approximately 200 visitors per month

• Student Experiences
• 76 on-campus, 73 DDP had used Griffin
• 49/50 learned Griffin on own
• 65/40 fairly confident w/Griffin (7-10 on scale)

12
Griffin Tutorial (Cont.)

• Hands-On Activity
• Portion of Griffin Tutorial completed that
  included author, title, keyword and subject
  heading searches w/concluding quiz
• Quiz Results
• Boolean operator confusion
• Keyword vs subject searching confusion
• Title field vs full-text of book
• Periodical misconception
• Articles in the OPAC

13
Griffin Tutorial (Cont.)

• Reflective Questions
• What did you learn?
• Truncation, Boolean operators, articles in OPAC
• Questions about Griffin?
• Articles in the OPAC, access from home, full-text
  availability
• Improvement?
• Increasing interactivity, more graphics etc

14
Assessment Online Tours

• Three versions offered
• Undergraduates, graduates and distance degree
  students
• Available since fall 2002
• August 2004-April 2005 statistics
• DDP 1016 uses, 113 per month average
• Undergraduate - 808 uses, 90 per month average
• Graduates 212 uses, 24 per month average

15
Online Tours (Cont.)

• Student Experiences
• 30 self proclaimed frequent library users
• 83 had checked out materials
• 76 used online databases
• 29 confidence level w/Libraries (8 or higher on
  scale)
• Hands-On Activity
• Portion of online tour completed including
  searching for a book, liaison librarian and
  library hours w/concluding quiz

16
Online Tours (Cont.)

• Quiz Results
• Only 51 completed the Griffin keyword search
  correctly
• 71 located correct liaison librarian
• Only 11 located correct library hours
• DDP students averaged higher correct responses
• Reflective Questions
• Confidence level of libraries raised from a mean
  score of 5.8 to 7.6

17
Assessment ProQuest Viewlet

• Three carbon viewlets offered for ProQuest
  Direct
• Locating scholarly sources, newspapers and
  reviews
• Created in fall 2004
• 163 visitors since its creation

18
ProQuest Viewlet (Cont.)

• Student Experiences
• 67 knew the difference between the OPAC and
  ProQuest and had used ProQuest previously
• 62 indicated they knew what was meant by
  scholarly articles
• Low confidence level using ProQuest
• Only 13 could give clear indications of what was
  and was not available in ProQuest
• 67 indicated they knew how to limit a search
  in ProQuest

19
ProQuest Viewlet (Cont.)

• Hands-On Activity
• Students watched the ProQuest Viewlet and were
  asked to proceed to Proquest to search for
  Weapons of Mass Destruction and use search
  results for follow-up questions
• Quiz Results
• 82 found ProQuest as the viewlet had indicated
• 52 correctly located scholarly sources on the
  topic

20
ProQuest Viewlet (Cont.)

• Quiz Results
• Some limiting confusion
• Mixed results with utilizing ProQuest help pages
  to define scholarly articles and peer-review
• DDP students generally more successful in
  searching process and with ProQuest activities as
  a whole
• Reflective Questions
• What did you learn from this viewlet
• What peer-review is, who is involved in peer
  review
• 46 increase in confidence level for using
  ProQuest

21
Assessment NetLibrary Viewlet

• Electronic book database with 9000 titles
• Completed in spring 2004
• Approximately 50 visitors per month
• Student Experiences
• Less than 10 indicated they had used electronic
  books
• 15 indicated they could try to locate e-books in
  Google or on the Internet
• Low confidence levels for locating e-books

22
NetLibrary Viewlet (Cont.)

• Hands-On Activity
• Students watched the carbon viewlet and then were
  asked to proceed to NetLibrary to complete tasks
  and respond to follow-up questions
• Quiz Results
• Perceptions of electronic books
• Nearly 50 compared them favorably to paper-bound
  books
• Special e-features added value to text
• 24 indicated e-books were convenient

23
NetLibrary Viewlet (Cont.)

• Quiz Results
• Confusion over book access and checkout periods
• Reflective Questions
• Confusion of database access via WSU subscription
• Increase in confidence levels for e-books
• Student confidence level for utilizing this
  resource fairly high for on-campus and DDP

24
Discussion

• Actual performance in all activities did not
  match confidence levels
• While self proclaimed confidence levels did rise
  in all activities, quiz results told a different
  story

25
Discussion

• DDP Students performed better in all activities
• Were the undergraduates rushing?
• Do the online tutorials better serve adult
  learners?
• Generation Y gap?
• More interactivity?

26
Discussion
Lack in confidence of locating items physically
in the libraries

• Student confusion of Library holdings
• What can be accessed remotely
• Commercial vs WSU Libraries website
• OPAC vs article databases

27
Future Plans

• Enhancing Tutorials
• Griffin Reading call numbers and locating items,
  more focus on information formats, Boolean
  searching and limits
• Tying licensed databases to the WSU Libraries
• Placing links to tutorials on specific pages for
  point of need refreshers

28
Conclusion

• WSU Tours and Tutorials available at
• http//www.wsulibs.wsu.edu/electric/trainingmods/i
  ndex_of_quizzes.htm
• Assessment activities available at
• http//www.wsulibs.wsu.edu/usered/tutorials/assess
  ment.html
• Slides Posted (on Monday)
• http//www.wsulibs.wsu.edu/science/Lara/lara1.html
  
• Questions, Comments??

29
For more information

• Corey Johnson Griffin Tutorial
• coreyj_at_wsu.edu
• Beth Lindsay Online Tours
• elindsay_at_wsu.edu
• Lara Cummings ProQuest Viewlet
• lursin_at_wsu.edu
• Jane Scales NetLibrary Viewlet
• scales_at_wsu.edu
• Read our article in the September 2006 Issue of
  CRL