Project SAILS results: What do they mean

1
Project SAILS results What do they mean?

• K. Jane Burpee and Judy Wanner
• Library Council
• January 2009

2
Outline

• Background information
• What is Project SAILS
• Logistics and test administration
• Results and comparison W2007 and F2008
• Demographic results
• Custom questions
• Results by skill sets
• Discussion
• How can we use these results?

3
What is Project SAILS

• Standardized test developed at Kent State
• Designed to address the following questions
• How do we know if students are IL?
• What are entry skills? Any changes upon grad?
• Does library contribute to IL?
• Does IL make a difference in success/retention?

4
What is Project SAILS

• Based on ACRL Standards (all but Standard 4)
• 131 questions organized in 8 skill sets
• Developing a research strategy
• Selecting finding tools
• Searching
• Using finding tool features
• Retrieving sources
• Evaluating sources
• Documenting sources
• Understanding economic, legal and social issues

5
Why participate?

• Exercise in needs assessment
• To inform IL teaching
• To provide empirical data to support initiatives
• To support our commitment to IL
• To track changes over time

6
Logistics and test admin

• Testing dates
• Testing population
• Logistics
• Ethics approval
• Web site
• Promotional campaign
• Scheduling

7
Results Demographics
8
Results Demographics
9
Results Summary
10
Informing teaching practices
11
Developing a Search Strategy (2008)

• Combined Arts and Science students at UG
  performed better than the institutional benchmark
  on this skills.
• Students at UG performed about the same as
  institutional benchmarks and Guelph average
  student benchmark

12
Selecting Finding Tools(2008)

• Students at UG performed about the same as
  institutional benchmarks and Guelph average
  student benchmark

13
Searching (2008)

• Students at UG performed about the same as
  institutional benchmarks and Guelph average
  student benchmark

14
Using Finding Tools (2008)

• Students at UG performed about the same as
  institutional benchmarks and Guelph average
  student benchmark

15
Retrieving Sources (2008)

• Students at UG performed about the same as
  institutional benchmarks and Guelph average
  student benchmark

16
Evaluating Sources (2008)

• Students at UG performed about the same as
  institutional benchmarks and Guelph average
  student benchmark

17
Documenting Sources (2008)

• Students at UG performed about the same as
  institutional benchmarks and Guelph average
  student benchmark

18
Understanding Economic, Legal, and Social Issues
(2008)

• Students at UG performed about the same as
  institutional benchmarks and Guelph average
  student benchmark

19
2008 Results Mapped to ACRL

• Students at the University of Guelph performed
  better than the institutional bench marks for
  Standard 2, 3, and 5
• Students at the University of Guelph performed
  about the same as the institutional bench marks
  for Standard 1

20
Results Custom questions
21
Results Custom questions
22
Results Limitations

• Standard error is too large to be meaningful
• Varies between 14 and 25
• Bottom line
• Not representative and not terribly generalizable

23
How can we use these results?

• Looking at low score skill sets
• Understanding legal, ethical issues
• Searching
• Documenting sources
• Use appendices
• Appendix E Test items per skill set and
  Standard
• Appendix F Standards with test items
• Informing Teaching Practice
• Your ideas? ..

24
Recommendations for Future administrations

• Continue SAILS as a longitudinal study
• More test takers
• Highly-monitored vs. unmonitored
• By faculty or department
• Collaborating with faculty members
• Your Ideas?....