Drew University Library

1
Drew University Library
2
NJ ACRL Chapter - Research Forum

• Assessing a First Year Information Literacy
  Program at an Academic Institution Intended and
  Unintended Consequences

3
First Year Seminar Program

• Mandatory for all new students.
• Requires students to read, consider and react to
  scholarly materials.
• Requires several written assignments, with
  bibliographies.
• A 75 minute library instruction session is an
  integral component of the program.

4
Step 1

• Attended an ACRL online course on Information
  Literacy assessment.
• Identified several learning outcomes for first
  year students, based on ACRL Information Literacy
  Competency Standards for Higher Education.

5
ACRL Competency Standard

• Competency Standard One Performance Indicator
  Two. The information literate student identifies
  a variety of types and formats of potential
  sources of information.
• Outcome 1.2 d. Identifies the purpose and
  audience of potential resources (e.g. popular,
  scholarly, current vs. historical).
• Distinguishes characteristics of information
  provided for different audiences.

6
Student assumptions v. student skills

• Findings from
• Session Report Crossing Boundaries
  Collaborating to Assess Information Literacy.
  Live from ACRL, Teaching and Learning,
  Collaborations. April 1, 2007. Jo Beld, St. Olaf
  College - Survey results.
• Educational Testing Service. 2006 ICT Literacy
  Assessment Preliminary Findings.
  www.ets.org/ictliteracy

7
Step 2

• Created short assessment tool, based on the
  A-B-C-D format for writing objectives.
• A the learner (student)
• B desired behavior of the learner
• C circumstances in which learner performs the
  behavior
• D competency of desired behavior displayed by
  the learner

8
Our objective for Outcome 1.2 d

• After a library instruction session C, students
  will identify characteristics of a scholarly
  article D. This activity demonstrates that
  students A can identify the purpose and
  audience of potential resources B.

9
Sample question for Outcome 1.2 d

• All the below statements about a scholarly
  article may be true. Pick the two best
  statements
• __has footnotes/bibliography
• __is longer than 8 pages
• __author has appropriate credentials (teaches at
  a college has advanced degrees)
• __contains an abstract
• __includes statistics, graphs or tables

10
Our Data

• 4 semesters of data - Spring 06 Fall 07
• Total number of students assessed 741
• Test iterations - 4

11
Step 3 Interpreting results

• ACRL Competency Standard One Performance
  Indicator Two Outcome 1.2 d
• Spring 06
• footnotes/bibliography 77.5
• credentials 75
• Fall 06 (different test question)
• footnotes 92.8
• bibliography 80.4
• credentials 62.7
• Spring 07
• footnotes/bibliography 80
• credentials 82
• Fall 07
• footnotes/bibliography 80.3
• credentials 84.9

12
Expected insights

• We learned how to write a better assessment tool
  less is more.
• The tool helped us standardize our teaching -
  gave us a commonality of focus.
• The process prompted lively discussions and
  brain-storming.
• Allowed us to assess our teaching.

13
Assume nothing - only 73 of students got this
right!

• Circle what you would click on to find the full
  text of the below article.

14
And.

• When one librarian neglected to demonstrate how
  the link resolver worked, only 1 student out of
  15 guessed the correct answer!

15
Lower level vs. higher level skills

• Identify journal title 73
• Identify Article Linker icon 73
• VS.
• Database choice - 98
• Scholarly article 79

16
Discipline-based access to resources

• Anthropology 63
• Philosophy 56
• VS.
• Biology 97
• Music 92
• Physics 97

17
Elizabeth Patterson epatters_at_drew.edu 973
408-3480Catherine Ryan cgryan_at_drew.edu 973
408-3483