Evaluation Research

1
Evaluation Research

• A review

2
Setting the Context

• Attention to mission, vision, and goals of our
  institution.
• Evaluating service and quality
• Assessing processes and programs
• Continuous improvement
• Looking at gaps or areas of weakness in the
  field- where has little research been focused?
• Evaluating new trends (folksonomy, digital
  reference, EAD)

3
Four Perspectives

• The institution and the communities served in the
  life of the library/archives
• The library/archives and academic departments and
  programs in the life of the institution

• The institution and the communities served in the
  life of the library/archives
• The library/archives and academic departments and
  programs in the life of the institution

4
Taking Measure Metrics

• Input
• Output
• Performance

• Customer-related
• Outcomes
• Student outcomes
• Student learning outcomes
• Other types--impacts

5
Immmmm Back The Problem Statement

• Lead in
• Originality
• Direction
• Significance/Justification/Value
• Not just the components, but why? Why are these
  elements important? Is it really important to
  co-locate in one place?

6
Literature Review Remember Me?

• Exploring prior research
• Setting the context
• Establishing a foundation
• Identifies variables
• Helps identify methodologies

7
Literature Review

• What else are we really interested in?
• How relevant are the cited works? Are they tied
  back to the present study?
• Skewing focus to support an angle
• How authoritative?
• How accurately are they used/ cited?
• Skewing focus to support an angle

8
Literature Review

• How scholarly?
• Look for patterns not just to see whos most
  cited, but what do we know about the authors and
  journals being cited?
• Scholarly, Trade, Conference Proceedings,
  Personal Communications, etc.
• Cook, S.J., Parker, R.S. and Pettijohn, C.E.
  (2005). The public library An early teens
  perspective. Public Libraries, 44 (3), 157-161.

9
Theoretical Framework

• Logically flows from literature review
• The theories, methods, or understandings on which
  your research is based.
• Marsteller, M.R. Mizzy, D. (2003). Exploring
  the synchronous digital reference interaction for
  query types, question negotiation and patron
  response. Internet Reference Services Quarterly,
  8 (1/2), 149-165.

10
Questions/ Objectives/ Hypotheses

• Continue to refine and focus the problem
  statement.
• Guide the data collection.
• Must relate specifically to the problem statement
  and methodology.
• Martinez, G. (2007). Partnering for reading
  readiness A case study of Maryland public
  libraries. Children and Libraries, 5(1), 32-39.

11
Research Design

• Type of study
• Exploratory
• Descriptive
• Explanatory
• Types of Designs
• Experiment
• Quasi-Experiment
• Case Study

12
Research Design Sampling

• Defining population, samples, units of analysis
• Choosing a representative group
• Random- each member has a known and equal chance
  of being selected
• Systematic- assembling a list and choosing every
  nth member
• Stratified- assembling a list by type/ social
  group, then selecting every nth number
• Convenience samples
• Typical, snowball, and quota

13
Methodology

• Quantitative, Qualitative, Mixed, and
  Triangulation
• How to select an appropriate method?
• What does quantitative data yield?
• What does qualitative data yield?
• What is the purpose of our study?

14
Methodology

• Some examples
• How has electronic access to journals affected
  inter-library loan?
• Dean, E., Goodier, R. (2004). Changing patterns
  in interlibrary loan and document supply.
  Interlending and Document Supply, 32(4), 206-214.

15
Methodology

• What do teenagers think of the public library as
  a place for entertainment, a place for research,
  a place to socialize?
• How satisfied are our customers with our New Book
  selection and circulation policies?
• How efficient is our cataloging department?

16
Methodology

• How do hobbyist geneaologists search for
  information in archives?
• How and to what extent do patrons use our book
  review system on our web page?
• Who are the non-users in our library? What
  factors keep them from using the library?
• How effective is our instruction program?

17
Why Research?

• Accountability
• Continuous Improvement
• Effectiveness
• Impact
• Efficiency
• Customer Relations
• Improved Decision-Making

18
Acknowledging Obstacles

• Ethical concerns
• Researcher bias
• Response bias
• Lack of institutional support
• Lack of time, budget, and staff
• Inappropriate application of evaluation- as a
  gesture, to torpedo unwanted programs, to justify
  pet projects