Heidi Julien

1
Evaluation of Information Services

• Heidi Julien
• LIS 503
• Fall 2009

2
Purpose

• assist in decision-making about value and
  sometimes how to improve, expand, contract,
  continue, or discontinue a function
• assist in formulation of policy and planning
• reduce uncertainty

3
Types of evaluation

• Formative (process evaluation)
• to improve an activity, service, or process
• Summative (outcome evaluation)
• determining value of activity, service, or
  process

4
Who cares?

• Stakeholders
• groups who fund, benefit, manage, or work in unit
  being evaluated
• clients

5
Types of evaluation

• Expert evaluation
• Personnel evaluations, promotion evaluations,
  peer review, accreditations, award decisions
• Goal-based evaluation
• Views the function in terms of whether or not it
  is achieving its goals (based on written goal
  statements)
• Need to specify goals in terms of measurable
  outcomes
• Sometimes evaluation reveals goals that have been
  misplaced, or unintended goals achieved
  (beneficial or not)

6
Types of evaluation (cont.)

• Goal-free evaluation
• Evaluation in absence of pre-determined goals
• Impact evaluation
• Explores differences a function makes to
  clients/participants
• Can examine level of participation, match of
  expectations and outcomes

7
Types of evaluation (cont.)

• Cost-benefit Cost-effectiveness evaluations
• Concerned with financial outcomes and outputs
• Assess resource expenditure against
  outcomes/outputs
• Compare different functions to select most
  efficient/effective

8
What is evaluated?

• Inputs - resources used to provide a service
• Processes - activities carried out to produce
  outputs
• Outputs - measures of the amount of services
  provided
• Outcomes - how things changed for the clients or
  the community

9
Managers are generally concerned with

• Impact did the function meet its goal?
• Efficiency costs are related to benefits
• Coverage who was served, and to what level?
• Service with what level of quality was the
  planned service delivered?
• Fiscal costs per client, or per service
  delivery
• Legal ethical considerations and fiscal
  accountability (any misuse of people or funds?)

10
Issues in evaluation

• validity
• concerned with soundness, effectiveness of
  measuring instrument
• are you measuring what you want to measure?
• reliability
• consistency with which a measuring instrument
  performs
• perspective
• the system or the client?