Welcome to LIB5080

1
Welcome to LIB5080

• The School Library Media Program

2
LIB 5080 The School Library Media Program

• Agenda
• Program Planning and Evaluation

3
Planning Evaluating Media Programs

• Logic Models - Providing focus for planning and
  evaluating school library media programs

Based on SERVE Centers Capacity for Applying
Project Evaluation (CAPE) available online at
http//www.serve.org/Evaluation/Capacity/
4
Sharing Logic Models

• Explain your logic
• What did you discover as you were creating your
  logic model?

5
Remember that Logic Models .

• Help to articulate the key elements of a program
• Provide focus for program evaluation
• Can lead to evaluation efficiency and
  effectiveness
• Promote stakeholder buy-in by helping clarify how
  the project works

6
What Happens Next?

• Define evaluation questions
• Determine benchmarks
• Select methods and measures
• Conduct the evaluation
• Analyze and draw inferences from data
• Modify the program

7
Writing Evaluation Questions

• Consider the purpose of your evaluation, what do
  you really care about?
• Quality of implementation
• Impact

8
Writing Evaluation Questions

• Evaluation questions about strategies ask how
  well the strategy is implemented.
• Evaluation questions about objectives ask about
  the impact of implementing the strategy.

9
Writing Evaluation Questions

• Try to avoid simple yes or no questions.
• Consider quantity questions (e.g., how many,
  how much, or how often).
• Consider quality questions (e.g., how well,
  how effectively, or in what ways).
• Be sure questions are self-contained avoid
  putting implementation questions in terms of
  impacts.

10
Healthy Living Logic Model
11
Writing Evaluation Questions

• Examples
• Implementation questions
• How many hours of sleep am I getting each week?
  (quantity)
• How soundly am I sleeping? (quality)
• Impact question
• How much weight have I lost? (quantity)
• How has my stress level changed? (quality)

12
Writing Evaluation Questions

• Using your logic model, select one strategy and
  brainstorm implementation questions that you
  might ask, to evaluate how well it is being
  accomplished.
• Select one objective and brainstorm impact
  questions that you might ask, to evaluate how
  well it is being met.
• Share and discuss.

13
Writing Evaluation Questions

• Insert a strategy from your logic model into
• the worksheet

1.
14
Writing Evaluation Questions

• Write at least one implementation question for
• the strategy.

2.
15
Writing Evaluation Questions

• Insert an objective from your logic model into
• the worksheet

1.
16
Writing Evaluation Questions

• Write at least one impact question for the
  objective.

2.
17
Data Sources/Analysis Methods

• What do we mean by data?
• Data are pieces of information.
• Common understanding is that data are numbers
  quantitative data or quantities of things.
• Data may also be qualitative about qualities
  that are not easily measured in numbers.
• Both qualitative and quantitative data should be
  analyzed.
• Data are what we use to conduct formative
  evaluation of projects.

18
Data Sources/Analysis Methods

• What data do you need?
• Your data needs are dictated by your evaluation
  questions.
• Ask yourself, What information do I
• need to answer this question?
• Gather all of the data you need, and only the
  data you need, to answer your questions.

19
Data Sources/Analysis Methods

• Where do you get the data you need?
• Check whether there is data already available
• to you that might help answer your questions.
• Determine the data sources you might
• use to meet remaining data needs.
• Note that data sources are not data.
• Example teachers lessons plans are
• a rich data source, but it is necessary
• to do something with them before
• you have actual data.

20
Data Sources/Analysis Methods

• What are analysis methods?
• The approaches and tools used to pull data out of
  data sources
• It may be helpful to think in four steps
• Collecting data
• Storing and organizing data
• Analyzing data
• Interpreting data
• Qualitative data may be quantified.

21
Data Sources/Analysis Methods

• What is informal data?
• For an evaluation to be purposeful and
  systematic, data must be relatively
• formal, collected, stored and organized, and
  analyzed with some degree of rigor.
• Data not handled in this way may be thought of as
  informal.
• The trick is making informal data formal.

22
Data Sources/Analysis Methods

• Review each evaluation question you wrote

1.
23
Data Sources/Analysis Methods

• Identify the data source to answer the question.

2.
24
Specifying Benchmarks

• Benchmarks
• Are your targets.
• Define levels of success.
• Help you stop periodically to examine progress.

25
Specifying Benchmarks

• Look at the Healthy Living logic model.
• Listen to the possible benchmarks.
• What do you notice?
• How are data used to specify benchmarks?
• Are the expectations reasonable?
• Do you think all the benchmarks will be met?
• Is it okay if the benchmarks arent met?

26
Healthy Living Logic Model
27
Specifying Benchmarks

• Objective Lose weight.
• Benchmarking
• Baseline data
• current weight (June 2007) 165 lbs.
• Set target
• weight 1 year from now (June 2008) 145 lbs.
• Specify benchmarks
• periodically stop and examine progress.

28
Specifying Benchmarks

• Specify benchmarks periodically stop and
  examine progress.
• By September, I will weigh
• 160 lbs.
• By December, I will weigh
• 155 lbs.
• By March, I will weigh 150 lbs.
• By June, I will weigh 145 lbs.

29
Specifying Benchmarks

• Benchmarks one possible format (there are
  others)

Possible Format How many of who (or what) is
going to do (or be) what by when?
For Example 70 of teachers will complete a
collaborative unit with the media coordinator by
mid-year.
30
Specifying Benchmarks

• Consider the questions and data sources you
• have written

1.
1.
31
Specifying Benchmarks

• Specify one or more benchmark for each data
• source.

2.
1.
1.
32
Specifying Benchmarks

• What baseline data do you have?
• What if you dont have baseline data?
• In order to make adjustments you will want to
  collect, analyze, and compare data to benchmarks
  periodically
• e.g. bi-monthly, monthly, each 9-weeks, at the
  end of each semester

33
Specifying Benchmarks

• Be realistic and reasonable.
• Consider how much time it will take to collect
  and analyze the data.

34
Making Decisions

• What decisions might you make if evaluation
  findings show
• Problems with implementation?
• Good quality implementation?
• Positive impacts (benchmarks reached)?
• Problems with impacts (benchmarks not reached)?
• Who will make the decisions?
• Who will the decisions effect?

35
Making Decisions

• Use the questions, data sources, and benchmarks
• you have written .

1.
1.
1.
36
Making Decisions

• What decisions will you make based on
• your findings?

1.
1.
1.
2.
37
Evaluation Plan Assignment

• Work in your groups
• Revise your logic model
• Taking the role of a school MTAC, select one
  objective the school will focus on for the school
  year.
• Complete an Objective worksheet for that
  objective.
• Complete a Strategy worksheet for all strategies
  associated with that objective.

38
Homework

• Work on evaluation plan
• Work on Principalship/Action Learning Project