Extension Methodology

1
Extension Methodology

• New Mexico State University
• IAER Project July 2007

2
Part VI Evaluation of Extension Programs
3
Evaluation

• Is a process that helps us understand and
  appreciate the relative merits and deficiencies
  of persons, groups, programs, situations,
  methods, and processes.
• An excellent source of additional materials is
• http//www.canr.msu.edu/evaluate/SourcesFiles/Trai
  ning20Manual.htm

4
Why should we evaluate?

• To prove something (accountability) - which is
  known as Summative Evaluation
• collect evidence for reports
• document achievement of objectives
• justify expenditures
• assess program impact and outcomes
• document that new practices were adopted
• increase visibility of programs
• promote public relations
• provide evidence of program accomplishments to
  stakeholders (especially funders)
• ensure that client expectations are met
• measure economic and other benefits to
  participants, communities, and society at large

5
Why should we evaluate?

• In a democracy, extension must be accountable to
  the people. This is particularly important in
  securing funding. In order to justify and secure
  appropriations, extension must show results and
  demonstrate efficiency.
• Economy and efficiency in the use of funds has
  been a tradition of the service. This grew
  partly from the integrity of officials
  responsible for administration and also from a
  close interest of rural clientele who are
  accustomed to practicing economy and thrift in
  their own affairs.

6
Why should we evaluate?

• 2. To improve something - which is known as
  Formative Evaluation
• find out what works or doesn't work with a
  particular audience and situation
• help educators grow professionally
• increase efficiency
• provide personal satisfaction

7
Why should we evaluate?

• Evaluations are seldom one or the other, their
  purposes usually overlap .
• Formative evaluations often occur while programs
  are in progress and are more likely to be
  something educators do on an informal basis.
• Summative evaluations are likely to occur after
  programs are over

8
The Process

• There are many different ways to evaluate, but
  they all include two elements
• gathering the evidence
• comparing the evidence with criteria
  (objectives).

Evidence Measurement and/or description
Judgment Comparison of evidence with criteria
Evaluation


The importance of clearly defined educational
objectives can hardly be over emphasized the
wording of the objectives directly affects the
type of evidence we must collect and record.
9
Objectives-Based Approach

• provides direction for the work
• serves as criteria for evaluation
• communicates expectations
• provides a framework for reports
• Extension is a system of education and therefore
  it is necessary to base its evaluation on
  educational objectives. These objectives must be
  stated in terms of changes in the behaviors and
  actions of people.

10
Behavior Change

• Behavior when considered as a result of education
  is usually not in terms of an immediate action.
  It is more likely
• an improvement in skills,
• a better understanding of a concept,
• an increased ability to solve problems,
• a changed attitude,
• an appreciation of different things,
• a shifting of values,
• a change of interest,
• an adoption of improve practices,
• or a general increase in knowledge.

11
Behavior Change

• Many behavior changes are difficult to measure
• They must be identified through the objectives
  before the evaluation begins
• See example in text
• Easier to measure behaviors that have tangible
  evidence
• Many times we must ask what a person has learned

12
Evidence

• May be in the form of numbers or words and is
  called data
• Quantitative data refers to evidence that can be
  expressed in numbers or statistics
• Qualitative data refers to information that
  describes program inputs and/or impacts in words
  or phrases or quotes
• Together quantitative and qualitative data
  support one another to create a more complete
  picture concerning program inputs and impacts.

13
Questionnaires

• See sample instrument
• Tips for constructing questionnaires
• Use the same scale for all questions people find
  it easier to respond.
• Scales with four or more points allow more
  insight to participants responses then yes or no
  answers.
• When rating people, especially if the purpose is
  to help people improve, words are less
  threatening and more encouraging than numbers.
• Put questions about age, gender, and income at
  the end. Beginning questions should be more
  interesting than "How old are you?"
• Take out any unneeded questions and make the
  instrument as short as possible.
• If a question is asking about two things, divide
  it into two questions.
• Choose words or phrases with very clear meanings
  -- for example, "Did you increase your net
  income?" instead of "Did you make more money?"

14
Analyzing the Evidence

• Extension in the US uses mostly descriptive
  statistics and qualitative/anecdotal data for the
  judgment and reporting phases of the evaluation
  process.
• Descriptive statistics include sums and
  percentages such as
• local cotton farmers increased production by 5
• 45 mothers attended a class on how to prepare
  nutritious meals for their children under five
  years of age
• 60 of the respondents to the workshop evaluation
  questionnaire indicated that they had increased
  or significantly increased their knowledge on
  orchard management

15
Analyzing the Evidence cont.

• Qualitative/anecdotal data include testimony and
  personal observations such as
• two participants personally expressed that they
  had learned more at this workshop than any other
  they had attended
• over the course of the three-month program, the
  coordinator said that she observed an overall
  increase in the enthusiasm for exercise during
  the school day.
• Formal statistical analysis is used on larger and
  more complex programs, mostly at the state and
  federal levels.

16
Reporting

• The evaluation process is not complete until the
  results are reported.
• The reporting process helps to keep divergent
  activities in check and promotes coordination of
  program building with daily work and final
  results.
• A common mistake is to assume that one type of
  report will fit all purposes. The format of the
  report needs to fit the audience.
• Another common mistake is waiting until the end
  of an evaluation to report. It may be necessary,
  especially for large evaluations, to provide
  progress reports which can take many different
  forms depending on the audience.

17
Summary

• Evaluation helps to establish a "benchmark"
• Shows how far our plans have progressed
• Shows whether we are proceeding in the right
  direction
• Indicates the effectiveness of a program
• Helps locate strong and weak points in any
  program or plan
• Improves our skill in working with people
• Helps to determine priorities for activities in
  the plan of work
• Brings confidence and satisfaction to our work

18
Discussion Questions

• Choose one of your current extension programs and
  decide whether formative or summative evaluation
  would be more appropriate and explain why.
• How will the accountability mandates from local
  and national governments affect the design of
  your extension evaluations?