Procedure

1
Evaluation for Real People
Drew L. Betz Laura G. Hill, Washington State
University Extension
Background
Hypotheses
Results

• Washington State University Extension selected
  evidence-based best-practice Strengthening
  Families Program for Parents and Youth 10-14 for
  statewide dissemination
• Needed strong evaluation component
• Examined needs of both research and field
  faculty
• Research faculty concerned about social
  desirability bias and favored true pretest model
• Field faculty concerned about response-shift
  bias and favored retrospective pretest model

Mean Pretest, Retrospective Pretest, and Posttest
Scores

• Posttest scores will show improvement from both
  true and retrospective pretests
• Improvement from pre to post will appear greater
  on items with socially desirable content
• Difference between true and retrospective
  pretests tests will also be greater on socially
  desirable items

Method
NOTE p lt .001 a Indicates column values
that are significantly different from one another

• Procedure
• Pretest administered 1st night
• Retrospective pretest and posttest last night
  
• Participants
• 100 adults from 15 SFP series completed true
  pretest, retrospective pretest, and posttest
• Measures
• Scales of Family Warmth, Consistency, Temper
  Management and Harshness modified. Half the
  items worded so that raters endorse desirable
  behaviors and the other half undesirable
• Examples of Desirable Items
• I enjoy spending time with my youth
• My youth talks to me when he or she is upset
• Examples of Undesirable Items
• It is hard for me to show love to my youth
• When Im upset with my youth I tend to blame and
  criticize him

Summary and Conclusions

• Both true and retrospective pretests demonstrate
  positive change
• Social desirability bias on retrospective
  pretests can be minimized with careful selection
  of items
• Aggregating statewide data allowed us to find
  significant differences between true pretest and
  both retrospective test and posttest
• Continue to use true pretest in combination with
  retrospective pretest
• Both have a valid place in real-world evaluation
• True pretest welcomed by research and funding
  world prepares parents for what to expect from
  program
• Retrospective evaluation allows parents to
  reflect on their learning and become engaged in
  evaluation process

Research Questions

• How would true pretest compare to retrospective
  pretest?
• Overall
• By item type

Contact Information

Laura Griner Hill PO Box 6236 Washington State
University Pullman, WA 99164 laurahill_at_wsu.edu
Drew Betz WSU Whatcom County Extension 1000 N
Forest St Suite 201 Bellingham WA
98225 betz_at_wsu.edu

Reference
Hill, L.G. Betz, D.L. Revisiting the
retrospective pretest. Manuscript under review.