Health Program Evaluation Data Analysis

1
Health Program EvaluationData Analysis

• CHSC 433
• Module 5/Chapter 11
• L. Michele Issel, PhD
• UIC School of Public Health

2
Objectives

• Calculate response rates
• Determine whether to use parametric or
  non-parametric statistical test
• Distinguish among the types of significance

3
Response Rates

• Calculate rate of response to survey
• original sample - ( ineligibles) -(
  non-responders)
• completed by eligibles
• eligibles

4
Basic Change Calculation

• Amount of Change
• score for program participants -
• score for control (non-participants)
• OR
• Amount of Change
• score on post test - score on pretest

5
How much Impact?

• Net impact is intervention effects only
• Gross impact is intervention effects, other
  effects, design effects

6
Factors that Contribute to Change (from Green and
Lewis, 1986)

• See the figure on the next slide
• All the internal and external validity factors
  affect the amount of change (program effect) that
  is detectable

7
(No Transcript)
8
Chose the Statistical Test based on

• 1. Focus of the evaluation question comparison,
  association among variables, or prediction of
  outcomes
• 2. Analysis level will be used individual,
  aggregate, or population
• 3. Measurement level of measurement used for the
  dependent and independent variables nominal,
  ordinal, interval/ratio
• Interval/ratio measures have a normal/parametric
  distribution or not

9
Choosing Stats test continued

• 4. Design used non-experimental,
  quasi-experimental with one group,
  quasi-experimental with two or more groups,
  quasi-experimental with other design, or
  experimental
• 5. Data from sample or a population
• 6. Interest and capacity of the stakeholders to
  understanding statistical analyses?

10
Choosing Statistical Tests

• Parametric or non
• Based on distribution
• Based on variable type
• Complexity of the Question
• Compare
• Association
• Prediction

11
Curves

• Its about distributions curves
• Normal distribution
• Bell shaped
• Abnormal distribution
• Lopsided or flat
• Not a distribution
• Yes/no, dead/alive

12
Diagnosing Abnormal

• Skewness
• Peak is off-center
• To the right or left
• Implies have outliers
• Kurtosis
• Steep peak
• Flat

13
Levels of Measurement
14
Levels of Measurement Examples of DV by IV
15
Parametric Stats by Complexity of Question
16
Non-Parametric Stats by Complexity of Question
17
Nominal Dependent Variable
18
Ordinal Dependent Variable
19
Continuous Dependent Variable
20
Significance

• Statistical significance - less likely than by
  chance
• Clinical significance - the potential to have
  noticeable benefit
• Statistical and Clinical do NOT always overlap!
• Which is more important to the evaluation, to the
  stakeholders?

21
Analysis of Data Across the Pyramid