Harold Kooreman, M'A'

1
State-Level Evaluation

• Harold Kooreman, M.A.

Contact Information IUPUI Center for Health
Policy 334 North Senate Ave. Indianapolis, IN
46204 Phone [phone] Email
hkoorema_at_iupui.edu Web http//urbancenter.iupui.e
du/healthpolicy/ (temporary)
2
Requirements for the State

• State-Level Instrument (SLI)
• Developed by the national evaluators to monitor
  how each funded state implements and carries out
  the SPF process.
• The SLI is composed of in-depth semi-structured
  interviews administered by the federal evaluation
  team.
• The SLI is completed by representatives from
  state agencies involved in the SPF.
• The SLI is conducted on an annual basis.
• Funded communities do not participate in the SLI

3
Requirements for Communities

• The National Outcomes Measures (NOMS) are
  federally and state mandated outcomes measures to
  be collected on individuals served by the funded
  communities.
• The NOMS record client-level consumption and
  consequence data related to alcohol, tobacco and
  other drug use.
• The state-level evaluation team will be adding
  some additional state-specific questions to the
  NOMS to help compare funded communities to one
  another.

4
Requirements for Communities

• Data Collection for NOMS
• Collection of NOMS will begin once a community
  implements its program/intervention.
• Communities will need to collect NOMS every 6
  months till the end of the grant period.
• Because it is anticipated that communities will
  all implement different programs and/or
  interventions communities will work with their
  local evaluators to develop the best method for
  collecting NOMS data.

5
Requirements for Communities

• SPF-SIG Fidelity Assessment
• The SPF-SIG fidelity assessment is a federally
  mandated assessment of the SPF-SIG process within
  each community.
• The goal is to determine the impact level of
  fidelity to the SPF-SIG process has on outcomes.
• The fidelity assessment will be completed by the
  state-level evaluation team with the assistance
  of the local-level evaluators and key informants
  from each community.

6
Requirements for Communities

• Data for the fidelity evaluation will come from
  review of the following
• Budgets
• Meeting minutes, agendas, attendance rosters,
  organizational charts
• Results of needs assessments/capacity assessments
• Strategic plans
• Memorandums of understanding
• Intervention implementation materials (curricula,
  programs, etc).
• Evaluation plans, findings, reports
• Program fidelity information (e.g., number of
  sessions, was agenda followed, was material
  presented properly, etc.) collected by local
  evaluators
• CLI data from each community (provided by federal
  evaluators)
• Site visit data (provided by IDMHA)
• Interviews with key informants (project director,
  program staff, local evaluator, etc.) when
  necessary

7
Requirements for Communities

• Process Evaluation
• The purpose of the community-level SPF-SIG
  process evaluation is to complete a case study on
  each communitys movement through the steps of
  the SPF-SIG planning framework.
• The process evaluation will allow for general
  comparisons across communities and also highlight
  the unique characteristics of communities which
  may affect implementation of the SPF framework
  across the state.

8
Requirements for Communities

• Communities will develop a process evaluation
  with the help of their local evaluator.
• It is anticipated that much of the data for the
  process evaluation will come from the following
  sources
• Budgets
• Meeting minutes, agendas, attendance rosters,
  organizational charts
• Results of needs assessments/capacity assessments
• Strategic plans
• Memorandums of understanding
• Intervention implementation materials (curricula,
  programs, etc).
• Evaluation plans, findings, reports
• Program fidelity information (e.g., number of
  sessions, was agenda followed, was material
  presented properly, etc.) collected by local
  evaluators
• CLI data from each community (provided by federal
  evaluators)
• Site visit data (provided by IDMHA)
• Interviews with key informants (project director,
  program staff, local evaluator, etc.) when
  necessary.

9
Requirements for Communities

• Local-Level Outcomes Evaluation
• The outcomes evaluation is a federally mandated
  requirement.
• The local-level outcome evaluation will track the
  community-level changes in consumption behaviors
  and substance-use related consequences outlined
  in the community strategic plan.
• Each community will develop their outcome
  evaluation with the help of their local
  evaluator.
• Data collected for the outcome evaluation will
  vary based on the intervention selected but could
  include interviews of participants, observations
  of interventions, staff-level questionnaires,
  telephone surveys of residents, reviews of
  community-level data sets on substance use and
  consequences, etc.

10
Thank You !

• Center for Health Policy
• 334 North Senate Ave., Ste. 300
• Indianapolis, IN 46204
• Phone [phone]
• Fax [phone]
• E-mail ewright_at_iupui.edu
• Web http//urbancenter.iupui.edu/healthpolicy/
  (temporary)