CONTINUOUS QUALITY IMPROVEMENT (CQI)/ Program Evaluation/ Efficacy Research

1
CONTINUOUS QUALITY IMPROVEMENT (CQI)/Program
Evaluation/Efficacy Research

• KNR 278

2
Evaluation

• Integral part of program planning
  implementation
• Can be conducted on a variety of levels
• Individual client
• Specific activity
• Specific program
• Comprehensive program

3
Evaluation

• Represented on TRAM in 3 areas
• See page 364
• One of the major ways professionals are held
  accountable

4
Evaluation Defined

• Systematic and logical process of gathering and
  analyzing selected information in order to make
  decisions about the quality, effectiveness,
  and/or outcomes of a program, function, or
  service
• Peterson Stumbo (2000) p. 288

5
Evaluation

• Generic evaluation design model (p. 368)
• Evaluation plan should be created during the
  program planning process
• Internal external
• Formative summative
• Planning evaluation (p. 366-374)

6
TR Evaluation

• Client outcomes evaluation
• Specific program evaluation
• Connollys Formative Program Evaluation Procedure
  (FPEP)
• Comprehensive Program Evaluation
• CQI
• Efficacy Research

7
ATRA Priority Ranking Efficacy Research Items

• TR on community integration
• TR on recidivism/prevention of further illness
• TR on independent functioning
• Leisure functioning on health
• TR on hospital costs/lengths of stay
• Increased efficacy and control through TR on
  rehab outcomes

8
ATRA Priority Ranking Efficacy Research Items

• Leisure education on quality of life
• TR on adjustment to disability
• TR on depression
• Leisure ed on independent leisure functioning
• TR on life satisfaction
• TR on socially appropriate behavior

9
ATRA Priority Ranking Efficacy Research Items

• TR on d-making planning ability
• Leisure participation addiction recovery
• Leisure education delinquency reduction

10
How is quality defined?

• Brainstorm for restaurant
• Quality per Peterson Stumbo (2000)
• Degree of adherence to standards of good practice

11
How do you define quality?

• Stakeholders are satisfied (satisfaction)
• Consumer goal/objectives are met (effectiveness)
• Efficient use of resources
• Appropriateness of services (utilization review)
• Safe environment/limited risk
• Standards of professional practice
• Malik, 2001

12
QA vs. CQI

• Quality Assurance (QA)
• Old term
• Can this organization provide quality health
  care?
• Schlenghe, 1991
• Focus on structure and maybe process

13
QA vs. CQI

• Continuous Quality Improvement (CQI)
• New term
• Does this organization provide quality health
  care?
• Demonstrate the client changed as a result of
  care
• Schlenghe, 1991
• Assume quality and always room to improve
• Increasing focus on outcomes

14
What should CQI or outcome measures evaluate?

• Professional paraprofessional performance
• Efficient use of resources
• Reduction of risk
• Consumer/stakeholder satisfaction
• Consumer outcomes
• Malik, 2001

15
CARF Requirement

• For each service for which it is seeking
  accreditation, the organization develops, at a
  minimum, one measure of effectiveness, one
  measure of efficiency, and one measure of the
  overall satisfaction of persons receiving
  services (Section 2.B-32, 1998) includes
  individuals served and other stakeholders
• Malik, 2001

16
Identify Program Stakeholders

• Identify all stakeholders for your program
• Prioritize the top 3
• Identify 3 quality outcome expectations
  (results!) for top 3 stakeholders
• Malik, 2001

17
Structure Measures /Efficiency

• Has to do with resources required for care
• Supplies, policy guidelines, staffconsumer
  ration, space for programming, etc.
• Identify 3-5 structure measures that you expect
  your agency to have
• Malik, 2001

18
Process Measures/Efficiency

• Reflects direct intervention techniques
• Assessment process, interaction between staff and
  consumers, etc.
• Reflects how things are done
• Assessments completed in a timely manner,
  incident reports, measurable goals objectives,
  etc.
• Identify 3-5 process measures that you expect of
  a quality program
• Malik, 2001

19
Outcome Measures/Effectiveness

• Consequences of intervention the end result
• Change in knowledge, skills, abilities
• Identify 3-5 outcome measures from your program
• Malik, 2001

20
Consumer/stakeholder Satisfaction

• Level of satisfaction with services, supports and
  programs provided
• How would you measure satisfaction?
• How often?
• Would you do anything differently between
  stakeholders?
• Malik, 2001

21
Ten Step Process of CQI

• Step 1 Assign responsibility
• Determine who is responsible for implementing and
  evaluating the CQI plan
• Step 2 Delineate scope of care
• Identify and determine the purpose of each
  program area
• Determine the purpose of the CQI plan

22
Ten Step Process (Cont.)

• Step 3 Identify important aspects/elements of
  care
• Important functions of TR
• Necessary elements of TR process
• Major responsibilities of TR professional
• High risk
• High volume
• Problem prone
• Outcomes identified previously today

23
Ten Step Process (Cont.)

• Step 4 Identify indicators
• For each aspect of care, one or more measurable
  indicators are identified. The indicator measure
  is either process or outcome oriented.

24
Ten Step Process (Cont.)

• Step 5 Establish criteria or thresholds
• Thresholds are established for each indicator as
  expectations of what constitutes good and
  desirable performance
• These are usually written as .

25
Ten Step Process (Cont.)

• Step 6 Collect organize data
• Determines the sample size, frequency, and time
  frame for collecting data
• Step 7 Evaluate care/results of study
• Evaluate the accumulated data to determine if
  threshold was reached. State findings and
  conclusions, opportunities for improvement, and
  problems identified.

26
Ten Step Process (Cont.)

• Step 8 Take action
• Based on the indicator, appropriate actions are
  taken
• In-service training, 1-1 staff counseling, etc.
• Step 9 Assess effectiveness of action
• Step 10 Communicate information

27
Quality Management

• Best practices
• Best way to perform business or process
• Explore what others have done
• Benchmarking
• Measure your performance compared to someone
  elses performance on pre-identified measurements

28
New Trends

• Interdisciplinary/entire team
• 2 levels of analysis of quality
• Micro Department or facility level
• Macro Regional, national, international level
• Computerized data sets
• Resident Assessment Instruments Minimum Data Set
  (MDS)
• Long term care
• burlingame Blaschko, 2002
• www.medicare.gov