Clinical governance and evaluation

1
Clinical governance and evaluation

• 2 of the 7 pillars of clinical governance
• Clinical effectiveness
• Clinical audit

2
At PCT strategic level

• Strategy and programmes for clinical
  effectiveness work, including research to
  identify effective clinical practice
• Involvement of partners in clinical effectiveness
  strategy, development and programmes
• Involvement of patient/service users and carers
  in clinical effectiveness strategy development
  and programmes
• Resources (staff and budget) to support research,
  development and implementation of the effective
  clinical practice

3
At the PCT and practice level

• Implementation and application of effective
  clinical practice e.g. integrated care pathways
  evidence based guidelines for disease management
• Collection and distribution of evidence based
  practice to the relevant teams and staff,
  including
• results of the own organisations own research
• published evidence of effective practice,
  including NSFs and guidance issued by NICE
• Local research projects to identify effective
  clinical practice

4
At the practice and provider level

• Accessibility of research results and evidence of
  effective practice e.g. libraries Internet
  Journals Intranet (or other local electronic
  library)
• Training for staff - e.g.
• Critical appraisal skills
• Literature, database and internet search skills
• Use of agreed performance indicators

5
How do we make this happen for CAM services?
                                                 
   
 
Use existing evidence to design CAM services
provide a rationale identify standards of best
practice
Internal evidence generated from service
evaluation
adjust
But, published evidence is thin in CAM So ? to
establish EBP, initial evidence net must be cast
wide and new evidence needs to be generated and
gathered  
assess
Audit cycle
implement
measure
New external evidence
6
Real world policy making

• 124 decisions
• RCT evidence to support decision 33.9
• RCT evidence not supporting decision 1.6
• RCT evidence equivocal 12.9
• NO relevant RCT evidence found 51.6
• Source Berkshire HA and PCT commissioning
  planning documents 1997-1998
• But 41 other studies (non experimental) found
  to support decisions in 5 out of 10 sampled
  decisions.
• Johnstone and Lacey. J Health Serv Res Policy
  July 2002

7
What is evidence based practice?

• Evidence-based medicine (EBM) is the integration
  of best research evidence with clinical expertise
  and patient values.
• Hierarchies of evidence

8
Tudor Harts plea for a wider and more
socialised evidence base

• We need to work within a different paradigm
  based on development of patients as co-producers
  rather than consumers, promoting continuing
  output of heath gain through shared decisions
  using all relevant evidence, within a broader,
  socialised definition of science.

9
HoL Report the place of CAM in the modern NHS

• Evidence desirable before a therapy can be
  advocated
• Efficacy above and beyond the placebo effect
• Safety
• Cost-effectiveness
• Mechanisms of action
• Poor general state of CAM evidence

HoL 2000
10
Patient satisfaction as outcome

• Patient satisfaction has its place in as part of
  the evidence base for CAM.
• Move towards incorporating patient views in
  conventional evaluation (NICE)
• Alone it cannot be taken as proof or otherwise
  of a treatments efficacy or as evidence to
  justify its provision.

HoL 2000
11
Mechanisms of action

• Any medicine with credible, accepted evidence
  for efficacy should be available, whatever the
  controversy over its underlying mechanisms.

HoL 2000
12
Efficacy

• Group 1 some evidence for all therapies in
  particular contexts.
• Much more research needed.
• Group 2 many claims of efficacy. Many are
  greatly appreciated for the comfort they bring in
  palliative care.
• As Adjuncts to conventional medicine, these are
  in lesser need of proof of specific effects.

HoL 2000
13
Para 4.40

• It is our opinion that as long as the
  treatments are known to carry no, or few, adverse
  effects, it would be against the principal of
  clinical freedom to prevent patients having
  access to therapies which fulfil these criteria
  and have never been restricted. This is
  especially the case if the patients believe that
  such therapies help them and the only argument
  against them is that an evidence base, derived
  from controlled trials, does not exist. (4.40)
• HoL 2000

14
The Government response

• Sir John Pattison Director of NHS RD
• Government has set challenging priorities in
  cancer, mental health, CVD, and CHD, the elderly
  and children. It would be in those areas that we
  would welcome proposals for CAM approaches.
• E.g. reflexology for patients after surgery for
  early breast cancer

15
CAM to support NSF patient priority groups in
primary care
Currently providing
Possible contribution
GP practices in England 2001
MCRU, University of Sheffield
16
BEST CAM Reports?

• Broad Evidence Synthesis paves the way for
  thinking about developing topic reviews for CAM ?
  Best Evidence Synthesis Topics (BEST reviews)
• Developing BEST reviews for CAM ? what would
  they look like? What information would they
  contain?
• Effectiveness
• Safety
• Delivery
• Value for money
• Access - What sources can we draw on?

17
Evidence sources and links

• http//www.nelh.nhs.uk/
• National Electronic Library Health (Cochrane
  Library and Bandolier)
• http//www.cebm.utoronto.ca/
• Center for Evidence Based Medicine,University of
  Toronto

18
Evidence sources

• The Desktop Guide to Complementary and
  Alternative Medicine. Ernst (Ed) (2001)
• FACT Focus on Alternative and Complementary
  Therapies (Quarterly digest)
• The Comp Med Bulletin

19
Feedback from TASK 1

• Draft proforma ? structure within which to
  collate broad evidence
• Types of evidence wanted
• Resources needed internal external
• Priorities
• Vote on BEST CAM Reports as possible title.

20
Clinical Performance Indicators

• Accepted indices of good practice
• Allow benchmarking
• How can we measure our own performance?
• Who/what needs to be involved in order to
  identify appropriate PIs for CAM?
• How do we create consensus about how the service
  is to be judged?

21
Consensus building on clinical Performance
Indicators (PIs)
22

Feedback from TASK 2

• What kind of indicators are right for CAM?
• What would the ideal team look like?
• What would their unique contributions be?
• Who would lead exercise?
• Who needs to sign up to chosen PIs?
• How would this happen?

23
Where evidence gaps are uncovered

• Can the clinical governance process contribute to
  the evidence base where gaps are identified?
• Audit or research?
• Audit means measuring performance against prior
  standards ? research means providing evidence for
  creating new standards.
• Added value
• Evaluation/research results could be fed back
  then into main source of data e.g. BEST CAM
  report (if someone responsible for collating)
  evidence base will grow.

24
Evaluation research ? methods and tools

• How can services contribute to the evidence base
  for effective practice?
• At what point should research be included in the
  audit cycle?
• Limited resources anticipated, but contributions
  could be made to new knowledge.

25
How can the evidence gaps be filled?

• Whats realistic?
• Asking the right questions
• Not possible to answer all the outstanding
  questions in one study ?less is more in
  research, if done well and targeted usefully.

26
Outcome measures ? whats available?

• Generic measures of outcome
• Satisfaction measures
• Resource use (measuring reduction in prescribed
  medication or referrals).
• What is it that the patient needs? ? patient
  generated tools (MYMOP).
• Using standardised disease specific outcomes
• Costs and cost?effectiveness

27
Evaluation designs

• Observational studies
• Clinical, patient and cost outcomes
• Service delivery questions (access via a GP or
  open access, capped appointments etc).
• Comparative studies
• Clinical groups (acute v. chronic)
• Therapy groups (acupuncture v. physio)
• Proving v. improving study designs
• Level of support needed for different kinds of
  research

28
A caveat
Research
Research
Research
The (research) tail wagging the (service) dog
29
Feedback from TASK 3

• Realistic research / evaluation tasks and topics
• Types of resource and support needed?