Making a difference - the CAMHS raison d

1
Making a difference - the CAMHS raison detre

• CAMHS Day 2 Training in Routine Outcome
  Measurement
• Peter Brann April 2003
• Eastern Health CAMHS
• Monash University Departments of Psychological
  Medicine and Psychology

2
And you say to yourself, well how did I get
here...

• The key question for health service delivery
• who receives
• what services
• from whom
• at what cost
• and with what effect?
• Leginski et al (1989)

• Too much data collection has been about
  perfecting the unexamined life - implementing
  processes without knowing their impact
• Local CAMHS experience
• Bickman et al. national report
• National Outcomes and Casemix Classification
  (NOCC)

(NOCC)
3
Outcomes and casemix measures for children and
adolescents

• Clinician rated
• Health of the Nation Outcome Scales for Children
  and Adolescents (HoNOSCA)
• Children's Global Assessment Scale (CGAS)
• ICD-10 Factors Influencing Health Status (FIHS)
• Consumer and carer self-report
• Strengths and Difficulties Questionnaire (SDQ)
• Common clinical data
• Diagnosis (chiefly responsible for care)
• Mental health legal status (involuntary?)

4
Why measure outcome?

• Clinically
• Inform treatment decisions by highlighting
  unexpected progress or deterioration
• Document progress of long-term cases
• Overt change during contact
• Prioritise supervision and reviews
• Service-wide
• Standardise assessment of strengths and
  difficulties
• Document program or service effectiveness
• Assess impact of training, procedures and
  policies
• Highlight strengths and weaknesses in outcomes
• Ground evidence-based treatments in local
  populations

5
Why measure casemix?

• Variation in outcomes cannot be understood
  without understanding the complexity of cases
• Myths about casemix
• 1 Casemix is a method of cutting costs
• Managers have used many methods of cutting costs
  - casemix may or may not be used but it is closer
  to patient needs than old boy networks
• 2 Casemix reduces quality
• Depends on how it is used. It may help ensure
  that treatment for complicated clients is not
  reduced to the lowest common denominator.

6
CAMHS processes and NOCC
Intake
Assessment
Review
Discharge
7
NOCC collection occasions
8
NOCC Episodes, cases, and collection occasions
Case 1
Case 2
Period of no care
6 mth review
Discharge from community
Discharge from inpatient
Intake to community
Intake to community
Admission to inpatient
Collection occasions
9
The case against leaving lounge suites covered in
plastic...

• Routine Outcome Measurement use it or lose it

10
Review checklist - example
11
Whats the point of knowing clinicians views?
12
HoNOSCA scores over time

• Over five years of outcome data
• Clear perception of clinicians?making a
  difference
• Equates to reductions in the number of scales
  with clinically significant scores

13
Changes from parents and adolescents
perspective, also!

• Note similarity to clinician measures of parent
  and adolescents SDQ
• Adolescents tend to rate themselves as less
  severe
• Now to the outliers

14
The professional development question and beyond
Average
15
Stand by your work and take a GLHL

• On average, we think we do better with some
  referrers
• And possibly better than some referrers think
• Why the variation?
• Complexity? Diagnosis? Iatrogenesis?

16
Use of feedback

• Used individually rather than with others
• Generally not included in review processes
• Reflects individualised structures within which
  feedback was provided (to case managers)

17
Never mind the psychometricsWhat do you think
of it?

• Survey of all EHCAMHS clinicians using HoNOSCA
• 85 return rate (42/48)
• True believers not necessary
• Average use 14 months
• Average time three mins

18
How well does HoNOSCA reflect symptoms and
functioning?

• 94 looked at the graphs
• Majority looked in moderate detail
• Seen to reflect symptoms and functioning well to
  very well by 60
• Seen as poor reflection by 11

19
How is it used clinically?

• Assessment
• Focus on problems rather than referrers
  perception
• Consider diagnostic options
• Reviews
• Structure reviews
• Compare progress
• Families/Carers
• Share perceptions of progress or not

• Treatment
• Prompts case reflection
• Discharge
• Consider timing
• Teams and services
• Strengths and weaknesses
• Professional development

20
What difference has it made?

• Other uses?
• Research
• Compare diagnostic groups
• Clinical
• Examine cases that do not change
• External system
• Demonstrate effectiveness - lobby for resources

• Difference to practice
• Clinical
• Reflective and structured space
• Providing perspective to parents
• Administrative
• More paperwork
• Awareness of others interest in outcome

21
Clinical file audits process and outcome?
Clarity of documentation
Reflects
Clarity of assessment and treatment
Results in
Better and more consistent outcomes
22
Perfecting the unexamined life

• Structure, process and outcomes as hypothesis
• Does properly implemented care lead to better
  subsequent health status (and functioning,
  satisfaction) and if not, which (combination?) of
  the two should we measure and rely upon
• What is properly implemented care?
• Advantages of a routine system is this can be
  commented upon locally
• Concluding

A neat file is a neat file, is a neat file, is
a neat file, is a neat file
23
What is good enough treatment?
24
No names in public and only that which is
relevant, thank you
25
HoNOSCA

• Overting clinicians perceptions of symptoms and
  functioning

26
What Is HoNOSCA?(Health of the Nation Outcome
Scales for Children and Adolescents)

• Clinician rated 0-4 point scale
• 13 items contribute to the total 2 optional
  items

Items (1-13 Core Scale)
Symptoms
Behaviour
7. Hallucinations 8. Somatic 9. Emotional 10.
Peer 11. Self-care 12. Family 13. School
1. Disruptive... 2. Concentration 3.
Self-injury 4. Substance misuse 5.
Scholastic/language skills 6. Disability/illness
Social Functioning
Impairment
Knowledge 14. Nature of difficulties 15.
Services available
27
Key HoNOSCA principles

• Rate each scale in order.
• Rate a problem or symptom only once (aggression
  for aggression, not to infer substance use).
• Use two weeks prior for community-based rating
  period (use three days for discharge from
  in-patient unit).
• Rate the most severe occurrence (even if two
  problems relate to the same scale).
• HoNOSCA is not an interview schedule. Use
  HoNOSCA to reflect your judgement from all
  sources.
• Clinically significant symptoms rate at least 2.
• Ratings do not equal active intervention.
• Preferably use same rater.
• Scales 1-9 use most severe example while Scales
  10-13 use typical level of functioning in rating
  period.

28
Scoring HoNOSCA

• Is it clinically significant?
• Severity
• Functional impairment
• Distress and burden
• Developmental pathway
• If confused by the glossary, remember ITS AN
  ORDINAL SCALE

29
HoNOSCA response sheet (For Paul V.)
30
Rating HoNOSCA

• Paul vignette walk through
• Melinda vignette
• When you encounter expert scores, remember
  expertise is relative
• The critical issue is
• Clinically significant vs non-significant
• Then the extent of impairment and severity
• The level of agreement required, which is a
  function of the purposes for which the score will
  be used

31
Paul V (the modified version)

• Paul is a 9 year old boy who lives with his
  family

32
1. Problems with disruptive, antisocial or
aggressive behaviour

• Include behaviour associated with any disorder,
  such as hyperkinetic disorder, depression,
  autism, drugs or alcohol.
• Include physical or verbal aggression (pushing,
  hitting, vandalism, teasing), or physical or
  sexual abuse of other children.
• Include antisocial behaviour (thieving, lying,
  cheating) or oppositional behaviour (defiance,
  opposition to authority or tantrums).
• Do not include overactivity rated at Scale 2
  truancy rated at Scale 13 self-harm at Scale 3.
• 0 No problems of this kind during the period
  rated
• Minor quarrelling, demanding behaviour, undue
  irritability, lying, etc
• Mild but definitely disruptive or antisocial
  behaviour, lesser damage to property, or
  aggression, or defiant behaviour
• Moderately severe aggressive behaviour, such as
  fighting, persistently threatening, oppositional,
  serious destruction of property, or moderately
  delinquent acts
• Disruptive in almost all activities, or at least
  one serious physical attack on others or animals,
  or serious destruction of property

33

• There are reports from teachers that during lunch
  breaks Paul has been pushing other boys off the
  swings
• Clinically significant ?
• Rate 2-3

34
Scale 1 Comments

• The behaviour is the issue, not the presumed
  cause, insight or intention
• aggression in the context of a psychotic disorder
  is rated here - the aggression does not
  contribute to the scale 7 rating
• The most severe incident in the relevant time
  frame (two weeks) is used for the rating
• Frequency does not deflate or inflate the rating
• Context may be considered to modify the
  disruptive nature of some behaviours, such as
  disagreements

35
2. Problems with overactivity, attention or
concentration

• Include overactive behaviour associated with any
  disorder such as hyperkinetic disorder, mania or
  arising from drugs.
• Include problems with restlessness, fidgeting,
  inattention or concentration due to any cause,
  including depression.
• 0 No problems of this kind during the period
  rated
• 1 Slight overactivity or minor restlessness, etc
• 2 Mild but definite overactivity or attention
  problems, but can usually be controlled
• 3 Moderately severe overactivity or attention
  problems that are sometimes uncontrollable
• 4 Severe overactivity or attention problems that
  are present in most activities and almost never
  controllable

36

• During class, Paul is well mannered and well
• behaved. He attends and participates well in
• Class.
• Clinically significant ?
• Rate 0

37
Scale 2 Comments

• Concentration difficulties for all disorders
  included here
• This is not the attention deficit scale
• A diagnosis of ADHD does not necessarily equate
  to an elevated score
• Conversely, many other disorders and
  presentations involve attention problems
• Remember, HoNOSCA is not diagnostically driven
• Manners are not the issue - good manners and good
  concentration are not equivalent

38
3. Non-accidental self-injury

• Include self-harm such as hitting and cutting
  self, suicide attempts, overdoses, hanging,
  drowning, etc.
• Do not include scratching, picking as a direct
  result of physical illness rated at Scale 6.
• Do not include accidental self-injury due, for
  example, to severe learning or physical
  disability, rated at Scale 6.
• Do not include illness or injury as a direct
  consequence of drug or alcohol use, rated at
  Scale 6.
• 0 No problems of this kind during the period
  rated
• Occasional thoughts about death or self-harm not
  leading to injury. No self-harm or suicidal
  thoughts
• Non-hazardous self-harm, such as wrist
  scratching, whether or not associated with
  suicidal thoughts
• Moderately severe suicidal intent (including
  preparatory acts such as collecting tablets) or
  moderate non-hazardous self-harm (such as small
  overdose)
• Serious suicidal attempt (such as serious
  overdose) or serious deliberate self-injury

39

• Pauls grandmother died at the start of term. His
  school counsellor said that at this time he often
  asked what would it be like to be dead. He hasnt
  asked this question in two months, and denies any
  thoughts on enquiry.
• Clinically significant ?
• Rate 0 (thoughts not in rating period -
  clinically I would follow up this issue rather
  than rely on school reports)

40
Scale 3 Comments

• As with all risk assessment, intention and
  likelihood of harm are relevant
• For example, using a fatal method, even though
  prevented, would still receive a severe rating.
• Although serious consequences may accompany a
  self-harm attempt, the rating may be deflated by
  clear intention to achieve minimal impact.
• Non suicidal self-harm included here

41
4. Problems with alcohol, substance or solvent
misuse

• Include problems with alcohol, substance or
  solvent misuse, taking into account current age
  and societal norms.
• Do not include aggressive or disruptive behaviour
  due to alcohol or drug use, rated at Scale 1.
• Do not include physical illness or disability due
  to alcohol or drug use, rated at Scale 6
• 0 No problems of this kind during the period
  rated
• Minor alcohol or drug use, within age norms
• Mildly excessive alcohol or drug use
• Moderately severe drug or alcohol problems
  significantly out of keeping with age norms
• Severe drug or alcohol problems leading to
  dependency or incapacity

42

• Paul had to see the headmaster last week because
  he was caught smoking at the bus stop. His
  teacher believes that he has been smoking since.
• Clinically significant ?
• Rate 2-3 (he is 9 years old)

43
Scale 4 Comments

• The key complexity here is that norms for the
  same drug use vary across ages and societal norms
• For example, tobacco is not a problem for which
  age group?
• However, even where a substance is commonly used
  (alcohol, marijuana), the issue is the extent of
  interference in their life.
• Consequences of substance use (aggressive
  behaviour, concentration difficulties) should be
  marked at their respective scales.

44
5. Problems with scholastic or language skills

• Include problems in reading, spelling,
  arithmetic, speech or language associated with
  disorder or problem, such as specific
  developmental learning problems or physical
  disability such as hearing problems.
• Include reduced scholastic performance associated
  with emotional or behavioural problems.
• Children with generalised learning disability
  should not be included unless their functioning
  is below the expected level.
• Do not include temporary problems due to
  inadequate education.
• 0 No problems of this kind during the period
  rated
• 1 Minor impairment within the normal range of
  variation
• 2 Minor but definite impairment of clinical
  significance
• 3 Moderately severe problems, below the level
  expected on the basis of mental age, past
  performance, or physical disability
• 4 Severe impairment, much below the level
  expected on the basis of mental age, past
  performance, or physical disability

45

• In the past two weeks Paul has not completed his
  homework. He has failed his last two Friday maths
  tests. He usually performs very well on these
  tests and hands in his homework.
• Clinically significant ?
• Rate 3

46
Scale 5 Comments

• Issue is under-performance compared with
  expectation. Past performance may be a guide
• Scholastic difficulties may arise from many
  causes
• Presentations of intellectual disability and
  autism are often concerning for people here
• If the performance is consistent with typical
  intellectual functioning, then it is not rated as
  a problem in the rating period.
• This is not a pseudo IQ rating

47
6. Physical illness or disability problems

• Include physical illness or disability problems
  that limit or prevent movement, impair sight or
  hearing, interfere with personal functioning.
• Include movement disorder, side effects from
  medication, physical effects from drug or alcohol
  use or physical complications of psychological
  disorders such as severe weight loss.
• Include self-injury due to severe learning
  disability or as of consequence of self-injury
  such as head banging.
• Do not include somatic complaints with no organic
  basis, rated at Scale 8.
• 0 No incapacity due to physical health problems
  during the period rated
• 1 Slight incapacity due to health problems (eg.
  cold, non-serious fall)
• 2 Physical health problem that imposes mild but
  definite functional restriction
• 3 Moderate degree of restriction on activity due
  to physical health problems
• 4 Complete or severe incapacity due to physical
  health problems

48

• Paul broke his arm six weeks ago. His arm is in a
  plaster cast on his non-dominant hand.
• Clinically significant ?
• Rate 2

49
Scale 6 Comments

• The issue is the recent effect of the disability
  or illness on the child
• Include physical complications of other
  psychological disorders here (for example, eating
  disorder impact on physical functioning)
• If in remission, rate the most severe impairment
  during the rating period

50
7. Problems associated with hallucinations,
delusions or abnormal perceptions

• Include hallucations, delusions or abnormal
  perceptions.
• Include bizarre behaviour associated with
  hallucinations and delusions
• Include problems with other abnormal perceptions
  such as illusions or pseudo-hallucations, or
  overvalued ideas such as distorted body image,
  suspicious or paranoid thoughts.
• Do not include disruptive/aggressive behaviour
  associated with hallucations or delusions, rated
  at Scale 1
• Do not include overactive behaviour associated
  with hallucinations or delusions, rated at Scale
  2
• 0 No evidence of abnormal thoughts or perceptions
  during the period rated
• 1 Somewhat odd or eccentric beliefs not in
  keeping with cultural norms
• 2 Abnormal thoughts or perceptions (paranoid
  ideas, illusions or body image disturbance), but
  little distress or manifestation in bizarre
  behaviour, that is, clinically present but mild
• 3 Moderate preoccupation with abnormal thoughts,
  perceptions or delusions, hallucinations causing
  distress or manifested in bizarre behaviour
• 4 Mental state and behaviour seriously and
  adversely affected by delusions, hallucinations
  or abnormal perceptions, with severe impact on
  person or others

51

• Paul's mother tells you that he sometimes tells
  her, he can hear the sound of one hand
  clapping. Paul's mother thinks this is very
  strange.
• Clinically significant ?
• Rate 2

52
Scale 7 Comments

• The key is the distress and/or the impact of
  these experiences on the child or adolescent
• Abnormal perceptions are included here, this is
  not a psychosis scale
• Abnormal perceptions and beliefs must take
  developmental stage into account (like all
  assessment)

53
8. Problems with non-organic somatic symptoms

• Include problems with gastrointestinal symptoms,
  such as non-organic vomiting, cardiovascular or
  neurological symptoms or non-organic enuresis and
  encopresis or sleep problems or chronic fatigue.
• Do not include movement disorders such as tics,
  rated at Scale 6.
• Do not include physical illnesses that complicate
  non-organic somatic symptoms, rated at Scale 6.
• 0 No problems of this kind during the period
  rated
• 1 Slight problems only, such as occasional
  enuresis, minor sleep problems, headaches or
  stomach aches without organic basis
• 2 Mild but definite problem with non-organic
  somatic symptoms
• 3 Moderately severe, symptoms produce a moderate
  degree of restriction in some activities
• 4 Very severe problems or symptoms persist into
  most activities the child or adolescent is
  seriously or adversely affected

54

• Paul tells you that last Tuesday night, the
  sound of one hand clapping kept him awake. This
  has happened a few times.
• Clinically significant ?
• Rate 2-3 (A 9 year old should sleep without this
  sound keeping him awake)

55
Scale 8 Comments

• The presumption is made that the symptom has no
  organic basis otherwise it should appear at Scale
  6
• This can be a fine line but is usually
  satisfactory in practice because the symptom will
  appear in one location or another
• This can include encopresis, sleep symptoms,
  gastrointestinal problems

56
9. Problems with emotional and related symptoms

• Rate only the most severe clinical problem not
  considered previously.
• Include depression, anxiety, worries, fears,
  phobias. Obsessions or compulsions, arising from
  clinical condition, including eating disorders.
• Do not include aggressive, destructive or
  overactivity behaviours attributed to fears or
  phobias, rated at Scale 1.
• Do not include physical complications of
  psychological disorders, such as severe weight
  loss, rated at Scale 6.
• 0 No evidence of depression, anxiety, fears or
  phobias during the period
• 1 Mildly anxious, gloomy, or transient mood
  changes
• 2 A mild but definite emotional symptom is
  clinically present, but is not preoccupying
• 3 Moderately severe emotional symptoms, which are
  preoccupying, intrude into some activities, and
  are uncontrollable at least sometimes
• 4 Severe emotional symptoms which intrude into
  all activities and are nearly always
  uncontrollable

57

• Paul's father tells you that Paul seemed very
  agitated last week when they went to the
  football. He has seemed more anxious than usual.
  But he says, Paul has always been a nervous kind
  of kid.
• Clinically significant ?
• Rate 2-3 (the always nervous does not reduce
  the rating)

58
Scale 9 Comments

• Anxiety, depression, obsessions, compulsions may
  occur here but only the most severe is used for
  the rating

59
10. Problems with peer relationships

• Include problems with school mates and social
  network, problems associated with active or
  passive withdrawal from social relationships or
  problems with over intrusiveness or with the
  ability to form satisfying peer relationships.
• Include social rejection as a result of
  aggressive behaviour or bullying.
• Do not include aggressive behaviour, bullying,
  rated at Scale 1.
• Do not include problems with family or siblings
  rated at Scale 12.
• 0 No significant problems during the period rated
• 1 Either transient or slight problems, occasional
  social withdrawal
• 2 Mild but definite problems in making or
  sustaining peer relationships. Problems causing
  distress due to social withdrawal,
  over-intrusiveness, rejection or being bullied
• 3 Moderate problems due to active or passive
  withdrawal from social relationships,
  over-intrusiveness, or to relationships that
  provide little or no comfort or support, such as
  a result of being severely bullied
• 4 Severe social isolation with hardly any friends
  due to inability to communicate socially or
  withdrawal from social relationships

60

• Paul says he had a fight last week with his best
  friend. He cant explain what precipitated the
  fight other than to say they shouldn't treat me
  bad. His teacher suggests that his friends are
  spending less time with him.
• Clinically significant ?
• Rate 2-3 (Key issue is relationships with friends)

61
Scale 10 Comments

• Unlike the preceding scales 1-9, which are
  concerned with the most severe marker in the
  two-week period, scales 10-13 are concerned with
  the average level of functioning during the
  rating period
• In Australia, this scale has shown the weakest
  absolute and consistency reliability estimates
• Concerned with the quality and appropriateness of
  the social network, school friends

62
11. Problems with self-care and independence

• Rate the overall level of functioning problems
  with basic activities of self-care such as
  feeding, washing, dressing, toilet and also
  complex skills such as managing money, travelling
  independently, shopping, taking into account the
  norm for the childs chronological age.
• Include poor levels of functioning arising from
  lack of motivation, mood or any other disorder.
• Do not include lack of opportunities for
  exercising intact abilities and skills, as might
  occur in an overrestrictive family, rated at
  Scale 12.
• Do not include enuresis and encopresis, rated at
  Scale 8.
• 0 No problems of this kind during the period
  good ability to function in all areas
• 1 Minor problems, such as untidy, disorganised
• 2 Self-care adequate, but major inability to
  perform one or more complex skills
• 3 Major problems in one or more areas of
  self-care (eating, washing, dressing) or major
  inability to perform several complex skills
• 4 Severe disability in all or nearly all areas of
  self-care or complex skills

63

• Wearing his new track pants, Paul caught the bus
  by himself to the movies last weekend to meet
  friends.
• Clinically significant ?
• Rate 0

64
Scale 11 Comments

• Rating performance rather than competence
• Need to have a sense of normative expectations
• Unlike the preceding scales 1-9 which are
  concerned with the most severe marker in the
  typically two-week period, scales 10-13 are
  concerned with the average level of functioning
  during the rating period.

65
12. Problems with family life and relationships

• Include parent-child and sibling relationship
  problems.
• Include rships with foster parents, social works
  or teachers in residential placements in the
  home with separated parents and siblings.
  Parental personality problems, mental illness,
  marital difficulties should only be rated here if
  they have an effect on the child/adolescent.
• Include problems such as poor communication,
  arguments, verbal or physical hostility,
  criticism and denigration, parental neglect or
  rejection, overrestriction, sexual or physical
  abuse.
• Include sibling jealousy, physical or coercive
  sexual abuse by sibling.
• Include problems with enmeshment and
  overprotection.
• Include problems with family bereavement leading
  to reorganisation.
• Do not include aggressive behaviour by the child
  or adolescent, rated at Scale 1
• 0 No problems during the period rated
• 1 Slight or transient problems
• 2 Mild but definite problem, such as episodes of
  neglect, hostility, enmeshment
• 3 Moderate problems, such as neglect, abuse,
  hostility. Problems associated with family or
  carer breakdown or reorganisation
• 4 Serious problems with feeling or being
  victimised, abused or seriously neglected by
  family or carer

66

• Paul's mother is very worried about him and the
  way he has changed. She tells you, I make
  sure he is never out of my sight now, you dont
  know what will happen even though he has been
  out by himself.
• Clinically significant ?
• Rate 2

67
Scale 12 Comments

• Family can include foster and alternative living
  arrangements
• Can be a challenge to give a global rating where
  they are involved in foster as well as biological
  family relationships
• Parents problems or parenting style (drug use,
  unemployment, mental illness) should only be
  rated if it has an impact on the child
• Unlike the preceding scales 1-9 concerned with
  the most severe marker in the typically two-week
  period, scales 10-13 are concerned with the
  average level of functioning during the rating
  period

68
13. Poor school attendance

• Include truancy, school refusal, school
  withdrawal or suspension for any cause.
• Include attendance at type of school at time of
  rating, for example hospital school, home
  tuition. If school holiday, rate the last two
  weeks of the previous term.
• 0 No problems of this kind during the period
  rated
• 1 Slight problems, such as late for two or more
  lessons
• 2 Definite but mild problems, such as missed
  several lessons because of truancy or refusal to
  go to school
• 3 Marked problems, absent several days during the
  period rated
• 4 Severe problems, absent most or all days
  include school suspension, exclusion or expulsion
  for any cause during the period rated

69

• On Wednesday, Paul refused to go to school.
• Clinically significant ?
• Rate 2

70
Scale 13 Comments

• If school holidays, use the last term
• Absence for any reason is considered
• Ways to minimise confusion
• Conceptualise this scale as assessing the
  normative occupational activity for this age
  group
• If attending alternative educational pathway
  (such as home tuition) or working, use attendance
  there

71
HoNOSCA optional scales Section B

• Scales 14 and 15 are concerned with problems for
  the child, parent or carer relating to lack of
  information or access to services. These are not
  direct measures of the child's mental health but
  changes here may result in long-term benefits for
  the child
• These scales do not contribute to the total score

72
14. Problems with knowledge or understanding
about the nature of the child or adolescent's
difficulties (in period rated)

• Include lack of useful information or
  understanding available to the child or
  adolescent, parents or carers.
• Include lack of explanation about the diagnosis
  or the cause of the problem or the prognosis.
• 0 No problems during the period rated parents
  and carers have been adequately informed about
  the child or adolescent's problems
• 1 Slight problems only
• 2 Mild but definite problems
• 3 Moderately severe problems parents and carers
  have very little or incorrect knowledge about the
  problem that is causing difficulties such as
  confusion or self-blame
• 4 Very severe problems parents have no
  understanding about the nature of their child or
  adolescent's problems

73

• Paul's mother is sure that Paul has
  schizophrenia just like his uncle
• Clinically significant ?
• Rate 2-3 (not that he may not develop a psychotic
  disorder but this view may well interfere with
  effective treatment currently)

74
15. Problems with lack of information about
services or management of the child or
adolescent's difficulties

• Include lack of useful information or
  understanding available to the child or
  adolescent, parents or carers or referrers.
• Include lack of information about the most
  appropriate way of providing services to the
  child or adolescent, such as care arrangements,
  educational placement or respite care.
• 0 No problems during the period rated the need
  for all necessary services has been recognised
• 1 Slight problems only
• 2 Mild but definite problems
• 3 Moderately severe problems parents and carers
  have been given little information about
  appropriate services, or professionals are not
  sure where a child should be managed
• 4 Very severe problems parents have no
  information about appropriate services or
  professionals do not know where a child should be
  managed

75

• During your interview with Paul's mother she says
  no-one could help his uncle until he was
  admitted to hospital. I just dont know who I
  should get to help Paul.
• Clinically significant ?
• Rate 2 (mother appears only aware of intensive
  and inappropriate adult intervention)

76
HoNOSCA-Section BEveryone likes to feel helpful

• Optional but outcomes ???????
• Scale 14 Lack of knowledge
• While ratings may reflect parents misdiagnosis,
  there can be substantial disagreement about the
  diagnosis between professionals
• Scale 15 Lack of information re management
• Confusion can arise based on a lack of consensus
  about appropriate management of a disorder
• Exacerbated by resource differentials between
  areas (rural c/f outer metropolitan c/f inner
  metropolitan)

77
Strength and Difficulties Questionnaire
78
Incorporating a carer and adolescent measure -
SDQ

• 25 items with 5 subscales
• conduct problems
• hyperactivity
• emotional symptoms
• peer problems
• prosocial behaviours
• Impact, burden, chronicity, distress supplement
• Public domain, Brief (A4), 40 languages (incl.
  Australian)
• Ref Goodman, R. J Am Acad Child Adolesc
  Psychiatry, 2001

• Parent, adolescent, teacher versions equivalent
  except
• Four social domains (P A) home friends
  learning leisure
• Two social domains (T) classroom learning peer
  relationships
• Age range 11-17 (A), 11-17 and 4-10 (P and T),
• Follow up versions
• UK norms available Australian norms being
  established

79
SDQ Research findings

• Correlates highly with Rutter scales
  (longstanding measure of parent informant of
  child symptomatology) (0.78-0.88-parent)
  (0.87-0.92 teacher)
• Discriminates clinical from community sample with
  self-report (11-16)
• Reasonable cross informant correlations and good
  internal consistency
• Multi-informant produces very good specificity
  (95) but lower sensitivity (63)
• Best predictor of caseness was impact (distress
  plus social impairment items) rather than
  symptoms or chronicity

80
SDQ Comparison with CBCL

• CBCL empirical from USA case files
• SDQ empirical and nosological (DSM4 and ICD9)
• SDQ 25 vs 118 CBCL items
• Both distinguish clinical from community samples
• SDQ correlates higher with clinical interview
  than CBCL, suggesting that CBCL overestimates
  hyperactivity
• On hyperactivity/inattention SDQ correlated 0.43,
  CBCL correlated 0.15 with clinical interview
• In community sample, mothers preferred SDQ to
  CBCL (Goodman and Scott, 1999)

81
Since coming to the service...
82
Role play SDQ

• Offering and receiving
• What did you find?
• What would you want to have happen?
• How could this process assist with engaging you?
• How could the process become negative?

83
Factors Influencing Health Status (FIHS)

• Checklist of psychosocial complications based on
  ICD 10
• Assists with understanding variations in outcomes
• Score Yes/No based on the question
• Have any of these factors required additional
  clinical input during the episode of care?
• Note This is a different question to whether
  these factors exist?

84
Factors Influencing Health Status (FIHS)

• Factor suggested ICD10 related codes
• Maltreatment syndromes Y00-07
• Problems related to negative life events in
  childhood Z61.x
• Problems related to upbringing Z62.x
• Problems related to primary support group,
  including family circumstances Z63.x
• Problems related to social environment Z60.x
• Problems related to certain psychosocial
  circumstances Z64.x
• Problems related to other psychosocial
  circumstances Z65.x

85
Rating the Children's Global Assessment Scale
(CGAS)

• 1-100 global scale of functioning
• Rate the childs most impaired level of general
  functioning for the previous two weeks by
  selecting the lowest level that describes their
  functioning on a hypothetical continuum of
  health-illness.
• Rate actual functioning regardless of treatment
  or prognosis.
• Use intermediate numbers.
• The examples of behaviour in the glossary are
  illustrative and are not required for a
  particular rating.

86
Childrens Global Assessment Scale

• 100-91 Superior functioning in all areas
• 90-81 Good functioning in all areas
• 80-71 No more than slight impairments in
  functioning
• 70-61 Some difficulty in a single area but
  generally functioning well
• 60-51 Variable functioning with sporadic
  difficulties or symptoms in several but not all
  social areas
• 50-41 Moderate degree of interference in
  functioning in most social areas or severe
  impairment of functioning in one area
• 40-31 Major impairment of functioning in several
  areas and unable to function in one of these
  areas
• 30-21 Unable to function in almost all areas
• 20-11 Needs considerable supervision
• 10-1 Needs constant supervision

87
Common items

• Principal diagnosis
• ...the diagnosis established after study to be
  chiefly responsible for occasioning the patient
  or clients care during the preceding period of
  care
• Legal status
• Was the person treated on an involuntary basis
  (under the relevant mental health legislation) at
  some point during the preceding period of care?
• Both derived from CMI registration/closure data

88
Melinda V - a case presentation

• NB This was developed for a brief case
  presentation, not as a HoNOSCA friendly
  vignette. It may appear more difficult but is
  more realistic
• Melinda is a 16 year-old girl whose mother
  referred her due to frequent outbursts of anger,
  and recent suicidal ideation. Melinda is the
  eldest of two children. Her 10 year-old brother
  was diagnosed with autism and ADHD at four years
  of age. Her father had a breakdown when
  Melinda was about five years of age. He was
  diagnosed with depression and is still being
  medicated for this. Melinda clashes with her
  father. They appear to have an intense
  relationship that they both recognise is due to
  their similarities in personality.
• Melinda has poor academic history but usually
  passes. She has recently discovered that she has
  failed at least four of six exams. She has
  recently been assessed as being of low average
  intelligence with a personal strength in visual
  attention to detail. She is a talented artist
  and top athlete, but has difficulty coping with
  her academic subjects. Her mother claims that
  she has always had difficulty making friends, and
  each week she seems to report a conflictual
  incident with her peers. She believes that
  others consider her weird and thinks that she
  is possessed by an evil spirit.

89
Melinda V - continued

• Melinda has had two experiences of auditory
  hallucinations over the last two weeks NB
  changed from mths on 24 clinicians results, that
  have scared her and made her feel out of control.
  She fears that she will some day be abducted by
  aliens, to the extent that she is at times too
  afraid to got out at night.
• Melinda finds her problems overwhelming and feels
  blue most days. She has constant suicidal
  ideation, although she claims she will not act
  upon it anymore, since she was frightened by her
  overdose in January, three months ago). Melinda
  worries excessively, especially about friendships
  with peers. At times, she is reticent to attend
  school and hopes to do a TAFE course next year.
  She wishes that she was beautiful, confident and
  popular like her alter-ego whom she fantasises
  about and has named Jasmine.
• She appears to be superstitious and has developed
  obsessive rituals that she feels compelled to do
  to prevent impending disaster, such as writing
  down good news in case it is jeopardised. Her
  parents have recently complained that her
  moodiness, irritability and verbal aggression at
  home have become unbearable and they are seeking
  some respite accommodation for her.

90
HoNOSCA/CGAS admission/assessment (response
(Melinda V)
91
The expert consensus and real clinicians

• Scale 12 3 Respite sought and fathers
  depression
• Scale 13 0-1 No problems noted
• Scale 14 3 Ms view is possession
  by evil spirit which may be incorrect
  knowledge
• Scale 15 2 Parents have some service
  info (eg respite) but may be limited
• HoNOSCA Total Score 17-25
• CGAS 48 Moderate degree of
  interference in functioning in most
  social areas or severe impairment
  of functioning in one area
• Now, How Did 24 Real CAMHS Clinicians Rate
  Melinda

• Scale 1 2-3 Continuing conflict/
  verbal aggression
• Scale 2 0-1 Little evidence
• Scale 3 2-3 Constant ideation
• Scale 4 0 Little evidence
• Scale 5 2 More problems than
  expected
• Scale 6 0 Little issues noted
• Scale 7 3-4 Impact of auditory hall
• Scale 8 0 No issues apparent
• Scale 9 3-4 Moodiness, depression
  anxiety leading to
  rituals
• Scale 10 2-3 Recurrent peer conflicts
• Scale 11 0-1 No issues noted

92
Ratings of the Melinda vignette by CAMHS
clinicians after minimal training
NB Scores of 0 or 1 (Ie Not Clinically
Significant Shown in Blues) and Scores of 2, 3 or
4 Shown in Yellow, Oranges and Reds (Ie
Clinically Significant)
93
Follow up of Melinda

• Progress Notes at three months Melindas parents
  reported two previous separations in their
  marriage, although they believe they have made
  significant improvements in their relationship
  since counselling commenced. They gave
  permission to contact this counsellor who remains
  involved with them. They also noted that they
  suspected that a teenage neighbour had sexually
  assaulted Melinda when she was six. They
  remained worried that they had been very
  unavailable to her around these years due to work
  pressures. On investigation, Melinda said that
  she was not worried about her parents separating
  however she was guarded about the sexual assault
  and it was decided to explore this issue very
  gently.
• Progress Notes six months ISP review Melinda
  has now withdrawn from school and has enrolled
  into the TAFE course for the next semester. Her
  parents are not happy with her decision but are
  willing to support her during the course. Her
  mother reported the odd argument with Melinda
  but nothing like it was before. Melinda appears
  to be happier on most days and less preoccupied
  with problems although she reports feelings of
  sadness at least once a week.

94
Follow up of Melinda

• Melinda reported one or two thoughts of self-harm
  but spoke to the psychologist about these
  incidents. These thoughts occurred after
  arguments with her friends and she attempts to
  minimise contact with her peers. She denied any
  current hallucinations and was not concerned
  about being abducted by aliens or possessed by
  evil spirits. Her parents confirm this.
• Melindas parents have arranged regular in-home
  respite with the local community youth service
  and attend family therapy every two weeks.
  Melindas mother claimed to manage her better
  within the home and encourages her to take the
  medication. The parents see the treating
  psychiatrist every month and the case manager
  every fortnight. They report clear improvements
  in Melindas behaviour yet remain concerned about
  her depression, potential for self-harm and
  ongoing conflict with friends.
• The psychologist reported that Melinda had low
  average intelligence with no significant
  impairment in memory, concentration or attention.
  Her relative strengths were observed to be in
  visual analysis and non-verbal reasoning.

95
Review/Discharge HoNOSCA/CGAS/FIHS (Melinda V)
96
Review/Discharge HoNOSCA/CGAS/FIHS (Melinda V)

• FIHS
• Y/N Maltreatment syndromes Y00-07
• Y/N Problems related to negative life events in
  childhood Z61.x
• Y/N Problems related to upbringing Z62.x
• Y/N Problems related to primary support group,
  incl. family circumstances Z63.x
• Y/N Problems related to social environment
  Z60.x
• Y/N Problems related to certain psychosocial
  circumstances Z64.x
• Y/N Problems related to other psychosocial
  circumstances Z65.x

97
The expert consensus

• Scale 13 0-1 TAFE now becomes the focus of
  this scale. Melinda has done all she can at this
  stage to indicate no current problems.
• HoNOSCA Total 7-11
• Scale 14 0-1 Both parents involved in
  treatment
• Scale 15 0-1 Parents maintain ongoing
  contact with mental health service and are
  informed about services
• CGAS 63 Variable functioning with sporadic
  difficulties or symptoms in several but not all
  social areas
• FIHS 3
• Neg. life events invest. sex. assault
• Primary support group contact rship couns. re
  sep.
• Social environment exclusion re friends

• Scale 1 1 Occasional minor arguments
  during rating period
• Scale 2 0 No evidence
• Scale 3 1 Little ideation
• Scale 4 0 No evidence of problem
• Scale 5 0-1 No evidence of problems in rating
  period
• Scale 6 0 Little issues noted
• Scale 7 0-1 No evidence of any impact
  of any hallucinations.
• Scale 8 0 No issues apparent
• Scale 9 2 Some periods of sadness
• Scale 10 2-3 Ongoing peer conflicts and
  social avoidance
• Scale 11 0 No issues noted
• Scale 12 1 Improved relationships and
  less conflict within family

98
Delivering training

• The importance of the local implementation
  committee
• what have you control over?
• what obstacles in training?
• not a hearts and minds campaign
• Imagine uses and then shape the implementation
• Responsibility for training
• your team
• assisting other trainers in your service
• troubleshooting implementation problems
• ensuring that your implementation committee is
  aware of problems
• being a resource for instrument questions

99
Key constructs for training

• Know there is a policy context
• Know the suite of instruments
• Know how the NOCC protocol applies and how that
  will be implemented at a your team level
• a flow chart describing where the forms are, when
  they are done, who enters the data, who can
  produce the standard reports for you, who will do
  further analysis if you have other questions, who
  can problem solve or trouble shoot
• Practice HoNOSCA
• Practice CGAS/FIHS
• Discuss SDQ administration and how to use the
  data

100
Training considerations and the dilemma of time

• How much time?
• everyone is busy
• limited opportunities to conduct training
• permission from your manager for training
• What are team members hopes and fears for this?
• how will that information be used within the
  service to guide implementation?

101
Resources for the train the trainer session

• Victorian training manual
• include instruments and their glossary
• Copy of Melinda assessment and follow up vignette
• Two copies of HoNOSCA/CGAS assessment sheet
• One copy of HoNOSCA/CGAS/FIHS sheet
• One copy of SDQ parent version (either youth or
  child)

102
Agency-level progress

• Done (hopefully)
• Local implementation groups
• Education of stakeholders
• IT and communications infrastructure
• Local protocol, clinical documentation
• Implementation plans signed off
• Department of Human Services support funding

• Next
• Two-day train the trainer program
• Check implementation schedule for rater training
  and OM data collection
• Commence routine OM
• Develop and refine local use of the data

103
Evaluation - CAMHS train the trainer day

• I understand the CAMHS outcomes protocol
• I feel confident that I can use
• HoNOSCA
• CGAS
• FIHS
• and make sense of the SDQ
• The questions I would like answered about outcome
  measurement are

• This training day has helped me feel more
  confident about providing training to my team in
  outcome measurement
• My reservations about the outcomes protocol are
• The barriers I anticipate include
• If the Mental Health Branch could help further in
  this rollout, I would like them to
• Any further comments about this day
• or about the outcomes rollout

1 2 3 4 5 Not at all
Completely
1 2 3 4 5 Not at all
Completely
1 2 3 4 5 Not at all
Completely
1 2 3 4 5 Not at all
Completely
1 2 3 4 5 Not at all
Completely
1 2 3 4 5 Not at all
Completely
104
If its good enough for rock and roll, its good
enough forUsing feedback during the Show
One-off Snapshot
Feedback
Treatment Modification
Service Modification
105
In this edition, EHCAMHS road tests Routine
Outcome Measurement
SDQ-parent version
HoNOSCA

• Expertise in mental health
• Increased rate of return

• Expertise in impact on child
• Independent to treatment

• Expertise in self
• The systems raison detre

SDQ-Adolescent
Child?
106
Clinical vignette (case presentation - not
specifically prepared for HoNOSCA rating) (1)

• Barry is an eight year-old boy from a separated
  Australian family who presents with oppositional
  behaviour at school and at home. He was a
  demanding baby and active toddler and has been
  aggressive in educational settings since he
  commenced kindergarten. He has tantrums at home,
  is destructive to property, and seems insensitive
  to the rights of others. He has some reading and
  writing difficulties which seem partly re