Expanding Your Assessment Toolbox

1
Expanding Your Assessment Toolbox

• Bradley Jackson, Ph.D.
• The Childrens Hospital
• Aurora, CO
• Robert Stadolnik, Ed.D.
• FirePsych, Inc/Brandon School
• Medway, MA
• May 14, 2009
• 3rd Annual Northeast Juvenile Firesetting
  Intervention Conference
• Worcester, MA

2
Evidenced Based Assessments

• Avoid clinical judgment alone which is a poor and
  inconsistent method. (Mills, 2005)
• Encourage frameworks that promote systemization
  and consistency, yet are flexible enough to adapt
  to individual needs. (Doyle and Dolan, 2002)
• Allow for integration of science and practice.
  (Borum, 2006)
• Collect, structure, and usually quantify the
  impressions of child, caregivers, and
  professionals. (Hunt and Johnson, 1990)

3
Evidence Based Assessment of Conduct
ProblemsMcMahon and Frick (2005)

• EB Assessment requires use of a multiple methods
  strategy
• Interviews (Parent, Child, Family)
• Behavioral Measures
• Behavioral Observations
• Evaluate Co-Morbid Adjustment Problems
• (ADHD, Anxiety, Depression, Social Rejection,
  Substance Abuse, Learning Disability)
• Issues Relevant to Assessment
• 1) Severe CPs cover a broad range of antisocial
  and aggressive behaviors.
• ODD vs. CD, CD
  overt-covert, destructive-nondestructive
• 2) Youths with CPs often have a number of
  adjustment problems.
• 3) There are a large number of risk factors that
  can have additive or interactive effects.
• 4) Impact of risk factors can differ across
  subgroups of youths with CP.

4
Evidenced Based Assessments and Firesetting

• Previous authors have reported on importance of a
  comprehensive diagnostic approach. (Kolko and
  Kazdin, 1989 Fineman, 1995 Sakheim and Osborn,
  1994 Stadolnik, 2000 Wilcox, 2006)
• Assessment is the combination of both scientific
  process and artistic endeavor. (Stadolnik, 2000)
• Evaluator must embrace a diverse array of data
  with increased need for collateral contacts.
  (Wilcox, 2006)
• Evaluator must collect and analyze data from
  multiple domains. (Fineman, 1995 Humphreys and
  Kopet, 1996 Kolko, 1999 Stadolnik, 2000
  Wilcox, 2006)

5
Constructing Your Assessment Protocol

• Factors
• Population Served
• Funding/Insurance, etc.
• Service Delivery Model
• Prior Training and Experience
• Supervision and Training Opportunities

6
Statistics 101
7
Statistics are our friend

• The normal curve
• Standard deviations
• Statistical confidence
• Establishing cutoff scores
• False positives and false negatives

8
Examples

• Not distributed evenly
• A few at the extremes pull the average/mean so
  that it becomes a confusing summary score
• Standardizing any distribution helps us to
  compare with more consistency and confidence

9
Standardized scores

• Mathematically transforming a raw score (or any
  score) into a standard score allows us to use
  what we know about the normal curve
• Here are some more well-known standard scores
• IQ scores (mean 100, std dev 15)
• GRE/SAT score (mean 500, std dev 100)
• T scores (mean 50, std dev 10)
• For all of these transformations, equal
  differences between people will result in equal
  differences between the scores, so now we can
  actually compare test scores and know what the
  differences mean.

10
Percentile

• The percentile of a score tells you what
  proportion of the population received that score
  or lower.
• The mean of percentiles is 50 and the range is
  0 to 100.
• The scores do not have to follow any particular
  distribution so be sure to use a program or chart
  that standardizes the percentiles.

11

• T scores help us determine how extreme a test
  score actually is

12
Reporting test results

• Raw score
• T score
• Percentile
• Total score
• Scale score
• Special score
• Critical item

13
Firesetting Assessment DomainsStadolnik, R.
(2000)

• Behavioral Functioning
• Social/Emotional Functioning
• Parent/Family Functioning
• School/Cognitive Functioning
• Firesetting Behavior History
• Fire Scene Evidence

14
Behavioral Functioning

• Assessment Practice
• Methods
• Record review
• Clinical and Collateral Interviews
• Observation reports
• Standardized measures
• Questions to answer
• Are behaviors acute or chronic?
• What is the rate of progression?
• Consensus among reporters?
• Is impulsivity present?
• Criminal charges or police contact?
• Direct aggression to people or animals?
• Periods of improved behavior?
• Current behavior?

• Behavioral Measures
• Achenbach CBCL, TRF, and YSR
• Behavioral Assessment System for Children (BASC)
• Connors Rating Forms
• Aggression Measures
• Aggression Questionnaire
• Novaco Anger Scale and Provocation Inventory
• Overt Aggression Scale
• Interview for Anti Social Behavior
• Others

15
Child Behavior Checklist (Achenbach)

• Self Report Format
• Parent, Teacher, and Youth Versions
• Ages 6 - 18
• Takes 15 - 20 minutes to complete
• Computerized scoring and reports

16
CBCL/Achenbach Subscales

• Competence Scales (20 items, 2 open- ended
  questions)
• Activities
• Social relations
• School performance
• Total
• (e.g., list your childs sports and rate how
  often and how well they do each one compared to
  other same-age children)

17
Child Behavior Checklist

• Behavior Problems Scales (118 items, plus 2
  open-ended items)
• Parents rate their child for how true each item
  is now or within the past 6 months using the
  following scale
• 0 not true (as far as you know)
• 1 somewhat or sometimes true
• 2 very true or often true
• Example items - argues a lot impulsive or acts
  without thinking sets fires unusually loud
  unhappy, sad, or depressed
• Internalizing, Externalizing, and Total Problem
  Scales

18
Child Behavior Checklist

• Cross Informant Syndromes
• Anxious/Depressed
• Withdrawn/Depressed
• Somatic Complaints
• Social Problems
• Thought Problems
• Attention Problems
• Rule-Breaking Behavior
• Aggressive Behavior

• DSM-oriented scales
• Affective Problems
• Anxiety Problems
• Somatic Problems
• Attention Deficit/
• Hyperactivity Problems
• Oppositional Defiant Problems
• Conduct Problems

19
Novaco Anger Scale and Provocation
InventoryNovaco, R. (2003)

• Two-part, self report measure with 85 total items
• Ages 9 to 84
• For use in research, individual assessment, and
  outcome evaluation
• Designed to assess anger as a problem of
  psychological functioning and physical health
• Hand Scored

20
Novaco Anger Scale and Provocation
Inventory(NAS-PI)

• NAS-PI Scores
• Cognitive (COG)-thoughts of justification,
  suspicion and hostility.
• Arousal (ARO)-elevated physiological response to
  anger.
• Behavior (BEH)-confrontational and antagonistic
  behaviors or verbalizations.
• Anger Regulation (REG)-suggests effective
  regulation skills, potential strength.
• NAS Total-overall levels of angry behaviors and
  thoughts.
• Provocation Inventory
• PI Total-an index score of anger intensity across
  a range of provocations disrespect, unfairness,
  frustation, annoyances.

21
The Aggressive Adolescent

• Cognitive Characteristics
• Narrow bands of imagination
  (concrete-operational), habitually ruminate on
  violent self perceptions, and appraise all
  situations in a hostile manner.
• Affective Characteristics
• Narrow band of emotional expression,
  frequently seen as unhappy or unwell (dysphoric),
  dichotomous expression of anger from
  overcontrolled to undercontrolled.
• Behavioral Coping Skills
• Lack basic social skills and have low
  assertiveness skills. Poor negotiators and
  unable to delay gratification. Need to be taught
  how to avoid conflicts.
• Davis, D. (2000). The Aggressive Adolescent
  Clinical and Forensic Issues. New York. Haworth
  Press.

22
Affective vs. Predatory Modes of Violence

• Predatory
• Minimal or no ans
• No conscious emotions
• Planned and purposeful
• No or minimal threat
• Many goals
• No target displacement
• No time limit
• Preceded by private ritual
• Primarily cognitive
• Focused awareness

• Affective
• Intense ans arousal
• Extreme experience of emotions
• Reactive and immediate
• Internal or external threat
• Goal is threat reduction
• Rapid displacement of target
• Time limited behavior
• Preceded by public posturing
• Primarily emotional
• Heightened and diffuse awareness

23
Social/Emotional Functioning

• Assessment Practice
• Methods
• Clinical Interviews
• Parent Interviews
• Review of Records
• Collateral Contacts
• Psychiatry Consultation
• Observations
• Standardized Measures
• Questions to Answer
• Acute or Chronic?
• Level of severity/impairment?
• Past interventions?
• Medication needs?
• Affect vs. thought?
• DSM IV Classification?

• Measures
• Jesness Inventory
• Millon Adolescent Clinical Inventory (MACI)
• Millon Pre-Adolescent Clinical Inventory (M-PACI)
  
• Rorschach Inkblot Test
• Childrens Depression Inventory
• Trauma Symptom Checklist for Children
• Personality Inventory for Youth
• Clinical Assessment of Interpersonal
  Relationships (CAIR)
• Thematic Apperception Test (Test)
• Social Skills Relationship Inventory (SSRI)
• Trauma and Attachment Belief Scale
• MMPI-A
• Projective Drawings
• Others

24
Jesness Inventory- Revised (JI-R)

• Youth self-report
• 160 true/false items
• Ages 8 yrs and older
• Computerized scoring, hand-scoring templates,
  online administration
• 30 45 minutes
• DSM-IV Scales
• Conduct Disorder
• Oppositional Defiant Disorder

25
Jesness Inventory-Revised

• Personality Scales
• Social Maladjustment
• Value Orientation
• Immaturity
• Autism
• Alienation

• Manifest Aggression
• Withdrawal-Depression
• Social Anxiety
• Repression
• Denial
• Asocial Index

26
Jesness Subtype Profiles

• Pragmatist
• Autonomy-Oriented
• Introspective
• Inhibited
• Adaptive

• Undersocialized, Active
• Undersocialized, Passive
• Conformist
• Group-Oriented
• Pragmatist

27
Trauma Symptom Checklist (TSCC)

• Youth self-report measure of post-traumatic
  distress
• 54 items
• Ages 8 16
• Hand Scored
• Profile Form

28
TSCC

• Validity Scales
• Underresponse
• Hyperresponse

• Clinical Scales
• Anxiety
• Depression
• Anger
• Posttraumatic Stress
• Dissociation (3)
• Sexual Concerns (3)
• Critical Items

29
Millon Adolescent Clinical Inventory (MACI)
Millon, T. (2006)

• MACI Features
• 160 items, True/False, self report measure
• Ages 13-19
• Summarized in computer generated narrative
  reports
• Examines three distinct categories Personality
  Patterns, Expressed Concerns, and Clinical
  Syndromes

30
Millon Adolescent Clinical Inventory (MACI)

• Base Rate (BR) Interpretations
• MACI raw scores are transformed into BR scores
• BR scores are a measure of the rate at which a
  characteristic is present in the norm population.
• For each MACI scale BR scores are anchored at 75
  and 85.
• 85 represents adolescents for whom this trait is
  most prominent
• 75 represents adolescents for whom the trait is
  prominent or present

31
Millon Adolescent Clinical Inventory (MACI)

• Personality Patterns
• -traits and features combine to form a pattern
• - Style derived from combining three polarities
  1) pain-pleasure, 2) active-passive and 3)
  self-other
• Scale Name
• 1 Introversive
• 2a Inhibited
• 2b Doleful
• 3 Submissive
• Dramatizing
• Egoistic
• 6a Unruly
• 6b Forceful
• Conforming
• 8a Oppositional
• 8b Self Deprecating
• Borderline
  Tendency

32
Millon Adolescent Clinical Inventory (MACI)

• Expressed Concerns
• -focus is on feelings and attitudes about issues
  that concern the adolescent
• -intensity of those feelings is reflected is
  score elevation
• - its perceptions, not objectively observable or
  behavioral criteria
• Scale Name
• A Identity Diffusion
• B Self Devaluation
• C Body Disapproval
• D Sexual Discomfort
• E Peer Insecurity
• F Social Insensitivity
• G Family Discord
• H Childhood Abuse

33
Millon Adolescent Clinical Inventory (MACI)

• Clinical Syndromes
• -assesses disorders that manifest themselves in
  specific form
• -best seen as an extension or distortion of the
  adolescents personality
• -are transient, they wax and wane depending on
  stressors
• Scale Name
• AA Eating Dysfunctions
• BB Substance Abuse Proneness
• CC Delinquent Predisposition
• DD Impulsive Propensity
• EE Anxious Feelings
• FF Depressive Affect
• GG Suicidal Tendency

34
Parent /Family Functioning

• Assessment Practice
• Methods
• Direct interview
• Data from collateral reporters
• Home visit
• Milieu observation reports
• Standardized measure
• Questions to answer
• Overall emotional climate?
• Relationship/attachment quality?
• Discipline practices?
• Family resources?
• Parent mental health?
• Marital strength?

• Measures
• Parenting Stress Index
• Family Conflict Scale
• Parent-Child Relationship Inventory
• Family Assessment Measure III
• Alabama Parent Questionnaire
• Others

35
Family Assessment Measure-Version 3Skinner, B.,
Steinhauer, P., and Santa-Barbara, J. (1995)

• FAM III Features
• Self report measure, takes 30-40 minutes to
  complete
• Ages 10 and older
• Provides a quantitative description of family
  strengths and weaknesses
• Can be completed by all members of the family
• 3 Scales General Scale, Dyadic Relationship
  Scale, Self-Rating Scale

36
Family Assessment Measure-Version 3FAM III

• FAM III Subscales
• Task Accomplishment- basic tasks met, flexible,
  alternative solutions are explored
• Role Performance-family understands expectations
  of roles, and agrees, adapt to new roles
• Communication- direct, clear, open, sufficient
• Affective Expression-full range of affect when
  appropriate and with correct intensity
• Involvement- empathic involvement, concern for
  others, nurturing
• Control- patterns of influence, adaptable,
  predictable yet flexible, constructive
• Values and Norms-consistent with family subgroup,
  explicit and implicit rules are consistent

37
Family Assessment Measure-Version 3FAM III

• Interpretive Guidelines
• Elevated scores (Tgt60) must be interpreted for
  alternative explanations (Ex Involvement)
• The more family members who indicate and elevated
  score in a particular area, the more likely it is
  problematic.
• Total number of elevated scores correlates to
  overall family pathology.
• Greater discrepancies among spouse profiles
  suggest marital discord
• Different scores elevated for different members
  of the family suggest perception differences.

38
Firesetting Behavior History

• Assessment Practice
• Methods
• Fire History Interview-Child
• Parent Interview
• Structured Interview Tools
• Record Review
• Collateral Reports
• Questions to Answer
• When ? How?
• Where? Who?
• What? Why?

• Structured Interview Tools
• Childrens Firesetting Interview
• Firesetting Risk Interview
• Graphing Technique
• Oregon Screening Tool
• FIRE Protocol
• Fire Risk Interviews (Child, Parent, Family)-FEMA
• Others

39
Drawings

• Use of drawings during the assessment interview
  can help to
• Immerse the child in the memory
• Engage the child in multiple modalities of
  sharing (verbal, visual)

• Types of drawings
• Self
• Tree
• Family doing something
• Safe fire
• Unsafe fire

40
Individual fire graphs

• From a list of all firesets/fireplay, the child
  selects a significant fire incident and details
  the sequence of events before, during and after
  the fire
• Cartoon or panel technique for fire drawings
• Written graph encourages child/teen to link
  thoughts and feelings to the sequence of events

41
Written Fire Graph

• Describe the situation the sequence of events
• Usually aware of events occurring during the
  fireset and after
• Focus on the events before fireset
• Slow down the description
• Ask clarifying questions
• Focus on possible precipitating triggers earlier
  in the day/week/month
• Add corresponding thoughts and feelings

42
Bibliography

•  
• Achenbach, T. (2001) Manual for the Achenbach
  System for Empirical Behavior Analysis.
  University of Vermont. Burlington, VT.
• Borum, R. (2006). Assessing risk for violence
  among juvenile offenders. In Forensic Mental
  Health Assessment of Children and Adolescents
  (Sparta, S and Koocher, G. Eds.). Oxford
  University Press. London. (pgs190-203). 
• Doyle, M. and Dolan, M.(2002). Violence risk
  assessment combining actuarial and clinical
  information to structure clinical judgments for
  the formulation and management of risk. Journal
  of Psychiatric and Mental Health Nursing. 9.
  649-657. 
• Fineman, K. (1995). A model for the qualitative
  analysis of child and adult fire deviant
  behavior. American Journal of Forensic
  Psychology. 13(1). 31-60.
•  Humphreys, J. and Kopet, T. (1996). Manual for
  the Juvenile Firesetter Needs Assessmnet
  Protocol. Oregon State Fire Marshal. Portland,
  OR.
•  Hunt, F. and Johnson, C. (1990). Early
  intervention for severe behavior problems the
  use of judgment based assessment procedures.
  Topics in Early Childhood Special Education.
  10(3). 111-122.
• Jesness, C. (2003). Jesness Inventory-Revised
  Manual. Multi-Health Systems. North Tonowanda,
  NY.
•  Kolko, D. (1999). Firesetting in children and
  youth. In V Van Hasselt M. Hersen (Eds.),
  Handbook of Psychological Approaches with Violent
  Offenders Contemporary Strategies and Issues.
  95-115. Kluwar Academic/Plenum Publishers. New
  York.
•  Kolko, D. and Kazdin, A. (1989). Assessments of
  dimensions of childhood firesetting among
  patients and nonpatients the Firesetting Risk
  Interview. Journal of Abnormal Child Psychology.
  17(2). 157-176.
•  McMahon, R. and Frick, P. (2005). Evidence-based
  assessment of conduct problems in children and
  adolescents. Journal of Clinical Child and
  Adolescent Psychology. 34(3). 477-505.
•  Millon, T. (2006). Millon Adolescent Clinical
  inventory Manual. NCS Pearson Inc. Minneapolis,
  MN.
•  Mills, J. (2005). Advances in the assessment and
  prediction of interpersonal violence. Journal of
  Interpersonal Violence. 20(2). 236-241.
•  Novaco, R. (2003). The Novaco Anger Scale and
  Provocation Inventory Manual. Western
  Psychological Associates. Los Angeles, CA.
•  Skinner, H., Steinhauer, P. and Santa-Barbara,
  J. (1995). Family Assessment Measure Version III
  Technical Manual. Multi-Health Systems. North
  Tonowanda, NY.
•  Stadolnik, R. (2000). Drawn to the Flame
  Assessment and Treatment of Juvenile Firesetting
  Behavior. Professional Resource Press. Sarasota,
  FL.
•  Wilcox, D. (2006). Assessing Firesetting
  Behavior in Children and Adolescents. In Forensic
  Mental Health Assessments of Children and
  Adolescents (Sparta, S. Koocher, G. Eds.).
  Oxford University Press, New York, NY.