The Logic of Informal Testing

1
The Logic of Informal Testing
Handout and references on line at ASHA.org
ASHA Convention Boston November 2007
Patrick Coppens, Ph.D. SUNY Plattsburgh, NY

• Purpose
• Informal testing is an important step in the
  assessment process before developing an effective
  rehabilitation program but its components have
  been poorly described and/or operationalized.
  This poster attempts to define concepts important
  for informal testing and to organize them into a
  logical framework.
• Formal testing
• Ability assessment which is exclusively
  quantitative and diagnostic.
• Static, inflexible, and decontextualized. At the
  most, they provide some limited insight into the
  type of errors, the spontaneous compensatory
  behaviors, and helpful cues, based on passive
  observation.
• Cannot shed light on the underlying nature of
  the problem (e.g., Muma, 1978) or identify the
  best parameters for clinical intervention
  (Butler, 1997 Lesser, 1988 McCauley, 1996).
• Not a clinical assessment. Although it is
  possible for an experienced clinician to diagnose
  a specific communication disorder fairly quickly,
  an effective clinical assessment should be much
  more encompassing and there is no such thing as a
  quick and easy clinical assessment (Muma, 1978,
  p. 216).
• What not to do
• Inexperienced clinicians may be tentative in
  implementing an effective informal assessment
  protocol because they focus their attention on
  doing something rather than the process of
  problem solving (Tomblin et al., 2000). When
  assessment is a matter of technique alone, with
  no regard for the specification of each of its
  stages and evaluation of those stages with
  reference to the obtained data, assessment
  becomes ritual rather than science (Damien
  Martin, 1988, p. 421). Informal assessment
  requires critical interpretation.
• Traditionally, clinicians who choose a type of
  intervention based on superficial information are
  said to provide cookbook therapy. Muma (1978)
  refers to those individuals as technicians (p.
  8) rather than clinicians.
• Flowcharts for assessment and therapy (Yoder and
  Kent, 1988) must not be followed rigidly lest it
  becomes a cookbook approach. Still, the steps
  involved address what to do next and not how
  or why to do it.
• Particularly with informal assessment, the
  thinking process behind the decision is the
  difficult aspect of the skill for beginning
  clinicians to master. Therefore, it may be more
  useful to generate a list of pertinent thought
  questions rather than a series of steps to
  follow.
• Informal testing
• Labels used for informal testing dynamic,
  authentic, alternative, nontraditional,
  descriptive, observational.
• Can be contextualized.
• The components and the objectives of informal
  testing should not only be used to articulate
  appropriate therapy goals at the onset of
  therapy, but also must be constantly used to
  adapt the therapy to the effected changes in the
  patient.

• The Tools
• Knowledge and skills are needed to select and
  use the tools appropriately.
• These tools allow clinicians to answer clinical
  questions.
• 1. Extension tasks
• Examiner-developed tasks to measure a behavior
  not covered by formal testing (e.g., oral apraxia
  task, written language tasks).
• Referred to as screening by Murray and Clark
  (2006).
• Difference with formal test no norms.
• Difference with dynamic assessment examiner
  still passive scorer, but may be extended into a
  dynamic assessment protocol by instructing the
  patient differently to influence their response.
• 2. In-depth tasks
• Garner more detailed information about skills
  and abilities already assessed in the formal
  testing phase (e.g., does my patient with anomia
  have more difficulty with abstract or longer
  words?)
• 3. Dynamic assessment
• Determine a clients range of performance,
  given help by the clinician (Haynes Pindzola,
  2004).
• Referred to as hypothesis testing (ASHA,
  2004), a mini-experiment in intervention (Lidz
  Peña, 1996), and mak(ing) the clinic a
  laboratory (Rosenbek et al., 1989).
• Clinician manipulates the clients response and
  the environment to maximize successful
  completion. Examiner attempts to (a) observe how
  the client performs the tasks, (b) identify what
  can prompt a modification of the observed
  behavior, and (c) identify the approach with the
  best potential for rehabilitation.
• Works best when clinicians rely on a theoretical
  model

• Knowledge and skills
• 1. Background Knowledge
• In order to ask pertinent hypothesis testing
  questions and eventually develop appropriate
  clinical interventions, clinicians need to know
  the theoretical underpinnings of the skills being
  assessed (Muma, 1978). If they do not, they must
  search the literature to educate themselves.
• Importantly, knowledge of the theoretical models
  will help clinicians develop hypotheses regarding
  the underlying mechanism for the impaired
  behavior and hence facilitate the development of
  therapy targets.
• Knowledge of rehabilitation techniques leads to
  appropriate candidacy hypothesis questions.
• 2. Critical Thinking Skills
• Many different definitions.
• Clinicians must be able to compare and contrast
  information (synthesis), to examine all
  possibilities and solve problems (analytical), to
  systematically evaluate results and conclusions
  (judgments and decisions).
• 3. Observation Skills
• Permeates all tasks including formal tests.
• A savvy clinician is constantly seeking relevant
  clinical information from the patients
  behaviors, in the event that one of these may
  become a useful addition to the therapeutic
  armamentarium.
• Used to formulate clinical hypotheses, in
  combination with background knowledge (one must
  anticipate what to look for).

• The Clinical Questions
• For the three clinical questions below,
  hypothesis testing questions will progressively
  increase in difficulty and depth. As well, the
  depth of knowledge and skills necessary to
  generate the appropriate hypotheses progressively
  increase. Consequently, there is an inherent
  increasing difficulty in formulating the most
  appropriate hypothesis testing questions. Tools
  to use vary.
• 1. What is the extent of the problem?
  (Non-skill-specific Q.)
• Defines the breadth of the impairment and the
  disability
• Hypothesis testing questions are mostly
  quantitative for the impairment, but could be
  qualitative for the disability (family/patient
  input).
• Examples Is my patient also apraxic?
• Tools formal testing, extension tasks.
• Hypothesis testing questions minimal
  difficulty.
• 2. Where does the behavior break down?
  (Skill-specific Q.)
• Hypothesis testing questions must bear on
  specific-skills.
• Clear understanding of the exact level of
  breakdown of an individual skill.
• Hypothesis testing questions limited to the
  knowledge of various components of the skill
  investigated.
• Examples How long can my TBI patient sustain
  attention? What level of sentence complexity
  impairs comprehension in a patient with Wernicke
  aphasia?
• Tools in-depth tasks.
• Hypothesis testing questions minimal
  difficulty.

KNOWLEDGE SKILLS
HYPOTHESIS TESTING Qs
TOOLS
Observation skills
Background knowledge
Extension tasks

• CLINICAL QUESTIONS
• What is the extent of the problem?
• Where does the behavior break down?
• What improves the behavior?
• What is the underlying nature of the behavior?

In-depth tasks
HYPOTHESIS TESTING Qs
HYPOTHESIS TESTING Qs
Dynamic assessment
Critical Thinking skills