Statement Validity Assessment

1
Statement Validity Assessment

• Vrij Chapter 8

2
What is Statement Validity Assesment?

• A verbal veracity assessment tool
• Originated in Sweden (1963) as a method to
  determine the credibility of child witnesses in
  sexual abuse cases
• Credibility of children in sexual abuse cases is
  critical, especially when there are no
  corroborating witnesses or physical evidence

3
So

• Unlike non-verbal deception detection techniques,
  you are not looking for tells as to when a
  person is lying

4
Problems with child witness testimonies

• Vrij cites Craig, 1995, stating estimates range
  between 6 to 60 that child witness statements
  about sexual abuse are inaccurate
• Due to parental influence, outside pressure,
  simple misidentification, or complete lies
• Adults tend to mistrust statements made by
  children

5
History of SVA

• Udo Undeutsch and the West German Supreme Court
• Presented case of a 14-year-old alleged victim of
  rape using a method called statement analysis
• Court ruled that outside psychologists had more
  and better resources to determine truthfulness
  than court fact finders
• 1955 court requires use of psychological
  interviews and credibility assessments in
  disputed cases

6
History of SVA continued

• Undeutsch was the first to create a comprehensive
  list of criteria to assess credibility
• In 1988, K?hnken and Steller refined the criteria
  and standardized it in to a formal assessment
  procedure
• Called it Statement Validity Analysis (SVA)

7
History of SVA continued

• So
• The current SVA method wasnt created until the
  1980s, more than 30 years after the German courts
  looked in to statement analysis
• Until this point, no studies had been done
  analyzing the validity of SA or SVA

8
Four Stages of SVA

• 1. Case-file analysis
• 2. Semi-structured interview
• 3. Criteria-Based Content Analysis (CBCA)
• 4. Evaluation of CBCA with the Validity Checklist

9
Stage 1 Case-File Analysis

• Analysis of facts in a case
• Expert forms hypotheses about what happened.
  Details from the analysis will help the expert
  focus on critical details later in the interview.

10
Stage 2 Semi-Structured Interview

• What the Criteria-Based Content Analysis (Stage
  3) will analyze
• Child gives his/her account of the allegation
• Can be very difficult do to lack of verbal or
  cognitive skills in young children
• Also highly influenced by personality factors
  such as anxiety or simple embarrassment
• Skill and knowledge of interviewer is critical

11
Stage 2 Semi-Structured Interview continued

• Interviewer must have a strategy for eliciting as
  much detailed information as possible
• Has to ask the right questions in the right way
• Must avoid leading, yes or no, questions
• Must get child (or adult for that matter) to tell
  story without interviewer influence

12
Stage 2 Semi-Structured Interview, continued

• Proper kinds of questions/techniques
• Open-ended (e.g. Tell me what happened.)
• Facilitative responses
• OK, mmhm, head nods, etc
• Focused questions
• Focus on specific details or aspects of the event
• Problematic questions
• Leading (e.g. Was it your dad?
• Option-posing (e.g. Was the man white or black?)

13
Stage 3 Criteria-Based Content Analysis (CBCA)

• Used on transcripts of the interviews
• Consists of 19 criteria judged on a three point
  scale.
• 0 if criteria is absent, 1 if criteria is
  present, 2 if criteria is strongly present
• Consists of four categories

14
Stage 3 CBCA The Four Categories

• 1. General Characteristics
• 2. Specific Contents
• 3. Motivation-Related Contents
• 4. Offence-Specific Elements

15
Stage 3 CBCA General Characteristics (1-3)

• 1. Logical Structure
• Statement is coherent and logically consistent
• 2. Unstructured Production
• Information is presented in non-chronological
  order
• 3. Quality of Details
• Statement is rich in details

16
Stage 3 CBCA Specific Contents (4-13)

• 4. Contextual Embedding
• Events are placed in time and location
• 5. Descriptions of Interactions
• Statements contain information that interlinks
  the alleged perpetrator and witness
• 6. Reproduction of Conversation
• Specific dialogue, not summaries of what people
  said
• 7. Unexpected Complications During the Incident

17
Stage 3 CBCA Specific Contents (4-13)
Continued

• 8. Unusual Details
• Tattoos, stutters, individual quirks
• 9. Superfluous Details
• Details that are non-essential to the allegation
• 10. Accurately Reported Details Misunderstood
• Mentioning of details outside a persons scope of
  understanding
• 11. Related External Associations

18
Stage 3 CBCA Specific Contents (4-13)
Continued

• 12. Accounts of Subjective Mental State
• Description of a change in a subjects feelings
  during the incident
• 13. Attribution of Perpetrators Mental State
• Witness describes perpetrators feelings

19
Stage 3 CBCA Motivated-Related Contents (14-18)

• 14. Spontaneous Corrections
• 15. Admitting Lack of Memory
• 16. Raising Doubts About Ones Own Testimony
• 17. Self-Deprecation
• 18. Pardoning the Perpetrator

20
Stage 3 CBCA Details Characteristic of the
Offence (19)

• 19. Offence-Specific Elements
• Descriptions of elements that are known by
  professionals to be typical of a crime

21
Stage 4 Evaluation of CBCA with the Validity
Checklist

• The CBCA score alone is not enough to determine
  if a person is being truthful
• The examiner must also take into account other
  factors that could have affected the outcome
• Leading by the interviewer, outside influences,
  witnesss cognitive abilities, etc
• The CBCA is NOT a standardized test

22
Stage 4 Evaluation of CBCA with the Validity
Checklist, continued

• Attempts to standardize the CBCA results through
  an 11 point checklist
• Allows the examiner to consider alternative
  reasons for CBCA outcomes
• As these alternative reasons are rejected, the
  CBCA results become stronger (in the assumption
  that the score represents the veracity of the
  statement)

23
Stage 4 Evaluation of CBCA with the Validity
Checklist, continued

• The Four Stages
• 1. Psychological Characteristics
• 2. Interview Characteristics
• 3. Motivation
• 4. Investigative Questions

24
Stage 4 Evaluation of CBCA with the Validity
Checklist Psych Characteristics

• 1. Inappropriateness of Language and Knowledge
• 2. Inappropriateness of Affect
• 3. Susceptibility to Suggestion

25
Stage 4 Evaluation of CBCA with the Validity
Checklist Interview Characteristics

• 4. Suggestive, Leading, or Coercive Interviewing
• 5. Overall Inadequacy of the Interview

26
Stage 4 Evaluation of CBCA with the Validity
Checklist Motivation

• 6. Questionable Motives to Report
• Both for witness and other parties involved
• 7. Questionable Context of the Original
  Disclosure or Report
• 8. Pressures to Report Falsely

27
Stage 4 Evaluation of CBCA with the Validity
Checklist Investigative Questions

• 9. Inconsistency with the Laws of Nature
• 10. Inconsistency with Other Statements
• 11. Inconsistency with Other Evidence

28
SVA Issues

• Effectiveness of individual criteria in CBCA
• Effectiveness of Validity Checklist
• Differences between laboratory and field studies
• Detection rates and false-positives
• Countermeasures
• Applicability to adults?
• The Daubert Standard

29
CBCA Issues

• Not all statements are equally effective
• A claim by a young child with less detail will be
  scored lower on the CBCA scale than that of an
  older child or adult
• Not all criteria are created equal
• Generally, the criteria in groups 1 and 2 are the
  most effective at distinguishing truth-tellers
  from liars

30
CBCA Inter-Rater Reliability

• Are CBCA scores found by one rater close to those
  of a second, independent rater?
• Good for most criteria, except unstructured
  production and spontaneous corrections
• Overall score agreement is higher than on
  individual criteria

31
Vrijs Literature Review

• Laboratory vs. Field studies
• Deficiencies for one type are the others
  strengths
• Lab Not realistic, often based off observation
  of a video
• Field Ground truth cannot always be
  established, methods of finding it are not always
  consistent
• In field studies, low quality statements are less
  likely to obtain a truthful diagnosis or a
  conviction/confession, even if true
• High CBCA scores on false claims can lead to
  false-confessions or convictions
• Therefore, relationship between CBCA scores and
  convictions or confessions may not be accurate

32
Esplin et al., (1988)

• Field study
• CBCA scored on 0-2 scale (range of scores could
  be 0-38)
• Confirmed statement average 24.8
• Doubtful statement average 3.6
• Differences between groups found in 16/19
  criteria
• However, there are criticisms

33
CBCA results from other studies

• Boychuck (1991) 13/19
• Lamb et al. (1997b) 5/14
• Plausible average 6.74
• Implausible average 4.85
• Parker Brown (2000) 6/18
• Rassin van der Sleen (2005) 2/5
• Craig et al. (1999)
• Confirmed average 7.2
• Doubtful average 5.7
• used a 0-1 pt scale on CBCA

34
Critical Difference to Non-verbal Studies

• All results found were in the expected direction,
  supporting the Undeutsch Hypothesis
• Results in non-verbal studies are highly erratic
• You may find non-verbal cues within individuals,
  but between groups these do not exist

35
CBCA Lab Studies

• Difficult to create realistic situations
• Accuracy rates ranged from 54 to 90
• Average rates for truths 70.81
• Average rates for lies 71.12
• Rates did not differ between children, adults,
  witnesses, victims, or suspects

36
CBCA Lab Studies, continued

• Serious methodological problems
• Different situations used
• Different analysis methods used
• Different amounts of training for raters
• Some studies do not use the Validity Checklist
  and base diagnoses purely upon the CBCA

37
CBCA Lab Studies, continued

• But some important results remain
• For the most part, all differences found were in
  the correct direction, once again supporting
  Undeutsch
• Some individual criteria are more effective than
  others
• Support percentages (differences found / studies
  investigated)
• Range from 76 (Criteria 3) to 0 (Criteria 17)

38
CBCA Lab Studies, continued

• Other effective criteria
• 4. Contextual embeddings
• 6. Reproductions of conversations
• 8. Unusual details
• Least effective
• 14-18 Motivational Criteria
• 17. Self deprecation actually occurred less in
  truth tellers in two studies

39
CBCA Classifications

• 1. Discriminate (statistical) analysis is the
  most common method
• 2. Rater makes own truth/lie classification
• Computer analysis better at detecting lies
• 80 vs. 60 for human raters
• People better at detecting truths
• 80 vs. 53 for computers
• 3. General decision rules
• E.G. Criteria 1-5, plus two others

40
Reviewing the Validity Checklist

• Focuses on three things
• 1. Age of interviewee
• Highly affects cognitive abilities
• Older age correlates with higher CBCA scores
• 2. Interviewers style
• Open-ended questions are most effective
• The Cognitive Interview
• 3. Coaching of interviewee
• Countermeasures
• Training of subject to include CBCA criteria in
  their statement
• Easily defeat the CBCA analysis (only 27 of
  coached liars caught)

41
What the lay-person believes

• Generally correct about number of details
  (Criterion 3) and descriptions of interactions
  (5)
• Generally believe liars include more contextual
  embeddings (Criterion 4), unusual details (8),
  and superfluous details (9) in stories
• Overall, the lay-persons view differs somewhat
  from the experts view
• This, potentially, is a good thing

42
Problems with the Validity Checklist

• Difficulty in identifying issues
• Coaching by an adult is hard to discover
• Difficulty in measuring issues
• E.g. susceptibility to suggestion
• Difficulty in determining impact of issues
• The validity checklist is much more subjective
  and less formalized than the CBCA
• It is therefore harder to study

43
Vrijs specific problems with VC

• Issue 2 Inappropriateness of Affect
• Cites research that suggests there are two main
  psychological reactions to a rape
• 1. Expressed style
• 2. Numbed style
• Issue 10 Inconsistencies between statements
• Human memory is not perfect, details can be
  unintentional
• A practiced lie will not contain as many
  inconsistencies
• Issue 9 and 11 (Consistency with laws of nature,
  consistency with other evidence)
• Childrens scope of understanding often include
  fantasies and other things not in agreement with
  natural laws
• Sometimes, even in a true allegation, no other
  evidence can be found

44
Vrijs specific problems with VC, continued

• Embedded false statements are difficult to detect
• False memories

45
The Daubert Standard

• Daubert vs. Merrel Dow Pharmaceuticals, Inc.
  (1993)
• Set standards for the inclusion of expert witness
  testimony in court cases in the United states
• Consists of 5 criteria that must be met for
  evidence to be admissible in court

46
The Daubert Standard, continued

• 1. Is the scientific hypothesis testable?
• 2. Has the proposition been tested?
• 3. Is there a known error rate?
• 4. Has the hypothesis and/or technique been
  subjected to peer review and publication?
• 5. Is the theory upon which the hypothesis and/or
  technique based generally accepted in the
  appropriate scientific community?

47
So, what about SVA?
48
Error rates

• Refer to subjects that are classified incorrectly
• Truth tellers classified as liars, and vice-versa
  
• Error rate for CBCA judgments made in laboratory
  research is nearly 30 for both truths and lies
• This is EXTREMELY high

49
Overall evaluation of SVA

• While results from research on SVA strongly
  support the Undeutsch Hypothesis, SVA does not
  meet the requirements of the five criteria
  established by the Daubert Standard
• 70 correct classification is OK
• 30 error rate is much too high for a valid test
• Certain criteria in the CBCA appear to be highly
  effective at discriminating truth tellers from
  liars
• Other criteria are wholly ineffective

50
In the end

• CBCA and SVA would be an effective tool for use
  in the initial stages of investigations
• Results from these tests can guide police
  throughout investigations
• CBCA and SVA appears to be effective on adults
  also, not just useful in situations of child
  sexual abuse