Statement Validity Assessment

1
Statement Validity Assessment

• Vrij Chapter 8

2
What is Statement Validity Assesment?

• A verbal veracity assessment tool
• Originated in Sweden (1963) as a method to
  determine the credibility of child witnesses in
  sexual abuse cases
• Credibility of children in sexual abuse cases is
  critical, especially when there are no
  corroborating witnesses or physical evidence

3
So

• Unlike non-verbal deception detection techniques,
  you are not looking for tells as to when a
  person is lying

4
Problems with child witness testimonies

• Vrij cites Craig, 1995, stating estimates range
  between 6 to 60 that child witness statements
  about sexual abuse are inaccurate
• Due to parental influence, outside pressure,
  simple misidentification, or complete lies
• Adults tend to mistrust statements made by
  children

5
History of SVA

• Udo Undeutsch and the West German Supreme Court
• Presented case of a 14-year-old alleged victim of
  rape using a method called statement analysis
• Court ruled that outside psychologists had more
  and better resources to determine truthfulness
  than court fact finders
• 1955 court requires use of psychological
  interviews and credibility assessments in
  disputed cases

6
History of SVA continued

• Undeutsch was the first to create a comprehensive
  list of criteria to assess credibility
• In 1988, K?hnken and Steller refined the criteria
  and standardized it in to a formal assessment
  procedure
• Called it Statement Validity Analysis (SVA)

7
History of SVA continued

• So
• The current SVA method wasnt created until the
  1980s, more than 30 years after the German courts
  looked in to statement analysis
• Until this point, no studies had been done
  analyzing the validity of SA or SVA

8
Four Stages of SVA

• 1. Case-file analysis
• 2. Semi-structured interview
• 3. Criteria-Based Content Analysis (CBCA)
• 4. Evaluation of CBCA with the Validity Checklist

9
Stage 1 Case-File Analysis

• Analysis of facts in a case
• Expert forms hypotheses about what happened.
  Details from the analysis will help the expert
  focus on critical details later in the interview.

10
Stage 2 Semi-Structured Interview

• What the Criteria-Based Content Analysis (Stage
  3) will analyze
• Child gives his/her account of the allegation
• Can be very difficult do to lack of verbal or
  cognitive skills in young children
• Also highly influenced by personality factors
  such as anxiety or simple embarrassment
• Skill and knowledge of interviewer is critical

11
Stage 2 Semi-Structured Interview continued

• Interviewer must have a strategy for eliciting as
  much detailed information as possible
• Has to ask the right questions in the right way
• Must avoid leading, yes or no, questions
• Must get child (or adult for that matter) to tell
  story without interviewer influence

12
Stage 2 Semi-Structured Interview, continued

• Proper kinds of questions/techniques
• Open-ended (e.g. Tell me what happened.)
• Facilitative responses
• OK, mmhm, head nods, etc
• Focused questions
• Focus on specific details or aspects of the event
• Problematic questions
• Leading (e.g. Was it your dad?
• Option-posing (e.g. Was the man white or black?)

13
Stage 3 Criteria-Based Content Analysis (CBCA)

• Used on transcripts of the interviews
• Consists of 19 criteria judged on a three point
  scale.
• 0 if criteria is absent, 1 if criteria is
  present, 2 if criteria is strongly present
• Consists of four categories

14
Stage 3 CBCA The Four Categories

• 1. General Characteristics
• 2. Specific Contents
• 3. Motivation-Related Contents
• 4. Offence-Specific Elements

15
Stage 3 CBCA General Characteristics (1-3)

• 1. Logical Structure
• Statement is coherent and logically consistent
• 2. Unstructured Production
• Information is presented in non-chronological
  order
• 3. Quality of Details
• Statement is rich in details

16
Stage 3 CBCA Specific Contents (4-13)

• 4. Contextual Embedding
• Events are placed in time and location
• 5. Descriptions of Interactions
• Statements contain information that interlinks
  the alleged perpetrator and witness
• 6. Reproduction of Conversation
• Specific dialogue, not summaries of what people
  said
• 7. Unexpected Complications During the Incident

17
Stage 3 CBCA Specific Contents (4-13)
Continued

• 8. Unusual Details
• Tattoos, stutters, individual quirks
• 9. Superfluous Details
• Details that are non-essential to the allegation
• 10. Accurately Reported Details Misunderstood
• Mentioning of details outside a persons scope of
  understanding
• 11. Related External Associations

18
Stage 3 CBCA Specific Contents (4-13)
Continued

• 12. Accounts of Subjective Mental State
• Description of a change in a subjects feelings
  during the incident
• 13. Attribution of Perpetrators Mental State
• Witness describes perpetrators feelings

19
Stage 3 CBCA Motivated-Related Contents (14-18)

• 14. Spontaneous Corrections
• 15. Admitting Lack of Memory
• 16. Raising Doubts About Ones Own Testimony
• 17. Self-Deprecation
• 18. Pardoning the Perpetrator

20
Stage 3 CBCA Details Characteristic of the
Offence (19)

• 19. Offence-Specific Elements
• Descriptions of elements that are known by
  professionals to be typical of a crime

21
Stage 4 Evaluation of CBCA with the Validity
Checklist

• The CBCA score alone is not enough to determine
  if a person is being truthful
• The examiner must also take into account other
  factors that could have affected the outcome
• Leading by the interviewer, outside influences,
  witnesss cognitive abilities, etc
• The CBCA is NOT a standardized test

22
Stage 4 Evaluation of CBCA with the Validity
Checklist, continued

• Attempts to standardize the CBCA results through
  an 11 point checklist
• Allows the examiner to consider alternative
  reasons for CBCA outcomes
• As these alternative reasons are rejected, the
  CBCA results become stronger (in the assumption
  that the score represents the veracity of the
  statement)

23
Stage 4 Evaluation of CBCA with the Validity
Checklist, continued

• The Four Stages
• 1. Psychological Characteristics
• 2. Interview Characteristics
• 3. Motivation
• 4. Investigative Questions

24
Stage 4 Evaluation of CBCA with the Validity
Checklist Psych Characteristics

• 1. Inappropriateness of Language and Knowledge
• 2. Inappropriateness of Affect
• 3. Susceptibility to Suggestion

25
Stage 4 Evaluation of CBCA with the Validity
Checklist Interview Characteristics

• 4. Suggestive, Leading, or Coercive Interviewing
• 5. Overall Inadequacy of the Interview

26
Stage 4 Evaluation of CBCA with the Validity
Checklist Motivation

• 6. Questionable Motives to Report
• Both for witness and other parties involved
• 7. Questionable Context of the Original
  Disclosure or Report
• 8. Pressures to Report Falsely

27
Stage 4 Evaluation of CBCA with the Validity
Checklist Investigative Questions

• 9. Inconsistency with the Laws of Nature
• 10. Inconsistency with Other Statements
• 11. Inconsistency with Other Evidence

28
SVA Issues

• Effectiveness of individual criteria in CBCA
• Effectiveness of Validity Checklist
• Differences between laboratory and field studies
• Detection rates and false-positives
• Countermeasures
• Applicability to adults?
• The Daubert Standard

29
CBCA Issues

• Not all statements are equally effective
• A claim by a young child with less detail will be
  scored lower on the CBCA scale than that of an
  older child or adult
• Not all criteria are created equal
• Generally, the criteria in groups 1 and 2 are the
  most effective at distinguishing truth-tellers
  from liars

30
CBCA Inter-Rater Reliability

• Are CBCA scores found by one rater close to those
  of a second, independent rater?
• Good for most criteria, except unstructured
  production and spontaneous corrections
• Overall score agreement is higher than on
  individual criteria

31
Vrijs Literature Review

• Laboratory vs. Field studies
• Deficiencies for one type are the others
  strengths
• Lab Not realistic, often based off observation
  of a video
• Field Ground truth cannot always be
  established, methods of finding it are not always
  consistent
• In field studies, low quality statements are less
  likely to obtain a truthful diagnosis or a
  conviction/confession, even if true
• High CBCA scores on false claims can lead to
  false-confessions or convictions
• Therefore, relationship between CBCA scores and
  convictions or confessions may not be accurate

32
Esplin et al., (1988)

• Field study
• CBCA scored on 0-2 scale (range of scores could
  be 0-38)
• Confirmed statement average 24.8
• Doubtful statement average 3.6
• Differences between groups found in 16/19
  criteria
• However, there are criticisms

33
CBCA results from other studies

• Boychuck (1991) 13/19
• Lamb et al. (1997b) 5/14
• Plausible average 6.74
• Implausible average 4.85
• Parker Brown (2000) 6/18
• Rassin van der Sleen (2005) 2/5
• Craig et al. (1999)
• Confirmed average 7.2
• Doubtful average 5.7
• used a 0-1 pt scale on CBCA

34
Critical Difference to Non-verbal Studies

• All results found were in the expected direction,
  supporting the Undeutsch Hypothesis
• Results in non-verbal studies are highly erratic
• You may find non-verbal cues within individuals,
  but between groups these do not exist

35
CBCA Lab Studies

• Difficult to create realistic situations
• Accuracy rates ranged from 54 to 90
• Average rates for truths 70.81
• Average rates for lies 71.12
• Rates did not differ between children, adults,
  witnesses, victims, or suspects

36
CBCA Lab Studies, continued

• Serious methodological problems
• Different situations used
• Different analysis methods used
• Different amounts of training for raters
• Some studies do not use the Validity Checklist
  and base diagnoses purely upon the CBCA

37
CBCA Lab Studies, continued

• But some important results remain
• For the most part, all differences found were in
  the correct direction, once again supporting
  Undeutsch
• Some individual criteria are more effective than
  others
• Support percentages (differences found / studies
  investigated)
• Range from 76 (Criteria 3) to 0 (Criteria 17)

38
CBCA Lab Studies, continued

• Other effective criteria
• 4. Contextual embeddings
• 6. Reproductions of conversations
• 8. Unusual details
• Least effective
• 14-18 Motivational Criteria
• 17. Self deprecation actually occurred less in
  truth tellers in two studies

39
CBCA Classifications

• 1. Discriminate (statistical) analysis is the
  most common method
• 2. Rater makes own truth/lie classification
• Computer analysis better at detecting lies
• 80 vs. 60 for human raters
• People better at detecting truths
• 80 vs. 53 for computers
• 3. General decision rules
• E.G. Criteria 1-5, plus two others

40
Reviewing the Validity Checklist

• Focuses on three things
• 1. Age of interviewee
• Highly affects cognitive abilities
• Older age correlates with higher CBCA scores
• 2. Interviewers style
• Open-ended questions are most effective
• The Cognitive Interview
• 3. Coaching of interviewee
• Countermeasures
• Training of subject to include CBCA criteria in
  their statement
• Easily defeat the CBCA analysis (only 27 of
  coached liars caught)

41
What the lay-person believes

• Generally correct about number of details
  (Criterion 3) and descriptions of interactions
  (5)
• Generally believe liars include more contextual
  embeddings (Criterion 4), unusual details (8),
  and superfluous details (9) in stories
• Overall, the lay-persons view differs somewhat
  from the experts view
• This, potentially, is a good thing

42
Problems with the Validity Checklist

• Difficulty in identifying issues
• Coaching by an adult is hard to discover
• Difficulty in measuring issues
• E.g. susceptibility to suggestion
• Difficulty in determining impact of issues
• The validity checklist is much more subjective
  and less formalized than the CBCA
• It is therefore harder to study

43
Vrijs specific problems with VC

• Issue 2 Inappropriateness of Affect
• Cites research that suggests there are two main
  psychological reactions to a rape
• 1. Expressed style
• 2. Numbed style
• Issue 10 Inconsistencies between statements
• Human memory is not perfect, details can be
  unintentional
• A practiced lie will not contain as many
  inconsistencies
• Issue 9 and 11 (Consistency with laws of nature,
  consistency with other evidence)
• Childrens scope of understanding often include
  fantasies and other things not in agreement with
  natural laws
• Sometimes, even in a true allegation, no other
  evidence can be found

44
Vrijs specific problems with VC, continued

• Embedded false statements are difficult to detect
• False memories

45
The Daubert Standard

• Daubert vs. Merrel Dow Pharmaceuticals, Inc.
  (1993)
• Set standards for the inclusion of expert witness
  testimony in court cases in the United states
• Consists of 5 criteria that must be met for
  evidence to be admissible in court

46
The Daubert Standard, continued

• 1. Is the scientific hypothesis testable?
• 2. Has the proposition been tested?
• 3. Is there a known error rate?
• 4. Has the hypothesis and/or technique been
  subjected to peer review and publication?
• 5. Is the theory upon which the hypothesis and/or
  technique based generally accepted in the
  appropriate scientific community?

47
So, what about SVA?
CBCA Lab CBCA Field Validity Checklist SVA
1. Is the scientific hypothesis testable? Yes Problematic Problematic Problematic
2. Has the proposition been tested? Yes No No No
3. Is there a known error rate? Yes, too high No No No
4. Has the hypothesis been subjected to peer review/publication? Yes Yes No No
5. Is the theory based on generally accepted principles? Unknown Unknown Unknown Unknown
48
Error rates

• Refer to subjects that are classified incorrectly
• Truth tellers classified as liars, and vice-versa
  
• Error rate for CBCA judgments made in laboratory
  research is nearly 30 for both truths and lies
• This is EXTREMELY high

49
Overall evaluation of SVA

• While results from research on SVA strongly
  support the Undeutsch Hypothesis, SVA does not
  meet the requirements of the five criteria
  established by the Daubert Standard
• 70 correct classification is OK
• 30 error rate is much too high for a valid test
• Certain criteria in the CBCA appear to be highly
  effective at discriminating truth tellers from
  liars
• Other criteria are wholly ineffective

50
In the end

• CBCA and SVA would be an effective tool for use
  in the initial stages of investigations
• Results from these tests can guide police
  throughout investigations
• CBCA and SVA appears to be effective on adults
  also, not just useful in situations of child
  sexual abuse