A field investigation of CriteriaBased Content Analysis

1
A field investigation of Criteria-Based Content
Analysis

• Dr Lucy Akehurst
• Sarah Manton
• Sabine Quandte
• International Centre for Research in Forensic
  Psychology
• Department of Psychology
• University of Portsmouth

2
Overall procedure for Statement Validity Analysis
(SVA)

•   1. Collection of background information
• regarding the case and the witness.
• Formulating hypotheses.
•  
• 2. Interview with the witness regarding
• the event(s) in question.
•  
•  

•  
• 4. Referral to a validity checklist.
•  

3
The Undeutsch hypothesis (1967)

• Truthful, reality-based accounts differ
  significantly and noticeably from unfounded,
  falsified or distorted stories

There are cognitive and motivational factors
involved (Köhnken, 1989)
4
CBCA criteria

• GENERAL CHARACTERISTICS
• 1. Logical structure
• 2. Unstructured production
• 3. Quantity of details

SPECIFIC CONTENTS 4. Contextual embedding 5.
Descriptions of interactions 6.
Reproduction of conversation 7. Unexpected
complications
PECULIARITIES OF CONTENT 8. Unusual details
9. Superfluous details 10. Details
misunderstood 11. Related external
associations 12. Accounts of own mental
state 13. Attribution of others mental state
MOTIVATION-RELATED CONTENTS 14. Spontaneous
corrections 15. Admitting lack of memory 16.
Raising doubts 17. Self-deprecation 18.
Pardoning the perpetrator

• OFFENCE-SPECIFIC ELEMENTS
• 19. Details characteristic of the offence

5
Hit rates

• For methodologically sound work
• Hit rates 65 - 90 when CBCA used
• Average for truthful statements around 73
• Average for fabricated statements around 72
• Therefore potentially very worthwhile as normal
  hit rates range from 45 to 60
• Based on Vrij (2005) review of the literature

6
Previous research

• Laboratory work
• Vs
• Field work

7
Methodological considerations for field studies
Our study ? - real statements

• Statement length

Statement type
? - verbatim transcripts
Raters
? - two, blind
Training
? - very thorough
Distribution
?? - in fact more liars in older age group
Decision rules
? - none used
8
Establishing ground truth truthful statements

• All truthful statements satisfied at least three
  of the following criteria and therefore had to
  include at least one of those marked with (i.e.
  independent of statement quality)
• Medical evidence
• Corroboration from other victim/witness
• Guilty verdict at court
• Admission of suspect

9
Establishing ground truth fabricated statements

• All fabricated statements satisfied at least
  three of the following criteria and therefore had
  to include at least one of those marked with
  (i.e. independent of statement quality)
• Contradictory evidence
• Proved incident could not have occurred
• Recantation at a later date (some charged with
  wasting police time)
• Persistent not guilty from alleged offender

10
Method

• West Midlands Constabulary
• Rigorous selection procedure from original sample
  of 175 records
• 61 - removed as no forensic interview recorded
• 7 couldnt track down videorecording
• 43 removed as classified as uncertain
• 30 removed as no independent case facts to help
  with classification
• 3 removed for misc. reasons interpreter,
  coughing etc!
• Resulting in 31 statements used
• 6 10 year olds 11 17 year olds
• Truthful 12 6
• Fabricated 4 9

11
Method

• Verbatim statements obtained
• Two, blind raters
• Rated each criteria
• (except details characteristic of the offence)
• On 5-point Likert scales
• Absent 1 2 3 4 5 Strongly present
• Also, answered, Is this statement true or
  false?
• And 7 point Likert scale to show how truthful

12
Results

• Impressive inter-rater reliability
• (Anson et al, 1993 .5 adequate, .6 good, .75
  excellent)

Logical consistency .83 Unstructured production
.56 Quantity of detail .76
Contextual embedding .96 Desc. of interactions
.69 Conversation .88 Unexpected complications
.89 Unusual details .84 Superfluous details
.59 Details misunderstood .62 Related
external assoc. .51 Own mental state
.75 Others mental state .71
Spontaneous corrections .88 Admitting lack of
memory .91 Raising doubts .60 Self
deprecation .17 Pardoning the perpetrator .47
TOTAL SCORE .82
13
Results Hit rates

• Q1 Is this is a true or fabricated statement?
• Overall Truths Lies
• Rater one 84 94 69
• Rater two 81 94 62

Q2Completely false 1 2 3 4 5 6 7
Completely true True mean False
mean t Rater one 5.06 3.15
4.52 Rater two 6.06 4.62 3.77
14
Total CBCA scores

• Means found for each criterion
• Totals dont include unreliable criteria
• ANOVA IVs age group (older/younger)
• veracity (truth/lie)
• DV total CBCA score
• Main effect for veracity F(1, 27) 6.83, p lt
  0.01
• No main effect for age
• No interaction effect

15
6-10 year olds 11-17 year olds Truthful 40.38
39.25 Fabricated 35.38 35.44
16
Still to analyse!

• Influence of individual criteria
• Correlations of age with presence of criteria
• Frequency data

17
Discussion points

• What if untrained observers asked to rate the
  verbatim statements?
• Stringent selection criteria vs generalisability
• Interesting how many times I felt a lot of
  statement true but some parts false
• Definitely some criteria that jump off the page
• SVA not just CBCA

18
The role of CBCA

• Germany, Sweden, The Netherlands
• Commonly used for expert testimony
• no questions asked?
• UK
• As an investigation tool early on?
• To give weight to judge/jury training?
• Issues with adequacy of training (leakage)
• Other countries
• Please let me know!
• lucy.akehurst_at_port.ac.uk