New Advances in Measurement

1
New Advances in Measurement

• Ronald D. Rogge

2
TOPICS

• RELATIONSHIP QUALITY
• T1 IRT Optimization
• Study 1
• T2 Responsiveness to Change
• Studies 2-5
• T3 Bi-Dimensional View
• Studies 6-7
• T4 Implicit Measures
• Studies 8-11
• ATTENTION
• T5 Screening for Error Variance
• Studies 12-16

3
Acknowledgements

• Couples Satisfaction Index
• Janette Funk, Mike Maniaci, Maria Saavedra,
  Soonhee Lee
• Positive-Negative Relationship Quality
• Frank Fincham, Richard Mattson, Matt Johnson
• C.A.R.E. Program
• Tom Bradbury, Rebecca Cobb, Matt Johnson, Erika
  Lawrence, Lisa Story, Lexi Rothman
• Implicit Assessment
• Soonhee Lee, Harry Reis
• Attention / Effort
• Mike Maniaci, Janette Funk, Soonhee Lee, Maria
  Saavedra

4
Relationship Quality

• Relationship satisfaction
• Self-report scales (DAS, MAT, QMI)
• 30-50 years of research (over 4K studies)
• Excellent correlational validity
• Level of noise?
• Responsive to change over time?
• Are these the best items?

5
TOPIC 1 IRT Optimization

• Large sample method
• N at least 1,000 in smallest group
• Large item pool
• Unidimensional
• Non-redundant
• Used by ETS
• SAT, GRE, MCAT, LSAT
• Quality of each item
• Information
• Noise
• Advantages
• Over correlations
• Over small sample methods

6
IRT Approach

• Latent scores (q) for each subject
• Like GRE scores
• Assessing relationship satisfaction
• Parameters for each item
• Response curves
• Higher qs ? higher responses?
• Item Responsiveness
• How informative?
• Where informative?
• Creates information profiles
• For individual items
• For sets of items

7
Study 1 - Measures

• 141 satisfaction items
• DAS, MAT, RAS, KMS, QMI, SMD
• 71 additional items
• 7 anchor scales
• Neuroticism (EPQ-N)
• Conflict / Communication (MCI, CPQ, IAI)
• Stress (PSS)
• Sexual Chemistry (Eros)
• Instability (MSI)
• 2 validity scales
• Inconsistency (PAI)
• Infrequency (PAI)

8
Study 1 - Sample

• 5,315 online respondents
• After removing
• Incomplete or invalid responses
• Multivariate outliers
• 26yo (10yrs)
• 83 Female
• 76 Caucasian
• 26 High school ed. or lower
• 27K average income
• 24 married, 16 engaged, 60 dating

9
Evaluating Previous Scales

• IRT results
• Simultaneous analysis
• 66 items of existing scales
• Some very informative items
• Many poor items

10
DAS-31 (Degree of happiness, all things
considered, of your relationship)
Response Curves
Information Curve
11
DAS/MAT 5Agreement on FRIENDS
Response Curves
Information Curve
12
MAT 12In leisure time, do you (and does your
mate) prefer to be on the go or to stay at home?
Response Curves
Information Curve
13
From Items to Scales

• A scales information
• sum of information from each item
• How informative
• Across different levels of happiness

14
Test Info for Current Measures
15
Analysis of Existing Measures

• Many uninformative items
• Particularly for DAS and MAT
• ? noise / error
• Modest test information
• For all scales
• Notably poor for MAT and DAS
• Room for improvement

16
Creating the CSI

• 141 item pool
• Screen for contaminating items
• Screen for redundant items
• IRT on remaining 66 items
• Select 32 most effective

17
Test Info for CSI Scales
18
Basic Psychometrics
Alpha Distress Cut Score Correlations Correlations Correlations Correlations Correlations Correlations
Alpha Distress Cut Score 1 2 3 4 5 6
1. DAS .94 97.5 -- -- -- -- -- --
2. MAT .84 95.5 .90 -- -- -- -- --
3. QMI .96 24.5 .85 .87 -- -- -- --
4. RAS .92 23.5 .86 .87 .91 -- -- --
5. CSI-32 .98 104.5 .91 .91 .94 .96 -- --
6. CSI-16 .98 51.5 .89 .90 .96 .95 .98 --
7. CSI-4 .94 13.5 .87 .88 .93 .94 .97 .97
19
Correlations with Anchors
Arguing IAI Instability MSI Comm. CPQ Stress PSS Conflict MCI Sexual Chemistry EROS Neuroticism EPQN
DAS -.79 -.74 .73 -.53 -.54 .42 -.40
MAT -.76 -.74 .69 -.49 -.49 .41 -.38
CSI 32 -.79 -.78 .71 -.52 -.48 .45 -.38
CSI 16 -.80 -.78 .71 -.53 -.49 .43 -.38
20
What have we gained?

• Identical correlational results
• Strong convergent validity
• Strong discriminant validity
• Strong construct validity
• ? Measuring same thing
• Higher information
• ? Should have
• Lower Noise
• Higher Precision
• Greater Power

21
Satisfaction Groups

• IRT satisfaction estimates
• For each subject
• Based on MAT, DAS, CSI items
• (equivalent of GRE scores)
• Created satisfaction groups
• N 265
• HIGHLY similar SAT within each group
• MAT, DAS CSI scores also similar?

22
Precision CSI-32 vs. DAS
23
Effect Size

• Ability to detect difference
• Between groups
• Pre Post
• Effect Size M1 M2 .
• pooled SD
• Difference in SD units
• Power for detecting Ds in SAT groups

24
Power CSI-32 vs. DAS
25
STUDY 1 - Conclusions

• CROSS-SECTIONALLY
• CSI assess same construct
• Higher precision
• Higher power
• NEXT STEP
• Longitudinal analysis
• Responsiveness to change over time

26
TOPIC 2 Responsiveness

• Detecting change
• Assumption
• Longitudinal
• External Criteria
• Treatment effect
• Clinician
• Interviewer
• Global report
• SERM (Sdiff)
• Noise over time
• Estimating
• Two main applications
• Individual change
• Clinically distinct groups

27
Studies 2 through 4

• Study 2
• 267 online respondents
• 1 2wk follow ups
• 468 change scores
• Study 3
• 156 online respondents
• 1, 2, 3, 4, 6 12mo follow ups
• 455 change scores
• Study 4
• 545 online respondents
• 1, 2, 3, 4, 6, 9 12mo follow ups
• 1,552 change scores

28
Studies 2-4 Measures

• Relationship satisfaction scales
• DAS-32
• MAT-15
• CSI-32
• CSI-16
• CSI-4
• 3 global relationship change items
• Change since last assessment

29
Individual Change

• How many points of change needed?
• (to show significant change)
• SERM in No Change
• RCI (Jacobson Truax, 1991)
• MDC95 (Stratford et al., 1996)
• MDC95 (SD units) 1.96SERM .
• SD
• PRESENTING
• Meta-Analytic Summary
• Standardized Units

30
Reliable Individual Change
C
C
C
B
A
31
Detecting Change

• Individual Change
• IRT optimization
• Longer scales
• Distinct Groups
• Can scales distinguish?
• Mild deterioration
• No change
• Mild improvement

32
Perceived Change
MuchWORSE SomewhatWORSE A littleWORSE Stayed the SAME A little BETTER Somewhat BETTER Much BETTER
-3 -2 -1 0 1 2 3

• How much have these changed?
• Overall happiness in the relationship
• Feeling close and connected
• Stability of the relationship

33
Perceived Change
MuchWORSE SomewhatWORSE A littleWORSE Stayed the SAME A little BETTER Somewhat BETTER Much BETTER
-3 -2 -1 0 1 2 3
SignificantDeterioration SignificantDeterioration MinimalDeter. NoChange MinimalImprove. SignificantImprovement SignificantImprovement

• Averaged responses
• Alpha .92
• Created change groups

34
Distinct Change Groups
35
Quantifying Group-Level Responsiveness

• MCID
• (Guyatt, Walter Norman, 1987)
• Noise over time (SERM)
• Effect Sizes
• (Avg Change)IMPROVE (Avg Change)NO CHANGE
• SERM
• (Avg Change)DETERIORATE (Avg Change)NO CHANGE
• SERM

36
Analytic Strategy

• Improving method
• Multi-wave data
• Global change continuous
• Moderation
• HLM
• PV Global change score
• Moderators
• Gender
• T0 Satisfaction
• DV Change scores on scales (n 2475)
• ? Change scores 1pt global change
• ? MCID effect sizes
• Meta-Analytic Summary

37
Responsiveness in Dissatisfied (1SD below M)
A
B
C
C
C
D
C
B
A
A
38
Responsiveness in Satisfied (1SD above M)
A
A
B
B
B
E
D
C
B
A
39
Responsiveness Conclusions

• Can be quantified
• Scale selection
• Power estimates
• Responsive scales
• Greater power
• Individual
• Group
• Cross-sectional ? Longitudinal
• Precision Power translate
• NEXT STEP ? Treatment Effects

40
Topic 3 Bi-Dimensional View

• Uni-Dimensional view
• Positive feelings opposite negative feelings
• Bi-Dimensional view
• Pos/Neg independent
• Moderately dissatisfied
• Ambivalent
• Indifferent
• Uni-Dimensional obscuring?

41
Background

• Fincham Linfield (1997)
• PN-QIMS
• Two 3-item scales
• Qualities of spouse
• Feelings toward spouse
• Feelings about marriage
• Considering only (pos/neg)
• Separated in survey
• CFA in 123 couples
• Unique information

42
Study 5

• Mattson et al. (under review)
• New pos-neg scale
• 7 SMD items of CSI
• Pos / neg separately
• Large online sample
• Ambivalent
• Indifferent

Positive Items Positive Items Negative Items Negative Items
InterestingEnjoyable Friendly Hopeful Sturdy Good Full Boring Miserable Lonely Discouraging Fragile Bad Empty
43
Study 5 - Sample

• 1656 online respondents
• Demographics
• 28yo (7yrs)
• 94 Female
• 87 Caucasian
• 30k income
• 5 high school
• Romantic relationships
• 38 married (6.5yrs)
• 19 engaged (3.6yrs)
• 41 dating exclusive (2.4yrs)

44
Ambivalence vs. Indifference

• Median Splits

Positives Positives
High Low
Negatives Low Satisfied Indifferent
Negatives High Ambivalent Distressed
45
Ambivalence vs. Indifference

• Median Splits

Positives Positives
High Low
Negatives Low Satisfied Indifferent
Negatives High Ambivalent Distressed
46
Uni-Dimensional Satisfaction
47
Negative Conflict
48
Negative Affect
49
Study 6

• IRT Optimized Positive Negative Scales
• Item Pools
• 20 positive items
• 20 negative items
• Large sample
• UG respondents
• Analyses
• EFA
• Redundancy
• IRT
• Precision / Power / Validity

50
Study 6 - Sample

• 1,814 undergrad respondents
• Demographics
• 19yo (2yrs)
• 77 Female
• 72 Caucasian
• Together 2.6yrs
• 26 dissatisfied
• Close relationships
• 54 romantic partners
• 38 friends
• 5 family members
• 3 roommates
• Romantic relationships
• 76 dating exclusive
• 21 dating non-exclusive

51
Positive-Negative Relationship Qualities

• New PN-RQ scales
• Best 4 8 items by IRT

Positive Items Positive Items Negative Items Negative Items
Enjoyable Pleasant Strong Alive Fun Full Exciting Energizing Bad Empty Miserable Lifeless Discouraging Unpleasant Weak Dull
52
PN-RQ Correlations
CSI-4 Pos-RQ Neg-RQ
CSI-4 1
Pos-RQ .68 1
Neg-RQ -.55 -.47 1
53
Information Provided
54
NEG Information Provided
55
Power Positive-Quality Groups
56
Power Negative-Quality Groups
57
Uni-Dimensional Satisfaction
58
Negative Conflict
59
Forgivingness
60
PN-RQ

• Power from Optimization
• More precise
• Unique Information
• Ambivalent vs. Indifferent
• NEXT Responsiveness

61
Study 7

• PREP
• Psycho-educational workshop
• Speaker-Listener Technique
• CARE
• Psycho-educational workshop
• Acceptance based techniques (IBCT)
• Awareness
• Self-guided
• Semi-strutured
• No Treatment

62
Study 7

• 173 Newlywed couples
• Engaged or married lt6mo
• Screened for severe discord (MAT below 85)
• Demographics
• AGE 29
• Caucasian H 58 W 54
• Latino/a H 17 W 23
• Asian Am H 9 W 11
• African Am H 5 W 5
• Assessed
• MAT, PN-QIMS
• 6 points over 3yrs

63
Slope-Intercept HLM Results

• MAT
• Drops over time for Men
• Negative Qualities
• ns
• Positive Qualities
• Drops only in No Treatment
• TX Sig better slopes in Men

64
Bi-Dimensional View

• Distinct individuals
• Distinct treatment effects
• Enhance
• Theories
• Clinical work

65
Topic 4 Implicit Assessment

• Limitations of Self-Report
• Insight
• Biases
• Limitations of Observational Coding
• Costly
• Evaluation apprehension
• Not all constructs observable
• Implicit assessment
• Indirect
• Inexpensive
• Unique insights

66
Previous Work

• Me/Not-Me task
• Aron, Aron, Tudor, Nelson (1991), Aron
  Fraley (1999), Slotter Gardner (2009)
• Rxn Time on Evaluations
• Fincham, Garnier, Gano-Phillips, Osborne
  (1995)
• Partner-focused IAT
• Zayas Shoda (2005)
• Banse Kowalick (2007)
• Scinta Gable (2007)
• Self-focused IAT
• Dewitt, de Houwer, Buysse (2008)
• Sequential priming task
• Scinta Gable (2007)

67
Go/No-Go Association Task

• Partner-GNAT
• Sort three types of words
• Good
• Bad
• Partner
• Presented
• One at a time
• In random order
• Spacebar for targets

68
GNAT Stimuli

• Partner words
• First name
• Nick name
• Pet name / Distinguishing characteristic

Good Words Bad Words
Studies 8 10 PeaceGiftVacation DeathAccidentTragedy
Study 9 AcceptingSharingUnderstanding AttackingNaggingCriticizing
69
GNAT Procedure

• Procedure
• Obtain partner stimuli
• Sorting task
• 16 practice trials good as target
• 16 practice trials bad as target
• 70 trials partner good as targets
• 70 trials partner bad as targets
• Complete counterbalancing

CriticalTrials
70
GNAT

• Fast task (600msec)
• Accuracy
• D index
• Proposed
• High performance on P-good
• ? Strong positive implicit attitude
• High performance on P-bad
• ? Strong negative implicit attitude

71
Studies 8 9 Samples

• Study 8
• 122 online respondents
• 39 married (for 3.3yrs)
• 13 engaged (together for 2.7yrs)
• 58 dating (for 2.4yrs)
• 79 Caucasian
• 87 Female
• 43 provided follow-up data
• 8 ended their relationships
• Study 9
• 100 online respondents
• 10 married (for 3.6yrs)
• 12 engaged (together for 3.2yrs)
• 77 dating (for 1.8yrs)
• 77 Caucasian
• 86 Female
• 63 provided follow-up data
• 11 ended their relationships

72
Method Variance

• P-good and P-bad
• r .45
• Shared method variance
• Ability
• Effort
• Attention
• Comfort with computers
• Enter as pairs
• Simultaneous PVs
• Partial correlations
• Shared variance dropped
• Examine interaction
• Pos Neg attitudes might interact

73
Study 8 Correlations
Initial Self Reports Initial Self Reports Initial Self Reports
Relationship Satisfaction Negative Conflict Neuroticism
Partner-Good .14 -.05 .00
Partner-Bad -.21 .12 .03
74
Studies 8 9 Analytic Strategy

• Discrete-time hazard modeling in HGLM

LEVEL 1 Prob(Breakup) P log P/(1-P)
p0 p1(time) e
75
Study 8 Prediction of Relationship Breakup over
1 year
B SE p Odds Ratio
Intercept Intercept Intercept Intercept Intercept Intercept
Intercept1 -4.14 0.36 .001 0.02
Predicting Breakups over 1 Year Predicting Breakups over 1 Year Predicting Breakups over 1 Year Predicting Breakups over 1 Year Predicting Breakups over 1 Year Predicting Breakups over 1 Year
Intercept2 2.21 0.51 .001 9.12
Relationship Satisfaction -0.05 0.03 .065 0.95
Neuroticism 0.10 0.10 .342 1.10
Hostile Conflict -0.07 0.07 .317 0.93
Partner with Good (d) -1.75 0.55 .002 0.17
Partner with Bad (d) 0.38 0.93 .685 1.46
P-Good X P-Bad -1.67 0.83 .048 0.19
NOTE B unstandardized beta SE standard
error. p lt .10 p lt .05
76
Study 8 Probabilities of Breakup
77
Study 9 Correlations
Initial Self Reports Initial Self Reports Initial Self Reports
Relationship Satisfaction Negative Conflict Neuroticism
Partner-Good .06 -.13 .00
Partner-Bad -.03 .07 .03
78
Study 9 Prediction of Relationship Breakup over
1 year
B SE p Odds Ratio
Intercept Intercept Intercept Intercept Intercept Intercept
Intercept1 -3.84 0.34 .001 0.02
Predicting Breakups over 1 Year Predicting Breakups over 1 Year Predicting Breakups over 1 Year Predicting Breakups over 1 Year Predicting Breakups over 1 Year Predicting Breakups over 1 Year
Intercept2 2.69 0.48 .001 14.75
Relationship Satisfaction -0.03 0.04 .363 0.97
Neuroticism 0.13 0.11 .232 1.14
Hostile Conflict -0.03 0.04 .399 0.97
Partner with Good (d) -1.03 0.37 .007 0.36
Partner with Bad (d) 0.86 0.47 .074 2.35
P-Good X P-Bad -0.22 0.61 .717 0.80
NOTE B unstandardized beta SE standard
error. p lt .10 p lt .05
79
Study 9 Probabilities of Breakup
80
Studies 8 9 Summary

• Partner-GNAT
• Predicts Breakup over 1yr
• After controlling for SR scales
• Possible interaction
• Suggests
• Partner-GNAT provides unique information
• Ps unable to report
• Ps unwilling to report
• Next Step Mechanism of action

81
Study 10

• Partner-GNAT
• Generic good or bad words
• Good stimuli freedom, pleasure, gift
• Bad stimuli death, accident, poverty
• Behavioral coding
• Two 10-minute Problem discussions
• Two 10-minute Social Support discussions
• Two teams of naïve coders
• Self-report data

82
Coding Process

• Two separate teams (5 and 7 coders)
• Weekly meetings
• Spouses coded in separate passes
• 30sec intervals
• Global codes
• Counterbalancing
• Order of couples
• Order of spouses (within each interaction)
• Order of topics (H vs. W)
• Rated 15-18 dimensions
• All coders coded all tapes
• Codes averaged within coders interactions
• Codes averaged across coders
• Created composite codes

83
Composite Codes

• Support Behavior/Affect
• Emotional Support
• Negative Behavior
• Conflict Behavior/Affect
• Empathic Listening
• Affection
• Negative Behavior

84
Study 10 Sample

• 57 couples
• 48 engaged to be married (in 4.8mo)
• 52 married (for 3.7mo)
• Relationships
• Together 3.3yrs
• Highly satisfied (avg. CSI 141)
• 81 premarital cohabitation
• 93 living together at T0
• 14 had children at T0
• Demographics
• Age 28yo
• 91 Caucasian
• 53k joint income
• 9 HS education

85
Study 10 Analytic Strategy

• Actor-Partner Interdependence Modeling in HLM
• Modeling trajectories over time
• Two level model
• Level 1 Individual differences
• GNAT indices
• Coded behavior
• Initial self-report
• Level 2 Dyadic variables
• Relationship length
• Number of children

86
Male Emotional Support(during support
interaction)
87
Male Negative Behavior (during support
interaction)
88
Female Empathic Listening(during conflict
interaction)
89
Female Affection(during conflict interaction)
Female Affection
90
Male Negative Behavior(during conflict
interaction)
Male Negative Behavior
91
Study 10 Summary

• Partner-GNAT
• Linked to own behavior
• Linked to partners behavior
• Across domains
• ? Might shape each other
• Tailoring GNAT
• Implicit assessment of attachment?

92
Self Report Attachment Scales

• ECR-R
• Attachment Anxiety
• I worry a lot about my relationship
• Attachment Avoidance
• I find it difficult to allow myself to depend on
  romantic partners
• Difficult to disentangle
• Attachment
• Preoccupied / Dismissive Behaviors
• Requires insight / honesty

93
Study 11

• Standard Battery of SR scales
• Implicit Attachment
• Partner-GNAT
• Self-GNAT
• New Valence Categories
• Relationally Worthy
• Relationally Worthless
• Hypotheses
• Partner-GNAT ? internal working model of others
• Self-GNAT ? internal working model of self

94
GNAT Stimuli

• Partner words
• First name
• Nick name
• Pet name / characteristic

• Self words
• First name
• Last name
• Nick name / characteristic

Relationally Worthy Relationally Worthless
Valence Stimuli LovedLiked AgreeableAcceptedCherishedValuedAdored InferiorRejected DisagreeablePatheticUnwantedAbandonedInsignificant
95
Study 11 Sample

• Recruitment underway
• First 48 couples
• 79 committed dating relationships (1.6yrs)
• 4 engaged (2.9yrs)
• 17 married (4.5yrs)
• Relationships
• Quite satisfied (CSI-16 70)
• Dissatisfied
• 6 of married
• 9 of dating
• Demographics
• Age 24yo
• 76 Caucasian
• 37k joint income
• 9 HS education

96
Study 11 Correlations
Initial Self Reports Initial Self Reports Initial Self Reports
Relationship Satisfaction Attachment Anxiety Attachment Avoidance
Partner-Good .27 .01 -.23
Partner-Bad -.14 .06 .17
97
Study 11 Correlations
Initial Self Reports Initial Self Reports Initial Self Reports
Relationship Satisfaction Attachment Anxiety Attachment Avoidance
Self-Good .04 .07 .03
Self-Bad .01 -.08 -.03
98
Future Directions

• Unique information
• Beyond SR
• Clinically useful?
• Shapes behavior
• Longitudinal mediation?
• Change over time?
• Can be Tailored
• Attachment?
• Alternate Targets
• Family
• Friends
• Behaviors
• Moderators
• Mindfulness
• Assimilation of Partner into Self-Concept

99
Topic 5 Attention / Effort

• Inattention
• Adds error / noise
• Reduces power
• Quantifying
• Large Clinical Inventories (e.g., PAI)
• Infrequency
• Inconsistency
• Experimental Research
• Instruction reading (IMC Oppenheimer, 2009)
• Survey Research
• Unknown

100
Study 12

• Quantifying Inattention
• Behavioral Measures
• 7 directed responses
• 20 pronoun task
• 2min video
• Self-Report
• Inattentive
• Patterned
• Rushed
• Instruction skipping

101
Study 12 - Sample

• 575 online respondents
• 54 Mturk.com
• 13 online forums
• 33 UG psychology students
• Demographics
• 29yo (12yrs)
• 70 Female
• 77 Caucasian
• 21 High School
• 30 30k / year

102
Behavioral Inattention
103
Self-Reported Inattention
104
Distinct from Desirability
Inattention Measure Self Deception Impression Management
Mistakes on directed Qs -.16 -.16
Time watching video .11 .12
Mistakes on pronouns -.06 -.07
SR Inattentive -.30 -.34
SR Patterned -.15 -.20
SR Rushed -.28 -.24
SR Skip Instructions -.15 -.18
105
Screening for Inattention

• Developing ARS
• Item pool
• Infrequent items
• Inconsistent pairs
• 3 large online samples
• Ability to discriminate
• P responses
• Random data
• Random responders

106
Studies 13 through 15

• Study 13
• 1195 online respondents
• 85 female
• 77 Caucasian
• 26yo (SD 8.4)
• Study 14
• 1878 online respondents
• 91 female
• 85 Caucasian
• 28yo (SD 7.1)
• Study 15
• 547 online respondents
• 74 female
• 72 Caucasian
• 20yo (SD 1.3)

107
Final ARS scales

• Two scales
• 11 infrequency items
• I enjoy the music of Marlene Sandersfield
• I look forward to my time off
• 11 inconsistency item pairs
• I am an active person
• I have an active lifestyle
• Agreement with PAI
• Study 14
• Continuous rs .64 and .83
• Categorical kappa .72

108
Ability to Detect Inattention
109
Convergent Validity

• Study 12 indices
• ARS inattentive respondents
• Higher on inattention indices?
• Behavioral Markers
• Self-Report
• Comparable regression results?

110
ARS Inattentive Ps
Index Effect Cohens D
Directed Qs 3 more mistakes / 7 -1.43
Video Watching 40 seconds less / 120 .95
Pronouns 7 more mistakes / 20 -1.01
SR Inattentive 1.0 pts higher / 6 -.91
SR Patterned 1.3 pts higher / 7 -1.11
SR Rushed 0.7 pts higher / 7 -.60
SR Skipping Instructions 0.8 pts higher / 7 -.53
111
ARS Inattentive Ps

• Robins et al. (2001)
• Big 5 ? Self Esteem
• R2 .34
• 3 sig coeffs
• Attentive Ps
• N 621
• R2 .41
• 3 sig coeffs
• Inattentive Ps
• N 55
• R2 .08 ns
• no sig coeffs

ns
112
ARS Convergent Validity

• ARS inattentive respondents
• Higher inattention
• Behavioral Markers
• Self-Report
• Adding noise
• Lowering power

113
Study 16

• Reading instructions?
• ARS vs. IMC
• Oppenheimer (2009)
• IMC Instructional Manipulation Check
• Single paragraph / item
• Eliminates 20-40 of Ps
• Enhances power
• Paragraph manipulations
• Sports ticket
• Can of pop

114
Sports Participation
Most modern theories of decision making recognize
the fact that decisions do not take place in a
vacuum. Individual preferences and knowledge,
along with situational variables can greatly
impact the decision process. In order to
facilitate our research on decision making we are
interested in knowing certain factors about you,
the decision maker. Specifically, we are
interested in whether you actually take the time
to read the directions if not, then some of our
manipulations that rely on changes in the
instructions will be ineffective. So, in order to
demonstrate that you have read the instructions,
please ignore the sports items below, as well as
the continue button. Instead, simply click on
the title at the top of this screen (i.e.,
Sports Participation) to proceed to the next
screen. Thank you very much.
Which of these activities do you engage in
regularly? (click all that apply)
skiing
soccer
snowboarding
running
hockey
football
swimming
basketball
tennis
cycling
Continue
115
Sports Participation
Most modern theories of decision making recognize
the fact that decisions do not take place in a
vacuum. Individual preferences and knowledge,
along with situational variables can greatly
impact the decision process. In order to
facilitate our research on decision making we are
interested in knowing certain factors about you,
the decision maker. Specifically, we are
interested in whether you actually take the time
to read the directions if not, then some of our
manipulations that rely on changes in the
instructions will be ineffective. So, in order to
demonstrate that you have read the instructions,
please ignore the sports items below, as well as
the continue button. Instead, simply click on
the title at the top of this screen (i.e.,
Sports Participation) to proceed to the next
screen. Thank you very much.
Which of these activities do you engage in
regularly? (click all that apply)
skiing
soccer
snowboarding
running
hockey
football
swimming
basketball
tennis
cycling
Continue
116
Study 16 - Sample

• 652 online respondents
• 60 Mturk.com
• 40 UG psychology students
• Demographics
• 28yo (11.5yrs)
• 70 Female
• 74 Caucasian
• 27 High School
• 30 30k / year

117
ARS IMC Agreement
Kappa .16 Kappa .16 IMC IMC Total
Kappa .16 Kappa .16 OK Exclude Total
ARS OK 435 138 91
ARS Exclude 24 30 9
Total Total 73 27 627
118
Inattention Indices
119
Sunk Cost Task (Thaler, 1985)

• Imagine that your favorite football team is
  playing an important game that you
• have paid handsomely for.
• have received from a friend.
• However, on the day of the game, it happens to
  be freezing cold. What do you do?

ns
120
Soda Pricing Task (Thaler, 1985)

• You are on the beach on a hot day. For the last
  hour you have been thinking about how much you
  would enjoy an ice cold can of soda. Your
  companion needs to make a phone call and offers
  to bring back a soda from the only nearby place
  where drinks are sold, which happens to be a
• run-down grocery store.
  fancy resort.
• Your companion asks how much you are willing to
  pay for the soda and will only buy it if it is
  below the price you state. How much are you
  willing to pay?

ns
ns
121
Inattention Summary

• Inattention
• Skipping instructions
• As high as 20-40
• Skimming items
• 5-10
• ARS effectively screens
• Enhances power

122
Summary

• RELATIONSHIP QUALITY
• T1 IRT Optimization
• Study 1
• T2 Responsiveness to Change
• Studies 2-5
• T3 Bi-Dimensional View
• Studies 6-7
• T4 Implicit Measures
• Studies 8-10
• ATTENTION
• T5 Screening for Error Variance
• Studies 11-15

123
Limitations

• Online samples
• Largely female
• Largely Caucasian
• Lacking behavioral criteria

124
Thank You.
125
Existing Measures
Measure Citations Cit./Year
DAS (32) 2,237 77.1
MAT (16) 1,476 32.1
QMI (6) 218 9.9
KMS (3) 179 9.4
RAS (7) 150 8.8
126
Criterion Validity

• DAS Distress groups
• Current gold-standard
• DAS score lt 97.5
• 1027 DAS distressed Ps
• ROCs to identify CSI cut scores
• Identified CSI distressed Ps
• 91 agreement w/ DAS

127
Precision CSI-16 vs. MAT
128
Power CSI-16 vs. MAT
129
Studies 2-4 Demographics

• SAMPLE
• N 2,056 initial respondents
• N 968 (47) respondents with longitudinal data
• AGE
• M 27.7yo (9.3yrs)
• GENDER
• 71 Female
• 29 Male
• RACE
• 83 Caucasian
• 5 Asian
• 4 African American
• 4 Latino
• SES
• 10 High school diploma or less

130
Studies 2-4 Relationships

• Relationship Types
• 37 Married 7.9 yrs (7.9 yrs)
• 13 Engaged 3.2 yrs (2.4 yrs)
• 50 Dating 1.8 yrs (1.9 yrs)
• Relationship Satisfaction (MAT)
• Married 108 (32)
• Engaged 122 (24)
• Dating 116 (24)
• Dissatisfied Respondents
• 24 (n 487)

131
Study 2 - Sample

• N596 initial respondents
• 27yo (SD 10yrs)
• 77 Female
• 84 Caucasian
• 8 High school
• 22K income
• 30 married, 14 engaged, 55 dating
• 16 dissatisfied
• 372 provided email (62)
• 267 completed follow ups (71)
• NS differences on
• Length of relationship Relationship satisfaction
• Age Education
• Gender

132
Study 3 - Sample

• N398 initial respondents
• 26yo (SD 8yrs)
• 86 Female
• 80 Caucasian
• 9 High school
• 20K income
• 30 married, 12 engaged, 58 dating
• 24 dissatisfied
• 252 provided email (63)
• 156 completed follow ups (62)
• NS differences on
• Length of relationship Relationship satisfaction
• Gender Ethnicity

133
Study 4 - Sample

• N1062 initial respondents
• 29yo (SD 9yrs)
• 79 Female
• 83 Caucasian
• 11 High school
• 29K income
• 44 married, 13 engaged, 43 dating
• 28 dissatisfied
• 746 provided email (70)
• 545 completed follow ups (73)
• NS differences on
• Length of relationship
• Age Ethnicity

134
RCI and MDC95 Equations

• RCI
• SERM v2SD2(1 rxx)
• SERM v2MSE
• RCI (x2 x1) / SERM
• If RCI gt 1.96
• ? Sig individual change
• MDC95
• Solve RCI eq for (x2 x1)
• MDC95 1.96SERM

135
Estimating Noise Reliable Individual Change
Guyatt, Walter Norman (1987) Jacobson Truax
(1991)
NOISE

SEM(Standard Error of Repeated Measurement)

2MSE(MSE Mean Squared Error from a Repeated
Measures ANOVA on the T0, F1, F2 scores of No
Change individuals)
Dx
Signal
RELIABLE CHANGE



1.98
Noise
SEM
Dx Minimal Detectable Change (MDC95)(smallest
change in scores needed in an individual to
suggest reliable change)
136
Reliable Individual Change
Range SERM MCD95 SD units

CSI-32 0-161 8.0 15.7 .49 SDs
DAS 0-151 5.8 11.4 .65 SDs

CSI-16 0-81 4.8 9.4 .55 SDs
MAT 0-158 10.7 21.0 .76 SDs

CSI-4 0-21 1.6 2.8 .70 SDs
137
Estimating Powerfor Detecting Perceived Change
Guyatt, Walter Norman (1987)
Sensitivity to Perceived Change (difference in
avg change scores between adjacent perceived
change groups)
Signal
POWER (Effect size)


Noise
SEM(Standard Error ofRepeated Measurement)
138
Sensitivity to Perceived ChangeCSI-16 vs. MAT
139
Responsiveness Model

• Level 1 repeated assessments
• X2 X1 p0 p1(global change)
• p2(deterioration) e
• Level 2 individuals
• p0 b00
• p1 b10 b11(T0 rel sat) b12(male) r1
• p2 b20 b21(T0 rel sat)

140
Differences by Gender

• Scales showed slightly smaller effect sizes in men

141
Study 5

• Responsiveness to Mild Intervention
• Reissman, Aron, Bergen (1993)
• Pos. activities over 10wks
• Fun/Exciting ? Enhanced satisfaction
• 158 randomly assigned to
• Control
• Fun / Exciting Activities Feedback
• 2wk follow up
• 25 Fun / Exciting behaviors
• Satisfaction
• Scales
• CSI-32 DAS-32 MAT-15 QMI
• SMD RAS KMS PN-RQ

142
Fun/Exciting Activities Feedback

• Background
• There is a large body of research supporting the
  importance of fun in relationships.
• Unfortunately, many couples slowly forget to
  make time to do fun things together the longer
  they are together.
• Request
• As part of this study, we would like you and
  your partner to make an effort to have more fun
  with each other over the next 2 weeks.
• Specifically, we would like you to try to do
  some fun activities that get you out of the house
  and/or out of your normal routines.
• These activities should be fun and exciting for
  both of you.
• These activities should also involve things that
  you can do together (like going to dinner) rather
  than more solitary activities (like reading).

143
Fun/Exciting Activities Feedback

• Based on your responses, here is a list of
  activities you rated as most fun and exciting

Fun activity How fun/exciting you rated it How often you currently do it
Engaging in intimate sexual activity Extremely 7x in 2 weeks
Going to a movie Extremely 1x in 2 weeks
Playing sports Extremely 2x in 2 weeks
Going camping Very 0x in 2 weeks
Spending time with friends Very 6x in 2 weeks
Going on a hike Very 3x in 2 weeks
Going on a picnic Very 0x in 2 weeks
Attending community events (e.g., festivals) Very 0x in 2 weeks
Going to the beach or lake Very 1x in 2 weeks
Going to a restaurant Somewhat 4x in 2 weeks
144
Study 5 - Sample

• 158 initial respondents (first 3 ½ days of
  recruitment)
• 30yo (SD 11yrs)
• 74 Female
• 83 Caucasian
• 18 High school
• 53K income
• 39 married, 10 engaged, 51 dating
• 18 dissatisfied
• xxx completed follow ups (73)
• NS differences on
• Length of relationship
• Age Ethnicity

145
PREPPrevention and Relationship Enhancement
Program

• 14 hour workshop over 4 sessions
• One weekend day
• Three weeknights
• Communication Skill Focus
• Speaker-Listener Technique
• Problem-Solving skills
• Time Outs
• Building Positive Behaviors
• Goals
• prevent conflict escalation (improve resolution)
• enhance/protect positive aspects of relationship

146
CARECompassionate and Accepting Relationships
through Empathy

• 14 hour workshop over 4 sessions
• One weekend day
• Three weeknights
• 3 Acceptance based skill modules
• Support skills
• Conflict skills
• Forgiveness skills
• Goal increase understanding/acceptance
• To buffer rough spots
• To smooth out conflict discussions
• To protect positives

147
Awareness Condition

• Movie Treatment
• List of relationship-focused movies
• Watched 5 movies together
• 40 min guided discussion after each
• First movie in a group setting (at UCLA)
• Yoked Control Group
• Equivalent time together
• Equivalent time discussing relationship
• No active psycho-educational component

148
Treatment Conditions
CARE PREP Aware No Tx
N assigned 53 45 45 52
Initial satisfaction 121 117 125 117
Tx Dropout 9 10 21 --
149
Hypotheses

• All treatment conditions would show better
  marital quality than no tx
• CARE and PREP would show better marital quality
  than the minimal tx
• CARE would demonstrate comparable tx effects to
  PREP

150
Longitudinal Assessments

• T0 1-2 months prior to workshop
• T1 start of workshop
• T2 6 months after workshop
• T3 1 year
• T4 2 years
• T5 3 years

151
Previous Work

• Me/Not-Me task
• Implicit Closeness ? 3mo shift in SR closeness
• Aron, Aron, Tudor, Nelson (1991), Aron
  Fraley (1999), Slotter Gardner (2009)
• Rxn Time on Evaluations
• High Accessibility ? Stronger effects among SR
  scales
• Fincham, Garnier, Gano-Phillips, Osborne
  (1995)
• Partner-focused IAT
• Pos Implicit Atttitude ? Secure attachment (
  lower attch avoidance)
• Zayas Shoda (2005)
• Pos Implicit Attitude ? Criterion validity
  (separating groups)
• Banse Kowalick (2007)
• Pos Implicit Attitude ? Current relationship
  satisfaction
• Scinta Gable (2007)
• Self-focused IAT
• Implicit Relational Worthiness ? lower attachment
  anxiety preoccupation
• Implicit Relational Anxiety ? preoccupied
  attachment

152
Composite Codes

• Support Behavior/Affect
• Emotional Support
• Understanding Reassuring
• Responsive Relieving blame
• Negative Behavior
• Frustration Hostility
• Disagreeing Blaming
• Tension
• Conflict Behavior/Affect
• Empathic Listening
• Tuned into Ps feelings Supportive
• Validating Interested / Curious
• Affection
• Warm / Affectionate Humorous / Playful
• Negative Behavior

153
Study 10 Analytic Strategy

• Actor-Partner modeling in HLM

LEVEL 1 Relationship Behavior p1(male
X own satisfaction) p2(male X spouses
satisfaction) p3(female X own
satisfaction) p4(female X spouses
satisfaction) similar sets of APIM terms
for hostile conflict neuroticism
p13(male X own partner-good) p14(male X
spouses partner-good) p15(female X own
partner-good) p16(female X spouses
partner-good) p17(male X own partner-bad)
p18(male X spouses partner-bad)
p19(female X own partner-bad) p20(female X
spouses partner-bad) APIM terms for
interactions between partner-good and partner
bad e
Self-Report Controls
Partner-GNAT Performance
LEVEL 2 p1 b10 p2 b20 p3
b30 b31(rel length) b32( of kids) r3
p4 b40 (similar equations for remaining
lvl2 effects)
154
POS-RQ Distinct Change Groups
155
NEG-RQ Distinct Change Groups
156
POS-RQ Distinct Change Groups