Analyzing Data from Small N Designs using Multilevel Models

1
Analyzing Data from Small N Designs using
Multilevel Models

• Eden Nagler
• The Graduate Center, CUNY
• David Rindskopf, Ph.D
• The Graduate Center, CUNY

2
Overview/Intro

• What is our current work?
• Where did we start?
• How does HLM fit into this framework?

3
2 Initial Datasets

• Stuart, R.B. (1967). Behavioral control of
  overeating. Behavior Research Therapy, 5,
  (357-365).
• Dicarlo, C.F. Reid, D.H. (2004). Increasing
  pretend toy play of toddlers with disabilities in
  an inclusive setting. Journal of Applied Behavior
  Analysis, 37(2), (197-207).

4
Stuart (1967)
5
Stuart (1967) Procedures for Getting data into
HLM
6
Stuart (1967) Procedures for Getting data into
HLM
7
Stuart (1967) Level-1 dataset
8
Stuart (1967) Level-2 dataset
9
Stuart (1967) HLM (Linear model)

• Linear Model
• POUNDS p0 p1(MONTHS12) e

10
Stuart (1967) HLM Linear Model Estimates

• Final estimation of fixed effects
• Standard Approx.
• Fixed Effect Coefficient Error
  T-ratio d.f. P-value
• --------------------------------------------------
  --------
• For INTRCPT1,P0
• INTRCPT2, B00 156.439560 5.053645 30.956 7
  0.000
• For MONTHS12 slope, P1
• INTRCPT2, B10 -3.078984 0.233772 13.171 7
  0.000
• --------------------------------------------------
  --------
• The outcome variable is POUNDS
• --------------------------------------------------
  --------
• POUNDSij 156.4 3.1(MONTHS12) eij

11
Stuart (1967) HLM Quadratic Model

• Quadratic Model
• POUNDS p0 p1(MONTHS12) p2(MON12SQ)e

12
Stuart (1967) HLM Quadratic Model Estimates

• Final estimation of fixed effects
• Standard Approx.
• Fixed Effect Coefficient Error T-ratio d.f.
  P-value
• --------------------------------------------------
  ---------
• For INTRCPT1, P0
• INTRCPT2, B00 158.833791 5.321806 29.846 7
  0.000
• For MONTHS12 slope, P1
• INTRCPT2, B10 -1.773039 0.358651 -4.944 7
  0.001
• For MON12SQ slope, P2
• INTRCPT2, B20 0.108829 0.021467 5.070 7
  0.001
• --------------------------------------------------
  ---------
• The outcome variable is POUNDS
• --------------------------------------------------
  ---------
• POUNDSij 158.8 1.8(MONTHS12) 0.1(MON12SQ)
  eij

13
Stuart (1967) HLM Linear vs. Quadratic Model
Stuart (1967) Actual Data
Linear Model Prediction
Quadratic Model Prediction
14
Dicarlo Reid (2004)
15
Dicarlo Reid (2004) Level-1 dataset
16
Dicarlo Reid (2004) Level-2 dataset
17
Dicarlo Reid (2004) HLM Simple Model

• Simple Model
• FREQRND p 0 p1(PHASE) e

18
Dicarlo Reid (2004) HLM Simple Model
Estimates

• Level-1 Model Level-2 Model
• logL P0 P1(PHASE) P0 B00 R0
• P1 B10 R1
• --------------------------------------------------
  --------
• Final estimation of fixed effects (Unit-specific
  model)
• Standard Approx.
• Fixed Effect Coefficient Error
  T-ratio d.f. P-value
• -------------------------------------------------
  ---------
• For INTRCPT1,P0
• INTRCPT2, B00 -0.769384 0.634548 -1.212 4
  0.292
• For PHASE slope,P1
• INTRCPT2, B10 2.516446 0.278095 9.049 4
  0.000
• -------------------------------------------------
  ---------
• LN(FREQRNDij) -0.77 2.52(PHASE) eij

19
Dicarlo Reid (2004) HLM Simple Model
Estimates

• LOG(FREQRNDij) B00 B10(PHASE) eij

For PHASE0 (BASELINE) LOG(FREQRNDij)
B00 FREQRNDij exp(B00)
For PHASE1 (TREATMENT) LOG(FREQRNDij) B00
B10 FREQRNDij exp(B00B10)
exp(B00)exp(B10)
Estimates B00 -0.77 B10 2.52
For PHASE0 (BASELINE) FREQRNDij exp(B00)
exp(-0.77)
0.46
For PHASE1 (TREATMENT) FREQRNDij exp(B00B10)
exp(-0.772.52) exp(1.75) 5.75
20
In conclusion

1. Other issues weve encountered and explored
2. Issues weve encountered, but not yet explored
3. Issues weve not yet encountered nor explored