The Receiver Operating Characteristic (ROC) Curve

1
The Receiver Operating Characteristic (ROC) Curve

• EPP 245/298
• Statistical Analysis of
• Laboratory Data

2
Binary Classification

• Suppose we have two groups for which each case is
  a member of one or the other, and that we know
  the correct classification (truth).
• Suppose we have a prediction method that produces
  a single numerical value, and that small values
  of that number suggest membership in group 1 and
  large values suggest membership in group 2

3

• If we pick a cutpoint t, we can assign any case
  with a predicted value t to group 1 and the
  others to group 2.
• For that value of t, we can compute the number
  correctly assigned to group 2 and the number
  incorrectly assigned to group 2 (true positives
  and false positives).
• For t small enough, all will be assigned to group
  2 and for t large enough all will be assigned to
  group 1.
• The ROC curve is a plot of true positives vs.
  false positives

4
datagen lt- function() truth lt-
rep(01,each50) pred lt- c(rnorm(50,10,1),rnorm(
50,12,1)) return(data.frame(truthtruth,predpre
d)) plot1 lt- function() nz lt-
sum(truth0) n lt- length(truth)
plot(density(pred1nz),lwd2,xlimc(6,18),
main"Generating an ROC Curve")
lines(density(pred(nz1)n),col2,lwd2)
abline(v10,col4,lwd2) abline(v11,col4,lwd2
) abline(v12,col4,lwd2) -------------------
---------------------- gt source(rocsim.r)gt
roc.data lt- datagen() gt attach(roc.data) gt plot1()
5
(No Transcript)
6
roc.curve lt- function(truth,pred,maxx) ntp lt-
sum(truth1) ntn lt- sum(truth0) n lt-
length(truth) preds lt- sort(unique(pred))
npred lt- length(preds) tp lt- vector("numeric",np
red1) fp lt- tp fp1 lt- ntn tp1 lt- ntp
for (i in 1npred) cutpt lt- predsi
tpi1 lt- sum((pred gt cutpt)(truth1))
fpi1 lt- sum((pred gt cutpt)(truth0))
plot(fp,tp, type"l",lwd2,xlimc(0,maxx))
title("ROC Curve") -----------------------------
-----------gt roc.curve(truth,pred,50)
7
(No Transcript)
8
datagen2 lt- function() truth lt-
rep(01,c(990,10)) pred lt- c(rnorm(990,10,1),rno
rm(10,12,1)) return(data.frame(truthtruth,pred
pred)) --------------------------------------gt
detach(roc.data) gt roc.data2 lt- datagen2() gt
attach(roc.data2)gt roc.curve(truth,pred,40)
9
ROC Curve for Rare Outcome