Procedures for Blind Analysis

1
Procedures for Blind Analysis

• Teruki Kamon (Texas AM)
• Jaco Konigsberg (Florida)
• Joe Kroll (Penn)
• Giovanni Punzi (Pisa)

CDF Executive Board 21 July 2005
2
What is the Issue?
Amend CDF Bylaws to make it possible to bless
an analysis procedure rather than a result we
usually refer to this procedure as making a blind
analysis
Spokespersons asked Kamon, Konigsberg, Kroll, and
Punzi to write these additions to the CDF Bylaws
Status circulating draft of additions to
conveners
3
Why is this Desirable?

• Many analyses can benefit from being carried out
  blind.
• new particle searches
• parameter measurements (MW, Mtop, ?ms)
• bump hunting (Bc, B!??)
• Need to define the procedures for labeling
  analysis blind
• for uniformity within CDF physics analysis groups
• to give term credibility outside CDF
• Current bylaws require the result at
  preblessing
• no provision to force scrutinizing analysis
  procedure before preblessing
• must wait at least two weeks to show result after
  unblinding
• contradicts philosophy of blind analysis

4
Basic Idea of Additions to Bylaws

• Follow the current preblessing and blessing
  protocol
• Monday prior to preblessing release CDF note
  describing analysis procedure
• at preblessing present the entire analysis
  procedure
• physics group has two weeks to comment on
  procedure
• at blessing sign off on procedure
• once procedure is blessed there are several
  possible paths
• apply procedure to data to get result from signal
  region
• wait before looking at data but present
  expectations (sensitivity)
• Remaining issue is defining the protocol after
  revealing results in signal region

5
Illustration New Particle Search

• CDF note and preblessing would cover
• optimized selection criteria for signal
• includes method used to establish these criteria
• expected signal rate (including systematics)
• includes acceptance and efficiency calculations
• background estimates in signal region (including
  systematics)
• possibly a validation of analysis in a control
  region
• method of calculating limit form of final
  result
• criteria or threshold for deciding there is a
  signal
• could have no such criteria decide result is a
  limit no matter what
• enough information is presented so anyone can
  reproduce above
• At blessing physics group signs off on all of
  the above
• can then look in signal region get result
• choose to wait for more data show expected
  sensitivity

6
Schedule

• Today draft of additions being discussed with
  conveners
• Thursday, Aug. 4th have draft that has the
  support of the conveners circulate to the CDF
  collaboration
• Friday, Aug. 19th discuss feed back from the
  collaboration at Joint Physics Meeting
• Thursday, Aug. 25th circulate final version to
  collaboration
• Thursday, Sept. 1st vote on amendments at the
  CDF executive board meeting

7
Some Final Observations
The idea is to make it easier to do analysis not
harder
In discussing blind analysis procedures always
get hung up on exceptions instead of majority
It is important to consider all possible
exceptions carefully define clear procedure for
all anticipated surprises
At present no defined procedure neither for
majority nor exceptions It is better to at least
have a procedure that covers the majority of the
situations we will encounter so lets not
debate procedure for surprises for eternity we
can always make further refinements