Hypotheses

1
Hypotheses Testing

• Two overarching Issues
• 1) What are CSEP-able prediction schemes?
• 2) How do we test these?

2
Predictability

• The best (and only) test of predictability is to
  make a prediction!
• We must remember that ultimate goal of CSEP is to
  test EQ prediction algorithms, not just formulate
  technicalities of testing procedures

3
What does it take to be CSEP-able?

• Based on authorized data streams
• Reproducible
• Code must be runnable, adequately documented,
  archived by CSEP
• Have a standard output (to be determined by us!)
• Exists for RELM-type schemes
• Requires generalizations for CSEP-able schemes
• Still needs to be formulated for fault-based
  approaches

4
CSEP-able Prediction Techniques

• Grid based
• RELM models
• KB technique?
• Short-Term Forecasts
• STEP
• ETAS/ETES
• Fault-based
• McGuire technique
• Bowman/AMR technique
• Hybrids
• National Hazard Maps
• Ensemble/synthetic seismicity approaches
• Virtual California
• Other Silver-Bullets?

5
Synthetic Catalogs

• Not really CSEP-able by themselves
• Exceptions might include Ensemble forecasts
  based on models like Virtual California

6
Miscellaneous Issues

• How do we accommodate predictions for small
  areas/individual events (i.e. McGuire)?
• How do we cope with non-standard techniques?
• General agreement to focus efforts on seismicity
  deformation approaches
• May require proof-of-concept before acceptance
  algorithms or data into testing center
• Create a standard prediction depository for
  nonCSEP-able predictions? NOT at this time!
• How to accommodate CSEP algorithms on
  non-authorized data streams?
• Authorization process via data standards
  committee?
• Non-seismicity data streams considered beginning
  in Phase II/III
• Need near-real-time geodetic data to be included
  as authorized data stream (doesnt currently
  exist)
• Need standard authorized data stream for focal
  mechanisms relocated seismicity data

7
Philosophy of CSEP testing

• Definition of a prediction
• Testable statistical statement (hypothesis)
• Principles of how to formulate these hypotheses
  rather than give a list (still needs to be done)
• List of null hypotheses to test against
• One or more nulls per prediction class?
• Must have common output information
• Time-stamp Algorithms, predictions, data test
  results
• Feel-good vs. formal tests
• Danger of teaching to the test

8
Preprocessing Issues

• Authorized data streams
• To decluster or not to decluster
• (that is the
  question!)
• Input data declustering
• May use CSEP-authorized declustered data, or
  decluster as part of the algorithm
• Target declustering (i.e.predicting mainshocks
  vs. aftershocks)
• 3 data streams
• Non-declustered data
• Declustered
• DP-tagged

9
Testing techniques

• ROC / Molchan diagram
• For alarm-based probabilistic predictions
• Likelihood
• For probabilistic predictions
• Contingency tables

10
(No Transcript)
11
(No Transcript)
12
(No Transcript)
13
(No Transcript)
14
Fault-based Testing Issues

• Plan of action for fault based alarms
• use AMR as a straw-man to fill out the necessary
  elements of the forecast
• interface with Working Groups for fault
  definition
• determine how to declare a hit
• Authorized fault segmentation maps? (i.e. library
  of scenarios)

15
Revisiting Algorithms

• Possibility to re-run algorithms
• For different catalogs (synthetic as well)
• For different parameters
• For different targets
• This is a unique tool for research (not only for
  testing)