Performance of Statistical Learning Methods

1
Performance of Statistical Learning Methods
Jens Zimmermann zimmerm_at_mppmu.mpg.de
Max-Planck-Institut für Physik,
München Forschungszentrum Jülich GmbH
Performance Examples from Astrophysics Performance
vs. Control H1 Neural Network Trigger Controlling
Statistical Learning Methods Overtraining Effic
iencies Uncertainties Comparison of Learning
Methods Artificial Intelligence Higgs Parity
Measurement at the ILC
2
Performance of Statistical Learning Methods MAGIC
Significance and number of excess events scale
theuncertainties in the flux calculation.
3
Performance of Statistical Learning Methods XEUS
4
Control of Statistical Learning Methods
There may be many different successful
applicationsof statistical learning methods.
There may be great performance improvementscompar
ed to classical methods.
This does not impress people who fear
thatstatistical learning methods are not well
under control.
First talk Understanding and Interpretation Now
Control and correct Evaluation
5
The Neural Network Trigger in the H1 Experiment
Trigger Scheme
H1 at HERA ep Collider, DESY
L1 2.3 µs L2 20 µs L4 100 ms
10 MHz
500 Hz
50 Hz
10 Hz
Each neural network on L2 verifies a specific L1
sub-trigger.
6
Triggering Deeply Virtual Compton Scattering
Theory
7
Determine the correct efficiency
50 training set
signalshouldpeak at 1
backgroundshouldpeak at 0
8
Determine the Correct Efficiency
training set

test set

9
Check Statistical Uncertainties
efficiency
10
Check Systematical Uncertainties
There is only a propagation ofsystematical
uncertainties of the inputs
Assumingx1 with absolute error s1x2 with
relative error s2 5x3 with relative error
s310
11
Check Systematical Uncertainties
example DVCS dataset
12
Comparison of Hypotheses
NN 96.5 vs. SVM 95.7 Statistically
significant? Build 95 confidence interval!
13
Comparison of Learning Methods
Compare performancesover different training sets!
Cross-Validation Divide dataset into k
parts,train k classifiers byusing each part
once as test set.
14
Artificial Intelligence
H1-L2NN TriggeringCharged Current
15
Artificial Intelligence
H1-L2NN Triggering J/y
16
Higgs Parity Measurement at the ILC
H/A t t- rn rn ppn ppn
17
Higgs Parity Measurement at the ILC
Statistical learning approach direct
discrimination
trained towards 0
trained towards 1
18
Conclusion
Statistical Learning Methods successful in
many applications in high energy and
astrophysics. Significant performance
improvements comparedto classical
algorithms. Statistical learning methods are
well under control - efficiencies can be
determined - uncertainties can be
calculated. Comparison of learning methods
revealsstatistically significant
differences. Statistical Learning Methods
sometimes show more artificial intelligence than
expected.