Geant4%20Physics%20Validation

1
Geant4 Physics Validation
Pablo Cirrone Giacomo Cuttone Francesco Di
Rosa Susanna Guatelli Alfonso Mantero Barbara
Mascialino Luciano Pandola Andreas Pfeiffer MG
Pia Pedro Rodrigues Giorgio Russo Andreia
Trindade Valentina Zampichelli

• M.G. Pia
• On behalf of the
• LowE EM and Advanced Examples
• Working Groups
• http//www.ge.infn.it/geant4/lowE

Geant4 Space User Workshop Pasadena, 6-10
November 2006
2
Geant4 Toolkit
Wide set of physics processes and
models Versatility of configuration according to
use cases
How accurate is Geant4 physics modelling? Which
is the most appropriate model for my simulation?

• Provide objective criteria to evaluate Geant4
  physics models
• Document their precision against experimental
  data
• Test all Geant4 physics models systematically
• Quantitative tests with rigorous statistical
  methods

3
Verification and Validationof Geant4 physics

• Verification
• compliance of the software results with the
  specifications (the underlying physics
  model)
• Unit tests (at the level of individual Geant4
  classes)
• Validation
• comparison against experimental data
• Quantitative estimate of the agreement between
  Geant4 simulation and reference data through
  statistical methods (Goodness-of-Fit)

A systematic, quantitative validation of Geant4
physics models against reference experimental
data is essential to establish the reliability
of Geant4-based applications
4
K. Amako et al., Comparison of Geant4
electromagnetic physics models against the NIST
reference dataIEEE Trans. Nucl. Sci., Vol. 52,
Issue 4, Aug. 2005, pp. 910-918

• Strategy
• Rigorous methods
• Systematic, quantitative comparisons
• Address all modeling options
• Statistical analysis of compatibility with
  experimental data
• Adopt the same method also for hadronic physics
  validation
• Start from the bottom (low energy)
• Progress towards higher energy based on previous
  sound assessments
• Guidance to users based on objective ground
• not only educated-guess PhysicsLists

Statistical Toolkit Goodness-of-Fit test
Quantitatitative comparison of experimental -
simulated distributions
5
Statistical Toolkit

• Launched as an ESA project
• 2nd development cycle
• Released April 2006
• Goodness-of-fit tests
• Binned distributions
• Unbinned distributions
• Performance analysis
• Power analysis

The most complete software tool for 2-sample GoF
tests
6
Recent validation activities

• Atomic relaxation
• Fluorescence and Auger transition energies
• Bremsstrahlung
• Angular distributions
• Proton Bragg peak
• Electromagnetic interactions
• Elastic scattering
• Pre-equilibrium
• Nuclear de-excitation
• other validation activities in Advanced Examples

More details see talks at IEEE NSS 2006
7
Geant4 Atomic Relaxation

• Geant4 Low Energy Electromagnetic package takes
  into account the detailed atomic structure of
  matter and the related physics processes
• It includes a package for Atomic Relaxation
• Simulation of atomic de-excitation resulting from
  the creation of a vacancy in an atom by a primary
  process

• Geant4 Atomic Relaxation models
• Fluorescence
• Auger electron emission

• It is used by Geant4 packages
• Low Energy Electromagnetic
• Photoelectric effect
• Low Energy electron ionisation
• Low Energy proton ionisation (PIXE)
• Penelope Compton scattering
• Hadronic Physics
• Nuclear de-excitation
• Radioactive decay

These physics models are relevant to many diverse
experimental applications
8
Geant4 fluorescence
Original motivation from astrophysics requirements
Cosmic rays, jovian electrons
X-Ray Surveys of Asteroids and Moons
Solar X-rays, e, p
Geant3.21
ITS3.0, EGS4
Courtesy SOHO EIT
Geant4
Induced X-ray line emission indicator of target
composition (100 mm surface layer)
250 keV
C, N, O line emissions included
Wide field of applications beyond astrophysics
Courtesy ESA Space Environment Effects Analysis
Section
9
Atomic Relaxation in Geant4

• Two steps
• Identification of the atomic shell where a
  vacancy is created by a primary process
  (photoelectric, Compton, ionisation)
• The creation of the vacancy is based on the
  calculation of the primary process cross sections
  relative to the shells of the target atom
• Cross section modeling and calculation specific
  to each process
• Generation of the de-excitation chain and its
  products
• Common package, used by all vacancy-creating
  processes
• Geant4 Atomic Relaxation
• Generation of fluorescence photons and Auger
  electrons
• Determination of the energy of the secondary
  particles produced

10
Modelling foundationin Geant4 Low Energy
Electromagnetic Package

• Calculation of shell cross sections
• Based on the EPDL97 Livermore Library for
  photoelectric effect
• Based on the EEDL Livermore Library for electron
  ionisation
• Based on Penelope model for Compton scattering
• Detailed atom description and calculation of the
  energy of generated photons/electrons
• Based on the EADL Livermore Library

11
Validation of Geant4 Atomic Relaxation

• Previous partial validation studies
  (collaboration with ESA Advanced Concepts
  Division)
• Pure materials limited number of elements
  examined
• Complex materials complex experimental set-up,
  large uncertainties on the target material
  composition
• Systematic validation project NIST database as
  reference

Authoritative, systematic collection of
experimental data
12
Method and tools

• Geant4 test code to generate fluorescence and
  Auger transitions from all elements
• Geant4 Atomic Relaxation handles 6 Z 100
• Selection of experimental data subsets from NIST
  database
• The NIST database also contains data from
  theoretical calculations
• Comparison of simulated/NIST data with
  Goodness-of-Fit test
• Data grouped for the comparison as a function of
  Z according to the initial vacancy and transition
  type
• Statistical Toolkit (http//www.ge.infn.it/statist
  icaltoolkit)
• Kolmogorov-Smirnov test
• The result of the agreement is expressed through
  the p-value of the test

13
Fluorescence Shell vacancy K
Shell-end Kolmogorov-Smirnov D p-value
5 0.0188 1
6 0.0185 1
10 0.0172 1
13 0.0667 1
14 0.0588 1
18 0.0714 1
19 0.0714 1
E (keV)
? Geant4 ? NIST
Z
14
Fluorescence Shell vacancy L1
Shell-end Kolmogorov-Smirnov D p-value
10 0.0192 1
11 0.0175 1
13 0.0250 1
14 0.0256 1
18 0.0294 1
19 0.0312 1
21 0.1429 0.997085
22 0.0588 1
? Geant4 ? NIST
15
Fluorescence Shell vacancy L2
Shell-end Kolmogorov-Smirnov D p-value
8 0.0147 1
11 0.0435 1
13 0.0139 1
16 0.0204 1
19 0.0526 1
21 0.0196 1
24 0.0714 1
? Geant4 ? NIST
16
Fluorescence Shell vacancy L3
Shell-end Kolmogorov-Smirnov D p-value
8 0.0145 1
10 0.0588 1
11 0.0556 1
13 0.0182 1
14 0.0179 1
16 0.0200 1
18 0.0667 1
19 0.0556 1
21 0.0588 1
22 0.0500 1
? Geant4 ? NIST
17
Auger electron emission

• Scarce experimental data in the NIST database
• Often multiple data for the same Auger
  transition ambiguous reference
• Analysis in progress comparison of Geant4
  simulation data against the NIST subset of
  experimental data
• Preliminary results good qualitative agreement
  as in the case of X-ray fluorescence
• Rigorous statistical analysis to be completed,
  will be included in publication

18
Geant4 electron Bremsstrahlung
2 electromagnetic physics packages
Standard
Low Energy
3 Bremsstrahlung processes
G4eLowEnergyBremsstrahlung
G4eBremsstrahlung
Tsai
Tsai
2BN
2BS
angular distribution
angular distributions
G4PenelopeBremsstrahlung
19
Validation of Geant4 EM physics
Ongoing large-scale project
NIST
Photon mass attenuation coefficient Range,
Stopping power (e, p, a)
K. Amako et al., IEEE Trans. Nucl. Sci. 52
(2005) 910
Atomic relaxation (fluorescence, Auger
effect) Proton Bragg peak Electron Bremsstrahlung
NSS 2006
Bremsstrahlung
Difficult to find reference data Thin/thick
target experiments Difficult to disentangle
effects (because of the continuous part)
1st validation cycle focus on low energy
20
Angular distributions
70 keV
Angular distribution of photons is strongly
model-dependent
21
The experimental set-up
e- beam(70 keV-10 MeV) incident on a slab of
material
Photon (energy, ?)
Electrons and d-rays are absorbed Bremsstrahlung
photons can be transmitted
electrons
Z axis
Yield, Energy and Polar Angle of the emitted
photons
Secondary production threshold 0.5 mm
Statistical Toolkit Goodness-of-Fit test in
progress
Quantitatitative comparison of experimental -
simulated distributions
22
Data sets
Preliminary results Work in progress! Simulation
production still running Statistical analysis
still preliminary, to be completed
N. Starfelt et al., Phys. Rev. 102 (1956)
1598 Thin target Be, Al, Au - 2.7, 4.5, 9.7
MeV Double differential cross sections
W.E. Dance et al., Journal of Appl. Phys. 39
(1968) 2881 Thick target Al, Fe 0.5, 1
MeV Double differential cross sections Integrated
g yield
R. Ambrose et al., NIM B 56/57 (1991)
327 Absolute and relative yield
23
Double differential s at 2.7 MeV on thin (2.63
mg/cm2) Be target
N. Starfelt et al., Phys. Rev. 102 (1956) 1598
Energy (MeV)
Energy (MeV)
24
Double differential s at 4.5 MeV on thin (2.63
mg/cm2) Be target
N. Starfelt et al., Phys. Rev. 102 (1956) 1598
Preliminary Kolmogorov-Smirnov p-value 0.13
Preliminary Kolmogorov-Smirnov p-value 0.17
Energy (MeV)
Energy (MeV)
25
Double differential s at 9.7 MeV on thin (2.63
mg/cm2) Be target
N. Starfelt et al., Phys. Rev. 102 (1956) 1598
Energy (MeV)
Energy (MeV)
26
Double differential s at 2.7 MeV on thin (0.878
mg/cm2) Al target
N. Starfelt et al., Phys. Rev. 102 (1956) 1598
Energy (MeV)
Energy (MeV)
27
Double differential s at 2.7 MeV on thin (0.878
mg/cm2) Al target
N. Starfelt et al., Phys. Rev. 102 (1956) 1598
Energy (MeV)
Energy (MeV)
28
Double differential s at 4.5 MeV on thin (0.878
mg/cm2) Al target
N. Starfelt et al., Phys. Rev. 102 (1956) 1598
Energy (MeV)
Energy (MeV)
29
Double differential s at 9.7 MeV on thin (0.878
mg/cm2) Al target
N. Starfelt et al., Phys. Rev. 102 (1956) 1598
Energy (MeV)
Energy (MeV)
30
Double differential s at 2.7 MeV on thin (0.209
mg/cm2) Au target
N. Starfelt et al., Phys. Rev. 102 (1956) 1598
Energy (MeV)
Energy (MeV)
31
Double differential s at 4.5 MeV on thin (0.209
mg/cm2) Au target
N. Starfelt et al., Phys. Rev. 102 (1956) 1598
Energy (MeV)
Energy (MeV)
32
Double differential s at 9.7 MeV on thin (0.209
mg/cm2) Au target
N. Starfelt et al., Phys. Rev. 102 (1956) 1598
Energy (MeV)
Energy (MeV)
33
Angular distribution
500 keV
W.E. Dance et al., Journal of Applied Physics 39
(1968) 2881
500 keV electrons on Al (0.548 g/cm2) and Fe
(0.257 g/cm2) Thick target experiment
Preliminary ?2 test p-value 0.10
Standard package
Red data Black simulation o ? Al ? ? Fe
Absolute comparison
34
Angular distribution
500 keV
W.E. Dance et al., Journal of Applied Physics 39
(1968) 2881
Preliminary ?2 test p-value 0.68
Preliminary ?2 test p-value 0.03
precise agreement!
35
Angular distribution
500 keV
W.E. Dance et al., Journal of Applied Physics 39
(1968) 2881
Preliminary ?2 test p-value 0.33
Preliminary ?2 test p-value not meaningful
36
Angular distribution
1 MeV
Preliminary Fe ?2 test p-value not meaningful
Same test for 1 MeV primary electrons (threshold
50 keV)
W.E. Dance et al., Journal of Applied Physics 39
(1968) 2881
Targets Al (0.548 g/cm2) and Fe (0.613 g/cm2)
Red data Black simulation o ? Al ? ? Fe
Absolute comparison
37
Angular distribution
1 MeV
W.E. Dance et al., Journal of Applied Physics 39
(1968) 2881
Preliminary Fe ?2 test p-value 0.68
Preliminary Fe ?2 test p-value 0.06
precise agreement!
Good agreement for Al - Reasonable also for Fe
(2BN)
38
Angular distribution
1 MeV
W.E. Dance et al., Journal of Applied Physics 39
(1968) 2881
Preliminary Fe ?2 test p-value 0.36
Preliminary Fe ?2 test p-value not meaningful
2BS good for Al and Fe (except in the backward
direction)
39
Integral g yield
Total g yield on Al integrated on ? (0 ? p) and
on energy (Eth ? Emax)
Also available for other flavours of Geant4
Bremsstrahlung models
W.E. Dance et al., Journal of Applied Physics 39
(1968) 2881
o ? dat a ? ? simul.
Preliminary
Preliminary
Further investigation in progress
40
Energy distribution at 70 keV
Penelope Low Energy - TSAI
R. Ambrose et al., Nucl. Instr. Meth. B 56/57
(1991) 327
Intensity/Z (eV/sr keV)
photon direction
70 keV e-
45 deg
Photon energy (keV)
70 keV electrons impinging on Al (25.4 mg/cm2)
41
Relative comparison at 70 keV
Low Energy - TSAI
Penelope
Intensity/Z (eV/sr keV)
Intensity/Z (eV/sr keV)
Photon energy (keV)
Photon energy (keV)
Relative comparison (45 direction) Shapes of
the spectra are in good agreement
42
Proton Bragg peak

• Compare various Geant4 electromagnetic models
• Assess lowest energy range of hadronic
  interactions
• elastic scattering
• pre-equilibrium nuclear deexcitation
• to build further validation tests on solid ground
• Results directly relevant to various experimental
  use cases

43
Relevant Geant4 physics models
Hadronic
Electromagnetic

• Parameterized (à la GHEISHA)
• Nuclear Deexcitation
• Default evaporation
• GEM evaporation
• Fermi break-up
• Pre-equilibrium
• Precompound model
• Bertini model
• Elastic scattering
• Parameterized models
• Bertini
• Intra-nuclear cascade
• Bertini cascade
• Binary cascade

• Standard
• Low Energy ICRU 49
• Low Energy Ziegler 1977
• Low Energy Ziegler 1985
• Low Energy Ziegler 2000
• New very low energy models

Subset of results shown here Full set of results
in publication coming shortly
44
Experimental data

• CATANA hadrontherapy facility in Catania, Italy
• high precision experimental data satisfying
  rigorous medical physics protocols
• Geant4 Collaboration members

Validation measurements Markus Ionization chamber
45
Geant4 simulation
Accurate reproduction of the experimental
set-up This is the most difficult part to achieve
a quantitative Geant4 physics validation Geometry
and beam characteristics must be known in detail
and with high precision
Ad hoc beam line set-up for Geant4 validation to
enhance peculiar effects of physics processes
Eproton 63.5 MeV sE 300 keV
46
Electromagnetic processes

• Electromagnetic options
• Standard EM
• Low Energy EM ICRU 49
• Low Energy EM Ziegler 1977
• Low Energy EM Ziegler 1985
• Low Energy EM Ziegler 2000

47
Electromagnetic processes
Standard EM

• Standard EM p, ions, g, e- e

1 M events
p-value p-value p-value
CvM KS AD
Left branch 0.418
Right branch 0.736
Whole curve 0.438
Geant4 Experimental data
CvM Cramer-von Mises test KS
Kolmogorov-Smirnov test AD Anderson-Darling
test
mm
48
Electromagnetic processes
LowE EM ICRU49

• Low Energy EM ICRU49 p, ions
• Low Energy EM Livermore g, e-
• Standard EM e

1 M events
p-value p-value p-value
CvM KS AD
Left branch 0.530
Right branch 0.985
Whole curve 0.676
Geant4 Experimental data
CvM Cramer-von Mises test KS
Kolmogorov-Smirnov test AD Anderson-Darling
test
mm
49
Electromagnetic processes
LowE EM Ziegler 1977

• Low Energy EM Ziegler 1977 p, ions
• Low Energy EM Livermore g, e-
• Standard EM e

1 M events
Geant4 Experimental data
CvM Cramer-von Mises test KS
Kolmogorov-Smirnov test AD Anderson-Darling
test
mm
50
Electromagnetic processes
LowE EM Ziegler 1985

• Low Energy EM Ziegler 1985 p, ions
• Low Energy EM Livermore g, e-
• Standard EM e

1 M events
Subject to further investigation
Geant4 Experimental data
mm
51
Electromagnetic processes
LowE EM Ziegler 2000

• Low Energy EM Ziegler 2000 p, ions
• Low Energy EM Livermore g, e-
• Standard EM e

1 M events
Subject to further investigation
Geant4 Experimental data
mm
52
Electromagnetic processes Summary
p-value p-value p-value
LowE ICRU49 LowE Ziegler 1977 Standard
Left branch (CvM) 0.530 0.418
Right branch (KS) 0.985 0.985 0.736
Whole curve (AD) 0.676 0.438
CvM Cramer-von Mises test KS
Kolmogorov-Smirnov test AD Anderson-Darling
test
53
Electromagnetic processes Elastic scattering

• Elastic scattering options
• HadronElastic process with LElastic model
• HadronElastic process with BertiniElastic model
• UHadronElastic process with HadronElastic model

54
EM Elastic scattering
LowE EM ICRU49
LElastic

• Low Energy EM ICRU49 p, ions
• Low Energy EM Livermore g, e-
• Standard EM e
• HadronElastic with LElastic

1 M events
p-value p-value p-value
CvM KS AD
Left branch 0.522
Right branch 0.985
Whole curve 0.697
Geant4 Experimental data
CvM Cramer-von Mises test KS
Kolmogorov-Smirnov test AD Anderson-Darling
test
mm
55
EM Elastic scattering
LowE EM ICRU49
HadronElastic

• Low Energy EM ICRU49 p, ions
• Low Energy EM Livermore g, e-
• Standard EM e
• UHadronElastic with HadronElastic

0.5 M events
p-value p-value p-value
CvM KS AD
Left branch 0.490
Right branch 0.735
Whole curve 0.669
Geant4 Experimental data
CvM Cramer-von Mises test KS
Kolmogorov-Smirnov test AD Anderson-Darling
test
mm
56
Electromagnetic processes Elastic
scatteringHadronic inelastic scattering

• Hadronic Inelastic options
• Precompound with Default Evaporation
• Precompound with GEM Evaporation
• Precompound with Default Evaporation Fermi
  Break-up
• Bertini

57
EM hadronic physics
LowE EM ICRU49
LElastic
Precompound default

• Low Energy EM ICRU49 p, ions
• Low Energy EM Livermore g, e-
• Standard EM e
• HadronElastic with LElastic
• Precompound with Default Evaporation

1 M events
p-value p-value p-value
CvM KS AD
Left branch 0.836
Right branch 0.985
Whole curve 0.946
Geant4 Experimental data
CvM Cramer-von Mises test KS
Kolmogorov-Smirnov test AD Anderson-Darling
test
mm
58
EM hadronic physics
Standard EM
LElastic
Precompound default

• Standard EM p, ions, g, e- e
• HadronElastic with LElastic
• Precompound with Default Evaporation

1 M events
p-value p-value p-value
CvM KS AD
Left branch 0.648
Right branch 0.760
Whole curve 0.666
Geant4 Experimental data
CvM Cramer-von Mises test KS
Kolmogorov-Smirnov test AD Anderson-Darling
test
mm
59
EM hadronic physics
LowE EM ICRU49
HadronElastic
Precompound default

• Low Energy EM ICRU49 p, ions
• Low Energy EM Livermore g, e-
• Standard EM e
• UHadronElastic with HadronElastic Precompound
  with Default Evaporation

0.5 M events
p-value p-value p-value
CvM KS AD
Left branch 0.973
Right branch 0.985
Whole curve 0.982
Geant4 Experimental data
CvM Cramer-von Mises test KS
Kolmogorov-Smirnov test AD Anderson-Darling
test
mm
60
EM hadronic physics
LowE EM ICRU49
LElastic
Precompound with GEM Evaporation

• Low Energy EM ICRU49 p, ions
• Low Energy EM Livermore g, e-
• Standard EM e
• HadronElastic with LElastic
• Precompound with GEM Evaporation

0.5 M events
p-value p-value p-value
CvM KS AD
Left branch 0.667
Right branch 0.985
Whole curve 0.858
Geant4 Experimental data
CvM Cramer-von Mises test KS
Kolmogorov-Smirnov test AD Anderson-Darling
test
mm
61
EM hadronic physics
LowE EM ICRU49
LElastic
Precompound with Fermi Break-up

• Low Energy EM ICRU49 p, ions
• Low Energy EM Livermore g, e-
• Standard EM e
• HadronElastic with LElastic
• Precompound with Fermi Break-up

0.5 M events
p-value p-value p-value
CvM KS AD
Left branch 0.814
Right branch 0.985
Whole curve 0.945
Geant4 Experimental data
CvM Cramer-von Mises test KS
Kolmogorov-Smirnov test AD Anderson-Darling
test
mm
62
EM hadronic physics
LowE EM ICRU49
LElastic
Bertini Inelastic

• Low Energy EM ICRU49 p, ions
• Low Energy EM Livermore g, e-
• Standard EM e
• HadronElastic with LElastic
• Bertini Inelastic

1 M events
p-value p-value p-value
CvM KS AD
Left branch 0.790
Right branch 0.985
Whole curve 0.936
Geant4 Experimental data
CvM Cramer-von Mises test KS
Kolmogorov-Smirnov test AD Anderson-Darling
test
mm
63
EM hadronic physics
LowE EM ICRU49
BertiniElastic
Bertini Inelastic

• Low Energy EM ICRU49 p, ions
• Low Energy EM Livermore g, e-
• Standard EM e
• HadronElastic with BertiniElastic
• Bertini Inelastic

0.5 M events
p-value p-value p-value
CvM KS AD
Left branch 0.977
Right branch 0.985
Whole curve 0.994
CvM Cramer-von Mises test KS
Kolmogorov-Smirnov test AD Anderson-Darling
test
Geant4 Experimental data
mm
64
Electromagnetic Hadronic Summary
p-value p-value p-value p-value p-value p-value p-value
Standard LElastic Precompound LowE ICRU49 LElastic Precompound GEM LowE ICRU49 LElastic Bertini Inelastic LowE ICRU49 LElastic Precompound Fermi Break-up LowE ICRU49 LElastic Precompound LowE ICRU49 HadronElastic Precompound LowE ICRU49 Bertini Elastic Bertini Inelastic
Left branch (CvM) 0.648 0.667 0.790 0.814 0.836 0.973 0.977
Right branch (KS) 0.760 0.985 0.985 0.985 0.985 0.985 0.985
Whole curve (AD) 0.666 0.858 0.936 0.945 0.946 0.982 0.994

• Precise electromagnetic physics
• Good elastic scattering model
• Good pre-equilibrium model

Key ingredients
65
and behind everything
Unified Process
A rigorous software process Incremental and
iterative lifecycle RUP? as process framework,
tailored to the specific project Mapped onto ISO
15504
66
Conclusion

• Geant4 physics validation carried on by a small,
  young team with rigorous methods
• Underlying vision
• Systematic approach
• Rigorous quantitative analysis
• Current projects
• Atomic relaxation final results
• Bremsstrahlung preliminary results
• Proton Bragg peak mature stage, refinements by
  end 2006
• Publications coming soon