Geant4 Physics Validation

1
Geant4 Physics Validation
Pablo Cirrone Giacomo Cuttone Francesco Di
Rosa Susanna Guatelli Alfonso Mantero Barbara
Mascialino Luciano Pandola Andreas Pfeiffer MG
Pia Giorgio Russo Paolo Viarengo Valentina
Zampichelli

• M.G. Pia
• On behalf of the LowE EM and Advanced Examples
• Working Groups
• http//www.ge.infn.it/geant4/lowE

Geant4 Space User Workshop Pasadena, 6-10
Novemver 2006
2
Geant4 Toolkit
Wide set of physics processes and
models Versatility of configuration according to
use cases
How to choose the most appropriate model for my
simulation?

• Provide objective criteria to evaluate Geant4
  physics models
• Document their precision against experimental
  data
• Test all Geant4 physics models systematically
• Quantitative tests with rigorous statistical
  methods

3
Verification and Validationof Geant4 physics

• Verification
• compliance of the software results with the
  underlying model
• Unit tests (at the level of individual Geant4
  classes)
• Validation
• comparison against experimental data
• Quantitative estimate of the agreement between
  Geant4 simulation and reference data through
  statistical methods (Goodness-of-Fit)

A systematic, quantitative validation of Geant4
physics models against reference experimental
data is essential to establish the reliability
of Geant4-based applications
4
Statistical Toolkit

• 2nd development cycle
• Released April 2006
• Whats new
• New tests
• ROOT User Layer
• New installation tools
• Performance analysis

The most complete software tool for 2-sample GoF
tests
5
Geant4 Validation

• Atomic relaxation
• Bremsstrahlung
• Proton Bragg peak
• other validation activities in Advanced
  Examples
• Statistical Toolkit
• Common features of the validation activities
• Collaborative, open, transparent work environment
• Rigorous, quantitative analysis
• Publication-quality methods and results

K. Amako et al., Comparison of Geant4
electromagnetic physics models against the NIST
reference dataIEEE Trans. Nucl. Sci., Vol. 52,
Issue 4, Aug. 2005, pp. 910-918
6
Geant4 Atomic Relaxation

• Geant4 Low Energy Electromagnetic package takes
  into account the detailed atomic structure of
  matter and the related physics processes
• It includes a package for Atomic Relaxation
• Simulation of atomic de-excitation resulting from
  the creation of a vacancy in an atom by a primary
  process

• Geant4 Atomic Relaxation models
• Fluorescence
• Auger electron emission

• It is used by Geant4 packages
• Low Energy Electromagnetic
• Photoelectric effect
• Low Energy electron ionisation
• Low Energy proton ionisation (PIXE)
• Penelope Compton scattering
• Hadronic Physics
• Nuclear de-excitation
• Radioactive decay

These physics models are relevant to many diverse
experimental applications
7
Geant4 fluorescence
Original motivation from astrophysics requirements
Cosmic rays, jovian electrons
X-Ray Surveys of Asteroids and Moons
Solar X-rays, e, p
Geant3.21
ITS3.0, EGS4
Courtesy SOHO EIT
Geant4
Induced X-ray line emission indicator of target
composition (100 mm surface layer)
250 keV
C, N, O line emissions included
Wide field of applications beyond astrophysics
Courtesy ESA Space Environment Effects Analysis
Section
8
Atomic Relaxation in Geant4

• Two steps
• Identification of the atomic shell where a
  vacancy is created by a primary process
  (photoelectric, Compton, ionisation)
• The creation of the vacancy is based on the
  calculation of the primary process cross sections
  relative to the shells of the target atom
• Cross section modeling and calculation specific
  to each process
• Generation of the de-excitation chain and its
  products
• Common package, used by all vacancy-creating
  processes
• Geant4 Atomic Relaxation
• Generation of fluorescence photons and Auger
  electrons
• Determination of the energy of the secondary
  particles produced

9
Modelling foundationin Geant4 Low Energy
Electromagnetic Package

• Calculation of shell cross sections
• Based on the EPDL97 Livermore Library for
  photoelectric effect
• Based on the EEDL Livermore Library for electron
  ionisation
• Based on Penelope model for Compton scattering
• Detailed atom description and calculation of the
  energy of generated photons/electrons
• Based on the EADL Livermore Library

10
Validation of Geant4 Atomic Relaxation

• Previous partial validation studies
  (collaboration with ESA)
• Pure materials limited number of materials
  examined
• Complex materials complex experimental set-up,
  large uncertainties on the target material
  composition
• Systematic validation project NIST database as
  reference

Authoritative, systematic collection of
experimental data
11
Method and tools

• Geant4 test code to generate fluorescence and
  Auger transitions from all elements
• Geant4 Atomic Relaxation handles 6 Z 100
• Selection of experimental data subsets from NIST
  database
• The NIST database also contains data from
  theoretical calculations
• Comparison of simulated/NIST data with
  Goodness-of-Fit test
• Data grouped for the comparison as a function of
  Z according to the initial vacancy and transition
  type
• Statistical Toolkit (http//www.ge.infn.it/statist
  icaltoolkit)
• Kolmogorov-Smirnov test
• The result of the agreement is expressed through
  the p-value of the test

12
Fluorescence - Shell-start 1
Shell-end Kolmogorov-Smirnov D p-value
5 0.0188 1
6 0.0185 1
10 0.0172 1
13 0.0667 1
14 0.0588 1
18 0.0714 1
19 0.0714 1
E (keV)
? Geant4 ? NIST
Z
13
Fluorescence - Shell-start 3
Shell-end Kolmogorov-Smirnov D p-value
10 0.0192 1
11 0.0175 1
13 0.0250 1
14 0.0256 1
18 0.0294 1
19 0.0312 1
21 0.1429 0.997085
22 0.0588 1
? Geant4 ? NIST
14
Fluorescence - Shell-start 5
Shell-end Kolmogorov-Smirnov D p-value
8 0.0147 1
11 0.0435 1
13 0.0139 1
16 0.0204 1
19 0.0526 1
21 0.0196 1
24 0.0714 1
? Geant4 ? NIST
15
Fluorescence - Shell-start 6
Shell-end Kolmogorov-Smirnov D p-value
8 0.0145 1
10 0.0588 1
11 0.0556 1
13 0.0182 1
14 0.0179 1
16 0.0200 1
18 0.0667 1
19 0.0556 1
21 0.0588 1
22 0.0500 1
? Geant4 ? NIST
16
Auger electron emission

• Scarce experimental data in the NIST database
• Often multiple data for the same Auger
  transition ambiguous reference
• Analysis in progress comparison of Geant4
  simulation data against the NIST subset of
  experimental data
• Preliminary results good qualitative agreement
  as in the case of X-ray fluorescence
• Rigorous statistical analysis to be completed,
  will be included in publication

17
Geant4 electron Bremsstrahlung
2 electromagnetic physics packages
Standard
Low Energy
3 Bremsstrahlung processes
G4eLowEnergyBremsstrahlung
G4eBremsstrahlung
Tsai
Tsai
2BN
2BS
angular distribution
angular distributions
G4PenelopeBremsstrahlung
18
Validation of Geant4 EM physics
Ongoing large-scale project
NIST
Photon mass attenuation coefficient Range,
Stopping power (e, p, a)
K. Amako et al., IEEE Trans. Nucl. Sci. 52
(2005) 910
Atomic relaxation (fluorescence, Auger
effect) Proton Bragg peak Electron Bremsstrahlung
NSS 2006
Bremsstrahlung
Difficult to find reference data Thin/thick
target experiments Difficult to disentangle
effects (because of the continuous part)
1st validation cycle focus on low energy
19
The experimental set-up
e- beam(70 keV-10 MeV) incident on a slab of
material
Photon (energy, ?)
Electrons and d-rays are absorbed Bremsstrahlung
photons can be transmitted
electrons
Z axis
Yield, Energy and Polar Angle of the emitted
photons
Secondary production threshold 0.5 mm
Statistical Toolkit Goodness-of-Fit test in
progress
Quantitatitative comparison of experimental -
simulated distributions
20
Data sets
Preliminary results Work in progress!
N. Starfelt et al., Phys. Rev. 102 (1956)
1598 Thin target Be, Al, Au - 2.7, 4.5, 9.7
MeV Double differential cross sections
W.E. Dance et al., Journal of Appl. Phys. 39
(1968) 2881 Thick target Al, Fe 0.5, 1
MeV Double differential cross sections Integrated
g yield
R. Ambrose et al., NIM B 56/57 (1991)
327 Absolute and relative yield
21
Double differential s at 2.7 MeV on thin (2.63
mg/cm2) Be target
N. Starfelt et al., Phys. Rev. 102 (1956) 1598
Energy (MeV)
Energy (MeV)
22
Double differential s at 4.5 MeV on thin (2.63
mg/cm2) Be target
N. Starfelt et al., Phys. Rev. 102 (1956) 1598
Energy (MeV)
Energy (MeV)
23
Double differential s at 9.7 MeV on thin (2.63
mg/cm2) Be target
N. Starfelt et al., Phys. Rev. 102 (1956) 1598
Energy (MeV)
Energy (MeV)
24
Double differential s at 2.7 MeV on thin (0.878
mg/cm2) Al target
N. Starfelt et al., Phys. Rev. 102 (1956) 1598
Energy (MeV)
Energy (MeV)
25
Double differential s at 2.7 MeV on thin (0.878
mg/cm2) Al target
N. Starfelt et al., Phys. Rev. 102 (1956) 1598
Energy (MeV)
Energy (MeV)
26
Double differential s at 4.5 MeV on thin (0.878
mg/cm2) Al target
N. Starfelt et al., Phys. Rev. 102 (1956) 1598
Energy (MeV)
Energy (MeV)
27
Double differential s at 9.7 MeV on thin (0.878
mg/cm2) Al target
N. Starfelt et al., Phys. Rev. 102 (1956) 1598
Energy (MeV)
Energy (MeV)
28
Double differential s at 2.7 MeV on thin (0.209
mg/cm2) Au target
N. Starfelt et al., Phys. Rev. 102 (1956) 1598
Energy (MeV)
Energy (MeV)
29
Double differential s at 4.5 MeV on thin (0.209
mg/cm2) Au target
N. Starfelt et al., Phys. Rev. 102 (1956) 1598
Energy (MeV)
Energy (MeV)
30
Double differential s at 9.7 MeV on thin (0.209
mg/cm2) Au target
N. Starfelt et al., Phys. Rev. 102 (1956) 1598
Energy (MeV)
Energy (MeV)
31
Angular distribution
500 keV
W.E. Dance et al., Journal of Applied Physics 39
(1968) 2881
Ethr 46 keV
500 keV electrons on Al (0.548 g/cm2) and Fe
(0.257 g/cm2) Thick target experiment
Standard package
Red data Black simulation o ? Al ? ? Fe
Absolute comparison
32
Angular distribution
500 keV
W.E. Dance et al., Journal of Applied Physics 39
(1968) 2881
precise agreement!
33
Angular distribution
500 keV
W.E. Dance et al., Journal of Applied Physics 39
(1968) 2881
34
Angular distribution
1 MeV
Same test for 1 MeV primary electrons (threshold
50 keV)
Standard package
W.E. Dance et al., Journal of Applied Physics 39
(1968) 2881
Targets Al (0.548 g/cm2) and Fe (0.613 g/cm2)
Red data Black simulation o ? Al ? ? Fe
Absolute comparison
35
Angular distribution
1 MeV
W.E. Dance et al., Journal of Applied Physics 39
(1968) 2881
precise agreement!
Good agreement for Al - Reasonable also for Fe
(2BN)
36
Angular distribution
1 MeV
W.E. Dance et al., Journal of Applied Physics 39
(1968) 2881
2BS good for Al and Fe (except in the backward
direction)
37
Integral g yield
Total g yield on Al integrated on ? (0 ? p) and
on energy (Eth ? Emax)
Also available for other flavours of Geant4
Bremsstrahlung models
W.E. Dance et al., Journal of Applied Physics 39
(1968) 2881
o ? dat a ? ? simul.
Preliminary
Further investigation in progress
38
Angular distributions
70 keV
Low Energy Package
Penelope Standard Low Energy (TSAI)
Penelope TSAI 2BS 2BN
Angle (deg)
Angle (deg)
Angular distribution of photons is strongly
model-dependent
39
Energy distribution at 70 keV
Penelope Low Energy - TSAI
R. Ambrose et al., Nucl. Instr. Meth. B 56/57
(1991) 327
Intensity/Z (eV/sr keV)
photon direction
70 keV e-
45 deg
Photon energy (keV)
70 keV electrons impinging on Al (25.4 mg/cm2)
40
Relative comparison at 70 keV
Low Energy - TSAI
Penelope
Intensity/Z (eV/sr keV)
Intensity/Z (eV/sr keV)
Photon energy (keV)
Photon energy (keV)
Relative comparison (45 direction) Shapes of
the spectra are in good agreement
41
K. Amako et al., Comparison of Geant4
electromagnetic physics models against the NIST
reference dataIEEE Trans. Nucl. Sci., Vol. 52,
Issue 4, Aug. 2005, pp. 910-918

• Adopt the same method also for hadronic physics
  validation
• address all modeling options
• start from the bottom (low energy)
• progress towards higher energy based on previous
  sound assessments
• statistical analysis of compatibility with
  experimental data
• Guidance to users based on objective ground
• not only educated-guess PhysicsLists

Statistical Toolkit Goodness-of-Fit test
Quantitatitative comparison of experimental -
simulated distributions
42
Proton Bragg peak

• Compare various Geant4 electromagnetic models
• Assess lowest energy range of hadronic
  interactions
• elastic scattering
• pre-equilibrium nuclear deexcitation
• to build further validation tests on solid ground
• Results directly relevant to various experimental
  use cases

43
Relevant Geant4 physics models
Hadronic
Electromagnetic

• Parameterized (à la GHEISHA)
• Nuclear Deexcitation
• Default evaporation
• GEM evaporation
• Fermi break-up
• Pre-equilibrium
• Precompound model
• Bertini model
• Elastic scattering
• Parameterized models
• Bertini
• Intra-nuclear cascade
• Bertini cascade
• Binary cascade

• Standard
• Low Energy ICRU 49
• Low Energy Ziegler 1977
• Low Energy Ziegler 1985
• Low Energy Ziegler 2000
• New very low energy models

Subset of results shown here Full set of results
in publication coming shortly
44
Experimental data

• CATANA hadrontherapy facility in Catania, Italy
• high precision experimental data satisfying
  rigorous medical physics protocols
• Geant4 Collaboration members

Validation measurements Markus Ionization chamber
45
Geant4 simulation
Accurate reproduction of the experimental
set-up This is the most difficult part to achieve
a quantitative Geant4 physics validation Geometry
and beam characteristics must be known in detail
and with high precision
Ad hoc beam line set-up for Geant4 validation to
enhance peculiar effects of physics processes
Eproton 63.5 MeV sE 300 keV
46
Electromagnetic processes

• Electromagnetic options
• Standard EM
• Low Energy EM ICRU 49
• Low Energy EM Ziegler 1977
• Low Energy EM Ziegler 1985
• Low Energy EM Ziegler 2000

47
Electromagnetic processes
Standard EM

• Standard EM p, ions, g, e- e

1 M events
p-value p-value p-value
CvM KS AD
Left branch 0.418
Right branch 0.736
Whole curve 0.438
Geant4 Experimental data
CvM Cramer-von Mises test KS
Kolmogorov-Smirnov test AD Anderson-Darling
test
mm
48
Electromagnetic processes
LowE EM ICRU49

• Low Energy EM ICRU49 p, ions
• Low Energy EM Livermore g, e-
• Standard EM e

1 M events
p-value p-value p-value
CvM KS AD
Left branch 0.530
Right branch 0.985
Whole curve 0.676
Geant4 Experimental data
CvM Cramer-von Mises test KS
Kolmogorov-Smirnov test AD Anderson-Darling
test
mm
49
Electromagnetic processes
LowE EM Ziegler 1977

• Low Energy EM Ziegler 1977 p, ions
• Low Energy EM Livermore g, e-
• Standard EM e

1 M events
Geant4 Experimental data
CvM Cramer-von Mises test KS
Kolmogorov-Smirnov test AD Anderson-Darling
test
mm
50
Electromagnetic processes
LowE EM Ziegler 1985

• Low Energy EM Ziegler 1985 p, ions
• Low Energy EM Livermore g, e-
• Standard EM e

1 M events
Subject to further investigation
Geant4 Experimental data
mm
51
Electromagnetic processes
LowE EM Ziegler 2000

• Low Energy EM Ziegler 2000 p, ions
• Low Energy EM Livermore g, e-
• Standard EM e

1 M events
Subject to further investigation
Geant4 Experimental data
mm
52
Electromagnetic processes Summary
p-value p-value p-value
LowE ICRU49 LowE Ziegler 1977 Standard
Left branch (CvM) 0.530 0.418
Right branch (KS) 0.985 0.985 0.736
Whole curve (AD) 0.676 0.438
CvM Cramer-von Mises test KS
Kolmogorov-Smirnov test AD Anderson-Darling
test
53
Electromagnetic processes Elastic scattering

• Elastic scattering options
• HadronElastic process with LElastic model
• HadronElastic process with BertiniElastic model
• UHadronElastic process with HadronElastic model

54
EM Elastic scattering
LowE EM ICRU49
LElastic

• Low Energy EM ICRU49 p, ions
• Low Energy EM Livermore g, e-
• Standard EM e
• HadronElastic with LElastic

1 M events
p-value p-value p-value
CvM KS AD
Left branch 0.522
Right branch 0.985
Whole curve 0.697
Geant4 Experimental data
CvM Cramer-von Mises test KS
Kolmogorov-Smirnov test AD Anderson-Darling
test
mm
55
EM Elastic scattering
LowE EM ICRU49
HadronElastic

• Low Energy EM ICRU49 p, ions
• Low Energy EM Livermore g, e-
• Standard EM e
• UHadronElastic with HadronElastic

0.5 M events
p-value p-value p-value
CvM KS AD
Left branch 0.490
Right branch 0.735
Whole curve 0.669
Geant4 Experimental data
CvM Cramer-von Mises test KS
Kolmogorov-Smirnov test AD Anderson-Darling
test
mm
56
Electromagnetic processes Elastic
scatteringHadronic inelastic scattering

• Hadronic Inelastic options
• Precompound with Default Evaporation
• Precompound with GEM Evaporation
• Precompound with Default Evaporation Fermi
  Break-up
• Bertini

57
EM hadronic physics
LowE EM ICRU49
LElastic
Precompound default

• Low Energy EM ICRU49 p, ions
• Low Energy EM Livermore g, e-
• Standard EM e
• HadronElastic with LElastic
• Precompound with Default Evaporation

1 M events
p-value p-value p-value
CvM KS AD
Left branch 0.836
Right branch 0.985
Whole curve 0.946
Geant4 Experimental data
CvM Cramer-von Mises test KS
Kolmogorov-Smirnov test AD Anderson-Darling
test
mm
58
EM hadronic physics
Standard EM
LElastic
Precompound default

• Standard EM p, ions, g, e- e
• HadronElastic with LElastic
• Precompound with Default Evaporation

1 M events
p-value p-value p-value
CvM KS AD
Left branch 0.648
Right branch 0.760
Whole curve 0.666
Geant4 Experimental data
CvM Cramer-von Mises test KS
Kolmogorov-Smirnov test AD Anderson-Darling
test
mm
59
EM hadronic physics
LowE EM ICRU49
HadronElastic
Precompound default

• Low Energy EM ICRU49 p, ions
• Low Energy EM Livermore g, e-
• Standard EM e
• UHadronElastic with HadronElastic Precompound
  with Default Evaporation

0.5 M events
p-value p-value p-value
CvM KS AD
Left branch 0.973
Right branch 0.985
Whole curve 0.982
Geant4 Experimental data
CvM Cramer-von Mises test KS
Kolmogorov-Smirnov test AD Anderson-Darling
test
mm
60
EM hadronic physics
LowE EM ICRU49
LElastic
Precompound with GEM Evaporation

• Low Energy EM ICRU49 p, ions
• Low Energy EM Livermore g, e-
• Standard EM e
• HadronElastic with LElastic
• Precompound with GEM Evaporation

0.5 M events
p-value p-value p-value
CvM KS AD
Left branch 0.667
Right branch 0.985
Whole curve 0.858
Geant4 Experimental data
CvM Cramer-von Mises test KS
Kolmogorov-Smirnov test AD Anderson-Darling
test
mm
61
EM hadronic physics
LowE EM ICRU49
LElastic
Precompound with Fermi Break-up

• Low Energy EM ICRU49 p, ions
• Low Energy EM Livermore g, e-
• Standard EM e
• HadronElastic with LElastic
• Precompound with Fermi Break-up

0.5 M events
p-value p-value p-value
CvM KS AD
Left branch 0.814
Right branch 0.985
Whole curve 0.945
Geant4 Experimental data
CvM Cramer-von Mises test KS
Kolmogorov-Smirnov test AD Anderson-Darling
test
mm
62
EM hadronic physics
LowE EM ICRU49
LElastic
Bertini Inelastic

• Low Energy EM ICRU49 p, ions
• Low Energy EM Livermore g, e-
• Standard EM e
• HadronElastic with LElastic
• Bertini Inelastic

1 M events
p-value p-value p-value
CvM KS AD
Left branch 0.790
Right branch 0.985
Whole curve 0.936
Geant4 Experimental data
CvM Cramer-von Mises test KS
Kolmogorov-Smirnov test AD Anderson-Darling
test
mm
63
EM hadronic physics
LowE EM ICRU49
BertiniElastic
Bertini Inelastic

• Low Energy EM ICRU49 p, ions
• Low Energy EM Livermore g, e-
• Standard EM e
• HadronElastic with BertiniElastic
• Bertini Inelastic

0.5 M events
p-value p-value p-value
CvM KS AD
Left branch 0.977
Right branch 0.985
Whole curve 0.994
CvM Cramer-von Mises test KS
Kolmogorov-Smirnov test AD Anderson-Darling
test
Geant4 Experimental data
mm
64
Electromagnetic Hadronic Summary
p-value p-value p-value p-value p-value p-value p-value
Standard LElastic Precompound LowE ICRU49 LElastic Precompound GEM LowE ICRU49 LElastic Bertini Inelastic LowE ICRU49 LElastic Precompound Fermi Break-up LowE ICRU49 LElastic Precompound LowE ICRU49 HadronElastic Precompound LowE ICRU49 Bertini Elastic Bertini Inelastic
Left branch (CvM) 0.648 0.667 0.790 0.814 0.836 0.973 0.977
Right branch (KS) 0.760 0.985 0.985 0.985 0.985 0.985 0.985
Whole curve (AD) 0.666 0.858 0.936 0.945 0.946 0.982 0.994

• Precise electromagnetic physics
• Good elastic scattering model
• Good pre-equilibrium model

Key ingredients
65
and behind everything
Unified Process
A rigorous software process Incremental and
iterative lifecycle RUP? as process framework,
tailored to the specific project Mapped onto ISO
15504
66
Conclusion

• Geant4 physics validation carried on by a small,
  young team with rigorous methods
• Underlying vision
• Systematic approach
• Rigorous statistical analysis for quantitative
  characterization of Geant4 physics
• Current projects
• Atomic relaxation final results
• Bremsstrahlung preliminary results
• Proton Bragg peak mature stage, refinements by
  end 2006
• Publications coming soon