Preprocessing With focus on NIR

1
PreprocessingWith focus on NIR

• Åsmund Rinnan , Frans van den Berg, Rasmus Bro,
  Søren Balling Engelsen, Lars Nørgaard , Jonas
  Thygesen

2
Introduction
Introduction Techniques Summary

• How often is it used?
• When is it used?
• What methods are used?
• Why is it used?
• NIR is a measurement technique screaming for
  preprocessing

3
IntroductionNIR
Introduction Techniques Summary
Fructose
Glucose
0 50 75
100 50 25
4
IntroductionTechniques
Introduction Techniques Summary

• Correction of light scatter
• MSC/ ISC
• PMSC
• EMSC/ EISC
• SNV
• Derivation
• NorrisWilliams
• SavitskyGolay

• Baseline-correction
• Normalization
• Detrend
• Use of reference values
• O-PLS
• OSC
• OS
• SIS

5
IntroductionWhy
Introduction Techniques Summary
Baseline with a slope/ curve
Nonlinearity
A fat baseline
6
MSC
Introduction Techniques Summary
ISC

• Offset
• Normalization

Raw spectrum
Reference

• P Geladi, D MacDougal, H Martens (1985)
  Linearization and scatter correction for
  near-infrared reflectance spectra of meat,
  Applied Spectroscopy, 39, 491-500

7
MSC/ ISC
Introduction Techniques Summary

• Offset
• Normalization

8
EISC
Introduction Techniques Summary

• Offset
• Baseline
• Normalization

• H. Martens and E. Stark (1991) Extended
  multiplicative signal correction and spectral
  interference subtraction new preprocessing
  methods for near infrared spectroscopy, Journal
  of Pharmaceutical and Biomedicinal Analysis, 9,
  625-635
• H Martens, JP Nielsen, SB Engelsen (2003) Light
  scattering and light absorbance separated by
  extended multiplicative signal correction. The
  application to near-infrared transmission
  analysis of powder mixtures, Analytical
  chemistry, 75, 394-404
• DK Pedersen, H Martens, JP Nielsen, SB Engelsen
  (2002) Near-Infrared Absorption and Scattering
  Separated by Extended Inverse Signal Correction
  (EISC) Analysis of Near-Infrared Transmittance
  Spectra of Single Wheat, Applied Spectroscopy,
  56, 1206-1214
• Thennadil, SN, Martin, EB (2005) Empirical
  preprocessing methods and their impact on NIR
  calibrations - a simulation study, Journal of
  Chemometrics, 19, 77-89

9
EMSC(w/ 2. degree wavelength)
Introduction Techniques Summary

• Offset
• Baseline
• Normalization

10
PxSC
Introduction Techniques Summary

• Offset
• Baseline
• Normalization

X EM EI M I

• Isaksson, T. Kowalski, B. R. (1993) Piece-wise
  multiplicative scatter correction applied to
  Near-Infrared diffuse transmittance data from
  meat products, Applied Spectroscopy, 47, 702-09

11
PMSC(window size 35)
Introduction Techniques Summary

• Offset
• Normalization

12
SNV
Introduction Techniques Summary

• Offset
• Normalization

• R.J. Barnes, M.S. Dhanoa, S.J. Lister (1989)
  Standard Normal Variate Transformation and
  De-trending of Near-Infrared Diffuse Reflectance
  Spectra, Applied Spectroscopy, 43, 772-777

13
SNV vs MSC
Introduction Techniques Summary
MSC
SNV
14
Detrend
Introduction Techniques Summary

• Offset
• Baseline

• R.J. Barnes, M.S. Dhanoa, S.J. Lister (1989)
  Standard Normal Variate Transformation and
  De-trending of Near-Infrared Diffuse Reflectance
  Spectra, Applied Spectroscopy, 43, 772-777

15
DerivationSavitsky-Golay
Introduction Techniques Summary

• Offset
• Baseline

• A Savitsky, MJE Golay (1964) Smoothing and
  differentiation of data by simplified least
  squares procedures, Analytical Chemistry, 36 (8),
  1627-1639
• P.A. Gorry (1990) General Least-Squares
  smoothing and differentiation by the convolution
  (Savitsky-Golay) method, Analytical Chemistry,
  62, 570-573

16
DerivationSavitsky-Golay
Introduction Techniques Summary

• Offset
• Baseline

17
DerivationSavitsky-Golay
Introduction Techniques Summary

• Offset
• Baseline

18
DerivationNorris-Williams
Introduction Techniques Summary

• Offset
• Baseline
• Normalization

• Norris, KH (1983) in Food Research and Data
  Analysis (Eds Martens H and Russwurm H Jr),
  Applied Science, London, 95-114
• Norris KH, Williams PC (1984) Optimization of
  mathematical treatments of raw near-infrared
  signal in the measurement of protein in hard red
  spring wheat I. Influence of particle size,
  Cereal Chemistry, 61, 158-165

19
DerivationNorris-Williams
Introduction Techniques Summary

• Offset
• Baseline
• Normalization

20
DerivationNorris-Williams
Introduction Techniques Summary

• Offset
• Baseline
• Normalization

21
Preprocessing
Introduction Techniques Summary

• PMSC Sensitivite to window size
• EMSC Small effect on baseline ? Detrend is
  better
• MSC and SNV are practical the same
• Savitsky-Golay is more robust than Norris-Williams

22
Preprocessing
Introduction Techniques Summary
23

• Thanks for the attention
• www.models.life.ku.dk