Obtaining limbscattered radiance data from an OMPSlike instrument

1
Obtaining limb-scattered radiance data from an
OMPS-like instrument
J. W. Bergman CPI Boulder CO J. L. Lumpe CPI
Boulder CO J. S. Hornstein NRL Washington
DC with special thanks to Q. P. Remund BATC
Boulder CO J. V. Rodriguez BATC Boulder CO
What begins as an investigation into an iterative
method for removing stray light ends up as
instrument inversion via multiple linear
regression
2
Outline

• Describe the instrument simulator
• Sources of instrument inversion error (i.e.,
  problems encountered while converting photon
  counts from the CCD to radiance as a function of
  wavelength and tangent height)
• Describe an iterative method for removing stray
  light contamination
• Works well but requires accurate knowledge of
• The PSFs and they are complicated
• Photon counts over the entire CCD which will
  be severely sub-sampled
• Use linear regression to reconstruct the full CCD
  (Caveat 2)
• Works well but
• Errors are amplified by instrument inversion
• PSF inversion computationally expensive
• Use linear regression to obtain radiance directly
  from sub-sampled data

3
The OMPS-like instrument simulator

• 6 image CCD (high and low gain 3 slits)
• Intensity reduction (mostly wavelength dependent)
• Loss in the detector (quantum efficiency)
• Loss in the optics (optical through put)
• Loss in stray light filters
• Photon dispersion (stray light)
• Described by point spread function (PSF)
• Noise
• Gaussian white noise signal-to-noise ratios
  20-1000

4
Projection onto CCD
6 images in close proximity Stray light
contamination between images can occur
Irregular mapping (i.e., the spectral
smile) Complicates inverse projection to
radiance/height space (we would prefer vertical
uniformity for spectral bands)
5
Simulated observations
Instrument simulator includes Quantum
efficiency Optical through put Stray
light Stray light filters Noise
Obvious impact of filters (6 panel
effect) Photon dispersion noticeable in
inter-image regions
6
Impacts of instrument error
Photon counts from a CCD cross section Perfect
instrument (thick black line), and impacts of
photon loss (thick blue line) and photon
dispersion (thick red line).
7
Instrument inversion problems

• Photon loss results in a wavelength dependent
  signal reduction
• Does not complicate inversion (much)
• Signal-to-noise ratio is preserved during
  inversion
• Irregular mapping of spectral/height radiances
  onto x-y pixels
• Complicates inversion
• Spectral bands are not vertically uniform
• Photon dispersion (stray light)
• Really complicates inversion
• Inversion can be sensitive to noise
• Requires accurate knowledge of PSFs
• Requires knowledge of photon counts over the
  entire CCD
• Data sub-sampling (to accommodate data rate
  constraints)
• Complicates stray light removal

8
Iterative method for stray light removal

• Stray light errors are a small perturbation to
  measured photon counts
• So, we use an iterative method based on Taylor
  expansion of inverse PSF matrix

Remove stray light with successive applications
of PSF One per iteration
9
Impact of stray light on ozone retrievals
Uncorrected stray light leads to 20-40
retrieval errors Iterative method is very
effective in only 2 iterations Low noise
sensitivity
Ozone retrieval errors from an ensemble of
atmospheric/viewing conditions as a metric for
assessing instrument inversion errors
72-member ensemble
24-member ensemble
24-member ensemble
10
Iterative method Summary of results

• Iterative method is very effective
• But,
• Requires knowledge of full CCD (OMPS will be
  sub-sampled)
• Reconstruct full CCD of observations from
  sub-sampled data
• Expensive each iteration of full PSF takes 2 hr
  on a linux box
• Simplify the PSF application (later)

11
CCD reconstruction via linear regression

• Obtain photon counts at each pixel i from a
  regression onto N (20) samples
• Training data includes radiances calculated
  directly from specified atmospheric conditions
  and the corresponding CCD arrays of photon counts
  from the instrument model (without noise)
• Perform the instrument inversion
• Quantify errors in terms of ozone retrieval errors

12
CCD reconstruction (simplified PSF)
Linear regression of full set of observations
from sub-sampled data With few exceptions, the
errors are less that 10 and typically on the
order of 1.
13
Ozone retrievals Reconstructed CCD
1 errors from the reconstruction lead to large
errors below 40 km.
No noise added
Noise added
14
Source of error

• Tests (not shown) demonstrate that 1 errors in
  the reconstruction are amplified by the PSF
  inversion
• To be useful, errors must be reduced by more than
  an order of magnitude
• Reality check Is this approach worth pursuing?
• Is it possible for the reconstruction to be
  accurate enough? (maybe)
• Is it possible to know the PSFs accurately
  enough? (maybe)
• Is it possible to simplify the PSF to reduce
  cost? (difficult to say)

So, why not use the linear regression model to
obtain radiances directly and by-pass the
explicit PSF inversion?
15
Ozone retrievals Reconstructed radiance
No noise added
Noise added
Ozone retrieval errors from the direct linear
reconstructed radiances. Data used here are
from our most realistic instrument model
(previous plots used a simplified PSF).
16
Conclusions

• Direct reconstruction of radiances via linear
  regression is
• Fast The expense is incurred during off-line
  training
• Solves all(?) instrument inversion problems at
  once including sub-sampling and the spectral
  smile
• These are only preliminary results
• Need rigorous testing
• Use larger sets of data train and test on
  independent data
• Use actual observations
• Test over a wider range of conditions
• Need to use a more realistic instrument model
• Will an instrument simulator ever be good enough
  (for a software approach to instrument
  inversion)?
• How do other retrieval problems complicate this
  method?
• Particularly altitude registration errors
• Our results are good enough to justify further
  investigation of these questions