System Level Approach to Satellite Instrument Calibration

1
System Level Approach to Satellite Instrument
Calibration

• Space Dynamics Laboratory at Utah State
  University Joe Tansock, Alan Thurgood, Gail
  Bingham, Nikita Pougatchev, Randy Jost
• NIST Raju Datla
• Ball Aerospace Technologies Corp. Edward
  Knight

2
Outline

• Calibration Philosophy
• Specmanship
• Workshop to Improve Calibration
• Calibration Planning
• Subsystem/Component measurements
• Ground Calibration
• On-Orbit Calibration
• Internal and external calibration sources
• Satellite Instrument Validation and End-to-end
  Error Model
• Summary

3
Calibration Philosophy

• Calibration
• Provides a thorough understanding of sensor
  operation and performance
• Verifies a sensors readiness for flight
• Verifies requirements and quantifies radiometric
  and goniometric performance
• Provides the needed tools to convert the sensor
  output to engineering units that are compatible
  with measurement objectives
• Provides traceability to appropriate standards
• Estimates measurement uncertainties

4
Calibration Philosophy Cal Domains

• A complete calibration will address five
  responsivity domains
• Radiometric responsivity
• Radiance and irradiance traceable to NIST
• Response linearity and uniformity corrections
• Nominal/outlying pixel identification
• Transfer calibration to internal calibration
  sources
• Spectral responsivity
• Sensor-level relative spectral response
• Spatial responsivity
• Point response function, effective field of view,
  optical distortion, and scatter
• Temporal
• Short, medium, and long-term repeatability,
  frequency response
• Polarization
• Polarization sensitivity

5
Calibration Philosophy Cal Domains

• The goal of calibration is to characterize each
  domain independently
• Together, these individually characterized
  domains comprise a complete calibration of a
  radiometric sensor
• Domains cannot always be characterized
  independently
• Complicates and increases calibration effort
• Example Spectral spatial dependence caused by
  Stierwalt effect
• Calibration parameters are grouped into two
  convenient categories
• Calibration equation
• Converts sensor output (counts, volts, etc.) to
  engineering units
• Radiometric model
• All parameters not included in calibration
  equation but required to meet calibration
  requirements

6
Calibration Philosophy Phases of Cal

• A complete and methodical approach to sensor
  calibration should address the following phases

Calibration planning during sensor design Calibration planning during sensor design
Ground measurements Subsystem/component measurements
Ground measurements Sensor-level engineering tests and calibration
Ground measurements Sensor-level ground calibration
Ground measurements Integration and test
On-orbit measurements On-orbit calibration
7
Establishment of Good Specifications Improves
Calibration

• Programs often start with a requirement such as
• The instrument shall be radiometrically
  calibrated to a 3 absolute error, 1.5 band to
  band error, and a 0.25 intra-band pixel to pixel
  error
• The designers are then asked for cost, schedule,
  and risk to meet this requirement, which could
  vary dramatically
• E.g., is error a 1-sigma or 3-sigma
  requirement?
• Furthermore, incomplete, changing, or impossible
  specifications are often the cause of cost and
  schedule overruns

8
So, What Makes a Good Specification?

• A good specification clearly communicates what
  must be accomplished
• To an audience that is reading (vs. oral
  communication)
• No other clues to help understanding
• To an audience that may not be able to ask
  questions easily
• Example reading the specification at the end of
  the program after theres been personnel turnover
• To an audience that may have a different
  background, training, or understanding of the
  problem than the author

Good Requirements Tests (examine every formal requirement with these tests)
A. Is the requirement complete (domains, interactions, worst cases)?
B. Is the requirement unambiguous (terminology, grammar)?
C. Is the specification free of errors (for example, typos, math mistakes)?
D. Is there at least one identifiable method to implement this requirement?
E. Is there at least one identifiable method to verify this requirement?
Also see E. Knight, Lessons Learned in
Calibration Specsmanship, CALCON 2005
proceedings.
9
Lessons Learned in Specifications

• Lessons
• Cover all domains (spectral, spatial, temporal,
  radiometric, polarization)
• Including interactions and worst case for
  requirements
• Scrub for ambiguity
• Use mathematical equations whenever possible to
  define requirements
• Have at least one idea for implementation in mind
  when writing the specification
• Or upon first round of review/questions
• Have at least one idea for verification in mind
  as well
• Conclusion
• The chance of an instrument
• Being poorly calibrated
• Overrunning cost and schedule targets can be
  reduced with improved calibration specsmanship

10
Workshop to Improve Quality of Calibration

• EO/IR Calibration Characterization Workshops
  held in Feb 2005 and March 2006 at SDL/USU
• Envision self governing community based
  organization with goal of improving calibration
  for all participating organizations
• Workshop Objectives
• Explore ways to improve the quality of
  IR/Visible/UV measurements, community-wide, based
  on an ISO 17025 standard, as pioneered by the RCS
  community
• Benefits based on experiences of RCS community
• Measurably and quantifiably improve the quality
  of measurements made in the community
• Facilitate data comparison between sensors,
  systems, facilities, programs and customers
• Increase in customer confidence in measurement
  results due to improved accuracy, uncertainty,
  repeatability, comparability, consistent
  documentation

11
Workshop to Improve Quality of Calibration

• Universal Agreement
• There is an unmet need that can not be addressed
  by any one organization
• Intermediate results will continue to be
  presented at annual CALCON (Calibration
  Conference)
• For more information
• CD available containing the presentations and
  recommendations of the 2005 and 2006 workshops.
• http//www.sdl.usu.edu/conferences/eo-ir/
• Based on attendee feedback provided at the 2006
  workshop, we have started the planning process
  for the next workshop, to be held Spring 2007, at
  NIST, in Gaithersburg, MD

12
Calibration Planning

• Calibration planning
• Start as soon as possible (I.e. requirements
  definition, concept design, sensor design, etc.)
• Influence sensor design to allow for efficient
  and complete calibration
• Encourages optimum sensor design and calibration
  approach to achieve performance requirements
• Planning phase can help shake out problems
• Schedule and cost risk can be minimized by
  understanding what is required to perform a
  successful calibration
• Calibration equipment needs should be identified
  early to allow time to build and test any
  required new equipment

13
Calibration Planning

• Identify instrument requirements that drive
  calibration
• Identify calibration measurement parameters and
  group into
• Calibration equation
• Radiometric model
• Flow calibration measurement parameters to trade
  study
• Schedule
• Sensor design feedback
• GSE hardware software
• Measurement uncertainty
• Risk
• Perform trade study to determine best calibration
  approach

14
Subsystem/Component Measurements

• Subsystem and/or component level measurements
• Help verify, understand, and predict performance
• Collect Parameters for the Radiometric Model that
  can't be measured well at the system level
• Minimize schedule risk during system assembly
• Identifies problems at lowest level of assembly
• Minimizes schedule impact by minimizing
  disassembly effort to fix a problem
• System/Sensor level model development and
  measurements
• Allow for the development of Measurement Equation
  and Performance prediction
• Allow for end-to-end measurements
• Account for interactions between subsystems and
  components that are difficult to predict

15
Subsystem/Component Measurements

• Merging component-level measurements to predict
  sensor level calibration parameters may bring to
  light systematic system-level uncertainties A,B
• Comparison of System-level estimate using
  component measurements with end-to-end
  measurement of SABER relative spectral
  responsivity (RSR)
• 9 of 10 channels lt 5 difference
• 1 channel ?24 difference (reason unknown)
• Helps to resolve and correct for component
  degradation and sensor performance after launch

A.) Component Level Prediction versus System
Level Measurement of SABER Relative Spectral
response, Scott Hansen, et.al., Conference on
Characterization and Radiometric Calibration for
Remote Sensing, 1999 B.) System Level Vs. Piece
Parts Calibration NIST Traceability When Do
You Have It and What Does It Mean? Steven
Lorentz, L-1 Standards and Technology, Inc,
Joseph Rice, NIST, CALCON, 2003
16
Engineering Ground Calibration

• Engineering calibration
• Performed before ground calibration
• Perform abbreviated set of all calibration
  measurements
• Verifies GSE operation, test configurations, and
  test procedures
• Checks out the sensor
• Produces preliminary data to evaluate sensor
  performance
• Feedbacks info to flight unit, calibration
  equipment, procedures, etc.
• Engineering calibration data analysis
• Evaluates sensor performance, test procedures,
  calibration hardware performance and test
  procedures
• Based on results of engineering calibration,
  appropriate updates can be made to prepare for
  ground calibration data collection

17
Ground Calibration

• Provides complete calibration needed to meet
  related requirements
• Is performed under conditions that simulate
  operational conditions for intended
  application/measurement
• Careful in-lab calibration minimizes problems
  that arise after launch
• Minimizes risk of not discovering a problem prior
  to launch
• Promotes mission success during on-orbit
  operations
• For many sensor applications
• Detailed calibration is most efficiently
  performed during ground calibration
• On-orbit calibration will not provide sufficient
  number of sources at needed flux levels
• Operational time required for on-orbit
  calibration is minimized
• Best to perform ground calibration at highest
  level of assembly possible
• Sensor-level at a minimum is recommended

18
Extending Calibration to Operational Environment

• Calibration continues after ground calibration

Sensor Design/Fabrication
Ground Calibration
On-Orbit/Field Operations
Internal Calibration Source Response Trending
On-Orbit/Field Calibration/Verification

• Internal Calibration Source Response Trending
• Trend sensor response to quantify relative
  response changes over time
• Source types
• Blackbodies, glow bars, diffusers, lamps, etc.
• Ensure source is stable and repeatable for sensor
  operational life

19
Internal Calibration Sources

• Challenges
• Ensure calibration source is stable and
  repeatable for sensor operational life
• Ideally, calibration source should use same
  optical path as external measurements
• Detailed trade to determine best approach is
  needed for each specific application
• Considerations gt source type, flux level,
  configuration, power, space, and weight
  limitations, etc.
• Sources of variability
• Temperature stability and/or temperature
  measurement
• Emissivity changes
• Thermal variations (external and internal)
• Separate drift in observed response between
  calibration source and sensor response
• Control and/or monitor electronics
• IR internal calibration source developments are
  required to achieve stringent stability
  requirements of many climate change measurements

20
On-Orbit Calibration Verifies Cal and Quantifies
Uncertainty

• Track, trend, and update calibration throughout a
  sensors operational life
• In addition to internal calibration sources make
  use of external calibration sources
• External On-orbit sources
• Standard IR stars
• Stars aBoo, aLyra, aTau, aCMa, bGem, bPeg
• Celestial objects
• Moon
• Planets provide bright variable sources
• Asteroids, etc.
• Sometimes you have to be creative
• Off-axis scatter characterization using the moon
• Other techniques
• View large area source located on surface of
  earth (often termed vicarious calibration)
• Cross-calibration between sensors
• Use of atmospheric lines
• Etc.

21
Satellite Instrument Validation

• The purpose of validation is to assess actual
  accuracy and precision of Satellite Instruments
  by comparison with validating measurements
• Apparent differences in results between
  validating and measurement system
• Satellite and validation data are not co-located
  in time and space
• Satellite and validating system have different
  vertical and horizontal resolution
• Satellite and validating system have finite
  accuracy and repeatability
• Physical measurement differences (I.e. spectral,
  sensor, platform, etc.)
• Validation Assessment Model makes comparisons
  more accurate by understanding and accounting for
  theses differences
• Make results comparable
• Validation Assessment Model can be used as a tool
  to better understand the tradeoff between
  validation approaches

22
End-to-End Error Model Overall Concept
Smoothing Parameter Noise Retrieval
Parameter Error - db Noise e Instrument
True Profile xsat
Radiance ysat
SDR
ˆ
y
EDR
ˆ
x
Performance Assessment
Validation Assessment ModelReconcile differences
to make results comparable

• ATMOSPHERE

True Profile xval
xval yval
Validation System Radiosondes, Aircraft
Measurement Systems, Cross-Calibration, etc.
23
Summary

• Calibration Philosophy
• What does calibration provide
• Calibration domains
• Phases of calibration
• Planning through operational environment
• Importance and benefit of good specsmanship
• Facilitate clear communication and minimizes risk
  of failure
• Workshop to improve quality of calibration
• Community wide participation working to improve
  calibration
• Calibration Planning
• Address all phases of calibration as early as
  possible
• Specification and design phases

24
Summary (cont)

• Calibration Measurements
• Subsystem/Component Measurements
• Minimizes schedule risk and facilitates
  development of instrument model and measurement
  equation
• Engineering Calibration and Calibration
• Methodical and careful approach leads to
  efficient and thorough calibration
• Extending Calibration to Operational Environment
• Internal calibration sources (I.e. in-flight
  internal sources)
• Challenges and need for improvement
• External on-orbit sources
• External sources and need for improvement
• Satellite Instrument Validation
• Overall concept and the need for validation
  assessment model to account for differences in
  space, time, resolution, etc.