Detection of Anthropogenic Signals that are Below Thermal Noise Power

1
Detection of Anthropogenic Signals that are
Below Thermal Noise Power

• Roger De Roo
• IEEE SEM Fall Conference
• 2012 Nov 14

2
Outline

• Motivation
• Physics of passive remote sensing
• Monitoring the soil moisture with microwave
  radiometry
• Radio Frequency Interference a major problem
• Summary of RFI detection approaches
• Kurtosis algorithm development
• Conclusions
• Thanks to Chris Ruf, U-M Joel Johnson, OSU
  Jeff Piepmeier, NASA Goddard Sid Misra, JPL

3
Soil Moisture who cares?

• Soil Moisture regulates plant transpiration
• Transpiration determines humidity
• Humidity gives rise to clouds
• No widespread measurements of soil moisture
  currently

4
Whats so great about Microwave Remote Sensing?
Long wavelengths (3mm to 30cm) dont scatter off
of objects the size of cloud droplets --
microwaves see through clouds
Radar (Active) Radiometry (Passive)

• Very high spatial resolution
• Power hungry expensive
• Sensitive to geometry of water eg. Movement of
  trees causes big signal changes

• Poor spatial resolution
• Low power requirements
• Insensitive to geometry of water

5
Planck Blackbody Radiation
6000K white hot the Sun
3000K red hot
300K room temp
30K
3K outer space
1 GHz
1 THz
1 PHz frequency wavelength
0.3 m
0.3 mm 0.3 um
6
Microwave Characteristics of the Atmosphere
from LeVine, Wilheit, Murphy and Swift, 1989
7
Products by frequency
Also -Sea surface salinity at
1.4GHz -Vegetation moisture content at 1.4 and
6 GHz -Vegetation temperature at 18 90 GHz
from LeVine, Wilheit, Murphy and Swift, 1989
8
Microwave Brightness and Moisture

• Water molecules have large electric dipole,
  unlike rest of nature

H - O H
Liquid water molecules will orient itself with
passing electromagnetic waves, slowing the wave
down The molecule can keep up with the wave until
9 GHz (index of refraction n 9 at 1GHz, but n
2 at 100 GHz)
e' n2
9
Microwave Brightness and Moisture

• An interface w/ high contrast of index of
  refraction leads to reflection
• Dry soils appear warm, while wet soils appear
  cold, at the same temp.

Transparent Atmosphere
Transparent Atmosphere
High Contrast at Interface
Low Contrast at Interface
10
Example Brightness Image from Space
NASDA
11
Sensitivity of Radiobrightness to Soil Moisture
Under a Vegetation Canopy
19 GHz
6.9 GHz
1.4 GHz
Courtesy of P. ONeill
12
University of Michigan Radiometers
L-band 1.4 GHz l 21 cm satellites SMOS Nov
09 Aquarius Jun 11 SMAP 14
C-band 6.7 GHz l 4.5 cm satellites AMSR-E
02-11
19 GHz 37 GHz l 1.6 cm l 0.8 cm
satellites SSM/I etc. 87 to present
Antenna size is proportional to wavelength
13
The Tundra Landscape
14
Diurnal Brightness Measurements
15
Brightness of Tundra and Shrubs
16
Trouble with the 1.4GHz Radiometer
Both of these ranges appear plausible
17
Potentially Interfering RadarsCobra Dane
Peak Transmit Power 16.8 MW Transmit Frequency
1.215-1.375GHz
Raytheon
18
Surrounded by Interfering Radars?
FPS-124
FPS-108 Cobra Dane
Observation site Toolik Lake
FPS-117
ITT, 05
19
AMSR-E Interference at 6.9GHz
If it is not purple, we cannot use the data from
that location If it is purple, the data from that
location might be OKor not
Li et al., 04
20
Traditional Radiometer Technology

• Use square law detector for signal power

v
k1 v2
21
Effect of Finite Samples RFI-free signals

• Variance of voltage waveform contains brightness
  power
• PIF kBTSYSB G
  ltv2(t)gt/Z
• TSYS TB TREC
• Finite number of samples results in a measurement
  variance
• s NE?T TSYS / vN
  NBt
• RFI always biases measurements of TB upwards
• Averaging preserves the bias thus, not a
  solution
• Wed like to see RFI at near the NE?T power level

22
Approaches to Detecting RFI

1. Time domain look for pulses
2. Frequency domain look for carrier frequencies
3. Amplitude domain look for non-thermal
   distribution

Gaussian pdf
Non-Gaussian pdf
Sinusoidal waveform
Thermal waveform
23
Digital Radiometry
Digital radiometers use fast analog-to-digital
converters to measure the voltage waveform
Power is determined by finding the variance
(2nd moment) of the quantized data Processing
capability allows for implementation of one or
more RFI mitigation strategies
24
Literature Search

• What has already been done on this problem, or
  related problems?
• For RFI mitigation, nothing in the amplitude
  domain. Some in time-domain and some in
  frequency domain.
• However, testing for normality of a distribution
  does have a rich literature. Lotsa ways to do
  it, and it is known how well they work.

25
Is it Normal?

• Statistical moments
• 0th event count
• 1st Mean
• 2nd Variance
• 3rd Skew
• 4th Kurtosis

26
Skew

• Measures how asymmetric a distribution is
• Normal distribution has zero skew
• So does RFI ?

27
Sources of Skew

• No skew for
• pulsed sinusoid (ps),
• Amplitude Modulation (AM), or
• Frequency Modulation (FM)
• Skew possible, but unlikely, with Phase
  Modulation (PM)

AM
FM
PM
28
Kurtosis

• Kurtosis measure peakedness of a distribution
• Normal distribution has kurtosis 3
• RFI can have any kurtosis

29
Definition of Kurtosis

• Desired radiometric (science) signals generated
  by thermal noise
• Gaussian (bell-curve) probability distribution
  function (PDF)
• RFI is man-made
• PDFs will be non-Gaussian in general
• Underlying Statistics
• all higher-order moments of a Gaussian are
  uniquely determined by its lowest two moments
• for example, the kurtosis
  equals 3 for a Gaussian v(t)
• where v(t) is the zero-mean pre-detected
  radiometer output voltage

30
Technology Approach

• Digitized IF waveforms lend themselves to moment
  estimation
• Use moment ratios to test for presence of RFI
• 1st moment, ?1, is a DC offset
• 2nd central moment, m2, is power the
  measurement objective
• Odd central moments are all zero
• The lowest moment for RFI detection is the 4th
  central moment m4

31
Alternative Technology Approach(explored by Jeff
Piepmeier of NASA)

• Use square law detector for signal power
• (traditional radiometer architecture)
• Use a second square law detector for the 4th
  moment

v
k1 v2 k2 v4
32
Probability Densities of signals w/ w/o RFI
Expected PDF of thermal noise with variance s2
with pulsed sinusoidal RFI of amplitude A and
duty cycle d (extension of Rice, 1948)

Noise w/ constant power, s2 RFI w/ Constant
Amplitude, A Varying duty cycle, d
33
All curves have the same variance A radiometer
will report all of these signals as the same
brightness
Pulsed sinusoid to noise ratio S dA2/2s2
34
Effect of Finite Samples RFI-free signals

• Exact kurtosis pdf unknown
• Kurtosis pdf is skewed
• less so as N?8
• kurtosis pdf is essentially Gaussian Ngt50k
• Mean of R3(N-1)/(N1)
• Variance of R ? 24/N as N?8

35
Detection Concepts
36
False Alarm Rate and Probability of Detection of
Pulsed Sinusoidal RFI

• For RFI power level at brightness temperature
  equivalent to 2NEDT, detection threshold can be
  set to give
• 90 probability of detection
• 3 false alarm rate
• 0.1 duty cycle case corresponds to ARSR-1
  operating mode
• Higher duty cycle reduces detectability

37
Minimum detectable RFI

• d is radar duty cycle
• PD1-FAR
• z is a FAR parameter
• z3 ?? FAR0.25
• z2 ?? FAR 5
• z1 ?? FAR30
• For large N,
• min detectable RFI
• TPS N-¼
• NEDT N-½

38
Blind Spot at 50 duty cycle, and solution
CFAR R4 (kurtosis)
13 blind spot
50 blind spot
61 blind spot
CFAR R6
R6 -0.085
R60.085
R60.085
N100kSa Threshold at 1s (30 FAR)
R4-0.0155
R40.0155
39
Laboratory Experiments

• Check assumptions about radiometer operation
• RFI is prescribed to conform to our theorys
  assumptions

40
Laboratory Experiment Results

• Kurtosis R m4/m22
• In the absence of RFI, R3
• For CW RFI (eg. Carriers) Rlt3
• For short duty cycle RFI (eg. RADAR), Rgt3
• But R3 also for 50 duty cycle

41
Kurtosis of RFI free signals

• Kurtosis False Alarm
• Rate confirmed with
• simulations
• PALS-ADD data
• a minute of apparently
• RFI free data
• RFI free
• assumption
• supported
• by kurtosis FAR
• predictions

Theoretical FAR1-erf( z /v2)
RFI flags from clean PALS-ADD data
z Rth-3 / sR0
42
Kurtosis of the RFI-free digitization effects

• Effects considered (and are very small)
• Kurtosis pdf itself is Gaussian for 50k
  independent samples
• Clipping of signals by finite Analog to Digital
  Converter (ADC) dynamic range 4 bits is enough
  3 bits, maybe
• Digitization (ADC bin size) effects are
  negligible.
• ADC null offsets can be corrected with 1st and
  3rd moments in addition to 2nd and 4th moments
  needed for kurtosis algorithm.
• Effects not yet considered
• Integral Nonlinearity and Differential
  Nonlinearity of Analog to Digital Converters
  likely is small effect because Flash ADC
  typically have small INL and DNL
• Correlated data we are still applying the
  theoretical tools to analyze the effects of
  sampling above the Nyquist rate on the kurtosis
  calculation.

43
Kurtosis of RFI-free effects of digitization

• Digitization reduces kurtosis
• Bin size effects decrease as noise amplitude
  increases
• Threshold locations not critical for sgt3/4
• Saturation of ADC at high noise amplitude
  distorts kurtosis

Number of ADC bins
s 3/4
44
Field Experiments

• A lot of fun to do!
• Takes lots of people () to do.

45
Example of RFI detection with Kurtosis (1)

• 1 minute of data from ADD back-end attached to
  PALS front-end at JPL
• Antenna looking to sky
• Kurtosis thresholds set to trigger 1 false alarm
  per minute
• Flagged observations some obvious RFI, some not

RFI flags
TSYS (counts2)
seconds
46
Example of RFI detection with Kurtosis (2)

• Another PALS minute of data same kurtosis
  thresholds
• Antenna looking to sky absorber placed over
  antenna
• Changes in brightness do not get flagged

RFI flags
TSYS (counts2)
sky
sky
absorber
seconds
47
Airborne Campaign Results
48
Soil Moisture Active and Passive (SMAP)

• NASA environmental satellite
• Currently in planning stages
• Launch Nov 2014
• Kurtosis is the main RFI detection algorithm

49
Conclusions

• Theoretical behavior of the kurtosis statistic as
  a detector of pulsed sinusoidal RFI has been
  explored.
• Kurtosis has a blind spot at 50 duty cycle
  sinusoids
• CW RFI lowers kurtosis below 3
• Low duty cycle sinusoidal pulses raise the
  kurtosis above 3
• Kurtosis is very sensitive to low duty cycle
  sinusoid pulses
• Kurtosis is minimally affected by digital
  receiver properties
• False Alarm Rate of kurtosis algorithm is
  confirmed
• Minimum detectable RFI is comparable to NEDT in
  realistic circumstances, may be less than NEDT
• Kurtosis false alarms do not bias the estimate of
  the brightness
• The kurtosis does not flag gradual changes in
  brightness.

50
Thank You!
51
Backup Slides
52
ADC offset and non-central moments

• offset in ADC ground requires 4 moments
• 3rd moment of questionable value
• elimination of 3rd moment can
• relax back-end data rate requirements,
• allow more subbands, and/or
• permit shorter integration periods

vi1
vi 0
vi -1
vi -2
53
Tanana River Breakup at Nenana
Guess the moment of breakup! Tickets cost 2.50
each Typical Jackpot 300,000 www.nenanaakicec
lassic.com
54
Observed Global Temperature Trends
IPCC 01
55
Projected Global Temperature Trends
2071-2100 temperatures relative to
1961-1990. Special Report on Emissions Scenarios
Storyline B2 (middle of the road warming).
IPCC 01
56
Carbon Stocks by Biome
Atmospheric stock is about 750PgC
IPCC 01
57
Permafrost extent
Global Terrestrial Network for Permafrost
58
20m Borehole Temperature Trends in AK
Hinzman et al 2005
59
Permafrost structure
NSIDC
60
Active Layer Depth Trends
Maximum Active Layer Depth (cm)
Year
Circumpolar Active Layer Monitoring Network
61
Strategy for Estimating Temperature and Moisture
Profiles
62
Calibrated LSP/R model of Prairie Grassland
Judge et al. 1999
63
Correlated Noise Calibration System
To Radiometer
From AWG
Low Noise Amplifier (LNA) input is a matched
source of sub-ambient noise it is an electronic
device which, at RF, looks like it is at LN2
temperatures CNCS concept Onto this very low
noise background, couple in some much stronger
noise. This much stronger noise can be
generated in a COTS Arbitrary Waveform
Generator CNCS extension This same concept
can be used to create known weak RFI
Ruf and Li, 03
64
Detection and Mitigation Testbed
65
Conclusions
Microwave Radiometry has been demonstrated to
have high sensitivity to surface soil
moisture. Hydrologic models can use this
measurement to constrain the evolution of
profiles of temperature and moisture. This
technique should work well for the low
vegetation content of the Arctic. Understanding
the evolution of the active layer will help us
understand the threat of carbon release from
Arctic soils in response to climate
change. Microwave observations are very
susceptible to interference. RFI mitigation for
microwave radiometry is an emerging research area
at Michigan
66
Microwave Brightness and Moisture

• Water molecules have large electric dipole,
  unlike rest of nature
• An interface w/ high contrast of index of
  refraction leads to reflection
• Dry soils appear warm, while wet soils appear
  cold, at the same temp.

H - O H
Liquid water molecules will orient itself with
passing electromagnetic waves, slowing the wave
down The molecule can keep up with the wave until
9 GHz (index of refraction n 9 at 1GHz, but n
2 at 100 GHz)
Transparent Atmosphere
Transparent Atmosphere
Low Contrast at Interface
High Contrast at Interface