Sebastian Torres

1
NEXRAD Range-Velocity Ambiguity Mitigation

• Sebastian Torres

Spring 2004 Technical Interchange Meeting
2
Part One

• Update on Previous Work

3
The Staggered PRT AlgorithmImages with artifacts
4
The Staggered PRT AlgorithmCorrect Images
5
The SZ-2 AlgorithmGround Clutter Filter Effects
6
Part Two

• The GMAP Ground Clutter Filter

7
How does GMAP work?

• Inputs
• Power spectrum
• from time-series data
• Apply window
• Compute DFT
• Compute magnitude squared
• Noise level (optional)
• Ground clutter spectrum width
• Outputs
• Filtered power spectrum
• Removed power

8
How does GMAP work?

• Noise power computation using rank order
  technique
• Sort power spectrum
• Compare with theoreticalcurve for white noise
• Find component at which power spectrum
  departsfrom white noise
• Identify noise components
• Compute noise power

9
How does GMAP work?

• Clutter filtering
• Generate clutter model based on sc, Ts, M, and
  window
• Determine notch width (gap) from clutter model
  and noise level
• Notch clutter components

10
How does GMAP work?

• Spectrum reconstruction
• Compute v and sv from signal components
• Fill gap with signal model
• Re-compute v and sv
• Repeat until v and sv converge
• Compute removed power
• Subtract reconstructed power from original power
  in the gap

11
GMAP Analysis Tool

• MATLAB port from RVP8 RDA 8.04 (4 Nov 2003)
• No significant changes to GMAP in subsequent
  releases
• Current release RVP8 RDA 8.05.2 (29 Mar 2004)
• Ported function fSpecFilterGMAP() and all
  supporting functions
• Tool is useful for qualitative analysis

12
GMAP Analysis Tool
13
GMAP Performance

• GMAP noise estimation
• Inadequate for small number of samples
• GMAP interpolation
• Restores weather signal power in the gap
• GMAP in the absence of clutter
• Some components are filtered nevertheless
• Spectrum can become distorted

14
Estimating Clutter Power from GMAP

• Simulation
• Input Signal Clutter
• Signal velocity is random in (-va, va)
• Signal spectrum width is 1, 2, 4, and 8 m/s
• CSR is varied from -30 to 50 dB
• CNR gt 20 dB for all cases
• Process GMAP
• Output Removed power PREM
• Goal Establish suitability of PREM as an
  estimate of clutter power

15
Estimating Clutter Power from GMAP
PREM 0 if clutter filtering adds power
CSR(dB)

• Removed power from GMAP is unreliable as an
  estimate of clutter power, especially for low CSR

16
Suitability of GMAP

• Algorithm is not available in the literature
• Details are proprietary
• Uses several empirical constants
• GMAP works better for large M
• Modifications made by SIGMET to handle small M
• Ice et al. (2004) reported compliance with NEXRAD
  requirements using black-box analysis
• Recommended using Blackman window and providing
  noise level to GMAP
• Good candidate for SZ no phase distortion
• Minor changes are required (stay tuned!)

17
Part Three

• GMAP and Phase Coding

18
GMAP and Phase Coding

• GMAP designed for uniform sampling, non-phase
  coded signals
• Issues
• Window effect
• Noise estimation
• Spectral reconstruction
• Filtered time series

19
Window Effect

• Window choice
• Sachidanda et al. (NSSL Report 2, 1998) recommend
  Von Hann window to minimize errors
• Ice et al. (ORDA Report, 2004) recommend Blackman
  window to achieve larger clutter suppression
• Blackman is more aggressive than Von Hann
• Should expect larger errors of estimates

20
Window Effect

• Simulation
• Input Signal in the 1st trip Signal in the 2nd
  trip
• 1st Trip Signal
• Velocity is random in (-va, va)
• Spectrum width varies from 0.5 to 8 m/s
• 2nd Trip Signal
• Velocity is random in (-va, va)
• Spectrum width is 1, 2, 4, and 8 m/s
• S1/S2 varies from 0 to 70 dB
• Process SZ-2 with GMAP
• Case 1 Blackman window
• Case 2 Von Hann window
• Output Statistics of v1 and v2 estimates

21
Window Effect
Blackman Window
Von Hann Window
22
Noise Estimation

• Issue Use GMAP noise estimation or provide noise
  to it?
• Out-of-trip echo looks like white noise
• GMAP noise estimation fails
• GMAP notch width based on over-estimated noise
  level is narrower than required

23
Noise Estimation

• Simulation
• Input Signal in the 1st trip Signal in the 2nd
  trip Clutter in the 1st trip
• 1st Trip Signal
• Velocity is random in (-va, va)
• Spectrum width varies from 0.5 to 8 m/s
• 2nd Trip Signal
• Velocity is random in (-va, va)
• Spectrum width is 1, 2, 4, and 8 m/s
• S1/S2 varies from 0 to 70 dB
• C/S1 varies from -30 to 50 dB
• Process SZ-2 with GMAP
• Case 1 GMAP with noise estimation
• Case 2 GMAP with provided noise
• Output Statistics of v2 estimates

24
Noise Estimation
GMAP with noise estimation
GMAP with provided noise
25
Spectrum Reconstruction

• Issue Use GMAP interpolation or apply notch
  filter?
• Interpolation helps reducing biases (in strong
  signal moments) due to clutter filtering
• Interpolation assumes coherent weather signal is
  present after clutter filtering
• Two cases to consider
• Clutter with the strong signal
• Clutter with the weak signal

26
Spectrum Reconstruction

• Clutter with the strong signal

v
27
Spectrum Reconstruction

• Simulation
• Input Signal in the 1st trip Signal in the 2nd
  trip Clutter in the 1st trip
• 1st Trip Signal
• Velocity is random in (-va, va)
• Spectrum width varies from 0.5 to 8 m/s
• 2nd Trip Signal
• Velocity is random in (-va, va)
• Spectrum width is 1, 2, 4, and 8 m/s
• S1/S2 varies from 0 to 70 dB
• C/S1 varies from -30 to 50 dB
• Process SZ-2 with GMAP
• Case 1 GMAP with interpolation
• Case 2 GMAP without interpolation
• Output Statistics of v1 estimates

28
Spectrum Reconstruction
GMAP with interpolation
GMAP without interpolation
29
Spectrum Reconstruction

• Clutter with the weak signal

v
30
Spectrum Reconstruction

• Simulation
• Input Signal in the 1st trip Signal in the 2nd
  trip Clutter in the 2nd trip
• 1st Trip Signal
• Velocity is random in (-va, va)
• Spectrum width varies from 0.5 to 8 m/s
• 2nd Trip Signal
• Velocity is random in (-va, va)
• Spectrum width is 1, 2, 4, and 8 m/s
• S1/S2 varies from 0 to 70 dB
• C/S1 varies from -30 to 50 dB
• Process SZ-2 with GMAP
• Case 1 GMAP with interpolation
• Case 2 GMAP without interpolation
• Output Statistics of v1 estimates

31
Spectrum Reconstruction
GMAP with interpolation
GMAP without interpolation
32
Filtered Time Series

• GMAP returns power spectrum
• Phases must be saved to reconstruct full spectrum
  and return to time domain
• Issue What are the phases of the reconstructed
  components?
• Original phases
• Zero phases
• Something else?

33
Filtered Time Series

• Simulation
• Input Clutter in the 1st trip Signal in the
  2nd trip
• 2nd Trip Signal
• Velocity is random in (-va, va)
• Spectrum width varies from 0.5 to 8 m/s
• C/S2 varies from 0 to 70 dB
• Process SZ-2 with GMAP
• Case 1 Reconstruction with original phases
• Case 2 Reconstruction with random phases
• Case 3 Reconstruction with zero phases
• Output Statistics of v2 estimates

34
Filtered Time Series
Weather is in the 2nd trip and Clutter is in the
1st trip
Original Phases
Random Phases
Zero Phases
35
GMAP and Phase CodingSummary

• Use Blackman window for required suppression at
  the expense of loss of accuracy
• Provide noise level to GMAP
• Reconstruct filtered spectrum using zero phases
  in the gap
• Use GMAP interpolation if clutter is with strong
  signal
• Dont use GMAP interpolation if clutter is with
  weak signal

36
GMAP vs. Elliptic GCF
37
GMAP vs. Elliptic GCF
38
GMAP vs. Elliptic GCF
39
GMAP vs. Elliptic GCF
40
Part Four

• Clutter Filtering in the SZ-2 Algorithm

41
Clutter Filtering in SZ-2

• Clutter filtering is controlled by map
• Bypass map automatically generated
• Clutter censor zones operator defined
• Issues
• Sequence of operations
• Conditions for filtering
• Recovery of weak-trip signal
• Ground clutter in any trip
• Overlaid ground clutter
• Anomalous propagation in any trip
• Overlaid ground clutter and AP

42
Basic Sequence of Operations

• Cohere for trip with clutter
• Apply clutter filter
• Cohere for trip with strong signal
• Recover strong-trip velocity
• Apply PNF
• Cohere for trip with weak signal
• Recover weak-trip velocity

43
Conditions for Filtering

• Ground clutter
• Determined by clutter map
• AP
• Determined by operator (censor zones) and GMAP
  during long-PRT scan
• Filter could be bypassed for low CSR
• CSR from GMAP is unreliable
• Issue Will clutter maps in ORDA contain clutter
  power?

44
To filter or not to filter?

• Simulation
• Input Signal in the 1st trip Signal in the 2nd
  trip Clutter in the 1st trip
• Process SZ-2 with GMAP
• Case 1 No filtering
• Case 2 GMAP with noise estimation
• Case 3 GMAP with provided noise
• Case 4 GMAP without interpolation
• Output Statistics of v1 and v2 estimates

45
To filter or not to filter?
46
To filter or not to filter?
47
To filter or not to filter?

• Simulation
• Input Signal in the 1st trip Signal in the 2nd
  trip Clutter in the 2nd trip
• Process SZ-2 with GMAP
• Case 1 No filtering
• Case 2 GMAP with noise estimation
• Case 3 GMAP with provided noise
• Case 4 GMAP without interpolation
• Output Statistics of v1 and v2 estimates

48
To filter or not to filter?
49
To filter or not to filter?
50
Clutter Filtering Issues

• If clutter is not with the strong signal, the
  weak signal cannot be recovered
• Weak trip must be censored
• Should use GMAP without interpolation (notch)
  when clutter is not with the strong signal
• Minor changes to function fSpecFilterGMAP() are
  required

51
Clutter in Any Trip

• Clutter can be ground clutter or AP
• Go after clutter first
• Cohere for trip with clutter
• Apply clutter filter
• Censor gates with overlaid clutter
• Clutter location can be obtained from bypass map
  and AP map (generated during long PRT from
  clutter censor zones and GMAP removed power)

52
Part Five

• Proposed SZ-2 Algorithm

53
Proposed SZ-2 Algorithm

• Basic algorithm as reported by NCAR-NSSL in joint
  report of Aug 15, 2003
• Minor changes to handle
• GMAP ground clutter filter
• Ground clutter in any trip
• Processing notch filter
• Spectrum width computation
• Censoring
• Prototype of proposed SZ-2 algorithm coded and
  tested in MATLAB

54
Changes to use GMAP

• Window data using Blackman window
• Compute DFT
• Save phases
• Compute power spectrum
• Apply GMAP
• Save number of coefficients with clutter
• Minor changes to function fSpecFilterGMAP() are
  required

55
Changes to Handle Clutter in Any Trip

• Analyze bypass map
• Determine whether ground clutter is present in
• No trips
• One trip
• Multiple trips
• If ground clutter is not present
• Do not filter
• If ground clutter is present in just one trip
• Cohere to trip with clutter and remove it
• Proceed as usual
• If ground clutter is present in multiple trips
• Censor

56
Changes to PNF Center

• PNF tries to remove most of the strong-trip
  signal and preserve 2 clean replicas of the
  out-of-trip signal
• If clutter is not present
• PNF is centered at vS (no change)
• If clutter is present
• PNF is centered at adjusted vS

57
Processing Notch Filter

• Location determined by vs and presence of clutter
• Notch Width determined by strong and weak trip
  numbers
• 8 replicas ? NW 3M/4
• 4 replicas ? NW M/2

58
PNF Center

• Simulation
• Input Signal in the 1st trip Signal in the 2nd
  trip Clutter in the 1st trip
• Process SZ-2 with GMAP
• Case 1 PNF centered at vS
• Case 2 PNF centered at vS/2
• Case 3 PNF centered at adjusted vS
• Output Statistics of v2 estimates

59
PNF Center
PNF centered at vS/2
PNF centered at adjusted vS
PNF centered at vS
60
Changes to PNF Center

• PNF must be centered such that
• vPNF is the closest to vS
• PNF stop band includes GCF notch
• PNF center is computed from
• vS
• NW
• kGMAP

61
Changes to Spectrum Width Computation

• Spectrum widths are obtained
• From the short-PRT scan for the strong trip
• From the long-PRT scan for the weak trip
• Strong-trip spectrum width computation
• Legacy algorithm uses S/R1
• S must be computed after determination of
  weak-trip power

62
Spectrum Width Computation
va 8.9 m s-1, ra 466 km, sv,max 5.15 m s-1
63
Spectrum Width Computation
va 23.7 m s-1, ra 175 km , sv,max 13.7 m s-1
64
Changes to Censoring

• Basic censoring remains the same
• Use same thresholds as in legacy processing
• SZ-2 censoring
• May need refined constants
• Plots of SD(v1) and SD(v2) on the S1/S2 vs. s1
  plane are useful to determine censoring constants
• Issue Have we only looked at SD(v2)?
• Additional censoring to handle clutter in any
  trip
• Gates with overlaid clutter are censored

65
Summary of Changes to the SZ-2 Algorithm
Proposed SZ-2
SZ-2 as of Aug 15 2003
Determine Overlaid Trips and Censoring
Determine Overlaid Trips and Censoring
Determine Location of Ground Clutter
Cohere for Ground-Clutter Trip
Cohere for First Trip
Filter Ground Clutter
Filter Ground Clutter
Compute Filtered Power
Compute Filtered and Unfiltered Powers
66
Summary of Changes to the SZ-2 Algorithm
Cohere for Trips A and B
Cohere for Trips A and B
Compute lag-one Correlations for Trips A and B
Compute lag-one Correlations for Trips A and B
Determine Strong and Weak Trips
Determine Strong and Weak Trips
Compute CSR
Compute Strong-Trip Velocity
Compute Strong-Trip Velocity
67
Summary of Changes to the SZ-2 Algorithm
Compute PNF Center Velocity
Apply Window
Compute DFT
Compute DFT
Apply PNF
Apply PNF
Compute IDFT
Compute IDFT
Cohere for Weak Trip
Cohere for Weak Trip
68
Summary of Changes to the SZ-2 Algorithm
Compute Weak-Trip Power
Compute Weak-Trip Power
Adjust Powers
Adjust Powers
Compute Strong-Trip Spectrum Width
Compute Weak-Trip lag-one Correlation
Compute Weak-Trip lag-one Correlation
Compute Weak-Trip Velocity
Compute Weak-Trip Velocity
69
Summary of Changes to the SZ-2 Algorithm
Assign Correct Range
Assign Correct Range
Censor and Threshold
Censor and Threshold
Compute Reflectivity
Compute Reflectivity
Clip and Scale
Clip and Scale
70
Part Six

• Further Refinements of
• the SZ-2 Algorithm

71
Further Refinements

• Proposed SZ-2 algorithm works fine
• However, theres room for improvement
• Improvements require more work
• Some are still under research
• All involve larger changes to proposed SZ-2
  algorithm
• Improvements are proposed for later releases of
  SZ-2

72
AP in Any Trip

• Operator defines zones with AP using systems
  clutter censor zones
• GMAP is used during the long-PRT scan to
  determine gates with significant clutter in these
  zones
• AP map is generated during long-PRT scan
• AP map and bypass map are combined
• Composite map is used in the algorithm

73
Strong Overlaid Echoes

• Situations where S1/S2 lt 5 dB may require
  double processing
• Cohere clutter-filtered time series to strong
  trip
• Apply PNF
• Cohere to weak trip
• Compute vw
• Cohere clutter-filtered time series to weak trip
• Apply PNF
• Cohere to strong trip
• Compute vs

74
Spectrum Width Computation

• Weak-trip spectrum width computed from long-PRT
  scan is limited
• Legacy maximum spectrum width va/v3
• Could use deconvolution
• Same drawbacks as in SZ-1
• Needs further testing

75
The End