National Precipitation Verification Unit

1
National Precipitation Verification Unit

• Letitia L. Soulliard
• RSIS NOAA/NWS/OCWWS

NPVU visits the NWRFC April 21, 2004 Portland,
Oregon
2
What is the NPVU?

• National
• Precipitation
• Verification
• Unit
• Verifies the QPF from the NGM, Eta, AVN, CONUS
  RFCs, and HPC

3
Who oversees the NPVU?

• Office of Climate, Water, and Weather Services
• Located at
• Hydrometeorological Prediction Center/NCEP

4
Why is the NPVU Important?

• Verification of direct NWP model, statistical,
  and forecaster value-added QPFs and PoPs is
  necessary to quantify and improve the skill of
  QPF/PQPF and PoP forecast at each step of the NWS
  End- to- End Forecast Process.
• Office of Meteorology (1999)
• One of the most important components of an
  effective national QPF program is a comprehensive
  objective comparative verification system
• National Weather Service (1999)

5
When did the NPVU start?

• Operation start date was October 1, 2000
• Not every RFC was implemented immediately
• Eastern RFCs
• Had day 1 totals starting in August 2000
• Had 6hr amounts starting October 2000
• Western RFCs
• CN and CB started in November 2000
• NW started in January 2001
• Mosaic of all RFCs
• Started January 2001

6
What are the different NPVU Products found on the
web?

• QPF Verification Statistics
• Comparison Plots
• RFC Live Data
• Archive Data
• RFC QPE
• HPC QPF
• Offline request form

7
Why does the NPVU produce statistics?

• It is the main function of the NPVU
• To see how forecast products change over time
• To see how each product is doing in general
• To see relative QPF performance measures

8
Why does the NPVU produce Comparison Plots?

• To see subjective comparison of performance
• To graphically demonstrate what is being compared
  in the statistical products
• It is useful for understanding anomalies in the
  statistical scores

9
Why does the NPVU post the Live Data to the web
site?

• It is Quick and Easy - One stop shopping
• To verify NPVU is getting good data
• To see mosaic of the Continental RFCs

10
Why does the NPVU provide a WWE comparison matrix?

• WFO requested it
• Already had access to Model and HPC forecast and
  RFC observations

11
Why does the NPVU give access to its archive?

• RFC QPE
• To view any observed rainfall event since the
  start of the NPVU
• To see what period might be good for a historical
  study
• HPC QPF
• To view any HPC forecast in the past month
• Offline Archive
• To allow anyone access to whatever historical
  data the NPVU has available

12
What is verified by the NPVU and when is it
updated?

• 12Z QPF for RFCs and HPC
• 00Z QPF for Models
• Updated Once a Month approximately around the
  10th for the previous month
• E-mail notification

13
What statistical scores are calculated by the
NPVU?
14
How does the data get from the RFCs to the NPVU?
RFCs QPF QPE
Network Control Facility
AWIPS WAN
SBN
Offices of Systems Operations
dbnet
NPVU
IBMSP
FTP
15
How are the forecast and observation grids
modified in order to be compared?

• Receive
• 4km QPE from RFC
• 10km QPF from RFC
• 1x1 degree AVN
• 32km Eta
• 90km NGM
• 32km HPC

• Convert to 32km by
• Spatial average
• Area preservation technique
• Area preservation technique
• Change of area and map projection
• Area preservation technique
• No conversion needed

16
Why should you use the NPVUs products?

• To see if there is an improvement over HPCs
  forecast
• To see if HPC is improving upon the models
• To see which model might be better for each
  season or for each RFC area
• To view QPF products for other RFCs and view
  continuity issues

17
What problems are there in the statistics?

• Small sample sizes can result in anomalous scores
• The eastern RFCs differ from the western RFCs due
  to Mountain Mapper
• QPE discontinuity problems

18
Example of when a small sample size can result in
anomalous scores

• Observation Profile shows the number of cases is
  dramatically less in 2003 than in 2002

19
Example of when a small sample size can result in
anomalous scores (cont...)

• RFCs Categorical Bias drastically improves from
  2002 to 2003.

20
Example of when a small sample size can result in
anomalous scores (cont...)

• Mean Absolute Error drastically improves from
  2002 to 2003.

21
Which RFCs are considered eastern and which are
western?

• Eastern RFCs
• ABRFC
• LMRFC
• MARFC
• MBRFC
• NCRFC
• NERFC
• OHRFC
• SERFC
• WGRFC

• Western RFCs
• CBRFC
• CNRFC
• NWRFC
• APRFC

22
What are the differences between the Eastern RFCs
and the Western RFCs?

• Eastern RFCs
• 00Z and 12Z Forecast
• 18Z forecast
• AB, MA, MB and NC RFC
• Forecast out to 24hr
• MA and SE out to 48hr
• QPE are stage III multi-sensor data

• Western RFCs
• 12Z forecast ONLY
• Forecast out to 72hr
• QPF is made for points
• QPE is gauge data
• Both are gridded using Mountain Mapper

23
What is Mountain Mapper and how does it affect
the NPVU's product/process?

• Developed at CBRFC to convert point data into
  gridded data
• Uses PRISM precipitation Climatology
• Since both QPE and QPF are run through this they
  are both biased toward the climatology where as
  the models and HPC are not

24
What is the NPVU doing about this Mountain Mapper
Issue?

• HPC provided forecast run through Mountain Mapper
• For the CNRFC, CBRFC, and NWRFC areas
• Since January 2003
• Statistics have been run for each month, warm and
  cool season, and annual
• Mountain Mapper has not improved HPCs forecast
  as much as expected

25
Sample of Mountain Mapper Results for 2003
26
What are some examples of QPE discontinuity
problems?

• Errors due to different methods (Stage III vs.
  Gauge Data)
• Errors due to interference with Radar
• Errors due to RFC QC process

27
Errors due to different methods
28
Errors due to interference with radar
29
Errors due to RFC QC Process
30
Thank You

• NPVU web site
• http//www.hpc.ncep.noaa.gov/npvu
• My e-mail
• letitia.soulliard_at_noaa.gov