Title: LEADS System Overview
1METEOSTAR LEADS
2WHAT IS LEADS
- Leading Environmental Analysis and Display System
- Development Began At Lockheed In Late 1989 To
Produce A New Commercial Weather Analysis Product - LEADS Grew Out Of Two Military Weather Contracts
- TESS(3) -- A Navy Program Which Uses Surface
Observations And Numerical Forecast Models To
Provide Tactical Weather Forecasts For Carrier
Fleets - Mark IVB -- An Air Force Program Which Uses
Satellite Imagery And Analysis Tools To Provide
Long-Range Strategic Forecasts - In 1994 Lockheed Was Brought Under Contract With
TCEQ To Add Pollution - In 1997 Information Processing Systems (IPS)
Acquired The MeteoStar Product Line
3SYSTEM CAPABILITIES
- Near Real-time Data Acquisition
- Pollution Sensor Data
- Water Probe Data
- Conventional Meteorological Surface Data
- Weather Satellite Imagery (Geostationary)
- Automated Gas Chromatograph (AutoGC)
- Upper Air Data
- Radar Profiler And Acoustic Sounder
- Visualization
- Long-term Data Archival
- EPA Reporting
- Analysis
- Ozone Action Day Prediction
- Transport Studies
- Pollution Source Determination
4SYSTEM PHILOSOPHY
- The MeteoStar LEADS At TCEQ Was Designed As An
Upgrade To An Existing Network - TCEQ Was Unable To Expand Their Existing Network
Due To Hardware Constraints In Their Data
Collection System - Individual Instruments And Sites Worked Well, But
No New Instruments Could Be Added - Communications And Data Collection Needed
Upgrading - A New Communications Network Was Designed And
Interfaced Into The Existing TCEQ Monitoring
Network - Changes At The Monitoring Sites Included A New
Datalogger, An Automatic Gas Calibrator And
Valves, And A New Modem - TCEQ Specified The Gas Calibrator Separate From
Other Equipment And Selected The Dasibi 5008 - The Zeno Datalogger Was Chosen Before The
Calibrator Was Selected Based On Its Capabilities
And The Willingness Of Coastal To Modify
Datalogger Firmware To Interface To The Calibrator
5SYSTEM INTEGRATION
- The Entire Data Collection System Is Closely
Integrated - The Datalogger And Automatic Gas Calibrator
Communicate With Each Other Via A Serial
Interface - The Interface Was Completely Defined By IPS
- This Interface Results In Preliminary Data
Flagging At The Monitoring Site Based On What The
Calibrator Is Doing - The MeteoStar LEADS Software That Collects Data
From The Datalogger Uses The Native Datalogger
Computer-To-Computer Language (cc-sail) - The Datalogger Keeps Track Of Which Data Records
Have Been Retrieved - Simplifies Retrieval Software
- Retrieval Software Queries Datalogger For Latest
Unsent Data Records - Data Collection Is Very Forgiving Of Missed Phone
Calls, Busy Signals, Network Outages, Etc.
6POLLUTION DATA COLLECTION
- LEADS Uses A Multi-Tier Data Collection System
- Reduces Communications Load On Any Single
Computer - Able To Rapidly Reconfigure the Communications
Network In The Event Of A Hardware Failure At A
Monitoring Site Or One Of The Regional Hub
Computers - Multi-Stage Archival Of Raw Measurements
- Possible To Retrieve Data From Any Of The Archive
Locations And Reintroduce It Into The System - The Datalogger Installed At Each Site Is Capable
Of Storing One To Three Weeks Of Data Based On
Number Of Instruments - Regional Hub Computers Are Located Across The
State That Communicate With The Dataloggers And
Retrieve Data From Them - The Regional Hub Computers Forward Collected Data
To Austin - The Hub Computers Store Three Months Of Data
- Once Data Arrives In Austin, It Is Decoded,
Checked For Errors And Stored In A Database - An
Independent Archive Is Also Created
7INSIDE A MONITORING SITE
NOx Analyzer
CO Analyzer
Data Logger/Modem
Calibration Valves
SO2 Analyzer
Ozone Analyzer
Gas Calibrator
Clean Air Supply
8AMBIENT AIR MONITORING SITE
9WATER MONITORING SITE
10TCEQ REGIONS
- TCEQ Divides Texas Into Sixteen Regions
- Regions Maintain And Operate Monitoring Sites
- TCEQ Collects Data From Fifteen Of The Sixteen
Regions - Data Collection System Based Around These Regions
- Communication Processing Is Distributed Across
The State - Green Highlighted Regions Have Hub Computers That
Service Monitoring Sites - Yellow Highlighted Regions Have Active Monitoring
Sites But No Hub Computer - Sites In Regions Without Hub Computers Are
Serviced From Other Regions
11MONITORING SITES
- There Are 177 Active Monitoring Sites - Including
1 Site In Indiana And 2 In Juarez - Sites Measure A Wide Variety Of Parameters
- Local Meteorological Data
- Air Pollutants
- Water Parameters
- Anything Else That Can Be Connected To The
Datalogger - Specific Parameters Measured At A Site Are
Determined By - Site Location
- What Is Located Nearby
- What Is Measured At Other Sites In The Area
- Public Concerns
- Federal Requirements
4
1
33
1
5
2
19
2
18
12
47
12
9
5
6
12GETTING DATA FROM THE SITES
- Monitoring Sites Serviced By Regional Hub
Computers - Communications Is A Balance Of Cost And Data
Retrieval - TCEQ Collects Data From Every Site In The System
Every 15 Minutes - That Means 96 Connections To Every Site Each Day
- Phone Lines And Modems Are The Least Expensive
- Used When Local (Non-Toll) Calls Can Be Made From
The Hub Computer To The Site - Major Metropolitan Areas Have Extended Dialing
Areas - Wireless Service Is Nice, But Limited To
Metropolitan Areas - Costs More Than Phone Lines But Less Than
Satellite Service - Satellite Service Can Be Established Anywhere
- Data Is Generally Available Within 20 - 22
Minutes From The Time It Is Collected - Data Available To The Public Within An Hour
- Public Data Has Gone Through Automatic QA/QC
Checks, But May Change Later Based On Human Review
13LEADS QA/QC PHILOSOPHY
- Standardize Data Handling
- Insure Consistency Across The Entire Network
- Minimize Subjective Data Quality Judgements
- Automate As Much As Possible
- Instrument Calibrations
- Periodic Instrument Challenges
- Data Quality Flagging
- Rigorously Check The Instruments In Each Station
And The Calibration System - Perform Automatic Instrument Challenges On A
Schedule - Augment With Periodic Station Visits To Perform
Independent Audits - Provide A Set Of Tools For Operators, Validators,
And Other Data Customers To View Not Only The
Data But QA/QC Information And System Performance
As Well
14LEADS QA/QC IMPLEMENTATION
- TCEQ, IPS, Coastal (Datalogger), And Dasibi (Gas
Calibrator) Developed Interfaces Between The
Various Components And Software - The Gas Calibrator Communicates Its Status To The
Datalogger Which In Turn Sets The Initial Data
Quality Flag - IPS-Developed Software Examines The Flags Coming
From The Datalogger And Determines Whether Or Not
To Interpret The Data As Ambient Or Some Type Of
Calibration Or Instrument Challenge - TCEQ Specified The Methods For Challenging The
Various Instruments - Five-Point Calibration
- Three-Point Span Check
- Span-Zero Check
- Limit Checks
- TCEQ Modified Their Existing QAPP To Incorporate
LEADS
15OTHER QA/QC CONSIDERATIONS
- The Introduction Of LEADS Into The TCEQ System
Did Not Affect Some Things - Documentation Of Procedures And Processes
- Traceability Of Measurements Back To A Basic
Standard - Independent Instrument Calibration And
Certification - Independent Station Audits
- Instrument Maintenance And Upkeep Procedures
- Major Impacts Of LEADS
- Ability To Expand Monitoring Network Without
Increase In Manpower - Higher Data Return
- High Confidence Level In Quality Of Data
- Ability To Rapidly Furnish Data To The General
Public
16QUALITY CHECKS
- There Are Several Types Of Automatic Quality
Checks - Calibration Sequences Are Run Periodically Or
When Equipment Is Changed To Establish A New
Slope And Intercept - Span Check Sequences Are Run Weekly To Ascertain
Whether Or Not A Particular Instrument Is
Drifting Or Malfunctioning - May Also Set A New
Intercept - Span-Zero Check Sequences Are Run Nightly To
Ascertain Whether Or Not A Particular Instrument
Is Drifting Or Malfunctioning - Any Of These Sequences Can Be Automatically
Scheduled On Either The Dasibi Calibrator Or The
Zeno Datalogger - Any Of These Sequences Can Be Manually Initiated
As Necessary
17AUTOMATIC TESTS
18OTHER DATA CHECKS
- Automatic Met Data Checks
- Optional Upper And Lower Limits
- Optional NEG Test
- Optional MUL Test
19OPERATIONAL DOCUMENTATION
- There Is An Electronic Operator Log On Each
Datalogger - Web Interface For Viewing Operator Logs
- Can Also Add Log Entries Via The Web
- Certain Web Pages Also Trigger Automatic Operator
Log Entries - Any Change Made By A Data Validator Results In An
Automatic Validator Note Entry - Web Interface For Viewing Validator Notes
- Certain Web Pages Also Trigger Automatic
Validator Note Entries - There Is A Web-Based Hardware Tracker That Can Be
Used To Record And Report Instrument Changes At
Each Site - There Is A Web-Based Contact Manager That Can Be
Used To Record Who Operates Sites And/Or
Particular Instruments As Well As Who Validates
The Data - Web-Based Data Reports
20DATA DISPLAY
21WEB INTERFACE
- Web Pages Written In Perl And Most Are Dynamic
- Web Pages Run Off A Combination Of Derived Files
And The Master Pollution Database - Derived Files Automatically Updated By Background
Processes - Derived Files Generally Contain Data Averages Or
Information That Is Not Stored In The LEADS
Pollution Database - Some Derived Files May Take Hours To Update - The
Derived Files Speed Delivery Of Information To
Web Users - Many Of The Web Pages Display System Health And
Status Information - Public URL
- http//www.tnrcc.state.tx.us/air/monops
- Internal URL
- http//dsr.tnrcc.state.tx.us
- The Web Pages Grow And Expand Based On User Input
And Feedback
22EXTERNAL WEB INTERFACE
23INTERNAL WEB INTERFACE
24POLLUTION RETRIEVAL/DISPLAY
- Also Known As Manual Validation
- Lets You Examine 5-Minute Data As Well As Hourly
Averages - Data Graphically Displayed
- Up To One Month Of Data Can Be Displayed
- Data Validators Use This Interface To Edit Data
- Requires Specific Privileges In Order To Make Any
Changes To Data - Can Change Data Flags
- Can Change Slope And Intercept
- Wind And Pollution Roses Included In This
Interface
25MANUAL VALIDATION WINDOW
Pull-Down Menus
Activate Panel
Horizontal Scroll Controls
Vertical Scroll Controls
Time Zone Select
Status Information
26STATUS INFORMATION
- Information About Each Data Point Is Displayed At
The Bottom Of The Window - The Information Displayed Is Dynamic And Is Based
On The Data Point Located Under The Cursor - Available Information Includes
- Date And Time Sample Was Collected
- Sample Value In Engineering Units
- EPA Data Flag
- The Slope And Intercept Applied To The Measured
Voltages - Whether Or Not The Data Point Has Been Validated
(Two Indicators) - Whether Or Not There Are Any Validator Notes
Attached To The Data - There Are Also Data Entry Areas For Manually
Entered Slopes And Intercepts
27WHATS IN THE DISPLAY
Site ID
Missing Data
Parameter Name
5-Point Cal
3-Point Span
Data Curve
Validated Bar
28ROSE TYPES
- There Are Four Types Of Roses Available
- Each Of The Roses Summarizes Data Over The Time
Span You Select - Data Is Accumulated Into Each Of The Direction
Bins - Wind Rose
- This Is A Standard Wind Rose Plot Which
Illustrates Wind Speed As A Function Of Direction - Average Concentration Rose
- This Pollution Rose Will Display The Average
Concentration Of A Particular Pollutant As A
Function Of Wind Direction - Maximum Concentration Rose
- This Pollution Rose Will Display The Maximum
Concentration Of A Particular Pollutant As A
Function Of Wind Direction - Percentile Rose
- This Pollution Rose Allows You To Filter
Pollution Concentrations In Order To Reject
Abnormally High Or Low Readings - The Smaller The
Percentile You Select, The Less Data Is Included
In The Pollution Rose - Selecting A Percentile Of 100 Is The Same As
Selecting A Maximum Concentration Rose
29WIND ROSE
30POLLUTION ROSE
31COMMUNICATIONS OPTIONS
32COMMUNICATION OPTIONS -MODEM-TO-MODEM
RS-232 Serial I/F
RS-232 Serial I/F
Phone Company
Datalogger
Modem
Modem
Hub Computer
- IPS-Developed Software On Hub Computer Initiates
Phone Calls - Control Software Written In Perl
- Comms Software Written In C
- Comms Software Driven By Configuration Files
- Phone Number Of Site
- Number Of Records Requested
- Comms Software Communicates Directly With
Datalogger - Each Hub Computer Services Multiple Monitoring
Sites
- Datalogger Comes With A Computer-To-Computer
Communications Interface - Maintained By Datalogger Vendor
- Hub Comms Software Written To Use This Interface
- Datalogger Keeps Track Of Which Records Have Been
Previously Sent To The Hub Computer
33COMMUNICATION OPTIONS -WIRELESS HANDHELD PC
Phone Company Hub
Ethernet
Hub Computer
- IPS-Developed Software On Hub Computer Initiates
TCP/IP Network Connection To Handheld PC - Hub Software Communicates Directly With
Datalogger Over Network Connection - Hub Computer Can Use Modems To Contact Other
Sites - CDPD Availability Limited To Larger Metropolitan
Areas
- Handheld PC Runs Windows CE
- Connection To Cellular Network Via Wireless Air
Card - Type II PCMCIA Card
- CDPD Protocol (Cellular Digital Packet Data)
- IPS-Developed Software On Handheld PC Establishes
Network Connection To Hub Computer - Can Also Use A Regular PC Or Laptop Instead Of
Handheld
34COMMUNICATION OPTIONS -Satellite
Provider Hub
RG-59 Cable
RS-232 Serial I/F
Ethernet
Ethernet
Internet
Hub Computer
Datalogger
PC
Satellite Modem
- IPS-Developed Software On PC Establishes A
Network Connection To Hub Computer - Approximate 6 Second Time Lag For Satellite
Transmission
- IPS-Developed Software On Hub Computer Initiates
TCP/IP Network Connection To Monitoring Site PC - Functionally Identical To Wireless Solution
35COMMUNICATION OPTIONS -CELLULAR MODEM
RS-232 Serial I/F
RS-232 Serial I/F
Phone Company
Modem
Hub Computer
Datalogger
Cellular Modem
- IPS-Developed Software On Hub Computer Initiates
Phone Call To Cellular Modem - Functionally Identical To Modem-To-Modem Solution
- Still In Test Phase
- Cellular Coverage Limits Usability
- Service Providers Trying To Phase Out This
Service In Favor Of CDPD And Other Protocols
36COMMUNICATION OPTIONS -PACKET RADIO MODEM
RS-232 Serial I/F
RS-232 Serial I/F
Packet Radio Modem
Datalogger
Hub Computer
Packet Radio Modem
- IPS-Developed Software On Hub Computer Triggers
Packet Radio Modem To Initiate A Connection - Once The Connection Is Made, This Is Functionally
Identical To The Modem-To-Modem Solution - Still In Test Phase
- Packet Radio Modem Operating At 5 Watts Output
- Designed For Line-Of-Sight Installations
- Highly Dependent Upon Terrain And Other Obstacles
- First Installations Will Be Approximately 5 Miles
- Other Configurations Can Use Repeaters Or Relays
To Extend Distance - Increased Radio Power Will Also Extend Distance
37AUTOMATIC QUALITY CHECKS
38QC SEQUENCES
- Sequences Are Composed Of Levels
- A Level Consists Of A Set Concentration From The
DASIBI 5008 Calibrator Introduced Into A Monitor
For A Set Number Of 5-Minute Sample Periods - Each Level Is Assigned A Letter Code (M, R, S, T,
Or G) By The Datalogger -- These Correspond To
Various Percentages Of The Instrument Full-Scale - A Set Number Of 5-Minute Samples (Usually 1 - 3)
In Each Level Are Allowed For Instrument
Stabilization - The Remaining Samples (Usually 3 - 4) Are
Processed By LEADS.
39FIVE-POINT CALIBRATION
40THREE-POINT SPAN CHECK
41TWO-POINT SPAN-ZERO CHECK
42POSSIBLE QC CHECK FLAGS
- Incomplete
- The Data Did Not Arrive In The Proper Sequence Or
There Were Either Too Few Or Too Many Samples At
Each Level - Invalid
- The Calibrator Did Not Deliver The Challenge Gas
At Correctly Spaced Intervals - Failed
- One Or More Of The Automatic QC Checks Exceeded A
Pre-Set Failure Limit -- Data Is Automatically
Rejected Accordingly - Warning
- One Or More Of The Automatic QC Checks Exceeded A
Pre-Set Warning Limit -- The Calibration Or Span
Check Is Still Valid And Is Still Used - Passed
- All Automatic QC Checks Were Passed
43AUTOMATIC TESTS
- Completeness Test
- Tests To See That Data Arrives In Correct Order
And The Correct Number Of Updates For Each Level
Are Present - Voltage Outlier
- Tests Are Based On The Measured Voltages From The
Instrument - Tests To See That No Single Voltage Measurement
For A Level Is More Than An Allowed Deviation
From The Average Voltage - Concentration Outlier
- Tests To See That No Single Concentration
Measurement For A Level Is More Than An Allowed
Deviation From The Average Concentration - This Test Was Specifically Designed To Catch
Calibration System Problems
44AUTOMATIC TESTS (Cont.)
- NO2 Concentration Outlier
- Based Entirely On NO Channel Performance On
Previous Non-Titration Level (M-Level) And
Current Titration Level (M-Level) -- NO2
Voltages Are Not Part Of This Test - Tests To See That No Single Concentration
Measurement For A Level Is More Than An Allowed
Deviation From The Average Concentration - Concentration Spacing
- Tests To See That The Calibrator Is Delivering
Evenly Spaced Concentrations Throughout The
Instruments Range - No Concentration For A Level Can Be More Than An
Allowed Deviation From The Ideal Concentration
45AUTOMATIC TESTS (Cont.)
- NO2 Concentration Spacing
- Based Entirely On NO Channel Performance On
Previous Non-Titration Level (M-Level) And
Current Titration Level (M-Level) -- Differs
From Normal Concentration Spacing Test - No Concentration For A Level Can Be More Than An
Allowed Deviation From The Ideal Concentration - Slope
- Tests To See That The Instrument Is Responding
Correctly. All Instrument Responses Should Be
Linear And The Calculated Slope Should Be Near
The Ideal - Ideal Slopes
- CO 20
- Other Pollutants 1000 (0 - 1000 ppb Instrument
Range) - Other Pollutants 2000 (0 - 500 ppb Instrument
Range)
46AUTOMATIC TESTS (Cont.)
- Intercept
- Tests To See That The Instrument Is Responding
Correctly And That The Calculated Intercept Is
Near The Ideal - Ideal Intercept 0
- Precision
- Tests To See That The Instrument Has A Stable,
Linear Response - The Voltage Average For Each Level Is Compared To
The Calculated Regression Line And Must Be Within
Limits - Linearity
- Tests To See That The Instrument Is Holding A
Stable, Linear Response - The Voltage Average For The T-Level Is Compared
To The Line Drawn Between The M-Level And The
G-Level And Must Be Within Limits
47AUTOMATIC TESTS (Cont.)
- Zero
- Tests To See That The Instrument Calibration Is
Holding Since The Last Good Calibration Or Span - The Voltage Average For The G-Level Is Compared
To The G-Level From The Last Good Calibration Or
Span And Must Be Within Limits - Span
- Tests To See That The Instrument Calibration Is
Holding Since The Last Good Calibration Or Span - The Voltage Average For The M-Level Is Compared
To The M-Level From The Last Good Calibration Or
Span And Must Be Within Limits
48AUTOMATIC TESTS (Cont.)
- NOx Balance Test
- Tests The Electronic And Flow Balance Between The
NO and NOx Channels. Compares The Ratio Of The
Average NO2 Span Response To The Average NO Span
Response During Non-Titration Steps. Sensitive
To NO2 Impurities In NO Cylinder. Impurities
Give Positive Errors. - Only Affects The Validity Of The NO2 Channel
Calibration - NO2 Converter Efficiency Test
- Tests The Efficiency Of the NO2 ? NO Converter
- Affects The Validity Of The NO2 And NOx Channel
Calibrations - H2S Converter Efficiency Test
- Tests The Efficiency Of the H2S ? SO2 Converter
- Based On The T-Level And The T1-Level. Test Is
Sensitive To Agreement Of The Calibrations Of The
H2S Cylinder To The SO2 Cylinder.
49AUTOMATIC TESTS (Cont.)
- SO2 Scrubber Efficiency Test
- Tests The Efficiency Of the SO2 Scrubber
50TEST LIMITS
- All QC Tests Performed On Calibration,Span Check,
Or Span-Zero Data Have Both Warning And Failure
Limits Except The Outlier Tests - Each Outlier Test Uses Only One Limit But The
Test Is Repeated If An Outlier Is Detected. - Each Warning Limit Is Chosen Statistically To
Represent The 3rd Standard Deviation Value About
The Mean Error Of A Test - There Should Be Only A 0.27 Probability Of
Exceeding A Warning Limit If The Monitoring
System Is Working Properly - Each Failure Limit Is Intended To Represent The
Maximum Error That Will Be Tolerated Without
Invalidation Of The Affected Data
51DATA VALIDATION RULES
- Automatic Invalidation Of Data Is Based Only On
Whether A QC Test Passes Or Fails - Warnings Are Not Considered In This Processing
- Failure Of A Concentration Outlier Test Or A
Concentration Spacing Test Indicates A Problem
With The Calibration System But Not With The
Monitor (Except NO2) - The Calibration Or Span Check Event Involved Is
Considered Invalid And The Ambient Pollution Data
Is Unaffected - In The Event Of A Test Failure, Data Is
Automatically Rejected (Flagged LIM) - To Stop Rejecting Data Forward, A Successful
Five-Point Calibration Must Be Run - Data Is Rejected Back To The Last Successful
Calibration, Span Check, Or Span-Zero Check
52SPAN SOURCE AUDITS
- Automatic Quality Control Is Augmented At TCEQ By
Quarterly Manual Span Source Audits - Span Source Audits Evaluate The Accuracy Of The
CAMS DASIBI 5008 Calibrator - Corrective Action Is Required If Audit Limits Are
Exceeded - If the Audit Passes, Then the Pollutant Monitor
Should Be Adjusted to Agree With the M-level
Concentration Produced by the CAMS DASIBI
Calibrator - This Sets the Slope of the Monitors Response to
the Ideal - If a Slope Test Warning Is Reported Thereafter,
Then Action Must Be Taken to Determine If the
Error Was Caused by Monitor Drift or Span Source
Drift - A Span Source Audit May Be Needed If There Are No
Obvious Instrument Problems
53OTHER DATA CHECKS
54UPPER AND LOWER LIMITS
- Limits Can Be Defined For A Specific Instrument
At A Site - Any Parameter Can Have Limit Checks Defined
- Can Define Either An Upper Or Lower Limit Or Both
- System Automatically Flags Any Data That Falls
Outside Specified Limits As LIM - LIM Data Is Automatically Excluded From Averaging
And Automatic Alerts
55AUTOMATIC MET CHECKS
56NEG AND MUL TESTS
- Only Performed On Hourly Averages - Does Not
Affect Five-Minute Averages - NEG Test Can Be Defined For Any Parameter That Is
Rolled Up Into An Hourly Average - Test Will Automatically Set Slightly Negative
Averages To 0.0 - If Data Is Too Negative, It Will Be Flagged NEG
- Data Flagged NEG Is Automatically Excluded By
Other Displays And Analysis Algorithms - MUL Test Designed Specifically For NOx Family
- Tests To Make Sure That Sum Of NO And NO2
Measurements Are Consistent With NOx Measurement - If Data Is Not Consistent, All Three Parameters
Will Be Flagged MUL - Data Flagged MUL Is Automatically Excluded By
Other Displays And Analysis Algorithms