Clarus Weather System Design

1
Clarus Weather System Design

• ICC Meeting 4
• August 8, 2006

www.clarusinitiative.org
2
Proof-of-Concept Demonstration

• Clarus System Status
• Development Process How We Got Here
• System Implementation
• System Testing
• System Deployment
• Clarus System How It Works
• Getting Observations
• Quality Checking
• Dissemination
• Lessons Learned
• Next Steps

3
Clarus System Status

• Base POC coding complete
• System collecting observations
• From POC states (Alaska, Minnesota, Utah)
• From ASOS (as background for spatial test)
• Quality checks being performed
• Observations available
• System tests ongoing

4
Implementation

• Coding standards emphasize stability,
  portability, extensibility
• Coded in C, Java, and PHP
• Using only open source libraries and
  Clarus-specific code
• Extensive comments and logging
• Compiles and runs in multiple operating systems

5
Implementation, contd

• Implementation methods balance performance and
  flexibility
• Metadata distributed
• MySQL database used for contributor, station, and
  sensor metadata
• Configuration files used for system metadata
• Component services distributed to multiple servers

6
Clarus Server Configuration
7
Testing

• Code review completed
• Unit/component testing complete
• System testing ongoing
• Testing within design team started
• Ready for POC state evaluations

8
Deployment

• Getting metadata is a challenge
• System deployed to servers in Mixon/Hill
  facilities
• Observations being collected from POC states and
  NWS sources
• All required quality checks running
• Observations available in Web browser as text
  report and on example map

9
Getting Observations

• Collector services are specific to each POC state
  system
• Observation file naming schemes differ
• File content templates differ
• Observation types in files differ
• Observation units and time specs differ
• All ASOS obs are collected (as back-ground for
  spatial quality checking)

10
Getting Observations, contd

• System will try to re-collect files missed due to
  communication or server errors
• Collected obs are converted to consistent format
  and units
• Individual observations are stored by type,
  location, and time

11
Quality Checking

• Quality checking services (QChS) are notified
  when new observations are available
• Sequence of QChS to be run are specific to obs
  type
• Results of checks are stored with each observation

12
Quality Checking Configuration
13
Dissemination

• Observations are available for dissemination as
  soon as cached
• Results requested through HTTP request
• In a Web browser
• From another application
• Results returned as comma-delimited text

14
Example Observation Results
http//clarus.mixonhill.com/Clarus/QEDCObsMgr2.jsp
? obs-type5733start-time200608020000end-time
200608040000 start-lat64841930end-lat58324410
start-lon-151658900end-lon-134457680
ObsTypeID,ObsTypeName,ClarusSensorID,ClarusStation
ID,ClarusContribID,Contributor,StationCode,Timesta
mp,Latitude,Longitude,Elevation,Observation(C),Qch
sSequenceComplete,QchsManualFlag,QchsServiceSensor
Range,QchsServiceClimateRange,QchsServiceStep,Qchs
ServiceLike,QchsServicePersist,QchsServiceBarnes,Q
chsServiceDewpoint,X,X,X,X,X,X,X,X,X,X,X,X,X,X,X,X
,X,X,X,X,X,X,X,ConfValue 5733,essAirTemperature,6
7,610,1,Alaska Dept. of Transportation and Public
Faciliti,351-45,8/2/06 61600 PM
UTC,60.532087,-149.55574,209000000,10.399999767541
885,P,-,P,P,N,N,N,N,N,-,-,-,-,-,-,-,-,-,-,-,-,-,-,
-,-,-,-,-,-,-,-,-,0.24843171 5733,essAirTemperatu
re,568,589,1,Alaska Dept. of Transportation and
Public Faciliti,351-17,8/2/06 73500 PM
UTC,59.424995,-136.10398,50000000,14.2999996803700
92,P,-,P,P,N,N,N,N,N,-,-,-,-,-,-,-,-,-,-,-,-,-,-,-
,-,-,-,-,-,-,-,-,0.2613462 5733,essAirTemperature
,424,602,1,Alaska Dept. of Transportation and
Public Faciliti,351-30,8/2/06 73800 PM
UTC,59.422258,-136.05841,67000000,13.4999996982514
86,P,-,P,P,N,N,N,N,N,-,-,-,-,-,-,-,-,-,-,-,-,-,-,-
,-,-,-,-,-,-,-,-,0.25753936 5733,essAirTemperatur
e,514,592,1,Alaska Dept. of Transportation and
Public Faciliti,351-20,8/2/06 74000 PM
UTC,61.113087,-145.80284,282000000,13.099999707192
183,P,-,P,P,N,N,N,N,N,-,-,-,-,-,-,-,-,-,-,-,-,-,-,
-,-,-,-,-,-,-,-,-,0.25585115 5733,essAirTemperatu
re,20,579,1,Alaska Dept. of Transportation and
Public Faciliti,351-7,8/2/06 74100 PM
UTC,61.49387,-149.24223,23000000,15.29999965801835
,P,-,P,P,N,N,N,N,N,-,-,-,-,-,-,-,-,-,-,-,-,-,-,-,-
,-,-,-,-,-,-,-,0.26687917 5733,essAirTemperature,
328,598,1,Alaska Dept. of Transportation and
Public Faciliti,351-26,8/2/06 74400 PM
UTC,61.129859,-145.73386,884000000,7.8999998234212
4,P,-,P,P,N,N,N,P,N,-,-,-,-,-,-,-,-,-,-,-,-,-,-,-,
-,-,-,-,-,-,-,-,0.2480511 5733,essAirTemperature,
223,586,1,Alaska Dept. of Transportation and
Public Faciliti,351-14,8/2/06 74500 PM
UTC,58.331906,-134.48244,5000000,11.09999975189566
6,P,-,P,P,N,N,N,N,N,-,-,-,-,-,-,-,-,-,-,-,-,-,-,-,
-,-,-,-,-,-,-,-,0.24967098 5733,essAirTemperature
,595,590,1,Alaska Dept. of Transportation and
Public Faciliti,351-18,8/2/06 74500 PM
UTC,61.833652,-147.4085,983000000,12.4999997206032
28,P,-,P,P,N,N,N,P,N,-,-,-,-,-,-,-,-,-,-,-,-,-,-,-
,-,-,-,-,-,-,-,-,0.25710365 5733,essAirTemperatur
e,618,588,1,Alaska Dept. of Transportation and
Public Faciliti,351-16,8/2/06 74700 PM
UTC,58.39726,-134.60823,65000000,12.49999972060322
8,P,-,P,P,N,N,N,N,N,-,-,-,-,-,-,-,-,-,-,-,-,-,-,-,
-,-,-,-,-,-,-,-,0.25359595 5733,essAirTemperature
,541,593,1,Alaska Dept. of Transportation and
Public Faciliti,351-21,8/2/06 74800 PM
UTC,61.819105,-145.21614,447000000,14.399999678134
918,P,-,P,P,N,N,N,N,N,-,-,-,-,-,-,-,-,-,-,-,-,-,-,
-,-,-,-,-,-,-,-,-,0.26186147 5733,essAirTemperatu
re,279,606,1,Alaska Dept. of Transportation and
Public Faciliti,351-34,8/2/06 74900 PM
UTC,58.327675,-134.49118,27000000,11.0999997518956
66,P,-,P,P,N,N,N,N,N,-,-,-,-,-,-,-,-,-,-,-,-,-,-,-
,-,-,-,-,-,-,-,-,0.24967098
15
Example Query Web Page
16
Dissemination, contd

• Example map interface available through Web
  browser

17
Example Map Presentation
18
Dissemination, contd

• Metadata reports available for

• Climate
• Climate Records
• Contacts
• Contributors
• Images
• Observation Types

• Organizations
• Sensors
• Sensor Types
• Sites
• Stations

19
Example Metadata Results
20
Lessons Learned (So Far)

• The system is only as accurate and complete as
  the metadata
• Location data
• Sensor types
• Interface standards, consistently applied,
  greatly reduce complexity and cost

21
Next Steps

• Complete system testing
• Get POC state feedback
• Complete interim design documents and
  operating/maintenance instructions
• Get operating experience (with POC states)
• Document/correct deficiencies
• Prepare final documentation

22
Project Information

• Project Contact Information
• Brenda.Boyce_at_mixonhill.com
• J.Kyle.Garrett_at_mixonhill.com
• U.S. DOT Contact Information
• Paul.Pisano_at_dot.gov
• James.Pol_at_dot.gov
• Clarus website address
• www.clarusinitiative.org