Firefighter Outreach, Testing, Validation

1
Firefighter Outreach, Testing, Validation

• Goal Maintain critical contacts with the
  firefighter and first responder community in
  order to guide, test, and validate our technology
  development efforts.

2
Firefighter Outreach, Testing, Validation

• Strategies
• Firefighter forums
• Joint exercises
• Distinguished speaker lectures
• Technology prototyping
• Data / Information sharing
•  After Action Reviews
• Studies / Experiments

3
Firefighter Outreach, Testing, Validation Overview

• Joint Drills / Technology Testing Exercises -
  Impact Analysis
• Live Burn CO Sensing Study OCFA, LA County
  Fire (23 FEB 09)
• HazMat, casualties, First Response drill and
  SAFIRE Deployment (16 SEP 08)
• HazMat drill (with multiple casualties) and SA
  Study (12 MAY 09)
• Tabletop exercise IC Usability Experiment (15 MAY
  09)

4
Firefighter Outreach, Testing, Validation Overview

• CERT sponsored Firefighter forums 
• Quarterly meetings 
• Roundtables, distinguished speakers, FF
  presentations
• One-on-one discussions, meeting, visits
• LA County Fire CO sensing
• Joint technology development
• Technology Prototyping
• Feedback on SAFIRE technologies

5
Technology Evaluation, Analysis, Validation

• Assessing effectiveness of developed technologies
  in improving FF safety is key component of project

5
6
Technology Evaluation and Analysis

• Responsphere Drills
• First HazMat drill (17 JUL 08)
• Second HazMat drill (12 MAY 09)
• Live Burn exercise (23 FEB 09)
• Immediate feedback on technology
• After-action reviews
• Occur after each exercise
• Impact Analysis
• - utilization experiment, tabletop exercise (15
  MAY 09)
• Reports on testing outcomes
• Data sharing
• CO Data available for Jeff Burgess (Arizona)

7
Technology Validation SAFIRE Studies

• SAFIRE / FICB Usability Study 15 MAY 09
• Goal To test the usability of SAFIRE technology
  for Incident Commanders and receive input on the
  technology.
• Methodology Mixed-methods (quantitative and
  qualitative data) design
• SAFIRE evaluated by panel of IC experts at FF
  forum Tabletop Exercise
• Drill Scenario stopped at 5 freeze points to
  assess 1) Usability, 2) Impact on decision-making

8
Technology Validation SAFIRE Studies

• SAFIRE / FICB Usability Study 15 MAY 09
• Freeze points were identified as critical
  junction / decision points
• IC Setup
• Spill location Efforts
• Information Traffic Flows
• Search and Rescue
• Localization of firefighter personnel

PRE-FICB INTERVENTION Given the capabilities of
the FICB technology, would you use it at this
freeze point?   YES /
NO   Optional Elaboration If yes, how? If no,
why?       POST-FICB INTERVENTION Would having
this capability and information from FICB assist
your decision on what to do next?   YES
/ NO   Optional Elaboration If yes,
how? If no, why?
9
Technology Validation SAFIRE Studies

• SAFIRE / FICB Usability Study 15 MAY 09

• Results
• Usability and decision-making impact
  significantly correlated with SAFIRE technology
  among ICs.
• Qualitative feedback overwhelmingly positive.
  Also, many suggestions for improvement.

10
Technology Validation SAFIRE Studies

• Situational Awareness (SAFIRE FICB) 12 MAY 09
• Goal To test the impact of SAFIRE technology on
  Situational Awareness during the Responsphere
  HazMat drill.
• Methodology Traditional Experimental Design
• Control Group Incident Commander without FICB
• Experimental Group Incident Commander with FICB
• Drill Scenario stopped at 6 freeze points to
  assess Situational Awareness

11
Technology Validation SAFIRE Studies

• Situational Awareness (SAFIRE FICB) 12 MAY 09
• Freeze points were identified as critical
  junction / decision points
• Initial Baseline (no FICB)
• IC Setup
• Spill location Efforts
• Information Traffic Flows
• Search and Rescue
• Localization of firefighter personnel
• At each freeze point, ICs were surveyed on their
  perception, comprehension, and projection
  (Endsley model of SA).

12
Technology Validation SAFIRE Studies

• Situational Awareness (SAFIRE FICB) 12 MAY 09

• Results
• Both ICs attained 100 comprehension. FICB
  reached this point earlier.
• FICB IC consistently higher in SA scores.
• SA with FICB 49
• SA without FICB 32 (was indirectly influenced
  by FICB by virtue of FF actions)

13
Technology Validation SAFIRE Studies

• Carboxyhemoglobin and CO Sensors 23 FEB 09
• Goal To test the efficacy and accuracy of
  commercial SpCO sensors and calibrate
  SAFIRE-created CO sensors in a live burn,
  field-deployment scenario.
• R

Event Timetable 957 -1021  -
Instrumentation/ calibration 1021- 1047
Burn 1047 1146 Break 1146- 1201
2nd Burn

• Results
• FF Movement (e.g., hand, arm motion) caused
  spikes in SpCO readings and instrumentation read
  errors by misaligning the sensor strip.
• Environmental factors (heat, smoke, etc.) caused
  no malfunctions.
• 6,929 data points obtained in order to calibrate
  SAFIRE CO sensors.