Evolving from WAS to IS: Societal Impacts in NWS Service Assessments

1
Evolving from WAS to IS Societal Impacts in
NWS Service Assessments
Kevin Barjenbruch, NWS, Salt Lake City, UT Daniel
Nietfeld, NWS, Omaha, NE Julie Demuth, NCAR
Societal Impacts Program
2008 Summer WASIS Workshop August 9, 2008
2
February 5-6, 2008, tornado outbreak

• 82 tornadoes, 5 rated a 4 on Enhanced Fujita
  Tornado Scale
• 57 fatalities most since May 31, 1985, and 13th
  of all time
• 350 injuries
• Over 400M in property damage

Dubbed Super Tuesday Tornado Outbreak
3
NWS Service Assessments

• Overarching Goal/Activation
• Conducted to evaluate NWS performance during
  significant (high-impact) events
• Major economic impacts
• Multiple fatalities or numerous serious injuries
• Unusually high public or media interest
• Usually convened just once or twice a year

4
NWS Service Assessments

• Team Composition
• Experts from both inside and outside the National
  Weather Service
• Process
• Study the event itself and as well as actions
  before, during, and immediately after
• Work with designated officials at National
  Weather Service Headquarters to author report
• Recommend changes in NWS procedures, products,
  and services to improve future performance
• Briefing provided to NWS Corporate Board
• Service Assessment posted for public consumption

5
NWS Service Assessments
Service assessments moving from WAS

• Inward Focus on NWS Procedures, actions, and
  equipment through
• Internal review of operations in National
  Centers, Weather Forecast Offices, Center Weather
  Service Units, and River Forecast Centers)
• External assessment with various customers of the
  information Trending toward IS!
• Emphasis on quantitative assessment
• Fatalities and injuries
• Damage
• Verification, etc.

6
NWS Service Assessments
To IS

• 2008 assessments have included a Societal Impacts
  Analysis Component
• Super Tuesday Tornado Outbreak
• Mothers Day Weekend Tornadoes in Oklahoma and
  Missouri, May 10, 2008
• Midwest Floods of June 2008
• Increased emphasis on qualitative assessment
• Actions taken
• Changes in information delivery, etc.

7
NWS Service Assessments
Desired outcomes

• Utilize lessons learned, best practices, and
  analysis of customers/partners of weather
  information to
• Improve delivery of hazard information (format
  and content) to our customers and partners
• Improve clarity of hazard information
• Focus research and training
• Allocate resources

Provide better and more understandable weather
information so that people will take action to
protect life and property!
8
Societal Impacts in Super Tuesday
9
Motivation

• This was a well-warned event with such good
  information
• .
• .
• .
• Why did so many people die?
• Why dont people do what theyre supposed to do
  to make the right decision?

We get frustrated when we put good weather
information out there and people dont make the
right decisions!
10
Societal impacts scope

• The task To try to understand why so many
  people died and the details of those fatalities
• E.g., age, gender, whether warning was heard,
  warning source, whether they heeded the warning,
  structure where they died, whether they sought
  shelter, whether safer shelter was available

11
Methodology

• Semi-structured interviews with the public
• Targeted, convenience, and snowball sampling
• 41 interviews in the 6 WFOs visited (assessment
  team broke into 3 sub-teams)
• Kevin and I did 17 public interviews over 4 days
  in the field, another day on the phone

12
Some of the questions

• When did you first realize there was a threat of
  a tornado in this area?
• How did you learn about the threat? (Sources,
  environmental cues)
• What were you thinking after you received that
  information? (Trust? Confusion? Uncertainty?
  Barriers to action?)
• What did you do next? (Confirmation?)
• Were there any unique circumstances about your
  day that affected your experience during the
  tornado event? What?
• Have you ever been in a similar type of
  extreme-weather situation in the past?
  (Experience, false alarms)
• Did anything from that experience influence what
  you did during this most recent event?
• Have you ever been warned about an extreme
  weather event in the past that did not occur?
• Think back over the entire tornado event, from
  the time you learned there was a tornado threat
  through when the tornado actually occurred.
• Is there any other information you would have
  liked to have had?

13
A Few Key Results

• Knowledge
• People get information from multiple sources
• Most commonly via television
• Also commonly from other people
• People get information multiple times
• Misconceptions about sirens

• Perceptions
• Integration of seasonality, weather salience,
  situational awareness about the event
• Personalization of the threat
• Seeking confirmation of the threat (e.g., Atkins
  woman, couple)
• Personal risk perception and optimism bias (e.g.,
  Hardin Co. family, Arkansas family)

14
A Few Key Results

• Decision-making and sheltering
• Decision-making is not a singular event it
  happens numerous times throughout the warning
  process
• Vast majority of people who received warning
  information sought shelter in best location
  available to them
• Most people heeded the warning and sought shelter
  in the best available location, but
• Most people did not have an underground shelter
  or safe room available to them

• Fatalities
• Collected as much good data as we could
• Nearly 2/3 of victims were in mobile homes

15
Reflections and Wisdom from a Veteran WASISer
16
WASIS Folded into Super Tuesday

• The value of WASIS
• Community to fall back on for support/knowledge
• Recognition that the NWS is not all that
• Through interaction at workshop
• Through partnership projects
• Through the assessment
• Some exposure to quantitative and qualitative
  research
• VisibilityOpportunity
• Coping with sleep deprivation

17
WASIS Folded into Super Tuesday

• The OMG now what!
• Surveys
• The art of interviewing
• Opportunity to observe and learn from Julie
• Exceldata analysis

18
Thoughts and Inspirations from a New WASISer
19
I was a ship without a sail
Adrift on the open sea of societal impacts

• Hurricane Charley Service Assessment 2004
• EM Interviews
• Media Interviews
• Hours and hours of transcribing
• Had a good teacher (Sociologist Betty Morrow)
• NO knowledge of good wine
• Super Tuesday Service Assessment 2008
• Learned from a great mentor (Julie and I
  suppose Kevin)
• Was inflicted by passion for doing this work
• Truly connected my tornado warnings with the
  impacts of them
• Became a layexpert in Italian Reds

20
Storm-Based Warnings

• Beneficial ?
• Confusing ?
• Media interpretation ?
• Implications to NWS ?

21
My Job

• Where should I focus my energy in the next 20
  years?
• How can I influence the forecasters in my office?
• The internal workings of the WFO should be driven
  by the external impacts of our actions
• How should my seminars be structured?
• When I collaborate with UNL and Creighton
  University, which research projects should I
  emphasize and pursue?

22
Little Sioux Boy Scout Camp

• June 11, 2008 EF3 Tornado
• 4 children killed
• 12 minute lead time
• Sirens sounded
• Sought shelter
• We now have a forecaster who wants to do a study
  on this event, from a societal impacts
  perspective
• I can at least now call Julie and say HELP!

23
My Other Job

• Teach Severe Storms at UNL
• Shared my Service Assessment experience with
  students
• Wow student interest!
• UNL Alumni Advisory Board
• Short Course?
• Curriculum?

24
June 3, 1980

• 28 years later
• Community rebuilding
• Economic Impact
• Psychological Impact
• Sheltering
• New perspective

25
Julies Reflections and Lessons Learned
26
The essentials

• Partnerships among social scientist, research
  meteorologists, operational meteorologist, policy
  makers, practitioners, etc.
• Kevin ? links operational meteorologists, users
• Julie ? links users, social science research
• Dan ? links research meteorologists, operational
  meteorologists
• The three of us are so much more effective
  together
• Interest and willingness to work together, to
  listen, learn, exchange ideas! To co-produce
  knowledge.
• Passion!

27

• 3 Huskers ISTJ ENFP INFJ LATE NIGHTS!
• But I function well (had some of my best thoughts
  even) at 630 a.m. after going to bed at 500
  a.m.!
• Even still, its important to know who else on
  the team needs Starbucks daily (or twice daily,
  or )
• Speaking of Starbucks, there are a lot of cops in
  Little Rock and theyll pull you over for
  almost running a red light.
• Speaking of Kevins driving, have a happy place
  that you can go to in an instant

Opportunities like this can change your life in
ways you never dreamed possible
28
Great strides in meteorology

• How has meteorology advanced in 15 years?
• How have warning and forecast operations advanced
  since 1993?
• Weve undergone a paradigm shift of warning
  services
• Extremely high expectations
• Relatively proud confidence in meeting them
• Will continue to advance and improve

But there is room for improvement ...
29
The next great strides

• To create good / effective products and
  services for users
• Provide people information that they actually
  want and use rather than what we think they do
  (or should) want and use