Proposed Research Topic

1
ANALYSIS OF AIR MOBILITY COMMAND WEATHER MISSION
EXECUTION FORECASTS METRICS OF FORECAST
PERFORMANCE AND IMPACTS ON WAR FIGHTING
OPERATIONS Capt Jeff Jarry, USAF, AFWA Prof.
Tom Murphree, Naval Postgraduate School Col David
Smarsh, USAF, Naval Postgraduate School Part of
an RTP Project Funded by ONR and SPAWAR BACIMO
Conference October 2005
2
Goals of This Metrics Study

• How Well is Air Force Weather (AFW) Doing?
• Improve Data Collection and Analysis Processes
• Establish Forecast Performance and Ops Impacts
  Benchmarks
• Allow Management by Fact
• Determine Return On Investment and Value Added
• Missions saved from delays
• Process improvements for potential value
• Make Recommendations for AF
• Metrics Regulations
• Augment AF Audit Agency MEFVER Report

3
Integration of MEF Performance and Operational
Impact Metrics
Operational Outcomes
MEFs
Observations
Operational Plans
1
Operational Performance Metrics
MEF Performance Metrics
2
1. Data Collection
Major Goal of Project
3
Metrics of MEF Impacts on Operational
Performance (Value Added)
2. Performance Analysis
3. Assessment of Ops Impacts
4
Mission Execution Forecast (MEF)

• Mission Execution Forecast (MEF)
• Weather forecast product delivered to a
  customer, for a specific mission
• DoD Forms 175-1s, 181-1s
• PowerPoint and other non- standard forms
• MEFs tailored from available guidance / products
• MEF used by mission commanders to make No Go or
  Go decisions

5
MEF Verification (MEFVER) Data
FOUO

• Air Force Wide Data Inconsistencies
• Non-standard MEF definition
• Non-standard reporting of MEFs / forecasts
• AFIs lacked details
• Data Selection
• AMC aircraft have common ops thresholds
• Data period FY 2004, first full FY of MEFVER
  data
• Spans OIF and OEF
• AMC FY 2004 Data Totals
• 228,692 Reported MEFs
• 74,000 missions supported (estimated)
• Estimated 2,100 experienced No Go conditions
  (2.8)

FOUO
6
Global Mobility Weather Flight (WXM) at the
Tanker Airlift Control Center (TACC)

• WXM
• Integrated into flight dispatch function of TACC
• Face to face briefs with flight mangers for
  missions at risk due to weather
• Weather ORM process
• Flight-following wheels up until wheels down
• Nearly 4,000 MEFs a month
• In future Will support all non-training ops for
  Air Mobility missions

7
Air Mobility Command (AMC) Operationsand Weather
Support Timelines
Long Range Planning
Short Term Planning
Mission Execution
T 20
T 72 hrs
T 96 hrs
T 0
T 48 hrs
T 24 hrs
T wks to mos

• Destination

Takeoff
T 3 to 4 hrs MEF Delivered
T 6 MEF Production
Effort Level / Actions Taken
Low / Routing and Staging Options
Data Collection Focus
Medium / Final Routing Decisions
High / Final Mitigation, WX Avoidance
Model Data Planning Products
Obs, TAFs, Perishable Data
Climo Products Extended Models
8
Calculation of MEF Performance Metrics
FOUO

• Combined MEF Forecast Accuracy
• C MEFAC
• 100 (AD) / (ABCD)
• Combined MEFAC is Currently the Only MEFVER
  Metric
• Primarily a Measure of Go MEF Performance
• 98.5 Go events
• No Go Performance is Nearly Invisible

FORECAST
TOTAL
Go
No Go
O B S E R V E D
A B 3379
B 1109
A 2270
No Go
Go
CD 225,313
C 1193
D 224,120
TOTAL
ABCD 228,692
BD 225,299
A C 3463
FOUO
9
MEF Performance Metrics Resultsfor AMC CWTs
FOUO

• Combined MEFAC looks great alone
• Compared to Go persistence, only 0.5 improvement
• Return on investment

FOUO
10
MEF Performance Metrics Resultsfor AMC CWTs
FOUO

• Performance Lower When Considering Other Metrics
• Consider for Benchmarks?

FOUO
11
MEF Performance Metrics Resultsfor Seven AMC CWTs
FOUO

• No Go MEFAC How Good are We When it Counts?
• Increase in Variation s 15.7
• Use AMC 7 Average as Benchmark?

FOUO
12
WXM Operational Impacts Metrics

• Missions at Risk Due to Weather (Phenomena)
• Mitigation
• WXM provides mitigation advice to flight managers
  (FM) for at risk missions
• FMs take advice and change plans
• FMs press with original plansroll the dice
• Mitigation Rate
• How often flight mangers took action on advice
  provided by WXM (advice taken / advice provided)
• Weather Delays to WXM Supported Missions

13
Operational Impacts Metrics ResultsWXM Monthly
Missions Saved
FOUO

• Estimated Missions Saved per Month
• Combination MEF Performance and Ops Impacts
  Metrics

• War Fighting Operational Impacts
• Nearly 90 of WXM missions supported war effort
• 45 war fighter support missions a month avoided
  delay / failure

Number of Weather Mitigation Actions Taken by
Operators Estimated Number of Missions
Saved Estimated Number of Unnecessary Actions
Taken
APR
OCT
NOV
DEC
JAN
FEB
MAR
MAY
JUN
JUL
AUG
SEP
FY04 AVG
FOUO
14
Operational Impacts Metrics ResultsWXM Monthly
Mitigation Rate
FOUO

• Mitigation Rate Correlated to Weather Delays
• Correlation coefficient -0.48, 93 certainty
• Good Information for WXM Flight Manager Training

Kandahar Push
FOUO
15
Summary and Conclusions

• No Single Performance Metric is Adequate
• Additional Performance Metrics Add Value
• Facts/benchmarks for management decisions
• Combine MEF Performance and Operational Impact
  Metrics to Show Return on Investment and Value
  Added
• WXMs estimated missions saved from weather
  delays
• Mitigation rates and mission delay correlations
• At risk totals for RD and training priorities
• Relation to War Fighting Ops
• During war time ops, estimated nearly 90 of WXM
  missions supported war effort
• 45 war fighter support missions a month avoided
  delay/failure

16
Future Air Force Weather MEFVER

• MEFVER FYI
• Response to AFAA report
• Complete Oct/Nov 05
• Basic program/tool set
• CWT/BWT air/ground
• Revision of AFI 15-114
• Define minimum performance metrics
• Foundation for ops impact metrics

17
FOUO
Operational ImpactsTerminal Weather That Placed
Missions at Risk

• Immediate Recognition for Where to Focus Resources

SVR ICG 4.9
FOUO
18
WXM Missions at Risk for TCs
.