Unmanned Aircraft in the NAS

1
Project on Unmanned Aircraft in the NAS Final
Review Panel Meeting
2
Integration of Unmanned Aircraft into the
National Airspace System

• A Project Course by
• Carnegie Mellon University
• Dept. of Engineering and Public Policy
• Dept. of Social and Decision Sciences
• May 1, 2007

3
Expert Review Panel

• Tom Curtin, AUVSI
• Bret Davis, AUVSI
• Lexa Garrett, America West Airlines
• Jim Geibel, GAO
• David Gerlach, FAA
• Tom Henricks, Aviation Week
• Ramon Lopez, Aurora Flight Sciences
• Edmond Menoche, GAO
• Rene Rey, FAA
• Melissa Rudinger, Aircraft Owners Pilots Assn.
• James Sizemore, FAA
• Larry Thomas, GAO
• Dyke Weatherington, DoD/OSD

4
Purpose of CMU Project Courses in Technology and
Policy

• Analyze a real world policy problem involving
  technology
• Combine diverse information and analytic
  frameworks to derive policy insights
• Learning objectives
• Problem decomposition, structuring and
  formulation
• Interdisciplinary problem solving
• Communication
• Teamwork

5
Examples of past project courses
6
Contributors to our UAS project

• 20 undergraduates majoring in
• Engineering
• Social Science
• Business Administration
• 3 Ph.D. student managers
• 3 faculty advisors
• Expert review panel
• Other experts

7
Background for this Project

• Increasing demand for UA
• Military (many current uses)
• Civilian (many potential uses)
• Federal Aviation Administration (FAA) is
  developing a roadmap for integrating UA into the
  NAS
• A few of the issues to be addressed
• Safety and reliability
• Public acceptability
• Market viability

8
Analysis Areas

• Economics
• How cost-effective are UA compared to alternative
  means of providing specific services?
• Risk, Technology and Standards
• What are the regulatory implications of different
  approaches to equivalent level of safety?
• Public Awareness and Perceptions
• Are risks of UA of greater public concern than
  risks of manned aircraft?
• Governance
• How can the current system for deliberation and
  decision-making on UA access be improved?

9
Project Outcomes

• 16 person-months of research completed across
  the four focus areas
• Economic model of market viability
• Risk model of fatality implications of UA
  introduction
• Better understanding of public awareness risk
  perception
• Actor roadblock analysis yields insight on
  deliberative process for UA integration
• Regulatory policy recommendations

10
Economics
Team Members Nathan Diorio-Toth Feng Deng Reiko
Baugham Victoria Morton Brad Brown Team
Manager Ryan Kurlinski
10
11
Purpose

• Assess the market viability of UAS applications
  using relative cost effectiveness
• Assess the effect of various regulatory measures
  on the market viability of UAS applications

11
12
Goals

• Develop UAS cost model
• Cost components
• Airframe
• Communications
• Insurance
• Pilot
• Etc.
• Apply cost model to chosen applications and
  alternatives to compare cost
• Examine sensitivity of overall cost to changes in
  each cost component
• Estimate cost implications of different
  regulatory measures and technology improvements

12
13
UAS Applications

• Weather Reconnaissance
• Alternative
• WC-130J Hercules high-wing, medium range
  aircraft
• Pipeline monitoring
• Alternative
• Concentric sensors pressure sensitive sensors
• Localized Surveillance
• Alternative
• Traffic Helicopter e.g. Bell JetRanger

13
14
Analysis Method

• Used triangular distributions to assign probable
  ranges to each input cost
• From this, generated a Probability Density
  Function
• Probability Density Function shows the entire
  range of possible costs with the associated
  likelihood of each cost
• Allows analysis of the most probable cost
  advantages

14
15
Importance Analysis
Contribution of uncertainty in each input to
uncertainty in total cost
Triangular probability distributions of all input
variables
Economic Model
16
Weather Reconnaissance

• Analyzed the use of Aersonde UAS for Weather
  Reconnaissance vs. the use of the WC-130J
  Hercules
• Aerosonde UAS currently in use for Weather
  Reconnaissance
• Hercules WC-130J currently in use by Keesler Air
  Force Base

16
17
Results Weather Reconnaissance
Probability Density of UAS Cost Advantage
(/flight hour)
17
18
Results Weather Reconnaissance
Importance Analysis of Model Inputs
Mission Hours per Year
Operational Lifetime
Com-Link Cost
18
19
Results Weather Reconnaissance

• Key Results
• UAS more cost effective than current manned
  alternative
• Most important inputs in determining overall cost
  effectiveness
• Mission hours per year
• Com link cost
• Operational lifetime
• Currently available sense-and-avoid equipment
  cause significant decrease in cost effectiveness,
  but does not cause the UAS to be more expensive
  than the manned alternative

19
20
Pipeline Monitoring

• Analyzed the use of the Aero Environment AeroPuma
  vs. the use of concentric wire sensors (6/m)
• Note the difference in monitoring style
• UAS monitors using thermal imaging with each pass
  and relays pertinent leak info to docking
  stations
• Concentric sensors constantly monitor pipeline
  and relay information

21
Results Pipeline Monitoring

• Key Results
• UAS cheaper depending on number in use
• Important to note difference in monitoring styles
  between UAS and concentric sensor
• Important inputs
• Relay/Docking station cost
• Number of UASs in use

21
22
Localized Surveillance

• Application based on the surveillance of a 1km2
  area for a short time (1-3 hours)
• Considered the use of a Cyber Defense Systems
  CyberBUG vs. the use of a traffic helicopter
• For model inputs, considered monitoring a large
  traffic accident over 2 hours
• For policy considerations, analyzed the addition
  of mandated sense-and-avoid hardware to the UAS

22
23
Results Localized Surveillance
PDF of Cost per Mission for UAS Compared with
Manned Alternative
Probability Density
Note no meaningful overlap
2000
4000
6000
8000
10K
8000
12K
14K
16K
Cost per Mission ()
23
24
Results Localized Surveillance
PDF of Cost per Mission for UAS Compared with
Manned Alternative with High-Range Fixed Cost
Variance
Probability Density
Note still no meaningful overlap
2500
5000
7500
17.5K
20K
22.5K
10K
12.5K
15K
Cost per Mission ()
24
25
Results Localized Surveillance
PDF of Cost per Mission for a Larger UAS Capable
of Carrying Sense-and-Avoid Equipment Compared
with the Cost of Manned Alternative
Probability Density
Note Significant overlap indicating that UAS
would likely no longer be a viable alternative to
manned craft
0
10K
20K
30K
40K
50K
60K
70K
Cost per Mission ()
25
26
Results Localized Surveillance
Missions per Year
Mission Related Costs
Flight Hours Per Mission
Input Importance for Cost Per Mission
Importance of inputs.
Input Costs
26
27
Results Localized Surveillance

• Key Results
• UAS less expensive in almost every case
• Levelized cost for manned more sensitive than to
  utilization hours discount rate than cost for
  unmanned
• UAS cost effectiveness reduced significantly by
  requirement for sense-and-avoid hardware
• Important inputs
• Missions per year
• Discount rate
• Flight hours per mission

27
28
Policy Implications

• Analyzed the effect of the following policies
• Mandated insurance premiums
• Mandated use of A/N hardware
• (Increased fixed cost)
• Mandated record-keeping practices
• (Increased yearly cost)
• Mandated airframe materials
• (Increased fixed cost)
• Mandated minimum amount of safety equipment
• (Increased fixed cost)
• Mandated pilot/operator training

28
29
Policy Implications Results

• All policies except mandated sense-and avoid
  hardware had little effect on the cost advantage
  of UAS over manned alternative
• Required sense-and-avoid hardware greatly affects
  cost-effectiveness, however
• Localized Surveillance and Pipeline Monitoring
  would no longer be viable as larger, much more
  expensive UAS would be necessary

29
30
Risk, Technologies, Standards
Team Members Samiah Akhtar Jonathan
Cornell Nicole Hayward Will Kim Nick Misek Doug
Robl Team Manager Keith Florig
30
31
Purpose

• Derive a risk model to explore how risk is
  related to UAS numbers, dimensions, and flight
  zones
• Research on elements of risk mitigation such as
  human factors, sense and avoid
• Exploration of alternative incident reporting
  systems

Predator
Source http//www.fs.fed.us/psw/news/PSW_News/20
05_09/images/uav.gif
32
Technology and Risk Outline

• Goals
• Risk Modeling
• Purpose
• Assumptions Approach
• Findings
• UAS Risk Mitigation

33
Risk Modeling Purpose

• Provide a way of modeling that creates some
  groundwork for future modeling
• Use model to compare relative risk calculations
• Pointer to the future, not the answer
• Points of interest
• Mid-air vs. single-craft crash
• Effect of sense and avoid technology
• UAS to displace manned aircraft

Source http//www.maximog.com/images/sublevel/uav
_left.jpg
34
Risk Modeling Assumptions

• Uniform national model
• Uniform traffic density
• Uniform ground population density
• Uniform aircraft per type
• Appropriate for
• VFR traffic
• Rural, less populated areas
• NOT Appropriate for
• Urban settings
• Airports
• High traffic densities

35
Risk Model
36
Risk Modeling Approach

• Number of midair collisions

N total number of aircraft in
defined airspace ? aircraft traffic
density D diameter of plane
(wingspan) S average aircraft speed P(A)
probability of avoidance (Used for calibration)
VFR operations only
UAV Picture Source http//www.evworld.com/press/s
pider_lion_uav.jpg
37
Risk Modeling UAs displacing Manned
Small risk from unmanned at lower extrema
Risk from unmanned at low levels less than
decreased risk from manned
Single-craft crashes still present less risk than
mid-airs
38
Risk Modeling Mid-Air vs. Single-Craft
At some point, manned risk surpasses unmanned risk
At low numbers, sense and avoid has little effect
Single-Craft generally less risk than mid-air
39
Risk Modeling Conclusions

• Mid-air collisions generally have more risk than
  single-craft crashes
• Displacing small to moderate amounts of manned
  craft represents decrease in risk
• Smaller, less reliable UAs can present less risk
  than larger more reliable manned aircraft
• For small numbers of UAs in low traffic
  densities, sense and avoid has small effect

40
Technology and Risk Outline

• Goals
• Risk Modeling
• UAS Risk Mitigation
• Human Factors
• Sense and Avoid

41
Human Factors Implications

• Risks - Caused Most Number of Accidents
• Sensory Isolation McCarley et al
• UAS operator does not receive same sensory cues
  as manned aircraft operator
• Automation
• Malfunction of automated components controlled by
  the UAS operator
• Operator Hand-Off
• Issues with handing off control of vehicle from
  one operator or crew to another

42
Human Factors Implications

• Recommendations
• Training and Procedures
• Up to date training as new technology
  advancements arise
• Ensure that operator has accurate knowledge of
  automated components within UAS
• Multimodal displays
• Prevent sensory isolation
• Allow for audio, visual and speech control
• Example simulated cockpit

43
Detect, Sense and Avoid

• Risks
• Market impact of single fatal collision
• Lack of standardization among DSA systems

44
Detect, Sense and Avoid

• Recommendations
• Create regulations specific to size, weight,
  application etc
• Testing Periods
• Phased Integration

45
Technology and Risk Outline

• Issues
• Goals
• Risk Modeling
• UAS Risk Mitigation
• Reporting systems

46
Current Reporting Systems

• Two Options
• NTSB Reporting (as required by FAR) - Accident
• NASA ASRS Voluntary Reporting - Incident
• Current Implementation
• NTSB mandates detailed information when
• Flight control system malfunction, Illness of
  crewmember, Turbine Engine Failure, In-flight
  fire, Mid-air collision or Damage in excess of
  25,000 to other property
• ASRS System is anonymous and does not have any
  reporting requirements

47
Reporting Recommendations

• Initially mandate reporting of all accidents and
  incidents
• Re-evaluate strategy after testing period

NTSB
- NTSB information helps FAA to assess
standards - FAA responds with rules for
reporting incidents.
- NTSB provides useful information on UAS
failures - UAS responds with improved design and
engineering
Communication Triangle
UAS manufacturers
FAA
48
Public Awareness Perceptions
Team Members Darian Ghorbi Jenny Kim Mark
Peterson Laura Seitz Patrick Snyder Team
Manager Pete Tengtrakul
48
49
Statement of Purpose

• Add the element of public perception to the
  discussions of UAS in the NAS
• Motivation the fact that there has never been a
  formal presentation of public perception on the
  topic
• Findings useful for the creation of regulations
  and policy implications

50
Objectives

• Compare public perceptions of the risks
  concerning manned and unmanned aircraft
• Find demographic groups with certain risk and
  benefit patterns of UAs
• Research implications of opinion of UAs
• Create survey to aid in completing objectives

51
Hypotheses

• Perceived Risk
• - Manned lt Remotely Piloted lt Autonomous
• Ground vs. Air
• - More risk of UAS perceived in air
• Prior Knowledge vs. Risk Perception
• - Prior knowledge, associate less risk
• Benefit vs. Risk Perception
• - Higher benefit, lower risk
• Education vs. Risk Perception
• - Technical education, associate less risk
• Age vs. Risk Perception
• - Older participants more cautious
• Frequency of Flight
• - Those that fly frequently, associate less risk

52
Layout of Survey

• First Page
• Provide information about UAS
• Autonomous
• Remotely Piloted
• Gauge previous knowledge
• Source
• Last Page
• Demographics
• Gender
• Age
• Education
• Frequency of Flight
• Voting (identify opinions of those that are
  politically engaged)
• Pilot

53
Layout of Survey

• Application
• Traffic Monitoring
• Pipeline Monitoring
• Disaster Relief
• Border Patrol

• Questions
• Quick Response
• Benefit
• Stakeholder
• Public
• Risk
• Ground
• Air
• 7 Point Scale
• 1 - Much Less
• 4 - Same
• 7 - Much More

Picture of UAS application

• Description of UAS
• Physical Information
• Current application

54
Obtaining Surveys

• Coding numerical code assigned
• Screening data obtained from those under 16
  years of age were not counted

55
Statistical Methods

• Paired T-tests
• Across applications
• ANOVA
• Significance of mean
• Regression
• Correlations
• Demonstrated the strength of the variables (risk
  and benefit)

56
ResultsDescriptive Statistics
57
Perceived Relative Risks Between Remotely
Operated vs. Autonomous
Autonomous applications are viewed to have more
risk in comparison to remotely operated UAs.
58
Relative Risks Across Applications

• Traffic Monitoring has the highest perceived risk.

59
Relative Benefit Across Applications
The more risky the public perceived the
application, the less benefit they associated
with the application.
60
Relative Perceived Risk to People on Ground vs.
Air
There is no difference between risk perceived on
ground vs. air. Also, there is no difference
between perceived benefit between stakeholders
and society.
61
Demographics and Risk
Those over the age of 65 perceived UAs as least
risky and least beneficial the mean value is
insignificant.
62
Risk Perception Conclusions

• Unmanned aircraft risk gt manned
• Autonomous risk gt Remotely piloted
• No difference
• Risk Ground vs. Air
• Benefit Stakeholders vs. Society
• 54 heard of UAS
• Need education programs
• 78 of those that heard of UAS obtained
  information from television
• The more familiar, the more comfortable
• Traffic Monitoring - higher risk
• Fear of operating around high population density
  areas

63
Impact on Policy

• Limit flight path/area
• Limited population density
• Implement education/outreach programs

64
Future Insights

• Limitations
• Time, Resources, and Budget
• Sample
• National scale-different regions
• Future Surveys
• Compare UAS to other risky technologies
• Size of aircraft
• Privacy concerns
• Economics concerns
• Lengthened

65
Governance
Team Members Nora Darcher Norma Espinosa Scott
Fortune Andrea Fuller Team Manager Leonardo
Reyes-Gonzalez
65
66
Purpose

• Evaluate current system of governance for UAS
  integration against principles of good
  governance
• Suggest measures that could improve the
  governance process

67
Analysis

• Principles of Good Governance
• Rules for FAA governance
• Historical Technologies
• Actor Interactions
• Roadblocks
• Cost and Benefits for each actor

68
Characteristics of Good Governancehttp//www.unes
cap.org/pdd/prs/ProjectActivities/Ongoing/gg/gover
nance.asp
69
Governance requirements on FAA

• OMB rule requires FAA standards adoption
  procedures to have the following
• Openness
• Balance of interest
• Due process
• Appeal process
• Consensus

70
Historical Analysis

• How did the governance system handle the
  introduction of new technologies?

Technology
What We Learned
71
Actor Analysis

• Objective
• Provide a systematic assessment of the actors
  involved in integrating UAs into the NAS
• Process
• Identified key actors, examined their goals and
  looked at problem from each actors perspective

72
Actor Analysis
73
Roadblock Analysis

• Objective prioritize problems inhibiting the
  integration of UAs
• Categories
• Technological
• Organizational
• Infrastructural
• Public Concern

74
Roadblock Analysis
75
Roadblock Analysis
of actors
6
Airspace Access
5
4
Equivalent/Acceptable Level of Safety
3
2
1
Transponders
Data Acquisition
0
Low
High
Complexity
76
Most obvious needs

• Defining an equivalent/acceptable level of safety
• Allowing UAS operations in scarcely-used airspace
  to facilitate testing and development for civil
  and commercial applications.
• Potentially large public concern about UAS safety
  argues for proactive public involvement in
  deliberations

77
Conclusions
78
Summary of Conclusions

• Economics
• Risk, Technologies, and Standards
• Public Awareness and Perception
• Governance

79
Economics

• Some civil UAS applications seem highly
  competitive with alternatives
• Initial policy ought to be tailored to the most
  commercially viable applications
• Cost models show that (i) costs are most
  sensitive to hours of utilization, (ii) safety
  equipment has modest cost effect, except for
  small systems using sense and avoid
• Foreign UAS firms may develop an advantage if
  they gain airspace first

80
Risk, Technologies, and Standards

• For some applications in some classes of
  airspace, unmanned aircraft result in fewer
  fatalities than manned aircraft used for the same
  task
• Sense and avoid is important only in airspace
  with significant traffic density
• Low risk areas could be used for experimentation
  and testing without posing a high risk to those
  on the ground or in other aircraft
• A mandatory incident reporting system has
  potential to greatly improve both airworthiness
  and human factors reliability

81
Public Awareness and Perception

• All UAS applications surveyed were considered
  more risky and less beneficial than the manned
  alternative
• Traffic monitoring perceived as most risky
  (likely due to flight over dense population)
• About half of participants had heard of UAs
• Those more familiar with UAS technology perceive
  less risk

82
Governance

• Integration problem is more complex than many
  people realize
• Incremental approach allows for policy
  experimentation at low risk (e.g., sparsely
  populated areas/airspace)
• Standards need to be established to provide
  benchmark and incentive for manufacturing
• Attention to public perception and involvement
  can greatly influence unfolding of UAS issue

83

• Thank You for Coming!
• Questions?