Intelligent Autonomy Update

1
Intelligent Autonomy Update

• Marc Steinberg
• Office of Naval Research (703) 696 5115,
  marc_steinberg_at_onr.navy.mil
• Naval Air Systems Command (301) 342 8567,
  marc.steinberg_at_navy.mil
• SAE Control Guidance Committee Meeting, 2
  March, 2005

2
Intelligent Autonomy Future Vision
External C4I, Data Filtering
Mixed-Initiative Human Interface
Common Operational Picture
High Level Mission Requirements/ROE
Common Information Management
Multi-Vehicle Mission Planning
Management of 5-10 UxVs
Task Scheduling, Routing, Constraints, Priorities
Share Mission Relevant Info Report Status
Multiple Types of Vehicle Interfaces/Comms
UGV Locally Controlled, Shares Info
UUV Local Planning SA
UAV Local Planning SA
USV Local Planning SA
Local Task Negotiation, Information Sharing
3
MOUT Site Demonstration Univ. of Penn., Georgia
Tech., USC, BBN, Mobile Intelligence

• Summary
• Team planning mechanisms geared towards
  maximizing communications capabilities in adverse
  conditions while on the move.
• Integrated mission specification capabilities for
  parsing the tasks of overall mission objectives
  and mapping them onto heterogeneous UxVs
• VV of autonomous algorithms
• Joint with DARPA

• Accomplishments
• Integration of large number of heterogeneous
  UxVs with different lower-level autonomy
  software
• Framework integrating communications, perception,
  and execution for UxVs
• New algorithm development implementation for
  communication sensitive behaviors, heterogeneous
  UxV tasking, distributed SA, and VV of
  autonomous systems

• Completed Demo at MOUT Site
• Operator tasks multiple types of UAVs and UGVs
  with high-level commands
• Mission execution while maintaining communication
  constraints

• Future Work
• Case-Based Reasoning Multi-UxV Task Allocation
  to support mission specification
• Extend VV approach to test IA systems

4
Risk-Aware Mixed-Initiative Dynamic Replanning
DemoDraper Laboratory/CRA

• Summary
• Single-operator mission management of multiple
  heterogeneous unmanned vehicles
• Extend UUV software w/ increased autonomy
  on-board UUV increased dynamic retasking
  capability for missions w/ comms constraints
• Integration with on-board vehicle sensors
  external systems

N
Region 3
Region 4
Region 2
Keep-out
Region 1
No Comms
Target region

• Accomplishments
• Initial software design implementation
• Integration of initial versions of SA, situation
  assessment, mixed-initiative interface, control
  station planning, UUV on-board planning/SA.
• Initial integration with existing UUV lower-level
  autonomous control software
• Usability analysis by NAVAIR NSWC to recommend
  improvements

• Completed Dynamic Replanning Demo w/
  high-fidelity UUV simulation
• Single UUV, UAV as comm link
• Operator provides high-level tasking/constraints
• Autonomously generates plan of activities for UUV
  to start the mission, transit, search shoreline
  for particular target classes, end mission
• Monitors execution

• Future Work
• Extend to increase UAV/UUV cooperation,
  complexity of tasking, integration with other
  systems, realism of simulation
• In-water demonstration

5
Mission Control for Multiple UxVs
DemonstrationBAE Systems, Aptima, MIT, BU, Univ.
of Minnesota

• Summary
• Allocates tasks based on operator high-level team
  tasking and constraints
• Determines team individual vehicle tactics to
  achieve objectives
• Schedules activities for heterogeneous resources
• Inputs to lower-level vehicle planners
• Cooperation w/ realistic comms limitations

• Accomplishments
• Extended Mission Control System (MCS) to provide
  tasking and routing for naval autonomous systems.
  
• Allocates tasks to vehicles including cooperative
  search and data collection
• Constraints include time windows, precedence
  constraints, and no activity zones
• Tasks prioritized as mandatory, high, med, or
  low. Drops lower priority tasks if not feasible.

• Multi-UxV Simulation Demonstration
• Firescout, J-UCAS, BAMS, USV
• Operator provides high-level team tasking, ROEs,
  constraints
• Replans following changes in environment or new
  tasking

• Future Work
• Simulation Demonstration in communication limited
  environment (joint with Air Force)
• Local planning under constraints on vehicle when
  outside of communication

6
Maritime Image UnderstandingNorthrop/Carnegie
Mellon University

• Summary
• Maritime Image Understanding for autonomous
  sensor-directed dynamic replanning
• Supports low elevation UUV mast with no
  mechanical stabilization of image
• Detects, classifies, and tracks targets
• Joint with DARPA

Completed In-Water Demo on Spartan USV

• Accomplishments
• Reliable surface object detection in clear and
  hazy conditions
• Image stabilization using software only
• Low false alarm rate for shoreline man-made
  object detection
• Interesting object detection to direct data
  collection
• Real-Time maritime scene segmentation/high speed
  video array limited classification capability

Surface Object Detection Tracking
Shoreline Detection

• Future Work
• Integration with on-board dynamic replanning
  software to enable replanning with sensed data
• Additional in-water experimentation
• Improved robustness

Shoreline Man-Made Object Detection
7
Intelligent Control Autonomous Replanning of
Unmanned Systems (ICARUS) - Lockheed

• Summary
• Integrated suite of components to enable rapid
  highly automated and fully autonomous mission
  planning/replanning by high-level objectives
• Determines optimized route that meets all
  constraints mission objectives while also
  optimizing secondary priorities

• Accomplishments
• Integrated Multi-UV Task Allocation, Replanning,
  Replan Assessment, Information/Alert Management,
  Operator Interface, Lower level GNC components
• Objectives incorporated include search for
  stationary mobile targets, EO/IR, SAR, Loiter,
  Steer-Point, Communication
• Constraints incorporated included no-fly zones
  and LOS comms requirements (moving stationary)

• Completed Simulation Demonstration Evaluation
  by 2 Navy 2 Marine Corps Operators with 7 UAVs
  (Firescout, BAMS, Future ISR)
• Replanning for Dynamic Mission Events
• New/Dropped Mission Tasks
• Change in order of tasks
• Failures/Weather/changes in environment

• Future Work
• Integration with naval control station
• Simulation Demonstration in warfare environment
  at NAVAIR
• Increased robustness/integration