Software Enabled Control UniversityLed Final Experiment Workshop June 2526, 2002

1
Software Enabled ControlUniversity-Led Final
Experiment WorkshopJune 25-26, 2002
controls.ae.gatech.edu/projects/sec
2
Description Objectives

• Develop software-enabled control methods for
  complex dynamic systems with application focus on
  intelligent UAVs
• Support-the-development and implement a
  plug-and-play, real-time software architecture
  for SEC integration (OCP-based)
• Provide Software-in-the-loop (SITL) and
  Hardware-in-the-loop (HITL) simulation and flight
  testing to SEC Technology Developers

3
University-Led Final Experiments
Optically Guided Glider Flown into Opening to act
as Sensor Possibly include sub-robot
Video
GTMAX Mother Ship
Simulated Structure with Opening
Urban, Homeland Defense Information Sensing,
Exploiting, Planning and Execution
4
Workshop Agenda

• Description of GTMax VTOL UAV
• Hardware Available
• Baseline Software
• Hybrid Controls API, and Integration
• Flight Tests Planned for late Summer
• Representative of Final Exam Approach to
  Collaboration
• Planned Collaborations on Final Experiments
• Final Experiments Definition
• Final Experiments Schedule

5
University Led Experiment

• GTMax VTOL UAV Description

6
GTMax

• Yamaha R-Max, 66kg, 3m Rotor Diameter
• Avionics and Simulation Tools Developed Over the
  Past Year
• Hardware-in-the-Loop Simulation and Ground
  Testing Started in November 2001
• Navigation System Ground Tests Completed
  February 2002
• Flights Testing (With Avionics)Began March 2002

7
Yamaha R-Max Industrial Helicopter

• Rotor Diameter 10.2 ft
• Length 11.9 ft
• Gross Weight 205 lb
• Payload 66 lb

• Gasoline, 2 Cycle
• 2 Cylinder, Water Cooled
• 246cc Displacement, 21 hp
• Endurance 60 min

8
GT Research UAV GTMax
GPS Reference
GPS
Georgia Tech Onboard Avionics
Data Link I
Data Link I
Ground Computer(s) And Network
GEORGIA TECH
Ethernet
Data Link II
Data Link II
Ground Control Station
3x RS-232 Serial
RC Receiver
Yamaha Attitude Control System (YACS)
YAMAHA
On-board Avionics
Safety Pilot
Actuators
9
GTMax Hardware Components

• Primary Flight Computer
• 266MHz Embedded PC, Ethernet, Flash Drive
• Sensors
• Inertial Measurement Unit (x2)
• Differential GPS
• Magnetometer
• Sonar and Radar Altimeters
• Vehicle Telemetry (RPM, Voltage, Pilot Inputs)
• Data Links
• 11 Mbps Ethernet Data Link
• RS-232 Serial Data Link

10
GTMax Hardware Integration

• Exchangeable modules
• Flight Computer Module
• GPS Module
• Data Link Module
• IMU/Radar Module
• Unused Module (Growth)
• Sonar/Magnetometer Assemblies
• Power Distribution System
• Each module has self-contained power regulation
  and EMI shielding
• Vibration isolated main module rack

11
GTMax Hardware Integration

• Power System
• On-board generator outputs 12V DC, 10 A
• Hot-swap between on-board and external power
• Each component has individual circuit breakers
• Interfacing and Wiring
• Wiring RS-232 Serial, Ethernet, 12V DC
• Easy access to modules from one side, see LEDs
• Aviation-quality wiring on the other

12
GTMax Flight Control Avionics
13
Sensors IMU

• ISIS Inertial Measurement Unit (ISIS-IMU)
• Acceleration and angular velocity triads
• Analog or 100Hz digital output
• 160 deg/sec max angular rate 11G max acceleration
  (as configured for the digital interface)
• 1.1 deg/sec and 0.003G standard deviations
  measured with rotor spinning at flight RPM
• Ours is a special order that takes a 12VDC input
• www.inertialscience.com

14
Sensors Differential GPS

• NovAtel RT-2 Differential GPS (MiLLenium Card)
• Carrier phase tracking
• Output of position and velocity used, WGS-84
• 5Hz update rate
• Differential correction from ground station
  communicated over both datalinks (every 2
  seconds)
• 2 cm CEP specification
• www.novatel.com

15
Sensors Magnetometer

• Honeywell 3-axis magnetometer HMR-2300
• Measures strength of magnetic field in 3
  directions
• 20 Hz digital output

16
Sensors Radar Altimeter

• Radar Altimeter
• Distance from terrain
• Range 5 to 2300 ft
• 10 Hz

17
Sensors Sonar Altimeter

• Custom built sonar ranging system
• Polaroid range sensor
• Distance from terrain
• 0.5 ft to 10 ft
• 10 Hz

18
Actuators Helicopter Interface

• Yamaha R-Max modified with the same digital
  interface used in their Autonomous R-Max
• Has four RS-232 interfaces
• Send actuator commands, turn on/off stability
  augmentation (50 Hz)
• Get commands as received from safety pilot radio
• Get Yamaha IMU data (for its SAS)
• Get other telemetry
• RPM, Voltage
• Actuator command source

19
Dalalinks FreeWave

• FreeWave wireless modem
• 9600 baud (other settings possible)
• Tested up to about 2 miles with helicopter on the
  ground (worst case)

20
Dalalinks AeroNet

• AeroNet wireless ethernet
• Have seen as high as 11 Mb per sec in flight
• Short range,

21
Processors

• Primary flight computer
• 266 MHz Pentium
• 30 Mb flash drive
• Secondary computer (under construction)
• 850 MHz Pentium
• 500 Mb flash drive
• Ethernet hub onboard

22
Baseline Software

• Operating Systems
• Onboard (Flight) Software VxWorks or QNX
• Simulation Windows or Linux
• Languages and Libraries
• C/C, Utilizes OpenGL API
• No Special Compilers Needed (We Use Visual
  Studio on Windows or gcc on Linux)

23
Baseline Software Overview

• Onboard software
• Primary flight computer
• GCS software
• Runs on ground control station
• Simulation software
• Not used in flight
• All of the above included in GCS/Simulation build
• Only onboard software include in onboard or OCP
  build

24
Baseline Onboard Software

• Onboard software can be executed stand-alone
  (single task)
• Onboard software can be called as OCP components
  
• IO component
• Navigation component
• Controller component

25
Baseline OCP Implementation
Controls API Input Port
Controls API Output Port
I/O Component
DataLink Interface
Ethernet Serial Port
Input datalink ports read _at_ 100 Hz
Serial port
ControlData_out
NavData_out
Ethernet Serial Port
1 Hz 10 Hz
m0 written at 10 Hz m1 written at 1 Hz
Serial port
1 Hz 10 Hz
50 Hz
100 Hz
ControlData_in
NavData_in
NavControl_in
50 Hz
50 Hz
NavControl_out
26
Baseline Onboard Software

• Navigation
• 17 State Extended Kalman Filter Navigation System
• Vehicle Position
• Vehicle Velocity
• Vehicle Attitude
• Accelerometer Biases
• Gyro Biases
• Terrain Height
• All Attitude Capable
• 100 Hz Updates
• Flight Operational

• Control
• Adaptive Neural Network Trajectory Following
  Controller
• Neural Network
• 16 Inputs
• 5 Hidden Layer Neurons
• 6 Outputs for 6 Degrees of Freedom
• Can Also Be Configured as a Conventional
  Inverting Controller
• Flight Operational

27
Baseline Navigation Filter

• Extended Kalman Filtering Approach Used to
  Integrate Sensor Data in Real Time
• IMU Data is Propagated at 100 Hz and Discretely
  Updated by All Other Sensor Measurements

Acc. / Rate
IMU
Position/ velocity
Vehicle
DGPS
Extended Kalman Filter
F M
X
azimuth
Magnetometer
h
Sonar/Radar
More Sensors
28
Baseline Navigation Updates

• Differential GPS
• Mounting position and latency compensation
• Updates position and velocity state (5 Hz)
• Magnetometer
• Measures magnetic field line vector
• Corrects heading drift of the filter (20 Hz)
• Sonar/Radar
• Updates the vertical position and terrain height
  state (10 Hz/variable)
• Filter Initialization
• Based on averaged sensor data while helicopter is
  on the ground
• Initial position GPS
• Initial roll and pitch attitude, gyro biases IMU
• Initial heading magnetometer
• All other states set to zero

29
Baseline Controller

• Adaptive Trajectory Following Controller
• Supports Trajectory Commands, e.g. From a
  Real-Time (Re)Planning System

Inner Loop
Outer Loop
Command Trajectory
Neural Network
30
Communications

• Generic communication routines
• Supports serial data reading and writing
• Can re-route this data through Ethernet or as
  memory within an a single executable
• Most operating system specific software is
  limited to these routines (maybe they should
  replaced with ACE routines)
• Used for
• all onboard serial interfaces
• all GCS serial interfaces
• a redundant datalink through the wireless
  Ethernet
• Data received on a link can be saved to a file

31
Data recording

• Record any onboard data at fixed rate into a
  binary file
• Can start/stop recording from GCS
• Download normally over wireless Ethernet

32
Baseline GCS Software

• Monitor onboard systems
• Send reconfiguration commands
• Probe onboard data
• Initiate data recording, download data
• Set up, preview, and upload flight plan

33
Available Data

• Available Onboard
• Raw Sensor Data, Pilot Inputs
• Processed Navigation Solution
• Ground Station Communication
• Available On the Ground
• Network Connection to Onboard Computers
• Available Inputs
• Desired Continuous Trajectory for Baseline
  Autopilot
• Servo Commands (If We Trust You)

34
Simulation Tools

• Hardware In the Loop Simulation Capable
• The Desktop Computer Simulation Utilizes
• Actual Flight Software
• Actual Ground Control Station Software
• Flight Test Verified Dynamic Model of Helicopter
• Flight Test Verified Model of All
  Sensors/Actuators
• Scene Generation Capability

35
Software in the Loop (SITL)
36
Flight Operations

• Network Connections Available At Ground Control
  Station from Hub
• Multiple Laptops Can Communicate with Onboard
  Computers Simultaneously
• Endurance Limited Only by Onboard Fuel Due to
  Generator ( 1 hour)
• Ground Equipment Can Operate on 115VAC or 12VDC
  and Has Battery Backup

37
McDonough GA Test Field
38
Results With Baseline Controller
Step Input of Altitude Command
39
Potential Actuator Failure Scenarios
Scenario
Description
Effect
Reconfiguration
Main rotor collective is stuck in a known
position.
Vertical acceleration can not be controlled by
the collective.
Use RPM to control vertical acceleration.
1. Stuck Collective
Use altitude (or vertical acceleration) to
compensate for loss of rpm control (i.e.,
Autorotation).
Throttle is stuck in a known position or goes to
zero (engine failure).
RPM can not be governed as is traditionally done.
2. Throttle Stuck / Engine Failure
Use thrust and collective control to control yaw
acceleration.
3. Tail Rotor Stuck / Transmission Failure
Tail rotor is stuck at a given pitch or stops
rotating altogether.
Yaw acceleration can not be controlled by the
tail rotor.
40
Modifications to GTMax

• OCP Implementation is expected to be expanded
• Two processors available
• As needed to support final experiment scenarios
• Hardware additions
• Video camera and video link
• As needed to support final experiment scenarios
  (weight and power available)

41
Controller Development and Testing GTMax Flight
Tests Late Summer

• Adaptive Mode Transition Control
• Fault Tolerant Control
• Limit Detection and Avoidance

42
Adaptive Mode Transition ControlImplementation
OCP
43
UAV FTC Setup
Fault Detection Identification
Faulted Controller
Nominal FlightController
ActuatorCommands
Engine
ThrottleCommand
Tail Rotor
Tail RotorCommand
HelicopterDynamics
Collective
Sensor Suite
CollectiveCommand
Lat. Cyclic
Lat. CyclicCommand
Long. Cyclic
Long. CyclicCommand
44
Limit Detection and Avoidance

• Adaptive neural-net-based limit detection for the
  dynamic trim prediction
• Hardware-in-the loop and flight tests using the
  GT-Max helicopter Summer 2002

Helicopter Dynamics

Approximation Math Model or Static Neural Net
-



K
Adaptive Net 1

Approximation Math Model or Static Neural Net
-



K
Adaptive Net 2
45
Experimental Plan

• Goal flight-test mid-level control implemented
  through the Hybrid Controls API
• Adaptive Mode Transition Controller
• Smooth obstacle avoidance (would like input from
  a trajectory planner)
• FTC
• Fault recovery using FDI and control
  reconfiguration
• Limit Detection and Avoidance
• Extreme maneuvers without violating vehicle
  safety limits
• Usable with other low level controllers
• opportunities for collaboration

46
University Led Experiment

• Hybrid Controls API and Mid-Level Control
  Integration

47
Adaptive Mode Transition ControlImplementation
OCP
48
Goals for Transition Management

• Reduce Development Time
• reuse common structure of Transition Management
• (dont figure it out from scratch for each
  design)
• should be easy to add new mode/controller
• should be easy to change transition strategy
• Reliability, Early Validation
• prototyping (employ modeling and simulation)
• reuse of pre-validated strategies
• Rapid Adaptation
• low performance overhead

Set Point
Mode Transition Controller
Mode
Mode Transition Manager
Process 2
C1
ACM1
C2
ACMn
u
Cn
49
Transition Management for Hybrid Controls
50
TM builds on OCP Behaviors
OCP Component
Behavior1 f1(A, B)
Signal A
Signal B
Signal E
Behavior2 f2(A, B, D)
Signal C
Signal D
Behavior3 f3(C, D)
Provided by Boeings Transition Service
51
In Hybrid Controls API Specify Coordinator and
Transition Functions
52
Layout of TM Objects
...
Configuration
OCP Component

• signal transition function

...
53
TM Code Generation

• Hybrid Controls API
• Coordinator (FSM)
• Configurations
• Signal Transition Functions

Code Generation
Application-Specific OCP Classes
Application-Specific TM Classes

• Transition Management Interface
• SwitchToFunction
• LookupFunction

54
Output Blending Example
Time
Clock
Plant1 Configuration
C1
Plant
PlantState
Time
Clock
Blend Configuration
C1
Signal Transition Function
Plant
PlantState
C2
Time
Clock
Plant2 Configuration
PlantState
C2
Plant
55
Output Blending Example
Connector
Plant 1 Configuration
Behaviors
Receiver
Transmitter
Blend Configuration
Plant State
Time
Plant 2 Configuration
Coordinator
56
Example Implementation using Hybrid Controls API
OCP_ComponentConnector" NamePlantBlendFunction" NameHCTime" Type"double"/ NamePlantState" Type"double"/
... "factor
(((1-factor)c1_x2)(factorc2_x2)-0.27)/0.6
PlantState ((1-factor)c1_X2)(factorc2_x
2)" NamePlantCoordinator" InitialState "INIT"


ConfigurationPlant1_Function

HCTimegt20"

" Plant1_Function.GetPlant1().SetY_Ref(0
.4)"
Name"BLEND Configuration"Plant_Blend_Function
"
"PlantStategt0.33"

nagement
Configuration
signal transition function
Coordinator
57
TM Code Generation
Hybrid Controls API (XML IR)
User Defined Specification
Code Generation
Application-Specific OCP Classes
Connector
Application- Specific TM Classes
Plant1_Function
Plant2_Function
Plant_Blend_Function
Plant_Coordinator

• Transition Management Interface
• SwitchToFunction
• LookupFunction

• Controls API (Behaviors, Ports, Signals)
• EnableBehavior
• DisableBehavior
• GetInputData
• GetOutputData
• CoherentGet
• CoherentSet

58
TM Rapid Prototyping
Ptolemy Model
XML Format
XML Parser
XML IR
Code Generation
Transition Management
Controls API (Behaviors, Ports, Signals)
59
Using TM in AMTC Example
Mode Transition Controller
Configuration Cn-1
Set Point Signal
Cn-1
Behavior1
Configuration Transition
U Signal
Cn-1
ACM
Cn
UAV State Signal
Behavior2
Configuration Cn
Cn
Behavior3
Mode Transition Manager
60
Progressive Series of Experiments
Goals Reduce Development Time thru reuse Reduce
Time to Change Design Reliability, Early
Validation thru prototyping Rapid Adaptation w/
low performance overhead
61
University Led Experiment

• Final Experiments

62
University-Led Experiments

• A Series of Progressive Experiments
• Benchmark - Single VTOL UAV with Reconfigurable
  Controls through the OCP on a single processor
• Mid-Term - VTOL UAV OCP on two separate
  processors
• Final - A single VTOL UAV Mother Ship which
  launches another small air and/or ground robot
  OCP on two separate processors
• Develop GTMax VTOL UAV, Ground Control Station,
  and Simulation Infrastructure
• Add additional Platforms
• Developed as an Open Experimental System

63
AUVS IARC Mission for 2001 and Beyond
(http
avdil
.
gtri
.
gatech
.
edu
/AUVS/
IARCLaunchPoint
.html)
In Less Than 15 Minutes
In Less Than 15 Minutes
Launch Area
Launch Area
3 km
3 km
Image Receiver
Image Receiver
Two Lights
Two Lights
( Other
Gound
Components)
( Other
Gound
Components)
Sign Over
Sign Over
Identify
Identify
Entry
Entry
Building
Building
Building and an
Building and an
Entry Point Found
Entry Point Found
1m
Vehicle or
Vehicle or
Subvehicle
Subvehicle
(s)
(s)
1m
Enter Building
Enter Building
Georgia Tech in 2001
Transmit an Image of Point of Interest
Transmit an Image of Point of Interest
Inside Building
Inside Building
64
IARC Levels

• Level 1 Follow Prescribed Waypoints for 3km -
  Successfully accomplished by Georgia Tech Aerial
  Robotics (GTAR) team with Fixed Wing Aircraft in
  Summer 2001
• Level 2 Locate Building and Find an Entry - Plan
  on using GTMax w/video camera image processing.
  Provides focused identification surveillance
  automation to confirm ID, and maintain target
  identity
• Level 3 Enter the Building 1) Can be a
  different vehicle or sub-vehicle than used above
  2) Can launch near target structure - Plan on
  launching a small glider (Glider may or may not
  include a small ground robot, as a sub-vehicle)
• Level 4 Image Desired Location within Building
  and Transmit Complete in Data Retrieval) - Plan on using inserted glider
  and/or sub-vehicle Also plan on using
  coordinated control with small ground robots that
  can enter the building

65
University-Led Final Experiments
Optically Guided Glider Flown into Opening to act
as Sensor Possibly include sub-robot
Video
GTMAX Mother Ship
Simulated Structure with Opening
Urban, Homeland Defense Information Sensing,
Exploiting, Planning and Execution
66
Potential Scenario Elements

• Takeoff, En-route, Landing GNC
• Image Processing, Locating Target
• Coordination Between Gliderand Helicopter
• Route Planning
• Hazard Avoidance
• Vehicle Failures/Damage
• Aggressive Maneuvering
• VV

67
Detail Scenario (Example)

• VTOL UAV
• Does autonomous takeoff
• Plans and executes route through known obstacles
• Aggressive
• Urban elements (obstacles)
• Locates known target building visually
• Maneuvers to correct location and deploys glider
• Glider hits (goes through) window of building
• Vehicle returns to an autonomous landing at
  desired location after a component failure (e.g.
  stuck tail rotor actuator)

68
Collaborators

• Boeing Integration support and OCP extensions
• Georgia Tech Hybrid controls API, limit
  detection, fault tolerant control, mode
  transition, aggressive maneuvering
• Honeywell RTARM, VV, Trajectory Generation
• Draper Trajectory planning for aggressive
  maneuvers
• Scientific Systems Fault detection, recovery,
  re-planning
• OGI Control algorithms that consider
  environmental factors
• Vanderbilt Fault detection and fault tolerant
  control
• MIT Validation methods
• UMN/UCB/CalTech/Colorado Modeless Control

69
Final Experiment Architectures

• Builds on Baseline Architecture
• Utilize Hybrid Controls API
• Collaborators
• Replace or Improve Controller Component
• Add Fault Detection Component
• Replace Planner Component

70
Baseline OCP Implementation
Controls API Input Port
Controls API Output Port
I/O Component
DataLink Interface
Ethernet Serial Port
Input datalink ports read _at_ 100 Hz
Serial port
ControlData_out
NavData_out
Ethernet Serial Port
1 Hz 10 Hz
m0 written at 10 Hz m1 written at 1 Hz
Serial port
1 Hz 10 Hz
50 Hz
100 Hz
ControlData_in
NavData_in
NavControl_in
50 Hz
50 Hz
NavControl_out
71
Collaboration Roadmap

• Attend Industry/University Workshop this Summer
• Define Collaboration
• Begin Implementation
• Test Utilizing Latest Release of GTMax Simulation
  (SITL, HITL)
• Work With Georgia Tech and Boeing to
• Perform Final HITL Tests on Flight Hardware
• Execute Flight Tests

72
Issues for Discussion

• Adjustments to baseline architecture
• Hardware and software changes needed for GTMax
• Virtual Obstacles
• Interfaces to be defined for collaborators
• Update Baseline to OCP Build 2
• Identify benefits to the warfighter

73
Issues for Discussion

• Relationship to Industry-Led Experiments
• Transition implementation from University-Led to
  Industry-Led
• Schedule for Final Exam Experiments
• Integration Steps
• Flight Testing

74
Project Status Schedule