UK CAA HUMS Research Project

1
UK CAA HUMS Research Project

• Project briefing, August 2005

2
UK CAA HUMS Research Project

• Following a competitive tendering process, in
  April 2004 Smiths Aerospace was awarded an
  important UK CAA contract to demonstrate
  how the effectiveness of helicopter Health and
  Usage Monitoring Systems (HUMS) can be enhanced
  by applying unsupervised learning techniques to
  HUMS data in an anomaly detection system.
• The system is based on a state of the art data
  mining tool developed on the ProDAPS
  (Probabilistic Diagnostic and Prognostic System)
  program.

FLIGHT INTERNATIONAL 22-28 JUNE 2004
3
Overview of CAA HUMS Research Project

• The project aims to enhance the
  effectiveness of HUMS by automatically analysing
  the vibration health monitoring (VHM) outputs in
  an anomaly detection system, based on the ProDAPS
  data mining tool.
• The goal is to provide earlier and better fault
  indications and also to reduce the occurrence of
  false alarms.
• Smiths are working with Bristow Helicopters on
  the project. The data modelling techniques are
  being developed, refined and validated using
  Bristows extensive historical VHM database.
• A demonstration system will be fielded in early
  2006, and Bristow will perform an in-service
  evaluation of the system against conventional VHM
  analysis techniques using vibration data
  downloaded from their North Sea AS332L Super Puma
  helicopter fleet.

4
AS332L MGB bevel pinion fault case

• A CHC Scotia AS332L MGB was removed after a
  gearbox chip warning.
• On the subsequent inspection, a large crack was
  found in the bevel pinion.
• The chip warning occurred purely because a
  secondary crack fortuitously released a fragment
  of material.
• Normally this type of crack would not generate
  any debris.
• The HUMS VHM system (not a Smiths system) did not
  generate any alerts.
• Even if the HUMS had triggered an alert, alerts
  are not uncommon and it would be extremely
  difficult to detect the significance of the
  indicator trends when viewing these individually.
• This fault case was a key factor in the CAAs
  decision to initiate the current HUMS research
  project.

AS332L MGB Bevel Pinion
5
An introduction to ProDAPS

• ProDAPS is a DUST program jointly funded by
  Smiths Aerospace and the US Air Force Research
  Laboratory.
• The ProDAPS program is developing intelligent
  tools and techniques for multiple Information
  Systems applications.
• ProDAPS provides
• AI-based data mining, anomaly detection,informatio
  n fusion, reasoning and decision aiding/action
  planning technology for multiple applications.
• Open architecture tools and software components
  to enable technology insertion into multiple
  legacy and future ground-based and on-board
  platforms.
• Two core tools have been produced
• Data mining tool, incorporating advanced learning
  algorithms.
• Reasoning tool (Probabilistic networks).
• These tools underpin many applications such as
• FDM/FOQA data mining (e.g. FAA demo with British
  Airways)
• HUMS anomaly detection (e.g. CAA HUMS project
  with BHL)
• Engine PHM (e.g. USAF F15 F100-229 engines)
• On-board reasoning (e.g. Boeing fuel system
  model)
• Many others

6
ProDAPS data mining tool

• The data mining tool developed under the ProDAPS
  program uses a powerful and flexible framework
  that extends far beyond the library or component
  utility you would expect from a third party
  software tool.
• This meant we could build a software component to
  sit on top of the framework to facilitate batch
  processing of anomaly models and rapid
  prototyping of new modelling approaches.
• A set of ProDAPS learning algorithms have been
  developed, including a powerful cluster
  algorithm. These are based upon the best
  techniques developed by the academic / industrial
  research community, and are targeted at solving
  practical issues that arise with real-world data.
• The ProDAPS data mining tool has proven to be
  essential to the progress of the CAA HUMS
  research project.
• Its flexibility permits new modelling approaches
  to be rapidly prototyped.
• The cluster functionality is key to tackling some
  of the difficult data issues.
• The tool has some advanced model diagnostic
  capabilities (facility to extract information
  about complex models).

7
Anomaly detection

• Used for all kinds of applications
• The underlying theme is that there is no large
  library of tagged data with which to train a
  model.
• E.G. in the case of HUMS, there is no library of
  HUMS historical data tagged with known faults.
• Conceptually simple
• Build a model of normal behaviour.
• For a new sample, assess its fit against this
  model.
• If the fit is not within a models threshold then
  flag it as anomalous.
• Nearly all approaches assume a set of normal data
  is available to construct a model of normal
  behaviour.
• Anomaly detection is usually difficult but HUMS
  data present significant additional challenges.
• Gearboxes tend to occupy their own space of
  normality (e.g. vibration levels vary between
  gearboxes).
• For this application, there is no independent
  signal source with which to normalise the data.
• The condition of the training data is unknown
  healthy or not? Due to the lack of feedback from
  gearbox overhauls, we must expect any training
  set to contain some anomalous data.

8
ProDAPS approach

• Pre-process the data to remove outliers, and also
  extract trends.
• Construct unsupervised probabilistic models one
  per shaft per each form of pre-processing
  (absolute values/trends).
• Use these models for anomaly trending.
• For each acquisition and each model output a
  predicted fitness score.
• This score is a single value that is a fusion of
  all indicators used to construct a model.
• A time history of the scores can be plotted.
• The fused score will react to any significant
  change in one or more indicators.
• Overcome the lack of knowledge about the health
  status of training data during predictions (of
  the fitness scores).
• Assume that it is possible to identify traits
  from a society of gearboxes to segment expected
  normal behaviour.
• Segmentation is achieved by turning regions of
  populated model space into barren subspaces.
• Target low support and non-social territory.
• We can also make the segmentation gearbox ID
  dependent and view the model space from the
  perspective of an individual gearbox.

9
Example Results

• An example output is presented from the
  statistical analysis performed to support the
  model investigation.
• Some example results are presented from one
  anomaly modelling approach, and for one form of
  pre-processing (trend model).
• Data presented
• Results are for a single shaft (bevel pinion
  shaft)
• 60 training gearboxes (approx 65000 acquisitions)
• 28 validation gearboxes (approx 35000
  acquisitions)
• Plus CHC Scotia gearbox with bevel pinion fault
  (MGB 999)

10
Example statistical analysis results for SO2
indicator
SO2 (Median Filtered data)
3SDs for Fleet
GB 999 (Scotia)
GB 126
Median, Variance, Max, Min
GB 281
11
Gearbox ID conditional cluster model Fitness
scores for training data Scotia MGB with
cracked bevel pinion
BHL feedback confirmed that the extremely low
fitness scores for gearboxes 126, 156 and 281
were due to sensor problems. The anomaly
detection process clearly identifies these
gearboxes as having unbelievable values, however
one gearbox had actually been rejected before a
sensor fault had been diagnosed.
12
Gearbox ID conditional cluster model Fitness
scores for training data Scotia MGB with
cracked bevel pinion
Repeat of previous plot, but with gearboxes 126,
156 and 281 removed
BHL feedback confirmed that the low fitness score
for gearbox 194 is actually due to an
unidentified gearbox change (an incorrect date
had been given for this), and the variable score
for gearbox 172 is due to a series of maintenance
actions.
13
Gearbox ID conditional cluster model Fitness
scores for validation data set (no known faults
present)
The validation results demonstrate that the
modelling approach is very robust - the fitness
scores for the validation data are higher and
less spread than for the training data this is
consistent with the observation that the
validation set is better behaved.
14
Example of model diagnostics

• Under the ProDAPS program we are implementing
  some advanced modelling diagnostics.
• We can use a set of indicator values to predict
  the expected value of another indicator output
  includes
• Predicted value
• Predicted variance
• Prediction support
• We are currently assessing the influence of
  training attributes on the fitness score.
• Still in its early stages but has the potential
  to reveal more information about a flights fit
  within the model.
• See for example the traces for gearbox 999 (CHC
  Scotia fault data).
• These show that SO2 and ESA_SD have the most
  anomalous trends given the behaviour of the
  remaining indicators.
• Querying the model for similar cases revealed
  that another gearbox that exhibited similar
  trends on GE22 and MS_2, which explains why these
  have not been identified as anomalous.

15
Indicator influence in model output for CHC
Scotia MGB with cracked bevel pinion
Note Charts have different scales
16
CHC Scotia MGB indicator traces
SIG_SD
SIG_PP
SON
SO1
MS2
GE22
ESA_SD
ESA_PP
17
Summary

• A robust HUMS VHM data modelling and anomaly
  detection approach has been defined.
• Testing has shown that this approach can clearly
  detect the cracked CHC Scotia MGB bevel pinion.
• The approach does not require the frequent
  re-datuming that is needed by some HUM systems to
  achieve acceptable performance.
• The approach can provide reliable alerts.
• Results to date indicate that the modelling
  approach will not trigger high numbers of false
  positive alerts.
• Equally importantly, the approach provides
  valuable information on the degree of abnormality
  to support the maintenance decision making
  process (e.g. should a component be rejected?).