DIFG_TARA:1

1
Data Information Fusion Status of US Data and
Information Fusion
Briefing to FUSION 2000 10 July 2000 Paris France
2
Outline

• Motivation / Benefits
• Fusion Process Model
• Programs
• Army / Navy / Air Force
• Assessment
• Summary

3
Data Fusion Benefits

• Sensor Systems
• Cross-Sensor Cueing
• More efficient search
• Enhanced detection (cued dwells reduced
  threshold)
• LO techniques (passive cueing of radar bistatic
  sensing)
• Combine Multi-Source/Multi-Discipline Information
• Exploit overlapping sensor capabilities
• Improved target estimation location, track,
  identification
• Multi-spectral imaging (e.g. E-OSAR)
• Maintain Track Continuity Correlate
  Time-Separated Observations
• Intermittent sensor passes
• Track through drop-outs

Consistent tailored views

• Command Systems
• Formation of Consistent Tactical Picture
• Leverage all sensors, sources, organic or
  distributed
• Insensitive to sensor failures, deception
• Enables Information Management
• Flow, Access, Priority of Information
• Assessment of Situation / Threats
• Use of environment and force context
• Blue distribution / vulnerability
• Red capability / intent
• Decision Support Advisories
• Alerts critical events over space, time

Sensors Sources
Targeting
CIS Fusion Processes
Collection requests
4
Data Fusion Transforms Data into Information
and Knowledge
The Key is Seamless FLOW to the Commander - A
Person Our Boxes can become artificial
restrictions to the Flow
Data Information rec'd from the environment
Data
Information
Knowledge
5
Hierarchy of Inference Techniques
Type of Inference
Applicable Techniques
High
- Threat Analysis
- Knowledge-Based Techniques
- Expert Systems- Scripts, Frames, Templating -
Case-Based Reasoning - Genetic Algorithms - etc.
- Situation Assessment
- Decision-Level Techniques
- Neural Nets- Cluster Algorithms - Fuzzy Logic
INFERENCE LEVEL
- Behavior/Relationships of Entities
- Estimation Techniques
- Identity, Attributes and Location of an Entity
- Bayesian Nets - Maximum A Posteriori
Probability (e.g. Kalman Filters, Bayesian), -
Evidential Reasoning
- Existence and Measurable Features of an Entity
Low
- Signal Processing Techniques
Data Fusion provides appropriate automation or
assistance
6
Data Fusion Domain
DATA FUSION DOMAIN
USERS
SOURCES
NATIONAL
DISTRIBUTED
LOCAL






INTEL

EW

SONAR

RADAR
.
.
.
DATA BASE MANAGEMENT SYSTEM

DATA
BASES
FUSION
SUPPORT
DATABASE
DATABASE

7
Multi-Level Fusion Inferencing
8
Data Fusion Definition
A process dealing with the association,
correlation, and combination of data and
information from single and multiple sources to
achieve - refined position and identity
estimates, and - complete and timely
assessments of situation and threats, and
their significance. The process is
characterized by continuous refinements of its
estimates and assessments, and by evaluation of
the need for additional sources, or modification
of the process itself, to achieve improved
results.
9
An Evolving Data Fusion Model Proposed Revisions
(1998)

• Level 0? Sub-Object Data Assessment estimation
  and prediction of signal/feature states on the
  basis of pixel/signal level data association and
  characterization
• Level 1? Object Assessment estimation and
  prediction of entity states on the basis of
  observation-to-track association, continuous
  state estimation (e.g. kinematics) and discrete
  state estimation (e.g. target type and ID)
• Level 2 ? Situation Assessment estimation and
  prediction of relations among entities, to
  include force structure and cross force
  relations, communications and perceptual
  influences, physical context, etc.
• Level 3 ? Impact Assessment estimation and
  prediction of effects on situations of planned or
  estimated/predicted actions by the participants
  to include interactions between action plans of
  multiple players (e.g. assessing susceptibilities
  and vulnerabilities to estimated/predicted threat
  actions given ones own planned actions)
• Level 4 ? Process Refinement (an element of
  Resource Management) adaptive data acquisition
  and processing to support mission objectives

10
Commercial Applications
Intelligent Agent and Data Mining tools Use of
mining and data warehousing in distributed market
analysis - e.g. Retail and internet sales,
Credit card usage Wide variety of pattern
discovery applications e.g. Link analysis,
behavior recognition RD in Automotive Sensor
Fusion using onboard radar and imaging sensors
for situation awareness and smart highway
environments Adaptive cruise control, lane
maintenance, sensor-aided navigation, parking
aids, etc. Multi-sensor ATR-like tools used in
medical diagnosis
11
Summary Data Fusion Technology Assessment (1 of 2)
DATA FUSIONLEVEL
SUMMARY OF THE STATE OF THE ART
DESIRED NEAR TERM CAPABILITIES
CURRENT LIMITATIONS

• Off-the-shelf software package for robust
  estimation
• Multi-technique approach for object I/D
• Methodology guidelines for algorithm
  selection
• Standard test beds, data sets
• Metrics - MOPs/MOEs

Level 1 Positional, Kinematic, Attribute
Estimation
Relatively mature- numerous tracking
techniques - current research in IMM, MHT,
JPDA trackers Object I/D fusion dominated by
feature decision methods- Advances in
model-based ATR - RD in NN syntactic
methods - Efficient Near-Optimal Search
Techniques enhance robustness

• Difficulty tracking targets in dense target
  environment, low SNR, maneuvering targets
• Selection of features for classification
• Selection/use of multiple techniques in concert
  
• Maintaining robust target models

Levels 2 and 3 Situation and Threat Assessment

• Robust techniques to solve subset of situation/
  threat refinement
• Exploitable cognitive models
• Recognition of sparse indicators

Immature - Fielded systems use heuristic
techniques include templating, expert systems -
numerous experimental prototypes
Doctrinal Basis not well-defined Translation
of decision makers needs to fusion
requirements Automated reasoning
techniques Human behavior models
Level 4 Process Refinement
Very immature - Available technology founded
on experience of single sensors - immature for
multi-sensors- Some MOPs defined
Metrics not well-defined Disconnect between
lab/experiment and real world capability Hybrid
architectures challenging

• MOE/MOP Consensus
• Metrics baseline
• Structure and techniques for multi-sensor
  control
• Mechanisms for imparting trust in automation

12
Summary Data Fusion Technology Assessment (2 of 2)
DATA FUSIONFUNCTIONAL AREA
SUMMARY OF THE STATE OF THE ART
DESIRED NEAR TERM CAPABILITIES
CURRENT LIMITATIONS
Human-Computer interface (HCI)

• Numerous tools for rapid prototyping
• Current research in display design, crew
  position layout, workload aspects (ergonomic vice
  cognitive focus
• Major commercial interest in Agents Data Mining

Limited HCI research specific to data fusion
Limited cognitive models for focus of
attention, stress management, alternative
decision styles

• Integrated exploitation of advanced
  presentation technology
• Intelligent Groupware
• Interactive Data Fusion/Data Mining tools
• Multi-person HCI

Data Base Management
Numerous commercial tools (relational models)
Fourth-generation query languages Trend
toward object-oriented DBMS
Simultaneous optimization of storage and
retrieval Distributed concurrency
Multi-level security
Natural language interfaces S/W based
solution to multi-level security COTS DBMS to
handle diverse data (image, text, data, KBS)
Development Environment
Robust development standards and procedures for
conventional systems Widespread development of
application specific prototypes Single vs.
multi-sensor models
Lack of Standard MOPs and test sets Disjoint
test beds and simulation tools Limited
tools/MOE for Level 2,3 fusion
Robust test-bed for Test and Evaluation
Metrics for MOP/MOE Fusion Software Library
and Clearinghouse Data Fusion System
Engineering methodology
13
Summary

• A Good Baseline -
• Significant progress has been made in Level 1
  fusion technologies - Correlation, tracking,
  feature aided classification, etc.
• Increased attention is being paid to higher
  levels of fusion capability - Very little
  fielded capability is evident
• Whats Needed -
• Technology architecture developments
• Theoretical foundations - common uncertainty
  calculus
• Reasoning algorithms development - automated
  situation understanding
• Seamless access to databases, stovepipes, etc.
• Collaborative networking
• Infrastructure to accelerate operational
  introduction by providing
• A common environment for coordination and
  communication - Library of tools Mapping
  of problems to solution sets
• Standardization and interoperability support to
  key technologies - Models Metrics
• An environment for comparing benefits of
  alternative techniques
• Performance criteria for transition of products

14
BACK-UPS
15
Whats Needed?

• Technology Architecture that enables -
  seamless access to databases, stovepipes, etc. -
  collaborative networking - common situation
  understanding - integration of Intel and
  Operations
• Infrastructure to provide common services, to
  promote standardization and leverage developing
  technologies - clearinghouse functions -
  software, models, data sets, etc. - information
  source to support management choices,
  design and acquisition tradeoffs, performance
  verification.

Data Fusion underlies the ability to manage
information Data Fusion is needed to fulfill
these functions
16
Needed Technology to Resolve Deficiencies in
Data Fusion Systems

• EFFECTIVENESS
• Performance Lack of timely, accurate target
  situation awareness
• Focus Information not tailored to
  decision-makers needs
• User Confidence Cant assess information
  quality
• Interoperability Legacy systems cant talk to
  one another
• Data Exploitation Reported data doesnt
  include some types of useful data
• AFFORDABILITY
• Every new system is designed from scratch

17
Key Technology Needs (A Sampling)

• Theoretical Foundations
• - General Theory of Data Fusion
• - Canonical Forms for Fusion Processes
• - Linguistic Algebra
• - Cognitive Models
• Reasoning Systems Development
• - Spatial Temporal Reasoning
• - Machine Reasoning for Situation Assessment
• - Automated Template Authoring
• Data and Knowledge Bases for Fusion Processing
• - Spatial-Object Oriented DBMS
• - Natural Language Interface Support for
  Decision Maker
• Algorithm Model Development
• - Library of Tools
• - Mapping of Problems to Solution Sets
• - Exploitation of Parallelism

18
Needed An Underlying Framework for Data Fusion
Development - An Infrastructure

• To accelerate operational introduction of data
  fusion by providing
• A common environment for coordination and
  communication
• Standardization and interoperability support
  to key technologies
• An environment for comparing benefits of
  alternative techniques
• Performance criteria for transition of
  products
• Related Elements are Coming into Focus Although
  None are Complete. For example
• TCP/IP Networks and Services (EGINTELINK) are
  Providing an Architectural Framework for
  Information Sharing Including Intelligence
• DII COE and GCCS have the Potential for Creating
  a Common Development and Operating Environment
• Large B/W Comms (e.g. GBS) can move massive
  amounts of data

19
Elements of the Infrastructure
Simulation / Testbed - Interoperable, Portable,
Non-Monolithic Standardized Evolvable
Operating Systems, Networks, Modular H/W and
S/W Library of Numerical Techniques,
Canonical Targets, Geometries, and
Contexts Models - Library of Models for
Sensors, Sources, Platform Behavior Media,
Propagation, and Process Phenomenology Metrics -
Evaluation Measures Relevant to Operations -
Measures of Effectiveness, Engineering -
Measures of Performance Real World Data Sets -
Benchmark Data for Qualification Testing
20
Infrastructure Supports These Technical Areas

• Matching Fusion Requirement Derived From a
  Mission Requirement to Appropriate Technology or
  Systems
• Algorithm Development, Analysis, Evaluation
• Database Development and Evaluation
• Theoretical Performance Calculations
• Fusion Test Beds Within DoD, Industry, and
  Academia
• Operational Utility Analysis (MOEs and MOFEs)
• Counter Measure and CCM Analysis and Evaluation
• Identification of Shortfalls