Leveraging Semantic Web for Situational Awareness

1
(No Transcript)
2
Leveraging Semantic Web techniques to gain
situational awareness Can Semantic Web
techniques empower perception and comprehension
in Cyber Situational Awareness? Talk at Cyber
Situational Awareness Workshop, Fairfax, VA Nov
14-15, 2007.
Amit Sheth LexisNexis Ohio Eminent
Scholar Kno.e.sis Center Wright State
University http//knoesis.wright.edu Thanks
Cory Henson and Sensor Data Management team (M.
Perry, S. Sahoo)
3
Outline

• Situational Awareness (SA)
• SA within the Semantic Web
• Situation Awareness (SAW) Ontology
• Sensor Web Enablement
• Provenance Context
• Spatial-Temporal-Thematic Analysis

4
Situation Awareness

• Situation awareness is the perception of
  elements in the environment within a volume of
  time and space, the comprehension of their
  meaning, and the projection of their status in
  the near future.
• (1988, Mica Endsley).

http//en.wikipedia.org/wiki/Situation_awareness
5
JDL Data Fusion Model
A. Steinberg, et al., Rethinking the JDL Data
Fusion Levels
6
JDL Data Fusion Model
Level 0 Signal/Feature Assessment pixels,
signals Level 1 Entity Assessment
object-event identification and tracking Level
2 Situation Assessment relational analysis of
objects-events Level 3 Impact Assessment
threat intent estimation and consequence
prediction Level 4 Performance
Assessment resource management, adaptive
search and processing
M. Kokar, Ontology Based High Level Fusion and
Situation Awareness Methods and Tools
7
Endsleys Model
M. Kokar, et al., Ontology-based Situation
Awareness
8
Endsleys Model

• Perception
• involves monitoring and simple recognition
• produces Level 1 SA, an awareness of multiple
  situational elements (objects, events, people,
  systems, environmental factors) and their current
  states (locations, conditions, modes, actions).
• Comprehension
• involves pattern recognition, interpretation
  and evaluation
• produces Level 2 SA, an understanding of the
  overall meaning of the perceived elements - how
  they fit together as a whole, what kind of
  situation it is, what it means in terms of one's
  mission goals.
• Projection
• involves anticipation and mental simulation
• produces Level 3 SA, an awareness of the likely
  evolution of the situation, its possible/probable
  future states and events. This is the highest
  level of SA.

http//en.wikipedia.org/wiki/Situation_awareness
9
Endsleys Model w/ Semantics

• Semantic Analysis
• thematic
• Spatio-Temporal
• trust

Relate Situation Entities
Identify Situation Entities
Provenance
Collect Relevant Data
M. Kokar, et al., Ontology-based Situation
Awareness (Modified Figure)
10
Data Pyramid
Situation Awareness Data Pyramid
Semantics/Understanding/Insight
Relationship Metadata (Comprehension)
Expressiveness
Information
Entity Metadata (Perception)
Data
Sensor Data (World)
11
Situation Awareness

• Situation Awareness Components
• Physical World Sensor Data
• Perception Entity Metadata
• Comprehension Relationship Metadata
• Semantic Analysis
• How is the data represented? Sensor Web
  Enablement
• What are the antecedents of the event?
  Provenance Analysis
• Where did the event occur? Spatial
  Analysis
• When did the event occur? Temporal
  Analysis
• What is the significance of the event?
  Thematic Analysis

12
Sensor Web Enablement
13
Open Geospatial Consortium
OGC Mission To lead in the development,
promotion and harmonization of open spatial
standards

• Consortium of 330 companies, government
  agencies, and academic institutes
• Open Standards development by consensus process
• Interoperability Programs provide end-to-end
  implementation and testing before spec approval
• Standard encodings, e.g.
• GeographyML, SensorML, Observations
  Measurements, TransducerML, etc.
• Standard Web Service interfaces, e.g.
• Web Map Service
• Web Feature Service
• Web Coverage Service
• Catalog Service
• Sensor Web Enablement Services (Sensor
  Observation Service, Sensor Alert Service, Sensor
  Process Service, etc.)

http//www.opengeospatial.org/projects/groups/sens
orweb
14
Sensor Web Enablement
Vast set of users and applications
Constellations of heterogeneous sensors
Satellite
Airborne
Sensor Web Enablement
Weather
Surveillance

• Distributed self-describing sensors and related
  services
• Link sensors to network and network-centric
  services
• Common XML encodings, information models, and
  metadata for sensors and observations
• Access observation data for value added
  processing and decision support applications
• Users on exploitation workstations, web browsers,
  and mobile devices

Network Services
Biological Detectors
Chemical Detectors
Sea State
http//www.opengeospatial.org/projects/groups/sens
orweb
15
SWE Languages and Encodings
Sensor and Processing Description Language
Information Model for Observations and Sensing
Observations Measurements (OM)
SensorML (SML)
GeographyML (GML)
Common Model for Geography Systems and Features
Multiplexed, Real Time Streaming Protocol
Sam Bacharach, GML by OGC to AIXM 5 UGM, OGC,
Feb. 27, 2007.
16
SWE Components Web Services
Access Sensor Description and Data
Command and Task Sensor Systems
Dispatch Sensor Alerts to registered Users
Sam Bacharach, GML by OGC to AIXM 5 UGM, OGC,
Feb. 27, 2007.
17
Semantic Sensor ML Adding Ontological Metadata
Situation Awareness Ontology
Event
Situation
Domain Ontology
Company
Person
Spatial Ontology
Coordinates
Coordinate System
Temporal Ontology
Time Units
17
Mike Botts, "SensorML and Sensor Web Enablement,"
Earth System Science Center, UAB Huntsville
Timezone
18
Situation Awareness Ontology
19
Ontology

• What is an Ontology?
• Ontology is about the exact description of
  things and their relationships.
• World Wide Web Consortium (W3C)

20
Situation Awareness Ontology

• Core SAW Ontology
• Provides a framework from which to build
  ontologies for arbitrary situations
• Represents objects and relationships as well as
  their evolution over time
• Captures sufficient information about a situation
  to support high-level reasoning
• Economical design permits its implementation in a
  working system

C. Matheus, et al., A Core Ontology for Situation
Awareness
21
Situation Awareness Ontology
C. Matheus, et al., An Application of Semantic
Technologies to Situation Awareness
22
Provenance Context
23
Provenance

• What is Provenance?
• The recording of details in a data process
  workflow
• Trace back to where the particular data entity
  originated
• The phenomena captured by the sensor
• The sensor characteristics associated with data
• What processing was done on data
• Enables effective interpretation of object or
  event - Trust
• Evaluate whether particular data entity is
  relevant in current situation based on its
  provenance
• Enhanced situation comparison through use of
  provenance

24
Semantic Provenance Context for Situation
Awareness

• Use of provenance associated with data to
  evaluate if situation awareness is correct
• Situation analysis utilizes Provenance Context
• Provenance Context components
• Type of Data
• Type of Process
• Type of Agent (e.g., Sensor)
• Provenance Context components defined in Ontology
• Well defined formal semantics
• Machine processable
• Scalable to large data sets

25
Spatial, Temporal, Thematic Analysis
26
Three Dimensions of Information
Temporal Dimension When
Thematic Dimension What
North Korea detonates nuclear device on October
9, 2006 near Kilchu, North Korea
Spatial Dimension Where
27
Where we are, where we need to go

• Semantic Analytics
• Searching, analyzing and visualizing semantically
  meaningful connections between named entities
• Significant progress with thematic data
• Semantic associations (Rho-Operator)
• Subgraph discovery
• Query languages (SPARQ2L, SPARQLeR)
• Data stores (Brahms)
• Spatial and Temporal data is critical in many
  analytical domains
• Need to support spatial and temporal data and
  relationships

28
Current Research Towards STT Relationship Analysis

• Modeling Spatial and Temporal data using SW
  standards (RDF(S))1
• Upper-level ontology integrating thematic and
  spatial dimensions
• Use Temporal RDF3 to encode temporal properties
  of relationships
• Demonstrate expressiveness with various query
  operators built upon thematic contexts
• Graph Pattern queries over spatial and temporal
  RDF data2
• Extended ORDBMS to store and query spatial and
  temporal RDF
• User-defined functions for graph pattern queries
  involving spatial variables and spatial and
  temporal predicates
• Implementation of temporal RDFS inferencing

• Matthew Perry, Farshad Hakimpour, Amit Sheth.
  "Analyzing Theme, Space and Time An
  Ontology-based Approach", Fourteenth
  International Symposium on Advances in Geographic
  Information Systems (ACM-GIS '06), Arlington, VA,
  November 10 - 11, 2006
• Matthew Perry, Amit Sheth, Farshad Hakimpour,
  Prateek Jain. Supporting Complex Thematic,
  Spatial and Temporal Queries over Semantic Web
  Data", Second International Conference on
  Geospatial Semantics (GeoS 07), Mexico City, MX,
  November 29 30, 2007
• Claudio Gutiérrez, Carlos A. Hurtado, Alejandro
  A. Vaisman. Temporal RDF, ESWC 2005 93-107

29
Upper-level Ontology modeling Theme and Space
Continuant
Occurrent
Dynamic_Entity
Named_Place
Spatial_Occurrent
located_at
occurred_at
Spatial_Region
rdfssubClassOf property
Occurrent Events happen and then dont
exist Continuant Concrete and Abstract Entities
persist over time
occurred_at Links Spatial_Occurents to their
geographic locations located_at Links
Named_Places to their geographic locations
Named_Place Those entities with static spatial
behavior (e.g. building) Dynamic_Entity Those
entities with dynamic spatial behavior (e.g.
person)
Spatial_Region Records exact spatial location
(geometry objects, coordinate system info)
Spatial_Occurrent Events with concrete spatial
locations (e.g. a speech)
30
Upper-level Ontology
Continuant
Occurrent
Named_Place
Dynamic_Entity
located_at
occurred_at
Spatial_Occurrent
Spatial_Region
City
Person
Speech
trains_at
gives
Politician
participates_in
Military_Unit
Bombing
Military_Event
Soldier
assigned_to
on_crew_of
rdfssubClassOf used for integration rdfssubC
lassOf relationship type
Battle
used_in
Vehicle
Domain Ontology
dynamic entities get spatial properties
indirectly through relationships with spatial
entities
31
Sample STT Query

• Scenario (Biochemical Threat Detection) Analysts
  must examine soldiers symptoms to detect
  possible biochemical attack
• Query specifies
• a relationship between a soldier, a chemical
  agent and a battle location (graph pattern 1)
• a relationship between members of an enemy
  organization and their known locations (graph
  pattern 2)
• a spatial filtering condition based on the
  proximity of the soldier and the enemy group in
  this context (spatial Constraint)

32
Using SW to enable perception and comprehension
Utilizing Semantic Web technologies to enable
perception and comprehension within Situational
Awareness

• Perception
• Leveraging current research in sensor data
  representation found in the Sensor Web Enablement
  metadata languages
• Using SWE languages to model sensors, processes,
  and data
• Comprehension
• Extending the Sensor Web Enablement languages
  with semantic metadata to provide the ability to
  model relationships between entities
• Semantic relationships provide meaning to
  objects and events within a situation
• Using Situational Awareness Ontology to model
  situations and provide a framework for Semantic
  Analysis
• Provenance Context provides a historical record
  of relevant objects and events within a situation
• Spatial, Temporal and Thematic analysis provides
  the where, when, and what of objects and
  events within a situation

33
References

• C. Matheus, M. Kokar and K. Baclawski, A Core
  Ontology for Situation Awareness, Sixth
  International Conference on Information Fusion,
  pp.545-552, Cairns, Australia, July 2003
• C. Matheus, M. Kokar, K. Baclawski and J.
  Letkowski, An Application of Semantic Web
  Technologies to Situation Awareness, 4th
  International Semantic Web Conference, ISWC 2005,
  Galway, Ireland, November, 2005
• M. Kokar, C. Matheus and K. Baclawski,
  Ontology-based situation awareness, Informat.
  Fusion, 2007, doi10.1016/j.inffus.2007.01.004
• M. Kokar, Ontology Based High Level Fusion and
  Situation Awareness Methods and Tools,
  Presentation, Quebec, 2007
• A. Steinberg and C. Bowman, Rethinking the JDL
  data fusion levels, National Symposium on Sensor
  and Data Fusion, 2004
• Wikipedia, Situation Awareness,
  http//en.wikipedia.org/wiki/Situation_awareness
• Open Geospatial Consortium, Sensor Web Enablement
  WG, http//www.opengeospatial.org/projects/groups/
  sensorweb
• Sam Bacharach, GML by OGC to AIXM 5 UGM, OGC,
  Feb. 27, 2007.