MDA DS COI Pilot

1
MDA DS COI Pilot

• DoD CIO Quarterly COI Forum
• 12 October 2006

John Shea Technical Director PEO C4I, PMW180
ISR/IO NMCI       john.shea_at_navy.mil
2
Presentation Outline

• MDA Data Sharing CoI introduction
• Pilot Overview
• Data Management WG presentation
• Pilot Implementation and Demonstration
• Federated Search
• Messaging Services
• QA

3
MDA DS CoI Introduction

• MDA - Maritime Domain Awareness
• Effective knowledge of all activities associated
  with the global maritime environment that could
  impact the security, safety, economy, or
  environment of the United States.
• Specific knowledge of Vessels, Cargo and People
• Vessels coming into the United States are
  required to submit a 96-hour of Arrival Notice to
  the USCG
• On average 400 vessels per day carrying
  thousands of cargo containers

4
MDA DS CoI Introduction
12 Oct
Kickoff MDA COI 21-22 Feb 06
Pilot Selected 31 Mar 06
Pilot Demo Oct 06
2nd MDA Meeting 13 Jul 06
Pilot UDOP Selection Aug 06
1st Anniversary 22 Feb 07
Operational
Pilot 3-way NCES Discovery Aug 06
Pilot 3-way NCES Messaging Sep 06
Define Future Spirals
WGs Stood up 4 Mar 06
Common Vocabulary Delivered 24 May 06
Funding 5 Jul 06
5
MDA DS CoI Introduction

• A collaborative group of people that must
  exchange information in pursuit of its shared
  goals, interests, missions, or business processes
  and therefore must have a shared vocabulary for
  the information it exchanges DOD Directive
  8320.2

Visible
Accessible
Data Sharing in a Net-Centric Department of
Defense
Understandable
6
Purpose of the MDA DS COI

• To establish an MDA information-sharing
  capability, employing net-centric applications
  and services, among the cadre of MDA stakeholders
• The MDA DS COI focuses on creating a data
  standard supporting the net-centric information
  sharing across the full spectrum of MDA
  stakeholders culminating in the visibility,
  accessibility, and understanding of data on a
  User Defined Operational Picture (UDOP)

SOA
Community Of Interest
Net Centricity
Info Sharing
Formed in February 2006
7
MDA DS CoI Governance
8
MDA DS CoI Pilot Charter

• MDA DS COI Pilot Charter
• Develop a repeatable process/capability to
  demonstrate MDA COI Services and products (using
  AIS data), by leveraging Net-Centric Enterprise
  Services (NCES) within a risk reduction pilot
  scheduled for Oct 2006.

Stated in February 2006
9
MDA DS Pilot Implementation Plan

• MDA DS COI Pilot Implementation Plan
• Demonstrate a global UNCLAS MDA Data Sharing
  net-centric capability based initially on
  multiple Automatic Identification System (AIS)
  data producers adopting a common (MDA Community)
  vocabulary and schema.
• Utilize the DHS Homeland Security Information
  Network (HSIN) and DoD Net-Centric Enterprise
  Services (NCES) Early Capability Baseline (ECB)
  to offer Data Producers and Consumers a
  single/common seamless methodology for exposing,
  discovering, publishing and subscribing to UNCLAS
  MDA data.

10
MDA DS Pilot Implementation Plan

• MDA DS COI Pilot Implementation Plan
• Initially identified three AIS Data Producers
  now four
• Navy (USN) Shipboard Organic
• Coast Guard (USCG) OSC
• Office of Naval Intelligence (ONI)
• DoT / VOLPE Center
• To implement NCES Federated Search and Messaging
  Services, according to the current NCES Early
  Capabilities Baseline (ECB) standards.

11
MDA DS Pilot Implementation Plan

• MDA DS COI Pilot Implementation Plan
• Select a representative cross section of User
  Defined Operation Picture (UDOP) - visualization
  tools to view AIS Pilot data
• iMapData Viewer on HSIN (pre-selected by HSIN)
• Google Earth on HSIN or NIPRnet
• TV32 on Shipboard ISNS
• Implementation methods are reusable / sharable
  (located on the HSIN MDA Portal)

12
MDA DS Pilot Implementation Plan

• The Pilot Implementation plan was expressed into
  two Demonstration Use Cases.
• The ability to
• Discover Data made visible and understandable on
  the GIG by data producers and existence
  discovered by data consumers
• Access Data made accessible on the GIG by data
  producers and accessed by data consumers

Data Sharing in a Net-Centric Department of
Defense
13
High-Level Pilot Architecture
NAIS Aggregation
Navy Organic AIS Aggregation
Infrastructure
COI
MDA DS CoI Data Producers
ONI - AMRS Aggregation
NCES SOA Foundation
Content Discovery
Navy Afloat
NAIS
VOLPE Aggregation
DHS Consumers
AMRS
Volpe
DOD Consumers
NCES SS authenticates HSIN portal authorizes
DHS user
NCES Security Service
DOD
DHS
DHS user logs into HSIN portal
Defense Online Portal
HSIN Portal
HSIN Identity Store (Portal Authentication)
NCES Service Discovery
Data Sharing
SOA Foundation
HSIN maintain their own Identity Store Acting
as an Enterprise COI
14
Data Management WG

• Key enabler was the vocabulary, schema and
  process developed by the MDA DS COI Data
  Management WG

15

• Data Management Working Group (DMWG) Presentation
• The DOD Net-Centric Data Strategy
• 5-Step Repeatable Process

Brian Freeman DMWG The MITRE Corporation (719)
572-8240 bfreeman_at_mitre.org
16
Net Centric Data Strategy Goals
17
Vocabulary Based On

• Needs of MDA DS COI stakeholders
• And the existing schemas within the organizations
  of the stakeholders
• Initiatives interested in our activities
• International Maritime Organization (IMO)
  specification for Automatic Identification System
  (AIS)
• Objectives of the MDA DS COI pilot
• Future plans to accommodate data fusion and
  aggregation

18
Vocabulary Development Process
UNCLASSIFIED
Auto Generate XSD - XML
Develop UML Use-Case
Class Relationship Diagram
Vocabulary Handbook
Determine Pilot Demonstration
UNCLASSIFIED
19
UML Use-Case
20
Class Relationship Diagram
21
XML Schema Definition Instances
ltxsdelement name"heading"gt ltxsdannotationgt
ltxsddocumentationgtHeading is measured with
respect to true north. Measured in degrees (0
lt heading lt 360 ) lt/xsddocumentationgt
lt/xsdannotationgt ltxsdsimpleTypegt
ltxsdrestriction base"xsddecimal"gt
ltxsdfractionDigits value"1"/gt
ltxsdmaxExclusive value"360"/gt
ltxsdminInclusive value"0"/gt
lt/xsdrestrictiongt lt/xsdsimpleTypegt lt/xsdeleme
ntgt
22
XML Schema Definition Instances

•   lt?xml version"1.0" encoding"UTF-8" ?gt lt!--
  Sample XML file generated by XMLSpy v2005 U
  (http//www.xmlspy.com)  --gt
• ltdmwgMessage xmlnsdmwg"http//some-dod-dhs-name
  space.mil/" xmlnsism"urnusgovicismv2"
• ismreleasableTo"USA" ismclassification"U"
  ismownerProducer"GBR" ismdisseminationControls
  "FOUO" releasableToDepartment"DHS"gt
•   ltversiongt0.1lt/versiongt
•   lttimeOrigingt2001-12-17T093047.0Zlt/timeOrigingt
  
•   lttimeReceiptgt2001-12-17T093049.0Zlt/timeReceipt
  gt
• - ltcollectorgt
•   ltDataSourcegtAMRSlt/DataSourcegt
•   ltreportStationIDgt54a2lt/reportStationIDgt
•   lt/collectorgt
• - ltconveyance xsitype"dmwgVessel"gt
• - lttimegt
•   ltstartTimegt2001-12-17T093047.0Zlt/startTimegt
•   ltendTimegt2001-12-17T093047.0Zlt/endTimegt
•   lt/timegt
• - ltlocation locationAttribute"isAtLocation"gt
•   ltlatitudegt26.158lt/latitudegt
•   ltlongitudegt80.1835lt/longitudegt
•   lts-minorgt10lt/s-minorgt

Classification Markings
Collector
Vessel
Time, Location, Vector, Heading, Rate of Turn
Message
Conveyance
AIS Transponder
Transponder
23
Vocabulary Handbook

• Participants
• Objectives
• Assumptions
• Organizational Structure
• Schemas
• Supporting Documentation
• Diagrams
• Lesson Learned

Accessible on the HSIN MDA Portal
24
DOD Metadata Registry
https//metadata.dod.mil
25
DMWG Summary

• Involved numerous Maritime stakeholders
• Developed a repeatable process
• Maintained close coordination with stakeholders
  and met regularly
• Developed vessel position schema/vocabulary for
  MDA DS COI Spiral 1 Pilot in 3 months
• Updates already being incorporated into spiral 2
• Registered MDA DS COI vocabulary in DOD Metadata
  Registry
• Positioned for Spiral 2

26

• Pilot Demonstration
• Federated Search (Discovery)
• Messaging Services (Access)

Jason DeShano Pilot Technical Lead DoD CIO
(BAH) [phone] deshano_jason_at_bah.com
Trey Rhiddlehoover Pilot Technical Team DoD CIO
(Solers) [phone] trey.rhiddlehoover_at_solers.
com
Adam White Pilot Technical Team DoD CIO
(Solers) [phone] adam.white_at_solers.com
27
Pilot Network Layout
HSIN Portal NOC-Washington DC
Navy Organic Data Source SSC San Diego
NIPR
HSIN
Security Services SPAWAR, Charleston, SC
Internet
Volpe Data Source Volpe-Boston, MA
AMRS Data Source ONI, Suitland MD
Test Portal
Messaging Services
NAIS Data Source OSC Martinsburg, WV
Content Discovery Services
Solers, Arlington, VA
28
MDA DS Pilot Implementation Plan

• The Pilot Implementation plan was expressed into
  two Demonstration Use Cases.
• The ability to
• Discover Data made visible and understandable on
  the GIG by data producers and existence
  discovered by data consumers
• Access Data made accessible on the GIG by data
  producers and accessed by data consumers

Data Sharing in a Net-Centric Department of
Defense
29
Overview of Discovery Use Case

• Demonstration of Discovery Use Case
• Authorized User discovers the existence of AIS
  content made visible on the GIG by MDA DS CoI
  Data Producers
• Leverage the NCES Federated Search (FS)
  Specification
• Returns DoD Discovery Metadata Specification
  (DDMS)-type metadata describing the underlying
  AIS data
• Leverage MDA DS COI Data Management Working Group
  (DMWG)-defined DDMS profile for mapping metadata
  to data resources
• Extend DDMS metadata with specific COI metadata
  tagging (any COI-extensions defined by DMWG)
• Utilize the Federated Search WebPart for
  SharePoint
• Standard and advanced web queries
• DDMS results displayed in a browser (can be
  further integrated into applications later)

30
Federated Search Use Case
Fed Search Query Web Part

• Example Input AIS Provider

Data Provider
Fed Search Aggregator
FS

• DDMS DoD Discovery Metadata Specification
• Specifies a set of information fields to
  describe any data or service asset that is made
  known to the Enterprise
• Discovery is the ability to locate data
  assets through a consistent and flexible search

• Fed Search Processing Order of Precedence
• Check for static keyword match. If match,
  return metadata results.
• If no match on static keyword, return no
  metadata results

Fed Search Results Web Part

• Output DDMS record that describes the AIS
  Provider, a URL Message Service (Subscription)
  Instruction Page, and a list of channels the
  provider publishes to.

31
Federated Search Demonstration
32
Discovery Pilot Architecture
NAIS Contributors
Navy Organic AIS Aggregation
Infrastructure
COI
MDA DS CoI Data Producers
ONI - AMRS Contributors
NCES SOA Foundation
Content Discovery
NAIS
VOLPE Aggregation
DHS Consumers
AMRS
Federated Search Requests
4
DOD Consumers
Interfaces
3
NCES SS authenticates HSIN portal authorizes
DHS user
Federated Search Web Service
FS
Federated Search Aggregator
Federated Search Aggregator
Query FSA and Display Results (one of the
metadata attributes in each AIS provider result
contains the Messaging channels to which that
provider is publishing)
2
5
User initiates Fed Search query
Aggregated Search Results Returned in DDMS format
NCES Security Service
DOD
DHS
1
DHS user logs into HSIN portal
Fed Search WebPart
Fed Search WebPart
Defense Online Portal
HSIN Portal
Fed Search Results
HSIN Identity Store (Portal Authentication)
FederatedSearch
NCES Service Discovery
RESULTS
Data Sharing
SOA Foundation
Pilot Notional Use Case 1 DHS User
Sequence Metadata Discovery - Federated Search
(FS)
33
Federated Search Results
34
Navy AIS Federated Search Tasks
35
Fed Search Configuration Set Up Tasks
1. Implement NCES Federated Search Web
Service 2. Implement NCES Security handler with
web service 3. Obtain JITC PKI Server Cert 4.
Add JITC PKI Server Cert to NCES Security
Services 5. Add users and roles to NCES LDAP and
HSIN Identity Store 6. Register Data Source with
Fed Search Aggregator 7. Register Data Source
with NCES UDDI
Infrastructure
MDA DS CoI Data Producers
NCES SOA Foundation
1
Content Discovery
NAIS
AMRS
2
3
DHS Consumers
Navy Afloat
Volpe
DOD Consumers
Interfaces
Federated Search Web Service
4
FS
Federated Search Aggregator
6
5
NCES Security Service
DOD
DHS
5
Fed Search WebPart
7
Fed Search WebPart
Defense Online Portal
HSIN Portal
HSIN Identity Store (Portal Authentication)
NCES Service Discovery
Data Sharing
SOA Foundation
Configuration Set Up Tasks
36
MDA DS Pilot Implementation Plan

• The Pilot Implementation plan was expressed into
  two Demonstration Use Cases.
• The ability to
• Discover Data made visible and understandable on
  the GIG by data producers and existence
  discovered by data consumers
• Access Data made accessible on the GIG by data
  producers and accessed by data consumers

Data Sharing in a Net-Centric Department of
Defense
37
Overview of Access Use Case

• Demonstration of Access Use Case
• Authorized User accesses AIS content made visible
  on the GIG by MDA DS CoI Data Producers
• Leverage the NCES Messaging Specification and
  Messaging CES
• Delivers MDA Content (AIS data) in the common COI
  Vocabulary and Schema
• Uses the publish/subscribe paradigm (asynchronous
  delivery)
• Data consumers subscribe to Messaging CES to
  specifically receive MDA content (access control
  subscription only possible with proper
  authorization to the specific data)
• Data producers publish data to the Messaging CES
  (messaging bus)
• Messaging CES delivers published data to all
  registered subscribers asynchronously
  (event-driven)
• Performs/Scales better since the burden of
  processing is placed on NCES
• Data access takes place after the content is
  discovered by means of the NCES Federated Search

38
Notional MDA DS COI Pilot Architecture(Messaging
Use Case)
MDA DS COI
Pilot Channels A NE USA B SE USA C MED
West D MED East E West USA F All G AMRS
Navy Organic
USCG OSC
ONI AMRS
Volpe
Channel A
Channel B
Channel C
Channel D
Channel F
Channel E
Channel G
NCES Messaging Service
Message (in COI Vocab) Received
Subscription Request
UDOP
39
Messaging Use Case
Native AIS Data Source
On incoming data event, data is translated to
common vocab and sent to publisher
2
Translation Logic to Common COI Vocabulary/Schema
AIS Data Publisher
WSC
publish AIS data in message
3
NCES Messaging Service
WSP
Subscribe to AIS data
Subscribe to AIS data
1
1
4
4
message delivered
message delivered
AIS Data Subscriber/Receiver
AIS Data Subscriber/Receiver
WSC
WSC
Logic to Consume AIS Data
Logic to Consume AIS Data
UDOP 1
Legacy App
5
5
Data consumed by Legacy App
Data consumed by UDOP
40
High-Level Pilot Architecture
NAIS Aggregation
Navy Organic AIS Aggregation
Infrastructure
COI
MDA DS CoI Data Producers
ONI - AMRS Aggregation
NCES SOA Foundation
Content Delivery
Navy Afloat
NAIS
VOLPE Aggregation
DHS Consumers
AMRS
AMRS
Volpe
DOD Consumers
3
When a data producer receives data externally, it
is published to the Messaging service
asynchronously
2
NCES SS authenticates authorizes MDA COI roles
(MDARestrictedAccess,MDAFullAccess) of data
providers and UDOPs
Interfaces
Messaging Service
M
NCES Messaging Service
Messaging Publisher
MP
4
The data is delivered to subscribing UDOP(s) with
access to data from that data producer
Messaging Subscriber
One or more UDOP(s) is initialized and subscribes
to receive AIS data
1
MS
NCES Security Service
Content Delivery (AIS Data)
MS
MS
DOD
DHS
Other UDOP
Pilot UDOP
HSIN Portal
HSIN Identity Store (Portal Authentication)
5
Users login to UDOP to view AIS vessel information
NCES Service Discovery
Collaboration
SOA Foundation
Pilot Notional Use Case 2 Access Access of
AIS Data, display in Pilot UDOP
41
Messaging Demonstration using iMapData Viewer
42
Navy AIS Data Producer Tasks
43
HSIN iMapData Data Consumer Tasks
44
Msging Configuration Set Up Tasks
1. Implement NCES Message Publishing Service 2.
Implement NCES Security handler with message
publisher 3. Obtain JITC PKI Server Cert 4. Add
JITC PKI Server Cert to NCES Security Services 5.
Add users and roles to NCES LDAP 6. Create
messaging channels in NCES Messaging 7.
Subscribe to messaging channels 8. Mediate to
UDOP supported format
Infrastructure
MDA DS CoI Data Producers
NCES SOA Foundation
1
Content Discovery
NAIS
AMRS
2
3
DHS Consumers
Navy Afloat
Volpe
DOD Consumers
Interfaces
4
Messaging Service
M
NCES Messaging
6
5
Messaging Publisher
MP
Messaging Subscriber
MS
NCES Security Service
7
7
MS
MS
DOD
DHS
8
8
iMapData UDOP
3
3
Google Earth
HSIN Portal
Data Sharing
SOA Foundation
Configuration Set Up Tasks
45
Observations and Lessons Learned
46
Observations and Lessons Learned

• Firewalls are uniquely restricted within many
  organizations
• Custom rule sets exist across the DoD no clear
  policy
• Access is often blocked to standard web
  interfaces (HTTP)
• Special requests and processes are needed to
  change the rules
• Leveraging shared software (NCES) across a
  network remains a challengewe need rigorous SOPs
• Developers can implement net-centric capabilities
  rapidly
• Net-Centricity is not difficult and existing
  resources have the right capabilities
• Net-Centricity is an extension of existing system
  development

47
Observations and Lessons Learned

• Defining smaller increments with clear milestones
  is imperative
• Days and weeks, not months and years
• Identify high-value capabilities and show results
  early
• NCES is pre-Milestone B
• We are early adopters, as defined in the NCES
  Early Capability Baseline (ECB) program
• NCES needs to grow and to be scaleable to support
  the various DOD Enterprises
• COI Pilots are key in helping NCES mature

48
Observations and Lessons Learned

• Knowledge from operational (for JIWG/PWG), system
  (for DMWG), and NCES Subject Matter Experts are
  key to jump-start an effort like this
• Concept of decoupling various services is real!
• A successful pilot needs
• A strong and committed pilot lead
• Senior leadership support
• Commitment of resources (money, people and time)
  from leadership early in the process
• Well defined goals and objectives (spirals)

49
Orchestrating Web Services

• The Pilot identified a need for orchestration
  between NCES Federated Search and Messaging
• Specifically, the ability to
• Execute a Federated Search via a UDOP
• Display the Channels that are available back to
  the user
• Upon selecting a channel, allow the user to
  decide whether or not he wishes to subscribe to
  it (e.g. Google Earth Places)
• Upon subscribing to a Channel, COI Messages need
  to be received in the native format of the UDOP.
• Or, the UDOP is redesigned to understand/consume
  the COI Message
• Data management strategies need to be
  defined/worked for each UDOP implementation.

50
Metrics
51
Software Re-Use During the Pilot

• NCES Content Discovery SDK
• NCES Messaging SDK
• Stubbed out Federated Search Implementation of
  WSDL
• Stubbed out Messaging Publishing Implementation
  of WSDL
• Stubbed out Messaging Subscription and Receiving
  Implementation of WSDL
• Common Aggregation server and software for SPAWAR
  SSC and Volpe
• Federated Search WebPart

As one provider finished coding, the next
provider could reuse part or all of the code
52
Federated Search Development Metrics
53
Data Producer (Messaging) Development Metrics
54
Data Consumer (UDOP) Development Metrics
55
Acknowledgements
56
MDA DS COI Pilot Team

• USCG
• Jay Spalding, Steve Munson, Kathy Shea, Stan Kim,
  Bob DeYoung
• USN
• Bob Poor, Jeff Thomas, LT Dan Wang, Steve Shell,
  Chris Acantilado
• NMIC/ONI
• Steve Milwicz, Terrell Singleton, CAPT Darren
  Sawyer
• DoT
• Kam Chin, David Phinney, Brian Long, Henry
  Wychorski
• HLS/HSIN
• Jim Meck, Mike Fischbein, Ed Frankenberg, Navid
  Keshavarz-Nia, Joel Thomas, Mark Townsend
• Data Management WG
• Mark Andress, CDR Matt Zamary, Brian Freeman,
  Eric Tollefson, Dave Reading

57
Questions?