Data Interoperability: An Introduction

1
Data Interoperability An Introduction

• Bruce Harold bharold_at_esri.com
• Dale Lutz Safe Software

2
Background

• Why Data Interoperability?

3
What is the Data Interoperability Extension?

• Connects your World of Apps

• Connect to Formats
• 200 added file formats
• Feeds Web sources
• Custom formats
• Quick Translation
• Any format to File GDB
• Any format to another Format
• Transformation
• Schema transformation
• Geometry transformation

• File, Photo, Parcel, BIM, MS Office, DBMS, CSV,
  Network, Military, Survey, Sensor, GPS

4
ArcGIS Data Interop Common Tasks

• Connected workflows

• Integration
• Migration
• Cleanup
• Validation
• Distribution
• Merging Changes

• Spatial ETL

5
Data Interoperability Differentiators

• For Data Triathletes

• Feature by Feature Processing
• Inline SQL across all your data
• Data Inspection
• Deep format standards support
• XML Documents as Features or Attributes
• JSON Documents as Features or Attributes
• Protocol Support (FTP, HTTP, RSS)
• URL and Compressed Inputs
• Flow Control Parallel Processing
• No Coding

6
Basic OGC Support

• What you get just by installing

7
WFS and GML

• Supported without Licensing

• GML
• Simple Features Profile
• WFS in Data Interoperability
• Cache lives 24 hours by default
• Numeric Identifier forces live access
• You must install the extension

• OGC GML WFS suitable for Geoprocessing

8
OGC GML WFS

• Demo

• Data Interoperability as an OGC client

9
Other OGC Stuff

• But wait theres more

• KML
• Schema object preserved
• KML Properties
• KML Styling
• CityGML
• CityEngine Tools coming
• To From File GDB

• OGC Formats in Data Interoperability

10
Using Formats

• Ways to read any format data

11
Direct Read/Write From Disk

• File Based recognized by extension

• Catalog browse tree
• No need to add file type
• Feature Dataset icon
• Available feature types shown
• Format defaults used
• As geoprocessing input
• Tools
• Models
• Scripts (enable the extension!)

• Recognized formats just work

12
Interoperability Connections

• Manually set up and persistent

• When default format connection is
• Infeasible, e.g. authentication required
• Undesirable, e.g. optional parameter wanted
• Authored in Catalog tree
• Off-disk feeds web sources
• May view source data prior to connecting
• Not cached, read when used
• Coordinate system may be selected
• May be shared (.fdl file)

• Yes, Excel can be read as Point data!

13
Selecting Formats

• Or, What was that FME Reader Gallery thing in
  the previous slide?

• Browse
• Search
• Customize

• Dont see what youre looking for? Ask us!

14
Direct Read Connections

• Demo

• Using any format in ArcGIS for Desktop

15
Quick Translation

• Quick Import Quick Export GP Tools

16
Quick Import Quick Export

• Geoprocessing tools in Data Interoperability

• Quick Export
• To Any Format
• Quick Import
• To File Geodatabase

• Rapid format conversion using format defaults

17
Automate Quick Translations

• Use the Quick tools in ModelBuilder

• Easy way to ship multiple formats
• Retain your GIS schema

• Supply popular formats with GIS-friendly
  properties

18
Quick Tools

• Demo

• Automated Format Supply

19
Leveraging Formats

• Custom Formats

20
What if you need your own format?

• Save time by canning a custom format workflow

• Based on a known format
• Build format definition workflow once
• Remains available in the gallery

Based on JSON

• You can share a custom format with other Data
  Interoperability users

21
Custom Formats

• Demo

• JSON read from the web

22
The Power of Transformation

• Spatial ETL

23
The Full Power of Spatial ETL

• Connect. Transform. Automate.

• Format reading and writing is only a beginning
• Information has no format
• Formats are only languages many have dialects
• Formats merely facilitate information (or wreck
  it)
• Information is often hidden or requires business
  logic
• Transformation reveals and derives information
• Calculate stuff, mess with words, ignore junk
• Use relational, dictionary, list network
  structures
• Way, way more
• Stop. Writing. Code. (Unless you find it fun)

• Transformation is the real power of Data
  Interoperability

24
Workbench Application

• Graphically Design Data Flows

• Create via context menu
• Follow the wizard

• The Workbench application authors Spatial ETL
  tools

25
Workbench Authoring

• Design Document

• Read
• Transform
• Attributes
• Geometry
• Style
• Write
• Run standalone
• Run as tool

• Workbench has a canvas and supporting windows

26
Transformers

• Hundreds of functions at your fingertips

• Eponymous
• Named after their function
• Categorized
• Browse
• Search
• Popup

• Drag or click transformers onto the canvas or use
  the popup functionality

27
ETL

• Demo

• Connect. Transform. Automate.

28
Wrapping Up

• A few more observations

29
Data Interoperability in ArcGIS for Server

• Considerations

• Map services can use supported formats
• 64 bit only
• Spatial ETL tools must be in Models
• Publish service as usual from Result
• Good Zip Ship solution
• Generic Writer into ZIP file
• Output format made parametric

• Data Interoperability is available in ArcGIS for
  Server

30
Coming in 10.3

• And its service packs

• FME 2014 SP 2 Engine
• Write images hyperlinks to Excel (maps!)
• Data Interoperability for ArcGIS Pro
• 64 bit
• More

• Data Interoperability will be released for ArcGIS
  Pro

31
Learn More

• Data Interoperability

• At the User Conference
• Esri Showcase Tuesday 9am until Thursday 130pm
• After this presentation
• 60 Day Software Evaluation
• http//www.esri.com/software/arcgis/extensions/dat
  ainteroperability/evaluate
• Training
• http//training.esri.com/gateway/index.cfm?fasear
  ch.resultssearchtermdatainteroperability

• Talk to us now!

32
Thank You! Now its Question Time!

• Data Interoperability An Introduction

• Please fill out the session survey
• First Offering ID 1221
• Second Offering ID 1402
• Online www.esri.com/ucsessionsurveys
• Paper pick up and put in drop box

• Have a great conference!

33
(No Transcript)