Sep' 2122, 2006

1
v
Sep. 21-22, 2006
FME Worldwide User Conference - Vancouver
Using FME Objects Python API to Provide Platform
Independent Translations Andrew Smith, Lagen
Spatial, Australia
2

• Using FME Objects Python API to Provide Platform
  Independent Translations
• Andrew Smith
• Spatial Technical Consultant
• Lagen Spatial - Australia

3
Using FME Objects Python API to Provide Platform
Independent Translations

• Introduction
• Who am I?
• Project Overview
• Python and FMEObjects
• Method for Development
• Pitfalls
• Tips and Tricks
• Output Examples
• Questions

4
Using FME Objects Python API to Provide Platform
Independent Translations

• Who am I?
• Name Andrew Smith
• Company Lagen Spatial Pty Ltd
• Official Title Spatial Technical Consultant
• Email andrew_at_lagenspatial.com.au

5
Using FME Objects Python API to Provide Platform
Independent Translations

• Project Overview
• Geoscience Australia
• plays a critical role by producing first-class
  geoscientific information and knowledge. This can
  enable the government and the community to make
  informed decisions about the exploration of
  resources, the management of the environment, the
  safety of critical infrastructure and the
  resultant wellbeing of all Australians.
• www.ga.gov.au

6
Using FME Objects Python API to Provide Platform
Independent Translations

• Project Overview
• Geographic Data Management Division
• Solution Name - Geoflex
• Project Leader - Tony Hunter
• Formats Supported
• ESRI Shape File (.shp, .shx, .dxf)
• Personal Geodatabase
• Mapinfo Native Format (.tab), Mapinfo Data
  Interchange Format (.mif), and
• Geographic Mark-up Language (.gml)

7
Using FME Objects Python API to Provide Platform
Independent Translations

• Project Overview
• Requirements
• Be platform independent i.e. the core extraction
  process is able to execute on both Windows and
  Unix Operating Systems
• Allow a User to Select the Feature Dataset to be
  extracted
• Allow a User to Extract the datasets to on 3
  scales
• National
• State/Province
• Map Tile
• Custom Symbology Developed by GA to be applied to
  the Output Datasets
• Dynamically update the list of Feature Datasets
  that are available for extraction through
  interrogating the underlining Spatial DBMS schema

8
Using FME Objects Python API to Provide Platform
Independent Translations

• Platform Independent Translations with FMEObject
  for Python
• Method
• FME Workbench to Develop the Translation Process

9
Using FME Objects Python API to Provide Platform
Independent Translations
10
Using FME Objects Python API to Provide Platform
Independent Translations

• Copy the Transformers into a text editor to get
  the FME Script that defines Translation

11
ArcSDEQuerier

• MACRO wheremac SDE_SEARCH_METHOD_FIELD
  __SEARCH_METHOD
• INCLUDE if SDE_AREA_INTERSECT SDE_NONE
  puts MACRO wheremac
• FACTORY_DEF TeeFactory
  \
• FACTORY_NAME ARCSDEQUERIER_Globber
  \
• INPUT FEATURE_TYPE clip_area_0
  \
• OUTPUT FEATURE_TYPE __GLOBBED__
• FACTORY_DEF SDE30QueryFactory
  \
• FACTORY_NAME ARCSDEQUERIER_Querier
  \
• INPUT FEATURE_TYPE __GLOBBED__
  \
• _at_SupplyAttributes(__SERVER,lagen-pc-1,__INSTAN
  CE,esri_sde,__DATASET,sde,__USERID,sde,__PASSWORD,
  sdeuser,__TARGET_TABLE,STONNING_CAD,__SEARCH_METHO
  D,SDE_AREA_INTERSECT,__WHERE_CLAUSE,"",__VERSION,S
  DE.DEFAULT) \
• SDE_SERVER_FIELD __SERVER
  \
• SDE_INSTANCE_FIELD __INSTANCE
  \
• SDE_DATASET_FIELD __DATASET
  \
• SDE_USERID_FIELD __USERID
  \
• SDE_PASSWORD_FIELD __PASSWORD
  \
• SDE_VERSION_NAME_FIELD __VERSION
  \

12
ClippingFactory

• FACTORY_DEF ClippingFactory
  \
• FACTORY_NAME CLIPPER
  \
• INPUT CLIPPER FEATURE_TYPE clip_area_1
  \
• INPUT CLIPPEE FEATURE_TYPE ARCSDEQUERIER_STONNI
  NG_CAD \
• MULTICLIP YES
  \
• CLIPPEE_ON_BOUNDARY INSIDE
  \
• MERGE_CLIPPER_ATTRIBUTES No
  \
• DO_NOT_AGGREGATE NO
  \
• CLIPPER_ATTR_PREFIX "clipper_"
  \
• OUTPUT INSIDE FEATURE_TYPE CLIPPER_INSIDE
  \
• OUTPUT CLIPPED_INSIDE FEATURE_TYPE
  CLIPPER_CLIPPED_INSIDE \
• OUTPUT EXTRA_CLIPPER FEATURE_TYPE
  CLIPPER__delete__ _at_Tcl2("FME_LogMessage fme_warn
  Deleting extra clipper feature encountered which
  entered transformer CLIPPER") \
• OUTPUT NONPOLY_CLIPPER FEATURE_TYPE
  CLIPPER__delete__ _at_Tcl2("FME_LogMessage fme_warn
  Deleting non-polygonal clipper feature which
  entered transformer CLIPPER")
• FACTORY_DEF TeeFactory
  \
• FACTORY_NAME CLIPPER_Nuker
  \
• INPUT FEATURE_TYPE CLIPPER__delete__
  \
• _at_Log("Offending feature was")

13
Joiner (Relate) Factory

• Relate CACHE_SIZE "JOINERall_symbology_lookup_tab
  le" 5000
• Relate TABLE_LOCATION "JOINERall_symbology_lookup
  _table" "d\User Documents\LagenSpatial\Projects\G
  A - FME Data Migration\Symbology Library
  (new)\version1.1 20060419\all_symbology_lookup_tab
  le.csv"
• Relate TABLE_DEF "JOINERall_symbology_lookup_tabl
  e" CSV \
• FACTORY_DEF TeeFactory
  \
• FACTORY_NAME JOINER
  \
• INPUT FEATURE_TYPE CLIPPER_INSIDE
  \
• INPUT FEATURE_TYPE CLIPPER_CLIPPED_INSIDE
  \
• OUTPUT FEATURE_TYPE JOINER_JOINED
  \
• _at_Relate(JOINER,Read)
  \
• _at_RenameAttributes(_matched_records,matched_
  records)

14
Using FME Objects Python API to Provide Platform
Independent Translations

• Save the pasted output as one or more .fmi files
• Use the FMEFactoryPipeline.addFactoriesFromFile()
  method to include your translation pipeline .fmi
  files into your PyFMEObjects Script.

15
Using FME Objects Python API to Provide Platform
Independent Translations

• WHY???
• Reduce possible errors in your script
• Special Characters
• \n New line
• \r Carriage Return
• \w White Space
• \t Tab !!!
• Special characters MEAN something in both Python
  and FME Mapping Files

16
Using FME Objects Python API to Provide Platform
Independent Translations

• Allows you to solve other bugs
• Logical
• Programmatic
• Data Flow
• Logical
• IfThenElse
• DoLoop
• SwitchCase

17
Using FME Objects Python API to Provide Platform
Independent Translations

• Allows you to solve other bugs
• Logical
• Programmatic
• Data Flow
• Programmatic
• Incorrect Method use
• String Manipulation
• Reading Multiple Source Datasets

18
Using FME Objects Python API to Provide Platform
Independent Translations

• Allows you to solve other bugs
• Logical
• Programmatic
• Data Flow
• Data Flow
• Combination of Logical and Programmatic Errors
• Unexpected output

19
Using FME Objects Python API to Provide Platform
Independent Translations

• Once you have your Python script reproducing the
  same output as Workbench, its time to Embed your
  Factories
• which is also where the fun begins!!

20
Using FME Objects Python API to Provide Platform
Independent Translations

• First of all
• Use the Python Lists i.e.
• stringListVariable green, eggs, and,
  ham
• Cuts out the possibility of naughty special
  characters sneaking their way into your Factory
  Definition
• i.e. DONT USE THE green eggs and ham
  strings it tells Python to use the string as it
  is, Naughty Special Characters and all

21
GOOD

• fme_session.config('Relate', 'RELATION_DEF',
  'JOINER', '10..1',
• 'TABLE', 'JOINERall_symbology_loo
  kup_table',
• 'UNIQUE(Esri_No,feature_type)',
• 'JOIN', 'Esri_No', 'TO',
  'SYMBOL',
• 'JOIN', 'feature_type', 'TO',
  'fme_type',
• 'MATCHED_RECORDS_ATTR',
  'matched_records',
• 'TRIM_TRAILING', 'No',
• 'TRANSFER', 'mapinfo_brush_backgro
  und', 'TO', 'mapinfo_brush_background',
• 'TRANSFER', 'mapinfo_brush_foregro
  und', 'TO', 'mapinfo_brush_foreground',
• 'TRANSFER', 'mapinfo_brush_pattern
  ', 'TO', 'mapinfo_brush_pattern',
• 'TRANSFER', 'mapinfo_pen_color',
  'TO', 'mapinfo_pen_color',
• 'TRANSFER', 'mapinfo_pen_pattern',
  'TO', 'mapinfo_pen_pattern',
• 'TRANSFER', 'mapinfo_pen_width',
  'TO', 'mapinfo_pen_width',
• 'TRANSFER', 'mapinfo_symbol_angle'
  , 'TO', 'mapinfo_symbol_angle',
• 'TRANSFER', 'mapinfo_symbol_color'
  , 'TO', 'mapinfo_symbol_color',
• 'TRANSFER', 'mapinfo_symbol_file_n
  ame', 'TO', 'mapinfo_symbol_file_name',
• 'TRANSFER', 'mapinfo_symbol_font',
  'TO', 'mapinfo_symbol_font',
• 'TRANSFER', 'mapinfo_symbol_shape'
  , 'TO', 'mapinfo_symbol_shape',
• 'TRANSFER', 'mapinfo_symbol_size',
  'TO', 'mapinfo_symbol_size',

22
BAD

• fme_pipeline.addFactoryFromString( """FACTORY_DEF
  SDE30QueryFactory \
• FACTORY_NAME ARCSDEQUERIER_Querier
  \
• INPUT FEATURE_TYPE __GLOBBED__
  \
• _at_SupplyAttributes(__SERVER,"""
  server """,__INSTANCE,port""" instance
  """,__DATASET,""" _dataset """,__USERID,"""
  username """,__PASSWORD,""" password
  """,__TARGET_TABLE,""" _owner "."
  featuredataset.upper() """,__SEARCH_METHOD,SDE_E
  NVELOPE,__VERSION,SDE.DEFAULT) \
• SDE_SERVER_FIELD
  __SERVER \
• SDE_INSTANCE_FIELD
  __INSTANCE \
• SDE_DATASET_FIELD
  __DATASET \
• SDE_USERID_FIELD
  __USERID \
• SDE_PASSWORD_FIELD
  __PASSWORD \
• SDE_VERSION_NAME_FIELD
  __VERSION \
• SDE_TARGET_TABLE_FIELD
  __TARGET_TABLE \
• SDE_SEARCH_METHOD_FIELD
  __SEARCH_METHOD
• QUERY_MODE
  QUERY \
• SEARCH_ORDER
  OPTIMIZE \
• COMBINE_ATTRIBUTES
  MERGE \
• GET_SPATIAL_RELATIONS
  \
• OUTPUT RESULT FEATURE_TYPE
  ARCSDEQUERIER_OUTPUT
  \
• _at_RemoveAttributes(__SERVER,__I
  NSTANCE,__DATASET,__USERID,__PASSWORD,__TARGET_TAB
  LE,__SEARCH_METHOD,__WHERE_CLAUSE,__VERSION) \
• _at_SupplyAttributes(_table_name,
  _at_FeatureType()) \

23
Using FME Objects Python API to Provide Platform
Independent Translations

• Why Use Python?
• Great Whipitupability
• It is an interpreted language
• Its free and powerful!!! With a great open source
  community
• Supported by ESRI and FME
• You can write your own transformers for FME

24
Using FME Objects Python API to Provide Platform
Independent Translations

• Example - Sydney

25
Using FME Objects Python API to Provide Platform
Independent Translations

• Example - Darwin

26
Using FME Objects Python API to Provide Platform
Independent Translations

• Example - Perth

27
Using FME Objects Python API to Provide Platform
Independent Translations

• Example Pt. Lincon

28
Using FME Objects Python API to Provide Platform
Independent Translations

• QUESTIONS???
• Name Andrew Smith
• Email andrew_at_lagenspatial.com.au
• Phone [phone]