Image Information Mining ESA Perspective and European Coordination Group

1
Image Information MiningESA PerspectiveandEuro
pean Coordination Group

• Sergio DElia
• Head of Service Support and Ground Segment
  Technology
• EO Programme Directorate

2
Table of Contents

• Background
• IIM at ESA Objectives and Approach
• IIM at ESA Results and Plans
• Semantics
• European IIM Coordination Group
• Links

3
EO support for real problems
Need intervene on environment after oil spill
Data (PBytes)

• Parallel catalogue search
• EO data access
• Information extraction
• Higher level processing fusion with non-EO data
  / information

Many actors and services
knowledge
Information (KBytes)
4
New Technology Opportunities
Need intervene on environment after oil spill
Service Support Environment
Data (PBytes)

• Parallel catalogue search
• EO data access
• Information extraction
• Higher level processing fusion with non-EO data
  / information

Knowledge- based Information Mining / centred
Earth Observation
knowledge
Many actors and services
Grid Processing On Demand
Information (KBytes)
5
SSE Capabilities

• Easy service publication, chaining, discovery,
  activation, monitoring
• Extend GS Interface vs. providers
• Parallel access to distributed catalogues
• Order submission
• Satellite data dissemination
• Services
• Remain by the providers
• Defined via SLA (incl. QoS)
• Automatic or manual
• Subscription or occasional
• From any domain (e.g. GIS)
• Based on
• Open standards (WS, BPEL, SOAP, ...)
• Neutral ESA Moderation

6
SSE Use Opportunities
Restricted Access
Public Access
SSE Portal
7
Processing On Demand

• Capabilities
• 40 TBytes 128 processors(2.6 GHz each based
  on Grid technology)
• Process large data sets
• Interoperable with ESA catalogues / archives
• Status / Plans
• Operational with following services
• MERIS Global Vegetation Index (JRC Level 3
  processor)
• Mapping of ice sheets at Earth Poles from ASAR
  Global Monitoring data
• Temporal aggregation of MERIS RR global data sets
  (L3 processor prototypes)
• Ozone validation (comparing ERS-2 GOME data
  against LIDAR measurements)
• Add new user processors and integrate with SSE

8
Table of Contents

• Background
• IIM at ESA Objectives and Approach
• IIM at ESA Results and Plans
• Semantics
• European IIM Coordination Group
• Links

9
IIM Technology Objectives

• Support / automate information extraction
• Interactive Information Discovery (Probabilistic
  Information Mining )
• Content Based Image Selection (also via SSE)
• Feature Access (also via SSE)
• Time series handling
• Explore combined use of
• Feature Extraction Algorithms
• Probabilistic Information Mining
• Address related issues
• Semantics (ontology / terminology)
• Knowledge management / discovery
• Information mining from heterogeneous sources

10
ESA Approach on IIM
IIM Technology
Operational Method
Capabilities
Project
Probabilistic IM
Feature Extraction Algorithms
11
Related ESA Projects vs. Area
KES-B
IIM Services
KEO, MIMS
Crater, ICDY, SURF
Feature Extraction
Probabilistic IM
KIM, KES, KIMV
12
Table of Contents

• Background
• IIM at ESA Objectives and Approach
• IIM at ESA Results and Plans
• Semantics
• European IIM Coordination Group
• Links

13
Knowledge-based Information Mining

• KIM permits to interactively train the system to
  search an entire image collection for large or
  well characterised areas
• KIM elements
• Ingestion (Primitive Feature extraction /
  clustering per image collection)
• Client / Server for
• System training (and training refinement) from
  sample images
• Application of training to the entire image
  collection
• Semantic labelling of trained features (for
  reuse)
• Output
• Image identifiers
• Feature maps / GIS objects

KIM
Data
14
KIM User Interface
15
Primitive Features
16
Test Collections
17
MERIS Cloud-free Subscription Service
SSE Service Support Environment KIM
Knowledge-based Information Mining MIMS MERIS
Information Mining Services
18
KIM / SSE MERIS Cloud-free Service

• KIM / MIMS
• Ingest systematically MERIS RR products from the
  ESA rolling archives
• Too many cloud objects cloud cover computed
  for each image cell (112x112 pixels)
• Special search in the cloud cover
  databasecloud cover lt user defined
  threshold within user area of interest
• SSE
• Activates KIM cloud cover database search
• Provides notification and product identifiers
  (via e-mail)
• Permits to download MERIS RR products (via FTP)
  from rolling archives

19
KEO Positioning

• Component-based Processing Environmentfor
  information extraction by authorised usersvia
  chained internal / external services

20
KEO Concept
SSE
KEO
21
KEO Local / Remote Contributions
Remote Contributions
Local Contributions Functions
Web Services / FTP
22
Image Information Mining

• Objectives
• Image search also by content
• Discovery in image archives
• Interactive image interpretation
• Extract / access features
• Available
• Feature Extraction Algorithms survey
• Knowledge-based Information Mining tool
• MERIS SSE Cloud-free Service
• Plans
• Distributed Component-base Processing Environment
  linked with SSE
• Extend to other missions (e.g. TerraSAR)
• Ontology / Terminology

23
KEO Phase 2 Plan

• Objective
• Prototype of a Component-based Processing
  Environment
• to automate extraction and provision of
  information
• by authorised users
• Capabilities
• Handling of local Reference Data Sets (incl.
  Documentation, Data Components, auxiliary data,
  )
• Use of remote Data Components
• Description, handling, discovery and use of local
  or remote Processing Components
• Service creation, chaining, publishing and use

24
Relations with Other Projects

• KIM reused in
• ESA projects MIMS, KEO, PIMS-DLR
• STREAM (FP6 Project on Technology to Support
  Sustainable Humanitarian Crisis Management)
• KEO SSE architectures converging
• Easy KEO publishing onto SSE of internal services
  for external access
• Easy KEO use / chaining of external SSE services
• KEO SSE architecture applied to / considered
  for
• PIMS-DLR, STREAM, EUSC Reference Facility
• Heterogeneous Mission Accessibility
  (harmonisation of ground segments)
• Wide Information Network (WIN) FP6 Integrated
  Project on Risk Management
• Other similar approaches emerging
• E.g. EOFrame from VTT

25
Table of Contents

• Background
• IIM at ESA Objectives and Approach
• IIM at ESA Results and Plans
• Semantics
• European IIM Coordination Group
• Links

26
Semantics

• Objective
• Permit easy, semantic identification from non-EO
  domains (using non-EO domain terms) of relevant
  EO
• Products / services
• Processing components
• Approach
• Identify an ontology
• As simple as possible
• With limited dependencies from evolution /
  changes
• Supporting multiple domains
• Permitting a (partial) Web Service implementation
• Supporting multiple languages (outside ESA
  implementation)
• Test the implementation in specific cases

27
ESA Path on Semantics

• ESA activities on EO ontology / terminology
  started in 2003 with the KES-B project
  (http//earth.esa.int/rtd/Projects/KES-B/index.htm
  l), aimed at easing
• Information extraction (permitting semantic
  description / search of related processors)
• User exploitation of EO products / services
  (permitting their semantic description / search)
• KES-B results analysed for
• Ontology simplification / generalisation
• Implementation stability (minimise changes when
  applied to the real world)
• Resulting approach being verified for
• A processing environment (KEO http//earth.esa.in
  t/rtd/Projects/KEO/index.html)
• Implementation for coastal applications in
  cooperation with Mississippi State University
  (SDD)
• Ontology to be extended in future to the Service
  Support Environment

28
KES-B Knowledgebase
29
KES-B Ontology
Definitions Product data or information packed
for user Service (controlled) product provision
Services
Processing
30
Minimum EO Ontology
Domain
Product
Service
Processor
Unstable many changes are possible below Domain
31
Separation to Improve Stability
Domain
Leaves define Service Categories (improving
stability)
Domain (Service Category)
Domain (Service Category)
Product
Service
Processor
32
Classification Ontology Thesaurus?
ISO 2788 Documentation Guidelines for the
establishment and development of monolingual
thesauri
Broader Terms (BT)
Term Synonyms (NPT) Definition (DEF)
Related Terms (RT) (linking Terms)
Related Terms (RT) (not linking Terms)
Narrower Terms (NT)
Definition, Synonyms and not-linking Related
Terms provide many entry points with a minimum
number of links (graph complexity)
33
Thesaurus Example
34
Thesaurus Example
35
Multi-domain Thesaurus
Environment
Resources
Marine Resources
Marine Envir.
Land
Offshore Envir.
Coastal Envir.
Operational Oceanography
Marine Pollution
Algae
Oil Spill
Oil Seepage
Algae Bloom
Oil Seepage Detection
Oil Spill Detection
Algae Bloom Detection
Oil Spill Mov. Forecast
Service
Product
Product
Service
Service
Service
Service
Service
System B
System A
System F
System E
System D
System C
Processor
Processor
36
Distributed / Shared Functions
37
Table of Contents

• Background
• IIM at ESA Objectives and Approach
• IIM at ESA Results and Plans
• Semantics
• European IIM Coordination Group
• Links

38
EuropeanImage Information MiningCoordination
Group

• S. DElia ESA / ESRIN

39
IIMCG History

• Voluntary organisation focusing onResearch and
  technological activities for automated
  anduser-centred extraction of information from
  EO images
• Created on May 26, 2003 by
• ASI - Italian Space Agency
• CNES - French Space Agency
• CNR - Italian National Research Council
• DLR - German Aerospace Center
• EC-IST - Information Society Technology
• ESA/ESRIN - European Space Agency / Research
  Institute
• ETHZ - Swiss Federal Institute of Technology
  Zurich
• EUSC - European Union Satellite Centre
• Extended to EARSC

40
Achievements (after 3 years and 9 meetings)

• Good cooperation
• Free exchange of information and experience
• Common definition of IIM roadmap
• Contributed to ESA technology plan definition
• Agreement on specific research projects

41
Table of Contents

• Background
• IIM at ESA Objectives and Approach
• IIM at ESA Results and Plans
• Semantics
• European IIM Coordination Group
• Links

42
Useful Links

• Feature Extraction Algorithms
• Survey results (2004) http//earth.esa.int/rtd/Do
  cuments/SURF_TN.pdf
• KIM demonstrator
• Client / documentation http//kes.esrin.esa.int/k
  es/
• Questions to EOHelp_at_esa.int
• MERIS Cloud-free Subscription service under test
• Authorised user access at http//services.eoportal
  .org
• KEO
• Phase 1 presentations http//earth.esa.int/rtd/Pr
  ojects/KEO/index.html
• IIMCG
• Information (under revision) http//earth.esa.int
  /rtd/IIMCG
• ESRIN Ground Segment Research Projects
• Information at http//earth.esa.int/rtd/