Image To Knowledge

1
Image To Knowledge
2
Outline

• Introduction
• Image To Knowledge (I2K) Problems
• Basic Image Operations
• Image Classification
• Image Modeling and Synthesis
• Geographical Data Analysis
• Image Matching and Alignment
• Image Analysis in Bioinformatics
• Software Environment D2K
• Summary

3
Introduction

• Image is a special type of data that is collected
  over a regular grid of measurement points
• Image Examples digital photographs, digital
  video, magnetic resonance images, computer
  tomography images, hyperspectral images,
  satellite images, microscope images, infrared
  images, radar images, laser scanned images
• Image Content An image is worth of 1000 words.
  (source unknown) language independence
• Image Variability comes from image formation,
  atmospheric conditions, view point variations,
  and so on

4
Introduction

• Image Applications
• bioinformatics, hydrology, precision farming,
  automation in semiconductor industry, topography,
  plant biology, medicine, automobile industry,
  database retrieval, surveillance, biometrics for
  identification, digital library

5
Introduction

• Objectives
• Image to knowledge (class, location and
  behavior)
• What objects or events
• Where in space
• When in time
• Relationship to other objects and events
• Knowledge representation
• Model (deterministic or statistical)
• Spatial and temporal information
• Association rules
• Difficulties with images
• Large amount of data (512x512x3 786,432)
• Information is globally distributed
• Search for 3D information in lower dimensional
  space
• Models change due to atmospheric changes and view
  point variations
• Expectation on image processing is set very high

Model
6
Introduction

• Variety of challenging problems
• Automatic image calibration
• Automatic image registration
• Unsupervised and supervised classification
• Image segmentation with or without texture models
• Fast model based search
• Fast tracking
• Statistical analysis of image regions
• Image simulations
• High-dimensional image visualization
• Image compression
• Image watermarking
• Multi-sensor image fusion

7

• Image to Knowledge
• (I2K)
• Problems

8
Application Specific Problems Being Addressed in
I2K

• Noise Reduction Problem
• Image de-noising and de-blurring in microarray
  data
• Band Selection Problem Rank bands applied to
  hyperspectral data
• Computational reduction versus accuracy
• Classification Problem Two- and N-class
  classification
• Building land cover and land use maps
• Detecting change in hydrology data
• Matching Problem Search for well defined
  landmarks
• Image calibration in Remote Sensing
• Geo-registration
• Tracking Problem Landmark tracking applied to
  microscopy data
• Registration and alignment based on motion
  imagery
• Alignment Problem 3D Medical Anatomy
• Tissue Cross Section Alignment
• Statistical Modeling Models applied to
  agricultural data
• Crop analysis

9
Application Specific Problems Being Addressed in
I2K

• Motion Detection
• Video analysis for monitoring and surveillance
  purposes
• Grid Alignment Problem
• Irregular or partly regular grid alignment in
  microarray imagery, aerial photography,
  semiconductor wafer processing
• Quality Assurance Problem
• Automatic quality assurance control for
  microarray imagery based on spatial properties,
  SNR and statistical distribution
• Analysis of Geospatial Raster and Vector Data
• Aggregation of georeferenced data based on
  statistical attributes of territories for
  hydrology and insurance applications
• Statistical processing of elevation maps and land
  use maps
• Contour Detection Iso-contour extraction from
  historical maps
• Environmental preservation project

10
Basic Image ProblemsImage CalculatorImage
CorrectionImage Enhancement
11
Image Calculator
Numbers
Image Color Inversion
Images
12
Image Operations
AND


AVG
13
Image Correction Low-Pass and High-Pass Filtering
Output Image
Input Image
Low-Pass
High-Pass
14
Image Enhancement Edge Detection
Edge Phase Image
Magnitude and Phase Output Image
Input Image
Edge Magnitude Image
15
Image Enhancement Edge Detection
Magnitude Edge Image
Input Image
Sobel Operator
Morphological Operator
16
Application of Image Filtering Tools

• Noise Reduction Problem Image de-noising to
  improve statistical accuracy

Input
Output
17
Application of Image Filtering Tools

• Contrast Enhancement Dynamic Range and Variation
  Analysis

Min Max
Input
Mean/-StDev
18
Band Selection Problem

• Band Selection Problem Rank bands to reduce
  computation requirements for processing
  hyperspectral imagery

One method
All methods
19
Clustering, Classification and Segmentation
ProblemsTwo-class classificationN-class
clustering and classificationN-class segmentation
20
Classification

• Two-class classification problem thresholding
  hyperspectral data (e.g., crop versus bare soil)

Input
Output
21
Classification

• Two-class classification problem thresholding
  microarray data (signal versus background)

Output
Input
Model
1102
Dist 2004 Box 902 Plane 2632
22
Clustering

• N-class clustering problem Isodata (advanced
  K-means) algorithm for precision farming
  discover distinct classes

Class Labels
Input
23
Classification

• N-class classification problem Hydrology
  analysis of eco-regions

Class Labels
Statistics
Time
24
Segmentation

• Difference between Clustering and
• Segmentation Labels Neighborhood constraint

Labels from Clustering
Input Image
Labels from Segmentation
Spatially Contiguous Labels
25
Segmentation

• Delineate contiguous regions for (a) Object
  recognition and (b) Land use and land cover maps

Mean Over Label
Input Image
Label Image
26
Image Modeling and Image Synthesis
27
Statistical Models

• Statistical Modeling Problem Selection of
  parametric PDFs

Suggested Statistical Models
28
Statistical Models

• Statistical Modeling Problem Classifier with
  Statistical Model

Compute parameters of a chosen statistical
model, e.g., Gaussian PDF model.
Visualize PDF parameters
Classify
29
Image Problems in Geo-Spatial Information Systems
30
Vector Data
Boundary Information
Counties
Census Tracks
Census Blocks
Zip Codes
31
Raster Data
Image Georeferencing Information
Digital Elevation Map of Illinois
32
Georeferencing
Geo-registering data sets using georeferencing
(Lat/Lng lt-gt UTM lt-gt Pixels)
Input Raster and Vector Data Sets
Geo-registered Data Sets
33
Statistical Analysis
Computing statistics from geo-registered data sets
Geo-registered Elevation Map and County Boundaries
Elevation Statistics Per County
Mean Elevation
Standard Deviation
Kurtosis
Skew
34
Segmentation of Geo-Spatial Attributes
Segmentation Problem Find all neighboring
counties with similar attributes
Input Geo-info About Counties
Output Aggregations of Counties
Aggregations
Statistics
County Index
Counties
35
Contour Extraction

• Extract iso-contours from historical maps

Click and Extract
Extract all Automatically
36
Data Fusion

• Processing Heterogeneous Multi-Modal Data Sets
• Why?
• Overlapping information content describing the
  same event in space and time.
• Each sensor can convey only a small piece of
  information (EO, SAR, IR, HS).
• Example Landslide detection

Annotation Analysis and Data Fusion
Topographic Map
Aerial Photos
Hyperspectral
Satellite
Surface Geology
DEM
37
Matching ProblemMatching in Remote Sensing
Optical Character Recognition Tracking in
MicroscopyAlignment in 3D Medical Anatomy
Modeling
38
Matching in Remote Sensing

• Matching Problem Search for well defined
  landmarks to geo-register and calibrate imagery

Calibration Using Tarps
Registration
Hyperspectral
Satellite
Train, Find and Register
Calibrate
39
Matching In Optical Character Recognition

• Matching Problem Convert a laser scan of a
  document into a MS Word document
• Partition document (text, images and background)
• Find lines, paragraphs, headings, font,
• Identify characters

20x20 2400 ? 10120 patterns
40
Tracking in Microscopy

• Tracking Problem Feature tracking for
  registration and alignment.

Output Alignment Information
Input Sequence
41
Alignment in 3D Medical Anatomy Modeling

• Matching Problem Search for transformation
  parameters to align medical slices.

Input slice x1
Visualization of Slice Transformation
Input slice x2
42
Matching in Anatomy

• Matching Problem
• Find a match to a new cross section in the
  existing 3D anatomical model.
• Find a match to a new 1D signal
• Find a match to an image acquired by another
  sensor, e.g., MRI, CT, Histology

Structure - 3D Anatomy
Function 1D Signal
Metadata Annotation
43
Image Problems in Bio-Informatics Microarray
Grid AlignmentMicroarray Grid ScreeningMicroarra
y Image Feature Extraction and Clustering
44
Input and Output of Microarray Data Analysis

• Input Laser image scans (data) and underlying
  experiment hypotheses or experiment designs
  (prior knowledge)
• Output
• Conclusions about the input hypotheses or
  knowledge about statistical behavior of
  measurements
• The theory of biological systems learnt
  automatically from data (machine learning
  perspective)
• Model fitting, Inference process

45
Microarray Image Analysis
46
Grid Alignment
Single Grid one or more channels
Speed
Rotation
Background Noise
47
Grid Alignment
Multiple Grids
Grid Regularity
Multiple Grid Setup
48
Error Detection Example of Spot Screening
Mask Image Location and Size Screening
Mask Image No Screening
Mask Image SNR Screening
49
Feature Extraction
Feature Selection and Visualization
Feature Selection
Mean Feature Image
50
Clustering and Classification
Class Labeling and Visualization
Clustering
Mean Feature Image
Label Image
51
Video Processing Problems
52
Motion Detection
Input EO Video
Detection of Moving Entities
Tracking of Moving Entities
Metadata
Entity Pickup truck Track Attribute Motion
Vector Model Linear Track Attribute Motion
Vector Parameters 0.97, 381, 0.07, 203 Track
Attribute Motion Vector Confidence 0.78
Entities and their temporal and spatial changes
53
Visualization
54
Image To Knowledge (I2K) Visualization

• Hyperspectral image with 120 bands

55
Image To Knowledge (I2K) Documentation

• Overview and tools

56

• Software
• Environment
• Data To Knowledge (D2K)

57
Software Engineering in Data Mining

• Conceptual Software Hierarchy
• Operating System (Windows, UNIX, Linux, MAC)
• Programming Language (Java)
• Modules Sequences of Programming Language
  Commands
• Itineraries Linked Modules
• Streamlines Linked Itineraries
• Software for
• Users With Various Levels of Programming Skills
• Collaborating Users
• Users Used to Standard Menu Driven Tools, e.g.,
  MS Word

58
D2K - Software Environment for Data Mining

• Visual programming system employing a scalable
  framework
• Robust computational infrastructure
• Enable processor intensive apps, support
  distributed computing
• Enable data intensive apps, support
  multi-processor, shared memory architectures,
  thread pooling
• Very low granularity, fast data flow paradigm,
  integrated control flow
• Reduction of development time
• Increase code reuse and sharing
• Expedite custom software developments
• Relieve distributed computing burden
• Flexible and extensible architecture
• Create plug and play subsystem architectures, and
  standard APIs
• Rapid application development (RAD) environment
• Integrated environment for models and
  visualization

59
Programming and Runtime Environment
Tool Menu
Tool Bar

• Side Tab Panes

Workspace
Jump Up Panes
60
Streamlined Processing Environment D2K SL
Processing Steps
Workspace
Processing Options
Session
61
Menu Driven Processing Environment
Result Visualization
Menu Options
Processing Dialog
62
Data Mining Techniques in D2K

• Discovery
• Association Rules, Link Analysis, Self Organizing
  Maps
• Predictive Modeling
• Classification Naive Bayesian, Neural Networks,
  Decision Trees
• Regression Neural Networks, Regression Trees
• Deviation Detection
• Visualization
• Text To Knowledge (T2K)
• Image To Knowledge (I2K)
• ----------------------
• Audio, Touch, Scent and Savor To Knowledge
• Knowledge To Wisdom (K2W)

63
Summary

• Overview of Image Data Processing and
  Applications
• Data Processing
• Raster and vector data
• High-dimensional data
• Spatial, temporal and spectral data types
• File format issues
• Data fusion of heterogeneous data
• Applications
• Bio-tools (Bio-informatics, Biology, Plant
  Science)
• Geo-tools (GIS, Remote Sensing, Agriculture,
  Hydrology, Water Quality Survey, Insurance,
  Atmospheric Science)
• Med-tools (Medicine, Biology)
• Edu-tools (Education)
• Other tools Video Analysis, Statistical
  Simulations, Target Recognition
• Software Environment
• D2K, D2K SL, Menu Driven
• Interested ? Useful ? Let us know.