Digital Image Databases and 3D Visualization Applications to Science and Industry

1
Digital Image Databases and 3D Visualization
Applications to Science and Industry

• Dr. Nikolay Metodiev Sirakov
• Dept. of Mathematics and Dept. of CS
• Texas AM University Commerce
• Commerce , TX 75 429
• E-mail Nikolay_Sirakov_at_tamu-commerce.edu

2
The amount of digital images increased
enormously over the last few years in Internet,
satellite, medical, and environmental imaging.
Thus the problem for efficient image database
managementand quick image retrieval is emerging
as an active area of research that attracts the
attention of mathematicians, computer scientists
and engineers.
3
STRATEGIES FOR IMAGE RETRIEVAL

• Text-based search
• Conventional text-matching techniques
• Tedious, labor-intensive to prepare text
  descriptions of images
• Content-based search
• Requires features derived from image content
• Many features can be automatically extracted
• Similarity based on feature vectors

4
MOTIVATION

• -Great interest in Content-Based Image Retrieval
  (CBIR) techniques, which employ automatically
  derived features color, texture, and shape as
  search criteria (Lee et al 2003).
• Applications
• Air and satellite imagery, GIS
• Biomedical Biological Imaging
• Chemical Agricultural Imagery
• -The CBIR is a challenging topic of current
  research.
• The use of extracted image features to rapidly
  locate a desired image from a large and dynamic
  collection is widely recognized as an area of
  active research (Lisani 2001, Yang 2002).

5

CONTENT BASED IMAGE RETRIEVAL

• A CBIR system must be capable of
• Image segmentation into distinct regions (Long
  1999, Zamora 2001, Yang 2002 )
• Feature Extraction, Classification (Long 2001)
• Indexing by features, (Antani 2003, Chan 2003)
• Image retrieval by matching (Latecki 2001, Lee
  2003).
• REQUIREMENTS
• 1. To retrieve images in real time
• 2. To retrieve small amount of images
• 3. To use as little as possible prior
  information.

6
IMAGE DATABASE

• Image DataBase Construction and Querying

7
IMAGE DATABASE
a) Medical
b) Satellite

• a) Typical X-Ray image.

8
IMAGE DATABASE

• Vales Marineris Canyon Mars, taken by a
  spaceship, launched by European Space Agency,
  from an altitude 275 km, Resolution 12 m per
  pixel.
• The greatest Canyon in the Solar System 4000 km
  long,
• 10 km deep

9
Image Database and Features
Feature vectors
Images
Feature extraction
I1
F1
Fk f1(Ik), , fm(Ik)


In
Fn
10
Image Database Query
Two common approaches
Feature vector index
Query image
Query feature vector
F1
Iq
Fq

compare and rank
Fn
OR
Direct specification
Retrieved images
11
An Example of CBIR system

• Blobworld is a system for content-based image
  retrieval by automatically segmenting each image
  into regions which roughly correspond to objects
  or parts of objects in a photos Image DataBase.
• The Image DataBase contains around 35000 images.

12
IMAGE SEGMENTATION

• Definition A central problem, called
  segmentation, is to distinguish objects from
  background.
• Different types of segmentation by using
• texture, color, shape.
• Approaches used
• Statistical, Vector Source Coding (Yung 2002)
• Active Shape Models (Zamora 2001)
• I propose a new shape segmentation approach based
  on the Heat DE.
• Features of Main Interest
• Global Number of regions Inter-regional
  distances.
• Local Convex Hull Essential boundary points
  Boundary support Number of concavities, Etc.

13
SEGMENTATION

• Segmentation of a satellite image to different
  shapes.

14
SEGMENTATION

• Result from the piecewise smooth Mumford-Shah
  level set
• algorithm with one level set function

15
COLOR IMAGE SEGMENTATION

• In April, a team of European and American
  astronomers used Yepun to detect a faint and very
  red point of light near the brown-dwarf star
  2M1207. The astronomers believe they may have
  taken the first direct image of a planed circling
  another star.
• And Anne-Marie Lagrange "Our discovery represents
  a first step towards opening a new field in
  astrophysics
• the imaging and spectroscopic study of
  planetary systems.

16
SEGMENTATION

• Edge detection and medical X-Ray image
  segmentation, using curve, evolved according to
  the Heat DE

17
SEGMENTATION

• National Library of Medicine CBIR system
• Active Contour Segmentation Tool Main Window.

18
Concavities-Boundary Support
b)
a)

• A section of an impermeable subsurface unit.
• Its boundary support.

19
INDEXING and PARTITIONING

• Matching a query shape against every shape in the
  entire database may be infeasible even with a
  powerful computer platform and a very fast search
  engine.
• Thats way a Search Space Partitioning (SSP) is
  needed.
• For an extensive IDB, SSP and indexing techniques
  significantly reduce the size of the search
  space that the query must traverse.
• I propose a new efficient approach capable of SSP
  by using an indexing based on the following shape
  features extracted from the database
• - Convex Hull
• - Number of boundary concavities.

20
Search Space Partitioning
Region Shape Index
Subclass C1

Subclass Ck
Class 1 Index
Class 2 Index
Class 344 Index

where
Classes represent convex hull, defined by
Subclasses represent the number of shape
concavities
21
MATCHING

• A large number of shape matching methods
• - Elastic deformation of templates (Del Bimbo
  1996)
• - Matching directional histograms (Androutsas
  1999)
• - Shocks skeletal rep. of object shape
  (Tirthapura 1998).
• - A fuzzy logic similarity measurement by Gadi,
  1999.
• - Lisani 2000, uses differential equations to
  develop a similarity metric based on the area
  of shape concavities
• - Lee 2003, proposed an approach that combines
  polygon curve representation with Fourier
  descriptors.
• For fast shape matching I am using regularities,
  finite numerical sequences and shape support.

22
Math Definition of an Image.

• Two models have been introduced to recognize
  existing of edges
• 1) Mumford Shah 1989 Object Edge Model
• 2) Rubin, Osher and Fatemi 1992 BV image
  model.
• assumes that an ideal image I consists of
  disjoint homogenous object patches
  with and
  regular boundaries .
• Free boundary model
  

23
Examples
Decomposition of page image to a text and picture
by employing 1).

• assumes that an ideal image has bounded total
  variation
• used as an energy model
• Regularly based image models are applicable to
  images with low texture.

Application of 2) to image noise cleaning and
debluring.
24
ACTIVE CONTOURS

• The active contours are a powerful tool employed
  in CBIR systems for image cleaning, edge
  detection and segmentation.
• To develop an Active Contour Model an PDE or
  energy functional are used to evolve an Euclidean
  curve.
• Vector Fields Inward normal vector shrinks the
  Euclidean perimeter. Outer normal enlarges the
  perimeter.
• Classification
• - parametric or geometric active contour models
• - open or close.

25
DIFFICULTIES

• First, the initial contour must be close to the
  true boundary or else it will likely converge to
  wrong result.
• Several methods have been employed to tackle
  with this problem
• multiresolution (Leroy, B., at ll, 1996),
• pressure and distance forces respectively
  (Cohen, (1991), Cohen Cohen, 1993)
• The active contours have difficulties progressing
  into boundary concavities.
• Several approaches are designed to tackle with
  different particular tasks (Abrantes, A.J.,
  Marques, J.S., (1996), Davatzikos, C., Prince, J.
  L., (1995)).

26
DIFFICULTIES

• Figure 3. Brains 2D section boundary
  approximation by an active contour.

The result obtained by a tool based on a model
that uses Heat Differential equation and objects
shells.
27
DIFFICULTIES
distance forces

• Figures 5,6. Convergence into boundary
  concavities.
• Xu and Prince (1998)-John Hopkins University
• J. Tang and S. Acton (2004)-Virginia State
  University

28
Description
Are the vertical and the horizontal component of
the vector field, f - is the edge map derived
from the image and defined by
is a 2D Gaussian kernel, with a standard deviation
K is a constant used to control the smoothness of
the Vector field.
29
DEMONSTRATIONS

• BLOBWORLD
• RetReg Results .

30
2. 3D Objects Model
Regularity is considered to be the direction of
point motion over a closed polygon and represents
just one edge.
Essential Points of a closed polygon link
different regularities.
Non-essential points link similar regularities.
Consistency is defined as a sequence of
consecutive connected regularities.

• a)
• a.) The plane points satisfy 8 regularities.
• b) Closed polygon and the regularities that
  its edges satisfy.
• The vertex (xmin, ymin) is used as a starting
  point of polygon description.

(1)
31
An Example Approach Capable of Smooth 3D Objects
Visualization

• Each consistency falls apart to four finite
  numerical
• sequences
• Rg - sequence of regularities that build the
  consistency
• Rp -sequence containing repetition indicator of
  each regularity
• An - the angles, that the regularities of Rg
  conclude with the positive direction of the axis
  Ox
• Le - the lengths of the regularities belonging to
  Rg.
• The polygon shown in Fig. 1.b) is described by
  means of

Rg 5,3,2,1,2,4,6,7,6 Rp 2,3,1,1,1,2,2,1,1
An ?1,....,?14 Le
l1,....,l14.
32
An Example Approach Capable of Smooth 3D Objects
Visualization

• 3D object model as an aspect graph tree.

SSOk k-th 3D reconstructed subsurface
Cki the i-th plane section that cut the k-th 3D
subsurface object. In case of 3D branching
objects we denote by Cji the following
vector .
the regularity that belongs to the i-th
consistency of the k-th ore body
33
3. Approach Capable of 2D Boundary Approximation
3.1. METHOD OF THE QUADRATIC ERROR SUM
Assume set of points N (xi , yI), i 1,..,n
situated on the plane Oxy. The best approximation
of N is
1.A circle with canter (x0, y0) and radius r, if
the points satisfy (2), where
x0 , z0 , r2 ,
-denote the second central moments of the points
from N di denotes the distance between the
points (xi , yj) and the circle and d0 is the
least distance larger than di, for i 1,.,n.
2.An ellipse with center (x0, y0) and axes a
and b if the points satisfy
(3), where a2
, b2 .
34
An Example Approach Capable of Smooth 3D Objects
Visualization
3.A hyperbola with imaginary axis Oy or Ox
respectively if the points satisfy
(4),
Disadvantage the approach yields good results
only if the points are normally distributed
around their mass center. Follows, it is not
applicable for many practical cases.
35
An Example Approach Capable of Smooth 3D Objects
Visualization
3.2. 2D BOUNDARY INTERPOLATION BY MEANS OF ARCS
The method is useful when the points are not
normally distributed around the center of the
mass. Consider the general mode of an equation
of a second order curve A.x22.B.x.yC.y22.D.x
2.E.yF 0 (5)
Theorem 1. If a set of plane points satisfies one
of the consistencies or then the
coefficient C 0.
Theorem 2. If a set of plane points satisfies one
of the consistencies or then the
coefficient A 0.
36
An Example Approach Capable of Smooth 3D Objects
Visualization
The use of the theorems and equation (5) yields
the following equations systems
i 1,2,3,4 (6),
i 1,2,3,4 (7).
If the co-ordinates of 4 plane points are given,
then the coefficients A,C,D,E and F may be
calculated solving the equation system (6) or (7).
Once the coefficients are calculated the
following criteria gives the type of the arc
interpolating the points
If A.C gt 0, an arc of ellipse interpolates the
points.
If A C, an arc of circle interpolates the
points.
If A.C lt 0, an arc of hyperbola interpolates the
points.
If A 0 or C 0, an arc of parabola
interpolates the points.
37
An Example Approach Capable of Smooth 3D Objects
Visualization
Five generated ore body sections using a file
containing 21375 classified samples. Ore type 3
is painted in black Ore type 2 in grey Ore type
1 in white grey.
The fifth sections of ore body after
interpolation of the borders.
38
4. 2D Objects Recognition and 3D Objects
Separation
The 3D-object separation problem falls into
dividing the entire set of 2D objects, to
non-crossing subsets of similar 2D-objects.
Each subset is used for surface reconstruction of
a single 3D object.

Assume set of objects
The set of all objects features
Numerical intervals, where each feature takes a
value
When fi takes a value from Di, for i 1,.,m
the obtained vectors represent a m-D Vector
Feature Space (VFS).
Norm of distance in the VFS
wj, j1,...,m , weights representing the
importance of the features.
39
An Example Approach Capable of Smooth 3D Objects
Visualization
If fni / 2 for n 1,..,m, then the
object , for i 1,..,k , is said be a
center of m-dimensional polyhedron Pi.
Definition 1. The object is said to be
similar to the object if .
We denote the similarity by Oi Oj.
Corollary 1. The object is similar to the
object if
.
The user select the features of the objects to be
used for Vector Feature Space definition.
Different ore types are determined and bounded
inside of each ore body section.
40
An Example Approach Capable of Smooth 3D Objects
Visualization Applying the recognition approach
the sent of 2D sections, given in Fig.5 is
divided to 3 different subsets. Each subset
represents one ore type.
The set of 2D sections ore type 1.
2D sections of ore type 2.
41
An Example Approach Capable of Smooth 3D Objects
Visualization
2D sections of ore type 3.
1)
2)
3)
Similar 2D sections among ore type 1) ore type
2) and ore type 3).
42
3D Reconstruction and Visualisation
The method developed for 3D objects
reconstruction and visualization is based on
notions such as regularities, consistencies,
corresponding essential border points, direction
and sequences of observation, Delaunay
Triangulation and Voronoi diagrams.
3D reconstruction and visualization of main ore
body. The 5th sections shown on Fig.3 were used
in order to perform the experiment.
43
An Example Approach Capable of Smooth 3D Objects
Visualization
3D reconstruction and visualization of the
objects belonging to ore type 1 in side of the
main ore body.
44
5. Interpolation of 2D Sections.
5.1. Morphological Interpolation
Let X and Y, X Y 0 be a couple of sections
to be interpolated. Denote by Z X?Y
To transform a 2D section X into another 2D
section Y, X would shrink and become X?Y. At the
same time, X?Y would grow and become Y. Hence,
this interpolation algorithm works only when
there is a non-empty intersection between the
sections.
Figure 10. Geodesic set Z as a union of X and Y
in case of intersection.
dX and dY are the distances between point x and
sections X and Y, respectively. dZ denotes the
distance to the boundary of section Z, and Zc
is the complementary of Z with respect to plane
Pi.
 
 
0 on X and 1 on the boundary of Z dX / (dX dZ),
on Z/X ? on Zc
0 on Y and 1 on the boundary of Z dY / (dY dZ)
on Z/Y ? on Zc


45
An Example Approach Capable of Smooth 3D Objects
Visualization
The interpolated 2D sections between X and Z, and
those between Y and Z, are obtained by means of
single threshold between 0 and 1
 
Once performed, the above interpolation enables
one to obtain the interpolated section between X
and Y, i.e. the section at a distance ? from X
and (1 - ?) from Y, by intersecting two partially
interpolated sections





- interpolated
section at a distance ? between X and Z







 
 
 


 
 
 
- interpolated section at a
distance (1 - ?) between Y and Z. Thus
In conclusion, applying the above expression, one
can generate an intermediate interpolated section
for certain ? values, where ? ? 0,1.
46
An Example Approach Capable of Smooth 3D Objects
Visualization
5.2. Geodesic Set Definition in Case of Empty
Intersection
Consider set of 2D sections S within a set of
finite planes P, together with the co-ordinate
system Oxyz, as well as a set of points N (xi,
yi, zi), i 1,..,n that belong to the 2D
section border. The following parametric
equation gives the polynomial curve L that
interpolate the points L x f(t) , y g(t) ,
z h(t)
x f(t) an-1.tn-1 an-2.tn-2 .
a0 (10) y g(t) bn-1.tn-1 bn-2.tn-2
.b0 z h(t) cn-1.tn-1
cn-2.tn-2 . c0
Denote by Un , Vn ,
Wn , Xn , Yn ,
Zn
Tn x n .
Thus, the equation system (10) takes the
following matrix form
Tn x n . Un Xn Tn x n . Vn Yn
Tn x n . Wn Zn (11).
47
An Example Approach Capable of Smooth 3D Objects
Visualization
Definition Consider point A belonging to the
border of a 2D section. We call A extreme left
(right) visible point with respect to the axis
Oz-, if its x co-ordinate is minimal (maximal)
with respect to the other border points.
Denote by NL the set of ELVP and by NR - the set
of ERVP, both belong to the elements of S.
Applying (11), we define a couple of spatial
polynomial curves CL and CR , that interpolate
NL and NR. The curves orthogonal projections,
in each plane, are used to bind a geodesic sets,
where the Morphological interpolation may work.
Geodesic set design using a couple of
non-crossing 2D sections.
Gi Op(Ai-1Bi-1)i Op(Bi-1)IC CD
Op(aR)i Bi Ai Op(aL)i
Geodesic set Gi created by the non-crossing 2D
sections Si and Op(Si-1).
48
An Example Approach Capable of Smooth 3D Objects
Visualization
Six simulated cross-sections (images) containing
impermeable (black), less permeable (dark gray),
permeable (white gray) 2D sections.
A set of impermeable 2D sections separated from
the images given in Fig.12.
49
Example Approach Capable of Smooth 3D Objects
Visualization
The set of impermeable 2D sections is divided
into subsets of similar 2D sections.
The set of ELVBP (ERVBP) is denoted by Ai (Bi)
for i 1,,6.
50
Example Approach Capable of Smooth 3D Objects
Visualization
The selected sets of ELVP and ERVP own the
following coordinates XL 150 106 174
114 35 178 ZL 396 345 423 368 423
360 YL t , XR 183 135 187 135 52
198 ZR 405 345 423 377 423 360 YR
t. The real distance between the cross sections
is 100 m. Therefore, without any restrictions of
our reasoning we may set the distance between the
images to be 1 and fix Tn (1,...,6).
XL u(t) -1.8t5 41.7t4 339.1t3
1211.8t2 1874.6t 1112 ZL v(t) -8.1t5
143t4 945t3 2893t2 4005.4t 2319
YL t (12), XR f(t) -0.7t5 22.8t4
213.2t3 826.7t2 1354.6t 902 ZR g(t)
-7.5t5 131.8t4 876.8t3 2706.2t2
3786.7t 2238 YR t. (13).
Using the equations (12) and (13), we define the
curve arcs aLi and aRi bordered by sections Si
and Si1, for i 1,2,,5 as well as Op(aLi)i1,
Op(aRi)i1, Op(Si)i1, Op(Ai)i1 and Op(Bi)i1.
51
Example Approach Capable of Smooth 3D Objects
Visualization
The initial 2D sections, together with the
generated between them.
52
Example Approach Capable of Smooth 3D Objects
Visualization
A. Initial 2D
sections, given in Fig.15. B. The initial
sections together with the
interpolated 2D ones between them.
A. 3D impermeable unit built by means of the
initial 2D sections. B. The impermeable
unit surface built by means of the
initial and interpolated 2D sections.
53
3D Reconstruction and Visualization

• a, b, c, d - the consecutive crack photographs
  taken after the slice cutting, i.e. the in-depth
  development A, B, C and D of the macrocrack

54
3D Reconstruction and Visualization

• The 2D crack extracted from the cement

The 3D shape of the crack
55
3D Reconstruction and Visualization

• The 3D object, reconstructed by using
• the top 4 sections
• Both cut and interpolated sections

• The top sections are cut from the object, the
  sections below are interpolated

56

• THANK YOU FOR ATTENDINGTHE SEMINARQUESTIONS!!!