Indexing and Mining Biological Images

1
Indexing and Mining Biological Images

• Christos Faloutsos
• CMU

2
Outline

• Motivation - Problem Definition
• Proposed method
• Experiments
• Conclusions

3
ViVo

• with Ambuj Singh, Vebjorn Ljosa, Arnab
  Bhattacharya (UCSB)
• Jia-Yu Tim Pan, HJ Yang (CMU)

4
Detachment Development
1 day after detachment
3 days after detachment
Normal
3 months after detachment
7 days after detachment
28 days after detachment
5
Data and Problem

• (Data) Retinal images taken from cats
• (Problem) What happens in retina after
  detachment?
• What tissues (regions) are involved?
• How do they change over time?
• How will a program convey this info?
• More than classificationwe want to learn what
  classifier learned

6
Why study retinal detachment
skip

• Common damage to retina
• No effective treatment
• Surgery or drugs (lt100 recovery)
• Need to understand more about detachment
  development

7
Retina, its image, and the detachment
skip

• retina

Layers of tissues
stained by 3 antibodies (R,G,B)
8
Computer Scientists View of Retinal Detachment
skip
normal
detachment
7 days after
9
Detachment Development
1 day after detachment
3 days after detachment
Normal
3 months after detachment
7 days after detachment
28 days after detachment
10
How do the treatments do?
28 days after reattachment surgery
6 days after O2 treatment
11
Outline

• Motivation - Problem Definition
• Proposed method
• Experiments
• Conclusions

12
Main idea

• extract characteristic visual words
• Equivalent to characteristic keywords, in a
  collection of text documents

13
More than classificationwe want to learn what
classifier learned
skip

• Proposed method visual vocabulary (vivo)
• Try to capture local tissue texture variations
• (from stage to stage)
• Quality of vivo
• Classification Biological meaning?
• Lessons learn?
• Which tissue/texture is significant at stage
  7-day?
• What changes between 3-day and 7-day?

14
Visual Vocabulary (ViVo) generation
Visualvocabulary
Independent component analysis (ICA)
Tile image
Extract color structure features
15
Proposed method ViVo
skip

• Textures are different.
• Wavelet (Daubechies-4), MPEG-7 color structure
• Local variation partitioned into 64x64 tiles.

f1, , fm tile-vector
16
ViVos
skip
17
Outline

• Motivation - Problem Definition
• Proposed method
• Experiments
• Conclusions

18
Evaluation of ViVo method

• how meaningful are the discovered ViVos?
• can they help in classification?
• generality?
• how else can they help biologists create
  hypotheses?

19
Example ViVos
20
Goals

• how meaningful are the discovered ViVos?
• can they help in classification?
• generality?
• how else can they help biologists create
  hypotheses?

21
Quality of ViVo by classification
Predicted
Truth
86 accuracy 46 ViVos (90 energy)
22
Goals

• how meaningful are the discovered ViVos?
• can they help in classification?
• generality?
• how else can they help biologists create
  hypotheses?

23
ViVos for protein images
24
Protein images MPEG7 CS
Predicted
Truth
84 accuracy 4 ViVos (93 energy) 1-NN classifier
25
Evaluation of ViVo method

• how meaningful are the discovered ViVos?
• can they help in classification?
• generality?
• how else can they help biologists create
  hypotheses? ViVo-annotation!

26
Automatic ViVo-annotation of images

• A tile represents a ViVo vk if the largest
  coefficient of the tile is along the kth basis
  vector
• A ViVo vk represents a class ci if the majority
  of its tiles are in that class
• For each image, the representative ViVos for the
  class are automatically highlighted

27
Which tissue is significant on 7-day?
28
6 days after O2 treatment
29
28 days after surgery
30
Most discriminative ViVos
n
1d6dO2
31
Conclusions

• how meaningful are the discovered ViVos?
• can they help in classification?
• generality?
• how else can they help biologists create
  hypotheses?

32
Outcome/status

• What are the key results so far?
• ViVos Automatic Visual Vocabulary generation for
  biomedical image mining, Bhattacharya, Ljosa,
  Pan, Yang, Faloutsos, Singh (under review)
• Software MATLAB code
• Tutorial in SIGMOD05 (MurphyFaloutsos)