Virtual and Contentbased Access of Multimedia Collections

1
Virtual and Content-based Access of Multimedia
Collections

• Eric Paquet and John Taylor
• National Research Council Canada

2
Visual Information Technology at NRC

• RD on 3D imaging modeling, display and
  multimedia management

3
Why do we need tools to manage multimedia?

• Various media text, sound, 2D images, video, 3D
  models
• Numerous objects
• Distributed Web, file systems, databases
• If it cannot be found, it does not exist and
  cannot be used!

4
Main topics

• Content-based management of 2D images
• Content-based management of 3D objets
• The Virtual Museum

5
What does content-based 2D mean?

• For a 2D picture the content includes the visual
  composition (the scene and the objects) as well
  as the colors

This complex photo includes a wall, an objects
with shape (Rosone), tripod camera and a figure
in blue.
6
KHEOPS - management of 2D images

• The images are described automatically and an
  index database is created
• Query by example
• Search based on color and composition

7
Content also means context - The Rosone
8
Three pictures of Venicegt 2,000 images
9
2D and shape gt 10,000 images
10
Main topics

• Content-based management of 2D images
• Content-based management of 3D objets
• The Virtual Museum

11
What does content-based 3D mean?

• For a 3D object it includes the 3D shape, the
  colors and, the spatial relations between the
  parts

CMC VII-B-136
12
ALEXANDRIA -management of 3D objects

• Triangular mesh representation 3D
  scanning, stereo, photogrammetry and computer
  generated
• Automatic description of 3D shape and color
• No standard orientation needed

13
Query for a car 14,000 3D objects in the database
14
Query for a chair 14,000 3D objects in the
database
15
Content-based advantages

• Content-based is not language dependent
• Many of the paradigms utilized in real life are
  in fact content-based paradigms
• I like the colors in this painting
• I like the shape of this vase

16
Main topics

• Content-based management of 2D images
• Content-based management of 3D objets
• The Virtual Museum

17
The Virtual Museum a 3D museum

• 3D objects scanned and computer generated
• Content-based search
• Media association video, HTML, text, ...
• Java 3D platform independent
• You can virtualize your own real museum by using
  digital pictures ShapeCaptureTM, ...

18
Natural History Museum...
19
From animals to birds
20
Furniture...
21
HTML or additional information
22
Virtualization of the museum
23
Conclusions

• Tools for managing 2D and 3D information by
  content
• The Virtual Museum
• Complete and integrated solution for virtual
  reality-based heritage applications
• More information www.cleopatra.nrc.ca