INSYS 300 Multimedia Storage and Retrieval

1
INSYS 300 Multimedia Storage and Retrieval
2
What is Multimedia? What is Media?

• Text, email
• Images
• Audio (Speech, Music)
• Video (Many types of video)
• Flash presentations, animations
• Maps
• CAD output
• Virtual reality, Games
• Learning objects
• Software?

3
Multimedia Storage and Retrieval

• Information needs gt Content requirements
• Descriptions / Controlled vocabularies
• Interaction model
• Indexing/representation strategy
• Building collections/repositories
• System Requirements

4
What can we do with Multimedia

• Processing
• Compression, Resizing, Merging
• Content analysis (recognition)
• Metadata
• Representation/matching queries
• Preservation
• Synthesis

5
Visual Object Recognition

• Bottom-up vs. Top Down Processing
• Attribute matching vs. Shape matching

6
Face and Person Recognition

• Gesture, posture, etc

7
Image Processing and Retrieval

• Image segmentation
• QBIC http//wwwqbic.almaden.ibm.com/
• Map images http//www.davidrumsey.com/
• Query by Sketching

8
Text as Images

• Scanning
• Document segmentation
• OCR

9
GIS (Geo-spatial information Systems)

• Types of queries
• Range queries
• Maps
• Processing maps
• Map metadata

10
Speech Processing

• Representing speech with Phonemes
• Basic sound units, In English there are about 56
  phonemes
• Vowels vs. consonants
• Types of consonants plosives, fricatives
• Many applications
• Speaker identification
• Word spotting
• Language recognition

11
Original Sound Wave
Sampled Sound Wave
Frequency Representation
Wave Representation
12
Speech Recognition

• Can we find the phonemes?
• Spectrograms From waveform to frequency
• Look for formants

13
(No Transcript)
14
Speech Libraries

• http//www.ngsw.org

15
Music

• Music is highly structured
• Music synthesis
• Note recognition
• Representation of music
• Score
• MIDI
• Audio contours
• Query by humming

16
(No Transcript)
17
MPEG

• MPEG-1
• 1.5MB compression with frame differences
• MPEG-2
• 45MB compression
• MPEG-4
• ISDN compression semantic compression
• MPEG-7
• Descriptions of content (e.g., Dublin Core for
  video)
• MPEG-21
• Composites of media (developing a photo album)

18
(No Transcript)
19
Video Retrieval

• IBrowse
• Now many commercial systems
• Virage
• Informedia