CIS750

1
CIS750 Seminar in Advanced Topics in Computer
ScienceAdvanced topics in databases
Multimedia Databases

• V. Megalooikonomou
• Introduction

2
General Overview

• Multimedia data
• Multimedia data applications
• Access of different media types
• Operations on multimedia data
• Infrastructure to support operations
• Content Extracting media content indexing
  queries
• Physical storage storing media data
• Creating presentations Creating media
  presentations (in response to queries)
  delivering them to users at remote sites

3
Media types

• Text/Document
• Image
• Video
• Audio
• Traditional data (e.g., relations, flat files,
  object bases, etc)
• Video and audio differ from the other media types
  (temporal info)
• retrievals must be continuous
• support fast-forward, rewind, pause

4
Multimedia DBMS

• A framework that manages different types of data
  potentially represented in a wide diversity of
  formats on a wide array of media sources
• Abilities of Multimedia DBMSs (MMDBMSs)
• Uniformly query data represented in different
  formats
• Simultaneously query different media sources and
  apply traditional (classical) database operations
  on them
• Retrieve media objects from local storage devices
  in a smooth jitter free (i.e., continuous) manner
• Develop a presentation of the answer generated by
  a query and deliver the presentation in way that
  satisfies Quality of Service

5
Sample Multimedia Scenario

• Scenario A police investigation of a large-scale
  drug operation
• Types of data
• Video from surveillance cameras that record
  activities at various locations
• Audio from legally authorized telephone wiretaps
• Images from still photographs taken by
  investigators
• Documents seized by the police
• Structured relational data containing background
  information, bank records, etc.
• Geographical information systems data containing
  geographic data relevant to the investigation

6
Example Images Queries for Multimedia scenario

• Query 1
• Retrieve all images from the image library in
  which the person appearing in the (currently
  displayed) photograph appears
• Query 2
• Retrieve all images from the image library in
  which Dennis Dopeman appears

7
Example Image Queries Issues

• Image based vs keyword-based queries
• Output a ranked list of images that are
  similar to the query image
• Similarity?
• Ranking?
• Efficiency in supporting these queries
• Association of different attributes with
  images(or parts of images)
• Indexing and retrieving images effectively

8
Example Audio Query for Multimedia Scenario

• Query 1
• Listening to a tape containing a conversation
  between individual A (person under surveillance)
  and individual B (somebody meeting person A).
  Query Find the identity of individual B, given
  that individual A is Denis Dopeman
• Query 2
• Find all audio tapes in which Denis Dopeman was a
  participant

9
Example Text/Video Query

• Text Query Find all documents (from the corpora
  of text documents, e.g., newspaper archives,
  police dept. files on old, unsolved cases,
  witness statements, etc) that deal with the Cali
  drug cartels financial transactions with ABC
  Corp.
• Video Query Find all video segments in which the
  victim of the assault appears.

10
Heterogeneous Query

• Complex queries will mix and match data from
  different media sources
• Textual example Find all individuals who have
  been convicted of attempted murder in North
  America and who have recently has electronic
  fund transfers made into their bank accounts from
  ABC Corp.
• Answering this query is problematic
• May requiring accessing a wide variety of
  databases belonging to different police
  jurisdictions, etc
• ABC Corp. may have accounts in hundreds of banks
  worldwide each using different formats and
  database systems

11
Heterogeneous Multimedia Query

• Find all individuals who have been photographed
  with Jose Orojuelo and who have been convicted of
  attempted murder in North America and who have
  recently has electronic fund transfers made into
  their bank accounts from ABC Corp.
• This query requires
• Accessing a database containing names and
  pictures of various individuals
• Accessing photograph database of still images,
  surveillance video database
• Accessing image processing algorithms to
  determine who occurs in which video/still
  photograph

12
Multimedia Research Issues Queries

• A single language within which multimedia data of
  different types can be accessed
• Be able to specify combination operations across
  different media types (in addition to across
  different relations through join, union,
  difference, etc)
• Be able to access metadata (describing the
  content of different media sources) and raw
  data supported by different media sources
• Be able to merge, manipulate, and join results
  from different media sources
• Optimize a single query or a set of queries

13
Multimedia Research Issues Content

• What is the content of a media source? Under what
  conditions can content be described textually and
  under what conditions must it be described
  directly through the original media type?
• How should we extract the content of an image,
  video-clip, audio-clip, free/structured document?
• How to index the results of the extracted
  content?
• How to efficiently retrieve media data on the
  basis of similarity?
• How one had to design a multimedia database
• How should queries be relaxed so that not only
  the original but also similar queries get
  processed?
• What are efficient algorithms for processing
  these queries?

14
Multimedia Research Issues Storage

• How do the following (standard) storage devices
  work?
• Disk systems
• CD-ROM systems
• Tape systems and tape libraries
• How is data laid out on such devices?
• How do we design disk/CD-ROM/tape servers so as
  to optimally satisfy different clients
  concurrently, when the operations are
• Playback
• Rewind
• Fast forward
• Pause

15
Multimedia Research Issues Presentations and
Delivery

• How to specify the content of multimedia
  presentations?
• How to specify the form (temporal/spatial layout)
  of this content?
• How to create a presentation schedule that
  satisfies these temporal/spatial presentation
  requirements?
• How to deliver a multimedia presentation to users
  when
• There is a need to interact with remote servers
  to assemble the presentation
• There are bounds on the bandwidth, load and other
  resources
• There is a mismatch between host servers
  capabilities and customers machine capabilities
• How to achieve Quality of Service?