MPEG

1
MPEG
2
MPEG

• MPEG-1 for high quality video on CDROM
  supporting bit rate of 1.5MBits/s, VCD (MP3 layer
  3)
• MPEG2 addresses applications involving video
  quality between NTSC/PAL and CCIR a range that is
  suitable for cable TV broadcast, narrow channel
  satellite broadcasting, 2 to 10Mbits/s, DVD
• MPEG1 and MPEG2 are meant for storage and
  transmission efficiency
• MPEG4 provides improved video compression
  efficiency. Targets bit rates of 5 to 64
  Kbits/s for mobile devices and 4Mbits/s for TV
  and film
• MPEG7 A sophisticated system meant for content
  management. Encodes high semantic information
• MM content description (metadata)
• MPEG21 Builds in the concept of digital rights
  management. Watermarking algorithms for videos
  and images

3
MPEG7

• What do we want to do
• Graphics draw a few lines and get set of images
  containing similar graphics or logos
• Images define objects using colour, shape or
  texture and get similar images
• Video describe object movements, camera motion
  and relations between objects and get list of
  videos with similar temporal and spatial
  relations
• Video Activity given video content describe
  actions and get list of videos in which similar
  actions occur

4
MPEG7 (Image descriptors)

• Colour descriptors
• Colour distributions, layout, dominant colour
• Texture descriptors
• Shape descriptors
• Region and contour descriptors
• 2D and 3D object boundaries
• Motion descriptors
• Camera, objects
• Face descriptors
• 49 basis vectors which span the space of possible
  face vectors

5
Texture descriptors

• No universal definition of texture
• Useful for browsing aerial databases
• E.g. car park with cars parked at regular
  intervals gives texture,
• vegetation gives variety of textures trees, corn
• Homogeneous (similar) texture descriptor
• Used for similarity
• Texture browsing
• Regularity, coarseness,
• directionality
• Edge histogram
• Useful for natural images, sketches, clipart

6
Shape descriptors

• Region based
• Compact and efficient way of describing disjoint
  regions
• Can cope with errors when object is split into
  disconnected regions
• Compact and resistant to noise

Distinguishes (i) from (g) And (h) (j), (k) and
(l) are similar
7

• Contour based
• Emulates human perception of similarity
• Robust to non- rigid deformations e.g. running
  person
• Compact
• Distinguishes
• differences in (b)
• similarities in (c), (d) and (e)

8
3D Descriptors

• 3D shape
• Can retrieve articulated objects with different
  postures
• Robust to scale and transforms
• Shape descriptors should be invariant to scale,
  rotation and position

9
Motion descriptors

• Camera motion
• Eight basic motions
• Fixed, Panning (horizontal rotation), Tilting
  (vertical rotation), Tracking (horizontal
  transverse movement), Booming (vertical
  transverse movement), Zooming (change of focal
  length), Dollying ( translation along optical
  axis), Rolling (rotation around optical axis)
• Rotations- overview of large scene or follow
  object
• Translations follow objects for longer period
  of time
• Zoom focus of attention, often precedes close
  up
• Roll high dive sequence

10
Motion descriptors

• Motion activity
• High speed car chase, goal scoring
• Low speed interview, news report
• Motion trajectory
• Displacement of an object in time
• Parametric motion
• motion of a region
• e.g. translation, rotation

11
Video Segments

• Can relate objects within or between segments
• By position
• Is left of
• Is near to
• In front of
• Moves towards
• By object
• Ball
• Player
• Goalkeeper

12
Sound Descriptors

• Type Feature Descriptors
• Audio Silence Silence
• Timbre InstrumentTimbre
• HarmonicInstrumentTimbre
• PercussiveInstrumentTimbre
• Speech Phoneme
• Articulation
• Language
• MusicalStructure MelodicContour
• Rhythm
• SoundEffects Reverberation, Pitch, Contour,
  Noise

13
Sound

• Context
• How to search for melodies?
• Needs to be both efficient and tolerant to pitch
  timing imprecision
• Approach
• Melody description encodes
• Pitch
• Timing/rhythmic information
• Other optional information (key )
• Applications
• Query by Humming / Playing / Singing
• Search for themes in music catalogues
• Musicology

14

• How to automatically recognize/identify a
  recording?
• Store unique MPEG-7 signature / fingerprint of
  original items in DB (robust, compact)
• Identification of unknown audio material by
  matching its signature with the DB signatures

15
Applications

• Content based queries from image and video
  databases
• Video browsing
• Find active parts of soccer match,
• Follow ball,
• Find where ball has been kicked high in the air
• Video repurposing
• Drop frames when action is low
• Slow down frame rate when camera motion is fast

16
More Applications

• Query by humming
• Query for spoken content
• Music recognition on handheld devices (PDA, cell
  phone),
• Content search on the Internet
• How to find metadata for a given piece of legacy
  content (CD, VCR Tape, ... No metadata
  attached!)?

17
More applications

• Storage and retrieval of audiovisual databases
  (image, film, radio archives)
• Broadcast media selection (radio, TV programs)
• Surveillance (traffic control, surface
  transportation, production chains, trigger
  alarms)
• E-commerce and Tele-shopping (searching for
  clothes / patterns)
• Remote sensing (cartography, ecology, natural
  resources management)
• Entertainment (searching for a game, for a
  karaoke)

18
More applications

• Cultural services (museums, art galleries)
• Journalism (searching for events, persons)
• Personalized news service on Internet (push media
  filtering)
• Intelligent multimedia presentations
• Educational applications
• Bio-medical applications
• Social (e.g. dating services).
• Architecture, real estate, and interior design
  (e.g., searching for ideas).