MPEGVideo Supplement

1
MPEG/Video Supplement
2
Standards

• Important for communications
• Customers prefer standards (freedom to choose)
• Increases volumes and bring down cost of service
  and SW/HW
• Reduce risk of deploying new technology
• Major players often participate
• Facilitate development on a common background
• Provide research opportunity

3
Global Standards

• International
• ISO International Organization for
  Standardization (www.iso.org)
• IEC International Electrotechnical Commission
  (www.iec.ch)
• ITU International Telecommunications Union (UN)
  (www.itu.int)
• ITU-T ITU Telecommunication Standardization
  Sector (CCITT)
• ITU-R ITU Radio Communication Sector (CCIR)
• JTC1 Joint Technical Committee on Information
  Technology
• National
• ANSI American National Standards Institute

4
Organizations of an ISO Standard Body

• Group WG1 (JPEG), WG11(MPEG)
• Convenor Danial Lee(JPEG),
• Leonardo Chiariglione
  (MPEG)
• Sub-Group Video, System, Audio and Conformance
• Ad Hoc Group Coding Efficiency, Encoder
  Optimization
• NB National Body Delegates
• HoD Head of Delegation
• Observer

5
ISO/IEC JTC1 SC29

• Study Committee (SC) 29
• Working Group (WG) 1
• Joint Bi-level Image Group(JBIP)
• Joint Photographic Experts Group (JPEG)
• WG11
• Moving Picture Experts Group (MPEG)
• WG12
• Multimedia Hypermedia Experts Group (MHEG)

6
How does Standards Work?

• Schedule
• 3 to 5 one week meeting in different nations each
  year
• 300 to 400 delegate from around the world
• 200 companies from over 50 nations
• A final standard in about 4-5 years

7
Proposal Review Process

• Call for proposal
• CE Core Experiments Process
• Complete descriptions with at least one
  independent verification
• One functionality one tool as reviewed by peer
• Consensus based decision process at AHG, SG level
  
• CE Core Experiment Description
• Proposal that is relevant and is supported by two
  companies
• VM Verification Model
• The best proposal is admitted to VM for everyone
  to implement
• The new reference for the best performance
• The proposal needs to be challenged by incoming
  proposal

8
Proposal Review Process (Cont.)

• WD Working Draft
• CD Committee Draft
• All the doors will be frozen
• First round vote by National Bodies with comment
• FCD Final Committee draft
• DIS Draft International Standard
• Second round vote by National Bodies without
  comment
• ISInternational standard

9
Why Does Company Work in the Standards

• Interoperability war of formats (VHS vs. Beta)
• Patent Royalties
• licensing fee for MPEG-2 box US4
• Total licensing fee for DVD US10
• Bi companies can avoid being taxed by other
  companies
• 250 Millions per year for RCA patent profiles
• Create new market
• VCD Video Compact Disk
• DVD Digital Versatile Disk
• DBS Direct Broadcast System
• HDTV or DVB (Digital Video Broadcasting)

10
MPEG

• Motion Picture Expert Group
• ISO/IEC JTC1/SC29/WG11
• MPEG Standards
• MPEG-1 (ISO/IEC 11172) 92
• MPEG-2 (ISO/IEC 13818) 94
• MPEG-4 (ISO/IEC 14496) 98
• MPEG-7 (ongoing) ref. N5525, Mar. 2003
• MPEG-21 (ongoing) ref. N5231, Oct. 2002
• Only bit stream syntax decoding are specified

11
History of MPEG

• MPEG-1
• started in 1988
• Coding of moving pictures and associated audio
  for digital storage media at up to about 1.5 Mb/s
• compression standards for progressive frame-based
  video in SIF (360x240) 352x240
• Applications VCD
• MPEG-2
• Generic coding of moving pictures and associated
  audio
• compression standard for interlaced frame-based
  video in CCIR-601(720x480) and HDTV (1920x1088)
• Applications DVD, SVCD, Direct TV, DVB, HDTV

SIF VCR-quality, 30Hz/25Hz Non-interlaced
411 Y360x240 (260x288)
12
History of MPEG (Cont.)

• MPEG-4
• Very low bit rate audio-visual coding
• Multimedia Standard for object-based video from
  natural or synthetic source
• Applications Internet, cable TV, virtual studio,
  home LAN
• MPEG-7
• Multimedia content description interface
• Applications Internet, video search engine,
  digital library

13
Applications of MPEG-1 2

• Digital storage media 11.5 Mb/s
• Asymmetric applications
• Electronic publishing
• education training
• travel guidance
• Videotext (animated video instruction)
• point of sale
• Games
• Entertainment
• Symmetric Applications
• Electronic publishing
• Video mail
• Video-telephone
• Video conferencing

14
(No Transcript)
15
Features of Video Compression Algorithm

• Random access (latency 0.5 seconds)
• Fast Forward/Reverse Searches
• Reverse playback
• Audio-Visual Synchronization
• Robustness to errors
• Coding/decoding delay (150ms of videotelephone)
• Editability
• Format Flexibility raster size frame rate
• Cost tradeoffs implementable

16
Principles of MPEG

• Spatial redundancy (Intra-frame)
• DCT
• Temporal redundancy(Inter-frame)
• block based motion compensation
• MB 16x16 macro block
• Each with an address

17
(No Transcript)
18
Predicted span GOP
19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
Motion estimation and compensation

• Motion estimation estimate motion parameters of
  moving objects in an image sequence
• At the encoder
• Motion compensation replace a picture or portion
  thereof, based on displaced pels of a previously
  transmitted frame in an image sequence
• At the encoder
• Why motion compensation
• Reduce interframe correlation
• Block motion compensation is adopted by
  H/261/H.263, MPEG1/2

23
Motion Estimation

• Predict current frame from previous frame
• Motion Estimation Algorithm
• Block matching method
• Pel based
• Block based
• Object based
• Differential (gradient) method optical flow
• Fourier method

24
Motion estimation problem

• Moving object a group of contiguous pels that
  share the same set of motion parameters not
  necessarily match the ordinary meaning of object
• Assumptions
• Objects are rigid body object deformation can be
  neglected for at least a few nearby frames
• Objects move only in translational movement for
  at least a few frames

25
Motion estimation problem (Cont.)

• Assumptions (cont.)
• Illumination is spatially and temporally uniform
  the observed object intensities are unchanged
  under movement
• Occlusion of one object by another and uncovered
  background are neglected

26
Block matching motion estimation

• Concept correlation technique that searches for
  the best match between the current image block
  and candidates in a confirmed area of previous
  frame
• Assumptions image are partitioned into
  non-overlapped rectangular blocks
• Each block is viewed as an independent object
• The motion of pels within the same block is
  uniform

27
Motion Estimation
28
Factors Affect Block-based Matching Algorithm

• Searching algorithm
• Matching criteria
• Searching range

29
(No Transcript)
30
(No Transcript)
31
Matching function

• Mean squared error (MSE) (min)
• MSE(d1, d2) 1/N1N2S S
• f(n1, n2 , t) f((n1 - d1, n2 d2 , t-1)2
• Mean absolute difference (MAD) (min)
• MAD(d1, d2) 1/N1N2S S
• f(n1, n2 , t) f((n1 - d1, n2 d2 , t-1)
• d1, d2 are distance away

32
(No Transcript)
33
MPEG-Decoder
34
MPEG Encoder
35
Parts of MPEG-1

• ISO/IEC 11172-1 Systems
• ISO/IEC 11172-2 Video
• ISO/IEC 11172-3 Audio
• ISO/IEC 11172-4 Conformance Testing
• ISO/IEC 11172-5 Software

36
Parameters of MPEG-1

• Picture size up to 4096x4096, normally 360x240
• Pel aspect ratio choices
• Picture rates 23.976, 24,25, 29.97, 30, 50,
  59.94, 60
• 420 format

37
MPEG Layers
38
(No Transcript)
39
Slice in MPEG
(MPEG2 not across line)
40
Parts of MPEG-2

• ISO/IEC 13818-1 Systems
• ISO/IEC 13818-2 Video
• ISO/IEC 13818-3 Audio
• ISO/IEC 13818-4 Compliance Testing
• ISO/IEC 13818-5 Software
• ISO/IEC 13818-6 DSM-CC
• ISO/IEC 13818-7 NBC Audio
• ISO/IEC 13818-8 10-Bit Video (dropped)
• ISO/IEC 13818-9 Real-Time Interface
• ISO/IEC 13818-10 DSM-CC Conformance

41
Differences between MPEG-1 MPEG-2

• MPEG-2 is is backwards compatible to MPEG-1
• MPEG-1 progressive, MPEG-2 interlace
• MPEG-1 fixed picture rate, MPEG-2 low delay
  mode for big picture
• Slice boundary
• Alternate scan for DCT Coef.
• MPEG-2 scalable video coding
• New VLC table for DCT Coef.
• Nonlinear quantization table
• IDCT mismatch control

42
Scalability

• SNR scalability
• Same luminance resolution and format, lower layer
  (420 more error correction MPEG-1)
• A single enhancement layer (422, less resilient
  to error)
• Spatial
• Base layer at lower resolution independent coded
• Enhancement difference between interpolated of
  base and source image

43
Scalability (Cont.)

• Temporal scalability
• Extension to higher temporal picture rate,
  backward compatible with lower-rate
• Base temporal rate coded independently
• Temporal prediction relative to base layer
• Data Partition extension
• Two channel transmission /storage
• Header, motion vector, low freq DCT coeff
• Less critical information, high freq DCT coeff,
  less error protection

44
Profiles

• Simple profile (SP)
• Main profile (MP)
• SNR scalable profile (SNR)
• Spatially scalable (Spt)
• High profile (HP)

45
Parameters of Profile
46
Color Space in MPEG

• YCbCr
• Luminance Chrominance ratio
• 420 (MPEG-1, MPEG-2)
• 444 (MPEG-2)
• 420 (MPEG-2)

47
Chrominance Sampling
48
Chrominance Sampling (Cont.)

• Given a CCIR typical frame 720x480

49
Scan Order in MPEG
50
Typical Frame Size of MPEG

• with an I frame distance of 15 and a P frame
  distance of 3
• I P B Average
• 30 Hz SIF _at_ 1.15Mbit/sec 150,000 50,000
  20,000 38,000
• 30 Hz CCIR 601_at_ 4Mbit/sec 400,000 200,000
  80,000 130,000

51
Frame Order of MPEG

• Time I B B P B B P B B P B B P B B I
• frame 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
• Dec. I P B B P B B P B B P B B I B B
• frame 1 4 2 3 7 5 6 10 8 9 13 11 12 16 14 15

52
MPEG Myths

• Compression Ratios over 1001
• MPEG-1 is 352x240 up to 4095 x 4095 and a bit
  rates up to 100 Mbit/sec
• Motion Compensation displaces macroblocks from
  previous pictures
• Display picture size is the same as the coded
  picture size
• Picture coding types (I, P, B) all consist of the
  same macroblocks types
• Sequence structure is fixed to a specific I,P,B
  frame pattern

53
Other Video Standards

• ITU-T H.261 px64 teleconferencing standard,
  64kb/s
• ITU-T H.263 low bit rate teleconferencing, 64kb/s
• ATSC(Advance Television Systems Committee)
• MPEG-2 video Dolby AC3
• DVB (European Digital Video Broadcast) MPEG-2
• DVC (Digital Video cassette Consortium) 6mm tape
  MPEG-2 I-frame-like coding 25Mb/s
• MJPEG(Motion JPEG)