Video As A Datatype

1
Video As A Datatype

• Ketan Mayer-Patel

2
The mothers of all video...

• NTSC - American standard
• PAL - European standard
• SECAM - French standard (ugly stepchild)
• For better or worse, this is where it all starts.

3
Color Spaces

• One luminance component
• Two chrominance components
• Original TV was black and white.
• Adding color had to be done in a compatible way.
• NTSC YIQ
• PAL YUV
• In general YUV and YCrCb used as terms.

4
YUV vs. RGB

• Simple linear transform from one cube to the
  other.
• Specific transform coefficients vary for specific
  systems, but a common one
• Y 0.299 R 0.587 G 0.114 B
• Cr -0.147 R - 0.289 G 0.437 B 0.5
• Cb 0.615 R - 0.515 G - 0.100 B 0.5

5
YUV Challenges

• YUV is like taking RGB cube, standing it on a
  corner with (0,0,0) on bottom and (1,1,1) at top,
  rotating slightly, and taking the bounding box.
• What problems might this incur?
• Some RGB colors are illegal.
• A lot of YUV colors are illegal.

6
A Note On Color

• Previous view of color is EXTREMELY simplified.
• Color is complicated
• Frequency dependent response for contrast,
  lightness, etc.
• Gamma correction.

7
Scanning

• Analog video signal is continuous voltage signal
  that gets scanned along the screen.
• The electron stream controlled by two orthogonal
  sets of magnets.
• Horizontal Beam is moved from left to right and
  then quickly back.
• Vertically Beam is moved from top to bottom and
  then quickly back.

8
Scanning illustrated
Voltage
Time
9
Structure of Video

• Blinns article.
• Macrostructure
• Frequency peaks at frame rate.
• Microstructure
• Frequency peaks at line rate
• Adding color is a good first example of
  source-aware channel coding.
• What was the problem?
• What was the solution?
• Why does it work?
• What are its drawbacks (i.e., when does it fail)?

10
Interlacing

• Progressive every scan line done in turn.
• Interlaced every other scan line done.
• Creates two fields odd field and even field.
• NTSC 262.5 lines per field at 60 fields per sec.
• PAL 312.5 lines per field at 50 fields per sec.
• Fields are separated in time.

11
Analog Bandwidths

• True meaning of the bandwidth.
• Y, U, and V signals are all continuous along a
  scan line.
• A bit of a hybrid signal discrete vertically,
  continuous horizontally.
• NTSC
• Y is 4.2 MHz wide, I is 2 MHz, Q is 1 MHz
• PAL
• Y is 6 MHz wide, U is 3 MHz, V is 3 MHz

12
Corresponding Data Rates

• How much data can you put in 6 MHz band?
• Depends on S/N ratio.
• Depends on modulation scheme.
• Typical 27 - 36 Mbs
• How many cable channels do you get?
• If 50, then 1.3 - 1.8 Gbs coming into your home.
• The real question for multimedia is
• Why havent we found a better use for 1.3 Gbs
  than continuous broadcast of Real World.

13
Digital Video Frames

• Almost always progressive
• 3 planes of pixel values (Y, U, and V)
• Pixel depth
• Geometry of each plane width x height
• Chrominance is generally subsampled.
• How the planes relate to each other.
• Frame rate.

14
CCIR-601

• Standard established for digitizing NTSC and PAL
  signals.

15
Why 8-bits for chrominance?

• Whats another way to cut chrominance bandwidth
  in half?
• Use 4-bits per pixel.
• Why wont that work?
• Need the dynamic range for color.
• But what about when Y is either small or large?
• Dont need the range, but lots of YUV
  combinations that wont ever be used.

16
422

• For every 4 luminance samples, take 2 chrominance
  samples from odd lines and 2 from even lines.
• Chrom. planes just as tall, half as wide.
• JPEG does this.

17
420

• 2 chrominance samples for every 4 luminance
  samples, odd lines only.
• Chrominance halved in both directions.
• MPEG generally does this.

18
411

• What should this be?
• 1 chrominance for every 4 luminance for both odd
  and even lines.
• And that is what it is.
• But sometimes used to refer to this

19
De-interlacing

• Since analog video fields are separated in time
  by 1/2 the frame rate, at least half of a digital
  frame is missing no matter where you sample from.
• Normal solution linear interpolation of even
  fields to produce matching samples for odd
  fields.
• Even better linear interpolation of both fields
  to produce matching samples and thus digital
  frame rate will equal field rate.

20
Film Frame Rates

• Film is a different beast altogether.
• Continuous both vertically and horizontally
• Projection is simultaneous for all parts of the
  picture.
• Expensive medium.
• Combination of all of this motivates 24 fps.
• Film to digital is easier than video to digital.
• No interlacing, sample where you want to.
• Film to video is harder.

21
32 Pulldown

• Converts 24 frames to 60 fields.

22
Overall sampling lesson

• Cant recover what you dont have.
• Conversion between representations requires
  estimation of missing samples.
• Interpolation causes errors
• Spatially at the edges.
• Temporally when moving.

23
Common Digital Video Sizes

• CCIR-601 720x480 422, 420
• SIF 360x240 420
• CIF 360x288 420
• 43 HDTV 1440x1152 422, 420
• 916 HDTV 1920x1152 422, 420
• 4CIF, 16CIF, QCIF

24
Digital Bitrates

• Current television
• 30 fps 720 480 1.5 8 124 Mb/s
• 916 HDTV
• 30 fps 1920 1152 1.5 8 796Mb/s
• This motivates compression.