NTHU%20H.264/AVC%20Video%20Encoder%20

1
NTHU H.264/AVC Video Encoder Decoder

• Youn-Long Lin
• Department of Computer Science
• National Tsing Hua University
• Hsin-Chu, TAIWAN 300
• ylin_at_cs.nthu.edu.tw

2007/02/10 IC-DFN, Las Vegas
2
Evolution of Video Coding Standards
DVD/DTV
VCD
MP3
DV/IPCam
3
Get More (Quality) for Less (Bit-rate)
H.264
MPEG2
4
Ref G. Sullivan T. Wiegand, Video
CompressionFrom Concepts to the H.264/AVC
Standard, Proceedings of the IEEE, Vol 93, No.1,
Jan 2005
Effectiveness of basic techniques

• No MC
• Adding Skip mode to form a CR coder.
• Allow only zero-valued MVs.
• Allow integer-pel MC.
• Allow half-pel MC
• Allowing 4-MV
• Allowing quarter-pel MC.

5
Features of Video Coding Standards
Standard MPEG-1 MPEG-2 MPEG-4 H.264
MB size 1616 1616(frame) 1616 1616
Block size 88 88 1616, 88 1616, 168, 816, 88, 84, 48, 44
Transform DCT DCT DCT/ Wavelet 44 int transform
Entropy coding VLC VLC VLC VLC, CAVLC and CABAC
ME, MC Yes Yes Yes 41 MVs per MB
Pixel accuracy ½ pel ½ pel ¼ pel ¼ pel
Reference frames One frame One frame One frame Multiple (5) frames
Picture type I, P, B I, P, B I, P, B I, P, B
Transmission rate Up to 1.5 Mbps 2-15 Mbps 64kbps2Mbps 64kbps 150Mbps
6
Competing Standards

• H.264 Advanced Video Coding (H.264/AVC)
• Also Called MPEG-4 Part 10
• WMV/VC-1 (MicroSoft)
• Chinese AVS (Audio Video Coding Standard)

7
H.264/AVC Profiles
Extended profile
B slice
SP, SI slice
Interlace
Main profile
Weighted prediction
Data partition
CABAC
FREext (High) profile
I slice
Slice group
8x8 transform
P slice
ASO
Quantization matrix
Baseline profile
CAVLC
Redundant Slice
Color Sampling
8/10/12 bit sampling

8
Global UniChip Multimedia SOC Platform
CPU
Accelerator (FPGA)
USB(PHY) Daughter Board
ROM/ Flash Memory SRAM
SDRAM
FPGA
VIC
USB 2.0
Static memory
SDRAM Controller(4-CH)
High-Speed Bus
JPEG Codec
DMA
SRAM
PWM
WDT
TIMER
APB Bridge
Capture
Display Controller
Peripheral Bus
DAI
SSI
SD
SM
UART
GPIO
12C
Audio Codec I2S
Flash memory with SSI
Flash Card
Button
LED
Video-In CCIR601
TV/LCD
9
H.264/AVC Decoder System Diagram
UART
TV
Timer
SD Card
ARM926EJS
Slave
Master
Slave
Slave
AHB1
Slave
Slave
Slave
SDRAM
SDC
SDC
H264
Slave
Slave
Master
SDRAM
LM
AHB2
10
H.264/AVC Decoder Architecture

11
Hierarchical FSM in Main Controller
Frame Level
MB Level
Main FSM
CABAC
CABAC FSM
MC
MC FSM
Type decoder
rden
IPRED
IPRED FSM
rd_addr
rd_data
IQ/IDCT
IQ/IDCT FSM
PICREC
PICREC FSM
DF
DF FSM
Main controller
12
AMBA interface
AHB A
LM
slave wrapper
control register
H.264 Decoder
MFU
VLC TV OUT
DF MC
SDC
arbiter 1
arbiter 2
master wrapper 1
master wrapper 2
AHB B
13
NTHU Design Flow
Software spec. in C Acceleration specify
User Spec.
SW lib. C models, drivers
Platform spec.
API
System configuration
System.h
Embedded Software
Compilation
System description
Acceleration
Acceleration
Software image
System generation
HW lib. HDL IPs
HW IP Synthesizer
Co-Sim
HW/SW co-simulation
Accelerator.v
System.v
Parameterized ISS
System Integrate
Area Timing Power evaluation
Evaluation
Integration
Platform model
No
Performance constraint
Yes
Pin assignment Hardware compilation
FPGA prototyping
Hardware image
FPGA Verify
14
Demo Video
15
Encoder Demo -- Platform
16
Encoder Demo Video Capturing
17
Encoder Demo --.264 file output to SDCARD
18
Encoder Demo -- Decode with reference software
19
Encoder Demo -- Display the result
20
NTHU-PKU Collaboration

• Porting NTHU H.264/AVC Encoder and Decoder to PKU
  SOC Platform

21
Results
22
Participants
23
(No Transcript)
24
Job Function
25
Schedule
26
Schedule
27
System Overview
28
FPGA Prototyping
29
Results
30
Future Systems
31
High-Performance QFHD
32
SuperHDTV (3840x2160 QFHD)
3840x2160 - SuperHDTV
33
Summary

• H.264/AVC Encoder Decoder IP
• Two Platforms UniChip and PKU
• Very High Performance (Cycles per MB)
• Future Directions
• Super High Resolution (4X HDTV, 16X HDTV)
• Low Power Design
• Enhanced Platform and Design Flow
• Chip Implementation

34
Block-Based Hybrid Video Coding

• Divide Frame into Blocks of Equal Size
• Code One Block after Another
• Apply Multiple Kinds of Coding Methods
  Sequentially on a Block

We may use macroblock and block interchangeably
35
Need More to Make it Works
Prediction
Reconstruct
Reference Image
36
Encoding Path (Inter-Frame Predicted)
Q
DCT

-
ME
MC
Compressed Video Stream
P
Intra prediction

Filter
IQ
IDCT

37
Encoding Path (Intra-Frame Predicted)
Q
DCT

-
ME
MC
Compressed Video Stream
P
Intra prediction

Filter
IQ
IDCT

38
Decoding Path
Q
DCT

-
ME
Compressed Video Stream
MC
P
Intra prediction

Filter
IQ
IDCT