H.

264 Analyzer
Utility Tool
By
Mukta Gore
Shantanu Kurhekar
Amruta Vaidya
Gouri Patil

1
 Project Statement
n Decoding the H.264 input bit stream step wise, restricting its purview
to the Baseline profile.
n Extracting NAL units from the bit stream, and displaying list of frames
in the video sequence along with list of parameter sets encoded in
the stream.
n Decoding and displaying the syntax elements in the parameter sets.
n Displaying user selected frame.
n Extracting selected slice in selected frame, and decoding and
displaying parameters in the slice header.
n Decoding and displaying selected macroblock information.

2
 Analyzers
n A software tool to help technical professionals parse bit streams
conforming to international media standards
n Video standards specify only a syntactic description of compressed
visual data and method for decoding it to reconstruct video
sequence
n Since encoding methods are not specified, analyzers are needed to
test for conformance to the standards
n Analyzers perform stepwise decoding of compressed bit streams
and display results at every step to check for non-conforming
parameters
n They also help in testing the working and efficiency of algorithms for
motion estimation, rate control and scene change detection

3
A peek into our utility …

4
H.264 ENCODER

5
H.264 DECODER

6
Implementation of our Analyzer
User Interface

API

Library

User Interface : GUI in Visual Basic

API : Application Program Interface
Library : Dynamic Link Library in C
7
 User Interface
n This acts as an interface between the user and the core functionality
of the analyzer
n It enables user-controlled picture-by-picture analysis of .264 stream
n The user can analyze the bit stream at various levels in the
hierarchy-sequence, frames, slices, macroblocks and macroblock
partitions
n It displays the parameter sets, reconstructed sample values, errors
in the bit stream, if any and the reconstructed picture

8
 Option to Hide grid Reconstructed
frame

Parameter list
Slice list

Frame list

Macroblock list

MB info choice

Detailed MB info
based on choice 9
selected
 Library
n We have implemented the core decoding functions as a Dynamic
Link Library because

¨ The analyzer uses decoding functions written in C which is roughly a

14000 line code
¨ DLLs link the object code to the executable at run time rather than
compile time as done in static libraries
¨ So less memory is required to run the analyzer utility, and the
executable file is compact
¨ Multiple applications can share the same DLL
¨ Multiple entry points are provided in DLLs to enable conditional
decoding

10
 Program Flow
n The functionality of the utility tool is event driven so depending on the
action of the user, various tasks are performed.
n Processing is done at two levels: GUI level and DLL level
n Following events actuate entry into the DLL –
¨ Opening a file by clicking on the submenu Open in the menu File
¨ Clicking on a frame in the frame list
¨ Clicking on a slice in the slice list
¨ Closing a file by clicking on the submenu Close in the menu File
n Certain events like
¨ clicking on SPS or PPS in the parameter list
¨ clicking on a macroblock in the MB list
¨ selecting MB info choice
are handled in the GUI domain without calling the DLL again
11
 Event 1: Clicking on submenu Open of menu File

GUI DLL

Open File

PRIME Prime

Display No. of
Extract
NAL units, NALunit
parameter set list
and frame list
Parse

12
Extract NAL unit Parse

13
 NAL unit structure

NAL unit
Start
code
NAL Header Raw Byte Sequence Payload

0x000001

Video Coding Layer (VCL) data

Bit 0 Bits 1-2 Bits 3-7
Forbidden bit NAL ref idc NAL type
Value:0 Value:0-3 Value:0-31

14
 Event 2: Clicking on an SPS from list Event 3: Clicking on a PPS from list

Parameter set mechanism enables an encoder to signal important, infrequently changing

sequence and picture parameters separately from the coded slices

Sequence Parameter Set is applicable to a complete video sequence and contains profile
type, the number of reference frames, decoded picture width and height, choice of
progressive or interlaced coding etc

Picture Parameter Set is applicable to one or more pictures within a sequence and
contains selected SPS id, flag to select VLC or CABAC entropy coding, number of slice
groups in use, number of reference pictures in list 0 used for prediction, initial quantization
parameters etc
15
Event 4: Clicking on a frame from the list

16
 Frame decoding

n The following process will be iterated till all slices in the selected
frame are decoded
n I or IDR frame:
¨ It contains only intra coded macroblocks and hence is decoded
independently of previous pictures
¨ The starting point is first slice of the frame
n P frame:
¨ It contains inter and intra coded macroblocks and hence its reference
pictures need to be decoded
¨ The starting point is first slice of the most recent IDR
¨ To improve efficiency,
if (recent IDR < last decoded frame < frame to be decoded)
then starting point is first slice of last decoded frame
else
starting point is first slice of recent IDR
17
 Slice decoding

Slice Header Slice data

n Slice header
¨ Slice type
¨ Picture Parameter Set referenced
¨ Frame number to which it belongs with respect to recent IDR
¨ Delta quantization parameter at slice level
¨ No. of active reference pictures in list 0
n Slice data
¨ Macroblock layer

18
 Macroblock layer

n MB type (Inter/Intra/Skipped)
n Macroblock partitions for prediction
¨ Intra – 4x4, 16x16, PCM
n 9 prediction modes for 4x4
n 4 prediction modes for 16x16
¨ Inter – allows sub-block partitioning up to 4x4
n Separate motion vector for each partition or sub-partition
n Quarter pel motion vector accuracy
n Separate reference picture for each macroblock partition allowed
n Delta quantization parameter at macroblock level
n Coded block pattern to indicate which of the blocks contain non-zero
coefficients

19
Event 5: Clicking on a slice from the list

20
 Event 6: Clicking on an MB from the list

The information extracted during frame decode is accessed in GUI domain for
display, without calling DLL

21
Event 7: Clicking on submenu Close of menu File

22
Error handling
n The utility checks if all parameters are within their specified range to
test conformance with H.264 baseline profile
n Bit stream errors are handled at 4 levels
¨ General - Unsupported file format, No SPS, No PPS, No IDR
¨ SPS,PPS
¨ Slice header
¨ Slice data
n On return from DLL, the error array is checked and corresponding
errors are displayed with the relevant frame number

23
 Challenges
n Datatype matching of structures in DLL and GUI domain
n DLL convention for parameter passing
n Accessing memory allocated in DLL from GUI
n Finding out reference frame for every macroblock partition
n Memory allocation and deallocation problems

24
 Utility Features
n User-controlled picture by picture analysis
n Supports CIF (352 x 288), QCIF (176 x 144), SQCIF (128 x 96) video formats
n Stream level statistics collection
n Only Baseline profile supported
n Conformance checks (Display of non conformant parameters)
n Parameter set extraction facility (PPS, SPS)
n Frame Level Analysis
¨ Frame type
¨ Number of slices in the frame
¨ Distribution of Macroblocks in the frame based on type
n Slice Level Analysis
¨ Slice Header
¨ Number and address of Macroblocks in the slice
n Macroblock Level Analysis
¨ Macroblock type, subtype and prediction mode
¨ Quantization parameters for luma and chroma
¨ MB grid overlay on picture with Hide Grid option
¨ Motion vector display - MB and sub-MB partitioning

25
Applications of analyzer
n Getting insight into the coding quality of H.264 / MPEG-4 part 10
video-encoding software and hardware
n Enabling rectification of the functions responsible for encoding non-
conformant parameters in the encoder
n Analysis of coded bit stream for no. of I and P frames, macroblock
and sub-block partition for rate control
n Monitoring and validating large volumes of digital media down to
inspecting and debugging any problems encountered at the
bitstream level

26
 Files tested
File name Brief Analysis
&
Compression

Akiyo.264 Low motion activity video of 150 frames (10 I and 140 P), 1 slice per frame,
1 : 168 QCIF size
Silent.264 Medium motion activity video of 150 frames (1 I and 149 P), 1 slice per frame,
1 : 118 QCIF size
Md_33.264 Low motion activity video of 60 frames (1 I and 59 P), 3 slices per frame, QCIF
1 : 208 size
News.264 Low motion activity video of 97 frames (1 I and 96 P), 1 slice per frame, CIF
1 : 158 size
Carphone.264 Medium motion activity video of 90 frames (1 I and 89 P), 1 slice per frame,
1 : 90 QCIF size
Football.264 High motion activity video of 90 frames (1 I and 89 P), 1 slice per frame, CIF
1 : 31 size
Stefan.264 High motion activity video of 90 frames, 1 slice per frame, CIF size
1 : 27

27
 Non - conformant bit streams

FILENAME ERROR DISPLAYED

noStartCode.264 Unsupported file format
foreman.264 Profile not supported
md.264 Weighted prediction and weighted bi-prediction flag not
zero
container_wipe.264 Illegal luma ac coefficients (Frame 3)
test.264 No IDR, no SPS, no PPS in bit stream

28
 Future Scope
n Adding support for Main and Extended profile
n Displaying reference and current picture simultaneously for selected
macroblock
n Visual display for motion vectors using arrows
n Slice group map type
n Displaying the macroblocks in the order they are decoded if arbitrary
slice order is allowed
n Support for SIF file
n Support for simultaneous audio and video analysis

29
30