Image Processing and Computer Vision: 9 1

Image and Video Coding

Compressing data to a smaller volume without
losing (too much) information
Image Processing and Computer Vision: 9 3
Why Code Data?
To reduce storage volume
To reduce transmission time
One colour image
760 by 580 pixels
3 channels, each 8 bits
1.3 Mbyte
Video data
Same resolution
25 frames per second
33 Mbyte/second
Image Processing and Computer Vision: 9 4
Redundancy
Spatial
Correlation between adjacent pixels
Chromatic
Correlation between colour channels
Temporal
Correlation between adjacent frames
Perceptual
Unnoticed losses
Image Processing and Computer Vision: 9 5
Example
Image Processing and Computer Vision: 9 6
Redundancy - Consequences
Data exceeds information
More data than content justifies
Can lose data without losing
information
Image Processing and Computer Vision: 9 7
Compression Ratio
image compressed of volume
image ed uncompress of volume
C
Image Processing and Computer Vision: 9 8
Lossy vs Lossless Compression
Can lose data without losing
information
Lossily compressed images can look
similar to the original
Lossy compression has greater C
Image Processing and Computer Vision: 9 9
Quality of Decoded Images
Measure differences between
Original
Coded/decoded images
Options
Maximum difference
Average difference
Subjective scales
Image Processing and Computer Vision: 9 10
Example
Image Processing and Computer Vision: 9 11
Subjective Quality Measurement
Scales - Example
0 Unusable
1 Annoying degradation
2 Adequate images
3 Barely perceptible degradation
4 No observable degradation
Image Processing and Computer Vision: 9 12
Difference coding
Adjacent pixels are similar
Difference is small

Uncompressed:


Compressed:
, , , , ,
4 3 2 1 0
s s s s s
, , , , ,
3
4
2
3
2 1
0
1
0
s
s
s
s
s s
s
s
s


Image Processing and Computer Vision: 9 13
Coding
Assign 4 bits to a difference
Can code 7, , +8

Overflow?
Use 7 and +8 to show larger differences
Code 6, , +7 directly
Use overflow codes to indicate shift of codes
Image Processing and Computer Vision: 9 14
Image Processing and Computer Vision: 9 15
Predictive Coding
If signals are well sampled
Adjacent samples are correlated
They have similar values
Differences are small
Can guess next sample from value of
current
Image Processing and Computer Vision: 9 16


Constants are correlation coefficient and mean
grey value
Difference between real and predicted
values are smaller
Code as for difference coding

I I
t t

1
Image Processing and Computer Vision: 9 17
Run Length Coding
Replace runs of equal brightness values by
(length of run, value)

1 2 2 3 3 4 4 4 5 6 5
(1 1) (2 2) (2 3) (3 4) (1 5) (1 6) (1 5)

More use when few brightness values
e.g. fax

Image Processing and Computer Vision: 9 18
Huffman Coding
Uses variable length codes
Most frequently occurring grey value
has shortest code
Least frequently occurring values have
longest codes
Image Processing and Computer Vision: 9 19
Example
Symbol Probability 2 1
4
3
A
B
C
D
E
F
0.4
0.3
0.1
0.1
0.06
0.04
0.4
0.3
0.1
0.1
0.1
0.4
0.3
0.2
0.1
0.4
0.3
0.3
0.6
0.4
Image Processing and Computer Vision: 9 20
Symbol Probability 2 1
4
3
A
B
C
D
E
F
0.4 1
0.3 00
0.1 011
0.1 0100
0.06 01010
0.04 01011
0.4 1
0.3 00
0.1 011
0.1 0100
0.1 0101
0.4 1
0.3 00
0.2 010
0.1 011
0.4 1
0.3 00
0.3 01
0.6 0
0.4 1
Image Processing and Computer Vision: 9 21
GIF
Applicable to images with 256 colours
Replace sequences of bytes with
shorter codes
Most common sequences use shortest
codes
Image Processing and Computer Vision: 9 22
Wavelet coders
Wavelet transform organises image
content efficiently
Can easily select data to be discarded
Image Processing and Computer Vision: 9 23
Wavelet coders
Image Processing and Computer Vision: 9 24
JPEG standard
A subcommittee of ISO
Optimised for a range of image subject
matter
Compression rates can be defined
Quality inversely proportional to C
Image Processing and Computer Vision: 9 25
Block Transform Coding
original image
decompose
transform
quantise
entropy code
Sequence of 8 by 8 blocks - different planes treated
separately (RGB, YUV etc.)
Transformed blocks reduce redundancy and
concentrate signal energy into a few coefficients
discrete cosine transformation (DCT)
Blocks with discarded information - goal is to smooth
picture and discard information that will not be
missed, e.g. high frequencies
Image Processing and Computer Vision: 9 26
Block Transform Encoding
DCT
Zig-zag
run
length
code
entropy
code
quantise
010111000111..
Image Processing and Computer Vision: 9 27
Block Encoding
139 144 149 153
144 151 153 156
150 155 160 163
159 161 162 160
1260 -1 -12 -5
-23 -17 -6 -3
-11 -9 -2 2
-7 -2 0 1
DCT
Zig-zag
run
length
code
Huffman
code
quantise
10011011100011.
79 0 -1 0
-2 -1 0 0
-1 -1 0 0
0 0 0 0
79 0 -2 -1 -1 -1 0 0 -1 0 0 0 0 0 0 0
0 79
1 -2
0 -1
0 -1
0 -1
2 -1
0 0
Original image
Image Processing and Computer Vision: 9 28
Block Transform Decoding
DCT
Zig-zag
run
length
code
entropy
code
quantise
010111000111..
Image Processing and Computer Vision: 9 29
Result of Coding and Decoding
139 144 149 153
144 151 153 156
150 155 160 163
159 161 162 160
Original block
144 146 149 152
148 150 152 154
155 156 157 158
160 161 161 162
Reconstructed block
-5 -2 0 1
-4 1 1 2
-5 -1 3 5
-1 0 1 -2
errors
Image Processing and Computer Vision: 9 30
Discrete Cosine Transform
F[u,v] =
4C(u)C(v)
n
2

n-1
S
j=0
n-1
S
k=0
f(j,k) cos
(2j+1) u p
2n
cos
(2k+1) v p
2n
with
C(w) =
1
2
For w = 0
1 otherwise
Image Processing and Computer Vision: 9 31
Discussion
Where are lossy steps?
Quatisation and subsampling before coding
How is quantisation matrix chosen?
Its predefined by the standard after much
experimentation
Image Processing and Computer Vision: 9 32
Video coding
Many specific standards
AVS, MOV, QT,
One universal standard
MPEG
Image Processing and Computer Vision: 9 33
MPEG Standards
Standard specifies audio, video and
system layers
MPEG-1: low data rates, poor quality:
VHS quality at 1.5Mbits
-1
MPEG-2: high quality hence high data
rates: studio quality, 15Mbits
-1

MPEG-4: low data rates, small images,
64 kbits
-1

Image Processing and Computer Vision: 9 34
MPEG-1
Audio and video designed to work at CD
ROM speeds: 1.5Mbits
-1

Video 1.150Mbits
-1

Audio 0.256Mbits
-1

System 0.094Mbits
-1

Image Processing and Computer Vision: 9 35
MPEG-2
Released in 1994
Aimed at digital TV, ATM.
Additions for
Interlaced video
Scalable video coding
Graceful degradation with noise
Implementation of full standard impractical
Varying profiles/levels of conformity
Image Processing and Computer Vision: 9 36
MPEG-4
Coding specifically for multimedia
objects
Image Processing and Computer Vision: 9 37
Coding Algorithms
Frame sequence
Motion compensation
Frame coding
Image Processing and Computer Vision: 9 38
Frame Sequence
I frames (Intraframes)
Coded independently of any other frame
P frames (Predicted frames)
Derived from previous I frame by motion
prediction
B frames (Bidirectionally interpolated)
Interpolate motion compensated blocks
between I and P frames
Image Processing and Computer Vision: 9 39
Motion Compensation
Image is divided into macroblocks (16 x
16 pixels)
Matching macroblocks are found by
minimising differences
Code differences and macroblock
displacement
Image Processing and Computer Vision: 9 40
Frame Coding
Use JPEG algorithms
Image Processing and Computer Vision: 9 41
Summary
Why code data?
Redundancy
Assessment of compression
Lossy vs. lossless compression
Algorithms
JPEG, MPEG
Image Processing and Computer Vision: 9 42
But what is it good for?
Engineer at the Advanced Computing
Systems Division of IBM, commenting on
the microchip in 1968