PRE AND POST PROCESSING

Table of Contents:

1.
2.
3.
4.
5.
6.
7.
8.

Introduction
Logo erasing
Logo inserting / KLV metadata overlaying
Gaussian blur
Deinterlacing and interlacing
Frame cropping
Frame scaling
Adding noise

1. Introduction
This document explains various pre and post processing filters and ways of using them with
FFMPEG and VLC.
VLC player is a free and open source cross-platform multimedia player and framework that
plays most multimedia files as well as DVDs, Audio CDs, VCDs, and various streaming
protocols and it is licensed under GNU GPL license. It is written by VideoLAN project, which
was originally started as academic project and it was developed by students at Ecole Centale
Paris, while now, it has been developed by contributors worldwide. More information about
VLC can be found in Error: Reference source not found.
FFmpeg is a complete, cross-platform solution to record, encode/decode, convert, mux/demux,
filter, play and stream audio and video. More information about FFMPEG can be found in Error:
Reference source not found.

2. Logo erasing
Logo erasing or logo removal filters have a purpose in eliminating static logo or watermark
from each video frame. There are various ways to implement this effect however we are
focusing on implementations that exist in VLC and libavfiter library from FFMPEG.
VLC has erase module which works as erase video filter. It applies mask over specific part of a
frame and fills it with a value of surrounding pixels. Size of the affected area is size of the PNG
picture which is provided as a mask and pixels with alpha channel > 127 are marked as pixels

that should be erased. Additionally we can specify X and Y shift of picture in pixels from top left
corner of the frame.
Similar to VLC's erase module we have removelogo filter in libavfilter. Slight difference is that
each non zero pixel is considered part of the logo and picture has to be same size as the video
frame.
When we need to clear rectangular logo or watermark, delogo filter from libavfilter has same
effect as removelogo filter but simpler usage. We just specify x,y coordinates of top left corner
of the rectangle being erased and width and height of that rectangle. Additionally we can
specify thickness of the fuzzy edge added to the rectangle and whether to show that rectangle
to simplify finding the right one.
Thoughts about implementing this filter
Simple usage from user perspective is essential. Looking from GUI client side having just in
time mask editor that takes current video frame and allows marking of pixels that need erasing
would be most appropriate so removelogo filter from libavfilter seems like most suitable one.
However, command line usage without having a picture as a mask makes delogo more
appropriate. All this leads to conclusion that libavfilter covers broader range of use cases.

3. Logo inserting / KLV metadata overlaying

Logo inserting and generally overlaying present new artifacts on top of existing frame. It can be
text, image, image with opacity and another video. Both VLC and FFMPEG support this.
VLC has logo module which inserts one or more images. We can set transparency and display
duration for each image as well as offset from top left corner. With marq filter it can display
custom text on top of a video.
However, libavfilter with much more powerful overlay module seems like more effective
solution for broader range of use cases.
Both VLC and FFMPEG have support for displaying subtitles and both use libass for displaying
styled subtitles.

4. Gaussian blur
Gaussian blur is the result of applying Gaussian function on a video frame. The effect of this
filter is a smoother image resembling a view of the frame through translucent screen. It is
commonly used to lower image noise which in turn is used as a preceding step to edge
detection. More about Gaussian blur can be found in Error: Reference source not found.
VLC has gaussianblur module. It applies blur effect on whole video and only accepts standard
deviation value as parameter.
Unlike VLC, FFMPEG has access to many different Gaussian blur implementations. Most

notable one is inside libopencv (more about in Error: Reference source not found), a powerful
image processing library which has smooth filter that has Gaussian blur as one of the possible
methods.
Beside libopencv there are also smartblur, boxblur and unsharp filters which are part of
libavfilter and support features like time-line editing.
Thoughts about implementing this filter
My recommendation is to use smartblur at first. Requirements refinement will and should show
need for different filter implementation.

5. Deinterlacing and interlacing

Deinterlacing is the process of converting source material that contains alternating half-pictures
(fields) to a computer screen that displays full picture at a time (frame). This always results in some
image degradation. We will try to present overview of useful deinterlace modes in both VLC and
FFMPEG.
Blending or doubling only algorithms aren't considered as they produce more image degradation.
Some sort of interpolation always adds benefit to the output.
Notable modes of deinterlace module in VLC are:
linear doubling with linear interpolation of input lines but instead of displaying each line twice, line 2
is created as the average of line 1 and 3 etc.
x interpolator, generates a full picture by taking the odd lines from the odd half-picture and creating
the even lines using motion estimation and motion compensation algorithms that uses information
from both half-pictures
yadif Yet Another DeInterlacing Filter from MPlayer project which takes odd lines from the odd halfpicture and creating even lines through algorithm that includes both temporal and spatial
interpolation
yadif2x similar to linear but uses yadif interpolation algorithm
However, comparing results from all of the above yadif mode produces lowest image degradation
and is almost best time wise.
FFMPEG on the other hand has access to a broader range of deinterlacing algorithms. Notable
ones beside the ones present in VLC as well are
w3fdif based on Martin Weston work for BBC. Subjectively it has slight less image degradation but
introduces slickering line at the bottom (can be cut off) and is around 20% slower than yadif.
Interlacing is opposite process which creates half-pictures from full frames and it doesn't incur image
degradation. No direct support for interlacing exists in VLC but it is possible to do it through one or
more available codecs (x264, xvid). FFMPEG has interlace filter which does the job perfectly.
Conclusion
Having yadif support is should be first step in deinterlacing support. Of course in future, more
deinterlacing filters can be added.

6. Frame cropping
Cropping is a simple image transformation in which we remove outer parts of the image. Both VLC
and FFMPEG support frame cropping.
VLC's croppadd module has a simple usage, you just specify number of pixels to be cropped from
which side of the frame. Additionally you can add custom number of black pixels back to desired
side.
FFMPEG has much more powerful crop module but at the expense of more complex usage. It
accepts expressions as parameters which can have time or input frame number, input and output
width and height, x and y as top left corner position. However with simple wrapper around it can be
user accessible as simple as croppadd module.
Conclusion
Being much more powerful, FFMPEG's crop is preferred choice. If simple use case scenario is
required it is easy to add wrapper around it.

7. Frame scaling
Scaling is a process of resizing a digital image. It is not a simple transformation and always involves
a trade-off between efficiency, smoothness and sharpness. It is usually a lossy transformation as we
add (upsampling) or remove (downsampling) parts of the image.
Both VLC and FFMPEG use same libswscale library for image scaling with various algorithms.

8. Adding noise
Removing plastic look from animations or removing too much noise and than introducing controlled
amount of it can make image look more real-like. Both VLC and FFMPEG have noise module that
can accomplish this effect.