CSCI 3290 Computational Photography

Assignment 1 Image Filtering and Hybrid Images

Due Date: 11:59pm on Monday, October 10th, 2016

I. Objectives
a) Understand image filtering
b) Understand image representation in frequency domain
c) Learn basics of Python image processing using OpenCV

II. Background

The goal of this assignment is to write an image filtering function and use it to
create hybrid images [1] using a simplified version of the SIGGRAPH 2006 paper [2] by
Oliva, Torralba, and Schyns. Hybrid images are static images that change in interpretation
as a function of the viewing distance.
The basic idea is that high frequency tends to dominate perception when it is available, but,
at a distance, only the low frequency (smooth) part of the signal can be seen. By blending
the high frequency portion of one image with the low-frequency portion of another, you
get a hybrid image that leads to different interpretations at different distances.

III. Details
a) Image Filtering
Image filtering is a fundamental image processing tool. Images filters are meant
to remove unwanted components of images, such as noise, textures or certain
bands in frequency domain (high-pass/low-pass/band-pass). Here we focus on
the simplest convolution filters.

OpenCV has many built-ins to perform image filtering, but you need to write your own
for this assignment. More specifically, you will implement my_imfilter().

In the above example, filtering process is to perform convolution between input image
(68 pixels) and kernel image (33 pixels). The output is the convolution result. As
specified in my_imfilter.m, your filtering algorithm must do the following.
(1) Support both grayscale and color images.

(2) Support arbitrary shaped filters (convolution kernels), as long as both

dimensions are odd (e.g. 79 pixels but not 44 pixels)
(3) Pad the input image with zeros or reflected image content (e.g., zero-padding,
replicated, symmetric).

Input

Pad Zero

Kernel

Pad Replicated

Pad Symmetric

(4) Return a filtered image which is the same resolution (pixels) as the input image.

b) Hybrid Images
A hybrid image is the sum of a low-pass filtered version of the one image and a highpass filtered version of a second image. There is a free parameter, which can be tuned
for each image pair, which controls how much high frequency to remove from the first
image and how much low frequency to leave in the second image. This is called the
"cutoff-frequency". In the paper it is suggested to use two cutoff frequencies (tuned
for each image). In the skeleton code, the cutoff frequency is controlled by changing
the standard deviation of the Gaussian filter used in constructing the hybrid images.

 Low-pass filter: convolve input image with Gaussian kernel.

High-pass filter: input minus low-pass filtered input.
The low-pass (blurred) dog and high-passed cat look like this:

The high frequency image is actually zero-mean with negative values so it is visualized
by adding 0.5 (assuming images in range 0~1). In the resulting visualization, bright
values are positive and dark values are negative.
Adding the high and low frequencies together gives you the image at the top of this page.
If you have trouble seeing the multiple interpretations of the image, a method is by
progressively downsampling the hybrid image as is done below.

The starter code provides a function vis_hybrid_image() to save and display

such visualizations.
We provide you with 5 pairs of aligned images which can be merged reasonably well
into hybrid images. The alignment is important because it affects perceptual grouping
(read the paper for details). We encourage you to create additional examples (e.g.
change of expression, morph between different objects, change over time, etc.). See
the hybrid images project page [1] for some inspiration. The project page also
contains materials from their SIGGRAPH presentation.

c) TODOs
i.

TODO 1: Implement image filtering function, considering padding type.

ii.

TODO 2: Perform low-pass filter for the first image.

iii.

TODO 3: Perform high-pass filter for the second image

iv.

TODO 4: Generate hybrid image

d) Useful Hints
We provide: hybrid.py
You CANNOT use:
cv2.filter2D()

numpy.pad(),

numpy.convolve()

IV. Submission
a) Upload package
i.

Create a folder with name:

<your student ID>-Asgn1 (e.g. 1155011111-Asgn1\)

and

ii.

Readme file containing anything about the project that you want to tell the
TAs, including brief introduction of the usage of the codes
<your student ID>-Asgn1\README.txt

iii.

Place all your code in subfolder code\

<your student ID>-Asgn1\code\

iv.

You must report in HTML. In the report, describe your algorithm and any
decisions you made to write your algorithms in a particular way. Show and
discuss your results in your report. Discuss algorithms efficiency and
highlight all extra credits you did.
Place all your html report in subfolder html\. The home page should be
index.html
<your student ID>-Asgn1\html\index.html

v.

Compress the folder into <your student ID>-Asgn1.zip, and upload to e-learn
system.

b) Hand-in via CUHK e-learn system

i.

Go to http://elearning.cuhk.edu.hk/

ii.

Go to: 2016R1-CSCI3290 : Computational Photography

iii.

Go to: Assignments tab

iv.

Click Assignment 1Browse My Computer Select your package

Click Submit.

V. Scoring & Extra Bonus

The maximum score for this assignment is 100.

a) Rubric
i.

20%: Working implementation of convolution filter.

ii.

5%: Working implementation for both gray and color images.

iii.

5%: Filtered output is the same size input.

iv.

30%: Working implementation for 3 padding methods (10% for each).

v.

5%: Working implementation of low-pass filter.

vi.

5%: Working implementation of high-pass filter.

vii.

10%: Working implementation of generating hybrid images.

viii.

20%: html report with discussions and at least 3 pairs of results.

b) Extra Credit
i.

10%: use Fast Fourier Transform to accelerate convolution filters and show
comparisons.

VI. Other Remarks

a)

20% off from each extra late day.

b)

The three assignments are to be completed INDIVIDUALLY on your own.

c)

You are encouraged to use methods in related academic papers.

d)

Absolutely NO sharing or copying code! NO sharing or copying reports! Offender will be

given a failure grade and the case will be reported to the faculty.

VII. Reference
[1] Hybrid image gallery: http://cvcl.mit.edu/hybrid_gallery/gallery.html
[2] Hybrid images:
http://cvcl.mit.edu/publications/OlivaTorralb_Hybrid_Siggraph06.pdf