HIGH DYNAMIC RANGE IMAGING

A
PROJECT REPORT
By

Pratyush Pandab1
Under The Guidance Of

Prof. B. Majhi
Professor and Head of Dept. Computer Science and Engineering, NIT Rourkela

Department of Computer Science Engineering

NATIONAL INSTITUTE OF TECHNOLOGY ROURKELA
ROURKELA-769008 (ORISSA)

1
Pratyush Pandab is a 4 th year B. Tech student from Dept. of Computer Science and Engineering at CET, BBSR..

1
 National Institute of Technology Rourkela

Certificate

This is to certify that the work in this Project entitled “High Dynamic Range
Imaging” by Pratyush Pandab has been carried out under my supervision during
the Summer Internship 2009 at NIT RKL in the Department of Computer
Science and Engineering.

Place: Rourkela Prof. Banshidhar Majhi

Date: July 15, 2009 Professor and Head of Dept. of CSE
NIT, Rourkela.

2
 Declaration by the Candidate

I hereby declare that the project report entitled “High Dynamic Range Imaging”
is a record of bonafide project work carried out by me under the guidance of
Prof. B. Majhi of NIT- Rourkela, Orissa.

Pratyush Pandab
Dept. of CSE, CET
Bhubaneswar.

3
 Table of Contents

Abstract ......................................................................................... 5
1. Introduction to HDRI ........................................................ 6
1.1. A Brief History of HDRI 6
1.2. Traditional Techniques used 7
1.3. Recent Technology 7
1.4. Tone Mapping 8

2. Problem Statement .............................................................. 9

3. Sample Example .................................................................. 10

4. Background Research .......................................................... 11

4.1. Recovering High Dynamic Range Radiance Map from 11-12
Photographs
4.1.1. The Algorithm 12
4.1.1.1. Film Response Recovery 13-16
4.1.1.2. Construction of High Dynamic Range Radiance 17
Map.
4.1.2. Implementation 17-18
4.2. A Simple Spatial Tone Mapping Operator for High 19
Dynamic Range Images.
4.2.1. The Need for Tone Mapping. 19
4.2.2. The Algorithm 19-20
4.2.2.1. Cases of Interest 20-21
4.2.3. Implementation 21-22

5. Implementation Results ...................................................... 23-26

6. Application of HDRI .......................................................... 27

7. Conclusions .......................................................................... 28

8. References ............................................................................ 28.

4
 Abstract

In image processing, computer graphics, and photography, High Dynamic Range Imaging
(HDRI) is a set of techniques that allows a greater dynamic range of luminance between light
and dark areas of a scene than normal digital imaging techniques. The intention of HDRI is to
accurately represent the wide range of intensity levels found in real scenes ranging from
direct sunlight to shadows.

Information stored in high dynamic range images usually corresponds to the physical values
of luminance or radiance that can be observed in the real world. This is different from
traditional digital images, which represent colors that should appear on a monitor or a
paper print.

One problem with HDR has always been in viewing the images. Typical computer monitors
(CRTs, LCDs), prints, and other methods of displaying images only have a limited dynamic
range. Various methods of converting HDR images into a viewable format have been
developed, generally called Tone Mapping.

Recent methods have attempted to compress the dynamic range into one reproducible by
the intended display device. The more complex methods tap into research on how the
human eye and visual cortex perceive a scene, trying to show the whole dynamic range
while retaining realistic color and contrast.

Figure 1: Here the dynamic range of the image is demonstrated by adjusting the "exposure" when tone -
mapping the HDR image into an LDR one for display.

In this project, it is considered that nearly any camera can actually capture a vast dynamic
range-- just not in a single photo. By varying the shutter speed alone, most digital cameras
can change how much light they let in by a factor of 50,000 or more. This project attempts
to combine these captured images into a single High Dynamic Range Image by utilizing the
different exposures characterized by these images composed of multiple exposures. The
difficulties that accompany HDRI are Halo effects and artifacts. Care has been taken to
avoid these pitfalls. The complete project was simulated using Matlab 7.6 software. The
simulation results are shown as figures. Results have shown very high quality rendering of
images by this technique.

5
 1. Introduction to HDRI

High Dynamic Range Imaging (HDRI) is a set of techniques that allows a greater dynamic
range of luminance between light and dark areas of a scene than normal digital imaging
techniques. High dynamic range (HDR) images enable photographers to record a greater
range of tonal detail than a given camera could capture in a single photo. This opens up a
whole new set of lighting possibilities which one might have previously avoided. The
intention of HDRI is to accurately represent the wide range of intensity levels found in real
scenes ranging from direct sunlight to shadows.

1.1. A Brief History of HDRI

In the past, photographers dealt with the limitation of dynamic range by experimenting
with media. Ansel Adams was perhaps the first to systematically measure the sensitivity
range of all of the equipment he used to precisely indicate what the photograph would
display depending on the length of time the shutter was open. Before HDR was put into
practice, photographers and film makers needed to find ways around the limitations of
modern digital imaging which fail to capture the total perceptual display that the human eye
is capable of seeing. The problem is that real-world scenes contain light ranges that exceed
a 50,000:1 dynamic range2 while digital equipment is limited to around 300:1 dynamic
range.

Also, through Adams' work it was discovered that one way to get really great dynamic
range with colour photography is to use black-and-white film and colour filters. Although
this was a practical solution, as technology advanced so did the digital imaging market, thus
creating HDR.

Images tend to result in the subject seen and the surrounding blanked out.

Figure 2: In the first image the shelf, the table and the sofa set are the subject, whereas the window is the
surrounding. Not much is seen outside the window, but the details on shelf and the table are seen pretty
clearly. In the second im age the surrounding is clear enough but nothing is visible inside the room. This is
the problem caused due to the dynamic r ange of the capturing equipment.

2
Dynam ic Range: The dynami c range is the ra tio between the ma ximum and mi nimum values of a physi cal
measurement. Its definition depends on wha t the dynami c range refers to.

6
 1. 2 Traditional Techniques Used

Most photographic systems struggle on high contrasts scenes. With the entire modern
technology one would be wrong to think a digital camera would resolve the issues related to
dynamic range. The reason for this is simple, current film and image sensors are developed
to record enough light that can be reproduced on paper or compute r screen and neither are
currently capable of displaying a dynamic range that we can see with our eyes. Until
recently, to overcome this, digital photographers have either been using the old approach of
ND graduated filters at the taking stage or combining the best bits of two different
exposures on the computer, much like the dodge & burn 3 technique used by experienced
darkroom users.

1. 3 Recent Technology

New technology is on the way to record and display at higher dynamic ranges, but for now
there are a range of software solutions.

The job of HDR processing is to allow photographers to produce images with a much larger
dynamic range. The photographer takes a series of different exposures of the same scene
and then uses HDR software to automatically merge the resulting images into one image,
referred to as exposure blending. This produces a file with the entire desired dynamic range,
known as a radiance map. Ideally the fewer images used the better, because it's less
memory intensive and there's less chance of subject been out of alignment on each shot as
a result of camera or subject movement.

HDR uses 16 to 32 bits values. This jump in resolution allows for a much greater range of
colours, greater than the monitor can display at once. As a result, to view these images, the
computer breaks the images into levels of brightness and darkness. The brightest part of the
image are viewed with the brightest colours the monitor can display the darkest are viewed
with the darkest the screen can display. This allows for each image to display details, bright
highlights and dark shadows.

3
Dodging and Burning are terms used in photography for a technique used during the printing process , to
manipulate the exposure of a selected area(s) on a photographic print, deviating from the rest of the image's
exposure. Dodging decreases the exposure for areas of the print that the photographer wishes to be lighter,
while burning increases the exposure to areas of the print that should be darker.

7
 1. 4 Tone Mapping

The above mentioned merged photographs are saved as 32-bit. This means there's very
little one can do to edit the photos, they cannot be saved as JPG without reducing the bit-
depth 4 and one cannot view the superb dynamic range on a conventional monitor. In such
cases, the whole dynamic range of the scene (e.g. 100,000:1 or higher) has to be reproduced
on a medium with less than 100:1 contrast ratio. Tone mapping deals with the issue of
reproducing the dynamic range captured. This is why it becomes necessary to use
techniques that scale the dynamic range down while preserving the appearance of the
original image captured. One way around this is to apply tone mapping which converts the
HDR image into a balanced exposure.

Tone mapping operators are divided into two broad categories, Global and Local
Global operators are simple and fast. They map each pixel based on its intensity and global
image characteristics, regardless of the pixel's spatial location. Global operators are OK for
mapping 12-bit sensor data but usually don't work well with HDR images. An example of a
global type of tone mapping is a tonal curve.

Local operators take into account the pixel's surroundings for mapping it. This means that a
pixel of a given intensity will be mapped to a different value depending on whether it is
located in a dark or bright area. This makes local operators slower but tends to produce
more pleasing results, given that our eyes react locally to contrast.

4
Bit depth of a ca pturing or displa ying devi ce gi ves an indi ca tion of i ts dynami c range capa city, i .e. the hi ghes t dyna mi c
range tha t the devi ce would be capable of reproducing.

8
 2. Problem Statement

Here is the problem in a nutshell:

a. Real-world scenes contain light ranges that exceed a 50,000:1 dynamic range.
b. Media has been limited to around a 300:1 dynamic range.
c. So we have a mapping issue: how do we represent light values in a scene using a
much more limited set of light values for a particular media?
d. What if the set of images with different exposures have unknown noise in it? Can we
get the HDR image sans the noise (noise gets added, so it has to be removed.).

9
 3. Sample Example

Figure 3 : Sample input set of im ages with different exposure time.

Figure 4: Output; tone mapped HDR im age. (Note the details present in this single image.)

10
 4. Background Research

The method of recovering high dynamic range radiance maps from photographs taken with
conventional imaging equipments [𝐼] is presented in this project work. The proposed
algorithm by Debevec & Malik [𝐼] uses these differently exposed photographs to recover
the response function of the imaging process, up to factor of scale, using the assumption of
reciprocity. With the known response function, the algorithm can fuse the multiple
photographs into a single, high dynamic range radiance map whose pixel values are
proportional to the true radiance values in the scene.

Besides this, the project also implements a simple and effective tone mapping operator
proposed by Biswas & Pattanaik [𝐼𝐼], that preserves visibility and contrast impression of
high dynamic range images. It use a s-function type operator which takes into account both
the global average of the image, as well as local luminance in the immediate neighbourhood
of each pixel. The local luminance is computed using a median filter. It is seen that the
resulting low dynamic range image preserves fine details, and avoids common artefacts such
as halos, gradient reversals or loss of local contrast.

4.1. Recovering High Dynamic Range Radiance Maps from

Photographs [𝐼]
Digitized photographs are becoming increasingly important in computer graphics. When
one photographs a scene, either with film or an electronic imaging array, and digitizes the
photograph to obtain a two dimensional array of brightness 5 values, these values are rarely
true measurements of relative radiance in the scene. For example, if one pixel has twice the
value of another, it is unlikely that it observed twice the radiance 6. Instead, there is usually
an unknown, nonlinear mapping that determines how radiance in the scene becomes pixel
values in the image. This nonlinear mapping is hard to know beforehand because it is
actually the composition of several nonlinear mappings that occur in the photographic
process.

In a conventional camera (see Fig.5), the film is first exposed to light to form a latent image.
The film is then developed to change this latent image into variations in transparency, or
density, on the film. The film can then be digitized using a film scanner, which projects light
through the film onto an electronic light-sensitive array, converting the image to electrical
voltages. These voltages are digitized, and then manipulated before finally being written to
the storage medium. If prints of the film are scanned rather than the film itself, then the
printing process can also introduce nonlinear mappings.

5
Brightness values represent the gra y levels of the ima ge.
6
Radiance is the i rradiance/l uminance value of the object a t the si te of the s cene.

11
 Lens Shutter Film Development CCD ADC Remapping

𝜠 𝑿 𝜡

Scene Sensor Sensor exposure Latent image Film density Analog Digital Final
radiance irradiance voltage values digital
Digital camera values

Figure 5 : Image Acquisition Pipeline: shows how scene radiance becomes pixel values for both film and
digital cameras. Unknown nonline ar mappings can occur during exposure, de velopment, scanning,
digitization, and remapping.

In the first stage of the process, the film response to variations in exposure 𝑿7 is a non-
linear function, called the “characteristic curve” of the film. The development, scanning and
digitization processes usually introduce their own nonlinearities which compose to give the
aggregate nonlinear relationship between the image pixel exposures 𝑿 and their values 𝒁8.

As with film, the most the most significant nonlinearity in the response curve is at its
saturation point, where any pixel with radiance above a certain level is mapped to the same
maximum image value. The most obvious difficulty is that of limited dynamic range—one
has to choose the range of radiance values that are of interest and determine the exposure
time suitably. Sunlit scenes, and scenes with shiny materials and artificial light sources,
often have extreme differences in radiance values that are impossible to capture without
either under-exposing or saturating the film. To cover the full dynamic range in such a
scene, one can take a series of photographs with different exposures. This then poses a
problem: how can we combine these separate images into a composite radiance map? Here
the fact that the mapping from scene radiance to pixel values is unknown and nonlinearity
begins to haunt us.

A
simple technique for recovering this response function, up to a scale factor, using
nothing more than a set of photographs taken with varying, known exposure durations has
been the proposed work by Debevec & Malik [𝐼] and is simulated in this project. With this
mapping, the pixel values from all the available photographs are used to construct an
accurate map of the radiance in the scene, up to a factor of scale. This radiance map will
cover the entire dynamic range captured by the original photographs.

4.1.1. The Algorithm

This section presents the algorithm cited in [𝐼], for recovering the film response function,
and then presents the method of reconstructing the high dynamic range radiance image
from the multiple photographs. The algorithm assumes gray scale images, but can be
extended to colour images with proper conversions 9.

7
𝑿 = 𝑬𝜟𝒕 ; i.e. 𝑿 is the product of the i rra diance 𝑬 the film recei ves and the exposure time 𝜟𝒕.
8
𝒁 is the final di gi tal value after processing. Refer fig. 5.
9
We ca n convert RGB images to YCbCr i mages and consider onl y the Y (luminance) component or else we can consider
ea ch component indi vidually and plot the mapping.

12
 4.1.1.1. Film Response Recovery

This algorithm is based on exploiting a physical property of imaging systems, both

photochemical and electronic, known as reciprocity.

Let us consider photographic film first. The response of a film to variations in exposure is
summarized by the characteristic curve (or Hurter-Driffield curve) [𝐼]. This is a graph of the
optical density 𝑫 of the processed film against the logarithm of the exposure 𝑿 to which it
has been subjected. The exposure 𝑿 is defined as the product of the irradiance 𝑬 at the film
and exposure time, 𝜟𝒕, so that its units are 𝑱𝒎−2 . Key to the very concept of the
characteristic curve is the assumption that only the product 𝑬𝜟𝒕 is important, and that
halving 𝑬 and doubling 𝜟𝒕 will not change the resulting optical density 𝑫. Under extreme
conditions (very large or very low 𝜟𝒕 ), the reciprocity assumption can break down, a
1
situation described as reciprocity failure. In typical print films, reciprocity holds to within 3
1
stop10 for exposure times of 10 seconds to 10000 of a second.

After the development, scanning and digitization processes, we obtain a digital number 𝒁,
which is a nonlinear function of the original exposure 𝑿 at the pixel. Let us call this
function 𝒇, which is the composition of the characteristic curve of the film as well as all the
nonlinearities introduced by the later processing steps. The first goal is to recover this
function 𝒇. Once we have that, we can compute the exposure 𝑿 at each pixel, as
𝑿 = 𝒇−𝟏 (𝒁). A reasonable assumption made is that the function 𝒇 is monotonically
increasing [𝐼], so its inverse 𝒇−𝟏 is well defined. Knowing the exposure 𝑿 and
the exposure time 𝜟𝒕, the irradiance 𝑬 is recovered as 𝑬 = 𝑿/𝜟𝒕, which will be taken to be
proportional to the radiance 𝑳 in the scene.

The input to the algorithm is a number of digitized photographs taken from the same
vantage point with different known exposure durations 𝜟𝒕𝒋 .11 It is assumed that the scene is
static and that this process is completed quickly enough that lighting changes can be safely
ignored. It can then be assumed that the film irradiance values 𝑬 𝒊 for each pixel 𝒊 are
constant. Pixel values are denoted by 𝒁𝒊𝒋 where 𝒊 is a spatial index over pixels and 𝒋 indexes
over exposure times 𝜟𝒕𝒋 . Thus the film reciprocity equation is:

𝒁𝒊𝒋 = 𝒇(𝑬𝒊 𝜟𝒕𝒋 ) (𝟏)

Since it is assumed 𝒇 is monotonic, it is invertible, and so we can rewrite (𝟏) as:
𝒇−𝟏 (𝒁𝒊𝒋 ) = 𝑬𝒊 𝜟𝒕𝒋 (𝟐)
Taking the natural logarithm of both sides of (𝟐) , we have:
𝐥𝐧 𝒇−𝟏 (𝒁𝒊𝒋 ) = 𝐥𝐧 𝑬𝒊 + 𝐥𝐧 𝜟𝒕𝒋 (𝟑)
1
10 1
1 stop is a photographi c term for a fa ctor of two ; 3 s top is thus 23 .
11 1 1 1 1 1 1
The a ctual exposure ti mes a re va ried by powers of two between s tops (64 , 32 , 16 , 8 , 4 , 2, 1, 2, 4, 8, 16, 32), ra ther
1 1 1 1 1 1
than the rounded numbers displa yed on the camera readout (60 , 30 , 15 , 8 , 4 , 2, 1, 2, 4, 8, 15, 30).

13
To simplify notation, let us define a function, 𝒈 = 𝐥𝐧 𝒇−𝟏 . We then have the set of
equations:
𝒈(𝒁𝒊𝒋 ) = 𝐥𝐧 𝑬𝒊 + 𝐥𝐧 𝜟𝒕𝒋 (𝟒)
where 𝒊 ranges over pixels and 𝒋 ranges over exposure durations. In this set of equations,
the 𝒁𝒊𝒋 are known, along with their corresponding 𝜟𝒕𝒋 . The unknowns are the irradiances 𝑬 𝒊
as well as the function 𝒈, although it is assumed that 𝒈 is smooth and monotonic.

Our aim now is to recover the function 𝒈 and the irradiances 𝑬 𝒊 that best satisfy the set of
equations arising from Equation 𝟒 in a least-squared error sense. It can be verified that
recovering 𝒈 only requires recovering the finite number of values that 𝒈(𝒛) can take since
the domain of 𝒁, pixel brightness values, is finite. Let
𝒁𝒎𝒊𝒏 and 𝒁𝒎𝒂𝒙 be the least and greatest pixel values (integers),
𝑵 be the number of pixel locations, and
𝑷 be the number of photographs.

Thus the existing problem can be formulated as one of finding the (𝒁𝒎𝒊𝒏 − 𝒁𝒎𝒂𝒙 + 𝟏)
values of 𝒈(𝒁) and the 𝑵 values of 𝐥𝐧 𝑬 𝒊 that minimize the following quadratic objective
function:
𝑵 𝑷 𝒛=𝒁𝒎𝒂𝒙 −𝟏
𝟐
𝓞= 𝒈 𝒁𝒊𝒋 − 𝐥𝐧 𝑬𝒊 − 𝐥𝐧 𝜟𝒕𝒋 +𝝀 𝒈′′ 𝒛 𝟐

𝒊=𝟏 𝒋=𝟏 𝒛=𝒁𝒎𝒊𝒏 +𝟏

(𝟓)

The first term ensures that the solution satisfies the set of equationsarising from Equation
(𝟒) in a least squares sense. The second term is a smoothness term on the sum of squared
values of the second derivative of 𝒈 to ensure that the function 𝒈 is smooth. In this discrete
setting 𝒈′′ 𝒛 = 𝒈 𝒛 − 𝟏 − 𝟐𝒈 𝒛 + 𝒈(𝒛 + 𝟏) is used. This smoothness term is essential
to the formulation in that it provides coupling between the values 𝒈(𝒛) in the minimization.
The scalar 𝝀 weights the smoothness term relative to the data fitting term, and should be
chosen appropriately for the amount of noise expected in the 𝒁𝒊𝒋 measurements.

𝒈(𝒛)’s, minimizing 𝓞 is a straightforward linear least

Because it is quadratic in the 𝑬𝒊’s and
squares problem. The over-determined system of linear equations is robustly solved using
the singular value decomposition (SVD) method.

Along with this equation some addition points need to be considered.

First, the solution for the 𝒈(𝒛) and 𝑬𝒊 values can only be up to a single factor 𝜶 ;[𝐼]. If each
log irradiance value 𝐥𝐧 𝑬 𝒊 were replaced by 𝐥𝐧 𝑬 𝒊 + 𝜶, and the function 𝒈 replaced by
𝒈 + 𝜶, the system of equation (4) and also the objective function 𝓞 would remain
unchanged. To establish a scale factor we [𝐼] introduced the additional constraint
𝟏
𝒁𝒎𝒊𝒅 = 𝟎, where 𝒁𝒎𝒊𝒅 = 𝟐 (𝒁𝒎𝒊𝒏 + 𝒁𝒎𝒂𝒙 ), simply by adding this as an equation in the

14
linear system. The meaning of this constraint is that a pixel with value midway between
𝒁𝒎𝒊𝒅 and 𝒁𝒎𝒊𝒅 will be assumed to have unit exposure.
Second, the solution can be made to have a much better fit by anticipating the basic shape
of the response function. Since 𝒈(𝒛) will typically have a steep slope near 𝒁𝒎𝒊𝒏 𝒂𝒏𝒅 𝒁𝒎𝒂𝒙
we should expect that 𝒈(𝒛) will be less smooth and will fit the data more poorly near the
extremes;[𝐼]. To recognise this, we need to introduce a weight function 𝒘(𝒛) to emphasize
the smoothness and fitting terms towards the middle of the curve. A sensible choice of 𝒘 is
a simple hat function:
𝟏
𝒛 − 𝒁𝒎𝒊𝒏 𝑓𝑜𝑟 𝒛 ≤ (𝒁 + 𝒁𝒎𝒂𝒙 )
𝒘 𝒛 = 𝟐 𝒎𝒊𝒏
𝟏 (𝟔)
𝒁𝒎𝒂𝒙 − 𝒛 𝑓𝑜𝑟 𝒛 > (𝒁𝒎𝒊𝒏 + 𝒁𝒎𝒂𝒙 )
𝟐

Thus equation (𝟓) now becomes:

𝑵 𝑷 𝒛=𝒁𝒎𝒂𝒙 −𝟏
𝟐
𝓞= 𝒘(𝒁𝒊𝒋) 𝒈 𝒁𝒊𝒋 − 𝐥𝐧 𝑬𝒊 − 𝐥𝐧 𝜟𝒕𝒋 +𝝀 𝒘 𝒛 𝒈′′ (𝒛) 𝟐

𝒊=𝟏 𝒋=𝟏 𝒛=𝒁𝒎𝒊𝒏 +𝟏

Finally, we need not use every available pixel site in this solution procedure. Given
measurements of 𝑵 pixels in 𝑷 photographs, we have to solve for 𝑵 values of 𝐥𝐧 𝑬 𝒊 and
(𝒁𝒎𝒂𝒙 − 𝒁𝒎𝒊𝒏 ) samples of 𝒈. To ensure a sufficiently over-determined system, we want
𝑵(𝑷 − 𝟏) > (𝒁𝒎𝒂𝒙 − 𝒁𝒎𝒊𝒏 ). For the pixel value range (𝒁𝒎𝒂𝒙 − 𝒁𝒎𝒊𝒏 ) = 𝟐𝟓𝟓, 𝑷 = 𝟏𝟏
photographs, a choice of 𝑵 on the order of 50 pixels is more than adequate. Since the size of
the system of linear equations arising from Equation (𝟓) is on the order of
𝑵 × 𝑷 + (𝒁𝒎𝒂𝒙 − 𝒁𝒎𝒊𝒏 ), computational complexity considerations make it impractical to
use every pixel location in this algorithm. The pixels are best sampled from regions of the
image with low intensity variance so that radiance can be assumed to be constant across the
area of the pixel, and the effect of optical blur of the imaging system is minimized.

A Matlab routine is given to show the simplicity of the evaluation of the equation (𝟒).

We can represent this idea in a graphical manner as below:

15
Figure 6: In the figure on the 𝑡𝑜𝑝 𝑙𝑒𝑓𝑡, the × symbols represent samples of the 𝒈 curve derived from the digital values
at one pixel for 5 different known exposures using Equation(𝟒). The + and ο symbols show samples of 𝒈 curve
segments derived by consideration of two other pixels.
(𝑏𝑜𝑡𝑡𝑜𝑚 𝑓𝑖𝑔𝑢𝑟𝑒)The unknown log irradiance 𝒍𝒏𝑬𝒊 has been arbitrarily assumed to be 0. It’s worth noting that the shape
of the 𝒈 curve is correct, though its position on the vertical scale is arbitrary corresponding to the unk nown 𝒍𝒏𝑬𝒊.; again
the vertical position of each segment is arbitrary. Essentially, what we want to achieve in the optimization process is to
slide the 3 sampled curve segments up and down (by adjusting their 𝒍𝒏𝑬𝒊’s) until they “line up” into a single smooth,
monotonic curve, as shown in the 𝑏𝑜𝑡𝑡𝑜𝑚 𝑓𝑖𝑔𝑢𝑟𝑒. The vertical position of the composite curve will remain arbitrary.

16
 4.1.1.2. Construction of High Dynamic Range Radiance Map

Once the response curve 𝒈 is recovered, it can be used to quickly convert pixel values to
relative radiance values, assuming the exposure 𝜟𝒕𝒋 is known.

From equation (𝟒), we obtain:

𝐥𝐧 𝑬𝒊 = 𝒈 𝒁𝒊𝒋 − 𝐥𝐧 𝜟𝒕𝒋
(𝟕)
For robustness, and to recover high dynamic range radiance values, we should use all the
available exposures for a particular pixel to compute its radiance. For this, we reuse the
weighting function in Equation (𝟔) to give higher weight to exposures in which the pixel’s
value is closer to the middle of the response function:

𝑷
𝒋=𝟏 𝒘 𝒁𝒊𝒋 (𝒈 𝒁𝒊𝒋 − 𝐥𝐧 𝜟𝒕𝒋 )
𝐥𝐧 𝑬𝒊 = 𝑷
(𝟖)
𝒋=𝟏 𝒘 𝒁𝒊𝒋
Combining the multiple exposures has the effect of reducing noise in the recovered
radiance values. It also reduces the effects of imaging artefacts such as film grain. Since the
weighting function ignores saturated pixel values, blooming 12 artefacts have little impact on
the reconstructed radiance values.

4.1.2. Implementation

The heart of the algorithm is the optimization routine that needs to be implemented
effectively. The implementation is given in [𝐼]. The Matlab routine is give below:

12
Blooming occurs when charge or light a t highl y satura ted si tes on the imaging surface spills over a nd affects values at
nei ghboring si tes.

17
18
 4.2. A Simple Spatial Tone Mapping Operator for High
Dynamic Range Images [II]

After obtaining the HDR scene values, the next task was to tone map these values onto a
LDR matrix for being able to see it on the screen. This project presents a simple and
effective tone mapping operator, which preserves visibility and contrast impression of high
dynamic range images. The method is conceptually simple, and easy to use. It uses a
𝑠 − 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛 type operator which takes into account both the global average of the image,
as well as local luminance in the immediate neighbourhood of each pixel. The local
luminance is computed using a median filter. It is seen that the resulting low dynamic range
image preserves fine details, and avoids common artefacts such as halos, gradient reversals
or loss of local contrast.

4.2.1. The Need for Tone Mapping

The real world scenes often have a very high range of luminance values. While digital
imaging technology now enables us to capture full dynamic range of the real world scene,
still we are limited by the low dynamic range displays. Thus the scene can be visualized on a
display monitor only after the captured high dynamic range is compressed to available
range of the display device. This has been referred to as the tone mapping problem in
the literature and a great deal of work has been done in this area by using a mapping that
varies spatially depending on the neighbourhood of a pixel, often at multiple scales
[𝐼𝐼𝐼];[𝐼𝑉]; 𝑉 . In this paper[𝐼𝐼] the authors have proposed a simple tone mapping operator
which allows us to preserve the visual content of the real-world scene without the user
having to manually set a number of parameters. We show that by using a log of relative
luminance at a pixel with respect to its local luminance in a small neighbourhood, the
standard 𝑠 − 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛 can be modified to yield visually pleasing results. This project
computes the local luminance using a median filter, which provides a stronger central
indicator than the mean filter.

4.2.2. The Algorithm

The global contrast helps us to differentiate between various regions of the HDR image,
which can be loosely classified as dark, dim, lighted,, bright etc. Within each region objects
become distinguishable due to local contrast against the background − either the object is
darker than the background or it is brighter than the background.

If the HDR image consisted of only regions of uniform illuminations, the following
𝑠 − 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛 would compress the range of illumination across the image, to displayable
luminance 𝒀𝑫 in range 0 − 1.

19
 𝒀𝑫(𝒙, 𝒚) = 𝒀(𝒙, 𝒚)/[𝒀(𝒙, 𝒚) + 𝑮𝑪] (𝟗)
where 𝑮𝑪 is the global contrast factor computed through

𝑮𝑪 = 𝒄 𝒀𝑨 (𝟏𝟎)
where 𝒀𝑨 is the average luminance value of the whole image and 𝒄 is a multiplying factor.
This would have the effect of bringing the high luminance value closer to 1, while the low
luminance values would be a fraction closer to zero. The choice of factor 𝒄 has an effect on
bridging the gap between the two ends of the luminance value.

While equation (𝟗) is able to provide a global contrast reduction, it does not attempt to
preserve local luminance variations within a region, causing many details to be lost.

A detail in the image can result from two possible reasons− either the pixel is brighter than
its surrounding pixels or the pixel is darker than its surrounding pixels. If luminance 𝒀 at a
pixel is more than 𝒀𝑳, the local luminance in its immediate neighbourhood, we would like to
increase the displayable value 𝒀𝑫 at that point, so that it appears brighter against its
neighbourhood. On the other hand, if the luminance 𝒀 at a point is less than the local
luminance 𝒀𝑳 in its immediate neighborhood, we would like to reduce the displayable value
𝒀𝑫, so that it appears darker than its neighbourhood. This has been achieve by modifying
the 𝑠 − 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛 as follows:

𝒀𝑫(𝒙, 𝒚) = 𝒀(𝒙, 𝒚)/[𝒀(𝒙, 𝒚) + 𝑪𝑳(𝒙, 𝒚)] (𝟏𝟏)

where 𝑪𝑳 is the contrast luminance at a point (𝑥, 𝑦) obtained by modifying the global
contrast factor 𝑮𝑪 with a term involving the logarithm of ratio of local low-pass filtered
value 𝒀𝑳 to original luminance value,

𝑪𝑳 = 𝒀𝑳[𝒍𝒐𝒈(𝜹 + 𝒀𝑳/𝒀)] + 𝑮𝑪 (𝟏𝟐)

where 𝜹 is a very small value and is included to avoid singularity while doing the 𝑙𝑜𝑔
operation.

Also 𝒀𝑳 is computed using a median filter in a [3 × 3] neighbourhood rather than a mean

filter. The median is a stronger central indicator than the average.

4.2.2.1. Cases of Interest

𝑪𝒂𝒔𝒆 𝟏:
Pixel (𝑥, 𝑦) belongs to a uniform region, i.e. the Luminance 𝒀 and the local luminance 𝒀𝑳
have same magnitude. In this case the log value is going to be close to zero and thus the
contribution of the first term is negligible. 𝑪𝑳 will be very close to 𝑮𝑪, and the new 𝒀𝑫
value using equation (𝟏𝟏), is almost same as computed by equation (𝟗).

20
𝑪𝒂𝒔𝒆 𝟐:
Pixel (𝑥, 𝑦) is darker than its immediate surrounding neighbourhood, i.e. the magnitude of
pixel luminance 𝒀 is less than the local luminance 𝒀𝑳. In this case, the first term in equation
(𝟏𝟐) is positive. The contribution from 𝒀𝑳 is moderated by a factor which depends on how
relatively high 𝒀𝑳 is with respect to 𝒀 itself at that pixel. If the difference is large, the first
term in equation (𝟏𝟐), is significantly large. In any case a positive term adds on to 𝑮𝑪, the
global contrast value, resulting in an overall reduction in 𝒀𝑫 using equation (𝟏𝟏).
(𝑐𝑜𝑚𝑝𝑎𝑟𝑒𝑑 𝑡𝑜 𝑪𝒂𝒔𝒆 𝟏).

𝑪𝒂𝒔𝒆 𝟑:
Pixel (𝑥, 𝑦) is brighter than its immediate neighborhood, i.e. the magnitude of 𝒀 is more
than the local luminance 𝒀𝑳. Since the denominator is higher in 𝒍𝒐𝒈(𝒀𝑳/𝒀), the first term is
going to provide a negative contribution, reducing the value of 𝑪𝑳 and consequently
resulting in a higher value of 𝒀𝑫 (𝑐𝑜𝑚𝑝𝑎𝑟𝑒𝑑 𝑡𝑜 𝑪𝒂𝒔𝒆 𝟏).

For most HDR images, the computed YD can be displayed directly.

4.2.3. Implementation

The luminance values are obtained from the RGB inputs with the formula:
𝒀 = 𝟎. 𝟐𝟗𝟗 × 𝑹 + 𝟎. 𝟓𝟖𝟕 × 𝑮 + 𝟎. 𝟏𝟏𝟒 × 𝑩
After computing the YD, the displayable luminance, the new R, G, B values are computed
using the formula [𝑉𝐼]:

𝑹𝑫 = (𝑹/𝒀)𝜸 × 𝒀𝑫;
𝑮𝑫 = (𝑮/𝒀)𝜸 × 𝒀𝑫;
𝑩𝑫 = (𝑩/𝒀)𝜸 × 𝒀𝑫;
where 𝜸controls the display colour on the monitor. 𝜸 = 𝟎. 𝟒 is used in this implementation.

The implemented Matlab code is given below.

21
22
 5. 1. Implementation Results

Figure 7: The input of the HDRI program. 𝑰𝒏𝒑𝒖𝒕 𝒊𝒎𝒂𝒈𝒆 −> 𝑐𝑕𝑎𝑖𝑟. Note the variations in the details of each image. The
exposure times are in decreasing order. i.e. the darkest image has the least exposure time while the brightest has the
maximum.

Figure 8: The HDR radiance map. 𝒙 − 𝒂𝒙𝒊𝒔 Represents “𝒍𝒐𝒈 𝑬𝒙𝒑𝒐𝒔𝒖𝒓𝒆 (𝑿)” while 𝒚 − 𝒂𝒙𝒊𝒔 represents
”𝒑𝒊𝒙𝒆𝒍 𝒗𝒂𝒍𝒖𝒆 𝒁”. The three curves represent the Red, Green, and Blue components of the images.

23
Figure 9: The HDR image obtained from the radiance map.

Figure 10: The final tone mapped HDR image (LDR).

24
 5. 2. Implementation Results

Figure 11: The input of the HDRI program. 𝑰𝒏𝒑𝒖𝒕 𝒊𝒎𝒂𝒈𝒆 −> 𝑠𝑢𝑛𝑠𝑒𝑡. It is obvious from the images that the first image
is overexposed (high exposure time) while the second one is underexposed (less exposure time).

Figure 12: The HDR radiance map. 𝒙 − 𝒂𝒙𝒊𝒔 Represents “𝒍𝒐𝒈 𝑬𝒙𝒑𝒐𝒔𝒖𝒓𝒆 (𝑿)” while 𝒚 − 𝒂𝒙𝒊𝒔 represents
”𝒑𝒊𝒙𝒆𝒍 𝒗𝒂𝒍𝒖𝒆 𝒁”. The three curves represent the Red, Green, and Blue components of the images.

25
Figure 13: (𝒍𝒆𝒇𝒕 𝒊𝒎𝒂𝒈𝒆) HDR image obtained from the radiance map. (𝒓𝒊𝒈𝒉𝒕 𝒊𝒎𝒂𝒈𝒆) The final tone mapped HDR
image(LDR).

26
 5. Applications of HDRI

Today, the main users of HDR imaging devices are specialized professionals working in the
film, animation and VR industries. Some applications are listed below.

Film - Tools such as HDRShop enables one to convert a series of photographs into a light
probe - a special image that represents the lighting environment in a room. One can then
use the light probe to light virtual objects, so that the virtual objects actually appear to be lit
by the light from the room. This technique is especially useful for compositing computer
graphic objects into images of real scenes. Hollywood films use light maps extensively to
blend CGI into a scene.

Panoramas - Another use for HDR is in panoramic images. Panoramas often have a wide
dynamic range, e.g. one part of the panorama may contain the sun, and another part may
be in deep shadow. Online web panoramas constructed from HDR images look much better
than non-HDR equivalents.

Games - A third use for HDR is in computer games. Recent computer graphics cards support
HDR texture maps. With HDR texture maps, you can render objects using light probes, in
real time, yielding much more dynamic and interesting lighting effects. "High Dynamic
Range Lighting Effects" are used in many new high-end games.

By determining the response functions of the imaging device, the method presented here
allows one to correctly fuse pixel data from photographs taken at different exposure
settings. As a result, one can properly photograph outdoor areas with short exposures, and
indoor areas with longer exposures, without creating inconsistencies in the data set.
Furthermore, knowing the response functions can be helpful in merging photographs taken
with different imaging systems, such as video cameras, digital cameras, and film cameras
with various film stocks and digitization processes.

Most image processing operations, such as blurring, edge detection, colour correction, and
image correspondence, expect pixel values to be proportional to the scene radiance.
Because of nonlinear image response, especially at the point of saturation, these operations
can produce incorrect results for conventional images. In computer graphics, one common
image processing operation is the application of synthetic motion blur to images . It can be
shown that by using true radiance maps one can produce significantly more realistic motion
blur effects for high dynamic range scenes.

27
 6. Conclusion

In this
project a simple implementation of High Dynamic Range Imaging along with tone
mapping has been shown. The results achieved are pleasing to the eye and the images are
comparable to any of the standard HDRI toolbox available in the market.

7. References

[𝐼] Debevec, P. E., & Malik, J. (n.d.). Recovering High Dynamic Range Radiance Maps from
Photographs.

[𝐼𝐼] Biswas, K. K., & Pattanaik, S. (n.d.). A Simple Spatial Tone Mapping Operator for High Dynamic.

[𝐼𝐼𝐼] Pattanaik, S. N., Ferwerda, J. A., Fairchild, M. D., & Greenberg, D. P. (1998). A multiscale model
of adaptation and spatial vision for realistic images display. SIGGRAPH,Annual Conference
Series,ACM , 267-276.

[𝐼𝑉] Fattal, R., Lischinski, D., & Werman, M. (2002). Gradient Domain High Dynamic range
compression. SIGGRAPH,Annual Conference series,ACM , 246-249.

[𝑉] Reinhard, E., Stark, M., Shirley, P., & Ferwerda, J. (2002). Photographic Tone reproduction for
digital images. SIGGRAPH,Annual Conf. Series,ACM , 267-276.

[𝑉𝐼] Duan, J., & Qiu, G. (2004). Fast Tone Mapping for high dynamic range images. International
Conference on Pattern Recognition, ICPR , 847-850.

28