Chapter 1 : INTRODUCTION AND OVERVIEW

Image stabilization (IS) is a family of techniques that reduce blurring associated

with the motion of a camera or other imaging device during exposure. It is used
in image-stabilized binoculars, still and video cameras, astronomical telescopes,
and also smartphones, mainly the high-end.

1. The stabilization effect can be achieved through hardware or software.

2. The concept behind image stabilization is that the image or lens is shifted in
opposite direction to the motion of camera shake, based on the technique
used.

There are two popular image stabilization techniques,

1. Electronic Image Stabilization (or EIS) and

2. Optical Image Stabilization (OIS)

Electronic image stabilization (EIS) is an image enhancement technique using

electronic processing. EIS minimizes blurring and compensates for device
shake, often a camera. More technically, this technique is referred to as pan and
slant, which is the angular movement corresponding to pitch and yaw.

Optical image stabilization, often abbreviated OIS, IS, or OS, is a mechanism

used in a still camera or video camera that stabilizes the recorded image by
varying the optical path to the sensor. This technology is implemented in the
lens itself, as distinct from in-body image stabilization, which operates by
moving the sensor as the final element in the optical path. The key element of all
optical stabilization systems is that they stabilize the image projected on the
sensor before the sensor converts the image into digital information.

[1]
1.1 WHAT IS OPTICAL IMAGE STABILIZATION?

OIS stabilizes the recorded image by varying the optical path to the sensor. It is
completely Hardware based.

What OIS does is compensate for small movements of the camera during
exposure. In general terms this is achieved by the use of a floating lens,
gyroscopes and small motors. The elements are controlled by a microcontroller
which moves the lens very slightly to counteract the shaking of the camera or
phone, i.e. if the phones of is being moved to the right, then the lens is moved
left.

The basic OIS system uses two gyros in the lens (one for yaw and one for pitch)
that detect both the angle and speed of the movement. This data is fed, in real-
time, to the microcontroller which then moves things around inside the lens to
compensate for the movement.

Figure 1.1.1 : OIS

[2]
1.2 WHAT IS ELECTRONIC IMAGE STABILIZATION?

It is also called Digital Image Stabilization. The concept is same as OIS but it is
completely Software based.

EIS reduces camera shake and controls stability by manipulating the image
electronically and is mainly developed for videos.

Using software within the digital camera or camcorder, this technique corrects
the electronic image from the image sensor (frame to frame) to counteract any
unwanted motion.

It does this by using pixels outside the border of the visible frame to provide a
buffer for the motion.

If the camera position is slightly shifted the frame inside the camera is
electronically shifted within this buffer area to the corrected position and keep it
centered.

The more centered it is the less apparent jitter or shake you see.

Contrary to OIS, EIS solves the issue of blur and shakiness at the programming
level.

1.3 WORKING MECHANISM OF EIS

It makes use of the light-sensing chip, the Charge Coupled Device (CCD), of the
camera. Once the image hits the chip, and if the system detects what it thinks is
camera shake through its sensors, it responds by slightly moving the image so
that it remains in the same place on the CCD.

[3]
For example, if the camera shakes to the left, then the image moves to the right
to compensate, thus eliminating the shake.

Basically, when the image zigs, the EIS system zags, by precisely the same
amount.

All of this happens after the picture has been captured or after the conversion of
the optical signal into a digital signal, which happens within a fraction of
seconds.

During the process, the EIS technology zooms in to the image slightly to
decrease image movement. The resulting inflated image makes it easier
for the system to detect shake-induced movement. However, such
increase in image size can decrease the resolution of the picture, which is
probably why most cameras (smartphone or conventional cameras) with
electronic image stabilization have a higher resolution.

There are two ways the EIS system works to reduce the movement of the image:

1. Increase the size of the image by digitally “zooming in on the image

2. Using an oversized CCD.

1) Increase the size of the image by digitally “zooming in on the image

The first method is to increase the size of the image by digitally “zooming
in on the image so that it is larger than the CCD. By making the image
larger, the system can then scan within the image to counter the movement
created by the shake.

[4]
 When the zoomed in image hits the CCD chip, there is an extra space
around the chip which the system uses as a buffer of pixels to shift the
image.

2) Using an Oversized CCD

The second method of the EIS system uses of an oversized CCD. As a video
image covers only about 90 percent of the chip’s area, there is system space
available for which to move the image. For example, when the image is
stable, the chip centers the image on the CCD. But if the camera shakes to
the right, there is space for the image to roam back to the left to compensate
for the shake, keeping it in exactly the same place on the CCD, thus
eliminating the shake.

Figure 1.3.1: EIS using image zooming

[5]
 Figure 1.3.2 : Video Stabilization using EIS

GC-BPM ALGORITHM
(Gray-coded bit plane matching algorithm)
A EIS system usually includes five major units:
1. preprocessing unit
2. motion estimation unit
3. motion decision unit
4. motion compensation unit for FM
5. digital zooming unit.

The FM or Frame Memory is used to store current image data and to output the
stabilized image data.

[6]
 Figure 1.3.3 : Block diagram of EIS system

The algorithm begins by pre-processing the image data to produce gray-coded

bit-planes from a standard binary representation.

We adopt the gray-coded bit-plane strategy to do motion estimation. Under an

acceptable increase of computation complexity, however, we divide an image
frame into finer blocks to do localized block matching. This finer division
enables a much more accurate estimation of the local movement inside the
captured frames.

Moreover, the increased number of local motion vectors (LMVs) facilitates the
employment of a more complex procedure for motion decision.

Furthermore, to distinguish the movement caused by camera shaking from the

movement caused by moving objects or intentional panning, the temporal
correlation of each local motion vector is also carefully investigated in our EIS
system.

[7]
In motion compensation, an affine model is evaluated to compensate
translational and rotatory movements.

f(x,y) is a K-bit gray-level image represented as:

f(x,y) = aK-1(x,y)2^(K-1) + aK-2(x,y)2^(K-2) +……………………..+ a1(x,y)2

+ a0(x,y) ...(1)

Then, the gray-coded bit-planes are defined as:

gi(x,y)=ai(x,y) XOR ai+1(x,y) , 0<=i<=(K-2) …(2)

and

gK-1(x,y) = aK-1(x,y) …(3)

where ai are the standard binary coefficients.

Figure 1.3.4 : Bit planes of 8-bit gray level image

The gray coding is important in this algorithm because it allows the next block
to estimate motion using a single bit-plane by encoding most of the useful image
information into a few of the planes. With the gray code, small changes in gray

[8]
level yield a small, uniform change in the binary digits representing the
intensity. For this reason, the next GC-BPM block can estimate local motion
using an extremely efficient Boolean binary operator to compare the gray-coded
bit-plane of the previous image to the current one.

The involved correlation measure is given by

where W is the sliding block used to search and p is the number of pixels to
search over

Figure 1.3.5 : Motion Estimation

The resulting values of (m,n) produce a local motion vector for each of the four
subimages. Then, the four local motion vectors along with the previous global
motion vector are subject to a median operator to produce the current global
motion vector estimate. Then, the global motion estimate is passed through a
filter that is tuned to let intentional camera motion be preserved while removing
the undesirable high frequency motion.

The final filtered motion estimate is then compensated for by shifting the current
frame by an integer number of pixels in the opposite direction of the motion.
[9]
 Chapter 2 : BACKGROUND

Camera and smartphone buyers nowadays are bombarded with terms such as
OIS, VR, steady shot and much more when it comes to the camera USPs. But
what is image stabilization and should you really be concerned about buying a
device with this tech?

The Beginnings

The first commercial camera – the daguerreotype camera – was invented by

Louis-Jacques-Mandé Daguerre and designed by Alphonse Giroux in 1839.
Back then, cameras were big and bulky and required photographers to work at a
slow pace while the camera sits comfortably on a tripod, offering a perfect shot.
It was only after the invention of the 35mm film that cameras started to become
smaller and more compact like the Leica 1 back released in 1925. With cameras
becoming more compact, photography picked up among users who would travel
with cameras to capture moments as and when they fancied.

Taking pictures was a luxury owing to the high cost of film and photograph
development. But the biggest problem was the blurred shot that resulted due to
motion or shakes while capturing images. This was more prevalent in smaller
cameras.

In 1994, Nikon produced the first ever optically stabilized lens that would
compensate for minor motion or jerks to get a more steady shot. That marked
the beginning of image stabilization and we have come a long way from there.

[10]
 Chapter 3 : LITERATURE REVIEWS

Real-Time Digital Image Stabilization

In this paper, digital image stabilization (DIS) algorithms are investigated with
respect to implementation feasibility and complexity. One of the most
computationally efficient algorithms was simulated to produce qualitative results
to characterize performance.

Digital camcorder image stabilizer based on graycoded bit-plane

block matching

We propose an efficient algorithm to eliminate the nonpleasing effect caused by

involuntary hand movement of camera holders. In our approach, 1-bit gray-
coded bit-plane block matching, instead of 8-bit gray-level block matching, is
used to greatly simplify the computation of motion estimation. This computation
saving makes possible a finer division of image frame and thus facilitates the
employment of a much more robust procedure for motion decision. To deal with
various interfering factors in motion estimation, the temporal information of
each local motion vector is also used to efficiently distinguish random-like
movement from temporally correlated movement. To compensate for camera
rotation, an affine model is used in the motion compensation unit without adding
too much computation load. Having considered both programming flexibility
and hardware efficiency, the motion decision unit and the motion compensation

[11]
unit are coded in a microprocessor that interconnects with the stabilization
hardware, which consists of the motion estimation unit and the digital zooming
unit.

A robust digital image stabilization technique based on inverse

triangle method and background detection

In this paper, a novel robust digital image stabilization (DlS) technique is

proposed to remove the unwanted shaking phenomena in the image sequences
captured by hand-held camcorders without affecting the moving objects in the
image sequence and intentional motion of panning condition, etc. It consists of a
motion estimation unit and a motion compensation unit. To increase the
robustness in the adverse image conditions, an inverse triangle method is
proposed to extract reliable motion vectors in plain images which are lack of
features or contain large low-contrast area, etc., and a background evaluation
model is developed to deal with irregular images which contain large moving
objects, etc. In the motion compensation unit, a CMV estimation method with an
inner feedback-loop integrator is proposed to remove the unwanted shaking
phenomena without losing the effective area of the image in panning condition.
We also propose a smoothness index (Sl) to quantitatively evaluate the
performances of different image stabilization methods. The experimental results
are on-line available to demonstrate the remarkable performance of the proposed
DIS technique.

[12]
Digital image stabilization technique for fixed camera on small
size drone

This paper explores a digital image algorithm to stabilize videos recorded from a
fixed camera (without stabilized machanical tools) on a small size drone. In
particular, this paper focuses on implementation of the Speed-Up Robust
Feature (SURF) method. The fundamental concept is to match 2 pictures, one
obtained from the current image frame and another from the previous (or
reference) frame. The matching process is achieved by locating common
keypoints between the current and reference frames and associating them
together. Then transformation is then implemented to translate and rotate the
current image frame so that keypoints remain in the same position or as close as
possible to those of the reference frame. Various video samples are used to
validate the SURF method's efficiency. The scenarios include recorded videos
under a normal light condition and having partial shadows on the image. The
movement due to drone's engine and environmental winds are also under this
study. The results indicate that the SURF method can be used to stabilize image
frame so that the processed video becomes smoother and more suitable for
viewing.

Robust Digital Image Stabilization Technique for Car Camera

This research studies the digital image stabilization technique for the In-Car
videos which are acquired from a car camera. Firstly, the relationship is
established between the lane-line positions in a camera coordinate and an image
plane. Then an analysis is performed to reveal the positions of the lane-lines in
In-Car videos. Next, a digital image stabilization method for car cameras is

[13]
proposed based on lane-line matching. This method begins with extracting the
lane-lines from an In-Car video. Then, feature triangles are constructed to
estimate the global inter-frame motions of the input video and a series of
compensating motion vectors are yielded by using Kalman filter based algorithm
with the inter-frame motions. Finally, repositioning the frames of the input
video, according to the compensating motion vectors, can produce a stabilized
In-Car video. The proposed method is resistant to the scene changes of In-Car
videos. The experimental results, both for the simulated In-Car videos and the
real ones, have demonstrated that the proposed method can robustly reduce the
effects of undesired car camera motions on In-Car videos.

[14]
 Chapter 4 : FINDINGS / HIGHLIGHTS

4.1 APPLICATIONS

1. Real-time digital image stabilization—also called electronic image

stabilization (EIS)—is used in some video cameras
2. Some still camera manufacturers marketed their cameras as having
"digital image stabilization"
3. Used in fixed cameras on small size drone such as Parrot Bebop 2 and
Yuneec Breeze drones
4. Used in Car cameras
5. Electronic Image Stabilization is used in most of the smartphones in
today’s world
6. It is used in the field of astrology
7. It can be used in the surveillance cameras mounted in an exposed location
such as on a high pole or a street sign near a busy road

4.2 ADVANTAGES

1. EIS systems are compact because they do not add any bulk to the lens
2. EIS systems are fast because there is no need for any physical movement,
as all the heavy lifting occurs electronically at ultra high speeds.
3. EIS systems are light
4. EIS systems are less expensive as compared to OIS

[15]
 5. With improvement in algorithms used for stabilization image degradation
is minimal.

4.3 LIMITATIONS

1. Similar to OIS, EIS cannot do much if the blurriness was from the
subject’s end or when the camera is shaken vehemently.
2. Also, at the algorithm front, EIS may have issues differentiating between
intentional tilts and haphazard camera movement. This is more an issue
when zooming in on subjects because that could result in the subject
taking too much frame space and the movement becoming more
noticeable.
3. It uses pixels to form the buffer zone that would otherwise be used in the
frame. This means that you have fewer pixels to work with in each frame
and thus you lose a bit of picture quality.
4. At low resolutions and in poor lighting conditions, there is a significant
degradation in the image or video quality

[16]
 CONCLUSION

Camera and smartphone buyers nowadays are bombarded with terms such as
OIS, VR, steady shot and much more when it comes to the camera USPs.
Taking pictures was a luxury owing to the high cost of film and photograph
development. But the biggest problem was the blurred shot that resulted due to
motion or shakes while capturing images. This was more prevalent in smaller
cameras.

With the rise of smaller cameras, users have started moving around with
cameras a lot more than in the past. And there are higher chances of images
getting blurred with motion and vibration while moving. In such a
scenario, image stabilization is required, and more and more cameras and
even the smartphone makers are now offering devices with image stabilization.

When you're shopping for a digital camera or camcorder just be sure to pay
close attention to what type of stabilization technology the manufacturer is
using.

Is it optical stabilization or digital stabilization…? It makes a big difference…!

EIS, compared to OIS, is cheaper and more feasible to be implemented on a

wider scale. For instance, it can be used on surveillance cameras found affixed
to traffic signal posts to cut out the jitteriness caused by passing traffic or winds,
resulting in a much smoother live capture. OIS can probably do a better job, but
it’s expensive and is not designed to be used in such locations since OIS has
moving components, which makes it a bit fragile.

[17]
 REFERENCES

1. https://en.wikipedia.org/wiki/Image_stabilization
2. https://www.motionelements.com/blog/articles/eis-or-ois-your-guide-to-
image-stabilization-systems
3. https://www.videoeditingsage.com/camera-image-stabilization.html
4. https://scialert.net/fulltext/?doi=itj.2011.335.347
5. https://ieeexplore.ieee.org/document/1467968/
6. http://www.explainz.com/explanations/camera/electronic-image-
stabilization-eis/
7. https://gadgetstouse.com/news/ois-vs-eis/7624
8. https://www.guidingtech.com/ultimate-guide-to-image-stabilization/

[18]