Vision &

Image
processing
John Hulskamp
E-mail: john@hulskamp.com.au
Consultant

john@hulskamp.com.au 1
Introduction
 Introducing the new subject 31049
Computer Vision & Image
Processing

 Guest Lecturer with previous

experience in teaching & research
in the area

john@hulskamp.com.au 2
Osteons
 Compact bone is comprised of many units called osteons: they consist of a central
canal surrounded by closely packed concentric layers called lamallae.
 This is an image of some osteons, specimen is from a 32yo male. Image is a
~2.8x3.5mm portion of a contact micro-radiograph taken from a 100micrometre
thick, un-embedded, hard-ground section. Black areas are voids, white = high
mineral density (maybe a couple of grains of carborundum can be seen).
 Attribution: David Thomas, Dental Sciences, University of Melbourne

john@hulskamp.com.au 3
Near IR Imaging
 Hot strip mill temperature measurement

john@hulskamp.com.au 4
Near IR Imaging
(2)
 Thermal Map
 Attribution: C. Lampe: “A Multi-Processor temperature profiling system
for real time control in a hot strip rolling mill” M.Eng. Thesis RMIT
1995

john@hulskamp.com.au 5
Topics for
Discussion
Today
 Subject objectives
 Fundamental Issues
 Applications
 Towards image understanding
 CV & IP Tools
 A Simple Beginning
 Conclusions

john@hulskamp.com.au 6
Subject
objectives
 adequate background knowledge about
computer vision and image processing

 practical knowledge and skills about

computer vision and image processing tools

 necessary knowledge to design and

implement a prototype of a computer vision
application

john@hulskamp.com.au 7
Fundamental
Issues
 An imaging system

john@hulskamp.com.au 8
Fundamental
Issues (2)
 Resolution - the smallest feature size on
your object that the imaging system can
distinguish
 Field of view - the area of inspection that
the camera can acquire
 Working distance - the distance from
the front of the camera lens to the object
under inspection
 Sensor size - the size of a sensor's active
area
 Depth of field - the maximum object
depth that remains in focus

john@hulskamp.com.au 9
Fundamental
Issues (3)
Resolution
 Resolution indicates the amount of object detail that the
imaging system can reproduce. You can determine the
required resolution of your imaging system by measuring in
real-world units the size of the smallest feature you need to
detect in the image.
 To make accurate measurements, a minimum of two pixels
should represent the smallest feature you want to detect in
the digitized image. In the picture , the narrowest vertical bar
(w) should be at least two pixels wide in the image. This
information can assist to select the appropriate camera and
lens for the imaging application.

john@hulskamp.com.au 10
Fundamental
Issues (4)
 Sensor resolution is the number of columns and rows
of CCD pixels in the camera sensor. To compute the sensor
resolution, you need to know the field of view (FOV).
 The FOV is the area under inspection that the camera can
acquire. The horizontal and vertical dimensions of the
inspection area determine the FOV. Make sure the FOV
encloses the object you want to inspect.

Once you know the FOV, you can use the following
equation to determine your required sensor resolution:

sensor resolution = (FOV/resolution) x 2

= (FOV/size of smallest feature) x 2

john@hulskamp.com.au 11
Fundamental
Issues (5)
Determine the focal length of your lens.

A lens is primarily defined by its focal length. This picture

illustrates the relationship between the focal length of the
lens, field of view, sensor size, and working distance which
is the distance from the front of the lens to the object
under inspection.

focal length = sensor size x working distance / FOV

john@hulskamp.com.au 12
Fundamental
Issues (6)
Lighting issues
One of the most important aspects of setting up your imaging
environment is proper illumination. Images acquired under
proper lighting conditions make your image processing
software development easier and overall processing time
faster. One objective of lighting is to separate the feature or
part you want to inspect from the surrounding background by
as many gray levels as possible. Another goal is to control the
light in the scene. Set up your lighting devices so that
changes in ambient illumination-such as sunlight changing
with the weather or time of day-do not compromise image
analysis and processing.

john@hulskamp.com.au 13
Fundamental
Issues (7)
 Backlighting is another lighting technique that can
help improve the performance of your vision system.
If you can solve your application by looking at only the
shape of the object, you may want to create a
silhouette of the object by placing the light source
behind the object you are imaging. By lighting the
object from behind, you create sharp contrasts which
make finding edges and measuring distances fast and
easy. This picture shows a stamped metal part
acquired in a setup using backlighting.

john@hulskamp.com.au 14
Fundamental
Issues (8)
Perspective
Perspective errors occur when the camera axis is not
perpendicular to the object under inspection. The figures
show both an ideal camera position (I.e. vertical) and a
camera imaging an object from an angle.

john@hulskamp.com.au 15
Fundamental
Issues (9)
Perspective and Distortion Errors
 Try to position your camera perpendicular to the
object under inspection to reduce perspective errors.
Integration constraints may prevent you from
mounting the camera perpendicular to the scene.
Under these constraints, you can still take precise
measurements by correcting the perspective errors
with spatial calibration techniques.

john@hulskamp.com.au 16
Examples of
Vision
Applications
 Attribution: National Instruments IMAQ Vision Product
Demonstration (www.ni.com/vision)

 Examples:
 Battery Clamp
 Spark Plug Gap Measurement
 Blister Pack Inspection
 PCB Inspection

john@hulskamp.com.au 17
Examples of
Vision
Applications (2)
 Battery Clamp

john@hulskamp.com.au 18
Examples of
Vision
Applications (3)
 Spark Plug Gap Measurement

john@hulskamp.com.au 19
Examples of
Vision
Applications (4)
 Blister Pack Inspection: Ensuring that blister packs contain the
correct number and type of pills before they reach pharmacies, ensuring
the integrity of the product and increase the yield of production by
automating the inspection of blister pack contents.

 Acquire colour images of the blister packs. Use colour location to count
the number of green areas in the image. With colour location, you create
a model or template that represents the colours that you are searching.
Then the machine vision application searches for the model in each
acquired image and calculates a score for each match. The surface area
of each pill in the pack must be at least 50% green to pass inspection.

john@hulskamp.com.au 20
Examples of
Vision
Applications (5)
 Blister Pack Inspection (cont’d)

john@hulskamp.com.au 21
Examples of
Vision
Applications (6)
 PCB Inspection
 To ensure that components are present and at the
correct orientation on a PCB.

john@hulskamp.com.au 22
Examples of
Vision
Applications (7)
 PCB Inspection (cont’d): Colour information simplifies
a monochrome problem by improving contrast or separation
of the components from the background. Colour pattern
matching can distinguish objects from the background more
efficiently than grayscale pattern matching.
 This example uses rotation-invariant pattern matching
because it can detect the components regardless of their
orientations. You can use the orientation information to
determine the correct placement of orientation-sensitive
components, such as capacitors or diodes.

john@hulskamp.com.au 23
Towards image
understanding
 Computer Vision: making useful decisions about real physical objects
and scenes based on sensed images

 To get to that understanding we need to process images: hence

Image Processing is a step towards this understanding

 Image Processing issues:

 Image Statistics & Histograms

 Image Enhancement

 Image Restoration

 Image Analysis: Edge Detection & Feature extraction

 Representation & Description

 Pattern Recognition

john@hulskamp.com.au 24
Histograms
 The cumulative histogram of a grey-level image f(w,h) is a
function H(k) which provides the total number pixels (number
of occurrences) that have grey-level less than the value k.

H (k ) = card{ f ( w, h) | f ( w, h) < k}
k ∈ [0, k max ]
( w, h) ∈ [(0,0), (W − 1, H − 1)]

 Where card stands for the cardinality (i.e. number of pixels)

of a set.
 The histogram of a grey-level image f(w,h) is a table h(k)
which is the discrete difference of the cumulative histogram. It
provides the total number pixels (number of occurrences) that
have a specific grey-level value k.
 Attribution: www.khoral.com

h(k ) = H (k ) − H (k − 1)

john@hulskamp.com.au 25
Histograms (2)

john@hulskamp.com.au 26
Histogram
Equalisation

john@hulskamp.com.au 27
CV & IP Tools
 NIH Image is a public domain image processing and analysis program
for the Macintosh. It was developed at the
Research Services Branch (RSB) of the National Institute of Mental
Health (NIMH), part of the National Institutes of Health (NIH). A free PC
version of Image, called Scion Image for Windows, is available from
Scion Corporation. Image can acquire, display, edit, enhance, analyse
and animate images. It reads and writes TIFF, PICT, PICS and MacPaint
files, providing compatibility with many other applications, including
programs for scanning, processing, editing, publishing and analysing
images. It supports many standard image processing functions, including
contrast enhancement, density profiling, smoothing, sharpening, edge
detection, median filtering, and spatial convolution with user defined
kernels.
 Image can be used to measure area, mean, centroid, perimeter, etc. of
user defined regions of interest. It also performs automated particle
analysis and provides tools for measuring path lengths and angles.
Spatial calibration is supported to provide real world area and length
measurements. Density calibration can be done against radiation or
optical density standards using user specified units. Results can be
printed, exported to text files, or copied to the Clipboard.

john@hulskamp.com.au 28
CV & IP Tools (2)
 Khoral Inc:It started out as Khoros about 10 years ago, for UNIX-
based X-systems, has become available for the Windows platform as
well. Costs money today!
 Their online DIP course is very good:
http://www.khoral.com/contrib/contrib/dip2001/index.html

john@hulskamp.com.au 29
CV & IP Tools (3)
 UTHSCSA ImageTool
 UTHSCSA ImageTool (IT) is a free image processing and analysis
program for Microsoft Windows 9x, Windows ME or Windows NT. IT
can acquire, display, edit, analyse, process, compress, save and print
gray scale and colour images.IT can read and write over 22 common
file formats including BMP, PCX, TIF, GIF and JPEG. Image analysis
functions include dimensional (distance, angle, perimeter, area) and
gray scale measurements (point, line and area histogram with
statistics). ImageTool supports standard image processing functions
such as contrast manipulation, sharpening, smoothing, edge
detection, median filtering and spatial convolutions with user-defined
convolution masks.
ImageTool was designed with an open architecture that provides
extensibility via a variety of plug-ins. Support for image acquisition
using either Adobe Photoshop plug-ins or Twain scanners is built-in.
Custom analysis and processing plug-ins can be developed using the
software development kit (SDK) provided (with source code). This
approach makes it possible to solve almost any data acquisition or
analysis problem with IT.

john@hulskamp.com.au 30
CV & IP Tools (4)
 Intel open source computer vision library OpenCV

 CVIPtools: is a UNIX/Win32-based software package developed \ under the

continuing direction of Dr. Scott E Umbaugh One of the primary purposes of the
CVIPtools development is to allow students, faculty, and other researchers to
explore the power of computer processing of digital images.
 CVIPtools is a collection of computer imaging tools providing services to the users
at three layers. At the bottom level are the CVIPtools libraries (the application
programming interface). Based on the CVIPtools libraries are the cviptcl and
cvipwish shells. The cviptcl shell is an extension of Tcl with additional CVIP
capabilities. With cviptcl, the user can either use the command line for interactive
image processing, or write cviptcl shell scripts for batch processing. The cvipwish
shell is the extension of cviptcl with the added functionality for building a graphical
user interface (GUI) which allows even the casual computer users to experiment
with many of the sophisticated tools available to computer imaging specialists
without the need for any knowledge of computer programming.

john@hulskamp.com.au 31
Topics in Course
 Image formation. Human vision. Computer vision.
 Image representation. Analog and digital images. Common image and
video formats. Image compression. JPEG and MPEG.
 Image acquisition. Image acquisition systems. Cameras and frame
grabbers.
 Image processing. Image enhancement. Convolution. Filtering. Edge
detection. Texture analysis. Labelling. Contour tracing. Image
morphology. Image segmentation.
 Image analysis. Feature extraction. Geometrical features. Hough
transform.
 Image sequences. Motion detection. Optical flow. Background
subtraction. Feature tracking.
 Pattern recognition. Object classification. Statistical, neural networks,
symbolic classifiers.
 Computer Vision. Model-based vision. Applications in industrial quality
inspection, video surveillance, robotics, medicine, multimedia .

john@hulskamp.com.au 32
A Simple
Beginning
 Paradigm for an image processing code:
Declarations

File In: Bring in image into Array I[r,c]

Processing on Array I[r,c] to produce

Array O[r,c]

File Out: Output Array O[r,c] as image

john@hulskamp.com.au 33
Conclusions
 An overview of what is ahead for
you in this exciting field.
 You might like to experiment with
the above simple code to attempt
to gain the histogram for an image,
and try out “histogram
equalisation”

john@hulskamp.com.au 34