ELSEVIER Computers in Industry 28 ( 19%) 103- 111

Automated inspection of printed circuit boards

through machine vision
Wen-Yen Wu a, Mao-Jim J. Wang b**, Chih-Ming Liu b
a Department of Industrial Management, Kaohsiung Polytechnic Institute, Kaohsiung, Taiwan, 84008, ROC
b Department of Industrial Engineering, National Tsing Hua University. Hsinchu, Taiwan, 30043, ROC

Received 24 May 1994; accepted 14 August 1995

Abstract

This paper introduces the development of an automated visual inspection system for printed circuit boards (PCBs). It
utilizes an elimination-subtraction method which directly subtracts the template image from the inspected image, and then
conducts an elimination procedure to locate defects in the PCB. Each detected defect is subsequently classified into one of
the seven defect types hy three indices: the type of object detected, the difference in object numbers, and the difference in
background numbers between the inspected image and the template. Finally, a 256 X 240 PCB image was tested to show the
effectiveness of this system.

Keywords: Machine vision; Automated inspection; Printed circuit board

1. Introduction Besides, automated visual inspection can be carried

out in an unfavorable environment, and the test
Industrial inspectiian plays a very important role
information can be recorded readily for subsequent
in the manufacturing process. In most automated
statistical analysis. In order to achieve high effi-
manufacturing systems, an attempt is often made to
ciency and effectiveness, the development of an
achieve 100% quality assurance for all parts, sub-
automated visual inspection system is essential.
assemblies, and finished products. As a result, the
Digital image processing can solve a wide range
inspection operation is usually the largest single cost
of problems encountered in quality control. Ad-
in manufacturing [ 1,2]. A printed circuit board (PCB)
vances in technology have resulted in better and
is a basic component of many electronic devices.
cheaper image-analysis equipment. Recently, an in-
The quality of PCBs will have a significant effect on
creasing number of machine vision systems have
the performance of many electronic products. Con-
been used for PCB inspection. Various automated
ventionally, visual inspection of PCBs is done manu-
visual inspection systems for PCBs have been devel-
ally by inspectors. It is known that humans are
oped in the past years [5- 161. The approaches can be
subject to make mistakes, and they are slow and less
classified into three categories: reference compari-
consistent than automated inspection systems [3,41.
son, non-reference verification, and hybrid tech-
niques.
Corresponding author. In the reference comparison approach, the board

0166-3615/96/$15.00 0 1996 Elsevier Science B.V. All rights reserved

SSDI 0166-3615(95)00063-l
104 W.-Y. Wu et al./Computers
to be inspected is first scanned and its image is
compared to that of a standard board to identify
defects. This method is also called the template
matching technique. The limitations of this approach
are large reference data storage, precise alignment
requirements, and sensitiveity to illumination and
sensor conditions.
In the non-reference verification approach, the
inspection is simply a dimensional verification. The
task is to test whether each wiring track and pad fall
within the predefined limits or not. The method is
also called the design-rule technique. This approach
does not have the disadvantages of the reference
method, but it may miss large flaws and distorted
features.
The hybrid technique is a combination of the
reference and the non-reference techniques. It has the
advantages of these two techniques. However the
technique may be too complex and may result in
high computational costs.
Since the hardware is becoming cheaper and
cheaper, the limitation of a large reference data
storage for direct comparison is no longer a serious
drawback. And the alignment of PCBs can be done
by a mechanical device as well as the software
technique. Hence the reference technique is desirable
because of its simplicity.
Ejiri et al. [7] applied one of three different
methods - expansion-contraction, boundary-averag-
ing, and reverse-averaging - to generate a pseudo-
standard pattern from the input pattern. Subse-
quently, the input pattern was subtracted from the
pseudo-standard pattern followed by the boundary
effect compensation and restoration to detect tiny
defects in a complicated pattern. Large faults may be
missed in this method. Besides, if the minimum track
width or the minimum distance between two tracks
is less than 3 pixels, some false alarms may be
involved. In addition, the system did not tell the
types of defects detected.
Later, Ahluwalia et al. [5] converted the binary
image into a skeletal form, by using the contraction
technique, to make it suitable for analysis. Five types
of PCB defects - shorts, breaks, partial cracks or
holes on the tracks, increased or decreased track
width, and defects on the soldering pad - can be
detected by evaluating the streak count, streak area,
and centroid.
in Industry 28 (1996) 103-111
This paper proposes a new automated visual in-
spection system which is based on the reference
method. The method involves two major stages:
Stage 1. Detect the defects by an elimination-sub
traction procedure.
Stage 2. Classify the detected defects by using
three indices: (1) the type of object detected, (2) the
difference in object numbers, and (3) the difference
in background numbers between the inspected image
and the template.
2. Defect detection
This method first stores a perfect PCB image as
the template in advance and then compares the in-
spected image with the template image to locate
defects. The differences found between the inspected
PCB image and the template image are mainly due
to defects involved in the inspected PCB. But the
differences may also be caused by uneven binariza-
tion of edges as well as noise. Hence, the method
first performs a direct subtraction followed by an
elimination procedure to detect the actual defects.
In order to reduce the amount of information
processed and the processing time, the gray images
are converted to binary images. The image subtrac-
tion is performed by assigning one of the three
outcomes - equal (El, positive (PI, or negative (N) -
to the pixel in the subtracted image. Table 1 shows
the rules of this subtraction procedure. For example,
if the pixel in the inspected image belongs to the
object, and the corresponding pixel of the template
image belongs to the background, then the corre-
sponding pixel in the subtracted image is called a
P-pixel. For convenience, the gray levels for the E-,
P-, and N-pixels can be set as 0, 255, and 127
Table 1
Possible outcomes of subtracting the template from the inspecting
image
Template image Inspected image
Object Background
N
Object E
Background P E
 W.-Y. Wu et al./Computers in Industry28 11996) 103-111 105

respectively. Thus each pixel in the subtracted image
is expressed in E-, P-, and N-pixels.
The next step is to eliminate the P- or N-pixels
caused by noise from the subtracted image. The
elimination procedure is to examine each remaining
pixel and to determine whether the corresponding
pixel in the template image is an edge or not. If it is
not an edge, then its gray level should not be changed
and it is considered as a defective pixel. If it is an
edge, we further test whether the number of pixels
that have the same gray level in a 3 X 3 mask of the
subtracted image is greater than or equal to the
prespecified allowance. This procedure is to examine
the possible deformation due to uneven binarization.
If it is greater than or equal to the allowance, then its
gray level should not be changed and it is also
considered as a defective pixel. If both the above
conditions are not met, then the pixel is considered
as an E-pixel, and it is eliminated from the sub-
tracted image. The resulting image after the elimina-
tion procedure is called the residual image.
The portions classified as E-pixels are the perfect
parts, but the portions containing P- or N-pixels are
considered as defects in the inspected PCB. Hence, if

Residual image Inspecting image Template image

n Etching problem

E El

ml Positive
deformation

Negative
deformation
El

Short
mm

cut

El

Missing hole

Pin hole
m

I P-object N-object

Background

Fig. 1. Seven types of defects in PCB.

106 W.-Y. Wu et d/Computers in Industry 28 (1996) 103-111

all the pixels in the residual image are E-pixels, then inspected image and the template. The specification
the inspected PCB is perfect. Otherwise, the defects of the values for each index is listed in the follow-
in the residual image should be classified further. ing:

- 1 if the detected object is an N-object,

TOD= 1
( if the detected object is a P-object.

3. Defect classification
-1 ifOI<OT,
DON = 0 ifOI=OT,
As described in the last section, if there are P- or i 1 ifOI>OT.
N-pixels located in the residual image, then we must
rescan all the P- and N-pixels in the residual image -1 ifBI<BT,
to classify the defects. Here, by using the g-con- DBN = 0 ifBI=BT,
nectedness concept, the P-pixels (or N-pixels) which i 1 ifBI>BT.
are S-connected to each other in the residual image
are defined as a P-object (or N-object). Here, 01 (BI) is the object (background) number in a
The PCB defects can be classified into seven small rectangular region of the inspected image and
categories which include etching problem, positive OT (BT) is the object (background) number in the
deformation, negative deformation, short, cut, miss- same region of the template.
ing hole, and pin hole. In order to distinguish these Since the 4- or 8-connectedness concept cannot be
seven type of defects, three indices are used to used for both the objects and backgrounds simultane-
classify them: the type of object detected (TOD), the ously [17], we use 8-connectedness for object count-
difference in object numbers (DON), and the differ- ing and 4connectedness for background counting.
ence in background numbers (DBN) between the The method of counting the number of objects (or

Tyveof Difference in Difference in

object detected ob]ectnumbers backgroundnumbers

/ -1 (Shorty -I (Missing hole )

l1
~ +I ,,t,,,~ +I F"tion )

problem )

Fig. 2. PCB defect classification diagram (* : impossible case).

 W.-Y. Wu et al./Computers
backgrounds) to get 01, OT, BI and BT, is accom-
plished by the following procedure:
Step 1. Search the first pixel of the next un-
counted object (background) in the defined region.
Step 2. Increase tlhe object (background) counter
if the first pixel is found. Otherwise, return the value
of the object (background) counter and stop.
Step 3. Change rhe gray levels for the pixels
which are 8-connected (4-connected) to the scanned
object (background) top down and then bottom up
iteratively. Go to Step 1.
Fig. 1 illustrates the defects of etching problems,
positive deformatiorrs, shorts, and missing holes in
the form of P-0bject.s in the residual image. On the
other hand, defects like negative deformations, cuts,
1 1. Get template image
2. Set parameters.
?-sP-
Fig. 3. PCB defect detection and classification
in Industry 28 (1996) 103-111 107
and pin holes are in N-objects. So, they can be
classified into two categories by the following rules:
1. If TOD = - 1, then the defect is a negative defor-
mation, cut, or pin hole.
2. If TOD = 1, then the defect is an etching prob-
lem, positive deformation, short, or missing hole.
Furthermore, as shown in Fig. 1, the following rules
hold:
1. IfTOD= -1,and
1.1. DON = - 1, then it is impossible;
1.2. DON = 0, then the defect is a negative de-
formation or pin hole;
1.3. DON = + 1, then the defect is a cut.
2. IfTOD= +l,and
2.1. DON = - 1, then the defect is a short;
1
Get the residual image by
elimination-subtraction method
I Search ob]ect In the residual image
Object numbers comparison
Classify defect by detected object
Background numbers COmpOrison
Classify defect by difference in
background numben
flow chart.
1OB W.-Y. Wu et d/Computers
2.2. DON = 0, then the defect is a positive defor-
mation or missing hole;
2.3. DON = + 1, then the defect is an etching
problem.
Since the above rules cannot further classify the
defects when TOD = - 1 and DON = 0, as well as
when TOD = + 1 and DON = 0, it is necessary to
use another criterion to distinguish them. The index
of the difference in background numbers (DBN) is
thus applied to further classify them:
Fig. 4. AI1 the possible homogeneous local patterns of a center-filled 3 X 3 mask.
in Industry 28 (1996) 103-111
1. IfTOD=-l,DON=O,and
1.1. DBN = - 1, then it is impossible;
1.2. DBN = 0, then the defect is a negative de-
formation;
1.3. DBN = + 1, then the defect is a pin hole.
2. IfTOD= +l,DON=O,and
2.1. DBN = - 1, then the defect is a missing
hole;
2.2. DBN = 0, then the defect is a positive defor-
mation;
2.3. DBN = + 1, then it is impossible.
 W.-Y. Wu et d/Computers in Industry28 (1956) 103-111 109

Fig. 5. An illustration of the defect detection procedure.

To summarize the above descriptions, the PCB area of the detected positive or negative deformation
defect classification rules can be represented by a defect is less than the deformation allowance, then it
decision tree as displayed in Fig. 2. Further, if the is considered as acceptable.

Table 2
An example of PCB inspection data output
Type of defect Area X Y DX DY Number
Etching problem 10 159 112 5 4 1
Positive deformation 19 162 81 8 4 1
Negatie deformation 10 200 121 5 3 1
Short 6 155 216 2 4 1
Cut 2 175 158 1 3 1
Missing hole 91 152 141 10 12 1
Pin hole 7 149 62 4 3 1

Resolution = 256 X 24OBinarization threshold = 1423 X 3 mask allowance = 4De.fotmation allowance = 6Expanded x and y sizes =
(3,3)Total number of defects = 7Processing time = 15.68 s
110 W.-Y. Wu et al./Computers
4. Implementation
The PCB automated inspection algorithm was
tested by using a real PCB pattern on a PC/AT with
a PCVISION-plus frame grabber. To facilitate the
quality of the grabbed images, polarized filters were
used both in the lighting device and the camera lens.
Fig. 3 demonstrates the overall inspection procedure.
In defect detection, while eliminating the uneven
binarization of edges as well as the noise pixels, the
P-pixel (or N-pixel) in the residual image is consid-
ered as a defective pixel if it is not an edge pixel in
the corresponding template image. This is because
the uneven binarization of edges can only occur in
&connected neighbors of an edge. Furthermore, a
pre-specified allowance must be given by the user at
the system setup stage through a trial and error
process. In here, it is suggested that the allowance is
set to be 4. For any P-pixel (or N-pixel) in the
subtracted image, if the focal pixel is defective, the
number of the same gray level pixels in a 3 X 3
mask is usually greater than or equal to 4. Fig. 4
shows all the possible homogeneous local patterns of
a center-filled 3 X 3 mask.
Fig. 5 illustrates the defect detection process. As
can be seen, the screen (resolution = 512 X 480) is
divided into four quadrants. A 256 X 240 PCB image
with seven different defects is displayed on the
upper-left quadrant, and the template is shown in the
lower-left quadrant. The subtracted image with noise
pixels is displayed in the lower-right quadrant.
Through the elimination procedure, all the unwanted
pixels are eliminated and the actual defective pixels,
as shown in the upper-right quadrant, are left for
further classification. The proposed method is inde-
pendent of the size and composition of the inspected
PCB pattern. If the PCB track is too small, a magni-
fying device can be used for the taking of proper
images.
The inspection and classification results are stored
in a data file (see Table 2). The data in Table 2
include the pre-specified binarization threshold, 3 X 3
mask allowance, the deformation allowance, the pro-
cessing time, and the total number of defects de-
tected. As we can see, all the defects were success-
fully detected. Besides, the location, the area, and the
type of each detected defect are also demonstrated in
the lower half of the table. The dimensions of the
in Industry28 (1996) 103-111
area and the location (X, Y, DX, DY) are in pixels.
The processing time is 15.68 s. It takes 5.26s for
defect detection and an average of 1.49 s for classify-
ing each detected defect. The processing time can be
greatly reduced by enhancing the capability of the
computer.
5. Conclusion
A two-stage PCB automated inspection method is
demonstrated in this paper. The method first uses a
direct subtraction and an elimination procedure to
detect defects. Further, it uses three indices - (1) the
type of object detected, (2) the difference in object
numbers, and (3) the difference in background num-
bers between the inspected image and the template -
to classify each detected defect into one of the seven
defect types. If the defect is identified as positive or
negative deformation, then a deformation allowance
test is checked to determine whether the deformation
is acceptable or not.
A real PCB pattern was tested by the proposed
method. All the defects have been properly detected.
For the speed performance, the method is considered
to be efficient because only simple features are used
to classify the defects. In addition, operations like
subtraction/elimination and counting the numbers of
the objects (or backgrounds) in the inspected image
and the template can be implemented by the use of
parallel processing, which can reduce the processing
time substantially. Further, the PCB alignment prob-
lem can be solved by using an external mechanical
fixture to aid for positioning. Overall, the proposed
automated PCB inspection method, which is inde-
pendent of the size and composition of the inspected
PCB pattern, is found to be very simple and effec-
tive.
References
[I] R.T. Chin and CA. Harlow, Automated visual inspection: a
survey, IEEE Tran.~. Pattern Anal. Muck. Intell. 4(6) (I 982)
557-573.
[2] R.T. Chin, Automated visual inspection: 1981 to 1987,
Comput. Vision Graph. huge Process. 41 (1988) 346-381.
[3] H.K. Tonshoff, H. Janocha and M. Seidel. Image process-
ing in a production environment, Ann. CIRP 37(2) (1988)
579-590.
 W.-Y. Wu et al./Computers
[4] A.M. Wallace, Industrial application of computer vision
since 1982. IEE Proc. 135(3) (1988) 117-136.
[5] R.S. Ahluwalia and S.A. Sitamman, Machine vision system
for online inspection of printed circuit boards, in A. Mital
(ed.), Advances in kdustrial Ergonomics and Safety I, 1989,
pp. 837-844.
[6] A.M. Darwish and A.K. Jain, A rule based approach for
visual pattern inspection, IEEE Trans. Pattern Anal. Mach.
Intell. lo(l) (1988) 56-68.
[7] M. Ejiri, T. Uno, M. Mese and S. Ikeda, A process for
detecting defects in complicated patterns, Comput. Graph.
Image Process. 2 (1973) 326-339.
[8] N. Goto and T. Kondo, An automatic inspection system for
printed wiring board masks, Pattern Recognition 12 (1980)
443-455.
[9] P.M. Griffin, J.R. Villalobos, J.W. Forst III and S.L. Mes-
simer, Automated visual inspection of bare printed circuit
boards, Comput. Ind. Eng. 18(4) (1990) 505-509.
[lo] Y. Hara, N. Akiyama and K. Karasaki, Automatic inspec-
tion system for primed circuit boards, IEEE Trans. Pattern
Anal. Mach. Intell. .5(6) (1983) 623-630.
[I I] Y. Ham, H. Doi, K. Karasaki and T. Iida, A system for
PCB automated inspection using fluorescent light, IEEE
Trans. Pattern Anal. Mach. Intell. 10 ( 1) ( 1988), 69-78.
[12] J.F. Jarvis, A method for automating the visual inspection
of printed wiring boards, IEEE Trans. Pattern Anal, Mach.
Intell. 2(l) (1980) 77-82.
(131 S.H. Oguz and L. Onural, An automated system for
design-rule-based visual inspection of printed circuit boards,
Proc. I992 IEEE Int. ConJ on Robotics and Auromarion,
1991, pp. 2696-2701.
[14] G.A.W. West, A system for the automatic visual inspection
of bare-printed circuit boards, IEEE Trans. Systems Man
Cybern. 145) ( 1984) 767-773.
[15] Q.Z. Ye and P.E. Danielsson, Inspection of printed circuit
boards by connectivity preserving shrinking, IEEE Trans.
Pattern Anal. Mach. Intell. lO(5) (1988) 737-742.
[16] H. Yoda, Y. Ohuchi, Y. Taniguchi and M. Ejiri, An
automatic wafer inspection system using pipelincd image
processing techniques, IEEE Trans. Patfern Anal. Mach.
Intelf. 10(l) (1988) 4-16.
in Industry 28 (1996) 103-111 111
[ 171 T.Y. Kong aud A. Rosenfeld, If we use 4 or 8-cormected-
ness for both tbe objects and the background, the Euler
characteristic is not locally computable, Pattern Recogni-
tion Let?. 11 (1990) 231-232.
Wen-Yen Wu is an Associate Professor
of Industrial Management at Kaohsiung
Polytechnic Institute, Kaohsiung, Tai-
wan, ROC. He received a B.S. degree
(1988) in Mathematical Sciences from
National Chengchi University, and M.S.
(1990) and Ph.D. (1993) degrees in In-
dustrial Engineering from National Ts-
ing Hua University. His research inter-
ests include industrial inspection, image
processing, and machine vision.
Mao-Jiun J. Wang is a Professor of
Industrial Engineering at National Tsing
Hua University, Taiwan, ROC. He re-
ceived his Ph.D. in Industrial Engineer-
ing from the State University of New
York at Buffalo in 1986. His research
interests include industrial inspection,
machine vision applications, occupa-
tional ergonomics, and applications of
fuzzy set theory. He has published more
than 40 journal papers in these areas.
Chih-Ming Liu received his MS. in
Industrial Engineering from Kansas State
University in 1980 and his Ph.D. in
Industrial Engineering from the Univer-
sity of Wisconsin-Madison in 1987. He
is a Professor in the Industrial Engineer-
ing Department at National Tsing Hua
University. His research interests in-
clude systems engineering, modeling and
performance evaluation of complex sys-
tems, and design of automatic manufac-
turing systems.