A CLUSTERING TECHNIQUE FOR SUMMARIZING

MULTIVARIATE DATA
By Geoffrey H . Ball and David J . Hall
Stanford Research Institute, Menlo Park, California

Scientific measurements frequently involve large numbers of variables whose complex

interactions are not easily found. A practical computing method termed ISODATA, which
h d s the cluster structure of such data, is described. The resulting description of the data
provides a fit to the data of a set of cluster centers that tends to miaimize the s u m of the
squared distances of each data point from its closest cluster center. An application to the
grouping or clustering of the answers of 209 people to an 80-question sociological survey
illustrates the utility of the method.
E*J

T HE work reported here grew out of

efforts to develop adaptive preprocessing
for pattern-recognition machines (Ball and
Hall, 1964).Itbecame apparent, however,that
the techniques developed were not limited to
this task. In fact, they seem applicable to a
recurring basic problem in many disciplines
-that of analyzing large numbers of sam-
ples of multivariate data.
This technique, termed ISODATA, groups
patterns in a way determined primarily by
the data itself. (ISODATA represents
Iterative Self-organizing Data Analysis
Technique (A)
This technique automatically clusters a11
of the data into distinct and independent
groups. An average response pattern is
used to represent a group of patterns, and
the iterative process creates new average
The application of ISODATA t o sociological
data waa aided by contracts from the Behavioral
Sciences Division of the Air Force Office of Scien-
tific Research, Washington, D.C. The develop-
ment of the technique was aided by the Graphical
Data Transducer Branch, Data Division, Com-
munications Department, USAEL, Fort Mon-
mouth, New Jersey, and by internal funding from
Stanford Research Institute. The questionnaires
described in this paper were designed and admin-
istered by the Systems Analysis Laboratory of
the Management Sciences Division of Stanford
Research Institute, under the direction of Dr.
Howard Vollmer. We would like to thank him for
providing the data and for his assistance in inter-
preting the results we obtained. (His interpreta-
tion appears in Vollmer, 1964.) Miss Wanda
Fedurek and Mrs. Klara Miracle rendered valuable
assistance in plotting and tabulating data, for
which we thank them.
153
Behavioral Science. Volume 12, 1967
response patterns to improve the accuracy
of trial or existing average response patterns.
The process also combines average response
patterns that are so similar that their being
separate fails to provide a significant amount
of information about the structure of the re-
sponse patterns. computation time for this
technique grows linearly with the number of
response patterns and the number of di-
mensions and the number of clusters. An
ALGOL program implementing this proce-
dure on a Burroughs B-5500 digital computer
has been applied to two actual research
problems as well as to artificial problems
(Ball, Brain, Burch, and Hall, 1964). (Run
times were typically 15 to 20 minutes.) In
outline form, the essential steps of the
technique are :
(1) Select a “typical” set of response
patterns to be used as initial “cluster
points.”
(2) Group together patterns that lie
closest in Euclidian distance to the
same cluster point. (The patterns and
these cluster points are both vectors.)
(3) Split into two points the average re-
sponse pattern of each group found in
Step (2) if the group’s associated
“within-group variability” exceeds a
threshold, ,e, which is set by the
operator. The group is split by form-
ing two new cluster points from the
average response pattern. The new
cluster points are identical to the
average response pattern, except for
that response to a single item, having
154 COMPUTERS
IN BEHAVIORAL
highest vsriability, which is given the
values +1 and -1, respectively, for
the two new cluster points. This re-
sults in ithe “birth” of new cluster
points.
Regroup the patterns using the new
cluster points, and then again find the
average response pattern within each
group.
Compute distances between all pairs
of average response patterns.
Combine groups whose average re-
sponse patterns are closer together
than a threshold value Bc. This corre-
sponds to the “death” of certain
cluster points through the combining
of two clusters.
Iterate the procedure.
The thresholds Bc and 13, are set by the
operator. By changing them he can obtain
any number of dusters between one and the
number required for every response pattern
to be in cluster by itself.
We now describe the analysis that the
technique makes possible of the responses by
209 Air Force scientists and engineers to an
80-item sociological questionnaire.
The procedure automatically sorted the
209 responses of patterns given by the Air
Force scientists into seven groups. (The col-
lective response of each subject to all 80
items will be called a response pattern.) These
seven groups are constructed so that the
average response pattern calculated from
the patterns within a group is reasonably
typical of that group. I n other words, any
response pattern in a group more closely
corresponded to the average response in its
own group than the average response pat-
tern of any other group.
These average response patterns were
used to provide a descriptive word “profile”
of each group, which helps the researcher to
get a “feel” for the structure of the group and
its relationship to the other groups. For ex-
ample, the word profile for Group 7 de-
scribes the people in this group as being be-
tween 30 and 50 years of age, (95 percent are
in this category); highly educated (50 per-
cent Ph.D., 50 percent M.S.); civilians, pri-
marily performing basic research (80 per-
Behavioral Science, V,>lume12. 1967
SCIENCE
cent) ; planning to work for the employing
organization in ten years; publishing pro-
fessional papers frequently (75 percent have
five or more in the last five years); quite
satisfied with their jobs (55 percent highly
satisfied and only 15 percent dissatisfied).
This can be compared to the word profile
of Group 6, where 20 percent are under 30
years of age; have a moderate amount of
education (33 percent M.S., 67 percent
B.S.) ; are civilians; primarily performing
applied or basic-applied research (60 per-
cent); planning to be working in the same
organization in 10 years; publishing in-
frequently (45 percent have 0-2 publications
in the last five years); only moderately satis-
fied with their jobs (30 percent highly satis-
fied and 35 percent dissatisfied).
The average response pattern in each
group is a real-valued vector. Using these
vectors, we were able to compute a distance
between all pairs of average response pat-
terns and to plot the average response pat-
tern for each of the seven groups.
We also examined the change in the aver-
age response pattern as we moved from one
group to another group. These changes were
descriptive of the relationships between the
groups.
It is often difficult to comprehend and
manipulate large tables of numerical data
and to extract the important relationships
from large quantities of numerical data. I n
this specific instance, the average response
pattern, calculated from all 209 responses, is
not an adequate representation of the data
-there is still too much variation around the
average for most of the responses. By cluster-
ing the response patterns into a number of
separate groups, we obtained seven “typi-
cal” average response patterns that were
much more meaningful because the variety
of response within each group had been re-
duced. These average response patterns can
be accurately related to a word profile that
is much better matched to human compre-
hension and evaluation. We found our
visual plot also helped to illustrate the rela-
tionships, although it is not essential and
may not be possible for larger numbers of
clusters.
 COMPUTERS SCIENCE
I N BEHAVIORAL 155

From our readings in the literature it is D. J. Graphical data processing research

evident that other people have considered
similar clustering techniques. We have
made a comparison of ISODATA with
these and other techniques (Ball, 1965).
We are continuing to modify and refine
ISODATA and have developed a program to
cluster real-valued as well as binary-valued
data (Ball and Hall, 1965). We are now ap-
plying it to the design of pattern recognition
preprocessing for use with meteorological
data. We are also using it in other applica-
tions including analysis of responses to
psychological tests and the examination of
physiological measurements.
REFERENCES
Ball, G. H. Data analysis in the social sciences-
what about the details. Proc. Fall Joint
Cornpuler Conference, 1965,27, Pt. I, 533-559.
Ball, G. H., Brain, A. E., Burch, G. H., & Hall,
study and experimental investigation. Tech-
nical Report 16, Contract DA 36439 AMC-
03247 (E), SRI Project 4565. Menlo Park,
California: Stanford Research Institute,
July, 1964.
Ball, G. H., & Hal1,D. J. Some fundamental con-
cepts and synthesis procedures for pattern
recognition preprocessors. Paper presented
a t the International Conference on Micro-
waves, Circuit Theory, and Information
Theory, Tokyo, September, 1964.
Ball, G. H., & Hall, D. J. ISODATA, A novel tech-
nique for data analysis, and pattern classi-
fication. Technical Report, Menlo Park,
California: Stanford Research Institute,
May 1965.
Vollrner, Howard M. Applications of the behav-
ioral sciences to research management; an
initial study in the office of aerospace re-
search. Technical Report, Menlo Park, Cali-
fornia: Stanford Research Institute, No-
vember 1964.
(Manuscript received September 13, 1966)

The pure desire for social good does not indeed operate in human
affairs unalloyed by egotistic motives, but on the other hand whst
wc call egotistic motives do not act without direction from an
involuntary reference to social good.
T. H. GREEN.Principles of political obligation, 1880

Behavioral Science, Volume 12. 1967