1

Using ISODATA Unsupervised Classification, Ground Truth Data, and ArcGIS Models to Classify the
Baker to Bay Image of Bellingham, Washington

Tori Niewohner

Abstract

In this lab, I ran an ISODATA unsupervised classification on the Baker to Bay image taken in

July 2011, which is a portion from a Landsat TM image. For each of the 50 spectral classes, I assigned an

information class using ground truth data, the color-IR image, and Google Earth. I further refined this

classification by using ArcGIS models to distinguish between several classes using roads, slope, and

elevation. The overall accuracy of the resulting image prior to ArcGIS refinement was 51.4% with 12

classes. After applying the ArcGIS models, the overall accuracy was 57.3% with 13 classes. Although the

accuracy improvement was only 5.9%, the second image looks much better because it removed large

areas of land on and around Mt. Baker that were being displayed as water, urban, and residential land.

However, it also classified several rivers as shadow instead of water. If I were to repeat this process and

aim for a higher accuracy, I would need better ground truth data with more extensive coverage and

further subdivided information classes. I also would alter the ArcGIS models to change the slope

threshold between water and shadow and alter the definitions of low, medium, and high elevation.

Methods

All of the methods that follow and the image used were provided by Dr. Wallin on his website

(Wallin 2016). I used ground truth data for my training and test data which also was provided by Dr.

Wallin (Wallin 2016). The initial image for this lab was a Baker to Bay image from July of 2011 of the

area from Bellingham Bay west to Mt. Baker, north to Canada, and about the same distance south. This

Baker to Bay image is a 1500 lines by 2500 columns piece of a full TM scene with TM Bands 1,2 3, 4, 5,

and 7 and a pixel size of 25 meters by 25 meters. I also used ground truth training and test data which is
 2

a compilation of data collected by Dr. Wallin, me and my peers, previous students, and many others,

which records the Landuse-Landcover (LULC) code at various points which people visited.

The first step in this lab was to run an ISODATA Unsupervised Classification using the program

ENVI on the Baker to Bay image. The result was an image with fifty spectral classes which all needed to

be assigned to information classes. I did some initial assignments of spectral classes to information

classes by simply identifying a location I knew had a certain land type, like a lake, and assigning that

spectral class to the information class water and changing its color. After a few classifications like this, I

reached a point where it was too difficult to determine the information class or it was a spectral class

with multiple information classes in it. To move forward, I grabbed the ground truth training data which

contained 19 information classes and created regions of interest from it. I obtained the statistics for the

regions of interest which displayed the number of pixels from each information class in a given spectral

class and various other information. I brought this data into Microsoft Excel to allow for easier

manipulation and viewing. I combined several of the information classes and added a class resulting in

14 classes as follows: residential, urban, pasture, crops, 1973-1992 clear-cut, 1992-2005 clear-cut, 2005-

2011 clear-cut, deciduous forest, coniferous forest, water, soil/rock, alpine, snow/ice, and clouds. Using

this information, I assigned information classes to spectral classes where the numbers were large

enough to be relevant and updated the image with those names and relevant colors. After I ran out of

easy classes to assign, I used a combination of the image in color-IR and Google Earth to assign

information classes to the remaining spectral classes. Following this, I combined all 50 spectral classes

into the 14 classes (one for each information class).

To determine the accuracy of my classification I used a confusion matrix with the ground truth

test data. The confusion matrix displayed an overall accuracy, producers accuracy (the probability that a

given area is classified as the correct information class), and users accuracy (the probability that a pixel

classified as a certain information class is actually that class). To increase accuracy I combined pasture
 3

and crops and the clear-cuts from 1975 to 2005. In order to continue improving accuracy I used ArcMap

models to refine my classification based on rules. Using a roads layer, I ran a model to determine

whether portions of the urban, residential, or rock classes would remain that class or switch classes

based on their distance to a road. I ran another model using a slope layer which stated that any water

with a slope of zero would remain as water and any other slope would be considered shadow. The final

model used an elevation layer which determined whether portions of the pasture, crops, 1973-1992

clear-cut, 1992-2005 clear-cut, 2005-2011 clear-cut, and alpine classes would remain that class or switch

classes based on their elevation (low: below 200 meters, medium: 200-1500 meters, and high: above

1500 meters). I moved these results to an ENVI file and colored and named the 16 classes (two

additional classes of shadow and unclassified were added). Following this, I generated a confusion

matrix with my new classification scheme. To increase accuracy I combined pasture and crops and the

clear-cuts from 1975-2005.

Results

After my initial classification, I ended up with an image that made much more sense than the

very first image that came after running the unsupervised classification. I ended up with 14 information

classes, each assigned to varying numbers of spectral classes. 2 of the spectral classes went into the

information class of residential, 4 into urban, 5 into pasture, 1 into crops, 2 into 1973-1992 clear-cut, 2

into 1992-2005 clearcut, 1 into 2005-2011 clear-cut, 1 into deciduous forest, 9 into coniferous forest, 4

into water, 9 into soil/rock, 3 into alpine vegetation, 5 into snow/ice, and 2 into cloud. After combining

spectral classes, I got an image that displayed the 14 information classes each in a different color. Before

settling on my image, I ran an accuracy assessment to determine what my overall accuracy was and

whether or not I should collapse some of the classes. My overall accuracy for the image was 44.6%, but

after merging pasture and crops and also the 1973-1992 and 1992-2005 clear-cuts, the accuracy went
 4

up to 51.4% (Table 1). I collapsed those classes together in my image which resulted in an image with 12

classes (Figure 1). The class which took up the most area was coniferous forest covering 641.5 km2 or

27% of the image (Table 1). Cloud takes up the least amount of the image at 8.8 km 2 or 0.4%, but the

class that is actually on the ground that takes up the least amount of the image is alpine vegetation at

13.7 km2 or 0.6% (Table 1). The class with the highest producers and users accuracy is snow/ice at

92.9% and 86.7% respectively (Table 1). Alpine vegetation had extraordinarily low producers and users

accuracy at 0% (Table 1).

Looking at my first image, there are many parts of the image that look qualitatively good.

However, there are some clear inaccuracies. Near Mt. Baker there are chunks of land being displayed as

water, urban, and residential land which is clearly not correct (Figure 1).

Table 1. The data from Figure 1, the Baker to Bay unsupervised classification image created using
ground truth training data, Google Earth, and the color-IR image are reported here. The image has 12
classes with data on how many pixels are in each class, the area in square kilometers, the percent of the
image that the class takes up, the producers accuracy, the users accuracy, and the overall accuracy.
Area Percent of Producers Users
Class Information Class Pixels
(km2) Image Accuracy (%) Accuracy (%)
1 Residential 158,983 99.4 4.2 51.2 44.3
2 Urban 53,355 33.3 1.4 43.2 57.7
3 Pasture/Crops 620,878 388 16.3 61.1 58.6
4 73-05 Clear-cut 1,004,314 627.1 26.4 59.8 47.8
5 05-11 Clear-cut 41,191 25.7 1.1 62.5 32.3
6 Deciduous Forest 346,813 216.8 9.1 20.5 22.8
7 Coniferous Forest 1,026,363 641.5 27 43.1 62
8 Water 196,877 123 5.2 80.6 85.7
9 Soil/Rock 71,321 44.6 1.9 35.3 18.8
10 Alpine Vegetation 21,976 13.7 0.6 0 0
11 Snow/Ice 249,886 156.2 6.6 92.9 86.7
12 Cloud 14,143 8.8 0.4 n/a n/a
Overall Accuracy: 51.4%
 5

Legend
1: Residential
2: Urban
3: Pasture/Crops
4: 1973-2005 Clear-cut
5: 2005-2011 Clear-cut
6: Deciduous Forest
7: Coniferous Forest
8: Water
9: Soil/Rock
10: Alpine Vegetation
11: Snow/Ice
12: Cloud

Figure 1. This image is the result of unsupervised classification on the Baker to Bay image from July of
2011. The spectral classes were initially assigned to 14 information classes using ground truth training
data, Google Earth, and the color-IR image, but that was collapsed down to 12 classes. The overall
accuracy for this image is 51.4%.

After I refined my classification using ArcGIS models, I ended up with an image with 16 classes

(the two additional classes are shadow and unclassified). However, looking at my accuracy assessment

with an overall accuracy of 47.9% caused me to collapse the classes of pasture and crops and 1972-1992

and 1992-2005 clear-cuts again which brought the overall accuracy up to 57.3%, which is a 5.9% increase

in accuracy from the first image (Table 1 and 2). The resulting image has 13 classes (Figure 2). The class

taking up the most area is coniferous forest, again taking up 27%, which makes sense because the

ArcGIS models made no changes to coniferous forest (Table 2). The two classes taking up the least area

are once again cloud and alpine vegetation, but this time both at 0.4% (Table 2). Residential, urban,

pasture/crops, 1973-2005 clear-cut, 2005-2011 clear-cut, water, soil/rock, alpine vegetation, and

shadow all changed in respect to area from the first to the second image with pasture/crops having the

largest change of increasing by 4.6% (Table 1 and 2). Residential, urban, 1973-2005 clear-cut, 2005-2011

clear-cut, water, and alpine vegetation all decreased in area and pasture/crops, soil/rocks, and shadow

all increased in area (Table 1 and 2). For the second image, the highest producers accuracy is for the
 6

snow/ice class as it was with the first image (Table 2). The classes with the highest users accuracy are

for the 2005-2011 clear-cut and alpine vegetation at 100% (Table 2).

Comparing the two images, the second one looks overall more accurate qualitatively even

though the overall accuracy quantitatively did not increase by that much. There are no longer areas

around Mt. Baker displayed as water, urban, or residential areas (Figure 2). However, much of the rivers

were incorrectly classified as shadow (Figure 2).

Table 2. The data from Figure 2, the Baker to Bay unsupervised classification image created using
ground truth training data, Google Earth, the color-IR image, and further refinement through ArcGIS
models are reported here. The image has 13 classes with data on how many pixels are in each class, the
area in square kilometers, the percent of the image that the class takes up, the producers accuracy, the
users accuracy, and the overall accuracy.
Percent of Producers Users
Class Information Class Pixels Area
Image Accuracy Accuracy
1 Residential 67965 42.5 1.8 51.2 51.2
2 Urban 27892 17.4 0.7 60.6 58.8
3 Pasture/Crops 794186 496.4 20.9 79 46.4
4 73-05 Clear-cut 861481 538.4 22.6 64 90.5
5 05-11 Clear-cut 18115 11.3 0.5 66.7 100
6 Deciduous Forest 346813 216.8 9.1 20.5 22.8
7 Coniferous Forest 1026363 641.5 27 43.4 62
8 Water 164651 102.9 4.3 78.3 88.7
9 Soil/Rock 187802 117.4 4.9 11.8 10.5
10 Alpine Vegetation 13677 8.5 0.4 33.3 100
11 Snow/Ice 249886 156.2 6.6 92.9 86.7
12 Cloud 14143 8.8 0.4 n/a n/a
13 Shadow 32226 20.1 0.8 n/a n/a
Overall Accuracy: 57.3%
 7

Legend
1: Residential
2: Urban
3: Pasture/Crops
4: 1973-2005 Clear-cut
5: 2005-2011 Clear-cut
6: Deciduous Forest
7: Coniferous Forest
8: Water
9: Soil/Rock
10: Alpine Vegetation
11: Snow/Ice
12: Cloud
13: Shadow

Figure 1. This image is the second image that is a result of unsupervised classification on the Baker to
Bay image from July of 2011. The spectral classes were initially assigned to 14 information classes using
ground truth training data, Google Earth, the color-IR image and then were further refined using ArcGIS
models, but that was collapsed down to 13 classes. The overall accuracy for this image is 57.3%.

Discussion

Assigning information classes to each of the fifty spectral classes in the ISODATA Unsupervised

Classification turned out to be much more difficult than I anticipated! The first few classes I assigned

were fairly straightforward because they were classes where the ground truth training data had the

majority of pixels for a class in a certain information class. For example, according to the ground truth

data, Class 1 (of the initial 50 spectral classes) had 20 pixels as water, one as deciduous, and nothing for

the rest. This class was very easy to assign as water. Once I got past these easy classes, I came to the

difficult task of assigning classes with conflicting or very limited ground truth data. For these classes I

had to rely on my color-IR image and Google Earth. This was quite challenging for some spectral classes

which were difficult to tell whether they were one information class or another. I struggled with Class 21

and 44 (both of the initial 50 spectral classes), which I eventually assigned to the information class of

alpine vegetation because there was hardly any ground truth data for them and when using Google
 8

Earth and the color-IR image, there was a mix of soil/rock and light vegetation. This was reflected in my

initial accuracy assessment because I had 0% users and producers accuracy for alpine vegetation.

One large challenge in assigning spectral classes to an information class was that there were

several spectral classes which had multiple information classes within them. One example of this is

between the classes of urban and soil/rocks. Both had similar reflectance values since most urban areas

are technically rock since they have a lot of concrete. This meant that many spectral classes had both

urban areas and soil/rock in them! Due to this, I had to use a combination of ground truth training data,

Google Earth, and the color-IR image to evaluate whether there seemed to be more urban areas or

soil/rock. One time that this occurred was for Class 29 (of the initial 50 spectral classes) which had

ground truth data reporting it as both urban and soil/rock. There was more ground truth data calling it

soil/rock and after using Google Earth to examine several locations listed as Class 29, I decided that

more of the class was soil/rock than urban areas.

Another similar issue was between coniferous forest, soil/rock, deciduous forest and the clear-

cuts. Old clear-cuts could look like coniferous forest if they have grown back significantly and new clear-

cuts could look like soil/rock. When looking at the ground truth data, there were several classes which

had similar amounts listed as a clear-cut and either deciduous forest or coniferous forest. However, Dr.

Wallin said that the people who had listed certain areas as a clear-cut were people whom he trusted to

correctly identify them. So, in cases where it was difficult to decide between a clear-cut and another

class (such as Class 8 and 9 of the initial 50 spectral classes), I assigned that spectral class as a clear-cut.

Class 10 (of the initial 50 spectral classes) was tricky because it had more ground truth listed as a clear-

cut between 1992 and 2005, but also many values listed as deciduous forest. As I did not have any other

spectral classes assigned to the deciduous forest information class, I went ahead and assigned Class 10

to deciduous forest.
 9

Because of the various challenges associated with the classification, this means that certain

information classes ended up being more variable than others. The urban and soil/rock classes were

quite variable because they both contained a mixture of spectral classes with both urban and soil/rock in

them. A similar variability occurred in all of the forest and clear-cut information classes. The information

class of water also had some variability because shadow was often classified as water.

After my initial accuracy assessment and the combination of the pasture and crops and the

1975-1992 and 1992-2005 clear-cuts, I had a lot of improvements to make! The ArcGIS models helped to

separate out some of the common issues by using distance to roads, elevation, and slope. However, if I

was going to redo this refinement, I probably would. I used the low, medium, and high elevations as set

according to Dr. Wallins methods, but doing it again, I would change those elevations and make the low

elevation even lower because I ended up with crops a further east than seemed appropriate (Wallin

2016). I also would alter the slope model. While the slope model turned the areas being displayed as

water on Mt. Baker into shadow, it also turned some rivers into shadow! This was because anything at a

slope above 0 was turned into shadow, but rivers are frequently at slopes above 0. So, doing it over, I

would probably change the slope threshold.

Comparing my initial and secondary accuracy assessments, I made good improvements, but it

was still lower than I would have liked. Dr. Wallins methods stated that an accuracy of above 70% is

considered good (Wallin 2016). So, why was my accuracy not up that high? One reason is probably

because I have never done unsupervised classification before. No one can expect ideal results on their

first try! Now that I have gone through the process, I believe that I could get a slightly better accuracy on

a second try. However, I still dont think I could get it very high because of the quality of the ground

truth data. The ground truth data was very helpful and useful, but could be problematic for several

reasons. One is that it has been collected by tons of different people with varying skill levels. This means

that there were some spectral classes listed primarily as coniferous forest for example, but were maybe
 10

actually a clear-cut because a random student walking upon a small conifer stand wouldnt necessarily

think that it had at one point been a clear cut. The other issue with the ground truth data is that there

were several information classes and areas which had little coverage. For example, there was not much

ground truth data on alpine vegetation because areas with it were less frequently visited by people

collecting the data. The same was true for other classes such as snow/ice. A final issue with the data is

that it has been collected over a large time period. I was attempting to classify an image from 2011, but

much of the data was from either before or after 2011. With improved ground truth data, it would be

possible to get a higher accuracy.

Another way to get higher accuracy would also involve a change in ground truth data. This

change would involve creating more information classes. With additional ground truth data, collected by

people who were trained in its collection, many of the information classes could be subdivided which

could improve accuracy by making it easier to assign classes. One such subdivision could be in the forest

classes. Maybe it could be subdivided into forest age, which could help in the distinction of clear-cuts

and forest. Another potential subdivision could be of the soil/rock class into different types of rock.

There are other subdivisions that could occur, but all of them would require more intense ground truth

data collection.

Up to this point, I mostly discussed my two images based on quantitative accuracy. But what

about visually? How do the images compare to one another? Even though the jump in quantitative

accuracy was not huge, the appearance of the second image is much better because of the ArcGIS

models. The first image was classifying much of Mt. Baker and the area surrounding it as water and

urban areas! The second image successfully changed those areas to shadow and soil/rock. The main

issue is that the rivers were classified as shadow, which I discussed earlier.
 11

The technique of unsupervised classification is an extremely useful one as it allows you to

classify a given image based on ground truth data, Google Earth, ArcGIS models, and other techniques to

give a representation of the land cover and land use of a certain area. If I were to perform this process

again, I would hopefully have improved ground truth data to increase my accuracy and would change

some of the ArcGIS models in terms of the elevation cut-offs and the slope for the cutoff between water

and shadow.

Literature Cited

Wallin, D. 2016. Lab III: Unsupervised Classification with ENVI.

http://faculty.wwu.edu/wallin/envr442/ENVI/442_unsup_class_ENVI.html