Basics of digital image processing

Lecture 4 September 23, 2006

What is image processing

Is enhancing an image or extracting information or features from an image Computerized routines for information extraction (eg, pattern recognition, classification) from remotely sensed images to obtain categories of information about specific features. Many more

Image Processing Includes

Image quality and statistical evaluation Radiometric correction Geometric correction Image enhancement and sharpening Image classification Accuracy assessment of classification Post-classification and GIS Change detection
Pixel based Object-oriented based

Image Quality
Many remote sensing datasets contain high-quality, accurate data. Unfortunately, sometimes error (or noise) is introduced into the remote sensor data by: the environment (e.g., atmospheric scattering, cloud), random or systematic malfunction of the remote sensing system (e.g., an uncalibrated detector creates striping), or improper pre-processing of the remote sensor data prior to actual data analysis (e.g., inaccurate analog-to-digital conversion).

154 Cloud 155 160

155

162 MODIS True 143

163

164

Clouds in ETM+

Striping Noise and Removal

CPCA
Combined Principle Component Analysis

Xie et al. 2004

Speckle Noise and Removal

Blurred objects and boundary

G-MAP
Gamma Maximum A Posteriori Filter

Univariate descriptive image statistics

The mode is the value that occurs most frequently in a distribution and is usually the highest point on the curve (histogram). It is common, however, to encounter more than one mode in a remote sensing dataset. The median is the value midway in the frequency distribution. One-half of the area below the distribution curve is to the right of the median, and one-half is to the left The mean is the arithmetic average and is defined as the sum of all brightness value observations divided by the number of observations.

k =

BV
i =1

ik

Cont
Min Max Variance Standard deviation Coefficient of variation (CV) Skewness Kurtosis Moment

vark =

(BV
i =1

ik

k )

n 1

sk = k = vark
CV

k k

Multivariate Image Statistics

Remote sensing research is often concerned with the measurement of how much radiant flux is reflected or emitted from an object in more than one band. It is useful to compute multivariate statistical measures such as covariance and correlation among the several bands to determine how the measurements covary. Variancecovariance and correlation matrices are used in remote sensing principal components analysis (PCA), feature selection, classification and accuracy assessment.

Covariance
The different remote-sensing-derived spectral measurements for each pixel often change together in some predictable fashion. If there is no relationship between the brightness value in one band and that of another for a given pixel, the values are mutually independent; that is, an increase or decrease in one bands brightness value is not accompanied by a predictable change in another bands brightness value. Because spectral measurements of individual pixels may not be independent, some measure of their mutual interaction is needed. This measure, called the covariance, is the joint variation of two variables about their common mean.
SPkl = (BVik BVil )
i =1 n

BV BV
i =1 ik i =1

il

SPkl cov kl = n 1

Correlation
To estimate the degree of interrelation between variables in aamanner not To estimate the degree of interrelation between variables in manner not influenced by measurement units, the correlation coefficient, is influenced by measurement units, the correlation coefficient, is commonly used. The correlation between two bands of remotely sensed commonly used. The correlation between two bands of remotely sensed data, rr ,,is the ratio of their covariance (covkl))to the product of their data, kl is the ratio of their covariance (covkl to the product of their kl standard deviations (skss); thus: standard deviations (sk l l); thus:

cov kl rkl = sk sl
If we square the correlation coefficient (rkl), we obtain the sample coefficient of If we square the correlation coefficient (rkl), we obtain the sample coefficient of determination (r22),which expresses the proportion of the total variation in the values of determination (r), which expresses the proportion of the total variation in the values of band l that can be accounted for or explained by aalinear relationship with the values band l that can be accounted for or explained by linear relationship with the values of the random variable band k. Thus aacorrelation coefficient (rkl))of 0.70 results in an of the random variable band k. Thus correlation coefficient (rkl of 0.70 results in an 2 rr2value of 0.49, meaning that 49% of the total variation of the values of band l in the value of 0.49, meaning that 49% of the total variation of the values of band l in the sample is accounted for by aalinear relationship with values of band k. sample is accounted for by linear relationship with values of band k.

Pixel (1,1) (1,2) (1,3) (1,4) (1,5)

Band 1 (green) 130 165 100 135 145

Band 2 (red) 57 35 25 50 65

Band 3 (ni) 180 215 135 200 205

Band 4 (ni) 205 255 195 220 235

example
SP 12

(675)(232) = (31,860)
5

540 cov12 = = 135 4

Band 2 57 35 25 50 65 232

Band 1 130 165 100 135 145 675

(Band 1 x Band 2) 7,410 5,775 2,500 6,750 9,425 31,860

Band 1 Mean (k) Variance (vark) (s k ) (mink) (maxk) Range (BVr) 135 562.50 23.71 100 165 65

Band 2 46.40 264.80 16.27 25 65 40

Band 3 187 1007 31.4 135 215 80

Band 4 222 570 23.87 195 255 60

Univariate statistics

Band 1 Band 1 562.25 Band 2 Band 3 Band 4 135 718.75 537.50

Band 2 264.8 0

Band 3 -

Band 4 570 Band 1

Band 1

Band Band 3 Band 2 4 -

Band 2 0.35 Band 3 0.95 Band 4 0.94

275.25 1007.5 0 64 663.75

0.53 covariance 0.16 0.87 -

Covariance

Correlation coefficient

Types of radiometric correction

Detector error or sensor error (internal error) Atmospheric error (external error) Topographic error (external error)

Atmospheric correction
Various Paths of Satellite Received Radiance

There are several ways to atmospherically correct remotely sensed data. Some are relatively straightforward while others are complex, being founded on physical principles and requiring a significant amount of information to function properly. This discussion will focus on two major types of atmospheric correction:
Absolute atmospheric correction, and Relative atmospheric correction.

Total radiance L S at the sensor Solar irradiance E

0

Remote sensor detector

90

Lp

LT

2 Diffuse sky irradiance Ed

60 miles or 100km
Atmosphere

1,3,5

v
3

0
5

LI

Scattering, Absorption Refraction, Reflection

Reflectance from neighboring area,

Reflectance from study area,

Absolute atmospheric correction

Solar radiation is largely unaffected as it travels through the vacuum of space. When it interacts with the Earths atmosphere, however, it is selectively scattered and absorbed. The sum of these two forms of energy loss is called atmospheric attenuation. Atmospheric attenuation may 1) make it difficult to relate handheld in situ spectroradiometer measurements with remote measurements, 2) make it difficult to extend spectral signatures through space and time, and (3) have an impact on classification accuracy within a scene if atmospheric attenuation varies significantly throughout the image. The general goal of absolute radiometric correction is to turn the digital brightness values (or DN) recorded by a remote sensing system into scaled surface reflectance values. These values can then be compared or used in conjunction with scaled surface reflectance values obtained anywhere else on the planet.

a) Image containing substantial haze prior to atmospheric correction. b) Image after a) Image containing substantial haze prior to atmospheric correction. b) Image after atmospheric correction using ATCOR (Courtesy Leica Geosystems and DLR, the atmospheric correction using ATCOR (Courtesy Leica Geosystems and DLR, the German Aerospace Centre). German Aerospace Centre).

relative radiometric correction

When required data is not available for absolute radiometric correction, we can do relative radiometric correction Relative radiometric correction may be used to
Single-image normalization using histogram adjustment Multiple-data image normalization using regression

Single-image normalization using histogram adjustment

The method is based on the fact that infrared data (>0.7 m) is free of atmospheric scattering effects, whereas the visible region (0.4-0.7 m) is strongly influenced by them. Use Dark Subtract to apply atmospheric scattering corrections to the image data. The digital number to subtract from each band can be either the band minimum, an average based upon a user defined region of interest, or a specific value

Dark Subtract using band minimum

Topographic correction
Topographic slope and aspect also introduce radiometric distortion (for example, areas in shadow) The goal of a slope-aspect correction is to remove topographically induced illumination variation so that two objects having the same reflectance properties show the same brightness value (or DN) in the image despite their different orientation to the Suns position Based on DEM, sun-elevation

Conceptions of geometric correction

Geocoding: geographical referencing Registration: geographically or nongeographically (no coordination system)

Image to Map (or Ground Geocorrection) Image to Image Geocorrection

The correction of digital images to ground coordinates using ground control points collected from maps (Topographic map, DLG) or ground GPS points.

Image to Image correction involves matching the coordinate systems or column and row systems of two digital images with one image acting as a reference image and the other as the image to be rectified. Spatial interpolation: from input position to output position or coordinates. RST (rotation, scale, and transformation), Polynomial, Triangulation Root Mean Square Error (RMS): The RMS is the error term used to determine the accuracy of the transformation from one system to another. It is the difference between the desired output coordinate for a GCP and the actual. Intensity (or pixel value) interpolation (also called resampling): The process of extrapolating data values to a new grid, and is the step in rectifying an image that calculates pixel values for the rectified grid from the original data grid. Nearest neighbor, Bilinear, Cubic

Image enhancement
image reduction, image magnification, transect extraction, contrast adjustments (linear and non-linear), band ratioing, spatial filtering, fourier transformations, principle components analysis, texture transformations, and image sharpening

Contrast Enhancement (stretch)

Materials or objects reflect or emit similar amounts of radiant flux (so similar pixel value) Low-contrast imagery with pixel range less than the designed radiometric range To improve the contrast:
Linear technique 20-100 for TM less than the designed 0-255
Minimum-maximum contrast stretch Percentage linear contrast stretch Standard deviation contrast stretch Piecewise linear contrast stretch Histogram equalization

Non-linear technique

Contrast enhancement is only intended to improve the visual quality of a displayed image by increasing the range (spreading or stretching) of data values to occupy the available image display range (usually 0-255). It does not change the pixel values, unless save it as a new image. It is not good practice to use saved image for classification and change detection.

Minimum-maximum contrast stretch

BVin min k = max min k k quant k

BVout

where: where: --BVin is the original input brightness value BVin is the original input brightness value --quantk is the range of the brightness values that can be quantk is the range of the brightness values that can be displayed on the CRT (e.g., 255), displayed on the CRT (e.g., 255), --mink is the minimum value in the image, mink is the minimum value in the image, --maxk is the maximum value in the image, and maxk is the maximum value in the image, and --BVout is the output brightness value BVout is the output brightness value

Percentage linear and standard deviation contrast stretch

X percentage (say 5%) top or low values of the image will be set to 0 or 255, rest of values will be linearly stretched to 0 to 255 ENVI has a default of a 2% linear stretch applied to each image band, meaning the bottom and top 2% of image values are excluded by positioning the range bars at the appropriate points. Low 2% and top 2% will be saturated to 0 and 255, respectively. The values between the range bars are then stretched linearly between 0 and 255 resulting in a new image. If the percentage coincides with a standard deviation percentage, then it is called a standard deviation contrast stretch. For a normal distribution, 68%, 95.4%, 99.73% values lie in 1, 2 , 3 . So 16% linear contrast stretch is the 1 contrast stretch.

original

Saturating the water Stretching the land

Saturating the land Stretching the water

Special linear contrast stretch Or Stretch on demand

Piecewise linear contrast stretch

When the histogram of an image is not Gaussian (bimodal, trimodal, ), it is possible to apply a piecewise linear contrast stretch. But you better to know what each mode in the histogram represents in the real world.

Stretch both land and water

Principle Components Analysis (PCA)

contains the largest variance
There are large correlations among remote sensing bands. PCA will result in another uncorrelated datasets: principal component images (PCs). PC1 The first two or three components (PCs) contain over 90% of information from the original many bands. It is a great compress operation The new principal component images that may be more interpretable than the original data.

Purposes of image classification

Land use and land cover (LULC) Vegetation types Geologic terrains Mineral exploration Alteration mapping .

What is image classification or pattern recognition

Is a process of classifying multispectral (hyperspectral) images into patterns of varying gray or assigned colors that represent either

data and must be identified and then labeled by the analyst.

Spectral classes are those that are inherent in the remote sensor Information classes are those that human beings define.

clusters of statistically different sets of multiband data, some of which can be correlated with separable classes/features/materials. This is the result of Unsupervised Classification, or numerical discriminators composed of these sets of data that have been grouped and specified by associating each with a particular class, etc. whose identity is known independently and which has representative areas (training sites) within the image where that class is located. This is the result of Supervised Classification.

unsupervised classification, The computer or algorithm automatically group pixels with similar spectral characteristics (means, standard deviations, covariance matrices, correlation matrices, etc.) into unique clusters according to some statistically determined criteria. The analyst then re-labels and combines the spectral clusters into information classes.

supervised classification. Identify known a priori through a combination of fieldwork, map analysis, and personal experience as training sites; the spectral characteristics of these sites are used to train the classification algorithm for eventual land-cover mapping of the remainder of the image. Every pixel both within and outside the training sites is then evaluated and assigned to the class of which it has the highest likelihood of being a member.

Hard vs. Fuzzy classification

Supervised and unsupervised classification algorithms typically use hard classification logic to produce a classification map that consists of hard, discrete categories (e.g., forest, agriculture). Conversely, it is also possible to use fuzzy set classification logic, which takes into account the heterogeneous and imprecise nature (mix pixels) of the real world. Proportion of the m classes within a pixel (e.g., 10% bare soil, 10% shrub, 80% forest). Fuzzy classification schemes are not currently standardized.

Pixel-based vs. Object-oriented classification

In the past, most digital image classification was based on processing the entire scene pixel by pixel. This is commonly referred to as per-pixel (pixel-based) classification.

Object-oriented classification techniques allow the

analyst to decompose the scene into many relatively homogenous image objects (referred to as patches or segments) using a multi-resolution image segmentation process. The various statistical characteristics of these homogeneous image objects in the scene are then subjected to traditional statistical or fuzzy logic classification. Objectoriented classification based on image segmentation is often used for the analysis of high-spatial-resolution imagery (e.g., 1 1 m Space Imaging IKONOS and 0.61 0.61 m Digital Globe QuickBird).

Unsupervised classification
Uses statistical techniques to group n-dimensional data into their natural spectral clusters, and uses the iterative procedures label certain clusters as specific information classes K-mean and ISODATA

For the first iteration arbitrary starting values (i.e., the cluster properties) have to be selected. These initial values can influence the outcome of the classification. In general, both methods assign first arbitrary initial cluster values. The second step classifies each pixel to the closest cluster. In the third step the new cluster mean vectors are calculated based on all the pixels in one cluster. The second and third steps are repeated until the "change" between the iteration is small. The "change" can be defined in several different ways, either by measuring the distances of the mean cluster vector have changed from one iteration to another or by the percentage of pixels that have changed between iterations. The ISODATA algorithm has some further refinements by splitting and merging of clusters. Clusters are merged if either the number of members (pixel) in a cluster is less than a certain threshold or if the centers of two clusters are closer than a certain threshold. Clusters are split into two different clusters if the cluster standard deviation exceeds a predefined value and the number of members (pixels) is twice the threshold for the minimum number of members.

Supervised classification: training sites selection

Based on known a priori through a combination of fieldwork, map analysis, and personal experience on-screen selection of polygonal training data (ROI), and/or on-screen seeding of training data (ENVI does not have this, Erdas Imagine does). The seed program begins at a single x, y location and evaluates

neighboring pixel values in all bands of interest. Using criteria specified by the analyst, the seed algorithm expands outward like an amoeba as long as it finds pixels with spectral characteristics similar to the original seed pixel. This is a very effective way of collecting homogeneous training information.

From spectral library of field measurements

Selecting ROIs

Alfalfa Cotton Grass Fallow

Supervised classification methods

Various supervised classification algorithms may be used to assign an unknown pixel to one of m possible classes. The choice of a particular classifier or decision rule depends on the nature of the input data and the desired output. Parametric classification algorithms assumes that the observed measurement vectors Xc obtained for each class in each spectral band during the training phase of the supervised classification are Gaussian; that is, they are normally distributed. Nonparametric classification algorithms make no such assumption. Several widely adopted nonparametric classification algorithms include: one-dimensional density slicing parallepiped, minimum distance, nearest-neighbor, and neural network and expert system analysis. The most widely adopted parametric classification algorithms is the: maximum likelihood. Hyperspectral classification methods Binary Encoding Spectral Angle Mapper Matched Filtering Spectral Feature Fitting Linear Spectral Unmixing

Supervised classification method:

Spectral Feature Fitting

Source: http://popo.jpl.nasa .gov/html/data.html

Accuracy assessment of classification

Remote sensing-derived thematic information are becoming increasingly important. Unfortunately, they contain errors. Errors come from 5 sources:
Geometric error still there None of atmospheric correction is perfect Clusters incorrectly labeled after unsupervised classification Training sites incorrectly labeled before supervised classification None of classification method is perfect

We should identify the sources of the error, minimize it, do accuracy assessment, create metadata before being used in scientific investigations and policy decisions. We usually need GIS layers to assist our classification.

training vs. ground reference

Several ways to do error evaluation
Based on training pixels (areas)
The problem is that the locations of training sites are usually not random. They are biased by analysts a priori knowledge of where certain LULC types exist in the scene. This will results in higher classification accuracies than the one below These sites are not used to train the classification algorithm and therefore represent unbiased reference information It is possible to collect some ground sites prior to the classification, perhaps at the same time as the training data But majority of test reference is often collected after classification.

Based on ground reference pixels

Landscape often change rapidly. Therefore, it is best to collect both the training and ground reference as close to the data of remote sensing data acquisition as possible. (for example, agriculture crops change fast)

Error (Confusion) Matrix

Producer (analyst) accuracy is a measure indicating the probability that the classifier has labeled an image pixel into Class A given that the ground truth is Class A. it is the probability of a reference pixel being correctly classified. Omission error represent pixels that belong to the ground truth class but that the classification technique has failed to classify them into the proper class. User accuracy is a measure indicating the probability that a pixel is Class A given that the classifier has labeled the pixel into Class A. it is the probability that a pixel classified on the map actually represents that category on the ground. Commission error represent pixels that belong to another class but are labeled as belonging to the class. Overall accuracy is total classification accuracy. Kappa coefficient (Khat) is a discrete multivariate technique of use in accuracy assessment. Khat>80% represent strong agreement and good accuracy. 40%-80% is middle, <40% is poor.

Example: they took 407 samples (pixels) based on the stratified random sampling after classification. First made 5 files (each contain one class), using a random number generator to get points.

Post-classification and GIS

saltandpepper

types
Majority/Minority Analysis Clump Classes Morphology Filters Sieve Classes Combine Classes Classification to vector (GIS)

Change detection
Change detect involves the use of multi-temporal datasets to discriminate areas of land cover change between dates of imaging. Ideally, it requires
Same or similar sensor, resolution, viewing geometry, spectral bands, radiomatric resolution, acquisition time of data, and anniversary dates Accurate spatial registration (less than 0.5 pixel error)

Methods

Independently classified and registered, then compare them Classification of combined multi-temporal datasets, Principal components analysis of combined multi-temporal datasets Image differencing (subtracting), (needs to find change/no change threshold, change area will be in the tails of the histogram distribution) Image ratioing (dividing), (needs to find change/no change threshold, change area will be in the tails of the histogram distribution) Change vector analysis Delta transformation

Example: stages of development

Sun City Sun City Hilton Head Hilton Head

1994 1994

1996 1996

1974 1,040 urban hectares

1994 3,263 urban hectares 315% increase