Introduction to Field Spectroscopy

NERC Field Spectroscopy Facility Course Handbook

COST Action ES0903 Monte Bondone, Italy

7th to 9th July 2011

NERC FSF Introduction to Field Spectroscopy Course Alasdair Mac Arthur School of GeoSciences, University of Edinburgh
7th to 9th July 2011 Programme Thursday, 7th of July 2011 14:30 15:00 Coffee and introductions -

15.00-16:15 - Introduction - The role of field spectroscopy in research 16.15 16:30 16:30 18:00 Break The principles of field spectroscopy

19.00 Dinner at Hotel Montana Friday, 8th of July 2011 8:00 9:30 9:30 10:15 10:15 10.30 10:30 12:30 The design and calibration of spectroradiometers Measurement in the laboratory and in the field (theory) Break Practical introduction to spectroradiometers (demo of bench top/laboratory spectroscopy measurements) (1 x assistant required)

12:30 Lunch at Hotel Montana

14:00 15:15

Sampling design and measurement uncertainty

15:15 16:00

Introduction to Measurements in the environment spectroradiometers and sun photometers. Outside but adjacent to training facility (1 x assistant required) Break

16:00 16:15

16:15 18:00

The processing and analysis of spectral datasets Part 1

Saturday, 9th of July 2011

8:00 10:15 10:15- 10:30

The processing and analysis of spectral datasets Part 2 & Conclusions Break

Field Spectroscopy Facility Introduction to Field Spectroscopy Session 1: The role of field spectroscopy in research
Alasdair Mac Arthur
Iain Woodhouse, Director Chris MacLellan, Equipment Manager Alasdair Mac Arthur, Operations Manager

Who are you?

Time for introductions

Structure of the course

Two day introduction to field spectroscopy Assumes you already have a background knowledge of the physical principles of remote sensing Lecture sessions, demonstrations, and a bit of hands on Not designed to be majorly practical Weather permitting field work Sunday and Monday

Timetable for the course Day 1

15.00-16:15 Introduction - The role of field spectroscopy in research Coffee break The principles of field spectroscopy Dinner at Hotel Montana 16.15 16:30 16:30 18:00 19.00

Timetable for the course Day 2

8:00 9:30 The design and calibration of spectroradiometers 9:30 10:15 Measurement in the laboratory and in the field (theory) 10:15 10.30 Coffee break 10:30 12:30 Practical introduction to spectroradiometers 12:30 14:00 15:15 15:15 16:00 16:00 16:15 16:15 18:00 Lunch at Hotel Montana Sampling design and measurement uncertainty Introduction to Measurements in the environment Outside if possible Coffee break The processing and analysis of spectral datasets Part 1

Timetable for the course Day 3

8:00 10:15 The processing and analysis of spectral datasets Part 2 & Conclusions Coffee break 10:15- 10:30

Session 1 The role of field spectroscopy in research

Structure of the lecture

What is field spectroscopy? Whats it used for? What kind of research questions can it address? What its basis? Why a Field Spectroscopy Facility and what does FSF do? The case for and the challenges of the hyperspectral domain Conclusions

What is field spectroscopy?

Definition: the quantitative measurement of radiance, irradiance, reflectance or transmission in the field

Definitions
Hyper: meaning many, over-many Contiguous: Next in space, immediately successive, neighbouring, situated in close proximity Continuous: having no breaks, unbroken, uninterrupted in sequence Continuous spectrum: a spectrum not broken by bands or lines

Definitions
Hyperspectral sensing therefore represents an extension and natural evolution of the concept of multispectral sensing, to sensing in an increased number of discrete contiguous bands (representing contiguous measurement of the optical spectrum) But, when does a sensor cease being MS and become HS?

What is spectroscopy used for?

To make direct measurements of reflectances in the field To convert image-based measurements of reflected radiance to calibrated radiance or reflectance (calibration of air- and spaceborne images) For validation of satellite/airborne measurements To better understand the nature of the interaction of ER with earth surface objects To improve the quantitative determination of earth surface objects To provide data for input into radiative transfer models To build spectral libraries

Applications

35 30

p1 p4 p5 p8 p9

Reflectance (%)

25 20 15 10 5 0 400

900

1400

1900

2400

Wavelength (nm)

Applications

Applications
Atmospheric science (spatial and temporal variation in atmospheric constituents) Water quality (chlorophyll estimation, CDOM) Ecology (vegetation biomass, physiology, productivity) Geology (surface mineral identification, mapping, geomorphic mapping)

What research questions can FS address?

What is the optimum spectral resolution required for detection and characterisation of a target? What are the optimum number of bands required to characterise a target given a range in biophysical parameters? What are the optimum algorithms what methods of analysis can be applied? What spatial resolution is required? To what extent does the signal vary with time? What is the best time of year/day? What signal to noise ratio is required?

The basis of field spectroscopy

Fundamentally physically based quantitative objective Replicable or at least it should be! The topic of the next lecture

Spectroradiometers
Portable instruments for the measurement of the optical characteristics of earth surface targets For use in laboratory or the field Data are recorded for a single sample unit Ability to control acquisition parameters, particularly in an experimental design more so than for airborne or spaceborne hyperspectral imaging

Sunphotometers
Portable instruments for the measurement of atmospheric optical properties For use in the field For: Atmospheric correction of image datasets Characterisation of atmospheric particulate components, for modelling

Instrumentation @ FSF
4 ASD FieldSpec Pros 1 SVC HR-1024 3 FSF VSWIRs 4 GER 1500s 6 MicroTops sunphotometers 2 CIMEL sunphotometer Calibration facilities and maintenance of standards to National Physical Laboratory standards

GRASS
System for estimating BRDF with bespoke VSWIR spectroradiometer

FTIR
Fourier Transform infrared (FTIR) instrument, new instrument now deployed Measures over 0.2 to 15 m

Accessories
A range of fore optics for different fields of view radiance; irradiance and reflectance Contact probes A range of field reference panels Tripods and mounts Light meters Stabilised power supplies and lamps GPSs Laptop and Toughbook computers

Discrete spectral bands The case for and the challenges of the hyperspectral domain
Landsat TM data 6 optical, 1 thermal ~ limiting?

1 2 3

4 5 7

6 (thermal)

Multispectral sensor
TM Spectral Bands
80 70 12 3 4 5 7

The spectral signature

50 40 Reflectance (%) 90% cover vegetation 50% cover vegetation Bare light soil Bare dark peak Clear lake water

Reflectance

60 50 40 30 20 10 0 400 700 1000 1300 1600 1900 2200 2500

30

20

10

0 400

600

800 1000 1200 1400 1600 1800 2000 2200 2400 Wavelength (nm)

Wavelenth (nm)

Discrete numbers of spectral bands Loss (or overlooking) of potentially useful information

MS is limiting when we want to exploit the more subtle differences between (often spectrally similar) earth surface objects

Limitations of the MS approach

Crude spectral characterization of the reflectance properties of earth surface objects using multispectral approaches Loss of potentially useful information in regions overlooked by the bands Loss of more subtle information contained in the fluctuations in the signature curve Doesnt allow us to better define something about the object of interest The limited information in multispectral imaging systems potentially limits classification

The argument for hyperspectral

Improves characterisation of the Earth's surface Provides more detailed information Can filter data to match any lower spectral resolution sensor Identify spectral features too small to be detected by multispectral sensors Hypothesized that higher resolution spectral information will lead to better characterization or classification of Earth surfaces

The argument for hyperspectral

Analysis Use of alternative techniques for data analysis (beyond the spectral ratio) Alternative processing techniques which identify features and materials through measurement of spectral absorption features Draws on concepts developed in laboratory spectral analysis (from chemistry), e.g.:
Spectroscopy Gas chromatography Liquid chromatography

The argument for hyperspectral

Improved understanding of: Earth surface reflectances / spectral signatures Interaction of light with earth surface objects Use of radiative transfer models can help us better understand interaction of ER & Earth surfaces and develop improved techniques for analysis

The argument for hyperspectral

Pushes us away from traditional approaches, E.g. image analysis limitations in 3-band colour display But complexity of the data complicates the issue of analysis Data rich

A rich dataset
Requires techniques for data reduction Allows new methods of data exploration not available in datasets covering fewer bands
35 30
p1 p4 p5 p8 p9

Reflectance (%)

25 20 15 10 5 0 400

900

1400

1900

2400

Wavelength (nm)

Conclusions
Field spectroscopy is a key component in hyperspectral (and MS) remote sensing underpins RS from aircraft and satellites Advantages Full exploitation of the spectral domain RT modelling Enhanced ability for spectral discrimination Chemical and molecular analyses Physiological and biogeochemical analyses of vegetation Chemical and mineralogic analyses of water Geological and mineralogical analyses

Conclusions
Disadvantages In the field may rely on the Sun Difficult to do well- easy to do badly! Processing requirements Cost Difficulty of analysis analysis in its infancy!

Structure of the lecture Session 2: The principles of field spectroscopy

Alasdair Mac Arthur
Radiation geometry BRDF and goniometers Measurement modes Reference measurements Reference panels Reflectance Conclusions

Incident illumination
Solar illumination contributes direct and diffuse fractions Solar elevation and atmospheric properties will affect the overall intensity and spectral characteristics of direct illumination On a clear day diffuse illumination can still contribute up to 10-20% of the total incident irradiance & wavelength dependent (20-30% in the blue) Scatter from surroundings may be important

Targets
Illumination consists of both direct (solar flux) and diffuse (scattered) elements Natural targets are illuminated by the whole hemisphere of the sky A proportion of that incident radiation is reflected depending on the target properties Because targets are generally not perfectly diffuse (Lambertian) reflectors, the intensity of the reflected flux varies with angle Thus, the radiation environment comprises both incoming and outgoing hemispherical distributions of electromagnetic radiation

From the literature - the possible measurements definitions

Zenith and azimuth

The positions of the Sun and the detector are defined by two angles: the angle from the vertical (the zenith angle, ) the angle measured in the horizontal plane from a reference direction (the azimuth angle, )

Of the 9 quantities defined by Nicodemus, 4 can be approximated(?) but only 2 are useful

SchaepmanSchaepman-Strub et al. (2006) adapted from Nicodemus (1977)

10

Geometry

Theory
BRDF - In direct illumination, incident and reflected can be regarded as confined to two slender elongated cones. If the solid angles of the cones (measured in steradians, sr), are infinitesimally small, the reflectance of the target can be defined as a function:
f ( i , i , r , r ) = dL( r , r ) dE ( i , i )

dL is the reflected radiance per unit solid angle dE is the irradiance per unit solid angle i and r denote incident and reflected rays

BRDF
To specify completely the reflectance must be measured at all possible source/sensor positions = Bidirectional Reflectance Distribution Function The fundamental physical property governing reflectance behaviour from the surface (Nicodemus 1982) A theoretical concept: not possible to measure in practice - we estimate it Commonly simplified to bidirectional reflectance factor (BRF) but BRF describes reflectance for parallel beams of radiance and irradiance To relate BRF to BRDF involves a number of assumptions not well understood

BRDF & BRF

This is for a point source

HDRF

(sometimes referred to as HCRF)

iHDRF
We measure an integrated reflectance or radiance from a surface area defined by the directional response function (DRF) of the spectroradiometer DRF commonly considered to be FOV = erroneous The response of spectroradiometer to photons has both directional and wavelength dependencies Caveat emptor if you purchase a spectroradiometer and believe the manufacturers specified FOV!

We need to consider an irradiance hemisphere (direct + diffuse) Also need to consider the direction of reflectance or radiance Leads to HDRF (R ) hemispherical directional reflectance factor

L is the reflected radiance per unit solid angle Eo is the direct and Ed diffuse irradiance per unit solid angle Note introduction of should have been included in all previous functions Measurement of light always has wavelength dependencies but not there yet .

integral - area x to x by y to y is the spectrometers directional and wavelength dependent response function (DRF)
(x,y) x,y)

11

iHDRF
Is determined by the DRF of the spectroradiometer/fore optic and the structural and optical properties of the surface
shadow-casting multiple scattering transmission, absorption and emission by surface elements transmission, absorption and emission by surface body facet orientation distribution and facet density

Why is (i)HDRF important?

Needed for:
Correction of view and illumination angle effects on images Deriving albedo Land cover classification Atmospheric correction To attribute reflecting components to gross reflectance recorded

Gives a boundary condition for any radiative transfer problem and hence its relevance for climate modelling and energy budget investigations

Directional reflectance illustrated

Why is (i)HDRF important?

Black spruce forward and back scattering

Bare soil forward and back scattering

Measurement of iHDRF

Visualising iHDRF
In principal plane The principal plane is the plane of illumination

Polar plots

FIGOS

12

FSF GRASS instrument

Gonio Radiometric Spectrometer System Currently in development at NPL 30 degree intervals around hemisphere on up to six arms, each with 5-6 collecting optics Fibre optics based, feeding through a multisplexer to a spectroradiometer Very rapid measurement (~30min per hemisphere!)

Two practical measurement geometries in the field

Cos-conical method Bi-conical method

Cos-conical method
Upward-looking spectrometer with a cosinecorrected receptor is used (measures irradiance) Is not dependent upon the zenith or azimuth angle of the incident flux Often used in dual mode

Cos-conical method
Reflectance
R ( i , i , r , r ) = dLt ( r , r ) k ( i , i , r , r ) dE

where dE is the irradiation as measured by the upward-looking cosine sensor k is correction factor relating cosine receptor to a perfectly diffuse white panel N.B. All other iHDRF functions omitted for brevity

What does a cosine head do?

The flux of a beam of light at an oblique angle delivers fewer photons per m2 than a beam perpendicular to the surface:

White diffuser
At low angles some light is reflected, causing a lower reading than reality To correct for this, sensors are enclosed in a black cylinder with a raised, small plastic diffuser on top This is called a cosine corrected head or remote cosine receptor (RCR)

The ratio of the flux densities of the two beams is the cosine of the angle of the oblique beam A sensor should respond to oblique beams with this ratio One that does is said to give a cosine response

13

White diffuser
The plastic used should: Have a low reflectance at low angles such that the cosine error is small Be spectrally invariant Because there are errors is why its corrected to a panel measurement

Responses
Plot of instrument angular response cf. the ideal: but this is only for PAR!

Bi-conical method
Compares target radiance to that from a diffuse, perfectly reflecting reference panel The target and reference are viewed under the same irradiation conditions and geometry

Bi-conical method
R( i , i , r , r ) = dLt ( r , r ) k ( i , i , r , r ) dL p ( i , i )

dLt is the radiance of the target dLp is the radiance of the panel under the same conditions k is the panel correction factor N.B. iHDRF functions omitted for brevity

Reference panels
Spectralon (Labsphere) and Ultralon thermoplastic PTFEbased resin supplied in flat panel form. Have high and flat reflectance over the optical range when clean! >99% over 400 to 1500 nm, >95% over 1500 to 2500 nm Assumed to be Lambertian Chemically inert (relatively), highly water resistant and thermally stable Durable and stable and has a surface that maintains reproducable reflectances under field conditions (well, thats what the advertising says)

Reflectance of Spectralon

14

Spectralon panels
Do have non-Lambertian reflectance
properties with respect to global radiation at very large solar zenith angles (above 60) Sun-angle correction factors available from panel manufacturers but !

FSF measured cosine responses

Results from work by C. MacLellan of FSF

Spectralon correction factors

Assumptions in reflectance field spectroscopy

Sensor field-of-view < ~20 degrees The panel must fill the sensor field-of-view There is no change in incident irradiation amount or its spectral distribution Direct solar flux dominates the irradiation field The sensor responds in a linear fashion to changes in radiant flux The reflectance properties of the standard panel are known and invariant over the course of the measurements The sensor is sufficiently distant from the target

What does it all mean?

Two measurements required: Target radiance Incident irradiance (cosine or panel) Measurements must be made simultaneously or as close in time as practical FOV < 20 Made under clear sky conditions The sensor is calibrated The panel is calibrated Height above target is important

Reflectance
Reflectance = Target/Reference Calculation of reflectance cancels out multiplicative effects such as: Spectral irradiance of the illumination source Optical throughput of the field spectrometer But assumes characteristics of the illumination were the same for the target and reference measurements if they were made at different times (sequentially)

15

Errors
In sequential measurements of target and reference up to 100% (Duggin 1981) Simultaneous measurement is better Presence of nearby objects (given diffuse component of radiance) Up to 18% with different clothing (Kimes et al. 1983)

Sun photometry
Narrow field-of-view radiometers designed to measure solar irradiance <2.5 FOV Pointed at sun, measurement is dominated by the solar disk Measure in a distinct number of spectral bands Used to derive atmospheric parameters:
Aerosol optical depth Precipitable water Sky radiance distributions Ozone concentration

Atmospheric irradiance

FSF sun photometers

MicroTops II Cimel CE318

Conclusions
FS is well grounded in theory Confusing terminologies/lack of standardisation Theoretical considerations must also be considered in practical application of FS Much still to be understood but the science advances

16

Structure of the lecture Session 3: Instrument design and calibration

Alasdair Mac Arthur Inside the black box Key instrument components Key issues and specifications Examples of field spectroradiometers Spectroradiometric calibration Conclusion

Whats inside a spectroradiometer?

Key components inside the box

{ Diffraction gratings { Detector arrays { Entrance slits Together, these components can make up whats known as a spectrograph, an instrument used to separate and measure the relative amounts of radiation in each wavelength in electromagnetic radiation + Beam splitters

GER 3700

Diffraction gratings
An optical element, which separates (disperses) polychromatic light into its constituent wavelengths (colours) Realized as fine parallel and equally spaced grooves on a surface When light hits a diffraction grating, diffractive and mutual interference effects occur, and light is reflected or transmitted in discrete directions, called diffraction orders

Diffraction Gratings
Exit Slit

Grating equation: d(sin + sin ) = m , where d is distance between adjacent grooves is angle of incidence is angle of diffraction m is order (-3rd to +3rd) is wavelength

Entrance Slit

17

Diffraction Gratings
Zero, first & second order spectra

Detectors
Made from a variety of different substances
Silicon photodiodes (200 1100 nm) Indium gallium arsenide photodiodes (~900 ~2500 nm) Germanium photodiodes (650 1800 nm) Lead sulphide detectors (1000 3300 nm at minus 45 C, require cooling)

First Order Input Slit Zero Order

SecondOr der

Overlapping 1st & 2nd Order

Detector arrays
Individual detectors arranged in a line (1-D) or 2-D matrix Silicon photodiode arrays Charge Couple Devices (CCD) PbS & InGaAs arrays Need order blocking requirements

Spectrographs
Flat field gratings image is projected onto a flat plane, focussed

Linear array means acquires whole spectrum simultaneously

Spectrographs key benefits

High speed (~ 10 ms) - means very short measuring times Photodiode array or CCD technology Parallel spectral channel acquisition by bolting several spectrographs together No moving parts (excluding shutter) High sensitivity with long integration times Needs a shutter to measure dark current, set exposure to ambient light conditions

Beam splitter
Used to split a beam of light in two

Transmitted Light

Means the same light beam can be used to feed a number of spectrographs

18

GER 3700 full wavelength device

350 2500 nm Essentially has three spectrographs, from the one input optic, to cover:
Visible and near infrared (Vis/NIR 512 elements) Short wave infrared (SWIR1 128 elements) Short wave infrared (SWIR2 64 elements)

Detectors

SWIR1

That means it has:

Two beam splitters { Three diffraction gratings { Three detector arrays Three spectrographs SWIR2 Vis/NIR

Diffraction gratings

Beam splitters

Optical path
SWIR1 & 2 Optical Chopper

GER 3700 in diagrammatic form

Diffraction Gratings Detector Arrays Lenses Beam Splitters Apertures & Slits Shutter 10 Fore Optic Visible & NIR SWIR 1

SWIR1

SWIR2
Shutter

Vis/NIR

SWIR 2

19

Inside the ASD FieldSpec Pro

Three spectrometers but the SWIR units are oscillating gratings
VNIR

Monochromators
Oscillating grating, oscillates over 20 Passes different wavelengths across detector Single high quality/sensitive detector, reduces cost
Input Slit

Fixed Diffraction Grating Oscillating Diffraction Grating Detectors & Detector Array Lenses & Mirrors Fibre Optic Light Guide

SWIR 1

SWIR 2

Exit Slit Detector

Other key issues

Stray light Order effects Input optics Detector noise and linearity Harmonics

Stray light and order effects

Radiation of the wrong wavelength activates a signal at a detector element. Comes from:
Ambient light Scattering from imperfect optical components Reflections of non-optical components Order overlap from diffraction gratings

Encasing the spectrometer in a light tight housing eliminates ambient stray light. Filters can be used to eliminate order effects

Input optics
Field-of-view lenses Fibre optic light guides Cosine corrected diffusers Ideally, these should have spatially uniform properties

ASD Non uniform input optics

Actual area of measurement support per detector Nominal FOV

With 10 lens fore-optic: FOV extends up to 270 mm beyond 10 limit at 1m

20

GER 3700 with 10 fore-optic DRF

Detector noise and linearity

There will always be responses in the system unrelated to scene brightness (noise) Instruments must be designed so that the noise levels are small relative to the signal (target brightness) Measured as the Signal to Noise ratio (SNR) = ratio of usable signal to non-interpretable portion of the signal (i.e. noise) at a given input signal level SNR needs to be large over the full dynamic range

450nm

700nm

950nm

SNR =

DN DarkCurrent

DarkCurrent

Noise Equivalent Spectral Radiance (NESR), also called Noise Equivalent Power (NEP): minimum
1500nm 2200nm

resolvable change in input. The lowest brightness that can be reliably measured, i.e. when SNR = 1

Dark current, linearity and dynamic range

Dynamic range Saturation
B

Field spectroradiometers
Key specifications Spectral range & sensitivity over full range Optical bandwidth & wavelength resolution Speed Size & weight Signal to noise Dynamic range Non-linearity

Image brightness

Ideal linear response Actual sensor response

Dark current signal Scene brightness

Resolution and optical bandwidth

Sampling resolution - set by number of pixels in detector array Optical bandwidth is defined by:
Dispersion of diffraction grating Focal length Width of the entrance & exit slits Pixel width

Resolution and optical bandwidth

Definition of resolution

Detector response (%)

100

Ideal response

50

Full Width at Half Maximum Achievable response

Bandwidth

Oversampling - Most instruments oversample the spectrum e.g. optical bandwidth of ~10 nm and wavelength sampling interval of ~2-5 nm

50%

Wavelength (nm)

21

Spectroradiometric calibration
Calibrations performed by FSF typically includes:

Calibration of spectroradiometers

Absolute spectral radiance calibration (300 2500nm) Absolute spectral irradiance calibration (300 2500nm) Wavelength calibration

Needs standards against which to perform the calibration

Calibration facility

Radiance calibration
Uses integrating sphere White coated, provides uniform light output Calibrated by the National Physical Laboratories (NPL)

Irradiance calibration
Uses FEL lamp Calibrated by NPL Good output over all wavelengths

Wavelength calibration
Erbium and holmium oxide panels, lamps (mercury, sodium, etc)
Major Peaks
Mercury Emission Lines

365.0nm 404.7nm 435.8nm 696.5nm 706.7nm 738.4nm 750.9nm* 763.5nm 772.4nm 801.1nm* 811.1nm*

491.6nm 546.1nm 578.1nm* 826.5nm 841.8nm* 852.1nm 866.8nm 912.3nm 922.4nm 965.8nm

Argon Emission Lines

* denotes doublet, (a1+ b2)

where a & b are relative intensities

Mercury-argon lamp

22

Calibration means
Quality assurance (NER, SNR radiometric and wavelength accuracies) Traceability (quantification of uncertainty) Stability monitoring of instruments Confidence in the accuracy of the data Credible comparison to historical/international data sets User reassurance

Sources of error
Input optics Stray light Spectroradiometric calibration Equipment instability These are only the errors related to the instruments themselves. It ignores errors associated with actual measurements in the field

Conclusions
Get to know your equipment (intimately) Know how it works Question the suitability of the instrument for your application Trial the instrument before you take the measurements that count

23

Structure of the lecture Session 4: Measurement in the laboratory and field

Alasdair Mac Arthur Introduction Laboratory measurements Outdoor measurements: Conclusions

Making laboratory measurements

The Hardy tent mobile lab

Designed to fit over boxes

Advantages
More stable illumination compared to outdoors No atmospheric interference (full spectrum) Controlled geometry, no variations in illumination angle Can make measurements during non-optimal conditions Potential increase in signal:noise ratio Lends itself to an experimental set up and approach Can control the background

24

Disadvantages
Is artificial illumination (not sunlight), difficult to set up Direct or direct and scattered illumination Variable spatial light field Less real Influence of backgrounds if not controlled?? Heat produced by lamps can affect target and spectroradiometer

Key issues
Lamps:
Want something that simulates sunlight Not many do Tungsten lamps are probably best or combination of tungsten and halogen 500 Watt security lamps are cheap Power supplies: Always check the stability of the power supply powering the illumination lamps Real issue with ASDs Often surprising amounts of variation in voltages, especially in laboratories Use a stabilised power supply required

Other issues
Backgrounds best kept black The lamps usually require a warm up time determine what it is Check the heat output of the lamps:
Determine likely impact on the target (if vegetation it might be significant) Put lamps further away or think about providing cooling for the target (e.g. fan?)

Spatial distribution of illumination

Measure the spatial distribution of the illumination over the target

Overcoming spatial illumination problems

Consider adding more lights, but
Altering shading patterns Confounding the heating problems

Always measure in the same configuration to minimise its impact (e.g. orientation of samples)

Making measurements in the field

25

Critical issues
in field measurements: Sampling design considered in next session Measurement geometry:
Dual beam versus single field-of-view Height above target Field of view !

Choice of Instrument
Some considerations Wavelength range Relevant wavebands Project aims

Timing (of day, time year) Supporting measurements Other factors to be recorded

Dual field-of-view

Instrument configurations
Reference Reflectance Panel

Reference Spectrometer

Target Spectrometer

Target Spectrometer

Irradiance Spectrometer

Plant Canopy

Plant Canopy

Bi-Conical Configuration

Cos-Conical Configuration

Uses two sensors One to look at target The other fitted with a cosine head or viewing a reflectance panel The two measurements are made (near) simultaneously More expensive

Dual field-of-view

Single field-of-view
Reflectance R Reflected Radiance Incident Irradiance
Field Spectrometer at NADIR

Field Spectrometer at NADIR

Cosine Diffuser (Castle Design)

Lambertian Reference Reflectance Panel

Canopy Reflectance

Cos-Conical Input Optics Bi-Conical Input Optics

Reference Panel

Single Channel Configuration

26

Single field-of-view
Single sensor head Alternated between target and reference (panel usually) Necessarily imparts a time delay between the two measurements Assumes the target and reference are viewed under the same irradiation conditions and geometry The greater the difference in time between the two measurements the greater the risk of error

Measurement geometry
Target viewing geometry:
Appropriate field of view (choice of instrument foreoptic). Appropriate sampling unit (area of target sampled) - Related to instrument field of view and height above the canopy View angle nadir/off nadir

Illumination geometry:
In direction of incident irradiance Beware shading of target and panel

Height above canopy

Depend on what your wish to achieve Daughtry et al. (1982) for row crops Number of measurements required for a given level of precision decreases with increasing sensor altitude Higher the sensor the greater the area integrated into the spectrum recorded Ultimately, a compromise between the theory and what is practical

Suspending a spectroradiometer

Sun angles and timing

Time of day: At midday? At time of sensor overpass? How cope with illumination effects induced by changing sun angle if measurements made through the day Timing through year: How frequent? What are you trying to characterise?

Other influences in the field

Clouds (always take sunglasses) Cirrus clouds are difficult to spot but can significantly alter absolute intensity and spectral shape Partial cloud cover also contributes significantly to diffuse skylight illumination Need to minimise the time between target and irradiance (reference) measurement Wind movement of vegetation canopies and water in particular can significantly alter reflectance Dew affects and wet canopies

27

Influence of atmosphere
50 40

Atmospheric influences
Possible reasons: Direct radiation (i.e. clear skies) more shading of canopy components Diffuse less shading, greater reflectance in regions where scattering is high. Lower reflectance in certain areas where greater chance of absorption (by water or pigments)
1900 2400

Overcast sky Clear Sky

Reflectance (%)

30

20

10

0 400

900

1400

Wavelength (nm)

Recording variations in incident light

Direct and diffuse

Ratio of direct to diffuse conditions Use a shade ring over a pyranometer to measure diffuse, or
DeltaT BF3 sunshine sensor records diffuse & total solar radiation (PAR) and Sunshine Hours

PAR sensor records PAR radiation

Pyranometer records total solar radiation

Use a sun photometer to measure aerosol optical thickness and water vapour

Important factors to be recorded

Spectrometer set-up: Field-of-view Raw output versus reflectance output File naming conventions Viewing geometry Atmospheric conditions: Clouds, humidity, haze, wind, pyranometer irradiance, horizontal line of sight Physical conditions of target: Biophysical parameters, slope, aspect, moisture conditions, GPS location, soil moisture

Take digital photos

Record the condition of the target at the time of measurement Take a photo of the sky Record the set up

28

Recording forms
Example logsheets

Considerations of the very practical kind

Warming up of instruments Charging batteries Logistics Check everything is there and everything works BEFORE you go into the field! Allow for the fact that your first visit wont give you useful data Transport You will need help very rarely is it a one person show

Sequence of events: ideal scenario

Choice of Instrument Instrument configuration Experiment design Fieldwork plan Training Fieldwork Post processing Publications

29

Structure of the lecture Session 5: Sampling design and measurement uncertainty

Alasdair Mac Arthur
Introduction Definitions accuracy and precision Design of field programs Site selection Experimental design / sampling design Uncertainties Vicarious calibration Conclusions

Definitions
Accuracy confidence in the relation of one set of measurements with another Getting it right Precision careful measurement under controlled conditions Confidence in successive measurements with the same equipment and operating conditions The repetitiveness of measurement of the same target Accuracy is telling the truth . . . Precision is telling the same story over and over again

Approaches to research
Inductive phenomena observed, generalisations made, conclusions drawn Deductive hypotheses posed, tested by observation, experiment Much of the spectroscopy literature is inductive, indicating that we are still evaluating different approaches, strengths and limitations of the data

:K\ GR ZH QHHG GHVLJQ"

A little planning saves a lot of time To gather information as efficiently and accurately as possible Statistics cannot rescue a badly designed method Poorly collected data lead to poor conclusions If you must break the rules, do so consciously

Design of field programs

Dependent on objectives of study, e.g.: Spectral libraries Vicarious calibration of image data, empirical line targets Establishing relationships But sampling strategy is key to all

30

Strategies
Dependent on approach, e.g.: Correlation vs manipulative study Field or laboratory or both (my preferred approach!) Being able to control what you can, and randomize the remaining effects Control tends to decrease the experimental error

Key considerations
Selecting the site/sample Relation to the objectives of the investigation Being practical and achievable Cost-effective in time and equipment Providing estimates of population parameters that are truly representative and unbiased How you plan to process the data

Site selection considerations

Uniformity versus heterogeneity Scale of several space sensor pixels Sampling sites span the radiometric range that occurs across the landscape Other considerations:
Slope and aspect Time Accessibility (some of the kit is not light!)

How many samples?

How many observations or measurements must be acquired to be confident of: Detecting differences between surface types or different conditions Sufficiently characterising the surface bidirectional reflectance properties? Required number of sample sites and number of measurement replicates Dependent on scale of study and objectives Do a pilot study the only way to have statistical validity!

Matching the radiometers characteristics to air or space sensor (spectral bands, viewing angle)

How many samples?

Ideally, representative samples should be:
Large enough to give sufficient precision Unbiased by the sampling procedure or equipment (i.e. accurate)

Sampling design
Approaches: Point sampling Transects Plots Approach should be established before the investigation proceeds Key factors: The dimensions and shape of the sampling unit (e.g. field of view (DRF!) versus height) The number of sampling units in each sample The location of sampling units within the sampling area

31

Point sampling

Plots

Sampling a plot How not to do it!

Transects

Acquiring spectral transect data

Grid sampling to acquire spectra

ASD on GPS enabled quad-bike

32

Grid sampling to acquire spectra

Replication
Replication within a treatment shows how variable the response can be Provides an estimate of experimental error Improves precision by reducing the standard error of the mean Increases the scope of inference of the results Required number dependent on statistical analysis to be applied (to establish significance) If in doubt, start with n = 3 but in the natural environment you will need many many more Be consistent in your data recording

Beware pseudoreplication
Pseudoreplication in spectroscopy would be taking three scans from the same point and assuming they are replicates
i.e. intra-sample variation versus inter-sample variation

Point sampling strategies

Random? Stratified? Systematic? Webster et al. (1989):
Relate sampling effort to the variation present The more variable a region the larger the sample should be Determine required sampling by pilot survey

Result is an unrealistically small SE, which may invalidate conclusions Can use intra-sample measurements to increase signal to noise but cant use them as the basis of a statistical analysis

Heterogeneous landscapes

Heterogeneous areas of interest

May Building

June Building

Bog pool approx 10x20m

33

Point sampling strategies

Many targets are spatially autocorrelated Random sampling is less optimal in this situation Thus, other approaches are better Webster et al. propose use of the semivariogram to define the spatial variation and to design an efficient sampling

Vicarious calibration
Includes empirical line correction Use earth surface sites (pseudo-invariant) to verify calibrations or calibrate sensors Needs in-situ radiometer measurements Vicarious calibration - sites with long-term stability to enable temporal calibration drifts to be determined Sites need to be large and homogeneous

Desert Calibration Targets

Libyan (Western) Desert - Egypt Sonora - Mexico

Vicarious calibration
Allows radiometric closure between sensor top-ofatmosphere radiance values and ground measurements (bottom-of-atmosphere) Measurements on ground must be synchronous with satellite overpass (due to changes in atmosphere and target properties) Scaling problems: satellite integrates over a larger area on the surface. How respresentative is the point measurement on the ground? Needs sufficient spectroradiometer measurements to characterise variability within the area of the target/pixel All measurements are usually normalised to a ground based, laboratory reference standard

Makhtesh Ramon Israel, Negev desert for wavelength calibration

Empirical line method

To establish empirical relationships between sensor radiance and ground reflectance Requires coincident measurement of two or more ground sites of contrasting brightness that are discernable on imagery Again, requires sufficient spectral measurements to characterise the variability in the targets Smith and Milton area 4 x image pixel size 20+ measurements

Empirical line measurements

Karpouzli, E., Malthus, T. (2003). The empirical line method for the atmospheric correction of IKONOS imagery. International Journal of Remote Sensing, 24(5):1143-1150.

34

Sufficient number of targets

80

Ensure range of brightness

50

B lue W av eba n d
C eme n t

50

Gre e n W av eba n d

C eme n t

Gr o und reflectance (% )

40
Groun d re fle cta nc e (%)
S a nd

40

Sa nd

IKONOS NIRed Band

70
Ground measurements reflectance (%)
3

30

30

1 2

20

20

60 50
5

10

Tarmac Grass La wn

y = 0 .1 22 9 x - 2 9 .2 64 R = 0 .8 7 31
2

10

La wn Tarmac

Grass

y = 0.090 3x - 1 5.6 87 R = 0.8 78

2

0 0 2 00 4 00 IK ON OS Rad ia nce 6 00 8 00

0 0 2 00 4 00 IKON OS R adia nce 6 00 8 00

R ed W av eba n d

40
7

8
Gr o und reflec tan ce (% )

50
Ceme n t S a nd

80 70 Gr o und reflectance (% )

NIR W av eba n d
Lawn

30 20 10
9 6

40

60 50 40 30
Ceme n t

Gra ss

y = 0.0559x - 4.5544 R = 0.9609

2

0 0 200 400 600 800 1000 1200 1400

Radiance recorded by the sensor (DN values)

Malthus, T.J., Karpouzli, E. (2003). Integrating field and high spatial resolution satellite based methods for monitoring shallow submersed aquatic habitats in the Sound of Eriskay, Scotland, UK. International Journal of Remote Sensing, 24(13):2585-2593.

30

Sa nd

20

y = 0 .09 6x - 4 .32 62 20 10
T arm ac

10

T arm ac

Gra ss Lawn

R = 0 .9 3 08

y = 0.089 9x - 7 .42 36 R = 0.8 86

2

0 0 2 00 4 00 IKON OS R adia nce 6 00 8 00

0 0 2 00 4 00 IKONOS Ra dia nce 6 00 8 00

Conclusions
Careful consideration must be given to the design of your sampling approach and the propagation of errors in measurement A careful approach maximises your chance of Obtaining meaningful data Obtaining reproducible data Reliably detecting differences or phenomena not previously explained

35

Structure
Session 6: The processing and initial analysis of spectral datasets
Alasdair Mac Arthur
Introduction Processing to reflectance and absolute reflectance Radiance calibration Averaging Software The calculation of broad bands Smoothing of spectra Conclusion

Initial processing
Downloading Initial quality assessment Radiance calibration (if required) Processing to reflectance Correction to absolute reflectance Removal of atmospheric water vapour bands Averaging Peddle et al. (2001) describe a spreadsheetbased approach

Initial quality assessement

Largely visual, looking for: Noise Steps at junction of different detector arrays (can be due to temperature and DRF effects) Anything out of the ordinary, signalling other problems Consider irradiance and sky condition data if any is available -

Correction to absolute reflectance

Panel correction factors, supplied with your panel by FSF Specific to each panel FSF generated every prior to providing panels for loan

Radiance calibration
Requires accurate radiance calibration for instrument If required, radiance calibration file(s) will be supplied with your instrument Probably best to collect uncalibrated data initially and correct afterwards (FSF approach) Specific to each fore optic

36

Other considerations
Correction for non-Lambertian panel reflectance at very large solar zenith angles (above 60) Using the published corrections for Spectralon Requires calculation of sun-angle of course (time and position dependent)

Uncertainties
Inherent Instrument errors Calibration errors Measurement uncertainty Influences of the atmosphere Variable illumination Influence of temperature Real variation

Influence of the atmosphere

50 40

Influence of temperature
3500 3000 Radiance (Raw DN) 2500 2000 1500 1000 500 0 900

Overcast sky Clear Sky

Reflectance (%)

30

20

10

1000

1100

1200

Wavelength (nm)
0 400 900 1400 1900 2400

Wavelength (nm)

Change in radiance measurement over reference panel in NIR inside the Hardy tent

Software demonstrations
The FSF spreadsheet ASD Viewspec software Better still FSF Matlab Toolbox

SAMS
Spectral Analysis and Management System Version 3.2 recently available From Centre for Spatial Technologies and Remote Sensing (CSTARS) at UC, Davis http://sams.casil.ucdavis.edu/

37

SAMS
Accepts ASD and GER formats Imports ASCII (plain text) files intelligently Save spectra as groups and then save in SAMS database Can link spectra to metadata Common spectral functions

Other tools PARLes

Merging data into a single file Transformations (smoothing, detrending, derivatives, wavelets) Principal components analysis (PCA) Partial least squares regression (PLSR) Bagging PLSR (bootstrap aggregation)
Viscarra-Rossel (2008), available on request from r.viscarra-rossel@usyd.ed.au

Calculating broad band reflectances

To simulate reflectances from different airborne and satellite borne sensors
AVHRR Spectral Bands
80

TM Spectral Bands
80 12 3 4 5 7

Reflectance

Reflectance

Calculating broad band reflectances

70 60 50 40 30 20 10 0 400 700 1000 1300 1600 1900 2200 2500

70 60 50 40 30 20 10 0 400 700 1000 1300 1600 1900 2200 2500

Wavelenth (nm)

Wavelenth (nm)

Broad band reflectances

Needs spectral response functions for the specific sensor(s) you will simulate Why? Because they are all different! They are not uniformly sensitive Set of filter functions incorporated into the FSF Matlab Toolbox

Landsat ETM+ bands 1 to 7

1

Spectral sensitivity

0.8

0.6

0.4

0.2

0 400

900

1400

1900

2400

Wavelength (nm)

Not particularly symmetrical, best approximation using available filters

38

Landsat sensors, bands 3 and 4

1.0 TM ETM+ 0.8 MSS

Red and near IR bands, others

1 0.8 Sensitivity 0.6 0.4 0.2 0 400

SPOTS 1 to 4

Sensitivity

0.6

500

0.4

600 700 800 Wavelength (nm)

900

1000

1.0

QuickBird IKONOS

0.8

Sensitivity

0.2

0.6

0.4

0.0 400

500

600

700

800

900

1000

1100

1200

IKONOS and QuickBird

0.2

Wavelength (nm)

0.0 400

500

600

700

800

900

1000

1100

1200

Wavelength (nm)

Broad band reflectances

Ideally should be applied to separate target and reference scans (Robertson and Biehl) ? Steps:
Multiply target reflectance by the spectral response function Apply same to a spectrum representing reflectance of 100% Integrate the results for both Normalise by dividing target value by 100% value
n

Smoothing of spectral data

Field spectra have higher noise due to: Variable illumination Lower incoming radiation in some wavelengths Atmospheric interference Noise is often not constant with wavelength due to effects of instrument (transition from one detector to another), water vapour, other atmospheric interferences Often require smoothing Caveat treat with caution know what the effects of your smoothing algorithms are. Would noise removal better?

Reflectance spectral response function Broad band reflectance =

0 n

100 100% spectral response function

Approaches to smoothing Noise

Moving average filter can be severe on flattening out fine variations in the spectrum Simple weighted filters Least-squares polynomials (Savitzky and Golay 1964) better, can smooth and calculate derivatives directly (r)lowess (r)loess worth considering Cubic splines Fourier transforms Wavelets (e.g. Schmidt and Skidmore 2002) Caveat treat with caution know what the effects of your smoothing algorithms are. Would noise removal better

39

Comparison of smoothing methods

Interpolation
Dont unless you have to! Can introduce artefacts Talk to a mathematician If you use an ASD FS Pro it will interpolate for you whether you like it or not!

Spectral libraries
Sets of measured spectra for components likely to be encountered in the study area For spectral matching, spectral mixture analysis, etc. Several available, particularly for rocks and minerals Easy to compile for some objects (e.g. minerals) For others (e.g. plants, water targets) the key is the coincident access to the metadata which describes the key characteristics of the target Find out how they were generated before you use them and consider the implications

Spectral signatures

Spectral libraries
Increasing availability of spectral data (more instrumentation), more widespread Increased focus on cal / val Integration with IP softwares / data analysis Moves to continuous monitoring of reflectances Initiatives and calls to develop archives of spectra use - wider community initiatives Incompatible, often internal, data formats, from different instruments, separated from metadata Needs for data preservation, legacy value, lineage

Uses of spectral libraries

Calibration e.g. potential use as a target for atmospheric correction Validation e.g. to validate a model-based atmospheric correction, model simulation End members for incorporation into image processing routines Global algorithms / analysis Retrospective analysis for test a new research hypothesis / analysis technique

40

Spectral libraries
Spectral libraries: USGS spectral library - ~500 spectra of minerals over 200 3,000 nm range (http://speclab.cr.usgs.gov) Johns Hopkins University JPL spectral library (160 spectra) Mainly mineral, obtained under controlled conditions, few biophysical targets Conditions of measurement and subsequent processing less well documented

Spectral databases
Spectral databases ASTER spectral library ~ 2000 spectra of rocks, soils, water, snow, man-made materials (http://speclib.jpl.nasa.gov) Hyperspectral.info SPECCHIO (www.specchio.ch) Again, metadata components are minimal

Issues
Make good measurements in the field Spectral data collections are most often project (campaign) based, obtained for different purposes (unique?) Different methods, different instruments Of highly variable (unknown) quality How to store and easily exchange such data Implications for data quality and assessment Coping with single spectra, nested data from projects, replicates, related targets, campaigns Efficient in metadata entry

The challenge of field measurement

What determines quality?

Quality of conditions under which it was obtained Quality of the instrument and its calibration Design of the study Experience of the user Visual quality of the spectrum obtained Quality of the documentation obtained with the measurement, including information on the properties of the target (even a photograph) Quality flags?

Metadata
Quality relies on spectral data themselves but associated metadata is fundamentally necessary The existence of extensively documented metadata ultimately determines long-term usability and quality Assists searching and selection Assists assessment of suitability for other research projects Critical if data are obtained in field data A current hot topic with NERC Workshop on FS metadata and spectral libraries being planned for Oct 2011 by Tim Malthus, CSIRO, Canberra.

41

Metadata requirements
Should document: Instrument characteristics Conditions of measurement (meteorological, physical, geometrical) Target properties Subsequent processing Data characteristics A spectral dataset of high quality is one where these characteristics are extremely well documented it indicates that care has been taken in data acquisition

Metadata
Instrument make and model Manufacturer Serial number Owner Detector types Spectral wavelength range Spectral bandwidth Spectral resolution Operator DarkSignal correction Signal to Noise Scan duration Optic Field-of-view dimension X Optic field-of-view dimension Y Gain settings (Automatic/Manual) Signal averaging (instrumental) Integration time Setup (single beam, dual beam) Mode (cos-conincal, biconical) Calibration Date Irradiance Radiance Dark noise Signal to Noise Linearity Stray light Calibration data Traceability (e.g. Yes, No) Standard (e.g. NIST, NPL) Reference standards None Cosine receptor Type Reference standard Reference (panel, cosine) Serial number Reference material Time of measurement Calibration standard

Metadata
Measurement and configuration General: Name of experiment/Project Date of experiment Relevant publication Relevant websites Project participants Acknowledgement Text Plot number Description of target/sample Type of measurement (field, lab, etc) Target type vegetation, water, rock, air, Target ID Target treatment Sky conditions Clear sky, % cloud, cloud type Cloud cover Horizontal sight Wind speed Optical measure of ambient conditions (direct, diffuse) Source of illumination (e.g. sun, lamp) Location Referencing Datum Map projection Base unit Coordinate source N-S Coordinates E-W Coordinates Longitude Latitude Altitude Viewing geometry Time of measurement Distance from target Distance from ground/background Area of target in field of view Illumination zenith angle Illumination azimuth angle Sensor zenith angle Sensor azimuth angle Optical measure of ambient conditions Ambient temperature Instrument temperature

Metadata
Data (post)processing and manipulation Software used, version number Interpolation None Algorithm applied Number of interpolated points per data point Atmospheric band removal (yes, no) Averaging (yes, no, if yes, how many?) Averaging (mean or median, or closest spectra?, Standard deviation reported?) Smoothing None Algorithm applied Filter size Polynomial order Number of times applied Other Difference spectra Derivative spectra Supplementary Target data: Vegetation Water Rock/soil Atmosphere etc. Vegetation: Common name Species Type Class Subclass Leaf / Canopy LAI Chlorophyll content Biomass Moisture content Leaf angle distribution Time of year Background (soil / other?) Soil type Soil moisture content Comment, etc.

Metadata
Water Location Waveheight Wind conditions Depth of measurement Suspended sediment concentration Chlorophyll concentration Secchi disk transparency, etc. Underwater substratum Target Substrate description Type (hard, soft, vegetation, animal) Specifications? Density of growth Presence of epiphytes Water type Spectrum type (in situ/on boat/in lab) Upwelling/downwelling radiance Spatial resolution Water surface conditions Wind conditions The data itself Data precision Data type (Reflectance, Radiance) First X value Last Y value First Y value Last Y value Min X value Max X value Min Y value Max Y value Number of X values Wavelength interval XTitle YTitle XUnits YUnits Scaling factors Xfactor Yfactor Wavelength data Spectrum

Solution
Research project which is developing an XML exchange format, not a database PhD research into metadata requirement for FS currently being conducted by <barbara.rasaiah@rmit.edu.au> Being rigorous promotes best practice in field measurement

42

Conclusions
Visual assessment of individual spectra is important Software tools to enable visual analysis and to allow for batch processing Watch broad band calculations, not as easy as it sounds Smoothing is more complicated than it looks! As is interpolation and extrapolation is fraught! Spectral libraries require work and use critically!

43

Structure of the lecture Session 7: The analysis of spectral datasets

Alasdair Mac Arthur
Introduction - Approaches Linear methods Multivariate methods Non-linear methods Conclusions

Introduction
The data richness of hyperspectral data presents a considerable challenge to the analyst They provide fine spectral detail, but at an overwhelming data volume There is much apparent redundancy between neighbouring bands Much of the analysis is an exercise in data reduction, but such that the differences in reflectances between ground surface objects can be retained (feature extraction / reduction) The curse of dimensionality

Introduction
The development of appropriate tools and approaches for visualising and analysing hyperspectral data is still very much ongoing Much of that is focussed on the spectral rather than the spatial, let alone directional the next frontier! Feature reduction transforms original data to dataset of lower dimensionality but retaining original information

Deductive approaches to analysis

Empirical approaches Correlation Derivative analysis Spectral decomposition Feature reduction Discriminant analysis Neural networks Many of the techniques are methods of data reduction

Software
Specific to field spectra: ViewSpec, SAMS package, PARLes Bespoke programs Spreadsheets (e.g. Excel) Software processing and visulaisation packages (e.g. Matlab, IDL) Matlab possibly preferred by field spectroscopists, IDL by image analysts A plead - Please learn a scripting language that can manipulate 3-D numerical arrays

44

Linear correlation analysis

Simple linear regression used to relate reflectance at any wavelength to (bio)physical parameters of interest Used to highlight those wavelengths which correlate highly with the parameters of interest But assumes a simple and direct relationship between reflectance and the parameter, or target (e.g. linear or curvi-linear relationship)

Dutch lake reflectances

Subsurface reflectance

Measured from a CASI sensor

Malthus and Dekker (1995)

Linear correlation analysis

Log(1/R)
Useful as inverse highlights absorption features Figure shows near-infrared spectra of soil samples with highest and lowest Carbon contents from set of 136 surface samples, from Reeves et al. (2002)

Dry weight (a measure of suspended sediment concentration) versus in-lake reflectance (left) and from a CASI sensor (right)

Narrow-band ratio approach

A form of data reduction Use the data to select narrow-bands in apparent features of interest Use these as the basis of indices (e.g. ratios) to correlate against (bio)physical parameters Can use a correlation matrix approach to test every possible ratio will generate hugh data set!

Narrow band selection

45

Indices

More complex indices

E.g. Normalised difference Three-band reflectance models:
Pigment concentration [R1 (1 ) R1 (2 )] R(3 )

Using reciprocal reflectances

Subsurface reflectance CASI data

3-band algorithm - Water

Chl (mg m3 ) [R1(665) R1(715)] R(750)

n-band algorithm - Vegetation

Chlorophyll in water. Gitelson et al. (2008) works across a range of waters

Chlorophyll in crops. Gitelson et al. (2005) works across different crops

Correlation matrices

NIRS
Near infrared reflectance spectroscopy Extensively used in agro-chemistry E.g. composition of grains and foods Require calibrations that relate spectral information to analyte being determined (e.g. C or N content) Translated to foliar biochemistry Often based on stepwise multivariate regression, PCA or PLSR, derivatives E.g. Workman (1992), Curran et al. (1992), Peterson et al. (1988)

Welsh lake subsurface reflectances and all possible ratios versus chlorophyll concentration George and Malthus (2001)

46

NIRS
Problems:
Overfitted (more wavelengths than samples selected) Selection of non-causal wavelengths Intercorrelation of variates Generality

Continuum removal

E.g. Curran et al. (2002)

Continuum removal

Continuum removal / band depth

Band depth can be depth of the feature or area of the feature Analysis on basis of continuum removal tends to better relationships (i.e. R2) than normal data Note subtle difference between approach used in these graphics

Fitting the continuum

Continuum removed

Huang et al. (2004)

Continuum removal
Removes irrelevant background reflectance Allows isolation and enhancement of absorption features Tends to give higher R2 values Combined with stepwise multiple regression and PLSR Often used by geologists of much more limited use to others

Stepwise regression

Leaf biochemistry from reflectances, from Curran et al. (2002) Widely used in NIRS, suffers from problems highlighted above

47

Partial least squares regression

An extension of multiple regression Similar to principal component analysis, but fits linear model Tries to find multidimensional direction in x-space that explains maximum multidimensional variance direction in y-space Suited to hyperspectral data when no. of predictors exceeds number of observations Overcomes overfitting problems and multicollinearity among X values Requires correlograms of reflectance to help identify optimal bands

Limitation
Stepwise multiple regression and PLSR both based on the assumption that linear relationships exist between variable and reflectance But those relationships may be non-linear Spectral features selected tend to be site-specific or be non-causal, therefore non-general in application (requires recalibration)

Derivative analysis
Derivatives give an indication of rate of change, or slope of the original spectrum Increasing reflectance = positive first derivative Decreasing reflectance = negative first derivative Can be of any order (e.g. 1st, 2nd, 3rd, 4th) but noise is amplified with every level smoothing often required

Derivative analysis
Commonly applied in analytical chemistry where peaks in 4th derivative can be shown to be related to peaks in absorbing compounds i.e. useful for the resolution of overlapping spectral features Allows for elimination of background signals (e.g. soils from vegetation spectra) Derivatives of second or higher should be relatively insensitive to variations in illumination intensity whether caused by changes in sun angle, cloud cover, or topography Examples: Demetriades-Shah et al. (1990), Malthus and Dekker (1995), Tsai and Philpot (1998)

Derivative analysis

Derivative correlation

Subsurface reflectance

CASI data

48

Derivative approaches

Neural networks
Caters from non-linear relationships Evidence that they give stronger relationships (higher R2) than linear methods (Huang et al. 2004) Does not need continuum removal Somewhat black box in operation

Shaw et al. (1998) Scots Pine regeneration study

Binary encoding

Binary encoding
Considerably reduces dimensions of dataset while preserving main features Field spectra can then be compared to a binary encoded spectral library for spectral matching

Goetz 1991, Goetz et al. 1985.

Spectral matching
Matching image spectra to those derived from spectral libraries Needs accurate conversion of image radiance to reflectance Scene should contain mainly pure pixels (of materials) Complicated by mixed pixel problem Uses goodness of fit to determine best match Can use continuum removal

Critique of spectral matching

Though an attractive and straightforward idea, spectral matching and [spectral libraries] have not proven to be very powerful in terms of their ability to extract information in a robust and practical sense Landgrebe (1999) Criticism in that it relies on the notion of the pure pixel is there such a thing?

49

Discriminant analysis
Multivariate analysis technique A form of statistical classification on a priori defined groups Stepwise methods can be used as a tool for data reduction to identify the key wavelengths leading to target separation Does have its faults

Wavelets
Mathematical functions that can be convolved and passed across spectra Analyse spectra across wavelengths for frequency content Decompose spectra into decreasing sets of coefficients Can be used for denoising data Can be used as a method of data reduction Can be used for analysis (Blackburn 2008)

RT models forward and inverse modelling

PROSPECT (Jacquemoud 1996) Models the optical properties of leaves LIBERTY (Dawson 198) Models the optical properties of needles ACRM (Kuusk 2001) Models canopy reflectance and can account for layers incorporates PROSPECT and LIBERTY 4Sail2 (Verhoef 2003) Models canopy reflectance and can account for layers can be coupled to PROSPECT

Other methods
Support vector machines (e.g. Melgani and Bruzzone 2004) shown to be more effective than other nonparametric classifiers (e.g. ANNs, K-nn) Spectral un-mixing (linear and non-linear) Spectral angle mapper (SAM) Spectral deconvolution Principal components analysis (PCA) ?????? We just keep nicking them for others!

Conclusions
Power of high spectral resolution allows the investigation of alternative analytical approaches Largely techniques of feature reduction Often adopt empirical approaches to algorithm development Algorithm development / analysis still in its infancy RT modelling offers a virtual laboratory

50

Methods for measuring spectral reflectance in the field

5 methods for measuring spectral reflectance Method I

Spectro #1 with FOV or bare fibre Sequential

Spectro #1 with FOV or bare fibre

DNTarget Reflectance = DN Ref Panel

x CalRef panel

Ref Panel

Target

Method II

Spectro #1 with cosine receptor

Sequential

Spectro #1 with FOV or bare fibre

DNUpwelling x CalRadiance Reflectance = DNDownwelling x CalIrradiance x 2

Target

Method III

Spectro #1 with cosine receptor

Sequential

Spectro #1 with FOV or bare fibre

Spectro #1 with cosine Sequential receptor

Spectro #1 with FOV or bare fibre

Inter-Calibration
Ref Panel Target

DNUpwelling Reflectance =DN

Downwelling

x Cal Inter-Cal

51

Method IV ( Dual System)

Spectro #1 with cosine Simultaneous receptor

Spectro #2 with FOV or bare fibre

Measurements

DN#2Upwelling x Cal#2 Radiance Reflectance = DN#1Downwelling x Cal#1 Irradiance x 2

Target

Method V (Dual System)

Spectro #1 with cosine Simultaneous receptor

Spectro #2 with FOV or bare fibre

Spectro #1 with cosine Simultaneous receptor

Spectro #1 with FOV or bare fibre

Measurements

Measurements

Inter-Calibration
Ref Panel Target

DN#2 Upwelling Reflectance =DN

#1 Downwelling

x Cal Inter-Cal

Notes: i) ii)

All DN values must be normalised for integration times and amplifier gains settings Inter calibration methods do not require the system response calibration files (CalIrradiance, CalRadiance) but a uniquely generated for the systems used and the ambient lighting conditions

52

Logsheets

53

54

55

56

57

Reference List

58

Introduction to Field Spectroscopy

Selected reference list

Asrar, G. (1989). Theory and applications of optical remote sensing. New York, John Wiley and Sons. (Chapter 2 summarises a few relevant approaches) Curran, P. J., Dungan, J. L., Macler, B. A., Plummer, S. E., & Peterson, D. L. (1992). Reflectance spectroscopy of fresh whole leaves for the estimation of chemical composition. Remote Sensing of Environment, 39: 153-166. (Foliar biochemistry, regression analysis) Curran, P.J., Dungan, J.L., Peterson, D.L. (2002). Estimating the foliar biochemical concentration of leaves with reflectance spectrometry: Testing the Kokaly and Clark methodologies. Remote Sensing of Environment, 76: 349-359. (Continuum removal, multivariate regression, foliar biochemistry) Daughtry, C. S. T., K. P. Gallo, et al. (1983). Spectral estimates of solar radiation intercepted by corn canopies. Agronomy Journal 75: 527-31. (Effects of variations in measurement height) Daughtry, C. S. T., V. C. Vanderbilt, et al. (1982). Variability of reflectance measurements with sensor altitude and canopy type. Agronomy Journal 74: 744-51. (Effects of variations in measurement height) Demetriades-Shah, T. H., M. D. Steven, et al. (1990). High resolution derivative spectra in remote sensing. Remote Sensing of Environment 33: 55-64. (The case for derivative analysis) Duggin, M. J. (1981). Simultaneous measurement of irradiance and reflected radiance in field determination of spectral reflectance. Applied Optics 20(22): 3816-3818. (Approaches to measurement) Duggin, M. J. (1983). The effect of irradiation and reflectance variability on vegetation condition assessment. International Journal of Remote Sensing 4(3): 601-608. (Influence of extraneous factors) Duggin, M. J. and T. Cunia (1983). Ground reflectance measurement techniques - a Comparison. Applied Optics 22(23): 3771-3777. (Comparison of methods) George, D.G., Malthus, T.J. (2001). Using a compact airborne spectrographic imager to monitor phytoplankton biomass in a series of lakes in north Wales. Science of the Total Environment, 268:215-226. (Empirical approach to analysis) Gitelson A.A., Vina, A., Ciganda, V., Rundquist, D.C., Arkebauer, T.J. (2005). Remote estimation of canopy chlorophyll content in crops. Geophysical Research Letters, 32: L08403 (3 band algorithms - vegetation) Gitelson, A.A., DallOlmo, G., Moses, W., et al. (2008). A simple semi-analytical model for remote estimation of chlorophyll-a in turbid waters: Validation. Remote Sensing of Environment, 112: 3582-3593. (3 band algorithms - water) Goel, N.S., and N.E. Reynolds (1989). Bidirectional canopy reflectance and its relationship to vegetation characteristics. International Journal of Remote Sensing, 10:107-132. (on BRDF). Goetz, A.F.H. (1991). Imaging Spectrometry for studying Earth, Air, Fire and Water. EARSeL Advances in Remote Sensing 1: 3-15. (binary encoding) Goetz, A.F.H., Vane, G., Salomon, J. and Rock, B.N. (1985). Imaging spectrometry for Earth remote sensing. Science, 228: 1147-1153. (binary encoding) Huang, Z., Turner, B.J., Dury, S.J., Wallis, I.R., Foley, W.J. (2004). Estimating foliage nitrogen concentration from HYMAP data using continuum removal analysis. Remote Sensing of Environment, 93: 18-29. (Continuum removal, stepwise regression, partial least squares, neural networks)

59

Karpouzli, E., Malthus, T. (2003). The empirical line method for the atmospheric correction of IKONOS imagery. International Journal of Remote Sensing, 24(5):1143-1150. (Example of spectroradiometry for atmospheric correction) Karpouzli, E., Malthus, T.J., Place, C.J. (2004). Hyperspectral discrimination of coral reef benthic communities in western Caribbean. Coral Reefs, 23:141-151. (Example of use of disriminant analysis of both reflectance and derivative data) Kimes, D. S., J. A. Kirchner, et al. (1983). Spectral radiance errors in remote sensing ground studies due to nearby objects. Applied Optics 22: 8-10. (Influence of variations in background on measurements obtained) MacArthur, A.A., C. MacLellan, T. J. Malthus (2006). What does a spectroradiometer see? Proceedings of the Remote Sensing and Photogrammetric Society Annual Conference, University of Cambridge, September 2006. (highlights deficiencies in uniformity of fieldsof-view of GER and ASD spectroradiometers). Malthus, T.J., Dekker, A.G. (1995). First derivative indices for the remote sensing of inland water quality using high spectral resolution reflectance. Environment International, 23:221232. (Example of analysis based on derivatives, correlation analysis) Malthus, T.J., George, DG, (1997). Airborne remote sensing of aquatic macrophytes in Cefni Reservoir, Anglesey, UK. Aquatic Botany, 58:317-332. (Example of use of discriminant analysis) Malthus, T.J., Karpouzli, E. (2003). Integrating field and high spatial resolution satellite based methods for monitoring shallow submersed aquatic habitats in the Sound of Eriskay, Scotland, UK. International Journal of Remote Sensing, 24(13):2585-2593. (Example of empirical line method applied) Melgani, F., Bruzzone, L. (2004). Classication of hyperspectral remote sensing images with support vector machines. IEEE Transactions In Geoscience and Remote Sensing, 42: 17781790. (Support vector machines) Meroni, M., Colombo, R. (2009). 3S: A novel program for field spectroscopy. Computers and Geosciences, 35:1491-1496. (Software package for processing spectroscopic data) Milton, E. J. (1987). Principles of field spectroscopy. International Journal of Remote Sensing 8: 1807-27. (Lays down the basic principles of the approach well worth a read) Milton, E.J., Schaepman, M.E., Anderson, K., Kneubuehler, M., & Fox, N. (2009). Progress in field spectroscopy. Remote Sensing of Environment, 113:S92-S109. (opinion paper). Milton, E. J., Rollin, E.M., Emery, D.R (1995). Advances in field spectroscopy. Advances in environmental remote sensing. F. M. Danson, Plummer, S.E. Chichester, Wiley: 9-32. (Recent advances to 1995) Nicodemus, F.E. (1970). Reflectance nomenclature and directional reflectance emissivity. Applied Optics, 9:1474-1475. (defining paper on BRDF) Nicodemus, F.E., C.J., Richmond, et al. (1977). Geometrical considerations and nomenclature for reflectance. US Government Printing Office, Washington DC 20402. (defining paper on BRDF) Peddle, D. R., H. P. White, et al. (2001). Reflectance processing of remote sensing spectroradiometer data. Computers & Geosciences 27(2): 203-213. (describes spreadsheet basesd approach to processing of spectral data) Pegrum, H., N. Fox, M. Chapman, and E Milton. (2006). Design and testing a new instrumentation to measure the angular reflectance of terrestrial surfaces. Proceedings of IGARSS06, Denver, Colorado (IEEE). (Describes basis and design of the GRASS goniometer). Peterson, D. L., Aber, J. D., Matson, P. A., Card, D. H., Swanberg, N. A., Wessman, C. A., & Spanner, M. A. (1988). Remote sensing of forest canopy leaf biochemical contents. Remote Sensing of Environment, 24: 85-108. (multivariate regression, foliar biochemistry)

60

Reeves, J., McCarty, G., Mimmo, T. (2002). The potential of di use reectance spectroscopy for the determination of carbon inventories in soils. Environmental Pollution, 116: S277S284. (Near infrared reflectance spectroscopy) Richardson, A. J. (1981). Measurement of reflectance factors under daily and intermittent irradiance variations. Applied Optics 20(19): 3336-3340. (Influence of changes in incident irradiance) Robinson, F. B. and L. L. Behl (1979). Calibration procedures for measurements of reflectance factors in remote sensing field research. Society of Photo-Optical Instrumentation Engineering 196: 16-26. (Importance of calibration) Savitzky, A. and M. J. E. Golay (1964). Smoothing and differentiation of data by simplified least squares procedures. Analytical Chemistry 36: 1627-39. (Paper outlining least squares polynomial smoothing, but errors contained in tables of coefficients) Schaepman-Strub, G., M.E. Schaepman, T.H. Painter, S. Dangel, J.V. Martonchik (2006). Reflectance quantities in optical remote sensing definitions and case studies. Remote Sensing of Environment, 103:27-42. (lays out the mathematics for different reflectance measurements in an attempt to standardise the terminology in field spectroscopy) Shaw, D. T., T. J. Malthus, et al. (1998). High-spectral resolution data for monitoring Scots pine (Pinus sylvestris L.) regeneration. International Journal of Remote Sensing 19(13): 26012608. (Example of use of derivatives) Smith, G. M., and Milton, E. J., (1999). The use of the empirical line method to calibrate remotely sensed data to reflectance. International Journal of Remote Sensing, 20, 2653 2662. Steven, M.D, Malthus, T.J., Baret, F., Xu, H., Chopping, M.J. (2003). Intercalibration of vegetation indices from different sensor systems. Remote Sensing of Environment, 88(4):412422. (Example of application of sensor broad band filters to spectroradiometer data) Tsai, F. and W. Philpot (1998). Derivative analysis of hyperspectral data. Remote Sensing of Environment 66: 41-51. (Use of derivatives) Viscarra Rossel RA. 2008. ParLeS: Software for chemometric analysis of spectroscopic data. Chemometrics and Intelligent Laboratory Systems, 90: 72-83. (more software) Webster, R., P. J. Curran, et al. (1989). Spatial correlation in reflected radiation from the ground and its implications for sampling and mapping by ground-based radiometry. Remote Sensing of Environment 29: 67-78. (One approach to the choosing sample sites, based on geostatistics and the semi-variogram). Workman, J.J. (1992). NIR spectroscopy calibration basics. In: Burns, D.A., Ciurczak, E.W., (Eds.), Handbook of near infrared analysis. Marcel Dekker, New York, pp. 247-280. (Near infrared reflectance spectroscopy)

61

Underwater field spectroscopy references

Bukata et al. (1995) Optical properties and remote sensing of inland and coastal waters. CRC. Dekker, A. G., Brando, V.E., Anstee, J.M., Fyfe, S., Malthus, T.J.M. & Karpouzli, E. (2006) Remote sensing of seagrass ecosystems: use of spaceborne and airborne sensors, Chapter 15 in : Larkum, A,., Orth, B and Duarte, C. (eds) Seagrass Biology, Ecology and Conservation , Springer Verlag, Germany: pp 630. Dekker, A. G., V. E. Brando, J. M. Anstee, N. Pinnel, T. Kutser, H. J. Hoogenboom, R. Pasterkamp, S. W. M. Peters, R. J. Vos, C. Olbert, and T. J. Malthus, (2001), Imaging spectrometry of water, Ch. 11 in: Imaging Spectrometry: Basic principles and prospective applications: Remote Sensing and Digital Image Processing, v. IV: Dordrecht, Kluwer Academic Publishers, p. 307 - 359. Green E.P., Mumby P.J., Edwards A.J., Clark C.D. (Ed. A.J. Edwards) (2000). Remote Sensing Handbook for Tropical Coastal Management. Coastal Management Sourcebooks 3. UNESCO, Paris. 316 pp. Hooker, S.B. et al. (1994). Editor of SeaWIFS Technical Report series (available on the SeaWIFS web site) Jerlov (1976). Marine optics. Elsevier. Kirk (1994). Light and photosynthesis in aquatic ecosystems. CUP. Kutser, T., Dekker, A. G., Skirving, W. (2003) Modeling spectral discrimination of Great Barrier Reef benthic communities by remote sensing instruments. Limnology & Oceanography 48:497-510 Mobley, C. (1994). Light and water: Radiative transfer in natural waters. Academic Press. Mobley C., et al. (1993). Comparison of numerical models for computing underwater light fields. Appl. Optics, 32:7484-7504. Robinson, I.S. (1994) Satellite oceanography. Wiley

62

Note Pages

63