This article was downloaded by:[Universidad De Chile 2205] On: 14 July 2008 Access Details: [subscription number 791781823]

Publisher: Taylor & Francis Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

International Journal of Geographical Information Science

Publication details, including instructions for authors and subscription information: http://www.informaworld.com/smpp/title~content=t713599799

Comparison of the structure and accuracy of two land change models

Gil R. Pontius a; Jeffrey Malanson a a Clark University, Department of International Development, Community and Environment, Graduate School of Geography, Worcester MA 01610-1477, USA Online Publication Date: 01 February 2005 To cite this Article: Pontius, Gil R. and Malanson, Jeffrey (2005) 'Comparison of the structure and accuracy of two land change models', International Journal of Geographical Information Science, 19:2, 243 265 To link to this article: DOI: 10.1080/13658810410001713434 URL: http://dx.doi.org/10.1080/13658810410001713434

PLEASE SCROLL DOWN FOR ARTICLE Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf This article maybe used for research, teaching and private study purposes. Any substantial or systematic reproduction, re-distribution, re-selling, loan or sub-licensing, systematic supply or distribution in any form to anyone is expressly forbidden. The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.

International Journal of Geographical Information Science Vol. 19, No. 2, February 2005, 243265

Downloaded By: [Universidad De Chile 2205] At: 16:54 14 July 2008

Comparison of the structure and accuracy of two land change models

GIL R. PONTIUS* and JEFFREY MALANSON
Clark University, Department of International Development, Community and Environment, Graduate School of Geography, 950 Main StreetWorcester MA 016101477, USA; Tel: 508-793-7761; Fax: 508-793-8881; Email: rpontius@clarku.edu

This paper compares two land change models in terms of appropriateness for various applications and predictive power. Cellular Automata Markov (CA_Markov) and Geomod are the two models, which have similar options to allow for specification of the predicted quantity and location of land categories. The most important structural difference is that CA_Markov has the ability to predict any transition among any number of categories, while Geomod predicts only a one-way transition from one category to one alternative category. To assess the predictive power, each model is run several times to predict land change in central Massachusetts, USA. The models are calibrated with information from 1971 to 1985, and then the models predict the change from 1985 to 1999. The method to measure the predictive power: 1) separates the calibration process from the validation process, 2) assesses the accuracy at multiple resolutions, and 3) compares the predictive model vis-a ` -vis a null model that predicts pure persistence. Among 24 model runs, the predictive models are more accurate than the null model at resolutions coarser than two kilometres, but not at resolutions finer than one kilometre. The choice of the options account for more variation in accuracy of runs than the choice of the model per se. The most accurate model runs are those that did not use spatial contiguity explicitly. For this particular study area, the added complexity of CA_Markov is of no benefit. Keywords: Cellular Automata; Geomod; Land change; Markov; Model; Predict; Resolution; Scale; Validate

Introduction Which model to use? Models of land-cover & -use change (LUCC) vary enormously in terms of assumptions concerning: number of possible categories, types of category transitions, spatial dependency, feedbacks, cross-scale linkages, data requirements, etc. Some of these assumptions concern highly complex behaviours, such as interactions among agents. Other assumptions concern basic issues, such as the format of required data. Therefore, it can be difficult for a modeller to decide which model to use for a given location and time. If highly complex data are available for a study area, should all that data be used in a highly complex model, or should the data be simplified as would be required to run a simpler model? The answer ultimately depends on the purpose of the modelling. Whatever the purpose, the scientist usually wants the model to behave in a manner similar to the true system;
*Corresponding author. Email: rpontius@clarku.edu
International Journal of Geographical Information Science ISSN 1365-8816 print/ISSN 1362-3087 online # 2005 Taylor & Francis Ltd http://www.tandf.co.uk/journals DOI: 10.1080/13658810410001713434

244

R. G. Pontius Jr and J. Malanson

Downloaded By: [Universidad De Chile 2205] At: 16:54 14 July 2008

therefore the scientist would want to select a model that has predictive power. Hence, it is important to have a method to compare the accuracy of two models and of several runs for any particular model to see how the models structure relates to its accuracy. It can be a challenge to compare various runs of a single model, because a model can have a large number of options and implied assumptions that interact. Consequently, when one run is different than another run, it is not necessarily immediately clear which of the possible interactions among which assumptions is responsible for the difference. Comparison of runs from different models can be even more challenging, because it can be difficult to tell whether the difference in the runs is attributable to the models detailed assumptions or to the models more basic assumptions. LUCC scientists are likely to learn most efficiently when we examine the influence of the most basic assumptions before the highly-complex assumptions. To this end, this paper compares several runs of two models that are different in fundamental ways. Overview of two models The first model, Cellular Automata Markov (CA_Markov), allows any number of categories and can simulate the transition from any category to any other category. The second model, Geomod, uses exactly two categories and can simulate only the transition from the first category to the second category. That is, Geomod can not simulate an additional simultaneous transition of the second category to the first. In this respect, CA_Markov is more complex than Geomod. For some applications, the added complexity of CA_Markov may be desirable, for other applications the opposite may be true. Therefore, it is important to compare models head-to-head. In the process of comparing these two models, we illustrate a new generalized method to measure the predictive power of land change models. Markov-type models constitute some of the historically most common methods of predicting change among various categorical states. At their core is the mathematics of Markov chains, derived by the Russian Scientist Andrei A. Markov (1907). The first Markov models were not spatially explicit, but since the middle of last century, the Markov model has been used to simulate changes in maps (Baltzer et al. 1998). Spatio-temporal Markov chain (STMC) models can be used to model changes over time among categories in a landscape, whereby each pixel on the landscape is classified as exactly one category, and each pixel has some probability of transitioning to some other category at every time step. The Cellular Automata (CA) component of the CA_Markov model allows the transition probabilities of one pixel to be a function of neighbouring pixels. Baltzer (2000) gives many examples of how Markov-type models are used. Geomod is a newer land use change model that was originally designed to simulate the loss of tropical forests and to estimate the resulting carbon dioxide emissions (Pontius 1994). Pontius et al. (2001) offers the single most complete description of Geomod, which used to be known as Geomod2. Geomod predicts a one-way conversion from one category to one other category, similar to other popular land change models such as SLEUTH (Clarke et al. 1996, Clarke 1998). The Geomod approach has been used to simulate deforestation in Massachusetts (Schneider and Pontius 2001), Costa Rica (Hall 2000, Pontius 2002), India (Pontius and Batchu 2003, Pontius and Pacheco in press) and the tropics globally (Hall et al. 1995a, Hall et al. 1995b). It is rapidly gaining popularity in designing baseline

Comparison of two land change models

245

Downloaded By: [Universidad De Chile 2205] At: 16:54 14 July 2008

scenarios of tropical deforestation in the context of carbon offset projects called for by the Clean Development Mechanism of the Kyoto Protocol (Brown et al. 2002b). It is now being used in private businesses, non-governmental organizations, and research institutes (Brown et al. 2002c). The methods section of this paper describes the qualitative and mathematical details of each model, and then applies each model to a landscape in central Massachusetts, USA. Several runs of each model are compared, so naturally, there must be a criterion for comparison. The next two subsections of this introduction motivate the selection of a criterion, which the methods section then explains in technical detail. Modelling strategy of calibration and validation In land-use change modelling, there is no agreed upon criterion to assess the performance of one model versus another model, or to compare one run versus another run of the same model. This lack of a uniform method of assessment is related to the fact that there is not agreement among scientists concerning the purpose of land change modelling. Some scientists prefer models that express the theory of the mechanisms of the processes of land change, while others place more weight on a models ability to extrapolate the observed pattern of change based on past empirical patterns. It either case, most scientists would like for models to be able to simulate true patterns. Therefore, there is a need to quantify a models predictive power. In order to quantify the predictive power, scientists should examine the goodnessof-fit of the validation, which should not be confused with goodness-of-fit of calibration. Calibration is the process whereby the scientist uses information about the landscape to help select the parameters of the model. The information used for calibration should be at or before some specific point in time (t1), which is the point in time at which the predictive extrapolation begins. For example, CA_Markov models are typically calibrated with reference maps from two points in time, t0 and t1; whereas Geomod requires maps from only one point in time, t1. In both cases, the model then extrapolates land change beyond time t1 to some subsequent time t2. Validation is the process of comparing the models prediction for time t2 to a reference map of time t2, where the reference map is considered a much more accurate portrayal of the landscape at time t2. In order for the extrapolation to be a legitimate prediction, the information used in the calibration must have existed at or before t1. Pontius and Pacheco (in press) stress that a good fit of calibration does not necessarily imply a good fit of validation, and that the later is the appropriate indicator of a models predictive power. Mertens and Lambin (2000) and Brown et al. (2002a) are two of the few examples that we have been able to find that clearly distinguish calibration from validation. Rykiel (1996) and Oreskes et al. (1994) give a more detailed and philosophical discussion of validation. Criteria for accuracy assessment Separation of the calibration process from the validation process is necessary, but not sufficient, in order to compute the goodness-of-fit of validation. The scientist must also select a criterion to compare the predicted map of time t2 to the reference

246

R. G. Pontius Jr and J. Malanson

map of time t2. Perhaps the most popular criterion is the percent of pixels classified correctly. The percent correct is popular because of its ease of computation and apparent ease of interpretation; however a high level of sophistication should be used to interpret the percent correct. Most importantly, the interpreter must realize that a large percent correct does not necessarily imply that the model has good predictive power, due to a variety of reasons. Most importantly, it is common to attain a large percent correct from a null model that predicts pure persistence (i.e. no change) between time t1 and time t2, due to temporal autocorrelation between the reference maps of t1 and t2. In most of the land change modelling literature that we examined, the typical amount of change on the landscape was about 10%, so a null model of pure persistence would be 90% correct, which is a measure of accuracy that most na ve interpreters would consider high. Hagen (2002, 2003) is the only other author that we have found that articulates the need to compare a predictive model to a null model of pure persistence. The criterion of percent correct has additional complications. The percent correct is usually based on a pixel-by-pixel analysis, so it is possible that percent correct will not correspond to a visual assessment of the maps, because pixel-by-pixel analysis fails to consider spatial patterns of the pixels. For example, if a pixel is classified as the wrong category, then the entire pixel is in error, regardless whether the correct category is found in the neighbouring pixel or nowhere near the pixel (Pontius 2002). Lastly, a large portion of the percent correct can be attributable to random chance (Pontius 2000). Consequently, this paper uses the null resolution as the criterion for the goodness-of-fit of validation. The null resolution is the resolution at which the accuracy of the predictive model is the same as the accuracy of a null model, where accuracy is measured in terms of percent correct of the entire landscape. The finer the null resolution the more accurate and precise is the models prediction. The methods section describes the multiple-resolution procedures necessary to compute the models predictive power in terms of the null resolution. Methods Data The data for this paper derives from the State of Massachusetts Executive Office of Environmental Affairs, which makes GIS files available through the agency called MassGIS. Vector files of historic land use are available for three times: t051971, t151985, and t251999. Each map shows 20 categories. For each year, these categories are reclassified into two categories, built or non-built, based on the Anderson level 1 classification system (Anderson et al. 1976). For this study area, built consists mostly of Residential, and includes Commercial, Industrial, Transportation, Recreation and Waste Disposal. All of the maps are converted to raster format on a 30-by-30 meter grid of 1116 rows and 1008 columns, which contains 651,951 pixels in the study area. The study site consists of the town of Worcester and the nine adjacent towns in central Massachusetts. Worcester is a common example of a post-industrial American city that has undergone suburbanization over the last three decades. Recently, Worcester has become one of the nations fastest growing housing markets, as many people prefer to live in central Massachusetts as opposed to the

Downloaded By: [Universidad De Chile 2205] At: 16:54 14 July 2008

Comparison of two land change models

247

nearest major population centre, Boston, where the housing prices are approximately double what they are in the Worcester area. As a result, real estate developers are converting much of the land in the Worcester region to residential uses, which is causing tremendous concern for the existing residents of central Massachusetts. Figure 1 shows that from 1971 to 1985, approximately 3.7% of the landscape converted from non-built to built and a negligible amount converted from built to non-built. Figure 1 is the signal used to calibrate the model. Figure 2 is the true pattern used eventually to assess the accuracy of the models prediction through the validation process. Figure 2 shows that between 1985 and 1999, approximately 5.4% of the landscape converted from non-built to built and 0.8% converted from built to non-built, resulting in a net increase in quantity of built on the landscape of 4.6%. The legend of figure 3 shows the detailed categories of land use of 1971. Historic land use is important as a predictive factor of future change because historic land use is related to the ease with which land can be converted to built uses. For example, old agricultural land has a higher propensity to transition to built than does forest. This is probably related to the fact that agricultural land is already cleared and has a closer proximity to infrastructure such as roads and water services. Figure 4 shows the categories of legal protection that restrict the conversion of land to new built uses. In Massachusetts, legal restrictions are clearly important in determining which parcels of land convert to the built category. Unfortunately the map of legal restrictions does not specify the years in which the laws were enacted; therefore it is unclear whether the map of legal restrictions is legitimate for calibration, since some of the laws portrayed in the map were probably created

Downloaded By: [Universidad De Chile 2205] At: 16:54 14 July 2008

Figure 1. Change in built category from 1971 to 1985, which is the signal used for calibration.

248

R. G. Pontius Jr and J. Malanson

Downloaded By: [Universidad De Chile 2205] At: 16:54 14 July 2008

Figure 2. Change in built category from 1985 to 1999, which is the pattern used for validation.

Figure 3.

Land use of 1971, which is a driver used to calibrate the suitability maps.

Comparison of two land change models

249

Downloaded By: [Universidad De Chile 2205] At: 16:54 14 July 2008

Figure 4. Contemporary legal constraints to development, which is an optional additional driver used to calibrate the suitability maps.

subsequent to 1985. This paper uses the map of legal constraints in half of the model runs in order to demonstrate both the impacts that such constraints can have on land change predictions, and also to show the maximum level of accuracy attainable if we were to use this potentially illegitimate data for model calibration. Structural comparison CA_Markov and Geomod have some major qualitative differences in basic structure, and other minor differences in more subtle aspects. Table 1 states the differences between the models in general order of most fundamental to most detailed. The first two characteristics in table 1 show that CA_Markov can simulate two-way transitions among any number of categories, whereas Geomod simulates either a one-way gain or a one-way loss from exactly one category to one alternative category. In this sense, CA_Markov can theoretically model a wider variety of simultaneous processes. There are a suite of challenges and complications associated with simulating transitions among several categories. For example, many different categories might simultaneously compete to claim any particular pixel. To deal with this situation, CA_Markov implements a multiple objective land allocation algorithm. Geomod avoids these complications by assuming only one type of transition from one category to one other category. For many applications, the process of interest can be expressed in terms of a transition from a non-disturbed category to a disturbed category, in which case Geomods approach could suffice.

250

R. G. Pontius Jr and J. Malanson

Table 1. Qualitative characteristics of CA_Markov and Geomod.

Downloaded By: [Universidad De Chile 2205] At: 16:54 14 July 2008

Characteristic 1) 2) 3) 4) 5) Number of Categories Transitions Information for Quantities Quantity Propagation Suitability Map

CA_Markov Two or more Gains and Losses Computed from 2 maps Multiplicative Must be supplied for each transition Filter Not part of the design

Geomod Two Gain or Loss Must be supplied Additive Can be created from driver maps and a land-use map of 1 point in time Constraint Part of the design

6) Proximity Method 7) Stratification

The remaining characteristics of table 1 are grouped in terms of the two basic tasks that a model must perform. Specifically, the model must predict the quantity of each category and the model must predict the location of each category. Both CA_Markov and Geomod specify the quantity of the predicted categories independently from the location of the predicted categories. In table 1, characteristics 34 refer to the rules by which each model predicts the quantity of each category. Characteristics 57 refer to the rules by which each model predicts the location of each category. The sections below give the details of how each characteristic manifests in the example. The characteristics in table 1 are related closely to figure 5, which shows the specific options that this paper uses to model the land change for the example. The first option of figure 5 concerns which model to use: CA_Markov or Geomod. The next option concerns the method to extrapolate the quantity of each land category. The Markov approach extrapolates both the gain and loss of each category based on the proportion estimated from the calibration information. Geomod recommends linear extrapolation to predict the net quantity of the built category. The remaining options concern the decision rules that each model can use to determine the location of the predicted land change. One important decision concerns the factors to include in the creation of a suitability map. Another option concerns whether or not to implement a contiguity rule that encourages pixels of the same category to be adjacent. Yet another option concerns the time step, which is the shortest interval of time over which the model predicts change. This final option is relevant only if the contiguity rule is selected. The options shown in figure 5 give a range of the most important options available, but they are not exhaustive. For Geomod, an additional option allows for stratification of the analysis. For CA_Markov, an additional option allows for consideration of the accuracy of the reference data. The sections below discuss the details of each option. In total, this paper examines the 24 different model runs shown in figure 5 to extrapolate the change between 1985 and 1999, based on calibration from 1971 to 1985. CA_Markov Predicting Quantity of Change. CA_Markov predicts the quantity of each category at time t2 by extrapolating both gain and loss of each category from time t1. This extrapolation is calibrated by computing the proportional gain and loss between the years of calibration, which are t051971 and t151985 in our example. The

Comparison of two land change models

251

Downloaded By: [Universidad De Chile 2205] At: 16:54 14 July 2008

Figure 5.

Combinations of options for model runs.

extrapolated quantity for each category is a function of the transition matrix as dictated by the matrix algebra of Markov chains. Specifically, the Markov calculation assumes that for every 14-year time interval, the proportion of land that transitions from one category to another category is the same as the proportion of land that made that particular transition during the 14-year calibration period. Figure 6 shows this extrapolation where both the calibration interval and the extrapolation interval are 14 years. During the calibration interval, about 3.7% of the landscape shows gain in the built category and negligible loss of built. Therefore, over the subsequent 14 years, CA_Markov predicts gain of built and negligible loss of built. The open triangles of figure 6 show the true gross gain in built, whereas the closed triangles show the true net gain in built after the loss of built is considered. The open diamond shows the extrapolated gain of built, and the closed diamond below the open diamond indicates the net gain of built after the loss is taken into consideration. The extrapolation of quantity is computed independently of the decision of where to locate the change among categories on the landscape. Predicting Location of Change. There are two principles that determine the location where CA_Markov predicts land change. The first is based on the concept of a suitability map and the second is based on the concept of a contiguity rule. CA_Markov uses a suitability map for each transition that it extrapolates. At every time step, CA_Markov determines the number of pixels that must undergo each transition, then selects the pixels according the largest suitability for the particular transition. The user can generate the suitability maps in a variety of ways, for example by using a deductive approach such as Multi-Criteria Evaluation, or an inductive approach such as logistic regression. In our example, CA_Markov predicts

252

R. G. Pontius Jr and J. Malanson

Downloaded By: [Universidad De Chile 2205] At: 16:54 14 July 2008

Figure 6. Quantity of the built category as a percent of the landscape for the calibration and extrapolation intervals, where figure 6b zooms in on the validation year of figure 6a.

two transitions: 1) from built to non-built, and 2) from non-built to built. The suitability map of the transition from built to non-built is the inverse of the suitability map of the transition from non-built to built. The suitability map that indicates the transition from non-built to built is a map that Geomod creates automatically from the land cover map of 1985 and two factor maps: slope and land use of 1971. Therefore CA_Markov and Geomod use the same map of suitability for

Comparison of two land change models

253

Downloaded By: [Universidad De Chile 2205] At: 16:54 14 July 2008

the transition from non-built to built. The subsection on Geomod describes the detail of that suitability map (figure 7), including the option to use the map of legal constraints in the creation of the suitability map. The suitability maps remain static for the duration of the simulation. A contiguity rule is the second principle that CA_Markov uses to determine the location of predicted change. The contiguity rule usually has the effect of predicting the growth of a category near locations where that category already exists. At every time step, the suitability value in each candidate pixel is temporarily recalculated to show spatial dependence, such that the suitability for the transition to a particular category is influenced by whether that category exists in nearby pixels. If few of the nearby pixels belong to the category, then the suitability value is down-weighted. If many of the nearby pixels belong to the category, then the suitability value is maintained. The definition of nearby is determined by a spatial filter that the user specifies. Our example uses a filter which causes the suitability at a particular pixel to be weighted as a function of its nearest 24 neighbouring pixels, such that closer neighbours exert stronger influence on the calculation. This temporary recalculation is performed at each time step; therefore the duration of the time step can have some influence on the location of predicted land change when the contiguity rule is used. When the time step is small, the model must apply the time step for many iterations in order to achieve the desired duration of the extrapolation. Therefore, smaller time steps lead to more frequent applications of the filter, hence more frequent updates of the spatial dependence. If the user does not want to simulate spatial dependence

Figure 7. 1971.

Suitability for conversion from non-built to built, based on slope and land use of

254

R. G. Pontius Jr and J. Malanson

Downloaded By: [Universidad De Chile 2205] At: 16:54 14 July 2008

explicitly, then the most appropriate filter has a 1 in the centre surrounded by 0s. CA_Markov always assumes some spatial dependency, but this filter exerts the least possible spatial dependency, so when it is used, the selection of time step is negligible. Geomod Predicting Quantity of Change. Geomod has no explicit method to extrapolate the quantity of change from its one category to the alternative category, because the purpose of Geomod is to predict the location of a one-way transition. Nevertheless, Geomod must specify some quantity of change in order to make any prediction; therefore Geomod allows the user to predict any quantity, based on any extrapolation method that the user desires. We recommend a simple approach, unless it is clear that a more complicated approach is justified. Therefore, the example in this paper uses linear extrapolation of the net quantity of built, based on the line that is interpolated through the quantities at 1971 and 1985. Figure 6 shows that Geomods linear extrapolation of the quantity is nearly identical to CA_Markovs extrapolation of the gain and loss. The reason for the similarity of the extrapolation is that the percent of change that took place on the landscape during the calibration interval is small. Half of the Geomod runs use the quantity obtained from the linear extrapolation and the other half of the Geomod runs use the net quantity obtained from the Markov extrapolation (figure 5). This allows eventual comparison between the Geomod runs and the CA_Markov runs. Among all runs, there are three possible quantities: Markov, net Markov, and linear. Predicting Location of Change. There are four principles that determine the location where Geomod predicts land change. First, and most importantly, Geomod predicts persistence for the category that grows during the extrapolation. For the example, the built category grows in net quantity, therefore if a pixel is built in 1985, then Geomod predicts that it will remain built in 1999. This is different than CA_Markov and the true process, both of which show some loss of the built category. As a consequence, Geomod is doomed to fail to predict any real conversion from built to non-built, which is about 0.8% of the landscape as shown in figures 2 and 6. Next, Geomod has the ability to stratify the entire analysis, meaning that Geomod can specify the quantity in each stratum and calibrate the suitability map separately in each stratum. This is helpful because it is common that data are available by political unit, and that different political units experience different patterns of land change. Our example is not stratified in order to allow a more direct comparison with CA_Markov, which does not allow for stratification. Geomods last two principles that determine the predicted location of land change are similar to the two principles of CA_Markov. Specifically, Geomod can use a suitability map and/or a contiguity rule. Similar to CA_Markov, Geomod examines a suitability map to find the pixels with the largest suitability value, and then predicts conversion of new built at the non-built pixels that have the largest suitability for built. The suitability map is completely independent of the duration and the time step of the extrapolation, and the suitability map remains static for the duration of the simulation. Unlike CA_Markov, Geomod has a method to create empirically and automatically a suitability map based on several driver maps and a land cover map from a single point in time. In our example, the single land cover map is the

Comparison of two land change models

255

built versus non-built map of 1985. The driver maps are slope and land use of 1971. The information to compute the suitability map should be restricted to information that would have been available during the calibration interval, 19711985. The legend of figure 3 shows the various categories of land use of 1971, where the darker shade indicates a larger proportion of each category that is built in 1985. Categories that are pure black are categories that were built in 1971, of which more than 90% remained built in 1985. The non-black categories were non-built in 1971. For example, in the 1971 landscape, mining is a non-built category that transitioned to 25% built by 1985. The non-built categories of Cropland, Water Based Recreation, Pasture, and Woody Perennial are between 5% and 20% built in 1985. All other non-built categories experienced less than 5% conversion to built during the interval 19711985. Information of figure 3 is combined with information of a slope map to generate a map of suitability for the built category. In general, flatter slopes have a larger suitability for the built category. Figure 7 is the suitability map for conversion of non-built to built for both Geomod and CA_Markov. Pontius et al. (2001) discuss the details of the method to calibrate the suitability map from any number of driver maps. In the example, equal weights are applied to the slope driver and the 1971 land use driver. A map of the present-day legal restrictions on development is available (figure 4), so it could be used as another driver to create the suitability map. The date of the information in the map is not well documented, as is typical of freely-available data. Presumably many of the laws that regulate land conversion today, were in place in 1985. But several laws were probably created subsequent to 1985, which would render our map of legal restrictions illegitimate to predict change between 1985 and 1999. Nevertheless, the legal restrictions are probably one of the most important factors in determining the land conversion; therefore we perform the model runs both with and without the map of legal restrictions as a factor in order to measure the maximum additional predictive power that the information of legal restrictions could possibly offer. When the map of legal restrictions is used, it forces the suitability for conversion to the built category to be zero in any pixel that has any minimal level of legal restriction, meaning that any pixel that is classified as permanent, limited, or unknown is given a suitability value of 0. If the contiguity option is used, then Geomods search for the largest suitability values is constrained to the non-built pixels that are near existing built pixels. This option is useful for landscapes where growth of a land category occurs adjacent to that category. The definition of near can be set by the user to include pixels that are within any number of pixels of the edge of the two categories. Our example uses a 5-by-5 window in order to make the spatial dependency consistent with CA_Markovs spatial filter. The 5-by-5 window forces the predicted land change to be within two pixels of the edge including the diagonal. The definition of edge is updated at every iteration of the time step; therefore it is possible for the time step to have some influence on the location of the predicted landscape when the contiguity rule is used. If the extrapolation has small time steps, then Geomod can predict incremental growth at the locations with large suitability values. If the extrapolation has a large time step, then Geomod is forced to locate the change near the edge, with less consideration of the suitability values. Similar to CA_Markov, smaller time steps lead to more iterations and hence more frequent updates of the spatial dependency. If the contiguity rule is not used, then the time step has no influence on

Downloaded By: [Universidad De Chile 2205] At: 16:54 14 July 2008

256

R. G. Pontius Jr and J. Malanson

Downloaded By: [Universidad De Chile 2205] At: 16:54 14 July 2008

the predicted results. Figure 8 gives an example of the output of a Geomod run, which is overlaid with maps of the real landscape for 1985 and 1999. The red and green in figure 8 show the locations where Geomod simulated additional built land between 1985 and 1999. We also consider three model runs that distribute the predicted quantity of change evenly across the landscape. According to the mathematics of statistical expectation, the accuracy of these runs is the accuracy that one would expect from a model that distributes the change at randomly selected locations. The three runs correspond to the three different methods to predict the quantity of change: Markov, net Markov, and linear. The net Markov and linear quantities give an increase in the built category, whereas the Markov quantities give simultaneous gain and loss of the built category. Goodness-of-fit Percent correct. Two criteria measure the goodness-of-fit of each model run. The first criterion is the percent of pixels classified correctly on the entire landscape at the resolution of the raw data, which are 30-by-30 meter pixels. The percent correct criterion is extremely popular and frequently misinterpreted. This paper shows how to interpret the percent correct in an appropriate and sophisticated manner. There are two crucial issues that a scientist should consider when using the percent correct criterion. First, the scientist should compare the percent correct of the predictive model to the percent correct of a null model that predicts pure persistence. In all cases that we have seen, a null model of pure persistence gives a larger percent correct than a predictive model at the resolution of the raw data. This indicates that the resolution of the raw data is finer than the resolution at which the simulation model can accurately predict the process of change. Second, the scientist should examine the percent correct at multiple-resolutions in a way similar to Costanza (1989) and Pontius (2002), whereby a pixel aggregation procedure simply averages neighbouring pixels into coarser pixels in order to quantify the agreement between the maps at coarser resolutions. As the resolution of the accuracy assessment becomes coarser, the percent correct tends to rise. The percent correct

Figure 8. Overlay of the real 1985, the real 1999, and the simulated 1999 for the model run that is most accurate. The two shades of gray and the red are classified correctly by the simulation model, whereas the blue, green and yellow are errors by the simulation model. A null model would correctly predict the green and the two shades of gray.

Comparison of two land change models

257

increases rapidly at resolutions that correspond to distances at which the model is accurate. At the coarsest resolution for which one extremely coarse pixel contains the entire study area, the only error in the accuracy assessment is the error due to misspecification of quantity. Figures 9 and 10 show this phenomenon for a null model of pure persistence and the best model prediction respectively. In each figure, the reference map for 1999 is compared to the modelled map of 1999. The horizontal axis shows resolution changing from fine to coarse as one moves from left to right. The vertical axis is the percent of the entire landscape. The figures show three components of agreement (agreement due to chance, agreement due to quantity, and agreement due to location) and two components of disagreement (disagreement due to location and disagreement due to quantity). The border that separates the agreement due to location from the disagreement due to location is the percent correct. At the coarsest resolution, location no longer has meaning, so there are no components associated with location and the only error is due to quantity. Readers who are especially interested in the details of the calculations should read Pontius (2000), Pontius (2002) and Pontius and Suedmeyer (in press). The change in resolution is accomplished by aggregating neighbouring pixels at the fine resolution into coarser pixels. The aggregation rule computes the aggregate membership of each category for each coarse pixel as the proportion of fine resolution pixels of each category that constitute the coarse pixel. Each additional progression in the aggregation process averages the membership of four neighbouring pixels, therefore the size of the resolutions progress as a geometric sequence, where the side of a pixel in each additional resolution is twice the length of the side of the pixels at the previous resolution. Null resolution. The second criterion to assess a model run is the null resolution. Figure 11 shows how to determine the null resolution, which is the resolution at

Downloaded By: [Universidad De Chile 2205] At: 16:54 14 July 2008

Figure 9. Components of agreement and disagreement between real 1985 map and real 1999 map at multiple resolutions.

258

R. G. Pontius Jr and J. Malanson

Downloaded By: [Universidad De Chile 2205] At: 16:54 14 July 2008

Figure 10. Components of agreement and disagreement between best predicted 1999 map and real 1999 map at multiple resolutions.

which the percent correct of the predictive model crosses the percent correct of the null model. Figure 11 has the same axes as figures 9 and 10, since figure 11 shows an overlay of the crucial information in figures 9 and 10. In figure 11, the solid line with the circles (or triangles) is the percent correct for the predictive model (or null model) respectively. The dashed line with the circles (or triangles) shows the error due to quantity for the predictive model (or null model) respectively. The solid lines

Figure 11. Illustration of the method to determine that the null resolution is 32 times the length of the raw pixel side.

Comparison of two land change models

259

Downloaded By: [Universidad De Chile 2205] At: 16:54 14 July 2008

approach the dashed lines as resolution becomes coarser, and reach the dashed lines by at least the coarsest possible resolution, which is the resolution at which one coarse pixel contains the entire study area. At fine resolutions, the percent correct for the predictive model is less than the percent correct for the null model. As resolution becomes coarser, the percent correct for both the predictive model and the null model rises, however the percent correct for the null model is constrained by its relatively large error due to quantity. Since the predictive model has a smaller error due to quantity, the percent correct for the predictive model becomes larger than the percent correct for the null model at some resolution, which is called the null resolution. At resolutions finer than the null resolution, the predictive model is less accurate than the null model of pure persistence. At resolutions coarser than the null resolution, the predictive model is more accurate than the null model of persistence only. More accurate models have finer null resolutions because the null resolution is the resolution at which the predictive model starts to be more accurate than a null model. For example, figure 11 shows a null resolution of 32 times the size of the side of a pixel of the raw data. Results Figure 8 displays the most important information of the best simulation run, which is 92% correct at the 30-meter resolution. Figure 8 overlays the 1985 truth map, the 1999 truth map, and the 1999 simulated map of a Geomod run that uses net Markov quantities, considered laws, and is unconstrained. The two shades of gray and the red are classified correctly by the simulation model, but the gray would be classified correctly also by a null model that predicts persistence only. Blue, green and yellow are errors by the simulation model, but the green locations would be predicted correctly by a null model. Both the null model and the simulation model fail to predict the blue and yellow. Figure 8 shows that the null model has a higher percent correct than the best simulation run because there is more green than red. Figure 12 plots the null resolution versus the percent correct at the 30-by-30 meter resolution for all 24 model runs. Among all runs, the percent correct at the finest resolution ranges from 91.06 to 92.15. These accuracies are less than the percent correct of the null model, which is 93.75, but are greater than the accuracies of the runs that distribute the change evenly across the landscape, which range from 90.75 to 90.91 percent correct. The null resolution for nearly all runs is 32 times the resolution of the raw 30-meter data, which corresponds roughly to a 1-kilometer resolution. Four of the runs have a null resolution of approximately 2 kilometres. Figure 12 shows that the model runs are grouped into distinct clusters that are based more on the options for each model and less on the selection of CA_Markov versus Geomod. The cluster that has the largest percent correct (ranging from 91.87 to 92.15 percent correct) contains the runs that do not use a contiguity rule. These runs allow predicted change to occur at the largest suitability values, regardless of whether those locations are near existing built areas. Within this most accurate cluster, the runs with the finer null resolution of 32 tend to be from Geomod and to use the laws in the suitability map. Among the runs that use the contiguity rule, there are three clusters. Among these three clusters, the one with the largest percent correct (ranging from 91.52 to 91.71 percent correct) contains all the constrained Geomod runs. Among these constrained Geomod runs, the ones that predict a smaller quantity of change have a larger percent correct. The filtered CA_Markov runs are organized in two sub-clusters. The first of these sub-clusters (ranging from

260

R. G. Pontius Jr and J. Malanson

Downloaded By: [Universidad De Chile 2205] At: 16:54 14 July 2008

Figure 12. Null resolution versus percent correct at finest resolution for all model runs. Diamonds denote Geomod, squares denote CA_Markov, triangles denote smooth distribution of location. Gray denotes Markov quantities, black denotes linear quantities. Small symbols denote that laws are considered, large symbols denote laws are ignored. Unfilled symbols denote no contiguity rule, filled symbols denote contiguity rule.

91.30 to 91.39 percent correct) contains the CA_Markov filtered runs that have a one 14-year time step. The other sub-cluster (ranging from 91.06 to 91.10 percent correct) contains the CA_Markov filtered runs that have fourteen 1-year time steps. Within these sub-clusters, the runs that predict the smaller quantity of change, give greater accuracy. In all cases, the use of the laws improves the predictive power of the extrapolation. The laws account for variation of percent correct within the clusters, but not variation among the clusters. The cluster in the far left of figure 12 shows the statistically expected results from runs that distribute the change randomly. The null resolution of the random runs is 32. In terms of percent correct, all the CA_Markov and Geomod runs perform better than the random runs, but they also perform worse than the null model. Furthermore, none of the predictions were more accurate in spatial location than would be expected by random chance. The range in percent correct among the CA_Markov and Geomod runs is about 1 percentage point. This should be viewed in the context that the range in percent correct from the random models to the null model is 3 percentage points. A visual assessment of the various prediction maps shows that the percent correct corresponds with the pattern agreement, meaning that the larger the percent correct the more similar the spatial pattern. Most of the variation in the visual assessment is attributable to the contiguity rule. Discussion Interpretation of Results Percent correct. For any particular modelling application, it is a challenge to separate the results that are true in general from those that true for only a specific

Comparison of two land change models

261

landscape. We attempt to interpret our results in terms of general concepts that are likely to apply to other landscapes and other models. First, it is extremely important that the modeller be able to control the contiguity rule. The behaviour of the contiguity rule explains the differences among the clusters and sub-clusters in Figure 12. The cluster with the largest percent correct uses no principles of contiguity, meaning that it does not force the prediction of new built to be near existing built. The next most accurate cluster is the unconstrained Geomod runs. The effect of the contiguity rule in Geomod is weaker than in CA_Markov. Geomods contiguity rule merely restricts the candidates for new Built to be near the existing Built, then Geomod selects the pixels based on the suitability map. CA_Markovs contiguity rule applies a spatial filter to the entire suitability map, with strong weighting towards predicting new built to be near existing built. This filter rule is applied at every time step. As a result, there are two distinct sub-clusters in figure 12 for the constrained CA_Markov runs. The least accurate cluster is the one where there are fourteen 1-year time steps, as the filter is applied at each of the fourteen steps. Some models assume that land change can be predicted accurately using spatial dependency as expressed in a contiguity rule. However, a quick glance at figures 2 and 3, show clearly that new built areas do not grow from existing built areas. Therefore, any model that forces spatial dependency is doomed on such a landscape. This is an important observation because the contiguity assumption is hard-coded into some LUCC models. For other landscapes at other scales, the best predictor of the location of new development may indeed be the proximity to existing development (Mertens and Lambin 2000, Pontius and Batchu 2003, Pontius and Pacheco in press), but the Massachusetts landscape is an example for which the land change occurs in entirely new patches, not connected to existing patches (Schneider and Pontius 2001). Therefore, it is important that models have an option to control the level of spatial dependency. Both CA_Markov and Geomod have such an option. Second, an important basic principle is that when the model predicts change, it usually predicts it in the wrong location with respect to the fine resolution, therefore models that predict a small quantity of change tend to have a higher percent correct than models that predict a larger quantity of change. This phenomenon explains the variation in accuracy within the clusters. There are three quantities in this analysis. They are in order of smallest quantity of change to largest: net Markov, Linear, and Markov. In all cases, the accuracy within clusters follows this ordering, where net Markov is most accurate and Markov is least accurate. In no cases did the small 1-year time step improve the accuracy beyond the level attained by the corresponding run that has one 14-year time step. This supports Ockhams razor in terms of both accuracy and CPU time. Null resolution. Nearly all of the runs have a null resolution of 32 times the 30-meter pixel side, which is about 1 kilometre. The only exceptions are four of the unconstrained runs, which have a null resolution of about 2 kilometres. It makes sense that the unconstrained runs can have coarser null resolutions because those runs allow predicted change to occur far from existing built locations, since the predictions are unconstrained in space. All the models predictions are less accurate than a null model at distances less than 1 kilometre. But at distances greater than 2 kilometres, all the models predictions are more accurate than a null models prediction.

Downloaded By: [Universidad De Chile 2205] At: 16:54 14 July 2008

262

R. G. Pontius Jr and J. Malanson

Downloaded By: [Universidad De Chile 2205] At: 16:54 14 July 2008

The model runs that distribute the change smoothly or randomly across the landscape also have a null resolution of 32. Therefore, there are no scales at which any of the model predictions are simultaneously more accurate than both a null prediction and random prediction. We suspect that this characteristic is typical of the accuracy of contemporary LUCC models. We know of no spatially explicit models that have been shown to predict for any landscape better than both a null model and a random model at any spatial resolution. Signal versus Noise The results show that it is important to focus on the most important signals of change on the landscape, as opposed to modelling the detail of the noise. In our example, the major signal is persistence, with a modest amount of gain of the built category arranged in patches distributed widely across the landscape. The some of the noise is the small patches of transition from built to non-built. The predictive power of the runs is higher for the cases where the model focuses on the signal and ignores the noise. For example, Geomod ignores the small conversion from built to non-built, whereas CA_Markov attempts to model the weak empirical pattern of conversion from built to non-built. When CA_Markov tries to predict this rare phenomenon, it usually predicts incorrectly. Consequently, Geomod gives more accurate results by ignoring the rare phenomenon. The importance of focusing on the signal and ignoring the noise is especially crucial in light of the quality of the data. Noise derives from both unsystematic trivial change on the landscape and error in the maps. We do not know exactly how much error there is in the data because proper metadata for the maps does not exist and a complete error assessment of the maps was not done by the map producer. It could be that the small conversion from built to non-built is primarily error. In this case, it is wise to ignore this transition. Models that attempt to reproduce all the fine detail in the calibration data run the risk of modelling the error, which would be counter-productive. Scientists need to improve our understanding of how map error influences the level of certainty we should have in models (Foody and Atkinson 2002). For now, one necessary step is to apply rigorous procedures that separate calibration information from validation information. When scientists fail to distinguish between the calibration process and the validation process, there is a temptation to refine the model parameters until the model reproduces the noise in the data. This flawed approach can cause a models output to be a close match with the reference maps, but the apparent accuracy is a result of modelling the noise, i.e. of over-fitting the model. Such a practice can lure scientists into thinking that a model has a greater predictive power than it really does. Importance of goodness-of-fit analysis Techniques to measure the goodness-of-fit of validation are the least sophisticated tools in the standard toolbox of the contemporary land change modeller. In the past, scientists assessed the accuracy of models by a qualitative visual assessment; more recently modellers have been looking for more sophisticated measures (Foody 2002). Intuitively, modellers know that a na ve interpretation of percent correct is not useful. This paper offers a sophisticated way to interpret a simple, familiar measure. The temptation among many scientists is to restrict the accuracy assessment to some subset of pixels, for example to only the pixels that are non-built at the

Comparison of two land change models

263

Downloaded By: [Universidad De Chile 2205] At: 16:54 14 July 2008

beginning of the extrapolation, or to only the pixels that truly change during the time interval of the extrapolation. We recommend strongly against the exclusion of any pixels from the accuracy assessment and advise in favour of including the entire landscape in the accuracy assessment. It is best to consider the entire landscape and compare the predictive model to a null model of pure persistence. Exclusion of pixels from the accuracy assessment can make the scientist blind to problems in the model. For example, Geomod predicts only a one-way change from non-built to built, which is a characteristic that some scientists might consider a weakness. If the accuracy assessment considers only the pixels that were non-built at the beginning of the extrapolation, then the accuracy assessment would overlook this potential problem. Moreover, if the accuracy assessment considers only those pixels that truly change during the time interval of the extrapolation, then there is an incentive for the model to predict change everyplace, so as to maximize the predicted change in the pixels that truly did change. The important message is that if a scientist wants to gain insight into the dynamics of the entire landscape then (s)he should consider the entire landscape in the accuracy assessment, and if (s)he would like to know the usefulness of a model, then (s)he should compare its predictions to the predictions of a null model. Specifics of the data Additional investigation is required to tell whether our results are representative of a wide variety of applications. This paper uses an example from one place in space and time, specifically central Massachusetts during the end of last century. For this landscape over this duration, most of the change is a steady one-way increase in the built category. If the same models are applied to different landscapes at different times, the conclusions might be quite different, but the appropriate methods for analysis would be the same. Some landscapes are more dynamic and complex than others, and the apparent complexity is related in part to the detail of the available data. Scientists should be aware that available data is usually much more precise in spatial resolution and number of categories than is the precision of a models predictive power, so the modeller should not be tempted to let the availability of complex data be the major criterion upon which to select a model. In our example, CA_Markov is poor at predicting the location of conversion of built to non-built. Geomod was more accurate by following a strategy of ignoring the transition from built to non-built, even though the reference data was sufficiently detailed to show this rare phenomenon. Conclusions The most important difference between Geomod and CA_Markov is that Geomod predicts only a one-way transition from one category to one alternative category, whereas CA_Markov has the ability to predict any transition among any number of categories. Therefore if the important dynamics of a landscape involve simultaneous gain or loss of a category or involve substantial interactions among more than two categories, then it would seem that CA_Markov is the better choice based on qualitative characteristics. Alternatively, it might be advisable to simplify the base data in order to focus on the major signal of change, which frequently is the conversion from a single non-disturbed category to a single disturbed category,

264

R. G. Pontius Jr and J. Malanson

Downloaded By: [Universidad De Chile 2205] At: 16:54 14 July 2008

especially in the case of urban landscape modelling. If the goal is to predict a single dominant process, then Geomod can also be a good choice. Besides this major conceptual difference, both Geomod and CA_Markov are similar in terms of the available options that allow for specification of the predicted quantity and location of categories on a landscape. In our example, the choice of the options accounted for more variation in accuracy of model runs than the choice of the model. Whatever model is used, the methods of this paper allow a scientist to assess the accuracy of any model run in a sophisticated manner that gives useful information for model refinement. The most important aspects of assessing the predictive power of a model are: 1) to separate the calibration process from the validation process, 2) to assess the predictive model vis-a ` -vis a null model, and 3) to perform the accuracy assessment at multiple resolutions. Acknowledgements This research was made possible through the HERO program that NSF supports through the grant Infrastructure to Develop a Human-Environment Regional Observatory Network (Award ID 9978052). We acknowledge also programmer Hao Chen, who was funded by the Center for Integrated Study of the Human Dimensions of Global Change. This Center has been created through a cooperative agreement between the National Science Foundation (SRB-9521914) and Carnegie Mellon University, and has been generously supported by additional grants from the Electric Power Research Institute, the ExxonMobil Foundation, and the American Petroleum Institute. The authors thank also Clarklabs, which has made the methods of this paper available in the GIS software Idrisi. Literature
ANDERSON, J.R., HARDY, E.E., ROACH, J.T. and WITMER, R.E., 1976, A Land Use And Land Cover Classification System For Use With Remote Sensor Data. Geological Survey Professional Paper 964. BALTZER, H., 2000, Markov chain models for vegetation dynamics. Ecological Modeling, 126, pp. 139154. BALTZER, H., BRAUN, P.W. and KOHLER, W., 1998, Cellular automata models for vegetation dynamics. Ecological Modeling, 107, pp. 113125. BROWN, D.G., GOOVAERTS, P., BURNUCKI, A. and LI, M.-Y., 2002a, Stochastic Simulation of Land-Cover Change Using Geostatistics and Generalized Additive Models. Photogrammetric Engineering & Remote Sensing, 68 (10): pp. 10511061. BROWN, S., SWINGLAND, I., HANBURY-TENISON, R., PRANCE, G. and MYERS, N., 2002b, Changes in the use and management of forests for abating carbon emissions: issues and challenges under the Kyoto Protocol. Phil. Trans. R. Soc. Lond. The Royal Society, 360, pp. 113. BROWN, S., HALL, M., RUIZ, F., FLAMENCO, A., DEJONG, B., AUKLAND, L., MASERA, O. and SHOCH, D., 2002c, http://www.winrock.org/general/Publications/ LandUseAndForestryProjects.pdf. CLARKE, K.C., 1998, Loose-coupling a cellular automaton model and GIS: long-term urban growth prediction for San Francisco and Washington/Baltimore. International Journal of GIS: 12, 17, pp. 699714. CLARKE, K.C., HOPPEN, S. and GAYDOS, L.J., 1996, A self-modifying cellular automaton model of historical urbanization in the San Francisco Bay area. Environment and Planning B, 24, pp. 247261. COSTANZA, R., 1989, Model goodness of fit: a multiple resolution procedure. Ecological Modelling, 47, pp. 199215.

Comparison of two land change models

265

FOODY, G., 2002, Status of land cover classification accuracy assessment. Remote Sensing of Environment, 80, pp. 185201. FOODY, G. and ATKINSON, P., (Eds.)., 2002, Uncertainty in Remote Sensing and GIS. (West Sussex, England: John Wiley & Sons), 307 pages. HAGEN, A., 2002, Multi-method assessment of map similarity. 5th AGILE Conference on Geographic Information Science, Palma, Spain. April 2527; 171182. HAGEN, A., 2003, Fuzzy set approach to assessing similarity of categorical maps. International Journal of Geographic Information Science, 17 (3): pp. 235249. HALL, C.A.S. (Ed.)., 2000, Quantifying Sustainable Development: The Future of Tropical Economies. (San Diego, CA: Academic Press), 761 pages. HALL, C.A.S., TIAN, H., QI, Y., PONTIUS, G. and CORNELL, J., 1995a, Modelling spatial and temporal patterns of tropical land use change. Journal of Biogeography, 22 (4/5): pp. 753757. HALL, C.A.S., TIAN, H., QI, Y., PONTIUS, G., CORNELL, J. and UHLIG, J., 1995b, Spatially Explicit Models of Land Use Change and Their Application to the Tropics, Carbon Dioxide Information Analysis Center, Oak Ridge National Laboratory, DOE Research Summary 31, February. MARKOV, A.A., 1907, Extension of the limit theorems of probability theory to a sum of variables connected in a chain. The Notes of the Imperial Academy of Sciences of St. Petersburg, VIII Series, Physio-Mathematical College XXII. MERTENS, B. and LAMBIN, E.F., 2000, Land-Cover-Change Trajectories in Southern Cameroon. Annals of the Association of American Geographers, 90 (3): pp. 467494. ORESKES, N., SHRADER - FRECHETTE, K. and BELITZ, K., 1994, Verification, Validation, and Confirmation of Numerical Models in the Earth. Science, 263 (5147): pp. 641646. PONTIUS JR, R.G., 1994, Modeling tropical land use change and assessing policies to reduce carbon dioxide release from Africa. Graduate Program in Environmental Science, Syracuse: SUNY-ESF, pp. 177. PONTIUS JR, R.G., 2000, Quantification error versus location error in comparison of categorical maps. Photogrammetric Engineering & Remote Sensing, 66 (8): pp. 10111016. PONTIUS JR, R.G., 2002, Statistical methods to partition effects of quantity and location during comparison of categorical maps at multiple resolutions. Photogrammetric Engineering & Remote Sensing, 68 (10): pp. 10411049. PONTIUS JR, R.G., CORNELL, J.D. and HALL, C.A.S., 2001, Modeling the spatial pattern of land-use change with GEOMOD2: application and validation for Costa Rica. Agriculture, Ecosystems & Environment, pp. 191203. PONTIUS JR, R.G. and BATCHU, K., 2003, Using the Relative Operating Characteristic to quantify certainty in prediction of location of land cover change in India. Transactions in GIS, 7 (4): pp. 467484. PONTIUS JR, R.G. and PACHECO, P., 2003, Calibration and validation of a model of forest disturbance in the Western Ghats, India 19201990. GeoJournal, in press. PONTIUS JR, R.G. and SUEDMEYER, B., 2003, Components of agreement between categorical maps at multiple resolutions. In R.S. Lunetta and J.G. Lyon, (Eds). Remote Sensing and GIS Accuracy Assessment. (Boca Raton, FL: CRC Press). RYKIEL, E.J., 1996, Testing ecological models: the meaning of validation. Ecological Modeling, 90, pp. 229244. SCHNEIDER, L.C. and PONTIUS JR, R.G., 2001, Modeling land-use change in the Ipswich watershed, Massachusetts, USA. Agriculture, Ecosystems & Environment, 85, pp. 8394.

Downloaded By: [Universidad De Chile 2205] At: 16:54 14 July 2008