Urban Studies, Vol. 41, No.

4, 757–772, April 2004

An Indicator-based Approach to Measuring

Sustainable Urban Regeneration Performance:
Part 2, Empirical Evaluation and Case-study
Analysis

Lesley Hemphill, Stanley McGreal and Jim Berry

[Paper first received, January 2003; in final form, October 2003]

Summary. This paper examines the sustainability of current urban regeneration practice,
through the application of weighted indicators and a points scoring framework. The analysis
applies the hierarchical model discussed in the preceding paper of this journal issue to case
studies of waterfront areas and cultural quarters in three European cities: Belfast, Dublin and
Barcelona. The evaluation permits performance comparisons to be made between the case studies
regarding the sustainability of regeneration areas and projects, variations on an indicator set
basis and the sensitivity of scores. Conclusions are drawn concerning regeneration practice, the
extent to which sustainability principles are adhered to, potential policy benefits and the
applicability of the model.

Introduction
Sustainable development and urban regener- sustainable development, the role of urban
ation have become increasingly recognised regeneration in fulfilling sustainability objec-
as complementary goals, but there remains tives remains in its infancy.
only limited evidence of performance evalua- Sustainable Development: The UK Strat-
tions that investigate the extent to which egy (DOE, 1994) recognised the importance
regeneration projects have achieved sustain- of urban regeneration in contributing to sus-
able outcomes (Carley and Kirk, 1998; tainability by developing areas in an efficient
DETR, 1998a; OECD, 2000). Likewise, the way whilst making them more attractive
dissemination of what constitutes ‘good sus- places in which to live and work. In a similar
tainability practice’ as discussed in the ac- vein, the Urban Task Force in its “Towards
companying paper and how this can be an Urban Renaissance” report (DETR,
benchmarked remains underinvestigated. 1999b) advocated the need for a
What is becoming increasingly obvious is the “comprehensive package of regeneration
prominent place given in UK policy to the measures to address both the physical regen-
issue of sustainability and ultimately how eration of an area and the economic and
this can be best achieved (DETR, 1998b, social needs of the local population” (p. 133).
1998c, 1998d, 1999a, 1999b, 2000). How- This is further supported by the UK govern-
ever, despite this policy drive to instigate ment’s Urban White Paper Our Towns and
Lesley Hemphill, Stanley McGreal and Jim Berry are in the Centre for Research on Property and Planning, School of the Built
Environment, University of Ulster, Shore Road, Newtownabbey, County Antrim, Northern Ireland, BT37 0QB. Fax: 028 9036 6826.
E-mail: la.hemphill@ulster.ac.uk; ws.mcgreal@ulster.ac.uk and jn.berry@ulster.ac.uk.
0042-0980 Print/1360-063X On-line/04/040757–16  2004 The Editors of Urban Studies
DOI: 10.1080/0042098042000194098
758 LESLEY HEMPHILL ET AL.
Cities: The Future—Delivering an Urban
Renaissance (DETR, 2000) which identified
bringing brownfield land and buildings back
into constructive use as a key objective to
exploit their potential in contributing to the
quality of urban life. The Sustainable Devel-
opment Strategy for the UK: A Better Quality
of Life (DETR, 1999a) lists regeneration as
one of a number of aspects contributing to
sustainable development whilst indicating
that building sustainable communities re-
mains a priority for government action. How-
ever, Imrie and Thomas (1995) consider that
the proliferation of new urban policy initia-
tives since the early 1980s has not been
matched by systematic monitoring and evalu-
ation. Likewise, Turok (1989) states that the
scope and form of evaluative research have
often been illogical, weak and increasingly
concerned with a narrow range of quantitat-
ive evaluative criteria. Such a focus on the
quantitative measurement of policy impacts
has failed to recognise the importance of the
subjective, experiential ‘life-world’ of human
well-being (Burns, 2000). Indeed, it is this
failure to appreciate the human aspects of
urban policy through the use of purely quan-
titative methods of evaluation that has often
led to the disquiet expressed over urban pol-
icy evaluation.
Hakim (2000) argues that too often the
secondary analysis of an existing data-set is
proposed at the expense of qualitative re-
search. Similarly Wong (2002) points to-
wards the need for more qualitative research
to measure satisfactorily intangible issues,
such as community identity and institutional
capacity. Perhaps of greater significance is
the potential to combine both quantitative
and qualitative techniques within any given
evaluation to ensure that their respective
strengths can be capitalised on. Indeed, qual-
itative research can be used in combination
with virtually all other types of study, in-
forming the interpretation of more imper-
sonal statistical data (Hakim, 2000).
The research reported in this article builds
upon the concepts developed in the preced-
ing paper in this issue and draws upon a
combination of techniques as a means of
gathering data appropriate for the indicators.
Expert opinion plays a major part in the
empirical investigation which seeks to assess
the measurement of urban regeneration prac-
tice in light of sustainability principles. A
detailed performance analysis of six indepen-
dent case studies is presented utilising the 52
indicators discussed in the accompanying pa-
per. The performance of each case-study area
is compared for each of the five indicator
groups in the model and against an overall
aggregated score to provide an indication of
relative performance. Sensitivity analysis is
applied to each indicator to assess different
options. The paper provides the basis for the
use of the model for evaluation and policy
purposes, notably in the identification of the
issues where present urban regeneration
practice is not fully embracing the sustain-
ability agenda. Conclusions are drawn on the
respective case-study performances and the
successes or shortcomings of present regen-
eration policy in creating sustainable out-
comes, potential policy benefits and
applicability of the model.
Sustainable Regeneration Performance
Benchmarking
The subject of measuring sustainability per-
formance has been assessed in detail in the
accompanying paper; however, an equally
important aspect of measurement is how to
benchmark this performance. Benchmarking
can be considered an effective means of
helping to deliver better services by compar-
ing performance against an accepted norm
and learning from other areas/organisations
(Audit Commission, 2000). This is all the
more evident in today’s society with headline
(DETR, 1999a) and best value (ODPM,
2002) indicators providing two examples of
circumstances where indicators have been
used to measure and subsequently bench-
mark performance. Benchmarking enables
the comparison of quantitative performance
data, identifies how performance differs from
other areas, how it has changed over time
and whether it can act as the spur for im-
 MEASURING URBAN REGENERATION, PART 2
provement or an example of how to achieve
‘good practice’ (Audit Commission, 2000).
Practitioners need to be able to establish
the baseline economic and social conditions
against which they can measure the progress
and improvement resulting from regeneration
projects (Bristow, 1999). Likewise, it is ac-
knowledged that local authorities and devel-
opers require practical tools to measure the
sustainability of developments (both build-
ings and infrastructure) at site or estate level
(Brownhill and Rao, 2002) to ensure that
they meet planning requirements and contrib-
ute to wider sustainability objectives. Simi-
larly, a crucial part of benchmarking and the
use of indicators is to help guide appropriate
action through the setting of suitable targets
(European Commission DGXI, 1996). The
Audit Commission (2000) identifies the key
elements of the benchmarking process—
namely, select service/project, identify re-
sources, identify partners, define and collect
measures, compare performance, find best
practice/targets, plan change, implement and
monitor outcomes.
This paper adopts a similar benchmarking
strategy to the measurement of sustainable
urban regeneration and utilises the model
developed and discussed in this journal issue.
Although the points scoring parameters
reflect current legislative or policy standards,
it is important to stress the dynamic charac-
teristics of the model and its flexibility to
permit change as and when new technology
or information renders current standards as
insufficient to meet sustainability require-
ments.
Evaluative Model Overview
The preceding paper provides a detailed re-
sumé of the methodology underpinning the
research identifying a four-phase process
(Figure 1). This paper deals primarily with
the application of the evaluative model at
case-study level. In particular, the focus is
upon phase four—the case-study application
and sensitivity analysis. The model is based
on 52 individual indicators which have been
weighted into five main groups: Economy
759
and work (21.5 per cent); Resource use (17.5
per cent); Buildings and land use (18.9 per
cent); Transport and mobility (22.1 per cent);
Community benefits (20 per cent). Further
discussion of the indicators, the points scor-
ing framework and how they were selected
and tested is provided in the accompanying
paper.
Case-study Selection
In testing the validity of the model and seek-
ing to achieve a meaningful set of results,
rigorous selection criteria were drawn up to
ensure that the same basic characteristics
were present in each case-study area thereby
enabling a full comparative evaluation be-
tween individual schemes. These criteria in-
clude: previous track record of the case-study
area, the hybrid of contacts available, indi-
cator coverage, completed schemes, size and
scale of scheme, and the occurrence of wa-
terfront and cultural regeneration projects
within prospective locations.
The selection of the most appropriate case-
study locations, as governed by these criteria,
identified a number of potential urban regen-
eration schemes embracing ‘good practice’.
However, the requirement for differing
schemes including both cultural and water-
front examples reduced the range of possibil-
ities. A further consideration was the number
of case-study areas. Literature sources sug-
gest that the significance and robustness of
research findings increases with the number
of sites in which a survey is conducted,
although the largest single gain occurs when
the number of sites is increased from one to
two (Sudman, 1976). This was supported by
Yin (1984) who indicated that survey results
can be strong with a small number of case
studies provided that they are carefully se-
lected. Based upon these considerations,
three case-study cities were selected with
each providing two individual studies to
reflect the waterfront and cultural themes
underpinning the ‘designated regeneration
area’. It is considered that the six case studies
provided a sufficient and diverse range of
examples to allow the indicators to be tested.
760 LESLEY HEMPHILL ET AL.

Figure 1. Methodological approach.

Belfast provides examples of mainstream Cathedral Quarter, formerly Northside, on

UK-based initiatives including the develop-
ment corporation model, Laganside Corpor-
ation, the last remaining development
corporation excluding the recent revival of
the model in Thames Gateway and extensive
use of gap-funding through urban develop-
ment grant. In Belfast, two key schemes
Laganside and Cathedral Quarter fit the
wider selection criteria of waterfront and cul-
tural quarters respectively (Table 1). The for-
mer has a proven national and international
track record. Laganside schemes received
recognition from the RICS in 1998 by win-
ning their ‘Reclamation of Contaminated
Land Award’ and were the subject of an
OECD evaluation study (OECD, 2000). The
the northern city-centre fringe of Belfast,
although possessing many key physical at-
tributes, has been a difficult area to regener-
ate with many different initiatives (Berry and
McGreal, 1993) the latest of which focuses
upon its ethos as a cultural location.
The second case-study city, Dublin, pro-
vides many contrasts with Belfast from a
policy perspective (Berry and McGreal,
1992). Both of the selected schemes in
Dublin (Table 2), the Docklands area and the
cultural quarter of Temple Bar have
benefited significantly from the use of tax-
ation breaks. Temple Bar is widely accepted
as being a good model of cultural regener-
ation with the area promoted as the flagship
 MEASURING URBAN REGENERATION, PART 2 761

Table 1. Belfast case studies: summary

Laganside—Waterfront Cathedral Quarter—Cultural

140 ha site situated 1 km from Belfast city centre 12.1 ha site located on the northern fringe of the
city centre
Area first designated in 1987 with the scheme Area designated in 1998 with the scheme pro-
details highlighted in the Laganside Concept posals highlighted in the Cathedral Quarter Re-
Plan generation Strategy (previously Northside)
Area falls under the control of the Laganside Area falls under the control of the Laganside
Corporation, a form of development corporation Corporation, but treated as a distinct area separ-
set up to oversee the development of the area ate from the main Laganside site
Area consists of 6 key commercial sites and 6 Area (will) consist of 2 apartment developments;
apartment developments, as well as environmen- a hotel; various cultural and arts developments;
tal improvements to River Lagan and the intro- building refurbishments; a multi-storey car park;
duction of a cross-harbour road and rail bridge environmental improvements to streetscape

Table 2. Dublin case studies: summary

Dublin Docklands—Waterfront Temple Bar—Cultural

526 ha site located in the northern area of the 12 ha site located in the heart of Dublin city
traditional dockland; 85 ha available for develop- centre
ment and a further 10 ha for open space
Area first designated in 1986 although an official Area designated in 1991 with development car-
Master Plan was not adopted until 1997 ried out in line with the Temple Bar Develop-
ment Programme 1996
Area falls under the control of the Dublin Dock- Area fell under the control of 2 independent
lands Development Authority (DDDA) which re- companies—Temple Bar Renewal Ltd (respon-
placed the Custom House Docks Development sible for deciding funding applications) and
Authority (CHDDA) in 1997 Temple Bar Properties Ltd (responsible for car-
rying out the development work)
Area consists of 6 main development areas each Development carried out in 2 main phases:
of which have various objectives ranging from 1991–96 and 1996–1999. Area consists of an
promoting residential use, developing landscap- Irish Film Centre, infrastructural improvements,
ing, promoting public transport, promoting apartment developments, retail lettings, building
tourism/hotel facilities, protecting natural habi- refurbishments, provision of innovative archaeo-
tats, supporting cultural development and the logical component, promotion of area as a cul-
expansion of enterprise uses tural quarter

project to mark Dublin’s year as European
City of Culture in 1991 (Montgomery, 1995).
In a similar vein, the Docklands area has
been widely advocated as a success (Gahan,
1993) and long-term sustainability is one of
the objectives of the Dublin Docklands Mas-
ter Plan making the area an ideal case study
in which to apply the indicators (DDDA,
1997). In developing the contrast to Belfast,
the Dublin case studies provided the oppor-
tunity to compare and contrast the differing
approach taken to the stimulation of private
investment between tax-based incentives and
grant-based mechanisms as applied in
Belfast.
The selection of Barcelona (Spain) widens
the research to a European perspective, facil-
itating comparison with the UK and Ireland
dimension. Barcelona also provided two very
different regeneration schemes—namely, the
Ciutat Vella (Old City) as the cultural exam-
ple and the Olympic Village as the water-
762 LESLEY HEMPHILL ET AL.

Table 3. Barcelona case studies: summary

Olympic Village—Waterfront Ciutat Vella—Cultural

47 ha site located between the Cuitadella Park 386 ha site located in the old historical centre of
and the centre of the Poblenou area—part of a the city and constitutes one of the 10 districts
zone of coastal facilities (130 ha) into which Barcelona is divided
Decision to regenerate the area coincided with Area designated as an Area of Integrated Re-
the announcement that Barcelona was to be the habilitation (AIR) in 1986 with 4 special plans
venue for the 1992 Olympic Games. In 1989, for internal reform (PERI) to cover the key areas
various changes were made to the Plan General in the old city: El Ravel, Casc Antic, Barceloneta
to allow a change in the function of the area and Barri Gotic
Area fell under the control of an ad hoc body Area fell under the control of a public–private
called Holding Olimpico SA (HOLSA) who company called Promocio Ciutat Vella SA
were responsible for co-ordinating the develop- (PROCIVESA) to carry out or co-ordinate the
ments main works envisaged in the special plans
Area consists of various 5-storey apartment The main elements of PERI plans included land
blocks with commercial and retail facilities be- clearance, redevelopment or refurbishment of
low, the twin Olympic Towers (Hotel and property, creation of public open space and the
Office); and high-quality public space provision of modern infrastructure

front example (Table 3). The decision to use
Barcelona as a case-study location was based
on a number of counts, but primarily the
international reputation that the city has
achieved since the Olympic Games. Indeed,
the Urban Task Force report (DETR, 1999b)
charted some of the regeneration work un-
dertaken as being a good example of the way
forward for future practice. Furthermore, the
urban design and architectural quality of
what has been developed have been docu-
mented in journals, culminating in the RIBA
awarding the area with a gold medal for the
outstanding quality of the transformations
(DETR, 1999b).
Case-study Results
Accessing information to enable each of the
indicators to be assessed required the utilis-
ation of a combination of highly varied meth-
ods, applied consistently across each of the
case studies. Some provided hard
quantifiable data that could be directly ap-
plied to the points scoring framework, whilst
others provided more qualitative information
requiring interpretation prior to the allocation
of points. The sources of information utilised
reflect the complexity and variety of indica-
tors and include: government departments/
agencies and other development organ-
isations, literature/official reports, other re-
search, end-user surveys of occupiers/
employers, consultation with a range of
professionals in the built environment
(architects/property developers/estate agents),
questionnaire surveys (employees/residents)
and site survey and investigation.
The five separate indicator sets—namely,
Economy and work, Resource use, Buildings
and land use, Transport and mobility and
Community benefits for each case study were
evaluated on both an individual and collec-
tive basis in accordance with the points scor-
ing criteria and rationale highlighted in the
associated paper. The case-study results are
considered under three headings: indicator
set analysis, benchmark analysis and sensi-
tivity analysis.
Indicator Set Analysis
This analysis initially reviews the scoring on
an indicator basis, followed by the overall
points for each indicator group, supple-
mented by a brief overview of the relative
performance of the individual case studies in
delivering sustainable regeneration out-
 MEASURING URBAN REGENERATION, PART 2 763

Table 4. Economy and work scoring performance

Economy and work indicators LS CQ DD TB OV CV

Number of jobs created per 1000 square metres 8 10 6 4 4 2

Net jobs created 6 4 6 4 8 8
Percentage of new enterprises still operating after 3 10 8 10 6 10 8
years
Quality of jobs created 4 10 4 8 6 8
Leverage ratios 4 4 4 4 2 2
Performance of incentive mechanisms 6 1 6 2 6 8
Partnership structure performance 6 5 8 5 8 8
Effectiveness of exit strategy 7 5 8 2 8 7
Incorporation of training programmes 8 7 7 7 6 6
End-user scheme satisfaction 8 5 6 6 8 7

Total score 67 59 65 48 66 64
Percentage score 67 59 65 48 66 64
Weighted score (weighting factor 21.5) 1440.5 1268.5 1397.5 1032.0 1419.0 1376.0
Ranking 1 5 3 6 2 4

comes. Derivation of these indicators has
been discussed in detail in the accompanying
paper. Their richness stems from the number
and comprehensive range of indicators em-
ployed across the five component areas and
the extensive nature of the evidence.
Economy and work. This constitutes a subset
of 10 indicators formulated to evaluate the
performance of economic strategies as well
as the tangible outcomes of the investment
process in terms of the employment gener-
ated. For this indicator set, the three water-
front localities of Laganside Belfast, the
Olympic Village Barcelona and Dublin
Docklands achieved the highest scores
reflecting the economic emphasis initially
needed in these areas to stimulate investment
(Table 4). Of the individual indicators, the
percentage of new enterprises still operating
after 3 years showed the best performance
across all case-study areas, supporting the
need for longevity of businesses in the long-
term sustainability of regeneration schemes.
The 2 Barcelona case studies are the only
examples to embrace significantly the indi-
cator relating to the percentage of net jobs
created: both provide over 30 per cent of
their workforce from their local area. The
Cathedral Quarter Belfast had notable suc-
cess in terms of the number of jobs created
and the quality of those jobs with both indi-
cators scoring highly. However, the rela-
tively poor performance of the incentive
mechanisms indicator, within the Cathedral
Quarter and Temple Bar areas, seems to indi-
cate that incentives had little influence on the
decision of businesses to move to these loca-
tions. This also reflects a high number of in
situ end-users. From the sustainability per-
spective, the findings of the Economy and
work indicators show that in most of the
regeneration schemes partnership structures,
exit strategies and training programmes are
important, although the low scores awarded
to the leverage ratios suggest a continuing
dependency on public-sector money to se-
cure long-term success.
Resource use. The Resource use indicators
relate to the effective use of the physical
resources whilst dealing with environmental
issues such as the reclamation of materials,
waste minimisation, energy efficiency and
conservation. This indicator group produced
a similar set of results across 5 of the case-
study areas with only 7 per cent covering the
top 5 performers (Table 5). The Barcelona
case studies lead on these indicators with the
Olympic Village scoring 64 per cent and the
764 LESLEY HEMPHILL ET AL.

Table 5. Resource use scoring performance

Resource use indicators LS CQ DD TB OV CV

Reclamation of building materials 2 2 2 6 2 8

Retention of environmental features 6 6 4 2 2 6
Household waste disposal 2 2 2 2 6 6
Firms undertaking waste minimisation 10 8 10 10 10 8
Energy efficiency: building lay-out/design 8 7 8 9 8 5
Energy efficiency: building materials/construction 6 4 8 8 6 5
Conservation of built heritage 6 10 2 10 6 10
Incorporation of environmental design 4 4 10 6 10 4
Performance of environmental management 8 6 5 1 8 5

Total score 52 49 51 54 58 57
Percentage score 58 54 57 60 64 63
Weighted score (weighting factor 17.5) 910.0 857.5 892.5 945.0 1015.0 997.5
Ranking 4 6 5 3 1 2

Ciutat Vella 63 per cent. The Cathedral
Quarter Belfast provided the poorest per-
formance in this group (54 per cent)
reflecting cost constraints involved in im-
proving the energy efficiency of older build-
ings. All case studies scored well in the
waste minimisation indicator demonstrating
a very high percentage of firms actively
undertaking waste audits. However, this
positive treatment of waste did not extend to
household waste disposal, with the Barcelona
case studies the only areas to perform above
the less than 20 per cent band—indicating
that in the UK and Ireland households are
falling well short of the 40 per cent recycling
target. As expected, the three cultural case
studies substantially outperformed the water-
front areas regarding the conservation of
built heritage, but conversely were lower
ranked in terms of incorporating environ-
mental design into the building stock. With
the exception of Ciutat Vella and Temple
Bar, the case studies illustrated that little or
no reclamation of building materials oc-
curred. This finding is somewhat surprising
in sustainability terms, given the high levels
of land filling of such waste.
Buildings and land use. The best overall
performance in five of the case studies is for
the Buildings and land use indicator group,
reflecting the emphasis of these particular
regeneration schemes upon property and de-
velopment outcomes. The top-ranked loca-
tions are the Olympic Village (78 per cent),
followed by Ciutat Vella (74 per cent) with
Laganside, Cathedral Quarter and Dublin
Docklands all scoring 73 per cent (Table 6).
These results place the Barcelona case stud-
ies collectively as the best performers in this
sector corresponding with their well-recog-
nised position at the forefront of urban de-
sign and the creation of buildings with
architectural distinction.
Contrasts are apparent in terms of the indi-
vidual indicator performances. The analysis
demonstrates that the three waterfront case
studies demonstrate a high ratio of open
space to built form, whilst the cultural case
studies balance this with having a much
higher ratio of converted/redeveloped build-
ings to new build. Nevertheless, all case
studies illustrate high density, a balance of
uses (apart from Dublin Docklands) and the
complete removal of any contaminated land.
The office rental indicator, with the excep-
tion of Temple Bar, implies that favourable
rents are achievable with relatively high lev-
els of occupancy in areas that previously
were classified as weak market locations.
The performance of the property variables
reflects the acceptable levels of design and
quality of the final built product. However,
the main vulnerability from a sustainability
 MEASURING URBAN REGENERATION, PART 2 765

Table 6. Buildings and land use scoring performance

Buildings and land use indicators LS CQ DD TB OV CV

Ratio of open space: built form 10 6 10 4 10 6

Ratio of converted buildings: new build 2 10 4 10 2 6
Reclamation of contaminated land 10 10 10 10 10 10
Density levels in relation to plot size 8 10 10 8 8 10
Balance of uses 8 6 4 10 8 10
Occupancy levels 8 6 10 8 6 10
Office rental v. CBD rents 8 8 8 2 10 8
Quality of final product 8 7 8 6 8 7
Design quality 8 6 7 9 9 7
Quality of public space 6 6 5 4 8 5
Usage of public space 6 8 6 6 8 6
Quality of private space 6 4 5 3 7 4

Total score 88 87 87 80 94 89
Percentage score 73 73 73 67 78 74
Weighted score (weighting factor 18.9) 1663.2 1644.3 1644.3 1512.0 1776.6 1682.1
Ranking 3 4 4 6 1 2

perspective, shown by the rankings, is the
perception of poorer quality private space in
four of the case studies, the exceptions being
the Olympic Village and Laganside.
Transport and mobility. Transport and mo-
bility indicators encapsulate matters pertain-
ing to infrastructure improvement, travel
habits in relation to a range of activities and
car parking provision. This indicator set pro-
duced a varied performance across the case
studies, with only the Olympic Village (73
per cent) and Temple Bar (70 per cent) pro-
ducing high scores reflecting their excellent
internal accessibility and proximity advan-
tages within their respective city centres
(Table 7). The Dublin Docklands was ranked
third (63 per cent) ahead of Cathedral Quar-
ter, Belfast (62 per cent). Laganside (56 per
cent) had the lowest score for Transport and
mobility, closely followed by Ciutat Vella
(58 per cent), making the Belfast case studies
the weakest collective performers. Given that
a gap of 17 per cent separates the top and
bottom performers, there is ample evidence
to suggest that the importance placed upon
accessibility and discouraging private car use
differs greatly between case-study locations.
Those indicators measuring public trans-
port links and travelling habits in relation to
leisure activities perform well. However,
with the exceptions of Temple Bar, the
Olympic Village and to lesser degree in La-
ganside, the land allocation dedicated
specifically for pedestrian space and move-
ment is ranked low with the converse that too
high a proportion of land is devoted to roads.
For residential car parking provision, the
scores reflect that fewer spaces are allocated
within schemes, whereas commercial car
parking provision points towards an overpro-
vision with the result that the public transport
option becomes less lucrative. This raises
particular concern in meeting sustainability
objectives. Clearly, further deliverability is
required across the case studies in integrated
land-use and transport planning.
Community benefits. Community benefits
capture a highly variable range of indicators,
with the performance of Temple Bar (78 per
cent) significantly ahead of the next-highest-
ranked case studies—namely, Ciutat Vella
(67 per cent) and Olympic Village (66 per
cent). The high score associated with Temple
Bar reflects attempts to foster close links
with the community (Table 8). In compari-
son, the Dublin Docklands (54 per cent) had
766 LESLEY HEMPHILL ET AL.

Table 7. Transport and mobility scoring performance

Transport and mobility indicators LS CQ DD TB OV CV

Land devoted to roads 4 6 8 4 6 4

Land devoted to pedestrians 6 2 2 8 8 2
Road improvements 8 5 4 5 7 5
Work travelling habits 4 4 6 6 6 6
Leisure travelling habits 6 10 10 10 10 10
Public transport links 8 8 10 7 10 8
Car-parking provision—residential 6 8 8 8 8 6
Car-parking provision—commercial 4 8 4 8 4 4
Integration of land use and public transport 5 5 5 7 7 7

Total score 51 56 57 63 66 52
Percentage score 56 62 63 70 73 58
Weighted score (weighting factor 22.1) 1127.1 1237.6 1259.7 1392.3 1458.6 1149.2
Ranking 6 4 3 2 1 5

Table 8. Community benefits scoring performance

Community benefits indicators LS CQ DD TB OV CV

Access to open space 8 8 8 10 10 10

Access to leisure facilities 8 6 8 10 10 10
Access to retail facilities 8 10 8 10 4 6
Access to educational needs 4 8 4 8 6 8
Access to medical facilities 2 6 4 4 10 6
Access to entertainment facilities 8 8 6 10 8 6
Access to cultural facilities 4 4 4 10 4 6
Access to housing 6 4 6 6 7 7
On-site retail facilities 4 5 5 8 5 8
LA21 effectiveness 7 4 6 6 8 6
Community ownership 4 5 5 6 5 6
Community group involvement 3 1 1 5 2 1

Total score 66 69 65 93 79 80
Percentage score 55 58 54 78 66 67
Weighted score (weighting factor 20.00) 1320.0 1380.0 1300.0 1860.0 1580.0 1600.0
Ranking 5 4 6 1 3 2

the lowest score followed by the two Belfast
examples of Laganside (55 per cent) and
Cathedral Quarter (58 per cent). For Dublin
Docklands and Laganside, this in part reflects
the predominant emphasis upon a physical
development programme particularly in the
early stages of the regeneration process. The
scores indicate that more attention needs to
be placed upon improving community rela-
tions at the outset of regeneration schemes
rather than being slowly addressed in later
phases.
On an individual indicator basis, there is a
low level of community group involvement
and, at best, only partly developed com-
munity ownership of the schemes across all
case studies reflecting that, with the excep-
tion of the Ciutat Vella, none is a truly
established residential community area. In
terms of accessibility, the outcome is vari-
able. Certain facilities such as retailing,
leisure and entertainment perform at reason-
ably encouraging levels in terms of sustain-
ability principles, although more alarmingly
 MEASURING URBAN REGENERATION, PART 2
Table 9. Overall points scoring summary
Case-study location Weighted score Percentage score
Laganside 6460.8
Cathedral Quarter 6387.9
Dublin Docklands 6494.0
Temple Bar 6741.3
Olympic Village 7249.2
Ciutat Vella 6804.8
others such as education and medical facili-
ties produced a less satisfactory result apart
from Olympic Village. Likewise, the fact
that accommodation in these areas has be-
come sought after and hence is highly priced
can exclude lower-income households. This
is apparent in the variable performance of the
housing accessibility indicator and raises so-
cial equity issues concerning—for example,
housing affordability. This grouping of indi-
cators shows that out-turn performance is
skewed more towards the remunerative de-
liverables, whereas the community compo-
nents lag considerably with the result that
sustainability objectives are not being fully
embraced.
Benchmarking Analysis
The overall weighted scores, the summation
of the product of the total points times the
weighting factor, permit an analysis of each
case study against the theoretical maximum
possible score of 10 382. The median score
of 5191 is taken to represent average per-
formance with the weighted points band
(4152.8 to 5078.18) in the decile below the
median considered to be of below-average
performance. A case study with a weighted
total below 40 per cent of the maximum
possible score is rated as poor in relation to
sustainability practice. However, as all six
case studies were selected on the basis of
recognised or potential achievements in re-
generation, it is important to have a better
discrimination of weighted scores above the
median. Again, deciles permit a rational cate-
gorisation of scores into nominal groups with
the band 50–59 per cent (weighted score
767
Scaling Overall ranking
62 ‘Good’ (5)
62 ‘Good’ (6)
63 ‘Good’ (4)
65 ‘Good’ (3)
70 ‘Good–excellent’ (1)
66 ‘Good’ (2)
5191.0 to 6125.38) indicating above-average
performance, 60–69 per cent (6229.2 to
7163.58) classified as good, 70–79 per cent
(7267.4 to 8201.78) good–excellent and
above 80 per cent (weighted score 8305.6)
excellent. However, the spread of scores and
associated banding have sufficient distinction
also to enable application of the model to
areas considered to be failing from a regener-
ation perspective.
Each of the six case studies received high
total scores and can be classified as falling
within the ‘good’ category, with the Olympic
Village Barcelona achieving the distinction
of falling within the ‘good–excellent’ cate-
gory. A relatively small percentage (8 per
cent) separates the top and bottom perform-
ers, reflecting the exemplar status of these
three cities in terms of regeneration. On the
basis of this analysis, the top-ranked case-
study location is the Olympic Village with 70
per cent (weighted score 7249.2) and Ciutat
Vella has the next-best performance with 66
per cent (weighted score 6804.8). Temple
Bar ranks third with 65 per cent (weighted
score 6741.3). Outside the top three case
studies in terms of sustainability perform-
ance, the difference between the scores is
small. Dublin Docklands (63 per cent,
weighted score 6494) just edges out Lagan-
side (62 per cent, weighted score 6460.8) for
fourth place, whilst the Cathedral Quarter (62
per cent, weighted score 6387.9) although in
the bottom position is still classified in sus-
tainability terms as good (Table 9). It is the
weighted score that an area or project re-
ceives that is important.
The findings illustrate that the model is
capable of discriminating between compara-
768 LESLEY HEMPHILL ET AL.
tive overall performance levels on a case-
study basis. Significantly, each of the case
studies confirmed that sustainability is be-
coming a more integral part of regeneration
practice, with none of the areas below the
‘good’ category. Furthermore, no noticeable
pattern was established between those case
studies undertaken in waterfront locations
compared with the more culturally based
projects; rather, differences are more related
to the city than defined locations within a
city.
The case studies that have been completed
or those nearing completion provided the
best performances, notably the Olympic Vil-
lage (70 per cent), Ciutat Vella (66 per cent)
and Temple Bar (65 per cent), as these areas
benefit from more established markets and
greater investor, resident and employee
confidence. This suggests that the perform-
ance levels of the other areas should improve
as schemes mature. In this respect, Dublin
Docklands (63 per cent), Laganside (62 per
cent) and Cathedral Quarter (62 per cent)
have performed well given these constraints
and just lag the performance levels set by the
top-ranked case studies.
Sensitivity Analysis
The regeneration characteristics evaluated by
the indicators are complex, with a degree of
qualitative evidence from specific expert
groups as discussed in the accompanying
paper. Hence, sensitivity analysis is deployed
to allow for any uncertainty and to analyse
the effect that a change in the scores may
have on the overall result. Sensitivity simula-
tions are undertaken whereby higher and
lower variations of the final scores of several
individual indicators were chosen to test
what impact, if any, this would exert on the
overall final points scoring. This demon-
strates how sensitive the results are to a
change in the individual indicator scores and
indicates whether or not a particular case
study should be reallocated to another per-
formance level on the benchmark scale. The
lower score variation was taken from alterna-
tive pessimistic responses received in the
data collection phase, substituting the orig-
inal score with the lower score where any
difference occurred, with the converse occur-
ring in the case of the higher variation. Both
sets of scores provide an alternative view of
performance and represent the extreme of the
distribution of scores.
The results of the sensitivity analysis illus-
trate the effect that a change in the points
allocated to any indicator would have on the
weighted scores and relative ranking of the
case studies. In terms of the low variation,
some case-study areas proved to be more
sensitive to a change in points total than
others (Table 10). For example, the Olympic
Village shows the smallest percentage drop
(4 per cent) in the overall performance level
indicating that it is the least sensitive to a fall
in points total. Further confirmation of top
ranking for Olympic Village (66 per cent) is
illustrated by an increase in the gap (to 6 per
cent) to the next-highest-scored location,
Ciutat Vella (60 per cent), on the basis of
lowest score variation. The largest drop in
points was experienced by Laganside and
Cathedral Quarter (8 per cent), followed by
Temple Bar (7 per cent), resulting in both
Belfast case studies displaying a similar per-
formance level (54 per cent of possible
weighted score) and confirming their relative
status as the weakest performers, although
still above average. The utilisation of the low
score variation also shows a narrowing of the
gap between the Dublin case studies to 1 per
cent, demonstrating that the Temple Bar re-
sults were more sensitive to change than the
Docklands (Table 10).
Perhaps the most significant result of ap-
plying the low score variation is the effect on
the overall ranking. In all cases, the appli-
cation of the low score variation saw a real-
location of the case studies from the ‘good’
category to the lower ‘above average’ tier of
the benchmark scale and the top-ranked
Olympic Village area (6823.9) was reallo-
cated to the upper half of the ‘good’ cate-
gory. Of the case studies reallocated to the
‘above average’ category, Ciutat Vella
(6195.7), Temple Bar (5982.8) and the
Dublin Docklands (5948.3) are in the upper
 MEASURING URBAN REGENERATION, PART 2 769

Table 10. Comparison and ranking of sensitivity analysis (low variation)

Case-study location Weighted Percentage Percentage Scaling Original Low

score score difference category ranking ranking

Laganside 5588.5 54 -8 ‘Average–good’ (5) (5)

Cathedral Quarter 5485.5 54 -8 ‘Average–good’ (6) (6)
Dublin Docklands 5948.3 57 -6 ‘Average–good’ (4) (4)
Temple Bar 5982.8 58 -7 ‘Average–good’ (3) (3)
Olympic Village 6823.9 66 -4 ‘Good’ (1) (1)
Ciutat Vella 6195.7 60 -6 ‘Average–good’ (2) (2)

Table 11. Comparison and ranking of sensitivity analysis (high variation)

Case study location Weighted Percentage Percentage Scaling category Original High
score score difference ranking ranking

Laganside 7003.9 68 ⫹6 ‘Good’ (5) (5)

Cathedral Quarter 7164.7 69 ⫹7 ‘Good’ (6) (3)
Dublin Docklands 6788.0 65 ⫹2 ‘Good’ (4) (6)
Temple Bar 7115.8 69 ⫹4 ‘Good’ (3) (4)
Olympic Village 7849.9 76 ⫹6 ‘Good–excellent’ (1) (1)
Ciutat Vella 7251.3 70 ⫹4 ‘Good–excellent’ (2) (2)

half, whilst Laganside (5588.5) and Ca-
thedral Quarter (5485.5) are placed in the
middle to lower part (Table 10). The analysis
suggests that the overall scores awarded for
the case studies are sensitive to a downward
change in points total, with all six experienc-
ing a reallocation to the next lower category.
Concerning the high score variation, some
case studies demonstrated a greater sensi-
tivity to points change than others (Table
11). The Dublin Docklands showed the
smallest increase (2 per cent) in points total,
followed by Temple Bar and Ciutat Vella (4
per cent), illustrating that the Docklands is
the least sensitive to a rise in points total
from the high score variation. Cathedral
Quarter (7 per cent), the Olympic Village
and Laganside (6 per cent) experience the
greatest increase in points. The analysis
based on a high score variation establishes
Cathedral Quarter (69 per cent), demonstrat-
ing its latent potential, as the third-ranked
case-study area behind the Olympic Village
(76 per cent) and Ciutat Vella (70 per cent),
which consolidate their positions at the top.
The application of the high score variation
moved Ciutat Vella (7251.3) to the margins
of the ‘good–excellent’ tier with the other
case studies consolidating their position
within the respective categories, notably the
Olympic Village (7849.9), Cathedral Quarter
(7164.7), Temple Bar (7115.8) and Lagan-
side (7003.9). The Dublin Docklands experi-
enced little change in weighted points total,
with only a slight improvement to the mid
position of the ‘good’ category (Table 11).
The overall scores awarded for the case stud-
ies have proved to be less sensitive to an
upward change in points total, suggesting
that respondents may have adopted optimis-
tic assessments of regeneration areas in their
original response to particular indicators.
As none of the case studies experienced
greater than a 7 per cent change in the points
total allocated by selecting either the high or
low score variations, it can be concluded that
the original scores awarded can be treated
with a high degree of confidence. Further-
more, the overall rankings developed from
the sensitivity analysis showed very little
movement, with the exception of the Ca-
thedral Quarter case, which experienced a
major upward change attributable to its in-
herent potential. The outcome offers con-
770 LESLEY HEMPHILL ET AL.
siderable support to the scores allocated and
the rankings developed as well as the re-
liability of the data sources used.
Conclusions
Within urban regeneration policy and prac-
tice, the application of performance indica-
tors is becoming increasingly significant in
measuring sustainability outputs. However,
the use of methodologies to benchmark and
assess the performance of sustainability re-
mains largely underresearched. Weighted in-
dicators and a points scoring framework
developed in the preceding paper are applied
to both waterfront-led and culture-led regen-
eration schemes in three European cities to
measure achievements in the delivery of sus-
tainable developments/environments. The
study demonstrates that the more established
and mature regeneration locations (Olympic
Village, Ciutat Vella and Temple Bar) de-
liver better sustainability performance than
those areas which are less well developed,
highlighting the importance of timing in re-
generation areas. The use of sensitivity
analysis to test the various options, particu-
larly for those indicators relying on qualita-
tive evidence, indicates that the points score
allocated to the various case studies in the
empirical investigation can be treated with
confidence. Significantly, the analysis shows
that both of the Barcelona case studies, the
Olympic Village and Ciutat Vella, and Tem-
ple Bar, Dublin, display a high degree of
adherence to sustainability principles.
A key parameter of the hierarchical model
developed in the accompanying paper and
tested in this analysis is its inclusiveness in
terms of 5 indicator sets in the components
tier and 52 separate indicators. The robust-
ness of the model is demonstrated by its
application in different regeneration typolo-
gies (waterfronts and cultural quarters) and
in different urban contexts in terms of city
characteristics, institutional structures and
policy frameworks. Development of the
model through the use of expert opinion is a
key aspect of the methodological approach,
the comprehensive nature of which allows
reliance to be placed upon the weights as-
cribed to the components tier. In addition, the
model has flexibility in that parameters can
be changed to reflect new standards arising
from shifts in policy or technological devel-
opments, thereby contributing to its effec-
tiveness in the evaluation of mechanisms in
creating sustainable urban environments.
The weightings applied to the respective
components enhance the utility of the model
as a comparative tool in the analysis of sus-
tainability practice and outcomes. In this re-
spect, the model is capable of application
across regeneration areas and projects of dif-
fering size, scale, level of perceived success/
delivery and at different phases of a
particular initiative. Indeed, the model is use-
able as a tool to test at different stages the
degree to which a particular regeneration
project is adhering to the principles of sus-
tainability. Thus, at strategic points over the
life cycle of the project, the model could be
utilised to inform and influence strategy in a
corrective capacity by highlighting any
significant divergence from sustainability ob-
jectives. Ideally, a project/regeneration area
should be surveyed before mid term to en-
sure that sufficient time is available to im-
plement change if scores are low implying
that the scheme that is unlikely to achieve
sustainability targets. In theory, resurvey
could potentially occur at several stages over
the period of a project as well as forming part
of the overall evaluation at the end of the
initiative.
Potential policy benefits stem from the
model’s holistic approach and its applicabil-
ity across different regeneration mechanisms.
In assessing the extent to which regeneration
areas are conforming to sustainability cri-
teria, the model provides a valuable tool in
policy analysis. Application, as demonstrated
by the case studies, illustrates where good
practice is occurring—notably, in the
Barcelona examples and particularly the
Olympic Village. An added feature and po-
tentially of greater significance is the ability
of the model to operate at a components level
which, from a policy perspective, discrimi-
nates between those aspects where the regen-
 MEASURING URBAN REGENERATION, PART 2
eration initiative is working well and those
matters which require further policy inter-
vention. From the analysis contained in this
paper, the strong performance of Laganside
(Belfast) in terms of Economy and work is
highlighted, but equally apparent is the
weaker performance in relation to Com-
munity benefits and Transport and mobility.
Hence, policy priorities in Laganside need to
be redirected to the latter issues, whereas in
the case of Temple Bar (Dublin) the model
highlights a converse situation.
The capacity of the model to drill down
and examine components is important from a
policy and decision-making perspective, par-
ticularly in the allocation and targeting of
funds and other resources by allowing pol-
icy-makers to identify areas requiring action.
This in turn offers the possibility to influence
future performance and guide policy and
practice. On a wider perspective, the appli-
cability of performance indicators allows
policy approaches to be assessed on their
ability to deliver sustainable development
and assess whether partnership structures are
effective. Regeneration performance outputs
are increasingly expected to show joined-up
holistic thinking across programmes by pro-
moting physical, social, economic and en-
vironmental criteria. The indicator-based
model advocated and developed in these two
papers allows an evaluation of the extent to
which integration is being achieved and sus-
tainability objectives delivered within urban
regeneration.
References
AUDIT COMMISSION (2000) Getting Better All the
Time: Making Benchmarking Work. London:
HMSO.
BERRY, J. N. and MCGREAL, W. S. (1992) Fiscal
mechanisms and inner city development, Jour-
nal of Property Finance, 3(4), pp. 543–552.
BERRY, J. N. and MCGREAL, W. S. (1993) Public
sector initiatives in the regeneration of Belfast,
in: J. N. Berry, W. S. MCGREAL and W. DEDDIS
(Eds) Urban Regeneration, Property Invest-
ment and Development, pp. 193–214. London:
E&FN Spon.
BRISTOW, H. (1999) Developing the tools to mea-
771
sure success, Urban Environment Today, 62(21
January), p. 13.
BROWNHILL, D. and RAO, S. (2002) A Sustainabil-
ity Checklist for Developments: A Common
Framework for Developers and Local Authori-
ties. Watford: CRC Ltd.
BURNS, R. B. (2000) Introduction to Research
Methods. London: Sage Publications Ltd.
CARLEY, M. and KIRK, K. (1998) Sustainable by
2020? A Strategic Approach to Urban Regen-
eration for Britain’s Cities. Bristol: The Policy
Press, for the Joseph Rowntree Foundation.
DDDA (DUBLIN DOCKLANDS DEVELOPMENT AUTH-
ORITY) (1997) Dublin Docklands Area—Master
Plan 1997. Dublin: DDDA.
DETR (DEPARTMENT OF THE ENVIRONMENT,
TRANSPORT AND THE REGIONS) (1998a) Sustain-
able Regeneration Good Practice Guide. Lon-
don: HMSO.
DETR (1998b) Planning for the Communities of
the Future. London: HMSO.
DETR (1998c) Planning for Sustainable Develop-
ment: Towards Better Practice. London:
HMSO.
DETR (1998d) Sustainability Counts. London:
HMSO.
DETR (1999a) Sustainable Development Strategy
for the UK: A Better Quality of Life. London:
HMSO.
DETR (1999b) Towards an Urban Renaissance:
Final Report of the Urban Task Force. London:
DETR.
DETR (2000) Our Towns and Cities: The Future
Delivering an Urban Renaissance. London:
HMSO.
DOE (DEPARTMENT OF THE ENVIRONMENT) (1994)
Sustainable Development: The UK Strategy.
Cm 2426. London: HMSO.
EUROPEAN COMMISSION DGXI (1996) European
Sustainable Cities. Expert Group on the Urban
Environment. Luxembourg: European Com-
mission.
GAHAN, J. (1993) Fiscal measures in promoting
urban regeneration: Custom House Docks,
Dublin, in: J. N. BERRY, W. S. MCGREAL and
W. DEDDIS (Eds) Urban Regeneration, Prop-
erty Investment and Development, pp. 95–107.
London: E&FN Spon.
HAKIM, C. (2000) Research Design: Successful
Designs for Social and Economic Research.
London: Routledge.
IMRIE, R. and THOMAS, H. (1995) Changes in
Local Governance and their Implications for
Urban Policy Evaluation, in: R. HAMBLETON
and H. THOMAS (Eds) Urban Policy Evalu-
ation: Challenge and Change, pp. 123–138.
London: Paul Chapman Publishing.
MONTGOMERY, J. (1995) The story of Temple Bar:
creating Dublin’s cultural quarter, Planning
Practice and Research, 10(2) pp. 135–172.
772 LESLEY HEMPHILL ET AL.
ODPM (OFFICE OF THE DEPUTY PRIME MINISTER)
(2002) Best Value Performance Indicators
2002/2003. London: ODPM.
OECD (ORGANISATION FOR ECONOMIC CO-OPER-
ATION AND DEVELOPMENT) (2000) Urban Re-
naissance: Belfast’s Lessons for Policy and
Partnership. Paris: OECD.
SUDMAN, S. (1976) Applied Sampling. New York:
Academic Press.
TUROK, I. (1989) Evaluation and understanding in
local economic policy, Urban Studies, 26,
pp. 587–606.
WONG, C. (2002) Developing indicators to inform
local economic development in England, Ur-
ban Studies, 39(10), pp. 1833–1863.
YIN, R. K. (1984) Case Study Research: Design
and Methods. London: Sage.