Journal of Cleaner Production 171 (2018) 1324e1341

Contents lists available at ScienceDirect

Journal of Cleaner Production

journal homepage: www.elsevier.com/locate/jclepro

Towards more equity concerning quality of Urban Waste Management

services in the context of cities
Toni Lupo a, *, Massimiliano Cusumano b
a
degli Studi di Palermo,
Dipartimento dell'Innovazione Industriale e Digitale (DIID), Ingegneria Chimica, Gestionale, Informatica, Meccanica, Universita
Viale delle Scienze, 90128, Palermo, Italy
b
Servizi Comunali Integrati R.S.U. S.p.a., Partinico, Italy

a r t i c l e i n f o a b s t r a c t

Article history: An innovative combined evaluation approach is proposed in the present paper to deal with the issue of
Received 7 February 2017 the performance measurement and comparison of Urban Waste Management (UWM) services in the
Received in revised form context of cities. The ELECTRE III method is initially considered as tool able to execute a multi-criteria
19 September 2017
analysis aimed at ranking UWM systems based on their integration and sustainability levels. The
Accepted 20 September 2017
Available online 20 September 2017
latter are comprehensively evaluated via the recently developed “Wasteaware” Integrated and Sustain-
ability Waste Management indicators framework, which takes into account physical components and
governance aspects of UWM systems. Then, on the basis on ELECTRE III results, a practical gaps inves-
Keywords:
Urban waste management
tigation is developed to comparatively highlight service weaknesses to be addressed to pursue more
ISWM indicators framework equity concerning integration and sustainability in the considered context. The issue is particularly
Sustainability measures relevant in light of the multi-dimensional nature and the substantial complexity of UWM systems, as
ELECTRE III well as for the high criticality level assumed by these systems in recent years. The proposed combined
Gaps investigation evaluation approach is applied to an empirical case which regards the UWM systems of the Optimal
Territorial Ambit of Palermo (Sicily). In the considered context, results highlight how stakeholders'
participation in decision-making activities represents a crucial factor to pursue shared and coordinated
strategies in line with integration and sustainability principles. Moreover, seventeen measures involving
operational, environmental, socio-economic as well as administrative waste management aspects are
selected from a comprehensive list of measures aimed at waste management integration and sustain-
ability, and then implemented in the investigated context. Through these measures, the resulting ser-
vices criticalities can be evaluated and properly addressed, and new required processes can be designed.
© 2017 Elsevier Ltd. All rights reserved.

1. Introduction services have taken on an increased relevance in view of the sig-

Urban Waste Management (UWM) is one of the most important
public services in the context of a city. It meaningfully affects public
health (Balaguer, 2004), the image of cleanliness and tidiness of a
city (Hong et al., 2010), as well as the authority and credibility of
Local Administrators (Plata-Díaz et al., 2014). UWM includes all
those activities of managerial and operational nature aimed at
handling urban solid waste, starting from waste generation up to its
final destination, including collection, transportation, recovery and,
finally, disposal (Chen et al., 2010).
In recent decades, difficulties and problems arising from UWM
* Corresponding author.
E-mail address: toni.lupo@unipa.it (T. Lupo).
nificant growth of the amounts of waste generated (Karak et al.,
2012) and of the high service costs (Greco et al., 2015). These cir-
cumstances along with the increase of both population and urban
areas make the provision of effective UWM strategies based on
responsibility logics increasingly more and more cogent (Arıkan
et al., 2017). Facing this scenario, the EU has recently proposed
possible strategies for an effective waste management via the Eu-
ropean Waste Directive 2008/98/EC (WFD2008). The latter makes a
vibrant distinction among waste treatment methods, by ordering
hierarchically them and by establishing a ranking regarding the
different types of possible interventions (Gharfalkar et al., 2015). In
particular, the reduction of both the amounts of waste generated at
sources and the hazardous content of that waste represent the
fundamental prerequisites to be pursued by any UWM scheme
(Zaman, 2014). Then, other possible strategies take into

https://doi.org/10.1016/j.jclepro.2017.09.194
0959-6526/© 2017 Elsevier Ltd. All rights reserved.
 T. Lupo, M. Cusumano / Journal of Cleaner Production 171 (2018) 1324e1341
consideration features concerning waste treatments and specif-
ically they refer to (Van Ewijk and Stegemann, 2014):
1. preparing for re-use - checking, cleaning or repairing operations,
by which materials that would have become waste are properly
prepared so that they will be reused;
2. recycling - converting or reprocessing activities of waste into
reusable materials to reduce the consumption of new materials;
3. recovery - selective extractions of disposed materials to recover
energy by incineration, anaerobic digestion or similar processes;
4. disposal, which represents the least desirable option for an
effective and environmentally sensitive UWM.
The implication of the European Waste Directive is that of
greater complexity in the UWM context (Butti, 2012). Actually,
UWM systems are intent on performing complex and expensive
processes aimed at adapting waste collection and treatment ser-
vices to the new environmental demands of reuse-recycling and
using the waste residues generated (Kanat, 2010). Due to these
conditions, nowadays UWM systems have become composite en-
tities including, in addition to the purely technological features
(Morrissey and Browne, 2004), also those related to environmental
impact, global and local economics, social responsibility, material
technology, and much more (Wilson and Velis, 2015). These fea-
tures can be combined within three fundamental UWM dimensions
including stakeholders, waste system elements and their aspects
(Klundert and Anschutz, 2001a). The first one takes into account
the stakeholders in the UWM, comprising Households, Commu-
nities and other Service Users, Non-Governmental Organizations,
National and Local Government, Private Sector Enterprises,
Informal Private Sector and External Support Agencies (Triguero
et al., 2016). The dimension of the waste system elements refers
to technical and performance characteristics of UWM services
(Vitorino de Souza Melare  et al., 2017). Finally, the aspects dimen-
sion includes technical-operational, environmental, financial, eco-
nomic and socio-cultural, as well policy-legal and institutional ones
(UN-HABITAT, 2010) through which UWM processes can be
assessed and improved (Klundert and Anschutz, 2001b).
An indicators framework was recently developed in order to
allow performance evaluations and comparisons in terms of inte-
gration and sustainability of UWM services in the context of cities
(Wilson et al., 2015). According to this Integrated and Sustainability
Waste Management (ISWM) indicators framework, an innovative
combined evaluation approach is proposed in this study. In
particular, the ISWM indicators framework is considered as a
suitable tool to comprehensively evaluate and compare integration
and sustainability levels of UWM services (Leal Filho et al., 2016),
while the non-compensative ELECTRE III method (Roy, 1978) is
applied to perform a multi-criteria analysis aimed at ranking the
examined alternatives (i.e. the UWM systems adopted) based on
their levels of integration and sustainability. ELECTRE III has been
extensively considered in various research fields of decision sci-
ences (Mardani et al., 2015a), however, in the field of social sciences
its use is still quite limited (Mardani et al., 2015b). Then, with
reference to the best alternative(s) resulting from the ELECTRE III
analysis, gaps among the indicators scores are evaluated and
considered to comparatively point out service aspects deemed
lacking, as addressed within their priorities (Lupo, 2015) to pursue
greater equity among UWM services in the analyzed context.
Taking into account perspectives and viewpoints of the funda-
mental stakeholders of the field of inquiry, the proposed combined
evaluation approach is applied to an empirical case which regards
the UWM systems of the Optimal Territorial Ambit of Palermo
(Sicily), and service criticalities to be addressed are pointed out and
1325
discussed.
The relevance of the present contribution is twofold. From the
methodological viewpoint, the multi-criteria analysis is carried out
in a non-compensative manner, thus avoiding that an alternative
may outrank another one if its score proves to be unacceptable
albeit on a single indicator (Certa et al., 2013) as it might happen,
for instance, in the Analytic Hierarchical Process (AHP) method
(Saaty, 2008). This feature is particularly appreciated considering
that stakeholders can assume a non-compensative behavior in
comparing performance of service alternatives, especially when the
discrepancy in scores overcomes a certain threshold of veto
(Ghobadian et al., 1994). Although several approaches combining
tools of multi-criteria methods have been proposed in the literature
to study performance of UWM services (Goulart et al., 2017), to the
best of the authors' knowledge this study represents the first
attempt in the field of inquiry at comparing service performance in
a non-compensative manner. Furthermore, with relation to the
empirical case carried out, the OTA of Palermo was strategically
analyzed under the integration and sustainability perspectives, and
critical service aspects were pointed out and discussed.
The remainder of this paper is organized as follows. Section 2
describes materials and methods of such a study, while in section
3 results of the performed empirical case are given. Section 4
provides an explanation of research findings. Finally, conclusions
summarize the work highlighting weaknesses and strengths of the
proposed combined evaluation approach, as well as possible future
research developments.
2. Materials and methods
The devised combined evaluation approach comprises three
fundamental stages. The first stage refers to the choice of alterna-
tives to be investigated and the identification of the objective to be
pursued. In particular, the UWM systems of the OTA of Palermo are
chosen as alternatives to be examined, while their improvement in
terms of integration and sustainability represents the related
objective to be pursued. Accordingly, the ISWM indicators frame-
work is selected as a suitable tool for investigating these UWM
systems. Such an indicators framework, which represents a stan-
dard internationally recognized for comprehensively evaluating
and comparing integration and sustainability levels of UWM sys-
tems in the context of cities (Ikhlayel and Nguyen, 2017), is detailed
in section 2.1.2. Lastly, the collection of scores concerning investi-
gated alternatives against ISWM indicators closes this stage.
The second stage involves the processing of collected data and
concerns the implementation of a multi-criteria analysis aimed at
evaluating and comparing the investigated UWM systems. Gener-
ally speaking, in a multi-criteria analysis the decision-making
process normally leads to a choice between several alternatives
the Decision Maker(s) must consider with reference to a series of
criteria (Al-Shemmeri et al., 1997). The following basic components
are involved into the analysis: criteria score; a set of decision rules
needed to evaluate alternatives against the information collected
and the preferences of the Decision Maker(s) (Huang et al., 2011).
One category amongst the multi-criteria analysis methods for
evaluating options and strategies is based on outranking relations
(Mardani et al., 2015a). This category of methods aims at building
the so-called “outranking” relations among contextual alternatives
to facilitate specific decision-making processes (Lopez et al., 2017).
In every outranking method, pairs of feasible alternatives are
compared to each other by each single criterion to determine
whether one is preferable or no worse than the other. Outranking is
based on the concordance-discordance principle, implying the ex-
istence of concordance of the criteria favoring one alternative over
another in the absence of any apparent discordance in judgments
1326 T. Lupo, M. Cusumano / Journal of Cleaner Production 171 (2018) 1324e1341
(Marzouk, 2011). The general category of outranking methods may
be subdivided into two main families: the methods oriented to-
wards choice or ordering (Govindan and Jepsen, 2016) and the
selection/segmentation methods, dealing with problems of classi-
fication (Yu, 1992). In this study, the ELECTRE III outranking method
oriented towards ordering is chosen for ranking the investigated
alternatives. Several specific features of such a method make its
choice certainly attractive for the aim of the present study. Firstly,
ELECTRE III is a non-compensative method does not allowing the
compensation of a very bad score of an alternative against an in-
dicator with good scores against other indicators. In particular, such
a non-compensation feature is applied when differences between
the alternative scores overcomes a certain value of the veto
threshold (Salminen et al., 1998). Furthermore, such a method is
based on pseudo criteria which enable to deal with the uncertain
and imprecise nature of comparison processes (Curcurù et al., 2013)
through the use of proper thresholds (Bana e Costa and Oliveira,
2012).
Then, the second stage of the proposed approach implements
the ELECTRE III multi-criteria analysis relying on alternatives scores
against ISWM indicators and the Decision Maker(s) preferences.
The latter include the indicators weights and three threshold values
for each indicator (Damaskos and Kalfakakou, 2005). The weight
associated with each indicator denotes a relative importance co-
efficient. The thresholds represent values introduced to try to
reduce two types of risk: that of falsely considering two situations,
whose conditions and evaluations actually differ only marginally, as
being distinct instead of essentially equivalent and, that of failing to
make appropriate distinctions between situations whose prefer-
ences are in fact diverse. In particular, for each indicator, the
Fig. 1. Combined evaluation approach.
following three thresholds are considered: the indifference
threshold (q), the preference threshold (p) and the veto threshold
(v), with v > p > q > 0. The indifference threshold (q) expresses the
minimal difference in scores to which the Decision Maker(s) at-
tributes significance in terms of indifference. The preference
threshold (p) expresses the minimal difference in scores to which
the Decision Maker(s) attributes significance in terms of strict
preference. Finally, the veto threshold (v) expresses the minimal
difference in scores above which the Decision Maker(s) considers
the gap in scores to not be offset by score values in the other in-
dicators (Bouyssou and Marchant, 2007).
Within the last stage, gaps in scores against each ISWM indi-
cator as regards each alternative vs those of the best alternative(s)
resulting from the ELECTRE III analysis are calculated. These gaps
are representative of discrepancies in performance levels among
the best UWM system(s), taken as reference point, and the other
ones under investigation. The knowledge and awareness of these
discrepancies provide useful information about priorities of efforts
to be undertaken to fix them (Yasin, 2002). In this study, gaps in
scores and values of ISWM indicators thresholds are employed as
input data to execute a practical gaps investigation to point out in
comparative manner service criticalities in investigated UWM
systems within their priorities (Lupo, 2015). Scheme in Fig. 1 refers
to the devised combined evaluation approach and emphasizes how
each stage involves aspects of investigated UWM systems, ISWM
indicators framework, ELECTRE III analysis, as well gaps
investigation.
These stages are detailed in the next sections.
 T. Lupo, M. Cusumano / Journal of Cleaner Production 171 (2018) 1324e1341
2.1. Preliminary stage
2.1.1. Optimal territorial Ambit of Palermo
In Italy, UWM systems are assigned by the Public Environment
and Territory Department to specific geographical areas on the
basis of the Region administrative subdivisions. These geographical
areas are known as Optimal Territorial Ambits (OTAs). The empir-
ical case performed takes into account the OTA of Palermo (Sicily),
covering a part of the northeast Sicilian territory between Carini
and Castellammare del Golfo, and including 12 cities each one
characterized by its own UWM system. Fig. 2 shows the
geographical area covered by the OTA and Table 1 specifies back-
ground information on involved cities and key waste related data.
2.1.2. ISWM indicators framework
The first conceptual approach for an integrated and sustainable
UWM was developed in 1995 (Schübeler, 1997). Such an approach,
further developed during the 2000s on (Zurbrügg et al., 2012)
introduced for the first time the concept of the Integrated and
Sustainable Waste Management (ISWM). The latter is typically
synthesized by means of two partially overlapping triangles, as
shown in Fig. 3.
The first triangle includes the three physical components of an
UWM system referring to its sustainability aspects over the long
term (Wilson, 2007), that is:

public health, primarily related to the waste collection stage;

environment, the environment protection during waste handling
and related processing operations;

3Rs e “reduce, reuse, recycle” waste hierarchy, to reduce the
amounts of waste generated and to improve UWM processes
and programs.
The second triangle takes into account governance aspects that
play a crucial role for a well-functioning and integrated UWM
system. These features comprise:

user inclusivity, to achieve the stakeholders' involvement;
Tyrrhenian Sea
Fig. 2. Geographical coverage of the OTA of Palermo.
1327

financial sustainability, which ensures that UWM services and
related activities are cost-effective and affordable;

sound institutions and pro-active policies, responsible and
transparent institutional context to carry out effective-efficient
UWM services.
Based on this conceptual framework, several versions of in-
dicators frameworks have been proposed to allow the performance
measurements and comparisons of UWM systems in the context of
cities (Menikpura et al., 2013). The revised “Wasteaware” ISWM
indicator framework, which effectiveness and practicality has been
recently shown in measuring and comparing performances of both
physical components and governance aspects of UWM systems
(Wilson et al., 2015), is considered to evaluate and compare UWM
systems of the OTA of Palermo. Fig. 4 summarizes this indicators
framework.
The ISWM indicators framework includes 4 quantitative and 8
qualitative composite indicators to assess the integration and sus-
tainability level of UWM systems. The former are related to directly
measurable service performance aspects, whereas the latter take
into account qualitative features of physical components, as well as
soft characteristics of governance aspects of UWM systems. These
indicators within their definitions are listed in Table 2.
2.2. ELECTRE III analysis and gap investigation
ELECTRE III is considered to rank alternatives on the basis of
their integration and sustainability levels. As aforementioned, input
data required by this method include the scores matrix, which
describes the performance of alternatives against ISWM indicators,
as well as the indicators weights and the three threshold values for
each indicator. The procedure to achieve the alternatives ranking
involves two steps, the construction of outranking relations and the
process of alternatives classification (Certa et al., 2017). These steps
are detailed into the Appendix A1.
Implications towards greater equity concerning integration and
sustainability in the OTA of Palermo can emerge by comparing in-
dicators scores of the best alternative(s) resulting from the ELECTRE
Sicily
1328 T. Lupo, M. Cusumano / Journal of Cleaner Production 171 (2018) 1324e1341

Table 1
Background information of the OTA of Palermo (2014).

Public heath - Sound Institutions & pro-

Collection active Policies

Environment – Financial
Treatment & Governance Sustainability
Disposal
W: Waste
Related Data Resource Value – Inclusivity – User
Reduce, Reuse, and Provider
Recycle (3Rs)

B: Background
Information

Fig. 3. ISWM conceptual framework (Scheinberg et al., 2010).

Fig. 4. “Wasteaware” ISWM indicators framework.

 T. Lupo, M. Cusumano / Journal of Cleaner Production 171 (2018) 1324e1341 1329

Table 2
Qualitative and quantitative ISWM indicators and definitions.

Table 3
Total questionnaires.
1330 T. Lupo, M. Cusumano / Journal of Cleaner Production 171 (2018) 1324e1341

ISWM indicators framework

(A.1) Waste collection coverage

(A)
Quantitative
indicators
(A.4) Recycling rate

UWM Systems

S(1) Balestrate
(B.1) Quality of waste collection
services
S(2) Borgetto
(B.2) Environmental protection
in waste treatment/disposal

(B.3) Quality of 3Rs – Reduce,

… reuse, recycle – provision
(B)
(B.4) User inclusivity Qualitative
S(12) Trappeto indicators

(B.5) Provider inclusivity

(B.6) Financial sustainability

(B.8) Adequacy of national

sustainability waste management
framework

Objective data Subjective data

Fig. 5. UWM systems and ISWM indicators mapping in ELECTRE III (Buchanan et al., 1999).

III analysis, which can be considered as a reference point in the
investigated context, with those of the other alternatives under
analysis. In particular, with regard to the ith alternative (with i ¼ 1,
2, …, n) and the jth indicator (with j ¼ 1, 2, …, J), Eq. (1) is employed
to evaluate the gap in scores:


 *  
gapj ¼ sj  sj i ci; j
i
where ðsj Þ* and ðsj Þi denote scores of the best alternative(s) and the
ith one respectively against the jth indicator.
For each indicator, obtained gaps in scores are compared against
the related indifference, preference and veto thresholds to point
out priorities of improvement efforts. Gaps greater than the related
veto thresholds need to be primarily fixed, because they highlight
critical discrepancies in performance levels. Secondarily, the other
lower gaps need to be fixed. Lastly, gaps lower that the related
indifference thresholds do not require interventions because they
represent non-appreciable discrepancies in performance levels
(Lupo, 2015).
3. Empirical case
3.1. Scores of indicators
Scores of involved UWM systems as regards the ISWM in-
dicators were obtained via a consultation process involving both
the Waste Departments of the OTA of Palermo and the fundamental
stakeholders of the field of inquiry. The former provided data
related to the quantitative indicators, whereas scores of qualitative
composite indicators were collected via a survey taking into ac-
count perspectives and viewpoints of Service Providers, Local Ad-
ministrators, as well Citizens of the involved 12 cities, which were
considered as representative of the key stakeholders for the
(1) investigated OTA (Petts et al., 2003). For each involved city, Man-
agers, Decision Makers and Employers of the UWM system were
interviewed in order to obtain the Service Providers viewpoints,
and Majors as well as some Council Members were consulted in
order to get the Local Administrators perspectives. Then, a random
sample of Citizens representative of the reference population in
terms of age, income and residential placement were selected for
each involved city. The survey was conducted face-to-face from
September 2015 to January 2016 by using the questionnaire form
reported into the Appendix A2. Totally 404 questionnaires were
obtained to undergo the study (Table 3). An informative note about
the main aspects of delivered UWM services was read to re-
spondents to provide a service overview also with reference to
aspects which may have poor visibility for some consulted
stakeholders.
On the basis of collected data, scores of alternatives against
ISWM indicators were obtained. These data represent a substantial
feedback on the reached integration and sustainability levels in
involved UWM systems. Tables A1eA4 reported into the Appendix
A3 show these data for each consulted stakeholder. Note that, in-
dicators (A.2) Waste captured by the system, (A.3) Controlled treat-
ment and disposal and (B.7) Adequacy of national sustainability waste
 T. Lupo, M. Cusumano / Journal of Cleaner Production 171 (2018) 1324e1341
Table 4
Ranges (Rj), weights (wj) and thresholds values (qj, pj, vj).
management framework have the same scores for the investigated
UWM systems as well the consulted stakeholders. Thus, they were
not treated in the multi-criteria analysis successively reported.
3.2. ELECTRE III results
ELECTRE III was then used to rank UWM systems taking into
account perspectives and viewpoints of Service Providers, Local
Administrators as well Citizens. Fig. 5 shows the structural aspect of
the multi-criteria analysis performed.
The objective data refer to the scores matrices as regards con-
sulted stakeholders, whereas subjective data concern the Decision
Maker(s) preferences including thresholds values (p, s, v) and
weights (w) required by ELECTRE III (Roberts and Goodwin, 2002).
Indicators weights were estimated via AHP on the basis of pair-
wise comparison judgments elicited from a multi-disciplinary team
of experts specifically selected for this task. Eighteen experts were
selected at this step of the study from a comprehensive list of
stakeholders including 2 Decision Makers, 2 Managers and 2 Em-
ployers of Service Providers, 6 Local Administrators and 6 Citizens.
Each expert was called to assess the relative importance of any pair
1331
of ISWM indicators using the Saaty linguistic comparison scale
(Saaty, 2008), and collected data were employed to construct the
related pairwise comparison matrices. The inconsistency of the 18
obtained pairwise comparison matrices was checked via the
traditional Consistence Ratio (CR) and Consistency Index (CI)
(Saaty, 2008), whereas the geometric mean was applied to aggre-
gate these matrices (Basak and Saaty, 1993), obtaining three
aggregated comparison matrices with reference to Service Pro-
viders, Local Administrators and Citizens. Based on the latter, the
indicators weights shown in Table 4 were obtained. Conversely,
values of the indifference (q), preference (p) and veto (v) thresholds
were estimated according to the practice of similar problems and
the risk attitude of stakeholders in the waste management
(Chatterjee et al., 2014). In particular, threshold values were
expressed in terms of percentage of scores differences assumed by
the contemplated indicators. For each involved stakeholder, Eq. (2)
refers to the relationship adopted to identify the range Rj in which
these threshold values have to belong.
1332 T. Lupo, M. Cusumano / Journal of Cleaner Production 171 (2018) 1324e1341

Fig. 6. ELECTRE III rankings.

Table 5
Ranges of thresholds and weights in which no change occurs in the first position of ELECTRE III rankings.

Lastly, a sensitivity analysis was performed to verify the

   
maxci si;j  minci si;j robustness of final rankings as concerns the identified best UWM
Rj ¼   cj (2) systems. With reference to the most important indicators for the
maxci si;j
consulted stakeholders, the ELECTRE III analysis was repeated by
where si,j denotes the score of the jth indicator with reference to the assuming different ranges of values for the indifference (q), pref-
ith UWM system. erence (p) and veto (v) thresholds, while all other thresholds were
Thus, set the values of indifference thresholds for each stake- unchanged, unless otherwise required by the v > p > q > 0 rule
holder on the bases of the analyst preference, Eq (3) were estab- (Khalili and Duecker, 2013). A further sensitivity analysis was per-
lished among the three thresholds: formed changing the weights of these most important indicators.
Since indicators' weights have to sum to 1, weights of the other
 indicators change as well. The consistence of results presented in
pj ¼ 2$qj
cj (3) Table 5 indicates that the ranking procedure is robust under
vj, ¼ 3$qj
possible variability on subjective data.
and thresholds values presented in Table 4 were obtained.
Then, ELECTRE III was applied taking into perspectives and 3.3. Gaps investigation results
viewpoints of consulted stakeholders obtaining rankings shown in
Fig. 6. The last step of the developed empirical case regarded the
The best UWM systems were S(12) Trappeto and S(1) Balestrate, implementation of the gaps investigation based on the ELECTRE III
respectively as regards viewpoints of Service Providers and Local results. Gaps in scores were estimated according to Eq. (1)
Administrators, while both these UWM systems resulted to be the considering viewpoints of consulted stakeholders. From Service
reference points with regard to perspectives of Citizens. Providers perspectives gaps in scores were evaluated considering
 T. Lupo, M. Cusumano / Journal of Cleaner Production 171 (2018) 1324e1341 1333

(a)

(b)

(c)

Fig. 7. Gaps in scores: Service Providers (a), Local Administrators (b) and Citizens (c). (A.1) Waste collection coverage; (A.4) Recycling rate; (B.1) Quality of waste collection services;
(B.2) Environmental protection in waste treatment/disposal; (B.3) Quality of 3Rs e Reduce, reuse, recycle e provision;(B.4) User inclusivity; (B.5) Provider inclusivity; (B.6) Financial
sustainability; (B.8) Adequacy of national sustainability waste management framework.
1334 T. Lupo, M. Cusumano / Journal of Cleaner Production 171 (2018) 1324e1341
Table 6
Service criticalities and priorities.
S(12) Trappeto as the UWM system of reference, while S(1) Bale-
strate as well S(1) Balestrate and S(12) Trappeto were the UWM
systems considered with regard to viewpoints of Local Adminis-
trators and Citizens respectively (Fig. 6). Fig. 7 shows gaps in scores
referred to Service Providers (Fig. 7a), Local Administrators (Fig. 7b)
and Citizens (Fig. 7c). The dashed lines refer to the indifference
(green), preference (yellow) and veto (red) thresholds.
4. Discussion of the research findings
From the performed investigation of gaps, critical ISWM in-
dicators were pointed out as regards perspectives and viewpoints
of consulted stakeholders. Results shown in Table 6 highlight that
consulted Service Providers appear to be quite demanding with
reference to service aspects mainly related to the waste collection
coverage (A.1), whereas quality of waste collection (B.1), provider
inclusivity (B.5) and local institutional coherence (B.8) represent
focal aspects for the involved Local Administrators. Finally, Citizens
generally tend to perceive as poorly performing aspects related to
coverage (A.1) and quality (B.1) of waste collection services, pro-
vider inclusivity (B.5) and local institutional coherence (B.8).
With relation to their viewpoints and roles, consulted stake-
holders prove to be characterized by substantial dissimilarities and,
in some cases, poorly aligned expectations as regards the provided
UWM services. The latter might arise from the different roles
played by stakeholders in the considered context, but also from the
partial involvement of some of them (Triguero et al., 2016).
Therefore, it represents a matter of great importance to support
stakeholders' participation in UWM decision-making activities,
Citizens role in decisions that directly affect them, and use of
effective feedback and communication mechanisms involving all
the interested parties of the field of inquiry. In fact, Public education
and awareness in UWM services represent crucial factors to pursue
shared and coordinated strategies in line with integration and
sustainability principles (Higgs, 2006). Hornsby et al. (2017) high-
light in their work carried out on Naples (Italy), as appropriate
participatory decision-making processes involving the stake-
holders of the context of inquiry lead to better UWM solutions in
terms of integration and sustainability, as well as to the prevention
of acceptance conflicts at the local level.
Conversely, considering UWM systems of the OTA of Palermo,
the following criterion was considered to point out priority of in-
terventions on service aspects resulting critical. Service aspects
which prove to be critical for all the consulted stakeholders need to
be primarily addressed (A). Then, all those resulting critical for two
consulted stakeholders (B). Finally, the other remaining critical
service aspects can be subsequently addressed (C). Table 6 sum-
marizes results of the performed priority analysis.
For instance, considering S(2) Borgetto, it is characterized by a
single critical service aspect with priority (A) as regards the user
inclusivity (B.4), which was proved to be critical for Service Pro-
viders, Local Administrators and Citizens. The other remaining
critical service aspects have priority (B) and they refer to (A.1) Waste
collection coverage, (B.5) Provider inclusivity and (B.8) Local institu-
tional coherence, which were proved to be critical for Service Pro-
viders and Citizens, while (B.1) Quality of waste collection services
resulted to be critical for Local Administrators and Citizens.
Conversely, S(1) Balestrate and S(12) Trappeto, which take the first
position of the ELECTRE III ranking (Fig. 6), and S(10) Terrasini, do
not have critical service aspects.
4.1. Measures implemented in the OTA of Palermo
Based on results obtained via the performed gaps investigation,
17 measures were carefully selected from a comprehensive list of
measures aimed at waste management integration and sustain-
ability (Hoornweg and Bhada-Tata, 2012), and then implemented as
management support tools in the investigated context. Table 7
shows these measures arranged with regard to the UWM aspects
involved.
The operational and environmental measures were selected to
support the continuous improvement process of service aspects
 T. Lupo, M. Cusumano / Journal of Cleaner Production 171 (2018) 1324e1341
Table 7
Measures implemented.
related to indicators (A.1) Waste collection coverage, (B.1) Quality of
waste collection services and (B.2) Degree of environmental protection
in waste treatment/disposal. The socio-economic measures were
considered to promote an effective Public participation and
awareness in UWM services. Finally, the administrative measures
shown in Table 7 were enabled to address service aspects regarding
indicators (B.5) Provider inclusivity and (B.8) Adequacy of national
sustainability waste management framework. Through these mea-
sures, the resulting service criticalities can be evaluated and fixed,
and new required processes can be suitably designed.
The last phase of these interventions concerns the assessment of
results arising from this measures implementation. This phase will
be carried out after an appropriate period by their implementation,
still adopting the proposed combined evaluation approach.
5. Conclusions
An innovative evaluation approach, which combines the ELEC-
TRE III method along with the “Wasteaware” Integrated and Sus-
tainability Waste Management (ISWM) indicators framework and a
1335
practical gaps investigation, was proposed in the present paper to
evaluate and compare integration and sustainability levels of Urban
Waste Management (UWM) services in the context of cities. Its
effectiveness was shown with reference to an empirical case
involving the Optimal Territorial Ambit of Palermo (Sicily), and
criticalities in UWM services to be addressed within their priorities
were pointed out and discussed.
In the Authors' opinion, the only limitation of the proposed
evaluation approach lies on the subjective assumption of thresh-
olds and weights values required by ELECTRE III. However, this
limitation can be reliably overcome, as it was done in the present
study, by selecting a suitable experts' team representative of the
key stakeholders for the investigated context to estimate these
parameters values. Conversely, the prerequisite of having to assign
weights and thresholds makes the proposed evaluation approach
highly versatile and easily adaptable to a wide range of possible
needs and constraints which may characterize the specific evalu-
ation context under analysis.
Nevertheless, the proposed evaluation approach presents
notable strengths. First of all, it is based on the ISWM indicators
1336 T. Lupo, M. Cusumano / Journal of Cleaner Production 171 (2018) 1324e1341

framework, which represents a standard internationally recognized Once J concordance and discordance matrices of dimension
as suitable to comprehensively evaluate and compare integration (I  I) are obtained, they are aggregated into the global concordance
and sustainability levels of UWM systems. Moreover, it follows a matrix by means of weights assigned to each indicator. The generic
standardized methodology which can be applied to support deci- element c (a, a') of this matrix can be expressed as:
sion making processes devoid of ambiguity and involving the X
fundamental stakeholders of the context under investigation. cða; a0 Þ ¼ wj $cj ða; a0 Þ (A2)
Furthermore, it assures reliable and consistent performance com- j2J
parison results in view of the non-compensative feature of ELECTRE
Subsequently, the credibility d (a, a') that (a) outranks (a'), which
III, so avoiding that an alternative may outrank another one if its
summarizes information expressed by concordance and discordance
score proves to be unacceptable albeit on a single indicator. Such a
relations, is defined as:
feature is especially suitable for the aim of the present work
because of the possible non-compensative behavior of stakeholders dða; a0 Þ ¼ cða; a0 Þ if dj ða; a0 Þ  cða; a0 Þ (A3)
in comparing service performance of alternatives, especially when
the discrepancy in scores overcomes a certain veto threshold. Y  
1  dj ða; a0 Þ
Finally, the proposed evaluation approach can be treated as a dða; a0 Þ ¼ cða; a0 Þ 0
otherwise
suitable tool to control each stage of the continuous improvement 1  cða; a Þ
cjjdj ða;a0 Þ > cða;a0 Þ
process: measuring initial performance, planning strategies and
policies, implementing improvement efforts and measuring final Next, the so-called descendent distillation is performed, that
performance. consists in ranking alternatives in a descendent order on the basis
Future researches in this field may concern the further devel- of the credibility parameter. In such a step, a further threshold d0 is
opment of the proposed evaluation approach by implementing the considered:
Fuzzy Set Theory to effectively manage possible uncertainty and
vagueness of stakeholders in evaluating ISWM qualitative com- d0 ¼ max0 dða; a0 Þ (A4)
ca;a
posite indicators.
A credibility level d0 is established at a level lower than d0, but
closes enough that the related interval (d0 - d0 ) can be considered an
Acknowledgments indifference credibility interval. Consequently, a Boolean matrix is
calculated. The generic element B (a, a0 ) of this matrix can be
The Authors are grateful to Dr. Silvia Cataldo for the support expressed as:
provided to carry out the data collection phase. 
0
1 ca; a0 dða; a0 Þ > d0
Bða; a0 Þ ¼ (A5)
0 otherwise
Appendix A1
Lastly, for each alternative t, it is calculated the difference Q (t)
ELECTRE III between the number of alternatives that are outranked by the
alternative t at a credibility level greater than d0 and the number of
In order to utilize ELECTRE III, both the basic data of the multi- alternatives k that outrank the alternative t at the same credibility
criteria analysis including a matrix indicating scores of investigated level. The alternatives a such that B (t, a) ¼ 1 are outranked by t with
alternatives against ISWM indicators, and Decision Maker prefer- a credibility greater than d0 . Analogously, the alternatives a for
ences must be at the user's disposal. which B (a, t) ¼ 1 are those t with a credibility greater than d0 . The
Let A ¼ {ai: i ε I} denotes the set of alternatives evaluated with first distillates are the alternatives t having:
reference to J indicators. A weight is assigned to each indicator to
obtain a vector of normalized weights such that: Q ðtÞ ¼ maxQ ðaÞ (A6)
cj
(0  w  1
X j If the set containing all the alternatives, for which the previous
wj ¼ 1 cj (A1) relationships are verified, has a cardinality higher than 1, the
j2J described procedure is recursively applied until coming up with the
set containing only one alternative, or a group of alternatives that
In the first phase, ELECTRE III is based on the definition of
cannot be further differentiated. Subsequently, an ascending
concordance and discordance indices for each indicator j ε J. For
distillation can be applied, ranking the alternatives in ascending
each pair (a, a') of alternatives, the value of the concordance index cj
order. This new ranking, coupled to that obtained by descending
(a, a') may be established with reference to the scores sj (a) and sj
distillation, leads to a unique final ranking list.
(a') of the considered alternatives under the jth indicator. In detail:

cj (a, a0 ) ¼ 1 if sj (a0 )  sj (a) þ qj Appendix A2

cj (a, a0 ) ¼ 0 if sj (a0 )  sj (a) þ pj
pj ½sj ða0 Þsj ðaÞ Questionnaire

cj ða; a0 Þ ¼ pj qj if sj (a) þ qj < sj (a0 ) < sj (a) þ pj
Dear Service Provider/Local Administrator/Citizen,
Thus, a concordance matrix is obtained for each indicator This questionnaire is based on a framework of performance
considered. Similarly, the values of the discordance indices can be indicators internationally recognized able to evaluate the level of
obtained. In detail: integration and sustainability of Urban Waste Management (UWM)
systems.

dj (a, a0 ) ¼ 0 if sj (a0 )  sj (a) þ pj By answering to the following service items, you give us your

dj (a, a0 ) ¼ 1 if sj (a0 )  sj (a) þ vj viewpoints on the service performance of your UWM system and
½sj ða0 Þsj ðaÞpj

dj ða; a0 Þ ¼ vj pj if sj (a) þ pj < sj (a0 ) < sj (a) þ vj such a knowledge is useful to assess the reached integration and
 T. Lupo, M. Cusumano / Journal of Cleaner Production 171 (2018) 1324e1341 1337

sustainability level, as well as to identify possible paths for the Thank you for your support.
continuous improvement.
1338 T. Lupo, M. Cusumano / Journal of Cleaner Production 171 (2018) 1324e1341
 T. Lupo, M. Cusumano / Journal of Cleaner Production 171 (2018) 1324e1341 1339

Appendix A3

Performance overview of UWM systems in the context of the

OTA of Palermo

Table A1
Scores of ISWM quantitative indicators.
1340 T. Lupo, M. Cusumano / Journal of Cleaner Production 171 (2018) 1324e1341
Table A2
Scores of ISWM qualitative composite indicators (Service Providers).
Table A3
Scores of ISWM qualitative composite indicators (Local Administrators).
Table A4
Scores of ISWM qualitative composite indicators (Citizens)
References
Al-Shemmeri, T., Al-Kloub, B., Pearman, A., 1997. Model choice in multicriteria de-
cision aid. Eur. J. Oper. Res. 97 (3), 550e560.
€ 2017. Solid waste disposal methodology
Arıkan, E., Şimşit-Kalender, Z.T., Vayvay, O.,
selection using multi-criteria decision making methods and an application in
Turkey. J. Clean. Prod. 142 (20), 403e412.
Balaguer, M.T., 2004. La eficiencia en las Administraciones Locales ante diferentes
especificaciones del output. Hacienda Pública Espan ~ ola 170, 37e58.
Bana e Costa, C.A., Oliveira, M.D., 2012. A multicriteria decision analysis model for
faculty evaluation. Omega 40 (4), 424e436.
Basak, I., Saaty, T., 1993. Group decision making using the analytic hierarchy pro-
cess. Math. Comput. Model. 17 (4e5), 101e109.
Bouyssou, D., Marchant, T., 2007. An axiomatic approach to non-compensatory
sorting methods in MCDM, I: the case of two categories. Eur. J. Oper. Res. 178,
217e245.
Buchanan, J., Sheppard, P., Vanderpooten, V., 1999. Project Ranking Using ELECTRE
III. http://citeseerx.ist.psu.edu/viewdoc/download?doi¼10.1.1.493.
6585&rep¼rep1&type¼pdf. (Accessed 9 November 2012).
Butti, L., 2012. Birth and death of waste. Waste Manag. 32, 1621e1622.
Certa, A., Enea, M., Lupo, T., 2013. ELECTRE III to dynamically support the decision
maker about the periodic replacements configurations for a multi-component
system. Decis. Support Syst. 55 (1), 126e134.
Certa, A., Enea, M., Galante, G.M., La Fata, C.M., 2017. ELECTRE TRI-based approach to
the failure modes classification on the basis of risk parameters: an alternative to
the risk priority number. Comput. Ind. Eng. 108, 100e110.
Chatterjee, P., Mondal, S., Chakraborty, S., 2014. A comprehensive solution to
automated inspection device selection problems using ELECTRE method. Int. J.
 T. Lupo, M. Cusumano / Journal of Cleaner Production 171 (2018) 1324e1341
Technol. 2, 193e208.
Chen, X., Geng, Y., Fujita, T., 2010. An overview of municipal solid waste manage-
ment in China. Waste Manag. 30 (4), 716e724.
Curcurù, G., Galante, G.M., La Fata, C.M., 2013. An imprecise fault tree analysis for
the estimation of the rate of OCcurrence of failure (ROCOF). J. Loss Prev. Process
Ind. 26, 1285e1292.
Damaskos, X., Kalfakakou, G., 2005. Application of electre III and DEA methods in
the BPR of a bank branch network. Yugosl. J. Oper. Res. 15 (2), 259e276.
DIRECTIVE, 2008. Directive 2008/98/EC of the European parliament and of the
Council on waste and repealing certain directives. Off. J. Eur. Union L312, 3e30.
Gharfalkar, M., Court, R., Campbell, C., Ali, Z., Hillier, G., 2015. Analysis of waste
hierarchy in the European waste directive 2008/98/EC. Waste Manag. 39,
305e313.
Ghobadian, A., Simon Speller, S., Jones, M., 1994. Service quality: concepts and
models. Int. J. Qual. Reliab. Manag. 11 (9), 43e66.
Goulart Coelho, L.M., Lange, L.C., Coelho, H.M.G., 2017. Multi-criteria decision
making to support waste management: a critical review of current practices
and methods. Waste Manag. Res. 35 (1), 3e28.
Govindan, K., Jepsen, M.B., 2016. ELECTRE: a comprehensive literature review on
methodologies and applications. Eur. J. Oper. Res. 250 (1), 1e29.
Greco, G., Allegrini, M., Del Lungo, C., Gori Savellini, P., Gabellini, L., 2015. Drivers of
solid waste collection costs. Empirical evidence from Italy. J. Clean. Prod. 106,
364e371.
Higgs, G., 2006. Integrating multi-criteria techniques with geographical information
systems in waste facility location to enhance public participation. Waste Manag.
Res. 24 (2), 105e117.
Hong, J., Li, X., Zhaojie, C., 2010. Life cycle assessment of four municipal solid waste
management scenarios in China. Waste Manag. 30 (11), 2362e2369.
Hoornweg, D., Bhada-Tata, P., 2012. WHAT a WASTE a Global Review of Solid Waste
Management. WorldBank, Washington D.C.
Hornsby, C., Ripa, M., Vassillo, C., Ulgiati, S., 2017. A roadmap towards integrated
assessment and participatory strategies in support of decision-making pro-
cesses. The case of urban waste management. J. Clean. Prod. 142 (20), 157e172.
Huang, I.B., Keisler, J., Linkov, I., 2011. Multi-criteria decision analysis in environ-
mental sciences: ten years of applications and trends. Sci. Total Environ. 409
(19), 3578e3594.
Ikhlayel, M., Nguyen, L.H., 2017. Integrated approaches to water resource and solid
waste management for sustainable development. Sust. Develop. https://doi.org/
10.1002/sd.1683 (in-press).
Kanat, G., 2010. Municipal solid-waste management in Istanbul. Waste Manag. 30
(8e9), 1737e1745.
Karak, T., Bhagat, R.M., Bhattacharyya, P., 2012. Municipal solid waste generation,
composition, and management: the world scenario. Crit. Rev. Environ. Sci.
Technol. 42 (15), 1509e1630.
Khalili, N.R., Duecker, S., 2013. Application of multi-criteria decision analysis in
design of sustainable environmental management system framework. J. Clean.
Prod. 47, 188e198.
Klundert, Ar, Anschutz, J., 2001a. Integrated Sustainable Waste Management e the
Concept. WASTE, Gouda, the Netherlands).
Klundert, Ar, Anschutz, J., 2001b. Integrated Sustainable Waste Management - the
Concept e Tools for Decision-makers - Experiences from the Urban Waste
Expertise Programme (1995-2001).
Leal Filho, W., Brandli, L., Moora, H., Kruopiene, J., Stenmarck, Å., 2016. Bench-
marking approaches and methods in the field of urban waste management.
J. Clean. Prod. 112, 4377e4386.
Lopez, J.C.L., Noriega, J.J.S., Chavira, D.A.G., 2017. A multi-criteria approach to rank
the municipalities of the states of Mexico by its marginalization level: the case
of Jalisco. Int. J. Inf. Technol. Decis. Mak. 16 (2), 473e513.
Lupo, T., 2015. Fuzzy ServPerf model combined with ELECTRE III to comparatively
evaluate service quality of international airports in Sicily. J. Air Transp. Manag.
42, 249e259.
Mardani, A., Jusoh, A., Nor, K.M.D., Khalifah, Z., Zakwan, N., Valipour, A., 2015a.
Multiple criteria decision-making techniques and their applications - a review
1341
of the literature from 2000 to 2014. Econ. Research-Ekonomska Istraz. 28 (1),
516e571.
Mardani, A., Jusoh, A., Zavadskas, E.K., Zainab, Z., Nor, K.M.D., 2015b. Application of
multiple-criteria decision-making techniques and approaches to evaluating of
service quality: a systematic review of the literature. J. Bus. Econ. Manag. 16 (5),
1034e1068.
Marzouk, M.M., 2011. ELECTRE III model for value engineering applications. Auto.
Constr. 20 (5), 596e600.
Menikpura, S.N.M., Sang-Arun, J., Bengtsson, M., 2013. Integrated Solid Waste
Management: an approach for enhancing climate co-benefits through resource
recovery. J. Clean. Prod. 58, 34e42.
Morrissey, A.J., Browne, J., 2004. Waste management models and their application
to sustainable waste management. Waste Manag. 24, 297e308.
Petts, J., Homan, J., Pollard, S., 2003. Participatory Risk Assessment: Involving Lay
Audiences in Environmental Decisions on Risk. R&D Technical Report E2-043/
TR/01. Environment Agency, Bristol, p. 281.
Plata-Díaz, A.M., Zafra-Go  mez, J.L., Pe
rez-Lo pez, G., Lo
pez-Hern andez, A.M., 2014.
Alternative management structures for municipal waste collection services: the
influence of economic and political factors. Waste Manag. 34, 1967e1976.
Roberts, R., Goodwin, P., 2002. Weight approximations in multi-attribute decision
models. J. Multi-crit. Decis. Anal. 11, 291e303.
Roy, B., 1978. ELECTRE III: un algorithme de classements fonde sur une represen-
tation floue des preference en presence de criteres multiples. Cah. CERO 20 (1),
3e24.
Saaty, T.L., 2008. Decision making with the analytic hierarchy process. Int. J. Serv.
Sci. 1 (1), 83e98.
Salminen, P., Hokkanen, J., Lahdelma, R., 1998. Comparing multicriteria methods in
the context of environmental problems. Eur. J. Oper. Res. 104 (3), 485e496.
Scheinberg, A., Wilson, D.C., Rodic, L., 2010. Solid Waste Management in the World's
Cities: Water and Sanitation in the World's Cities 2010. Earthscan on Behalf of
UN-habitat, London. http://www.waste.nl/sites/waste.nl/files/product/files/
swm_in_world_cities_2010.pdf. (Accessed 27 September 2014).
Schübeler, P., 1997. A conceptual framework for municipal solid waste management
in developing countries. Waste Manag. Res. 15 (4), 437e446.

Triguero, A., Alvarez-Aledo, C., Cuerva, M.C., 2016. Factors influencing willingness to
accept different waste management policies: empirical evidence from the Eu-
ropean Union. J. Clean. Prod. 138, 38e46.
UN-HABITAT, 2010. Solid Waste Management in the World's Cities/Water & Sani-
tation in the World's Cities 2010. UN-Habitat, Malta.
Van Ewijk, S., Stegemann, J.A., 2014. Limitations of the waste hierarchy for achieving
absolute reductions in material throughput. J. Clean. Prod. 132, 122e128.
Vitorino de Souza Melare , A., Montenegro Gonza lez, S., Faceli, K., Casadei, V., 2017.
Technologies and decision support systems to aid solid-waste management: a
systematic review. Waste Manag. 59, 567e584.
Wilson, D.C., Rodic, L., Cowing, M.J., Velis, C.A., Whiteman, A.D., Scheinberg, A.,
Vilches, R., Masterson, D., Stretz, J., Oelz, B., 2015. 'Wasteaware' benchmark
indicators for integrated sustainable waste management in cities. Waste Manag.
35, 329e342.
Wilson, D.C., 2007. Development drivers for waste management. Waste Manag. Res.
25 (3), 198e207.
Wilson, D.C., Velis, C.A., 2015. Waste management - still a global challenge in the
21st century: an evidence-based call for action. Waste Manag. Res. 33 (12),
1049e1051.
Yasin, M.M., 2002. The theory and practice of benchmarking: then and now.
Benchmarking An Int. J. 9 (3), 217e243.
Yu, W., 1992. ELECTRE TRI: Aspects methodologiques et manuel d’utilisation.
Document du LAMSADE No. 74. Universite  Paris-Dauphine.
Zaman, A.U., 2014. Measuring waste management performance using the “Zero
Waste Index”: the case of Adelaide. Aust. J. Clean. Prod. 66, 407e419.
Zurbrügg, C., Gfrerer, M., Ashadi, H., Brenner, W., Küper, D., 2012. Determinants of
sustainability in solid waste management - The Gianyar Waste Recovery Project
in Indonesia. Waste Manag. 32 (11), 2126e2133.