Reconfiguring the Ecosystem for Sustainable Healthcare

Difficult Decisions in Times of Constraint: Criteria-Based Resource Allocation

Craig Mitton Franois Dionne Diane Schmidt
Article information:
To cite this document: Craig Mitton Franois Dionne Diane Schmidt . "Difficult
Decisions in Times of Constraint: Criteria-Based Resource Allocation" In Reconfiguring
the Ecosystem for Sustainable Healthcare. Published online: 10 Oct 2014; 195-216.
Permanent link to this document:
Downloaded by University of Sydney Library At 22:13 19 March 2017 (PT)

http://dx.doi.org/10.1108/S2045-060520140000004011
Downloaded on: 19 March 2017, At: 22:13 (PT)
References: this document contains references to 0 other documents.
To copy this document: permissions@emeraldinsight.com
The fulltext of this document has been downloaded 81 times since 2014*
Users who downloaded this article also downloaded:
(2003),"Internet based marketing research: a serious alternative to traditional research
methods?", Marketing Intelligence & Planning, Vol. 21 Iss 2 pp. 79-84 http://
dx.doi.org/10.1108/02634500310465380
(2014),"Factors used by Ghanaian students in determining career options in the
tourism and hospitality industry: Lessons for developing countries", Worldwide
Hospitality and Tourism Themes, Vol. 6 Iss 2 pp. 166-178 http://dx.doi.org/10.1108/
WHATT-12-2013-0053

Access to this document was granted through an Emerald subscription provided by

emerald-srm:216535 []
For Authors
If you would like to write for this, or any other Emerald publication, then please
use our Emerald for Authors service information about how to choose which
publication to write for and submission guidelines are available for all. Please visit
www.emeraldinsight.com/authors for more information.
About Emerald www.emeraldinsight.com
Emerald is a global publisher linking research and practice to the benefit of society.
The company manages a portfolio of more than 290 journals and over 2,350 books
and book series volumes, as well as providing an extensive range of online products
and additional customer resources and services.
Emerald is both COUNTER 4 and TRANSFER compliant. The organization is a partner
of the Committee on Publication Ethics (COPE) and also works with Portico and the
LOCKSS initiative for digital archive preservation.
 *Related content and download information correct at time of download.
Downloaded by University of Sydney Library At 22:13 19 March 2017 (PT)
 CHAPTER 7

DIFFICULT DECISIONS IN TIMES

OF CONSTRAINT: CRITERIA-
BASED RESOURCE ALLOCATION
Downloaded by University of Sydney Library At 22:13 19 March 2017 (PT)

Craig Mitton, Francois Dionne and Diane Schmidt

ABSTRACT

Purpose The purpose of this chapter is to describe a method for prior-

ity setting that can be used to identify options for disinvestment, and
is also meant to serve as a tool for re-allocation of resources to achieve
better outcomes with a given pot of resources.
Approach This chapter draws on findings from the application of a
priority setting and resource allocation framework known as Program
Budgeting and Marginal Analysis (PBMA). Case studies are used to
illustrate key points around implementation including factors for success
and guidelines for improving priority setting in practice.
Findings PBMA has been applied in over 150 settings over the last
30 years. Purposes varied from focusing strictly on disinvestment to
examining opportunities for re-allocation. Many organizations report
continued use of the framework and decision makers typically express a
desire to not revert to historical allocation or political negotiation in
deciding on the funding for programs.

Reconfiguring the Ecosystem for Sustainable Healthcare

Organizing for Sustainable Effectiveness, Volume 4, 195 216
Copyright r 2014 by Emerald Group Publishing Limited
All rights of reproduction in any form reserved
ISSN: 2045-0605/doi:10.1108/S2045-060520140000004011
195
 196 CRAIG MITTON ET AL.

Practical implications Practical implications of this body of work on

priority setting abound in that there are significant opportunities to
improve resource allocation practice including better engagement of
staff, clinicians and public members, greater use of evidence in decision
making and improving process transparency.
Social implications As healthcare resources are limited, particularly
in predominantly publicly funded health systems, prudent use of
resources is critical. Actually applying the appropriate tools to ensure
that funding aligns with organizational and system objectives is
Downloaded by University of Sydney Library At 22:13 19 March 2017 (PT)

paramount.
Originality/value Although there is a large body of literature on prior-
ity setting particularly in countries like the United Kingdom and
Canada, this chapter serves to highlight key messages specifically in the
context of fiscal constraint and in relation to the concept of disinvestment
or service reduction.
Keywords: Priority setting; resource allocation; disinvestment

INTRODUCTION

Countries the world over are experiencing fiscal constraint to a level that
has potentially not been seen previously in history (Appleby, Crawford, &
Emmerson, 2009; Callan, Nolan, & Walsh, 2010; Canada Institute for
Health Information, 2009). Disinvestment, decommissioning, service
reduction, and other related terms have jumped into the vernacular of
many healthcare decision makers who previously held that annual budget
growth outpacing inflation by 2 3 times or more was the norm. In most
OCED countries, this new fiscal reality results in the slowing of year on
year healthcare spending growth if not outright reduction in real terms
(Callan et al., 2010). While operational efficiencies achieved with processes
such as LEAN have made some inroads in reducing expenditure or freeing
up capacity, significant budget gaps often still exist. As a result, in Canada,
as elsewhere, healthcare decision makers are moving toward evidence-based
processes for disinvestment.
In this chapter, we explore the concept of disinvestment, identifying
different types of cost savings and clarifying what we would put forward
as an appropriate understanding of disinvestment. We then explain why
 Criteria-based Resource Allocation 197

disinvestment should be a fundamental tool in the decision makers toolbox

regardless of the given fiscal constraints and how it is key to improving
quality and health gain. Following this overview, the chapter then focuses
on a specific method for priority setting and resource allocation that has
been used many times over in different countries. We review the steps of
this method, including development of a rigorous set of decision criteria,
and then describe two case studies from Canada in which the approach was
used to identify disinvestments. The case studies serve to illustrate that
disinvestment is not only achievable but can readily become part of
Downloaded by University of Sydney Library At 22:13 19 March 2017 (PT)

ongoing resource management. Finally, we will review key success factors

for effective resource management, drawing on recent nationally funded
research from Canada that describes high performance with respect to
priority setting. This will lead us to our conclusion that having an explicit,
transparent, criteria-based process for priority setting and resource alloca-
tion decisions results in better alignment with organizational objectives and
also serves to improve the legitimacy and defensibility of the decisions
themselves. Indeed, decision makers report that after using this type of
process they are extremely reticent to return to an approach involving
historical allocation patterns and political negotiation.
The overall purpose of this chapter is to describe a rigorous method
for priority setting and resource allocation that has been used many times
over not only in Canada but across many different countries, and then to
illustrate its use in real-world contexts to draw out key aspects of process
implementation relevant for application elsewhere. Through this, it is our
intent that researchers and decision makers from other countries will gain a
better understanding of methods available to assist in addressing fiscal
challenges in healthcare without sacrificing patient outcomes and key
organizational objectives.

BACKGROUND

Due to limited healthcare resources, choices must be made about what

services to fund and what not to fund (Auld, Donaldson, Mitton, &
Shackley, 2002; Mitton & Donaldson, 2004). Priority setting is the process
whereby these choices are made; one key outflow of a priority setting pro-
cess is resource allocation decisions. In Canada and elsewhere, legislation
has been adopted that requires healthcare organizations to set priorities in
order to meet the health needs of the local population and develop effective
 198 CRAIG MITTON ET AL.

methods for this purpose (Her Majestys Government, 1999; Province of

Quebec, 2001; Province of Saskatchewan, 2002). Over the last two decades,
research has shown that healthcare decision makers face two main chal-
lenges with respect to setting priorities and allocating resources. The first is
a lack of skills related to those activities (Lomas, Veenstra, & Woods,
1997), while the second pertains to the organizational culture of healthcare
management where attitudes and incentives have been shaped by, and
implicitly support, allocation based on historical patterns and/or political
negotiation (Mitton & Donaldson, 2003).
Downloaded by University of Sydney Library At 22:13 19 March 2017 (PT)

In order to respond to these challenges, work in this field has focused on

building capacity around priority setting (Cohen, 1995; DH World Class
Commissioning, 2009; Donaldson et al., 2002; Gibson, Martin, & Singer,
2004). In particular, advances have been made in developing formal pro-
cesses for priority setting at both meso and macro organizational levels
(Mitton & Donaldson, 2001), examining ethical aspects of priority setting
(Daniels & Sabin, 2002; Martin, Giacomini, & Singer, 2002), understanding
how input from the public has been used in priority setting (Menon &
Stafinski, 2008; Mitton, Smith, Peacock, Evoy, & Abelson, 2009), and
driving the use of evidence in decision processes (Astley & Wake-Dyster,
2001; Peacock, 1998). Research in these and related areas has moved the
priority setting agenda away from technical solutions toward more
manager-friendly processes that typically involve some form of knowledge
exchange (Donaldson, Bate, Mitton, Peacock, & Ruta, 2005).
Despite these advances, it has become clear both for researchers and
decision makers that while identifying investment opportunities can be
straightforward, this is not at all the case for disinvestments. This is critical
since resource allocation activity must include consideration of both
investment and disinvestment options, except in the rare instances where the
process is focused solely on the allocation of new money. That is, regardless
of the size of the overall pot of resources available, in order to get the most
out of a given set of resources, except in the rare cases that deal only with
new money, some notion of shifting or re-allocating resources must be taken
on board. This can only happen through explicit consideration of proposed
disinvestments alongside options for investment (Donaldson, Bate, Mitton,
Dionne, & Ruta, 2010). Further, such activity must be carried out on an
ongoing basis if we are to maintain and sustain publicly funded health
systems (Mitton & Donaldson, 2004). That is, identifying disinvestments
is not simply a required activity when deficits are projected but rather is
necessary as part of ongoing management of limited resources, without
which overall performance is compromised and sustainability undermined.
 Criteria-based Resource Allocation 199

Disinvestments have long been a challenge for decision makers. Twenty

years ago, priority setting researchers described decision makers natural
reluctance to participate in the identification of disinvestment options
within formal priority setting processes (Cohen, 1994). More recently,
the identification of disinvestment options was still regarded as a major
hurdle in the implementation of priority setting activities (Mitton &
Donaldson, 2004, p. 118). In reference to one approach, Coast (1997) stated,
Attempts at programme budgeting and marginal analysis have shown that,
while happy to decide what should go on an incremental wish list, groups
Downloaded by University of Sydney Library At 22:13 19 March 2017 (PT)

are much more unwilling to identify services for explicit disinvestment.

This was confirmed by subsequent research which identified considerable
barriers to disinvestment at local levels (Williams & Bryan, 2007). Pearson
and Littlejohns (2007) have noted in the United Kingdom that the National
Institute for Clinical and Health Excellence (NICE) has received enthusias-
tic backing for the idea of appraising existing technologies to seek
opportunities for disinvestment; but, when followed by requests for specific
suggestions, the subsequent silence has been striking (p. 162). While
disinvestment is a necessary part of priority setting there is no doubt that it
has been difficult to achieve over time in almost all settings.
Several recent papers have addressed the issue of disinvestment, discussing
potential avenues for such action (Dionne, Mitton, Smith, & Donaldson,
2009; Donaldson et al., 2010; Elshaug et al., 2009; Robinson, Dickinson,
Freeman, & Williams, 2010). There are also practical examples of priority
setting exercises in which both investment and disinvestment options have
been identified and acted upon (Dionne et al., 2009; Haas, Viney, Kristensen,
Pain, & Foulds, 2001; Mitton, Patten, Waldner, & Donaldson, 2003;
Urquhart, Mitton, & Peacock, 2008). Further, public administration
research has considered political, economic, and other factors associated
with decisions to cutback or terminate policies or programs. This broader
evidence base suggests that in order to successfully set priorities, decision
makers require: evidence and analytical skills; organizational leadership and
support; public and stakeholder engagement; and robust decision processes
(Robinson et al., 2010; Sibbald, Singer, Upshur, & Martin, 2009).
In contrast, in most cases standard practice is to allocate resources on a
historical basis with no built-in mechanism to examine re-allocation
through identification of both investment and disinvestment options. Thus
the majority of healthcare organizations are approaching the critical issue
of disinvestment with limited experience. We would submit that this is
because of a lack of understanding of what is meant by disinvestment on
the part of decision makers and researchers as well as a lack of knowledge
 200 CRAIG MITTON ET AL.

of the tools that decision makers can use as they approach disinvestment
within priority setting processes.

What is Meant by Disinvestment?

Before going any further, it would be useful to take stock and define what is
meant by disinvestment in this context. This is important because the term
disinvestment is used with a range of meanings and there are different ways
Downloaded by University of Sydney Library At 22:13 19 March 2017 (PT)

of examining this concept definitions are diverse there are no agreed

operational criteria (Gallego, Haas, Hall, & Viney, 2010, p. 13). For
instance, Elshaug and colleagues in Australia have focused on the elimina-
tion of services that are demonstrated to be ineffective or provide no value
for money (Elshaug, Hiller, Tunis, & Moss, 2007); Nuti, Vainieri, and
Bonini (2010) and Gallego et al. (2010) use essentially the same definition.
Pearson and Littlejohns (2007) focus on adjusting the service mix rather
than eliminating services, but their guidance is still driven by clinical indica-
tions of effectiveness. Mitton and Donaldson (2004) refer to services which
are providing positive benefits but where it is considered that there may yet
be greater benefits to be had by shifting resources elsewhere. Rye and
Kimberly (2007) provide further specification by distinguishing between
discontinuation and substitution of interventions and practices. Table 1 out-
lines some of the key terms used in the disinvestment literature as well as
relevant elements for disinvestment.
In actuality a wide range of meanings are in play: stopping ineffective ser-
vices versus reducing spending on effective services; not giving planned
increases or not filling vacancies versus actual cuts in resources allocated to
certain programs or services; being more efficient (i.e., doing more with
less) versus actually reducing services; reducing service levels versus
complete termination or elimination of a service or program. In the end, we
would favor a simple definition of disinvestment to describe instances where

Table 1. Disinvestment Terms and Elements.

Terms Process Attributes Process Facilitators

Disinvestment Comparator assessment Alignment to organizational

Cost savings Involvement of staff and public mandate
Service reduction Assessment using explicit criteria Effective communication
Decommissioning Collaboration across disciplines
Divestiture Strong leadership
 Criteria-based Resource Allocation 201

overall (i.e., net) costs are decreased. The purpose and mechanism for
achieving this may indeed vary, but the point is, any cost saving is a form of
disinvestment. The implication of disinvestment may be that quality or
quantity of service is reduced, but this will not necessarily be the case as it
may be that a cheaper service re-design can still provide the same, or even
better, quality of service. Furthermore, at the organizational level, if quan-
tity or quality of a given service is reduced but the savings are re-allocated
to a higher producing service, overall quantity and quality increases.
Our personal experience suggests that the literatures lack of clarity on
Downloaded by University of Sydney Library At 22:13 19 March 2017 (PT)

the concept of disinvestment is actually mirrored amongst decision makers.

Compared to investment, which is a relatively straightforward concept,
decision makers give different and perhaps incompatible meanings of the
term disinvestment. Interestingly though, decision makers in Canada
have identified disinvestment as an area they would prioritize for further
research. In a recent management forum in the Province of British
Columbia with a broad set of healthcare decision makers, a key area identi-
fied for additional research was to examine the different ways in which
decision makers understand and apply the concept of disinvestment
(Smith, Mitton, Peacock, Cornelissen, & MacLeod, 2009).

Why is Disinvestment Required?

The basic need for disinvestment stems from an understanding of the

economic principle of opportunity cost. It stands to reason that a given set
of resources can only be spent once. In making an allocation decision, the
decision maker has implicitly or explicitly made a choice not to spend those
funds elsewhere. In so doing, there was some potential benefit attached to
the unfunded program or service area that was foregone. The opportunity
cost is that foregone benefit. The implication is that in order to get the most
out of the funds available, none of the services that are funded can produce
less benefit per dollar spent than the opportunity cost (i.e., the benefit
forgone). The opportunity cost is measured by assessing the value of the
investment options; the best alternative use of resources is the one with the
highest opportunity cost. But how do we know the value produced by
the services currently funded? Determining this value is what disinvest-
ment options are for. Disinvestment options allow us to measure the pro-
ductivity of the least marginally productive, currently funded services.
This means that disinvestment options are a necessary condition to ensure
optimal resource allocation. This applies irrespective of the financial
 202 CRAIG MITTON ET AL.

situation that is, this applies equally whether the budget is balanced, in
deficit or in surplus. One could argue that if new funds were available, tak-
ing resources from one area to re-invest elsewhere would in fact not be
necessary. But this could only be accepted if it were shown that all resources
are currently spent in the optimal manner possible (i.e., marginal benefits
exceed opportunity costs for all funded services). It is not to say that this
state is not possible but it is far more likely that any given health system will
require some shifting in order to receive more benefit out of the existing
pot of resources. The problem with new money, in our view, is that this
Downloaded by University of Sydney Library At 22:13 19 March 2017 (PT)

masks inefficiencies because decision makers have no pressing incentive (at

least in predominantly publicly funded systems) to shift resources to get the
most benefit. So the real issue in such systems is not about the concept of
opportunity cost (which is undebatable) nor even whether disinvestment
should occur (because it most likely has to in order to optimize the produc-
tivity of existing resources) but rather it is about appropriately incentivizing
decision makers to make the best decisions (subject to data and knowledge
constraints) within the available funding.
Disinvestment is a necessity in all health systems (i.e., both public and
privately funded) that have an interest in optimizing resources. We would
suggest that while the purpose of healthcare disinvestment is clear, the pro-
cesses used to achieve disinvestments, if even such a process is in place, are
typically complex and fraught with political and ethical challenges.
Without clear guidance, decision makers often defer to historical practices
or implement across the board cuts in an attempt to adopt a more sys-
tematic approach (Schmidt, 2012). While applying uniform reductions
across all programs and services may appear to be equitable, such practices
in fact often lead to increasing inequities or a deepening of historical injus-
tices under the rationale of sharing the pain equally. What is needed is a
pragmatic tool or model that is based on the economic principle of oppor-
tunity cost, and which assists decision makers in making both disinvest-
ment and investment decisions in order to allocate resources in a fair and
legitimate manner so as to achieve greater benefit overall. The approach
would need to be multiphased and lead decision makers from process
development to implementation and then tracking of outcomes.

PROGRAM BUDGETING AND MARGINAL ANALYSIS

One approach that has been used many times over in health systems across
countries is Program Budgeting and Marginal Analysis (PBMA). The aim
 Criteria-based Resource Allocation 203

of PBMA is to allocate and/or re-allocate resources within health organiza-

tions so that local or regional objectives are best met. The PBMA process,
as it has been used conventionally, asks four fundamental questions about
resource use (Mitton, Donaldson, Waldner, & Eagle, 2003; Mitton,
McGregor, Conroy, & Waddell, 2002; Ruta, Donaldson, & Gilray, 1996):
1. Within a given health organization (or specific program area), what
resources are available and how are these resources currently spent?
2. What new and existing services should receive additional funding (and
what are the costs and potential benefits of putting resources into such
Downloaded by University of Sydney Library At 22:13 19 March 2017 (PT)

growth areas)?
3. Can any existing services be provided as effectively, but with fewer
resources, allowing items on the growth list to be funded?
4. If some growth areas still cannot be funded, are there any services that
should receive fewer resources, or even be stopped, because more benefit
would be obtained by funding a growth option instead of an existing
service?
These questions relate to the two fundamental resource allocation princi-
ples underpinning PBMA: opportunity costs, as discussed earlier in the
chapter, and marginal net benefits. Answering the first question above
brings clarity to the current spending pattern. The actual budget numbers
from a given organization provide essential information but are often
not broken down along service or program lines and usually do not relate
overhead to specific services or programs.
The answers to the second through fourth questions provide the basis for
resource allocation decisions. The use of marginal analysis of benefits and
costs is required as, typically, healthcare costs do not increase at a constant
rate per patient or client and benefits are not constant across all patients or
clients. The marginal benefits and costs are those directly related to the
addition or the elimination of an incremental unit of a given program or ser-
vice. Through examining areas for service growth (i.e., question 2) alongside
of areas for resource release (i.e., questions 3 and 4), resource shifts can be
made at the margin to improve benefit overall. In this case benefit is
defined against a set of decision criteria that reflect the objectives and
strategic priorities of the given organization or health system under review.
PBMA does not alleviate the need to make difficult decisions. It does
however provide an approach that is explicit and evidence-based, and which
makes decision making more transparent and hence more accountable to
relevant stakeholders. This leads to decisions that conform more closely
with articulated values and preferences, and should prevent legal challenges
 204 CRAIG MITTON ET AL.

to resource allocation decisions (Greschner & Lewis, 2003). In short, the

marginal analysis process provides a practical mechanism to achieve gains
in any given expenditure environment through the shifting or re-allocating
of resources. The criteria serve as the basis for the entire process and, there-
fore, the same criteria must be used in assessing both investment and disin-
vestment options. Table 2 outlines the basic steps of PBMA.
Typically, decision makers manage the process themselves and lead the
evaluation of options for investment and disinvestment. Initially, health
economists or other health service researchers with knowledge of using a
Downloaded by University of Sydney Library At 22:13 19 March 2017 (PT)

criteria-based allocation process can be drawn in to support the process.

This is important as part of the aim within this context has to be around
building capacity for ongoing resource management as opposed to institut-
ing a process simply to meet a one off budget deficit. In terms of specific
examination of proposals vis-a`-vis a set of criteria, again this needs to be

Table 2. Steps of PBMA.

1. Determine the aim and scope of the priority setting exercise
Determine whether PBMA will be used to examine changes in services within a given
program or between programs; identify in and out of scope programs.
2. Compile a program budget
The resources and costs of programs may need to be identified and quantified, which, when
combined with activity information, comprises the program budget.
3. Form a marginal analysis advisory panel
The panel is made up of key stakeholders (managers, clinicians, consumers, etc.) in the
priority setting process.
4. Determine locally relevant decision making criteria
To be elicited from the advisory panel (e.g., maximizing benefits, improving access and
equity, reducing waiting times, etc.), with reference to national, regional, and local
objectives, and specified objectives of the health system and the community.
5. Identify options for (a) service growth (b) resource release from gains in operational
efficiency (c) resource release from scaling back or ceasing some services
The program budget, along with information on decision making objectives, evidence on
benefits from service, changes in local healthcare needs, and policy guidance, are used
highlight options for investment and disinvestment.
6. Evaluate investments and disinvestments
Evaluate in terms of costs and benefits and make recommendations for (a) funding growth
areas with new resources (b) moving resources from 5 (b) and 5 (c) to 5 (a).
7. Validate results and reallocate resources
Re-examine and validate evidence and judgments used in the process and reallocate
resources according to cost-benefit ratios and other decision making criteria.

Source: Adapted from Peacock et al. (2010).

 Criteria-based Resource Allocation 205

conducted by those in decision making roles in conjunction with clinical

counterparts along with finance and/or decision support personnel. Ideally,
the process is undertaken each year to inform annual budgeting decisions,
and thus in this way can become embedded into the business planning cycle.
In the mid-2000s, work completed in Canada drew together contributions
from both economics and ethics to devise a comprehensive approach
to priority setting (Gibson, Mitton, Martin, Donaldson, & Singer, 2006).
Drawing on the ethical conditions found in the Accountability for
Reasonableness framework (Daniels & Sabin, 2001), researchers and
Downloaded by University of Sydney Library At 22:13 19 March 2017 (PT)

decision makers worked collaboratively to evolve PBMA into a process that

not only focused on value for money but additionally ensured the process
itself was fair and legitimate (Gibson, Mitton, & DuBois-Wing, 2011). As a
result, the most recent form of PBMA being applied in Canada (Mitton,
Dionne, Damji, Campbell, & Bryan, 2011) and elsewhere has greater
emphasis on transparency as well as incorporating a review or appeals
mechanism along with the long-standing focus on a broad set of decision
criteria and use of best available evidence to support decisions. It is to the
application of this model that we now turn in order to illustrate its use and
the types of possible outcomes that are generated with real-world examples.

Case Studies: British Columbia and Nova Scotia

In this section, we draw on empirical work in two Canadian provinces

to demonstrate the use of PBMA and illustrate key lessons learned. In
Canada, the provinces hold primary responsibility for the delivery of health
services, so in actuality there are 10 provincial health systems (and three
territorial health systems in the north). The federal government plays a
minor role in health service delivery and focuses on financing, covering
about 20% of each provinces expenditure on healthcare, and ensuring a
common standard of services across all the provinces and territories. The
Canada Health Act sets out the minimum conditions that every provincial
insurance system must meet to be eligible for federal funding. There are
five basic conditions that must be met by the provincial insurance plans
which can be describe succinctly as follows: (1) public administration the
plans must be administered by a public authority on a nonprofit basis;
(2) comprehensiveness all necessary hospital and physician services must
be covered; (3) universality every citizen must be covered by the plan;
(4) portability individuals must be covered while in another province;
and (5) accessibility individuals must have reasonable access to
 206 CRAIG MITTON ET AL.

covered services which principally translates in practice by meaning that

no extra fees or co-payment can be charged to patients by hospitals or
physicians for covered services.
In most provinces, the systems have been devolved whereby regional
health boards or authorities oversee health services across the full conti-
nuum of care for a geographically defined area. Home care, residential care,
public health, emergency and acute services fall under the single board or
authority which receives funding from the provincial Ministries of Health to
deliver services largely autonomously. All Canadians have access to such
Downloaded by University of Sydney Library At 22:13 19 March 2017 (PT)

services through mandatory public insurance. In some but not all provinces
there is an income-adjusted premium set for individuals or families that
ranges from $50 to150 per month (and is waived entirely for low income
groups).
Physicians for the most part act outside of the regional health boards
(although they are granted admitting privileges from hospitals that are oper-
ated within the boards) and the majority are paid directly from the provin-
cial Ministry on a fee-for-service basis. Patients are free to access physician
and hospital services of their choosing although in order to see a specialist
the patient must be referred through a general practitioner. Overall in
Canada healthcare expenditure exceeds CAD$200B ($1 CAD = $0.9 US)
(about CAD$6,000 per capita) and equates to just under 11% of GDP.
In both case studies, the research team were hired in a consulting role
with the task of assisting the organization in developing a plan to address a
specific financial challenge.
The first case study comes from an organization in British Columbia
which is in one of the largest health regions in Canada serving a population
of approximately 1.5 million people with an annual operating budget of
CAD $3B. The specific financial challenge being addressed was the neces-
sary elimination of a forecasted deficit of approximately CAD$5M in one
of the divisions of the organization.1 Despite a challenging timeline of only
three months, managers and clinical leaders (programs are typically lead
jointly by a professional manager and a physician, those physicians are the
clinical leaders) worked together to not only eliminate the deficit through
application of a transparent set of criteria but to also go further and
identify options for re-allocation to program areas of high need.
A priority setting committee composed of directors and clinical leaders
within the division was established, as was an advisory panel that included
a mix of senior executive members and directors from the division. An
internal project manager worked closely with the external research team
throughout the process. Process features included a formal communication
 Criteria-based Resource Allocation 207

plan, assessment criteria, proposal rating tool, and structured business case
template. Some examples of criteria used were: alignment with mandate,
number of individuals to be affected, equity, impact on clinical outcomes,
and impact on service utilization. The criteria were weighted by decision
makers within the organization to reflect their relative importance.
Proposals for disinvestment and investment were submitted to the prior-
ity setting committee for assessment and ranking using a formal multi-
criteria decision analysis rating tool with recommendations forwarded to
the advisory panel (examples of rating in Table 3). The recommendations
Downloaded by University of Sydney Library At 22:13 19 March 2017 (PT)

Table 3. Anonymized Examples of Proposals For Change and Ratings.

Criteria Proposals for Change

Bed Reduction on 0.5 FTE Reduction of

Unit X Professional Time in
Program Y

Alignment to mandate 3 0
Efficiency, effectiveness, and appropriateness 1 1
Access 1 1
Flow and integration 0 0
Number affected 1 1
Equity 3 1
Significance of impact 3 1
Health promotion and prevention 0 0
Client experience 2 1
Workplace environment 3 1
Innovation and knowledge transfer 0 0
Implementation challenges 2 0
Impact on future service utilization 0 1
Weighted score 1.45 0.73

Note: The weighted score is a measurement of the opportunity cost for each of the proposal,
that is, if we do not keep the number of beds as it is we forego 1.45 units of benefits per dollar
saved and if we remove 0.5 FTE of professional services we forego 0.73 units of benefits per
dollar. The scoring for each criterion, for each proposal, is done by consensus and is not an
averaging of the estimates of every member of the committee charged with the proposals
evaluation. In this case the final decision, taking into account the score of each proposal, the
values of the projected savings and the total amount of savings required was to implement the
community services reduction but not the bed reduction. If investment proposals had been
considered at that stage, the minimum score of an investment would have to be above 0.73
units of benefits per dollar invested. If there were such investment opportunities, the decision
on proceeding or not would include consideration of the level of certainty around the estimates
of impact.
 208 CRAIG MITTON ET AL.

of the committee included implementation of 45 disinvestment options with

an annualized value of CAD$4.9M as well as consideration of possible
investments if the realized savings matched or exceeded expectations. These
recommendations were approved by the advisory panel and then presented
to senior executives who agreed to implement all but one of the disinvest-
ment recommendations and to support the further development of invest-
ment options.
The second case study involved an organization-wide process at a stand-
alone pediatric tertiary care facility in the Canadian Maritimes. With a
Downloaded by University of Sydney Library At 22:13 19 March 2017 (PT)

CAD$400M annual operating budget and facing significant pressure from

government to reduce expenditures, this organization applied a criteria-
based process to reduce costs and make significant re-investments to move
the organization closer to its objectives while addressing its financial
challenges. Similar to the case from British Columbia, a multi-disciplinary
priority setting committee was formed to develop and weight decision
criteria, and then review and rank both investment and disinvestment
service change options. In this case, considerable attention was paid to staff
and clinician engagement through one-on-one and small group meetings to
assist in understanding the process, to validate the decision criteria and to
engage a wide range of participants in proposal submission.
Over 350 proposals for change were submitted for review by the priority
setting committee. Of these, 112 were not advanced beyond the idea
stage. Of the almost 240 that were fully developed, approximately 75 were
investment proposals, about 75 were efficiency proposals and about 85
were service reduction opportunities. In terms of dollar values, annualized
net savings of over CAD$3M were realized (just under CAD$500,000 in
year one due to severance and other one-time costs incurred, increasing
to CAD$3M in year two and over CAD$3.5M in year three and beyond).
An important feature of this case was the participation of a public member
on the priority setting committee. Not only did this individual participate
fully in criteria development and proposal assessment, but also brought a
values lens, as a respected patient advocate.
One of the key comments in the evaluation of both of these case studies
was the need to improve process efficiency both in terms of time required
to complete the submission process for proposals and of the demands on
the priority setting committee members in managing the process. A recent
innovation in Canada is the advent of a proprietary software package
(Prioritize Softwarer) that significantly streamlines the process and
enables day-to-day single-click tracking of the process against the targets.
While other technological advancements will likely further facilitate process
implementation, it should be remembered that at the core of any priority
 Criteria-based Resource Allocation 209

setting process is human judgment and the individual and organizational

values that underlie healthcare decision making.
Several general points can be made about the case studies. First, it is
very much the case that context matters. In both of these cases, the key
impetus for initiating the activity around disinvestment was a large deficit.
This is both encouraging because they could have elected for a blunt instru-
ment like across the board cuts as the mechanism to achieve cost savings,
but on the other hand is also discouraging because as mentioned above a
key element of sound resource management is identifying options for
Downloaded by University of Sydney Library At 22:13 19 March 2017 (PT)

investment and disinvestment regardless of the fiscal climate. So the main

issue here then is drawing decision makers attention to the fundamental
need to look within to generate disinvestment proposals as part of ongoing
or routine business planning. Second, in both cases the actual process
mechanics were actually quite mundane. In these two exercises the steps
outlined in Table 2 above were followed closely. A panel was struck,
criteria were established, proposals were generated and assessed, and
recommendations followed. We point this out because truly this is not
rocket science. Yet, from a social science perspective, it is fascinating that
the use of a well-developed, tested and evaluated process, that is quite
frankly simple to use, is still very limited. Arguably the multi-criteria
approach does require some expertise, particularly in the first application,
but even there, the literature is robust and accessible to decision makers. In
our view, particularly in predominantly publicly funded systems, the chal-
lenge very clearly is not with the process or the expertise or the ability to
build capacity but rather lies in the incentives (or lack thereof) to maximize
benefit for a given budget. Third, public involvement, as illustrated by the
second case study, is not only possible but can indeed have a very clear
added value. Often decision makers shy away from transparency in deci-
sion making and at times hold to a belief that their values are an adequate
proxy for the values of members of the public. The literature on public
involvement in priority setting suggests that public members often have a
different set of views and are best drawn on for input on values rather than
technical expertise. In the second case study a public member (who was
also a patient advocate) was involved and consistently provided insight
from a patients perspective that was well received and respected.

Success Factors and High Performance

Despite some successful examples of priority setting involving an explicit

disinvestment component, clearly, releasing resources from within an
 210 CRAIG MITTON ET AL.

existing budget is not easy and in order to be successful, organizations

must ensure strong leadership is in place (Reeleder, Goel, Singer, &
Martin, 2006). Leadership has been consistently shown in the priority set-
ting literature to be the key success factor and was very clearly present in
both the case studies outlined above. That said, while research to date has
focused considerable attention on developing methods and processes for
priority setting (Mitton & Donaldson, 2004; Peacock et al., 2010), there
has been limited work understanding the factors on which decisions are
based, and, critically, the context and culture that enables these methods
Downloaded by University of Sydney Library At 22:13 19 March 2017 (PT)

and processes to succeed. In order to improve the practice of healthcare

priority setting, a better understanding of the basic factors underpinning
priority setting decisions and the associated organizational structures,
processes and behaviors is required.
With this in mind, building on previous work on success factors (Sibbald,
Gibson, Singer, Upshur, & Martin, 2010), a recent nationally funded
research project in Canada sought to identify key elements that contribute
to high performance in priority setting. The question addressed was
what is it about an organization that enables it to achieve excellence in
priority setting? Through a national survey of health authority senior
executives and then in-depth case study analysis of six geographically dis-
persed health regions from across Canada, the concept of high performance
was addressed and a table of elements that describe high performance was

Table 4. Elements of High Performance in Priority Setting.

Structure Processes Attitudes/Behaviors Outcomes

Senior management Priority setting Fit of priority setting Actual reallocation

has the ability and involves well- decisions with social of financial
authority to move defined, weighted and community resources is
financial resources criteria which reflect values is sought achieved
within and across the organizations through public
silos values and strategic participation
priorities

Staff and clinical Effective Strong leadership is Resource allocation

engagement communication displayed and decisions are
sought out leading to demonstrated justified in light of
transparency particularly when the organizations
tough decisions core values
are in play

Source: Adapted from Mitton et al. (2014).

 Criteria-based Resource Allocation 211

devised. Key factors deemed to be of critical importance included: (1) having

an identifiable, formal criteria-based process; (2) ensuring transparency and
buy-in through staff and clinician engagement; (3) identifying clinical cham-
pions to participate and assist in generating the evidence base for proposals;
(4) leadership that has the ability to re-allocate resources accordingly; and
(5) being able to justify criteria-based decision in light of strategic values.
The two case studies serve to support those findings in that all those factors
were present in both cases. A summary of the elements of high performance
is found in Table 4.
Downloaded by University of Sydney Library At 22:13 19 March 2017 (PT)

DISCUSSION

The basic claim made in this chapter is that the identification of disinvest-
ment options is an essential component of proper resource management and
that a formal, explicit process for priority setting and resource allocation
fosters organizational ability to fulfill this necessary function. In times of
fiscal constraint it may seem that disinvestment is of even greater
importance, but we have explained that regardless of the external financial
pressures, organizations can only optimize resource use if consideration of
disinvestments is part of routine resource management activity. Developing
organizational capabilities that is, the skill set to do this well underpins
much of what is being conveyed in this chapter. External expertise can be
drawn upon for a short time but in reality all organizations should be foster-
ing these skills internally, building up capacity so that resource management
activity can be carried out routinely and proactively.
PBMA is a resource allocation framework that has been around for
many years and has been applied in a wide range of contexts. The draw of
PBMA is that it forces decision makers to become more rigorous in their
priority setting activities through development and implementation of expli-
cit decision criteria. As the research work on high performance has shown,
at a very minimum, all healthcare organizations should have a well-defined,
clearly articulated process for priority setting and resource allocation that is
based on pre-established criteria that reflect the stated objectives and strate-
gic values of the organization or system (Smith et al., 2013). To put it sim-
ply, this is a non-negotiable which taxpayers in publicly funded health
systems should demand of all senior management teams. Anything less and
not only will transparency and therefore legitimacy be comprised but
resource use will most likely not be optimized, other than by good fortune.
 212 CRAIG MITTON ET AL.

Having said all of this, the skeptic might concede that PBMA, or some
other criteria-based management tool, is all well and good in publicly
funded systems, or, in the case of the United States, perhaps could concei-
vably have merit in Medicare and Medicaid programs where transparency
and stakeholder involvement should be a natural expectation. But what of
the rest of the US system? We would contend that in the United States
where systems of care are in place, a tool like PBMA is transferable, and
profit can be one criterion amongst other key criteria such as health gain,
equity and access, for example. In the end, each management team is
Downloaded by University of Sydney Library At 22:13 19 March 2017 (PT)

accountable to some entity, be it the public or shareholders, and the deci-

sion makers must be able to show why one set of choices was made over
another. The fact is, opportunity cost is a concept which transcends the
type of payment system. In order to best allocate a budget in alignment
with set objectives, multiple criteria must be defined (as even in private
systems, profit is very rarely the only relevant consideration in resource
allocation decision making) and service change proposals must be
generated. While the criteria will differ across organizations, the process to
assess proposals and the need for examining disinvestment alongside of
investment options remains.
Further research and advances in practice are required in this as in any
other field. In particular, more work on benefit measurement using a multi-
criteria decision analysis approach is required, specifically in gauging the
relevant criteria across different jurisdictions. In addition, while the need
for public involvement is well-recognized, further primary research is
needed in order to evaluate different engagement methods and solidify how
best to obtain meaningful input from different segments of society. Finally,
we need a deeper understanding of how best to employ incentives whereby
decision makers are motivated to act in the best interests of the populations
being served.

CONCLUSION
In the end, a criteria-based approach for assessing investment and
disinvestment proposals must be included in the decision maker toolbox in
supporting priority setting and resource allocation activity. This chapter
has outlined one such tool but let us be clear that we are less interested in
an organization taking up PBMA and much more interested in seeing all
decision makers charged with making funding decisions act in the best
 Criteria-based Resource Allocation 213

interest of the populations they serve. In terms of priority setting and

resource allocation, this firstly means having a process in place that is not a
simple roll over of last years budget and/or in times of constraint simply
making across the board cuts. Once a need for a process has been deter-
mined, then having identifiable and well-defined decision criteria, ensuring
involvement of staff and clinicians in formulating recommendations, and
drawing on a range of data sources to ensure decisions are based on the
best evidence available all become key tenets to success. The final catch
is properly incentivizing decision makers, particularly those in the public
Downloaded by University of Sydney Library At 22:13 19 March 2017 (PT)

sector, to take up the cross and ensure better resource stewardship, as it is

the only way that systems will be truly sustainable over time.

CONFLICT OF INTEREST STATEMENT

The authors are partners in Prioritize Consulting Ltd., a consulting firm

that owns the Prioritize Software package mentioned in the text of this
chapter.

NOTE

1. The British Columbia health authority referred to is divided into three

sub-regions, with each sub-region having separate divisions, each with its own
budget: acute care, residential care, mental health, corporate, community care, and
population health. The case took place in one of the five divisions of one of the
sub-regions.

REFERENCES
Appleby, J., Crawford, R., & Emmerson, C. (July 2009). How cold will it be? Prospects for
NHS funding: 2011 17. London, UK: The Kings Fund & Institute for Fiscal Studies.
Astley, J., & Wake-Dyster, W. (2001). Evidence-based priority setting. Australian Health
Review, 24(2), 32 39.
Auld, C., Donaldson, C., Mitton, C., & Shackley, P. (2002). Economic evaluation. In
R. Detels, W. Holland, J. McEwan, & G. Omenn (Eds.), Oxford textbook of public
health (4th ed.). Oxford: Oxford University Press.
Callan, T., Nolan, B., & Walsh, J. (2010). The economic crisis, public sector pay and the income
distribution. Bonn, Germany: Institute for the Study of Labor.
Canada Institute for Health Information (CIHI). (2009). Health care in Canada: A decade in
review. Ottawa, Canada: CIHI.
 214 CRAIG MITTON ET AL.

Coast, J. (1997). The case against. British Medical Journal, 314, 1118 1122.
Cohen, D. (1994). Marginal analysis in practice: An alternative to needs assessment for
contracting health care. British Medical Journal, 309(6957), 781 784.
Cohen, D. (1995). Messages from Mid Glamorgan: A multi-programme experiment with
marginal analysis. Health Policy, 33(2), 147 155.
Daniels, N., & Sabin, J. (2001). What are fairness and consistency in a national pharmacy
benefit? Medical Care, 39(4), 312 314.
Daniels, N., & Sabin, J. (2002). Setting limits fairly: Can we learn to share medical resources?
Oxford: Oxford University Press.
DH World Class Commissioning. (2009). World class commissioning assurance handbook
year 2. Retrieved from http://www.vision2020uk.org.uk/core_files/Assurance%20hand
Downloaded by University of Sydney Library At 22:13 19 March 2017 (PT)

book.pdf. Accessed on September 8, 2010.

Donaldson, C., Bate, A., Mitton, C., Dionne, F., & Ruta, D. (2010). Rational disinvestment.
QJM: An International Journal of Medicine, 103(10), 801 807.
Donaldson, C., Bate, A., Mitton, C., Peacock, S., & Ruta, D. (2005). Priority setting in
the public sector: Turning economics into a management process. In J. Hartley,
C. Donaldson, C. Skelcher, & M. Wallace (Eds.), Managing improvement in public
service delivery: Progress and challenges. London, UK: Nuffield Trust.
Donaldson, C., Mitton, C., Martin, H., Hasselback, P., Nelson, D., Dean, S., West, B., &
Sommer, S. (2002). Priority setting within regional funding envelopes: The use of program
budgeting and marginal analysis. Ottawa, Canada: Canadian Health Services Research
Foundation.
Dionne, F., Mitton, C., Smith, N., & Donaldson, C. (2009). Evaluation of the impact of
program budgeting and marginal analysis in Vancouver Island health authority.
Journal of Health Services Research and Policy, 14(4), 234 242.
Elshaug, A. G., Hiller, J. E., Tunis, S. R., & Moss, J. R. (2007). Challenges in Australian
policy processes for disinvestment from existing, ineffective health care practices.
Australia and New Zealand Health Policy, 4, 23. doi:10.1186/1743-8462-4-23
Elshaug, A. G., Moss, J. R., Littlejohns, P., Karnon, J., Merlin, T. L., & Hiller, J. E. (2009).
Identifying existing health care services that do not provide value for money. Medical
Journal of Australia, 190(5), 269 273.
Gallego, G., Haas, M., Hall, J., & Viney, R. (2010). Reducing the use of ineffective health care
interventions. Working Paper No. 2010/5. University of Technology Sydney, Centre for
Health Economics Research and Evaluation (CHERE), Sydney, Australia.
Gibson, J. L., Martin, D. K., & Singer, P. A. (2004). Setting priorities in health care organiza-
tions: Criteria, processes and parameters of success. BMC Health Services Research, 4,
25. doi:10.1186/1472-6963-4-25
Gibson, J. L., Mitton, C., & DuBois-Wing, G. (2011). Priority setting in Ontarios LHINs:
Ethics and economics in action. Healthcare Quarterly, 14(4), 35 46.
Gibson, J. L., Mitton, C., Martin, D. K., Donaldson, C., & Singer, P. A. (2006). Ethics and
economics: Does program budgeting and marginal analysis contribute to fair priority
setting? Journal of Health Services Research & Policy, 11(1), 32 37.
Greschner, D., & Lewis, S. (2003). Auton and evidence-based decision-making: Medicare in
the courts. Canadian Bar Review, 82, 501 533.
Haas, M., Viney, R., Kristensen, E., Pain, C., & Foulds, K. (2001). Using programme budget-
ing and marginal analysis to assist population based strategic planning for coronary
heart disease. Health Policy, 55(3), 173 186.
 Criteria-based Resource Allocation 215

Her Majestys Government. (1999). The health act. London, UK: The Stationery Office.
Lomas, J., Veenstra, G., & Woods, J. (1997). Devolving authority for health care in Canadas
provinces: 2. Backgrounds, resources and activities of board members. Canadian
Medical Association Journal, 156(4), 513 520.
Martin, D. K., Giacomini, M., & Singer, P. A. (2002). Fairness, accountability for reason-
ableness, and the views of priority setting decision-makers. Health Policy, 61(3),
279 290.
Menon, D., & Stafinski, T. (2008). Engaging the public in priority-setting for health technol-
ogy assessment: Findings from a citizens jury. Health Expectations, 11(3), 282 293.
Mitton, C., Dionne, F., Damji, R., Campbell, D., & Bryan, S. (2011). Difficult decisions in
times of constraint: Criteria based resource allocation in the Vancouver Coastal Health
Downloaded by University of Sydney Library At 22:13 19 March 2017 (PT)

Authority. BMC Health Services Research, 11, 169. doi:10.1186/1472-6963-11-169

Mitton, C., & Donaldson, C. (2001). Twenty-five years of program budgeting and marginal
analysis in the health sector, 1974 99. Journal of Health Services Research and Policy,
6(4), 239 248.
Mitton, C., & Donaldson, C. (2003). Setting priorities and allocating resources in health
regions: Lessons from a project evaluating program budgeting and marginal analysis.
Health Policy, 64, 335 348.
Mitton, C., & Donaldson, C. (2004). The priority setting toolkit: A guide to the use of economics
in health care decision making. London, UK: BMJ Books.
Mitton, C., Donaldson, C., Waldner, H., & Eagle, C. (2003). The evolution of PBMA:
Towards a macro-level priority setting framework for health regions. Health Care
Management Science, 6(4), 263 269.
Mitton, C., McGregor, J., Conroy, M., & Waddell C. (2002). Making choices in health care:
The reality of scarcity. Hospital Quarterly, 6(1), 48 54.
Mitton, C., Patten, S., Waldner, H., & Donaldson, C. (2003). Priority setting in health
authorities: A novel approach to a historical activity. Social Science and Medicine, 57,
1653 1663.
Mitton, C., Smith, N., Hall, W., Bryan, S., Donaldon, C., Peacock, S., Urquhart, B. (2014).
High performance in healthcare priority setting and resource allocation: Key elements.
Paper under review.
Mitton, C., Smith, N., Peacock, S., Evoy, B., & Abelson, J. (2009). Public participation in
health care priority setting: A scoping review. Health Policy, 91(3), 219 229.
Nuti, S., Vainieri, M., & Bonini, A. (2010). Disinvestment for reallocation: A process to
identify priorities in health care. Health Policy, 95, 137 143.
Peacock, S. (1998). An evaluation of program budgeting and marginal analysis applied in South
Australian Hospitals. Melbourne, Australia: Center for Health Program Evaluation,
Monash University.
Peacock, S., Mitton, C., Ruta, D., Donaldson, C., Bate, A., & Hedden, L. (2010). Priority
setting in healthcare: Towards guidelines for the program budgeting and marginal
analysis framework. Expert Review of Pharmacoeconomics & Outcomes Research, 10(5),
539 552.
Pearson, S., & Littlejohns, P. (2007). Reallocating resources: How should the National
Institute for Health and clinical excellence guide disinvestment efforts in the National
Health Service? Journal of Health Services Research & Policy, 12(3), 160 165.
Province of Quebec. (2001). An act representing health services and social services: Chp. S-4.2, s
(pp. 463 465). Quebec City, Canada: Queens Printer of Quebec.
 216 CRAIG MITTON ET AL.

Province of Saskatchewan. (2002). The regional health services act: Chp. R-8.2, s. 4(2). Regina,
Canada: Queens Printer of Saskatchewan.
Reeleder, D., Goel, V., Singer, P. A., & Martin, D. K. (2006). Leadership and priority-setting:
The perspective of hospital CEOs. Health Policy, 79(1), 24 34.
Robinson, S., Dickinson, H., Freeman, T., & Williams, I. (2010). Disinvestment after the new
White Paper: The challenges facing GP commissioners. Public Money and Management,
31(2), 145 147.
Ruta, D., Donaldson, C., & Gilray, I. (1996). Economics, public health and health care
purchasing: The Tayside experience of programme budgeting and marginal analysis.
Journal of Health Service Research and Policy, 1(4), 185 193.
Rye, C. B., & Kimberly, J. R. (2007). The adoption of innovations by provider organisations
Downloaded by University of Sydney Library At 22:13 19 March 2017 (PT)

in health care. Medical Care Research and Review, 64(3), 235 278.
Schmidt, D. (2012). The development of a disinvestment framework to guide resource allocation
in health service delivery organizations. Masters of Science Thesis. University of British
Columbia, Canada.
Sibbald, S. L., Gibson, J. L., Singer, P. A., Upshur, R., & Martin, D. K. (2010). Evaluating
priority setting success in healthcare: A pilot study. BMC Health Services Research, 10,
131. doi:10.1186/1472-6963/10/131
Sibbald, S. L., Singer, P. A., Upshur, R., & Martin, D. K. (2009). Priority setting: What
constitutes success? A conceptual framework for successful priority setting. BMC
Health Services Research, 9, 43. doi:10.1186/1472-6963-9-43
Smith, N., Mitton, C., Bryan, S., Davidson, A., Urquhart, B., Gibson, J., Peacock, S., &
Donaldson, C. (2013). Decision maker perceptions of resource allocation processes in
Canadian Health Care Organizations: A national survey. BMC Health Services
Research, 13, 247. doi:10.1186/1472-6963-13-247
Smith, N., Mitton, C., Peacock, S., Cornelissen, E., & MacLeod, S. (2009). Identifying research
priorities for health care priority setting: A collaborative effort between managers and
researchers. BMC Health Services Research, 9, 165. doi:10.1186/1472-6963-9-165
Urquhart, B., Mitton, C., & Peacock, S. (2008). Introducing priority setting and resource allo-
cation in home and community care programs. Journal of Health Services Research &
Policy, 13(Suppl. 1), 41 45.
Williams, I., & Bryan, S. (2007). Cost-effectiveness analysis and formulary decision making in
England: Findings from research. Social Science and Medicine, 65, 2116 2129.
 This article has been cited by:

1. Abraham B. (Rami) Shani Susan Albers Mohrman Reconfiguring the Ecosystem

for Sustainable Healthcare: Integrating Outside-In and Inside-Out Perspectives
217-242. [Abstract] [Full Text] [PDF] [PDF]
2. Susan Albers Mohrman Abraham B. (Rami) Shani Healthcare: An Ecosystem in
Transition 1-29. [Abstract] [Full Text] [PDF] [PDF]
Downloaded by University of Sydney Library At 22:13 19 March 2017 (PT)