24 June 2005 Working Paper No 8 - Benchmarking Principles

TIER 1 WORKING PAPER No. 8

BENCHMARKING PRINCIPLES

INTRODUCTION
Working Paper No. 1 ‘How Water Prices are Set’ provided an overview of how water prices are
set on the basis of lower bound costs. As noted in that paper, the expectation is that lower
bound cost pricing is determined on the premise of using the efficient costs associated with the
delivery of the particular service(s) being priced. Efficient costs are best described as the lowest
sustainable level of costs of supply incurred to provide the required level of service.
Benchmarking is often the process used to compare the cost structures of organisations carrying
out similar activities in an effort to identify the most efficient costs of operation based on
sustainable best practice management.

The purpose of Working Paper No. 8 is to provide an introduction to the principles of

benchmarking and indicate what issues are important when considering the results of a
benchmarking exercise.

WHAT IS BENCHMARKING?
The term benchmarking can be defined in many different ways and will most certainly mean
different things to different audiences. A very general definition describes benchmarking as
representing a process of identifying an organisation’s key cost drivers and performance
indicators, documenting past performances, making informed comparisons amongst peer
organisations and establishing baselines for gauging achievable improvements in the future.

In a more formal sense, benchmarking can be defined as the process of comparing the cost
structures of organisations carrying out similar activities to identify the most efficient costs of
operation based on sustainable best practice management, having regard to factors that may
cause structural differences such as the size of the organisations, geographic dispersion and
business complexity. It is also a useful tool to make comparisons of an organisation’s
performance over time.

WHY BENCHMARK?
In many ways, the process and outcomes of benchmarking represent an important exercise in
learning, understanding and appreciating the many factors that contribute to the environment in
which an organisation operates. Benchmarking also allows interested parties that are external
to the organisation itself to derive a level of confidence that the organisation is efficient or is in a
position to embrace opportunities for continuous improvement.

Apart from providing the comparative information to understand past and current performance
levels by a comparison with peer organisations, an appropriate benchmarking exercise will also
provide:
ƒ identification of potential opportunities for improvement in the future
ƒ the prospective value, in terms of both cost and service levels, to be gained from the
potential improvement opportunities
ƒ identification of any underlying factors that could impede potential improvement
opportunities or the realisation of the full value of making the improvements
ƒ prioritisation of potential improvement opportunities
ƒ realistic timelines and costs involved in achieving any potential improvement.

Page 1 of 6
24 June 2005 Working Paper No 8 - Benchmarking Principles

Ultimately, one of the outcomes of benchmarking is to ensure the prices an organisation charges
its customers reflect efficient costs of production. However, it is inappropriate to set the
objective of benchmarking as a focus on reducing prices or forcing the convergence of the prices
charged by peer organisations. The objective of benchmarking is to encourage sustainable and
optimal pricing by determining whether, in the current environment, changes or adjustments are
necessary to enhance the productivity, effectiveness and efficiency of an organisation.

BENCHMARKING METHODOLOGIES
The simplest form of benchmarking is to undertake a simple comparative performance analysis
using publicly available information. Utility industries (including the water industry) which have
been progressively reformed and deregulated over the past two decades have been subjected to
increasing requirements for public reporting of costs and service levels. Governments and
industry regulators have also required increased levels of information provision for the purposes
of monitoring and benchmarking performance. In many cases, this has been the underlying
basis for regulating actual levels of price changes and standards of service.

In more recent times, regulators have begun to look at the merits of developing more
sophisticated tools for benchmarking. Generally, one of the following three methodologies is
considered:
ƒ An extension of the simple comparative analysis approach using more sophisticated and
disaggregated performance indicators, average cost ratios and productivity indicators
ƒ Econometric Benchmarking – use of statistical and mathematical methods to generate
functional relationships between costs and outputs, input prices and other business
parameters. Econometric techniques are used to derive cost functions which are then
compared to historic and forecast data to predict any efficiency gaps. Popular techniques
including Data Envelope Analysis (DEA) and Total Factor Productivity (TFP)
• DEA – a linear programming technique which identifies best practice within a sample of
data and measures efficiency based on differences between observed and best
practice unit costs. DEA is typically used to measure technical efficiency
• TFP – the ratio of the quantity of all outputs to the quantity of all inputs
ƒ Economic/engineering analysis to develop a ‘model’ or ‘virtual’ firm.

The extent of data availability, comparability and reliability often affects the choice of which
methodology is most appropriate. Use of the more technical approaches is yet to be used on a
wide scale basis in Australia and, where they are being considered, it is more in the context of
establishing industry wide benchmarks where several utilities generally compete in the same
market (e.g. in the Victorian electricity industry where there are multiple electricity distributors).1
In other cases, where firms do not operate in a fully competitive market, theoretical yardstick
benchmarking2 methodologies are being investigated.3

1. As part of its current electricity distribution price review, Victoria’s Essential Services Commission (ESC) recently engaged
consultants (Pacific Economics Group) to assemble the necessary data and estimate total factor productivity (TFP) trends for
the five Victorian distribution businesses. Organisations such as the Productivity Commission and the Utility Regulators
Forum associated with the Australian Competition and Consumer Commission have also begun research into alternative
methods of determining and comparing utility performance.
2. Yardstick benchmarking is a concept of benchmarking of firms where there is no effective competition in the same market and a
firm’s ‘allowed revenues’ are regulated on the basis of the costs and performance indicators of firms in other markets.
3. A June 2002 report prepared by Farrier Swier Consulting for the Utility Regulators Forum (Comparison of Building Blocks and
Index-Based Approaches) indicated that although the more sophisticated econometric benchmarking techniques are
conceptually of high value, the key practical issue is the adequacy of the time series and sample size of available data.

Page 2 of 6
24 June 2005 Working Paper No 8 - Benchmarking Principles

Other potential downsides of these more sophisticated and technical benchmarking

methodologies are the technical skills required to interpret and translate the results into
meaningful implementation programs. Unfortunately, the greater technical skills needed to
reach a level of understanding of the processes and results could reduce the general confidence
of customers and limit their worthiness.

Although it engaged consultants to undertake a DEA benchmarking study of SunWater's

functional areas, the WRU also acknowledged that the DEA technique was in its infancy as
applied in the Australian rural water industry and that the true value of a DEA benchmarking
study can only be realised in the future when sufficient time series data can be collected.

KEY BENCHMARKING CONCEPTS AND ISSUES

Understanding the Relevant Cost Drivers

It is to be expected that whatever the methodology used to carry out a benchmarking analysis,
the results will invariably show differences in the costs of the organisations being benchmarked.
However, this result is in itself meaningless because a substantial component of the difference in
observed costs will be due to legitimate or uncontrollable differences in factors which affect the
level of costs incurred by individual firms, such as:
ƒ the nature or type of services provided by each firm
ƒ the range or mix of different services provided by each firm
ƒ the different of economies-of-scale that may be achievable given the volume of services
able to be provided by each firm
ƒ the quality or reliability of services provided by each firm
ƒ the prices of inputs able to be sourced by each firm
ƒ differences in business conditions, including the general business environment,
government regulations, environmental management requirements, and social obligations.

Benchmarking Inputs, Outputs or Outcomes

It is possible to undertake a benchmarking analysis based on outcomes, outputs or inputs.

Outcomes refers to the consideration of whole community well-being taking into account the
need to satisfy the total dimension of social, economic and environmental requirements.
Outcomes generally refer to the achievement of broad or global results across more than one
industry and are primarily determined and influenced by government policies. In the current
context, the pending government policy response to the rural water pricing discussion papers
released last year will contribute, but only in part, to establishing the outcomes the government
considers need to be achieved for the rural water industry. Examples of outcomes may be to
achieve a given minimum level of community well-being or the promotion of certain regional
development goals.

Outputs are the goods and services provided to the community to help achieve the outcomes.
Multiple outputs can collectively contribute to one or more outputs. Examples of SunWater's
outputs include the volume and quality of water.

Input benchmarking refers to making assessments related to the financial, physical and human
resources necessary to produce the outputs. Input benchmarks consider performance indicators
directly related to the efficacy of management’s decision making and include a focus on
corporate and indirect overhead parameters and strategic maintenance and planning activities.
Some examples of activities that are typically looked at include maintenance, operations,
customer service, human resources and information technology.

Page 3 of 6
24 June 2005 Working Paper No 8 - Benchmarking Principles

It is extremely important to recognise the differences between these three concepts and
recognise the extent to which SunWater can influence each one, particularly in the environment
in which SunWater now operates.
ƒ As already noted, SunWater does not control the determination of what outcomes need to
be satisfied as this is a function of government policy making. Generally, SunWater would
be only one of many organisation and individuals that contribute outputs to achieve an
outcome.
ƒ Whilst SunWater has a significant influence on the outputs it provides, these outputs are in
many cases determined by both the outcomes they are required to achieve and the
relevant preferences of customers to receive a given standard and quality of service for a
given price.
Benchmarking outputs requires the difficult process of normalising the quantitative results
to try to take account of the qualitative factors related to differing service standards and
quality of service.
ƒ Given the level of outputs it must provide, SunWater has a high degree of influence on the
quantity and mix of inputs as well as the processes it can utilise to produce those outputs
at least cost.

Undertaking benchmarking in the context of regulating, setting or reviewing prices of a utility

organisation focuses on processes related to the inputs of the utility rather than the resulting
outputs or the achievement of broader outcomes. Therefore, it would appear more reasonable
and valuable to benchmark SunWater on the input variables over which it has the greatest
degree of control and over which management makes most of its day-to-day decisions.

Targeting Efficiency and Best Practice

No matter how detailed the benchmarking exercise undertaken, the results really are only an
indication of the overall performance or efficiency of any organisation and caution must be
adopted if relying on the results of benchmarking to establish cost or performance targets. This
is because the very nature of most benchmarking is that it is backward-looking, concentrating on
past performance and does not really provide a solid indicator as to the capability of any
organisation to adapt to future needs and challenges.

Running any business involves a myriad of interrelated activities involving multiple processes
and sub-processes. It is unrealistic to assume that any business can reach best practice or be
efficient in each and every one of these processes. In fact, the pursuit of best practice in every
single activity or process undertaken by a firm could achieve a less than efficient result if the
increased effort required is greater than the associated cost. This also indicates why there is
danger is “cherry-picking” individual benchmark results – for example, concentrating on setting
efficiency targets for a particular activity where the benchmarking reveals the organisation is not
operating at best practice in relation to that activity without understanding the inter-relationships
between different activities and benchmark indicators. Improving performance in one activity
could mean reducing efficiency in other activities.

CONCLUSIONS
Benchmarking can be a very useful means of providing an important tool for comparative
analysis. Benchmarking can help an organisation by providing better information for decision
making purposes as well as providing regulators and customers with sufficient confidence that
the organisation is basing its decisions on the best available information.

However, use of benchmarking in a forward pricing exercise needs to recognise that the results
of any benchmarking analysis are only theoretical predictions of potential efficiency gains.
These gains may not be realised as actual gains when taking into account factors such as

Page 4 of 6
24 June 2005 Working Paper No 8 - Benchmarking Principles

service quality, structural differences between the benchmark organisations and the services
they provide as well as the costs of implementing any changes to reach best practice.

Page 5 of 6
24 June 2005 Working Paper No 8 - Benchmarking Principles

APPENDIX A
BENCHMARKING TERMINOLOGY

Best Practice
The set of management and work practices which results in the optimal (or highest potential)
quantity and combination of outputs for a given quantity and combination of inputs.

Effectiveness
The degree to which the output of a service provider achieves the stated objective of that
service.

Efficiency
The degree to which the actual use of resources to produce a given level of outputs of a given
quality matches what is considered or determined to be the optimal use of resources to produce
the same given level of outputs of the same given quality.

Efficiency is usually assessed in terms of maximising productive (technical) efficiency, allocative

efficiency and dynamic efficiency.

ƒ Productive (Technical) Efficiency – given the available technology and input prices, the
potential to increase the quantity of output from a given quantity of inputs, or the potential
to reduce the quantities of inputs used to produce a given quantity of output.
ƒ Allocative Efficiency – given the costs of different inputs, allocating inputs in the proportion
which minimises the cost of production;
ƒ Dynamic Efficiency – the extent to which an organisation can alter technology and
managerial practices to improve the quality and cost of production to respond to changes
in consumer preferences or market opportunities. Relates to the ability to implement
technological change, and undertake capital investment, research and product innovation.

Cost Efficiency
The achievement of technical efficiency and allocative efficiency. An organisation is cost
efficient when it produces a given quantity, quality and mix of outputs at minimum possible cost
given the existing level of technology and customer preferences.

Productivity
A measure of the physical output produced from the use of a given quantity of inputs. This may
include all inputs and all outputs (Total Factor Productivity) or a subset of inputs and outputs
(partial productivity).

Productivity can vary as a result of differences in production technology, differences in technical

efficiency and the external environment in which production occurs.

Page 6 of 6