A PERFORMANCE MEASUREMENT-BASED EARLY WARNING SYSTEM IN

A GOVERNMENT REVENUE ORGANIZATION

by
Elnur Asgarli
A Dissertation Presented in Partial Fulfillment
of the Requirements for the Degree
Doctor of Management in Organizational Leadership

UNIVERSITY OF PHOENIX
July 2012

UMI Number: 3536190

All rights reserved

INFORMATION TO ALL USERS
The quality of this reproduction is dependent upon the quality of the copy submitted.
In the unlikely event that the author did not send a complete manuscript
and there are missing pages, these will be noted. Also, if material had to be removed,
a note will indicate the deletion.

UMI 3536190
Published by ProQuest LLC (2013). Copyright in the Dissertation held by the Author.
Microform Edition ProQuest LLC.
All rights reserved. This work is protected against
unauthorized copying under Title 17, United States Code

ProQuest LLC.
789 East Eisenhower Parkway
P.O. Box 1346
Ann Arbor, MI 48106 - 1346

 2012 by ELNUR K. ASGARLI

ALL RIGHTS RESERVED

Abstract
The present quantitative study explored the relationship between the application of the
performance measurement system and the ability of the decision-makers of a government
revenue agency of the Republic of Azerbaijan to detect the events that could harm the
operations of the entire organization. Based on the literature review, the expectation was
that the organizational decision makers could detect organizational crises in earlier precrisis stages using a properly designed performance measurement system. Two
quantitative research designs were used by exploring one research question for each
design. A correlational design was used by running a multiple regression on 28 quarterly
observations on the quarterly tax revenue, quarterly performance report scores on audit,
cash operations control, tax debts, tax return processing, and taxpayer services, and
quarterly GDP figure of Azerbaijan. A survey design was used by administering an
online questionnaire to 344 managers of the revenue agency. The results of both research
questions indicated an overall positive relationship between the performance
measurement system and the ability of the decision-makers of a government revenue
agency to detect events that could harm the operations of the entire organization.
However, several deviations from the overall observed relationship were detected
indicating that further research may be needed to understand the relationship between
performance measurement and crisis leadership. Recommendations were also included
for the organizational leaders on improving the predictive capacity of the performance
measurement systems of their organizations.

v
Dedication
I dedicate this dissertation to my parents who always inspired me for continuous
education, and I owe to them for everything that I have achieved in this life. I also
dedicate this dissertation to my wife and my daughters who kept me energized
throughout my study and were my source of joy and happiness.

vi
Acknowledgments
I am very grateful to everyone who helped me throughout my doctoral studies. I
especially thank my mentor Dr. Richard Brunkan. Dr. Brunkan was very helpful in
providing timely and quality guidance and directing me in the right way towards
completion of my dissertation.
I thank my committee members Dr. Richard Turner and Dr. Mohammad
Shraifzadeh who supported me with their valuable feedback. I also thank my academic
advisor Carola Garfias who would regularly check my progress and help make my
doctoral studies more efficient.
I also thank Dr. Gafar Gurbanov for helping me to approach my doctoral
dissertation from a student's perspective. Dr. Gurbanov, based on his online doctoral
experience, provided me with very practical help in balancing my personal life with
doctoral studies ad making it fun and useful at the same time.
My special thanks go to the people at the Ministry of Taxes of Azerbaijan. Mr.
Fazil Mammadov, Minister of Taxes, provided overall support for my doctoral studies,
and I am very grateful to him for this support. I thank Mr. Sahir Mammadkhanov,
Deputy Minister of Taxes, who personally helped me in organizing and conducting data
collection for the dissertation at the Ministry of Taxes. I especially thank Dr. Asaf
Asadov, Advisor to the Minister of Taxes, for his exceptional support in formulating my
research idea and validating my research instrument. My thanks also go to the team of
the performance measurement section of the Ministry of Taxes, Mirgulu Mursaliyev,
Subhan Kamilzada, Ismayil Agalarov, and Ziya Askarov for their continuous support in
data collection.

vii
Table of Contents
List of Tables ............................................................................................................ xi
List of Figures .......................................................................................................... xii
Chapter 1: Introduction .............................................................................................. 1
Background of the Problem ....................................................................................... 1
Statement of the Problem ........................................................................................... 2
Purpose of the Study .................................................................................................. 4
Significance of the Problem ....................................................................................... 4
Nature of the Study .................................................................................................... 5
Appropriateness of research method .................................................................. 6
Appropriateness of research design. ................................................................... 7
Research population and instrument ................................................................... 8
Research Questions and Hypotheses ......................................................................... 9
Theoretical Framework ............................................................................................ 10
Signal detection theory. .................................................................................... 10
Performance measurement theory .................................................................... 12
The study as a bridge between the framework theories .................................... 12
Controversies in the field .................................................................................. 13
Definition of Terms.................................................................................................. 14
Assumptions............................................................................................................. 15
Scope and Limitations.............................................................................................. 15
Delimitations ............................................................................................................ 15
Summary .................................................................................................................. 16

viii
Chapter 2: Review of the Literature......................................................................... 17
Title Searches, Articles, Research Documents, and Journals .................................. 18
Literature Review..................................................................................................... 19
Historical overview. .......................................................................................... 20
Performance measurement in the first decade of the 21st century .................... 22
The purpose of performance measurement in public organizations ............ 23
Performance measurement as an early warning system .............................. 24
Crisis leadership................................................................................................ 26
Signal detection ............................................................................................ 27
Crisis response ............................................................................................. 30
Stakeholder theory of crisis management .................................................... 32
Crisis communication .................................................................................. 32
Organizational complexity ........................................................................... 33
Organizational learning from crises ............................................................. 35
Variables .................................................................................................................. 36
Research question 1. ......................................................................................... 36
Research question 2. ......................................................................................... 38
Research Gap ........................................................................................................... 39
Conclusion ............................................................................................................... 40
Summary .................................................................................................................. 41
Chapter 3: Method ................................................................................................... 43
Research Method ..................................................................................................... 43
Research Design....................................................................................................... 44

ix
Research Questions and Hypotheses ....................................................................... 46
Population ................................................................................................................ 48
Sampling Frame ....................................................................................................... 49
Sample...................................................................................................................... 50
Sample for Research Question 1....................................................................... 50
Sample for Research Question 2....................................................................... 50
Informed Consent and Confidentiality..................................................................... 52
Geographic Location ................................................................................................ 53
Instrumentation ........................................................................................................ 53
Validity and Reliability ............................................................................................ 55
Data Analysis ........................................................................................................... 58
Research question 1 .......................................................................................... 58
Joint hypotheses test. ................................................................................... 58
Research question 2. ......................................................................................... 59
Summary .................................................................................................................. 60
Chapter 4: Analysis and Results .............................................................................. 62
Data collection procedure. ................................................................................ 63
Results for Research Question 1 .............................................................................. 64
Descriptive statistics ......................................................................................... 64
Hypothesis results for research question 1. ...................................................... 67
Results for Research Question 2 .............................................................................. 71
Demographics of the sample ............................................................................ 71
Descriptive statistics ......................................................................................... 72

x
Hypothesis results for research question 2. ...................................................... 74
Summary .................................................................................................................. 76
Chapter 5: Conclusions and Recommendations ...................................................... 77
Research Questions and Hypotheses testing ............................................................ 77
Research question 1 .......................................................................................... 78
Research question 2 .......................................................................................... 78
Interpretation of the results. .............................................................................. 79
Limitations ............................................................................................................... 79
Implications.............................................................................................................. 80
Recommendations for Key Stakeholders ................................................................. 83
Recommendations for Future Research ................................................................... 84
Summary .................................................................................................................. 86
References ................................................................................................................ 88
Appendix A: Authorization Letter ......................................................................... 101
Appendix B: Informed Consent Form ................................................................... 103
Appendix C: Research Instrument ......................................................................... 107
Appendix D: Sample Performance Measurement Report ...................................... 120

xi
List of Tables
Table 1 Distribution of Key Word Searches ............................................................ 19
Table 2 Possible Choices in a Simple Signal Detection Case ................................. 28
Table 3 Summary Statistics of the Variables ........................................................... 65
Table 4 Predictors of Tax Revenue .......................................................................... 68
Table 5 Predictors of Tax Revenue in a Parsimonious Model ................................ 70
Table 5 Normality Tests ........................................................................................... 74

xii
List of Figures
Figure 1. Graphical representation of Revenue and GDP variables. ....................... 65
Figure 2. Graphical representation of independent variables. ................................. 66
Figure 3. Distribution of QPM variable. ................................................................. 72
Figure 4. Distribution of Leadership variable. ........................................................ 73

1
Chapter 1: Introduction
Organizations perform in complex environments, and changes in these
environments may result in crises and disruptive events (Mitroff, 2004; Parskevas, 2006).
Use of early warning systems and signal detection theory in management and leadership
is an emerging managerial practice for increasing organizational agility and
responsiveness before and during crises (Mitroff, 2004). The purpose of this study was to
explore whether a properly devised balanced performance measurement system may be a
basis for an early warning system of an organization.
Chapter 1 discusses the importance of the problem from social context and
research perspectives. Then, the general and specific problems of the study and the
general population for the study are defined. Afterwards, the research purpose is
discussed by explaining the research method and design and their appropriateness for the
study. The discussion goes on by explaining the significance of the study for research
and leadership, the nature of the study, research questions and hypotheses, theoretical
framework, assumptions, limitations, and delimitations.
Background of the Problem
Any discontinuity in the performance of a government agency responsible for
collecting taxes may harm the governments revenues and taxpayers rights. A
discontinuity in a revenue organizations performance may occur for various reasons, for
example, a terrorist attack, an earthquake, or a major failure of an organizations
computerized information system. Organizations need to have contingency plans for
mitigating possible and impossible crises; however, for this study, only discontinuities
that can result from misrepresentation of performance measures will be considered. Tax

2
revenues may decrease if taxpayers become increasingly dissatisfied with the
performance of the tax agency and if voluntary compliance is reduced. The revenue
agency could have predicted a decrease in taxpayer satisfaction by measuring and
analyzing the taxpayer satisfaction level in a timely fashion (DBM Consultants, 2009).
To avoid discontinuities, a revenue agency needs to scan its external and internal
environments for the signals of potential major events that may cause discontinuity or
crisis. Deverell and Olsson (2010) suggested that detecting and reading early warning
signals might improve the quality and timeliness of organizational decisions.
Organizations need some tools for predicting and preventing events that may harm their
performance.
The problem is important both socially and from a research perspective. As a
social concern, a disruptive event in the operation of a government agency of a
nationwide scale may negatively affect the lives of many citizens and organizations. As a
research focus, the problem is important because crisis leadership research is relatively
young, and theoretical and empirical research in this field may help develop the area by
incorporating other well-researched fields such as performance measurement.
Statement of the Problem
According to Mitroff (2004), early warning signals precede crises, and
organizations may sense these signals using signal detection mechanisms. The problem
is that the early warning signals are usually weak. In many cases, organizational systems
may lack proper detection mechanisms for sensing and transferring these signals to the
right people in the organization (Mitroff, 2004). If the government revenue organization
fails to predict early warning signals of the disruptions in the activities of the

3
organization, the results may be decreased revenue to the state budget, poor taxpayer
services, increased tax burden on taxpayers because of the poor tax administration and
many other negative effects (U.S. Government Accountability Office (GAO), 2009a,
2009b, 2010).
Organizational decision-makers may use appropriate tools to detect the weak
signals about the opportunities and threats facing the organization (Mitroff, 2004). One
of these tools is the performance measurement system of the organization. The specific
problem is that performance measurement systems are useful as early warning systems
only if performance measures are balanced, appropriately designed, and aligned with the
strategy of the organization (Kaplan & Norton, 2007; Parida & Chattopadhyay, 2007).
The quantitative descriptive survey research was used to study the quality of the
performance measurement system of a government revenue organization as a tool for
detecting crises. The quantitative method and the survey research design were
appropriate for this study because the purpose of the study was to test the relationship
among variables identified before the study and not emerging during the study (Creswell,
2005). The study used two data sets. Managers of government revenue organizations
who participate in making decisions under uncertainty constituted the general population
for the survey. Tax revenue, gross domestic product (GDP), and performance scores on
core tax agency processes constituted the general population for the regression analysis.
The reason for selecting the Ministry of Taxes of the Republic of Azerbaijan (the
MOT) as the source of data was that performance measurement systems used across
organizations differ from one another that results in inconsistent performance
measurement data across organizations. Another reason was that access to detailed

4
performance measurement results of many organizations was restricted. Obtaining
required data for this study from many organizations would be difficult.
Purpose of the Study
The purpose of the present study was to measure the relationship between the
application of the performance measurement system and the ability of the decisionmakers of a government revenue agency to detect potentially disruptive events or the
events that could harm the operations of the entire organization. The study may be
important for understanding the role of performance measurement as an early warning
system in the organization and how this phenomenon is accepted by the managers of the
organization. Quantitative research methods were used for the study because the
explanation of trends and relationship among variables was needed for exploring the
problem (Creswell, 2005).
The specific population for the correlational research design using multiple
regression analysis consisted of tax revenue and GDP data of the Republic of Azerbaijan
and performance measurement scores of the Ministry of Taxes of Azerbaijan. The
specific population for the survey design included 577 managers of the Ministry of Taxes
of Azerbaijan. The managers worked in different areas of tax administration that was
useful in gathering information from all functional areas of the organization. The
population for the survey study was geographically located in the territory of the
Republic of Azerbaijan.
Significance of the Problem
Fink, Siebe, and Kuhle (2004) indicated that organizations make decisions in the
presence of uncertainty, complexity, and rapid change. These three factors can produce

5
unexpected disruptive events. Fink et al. suggested that some forms of weak signals
precede these events. Organizations need to use early warning systems to guide their
decisions.
From this perspective, the results of the current study may be useful to leadership
research by combining leadership theory and decision-making theory with signal
detection theory and by providing an empirical investigation about the usefulness of this
combination. The results of the study may contribute to government performance
measurement as part of ongoing efforts to modernize the activities of the government
sector and to make government agencies as effective as private organizations (Currie,
Humphreys, Ucbasaran, & McManus, 2008; Wallis & McLoughlin, 2007). Although the
results of the study cannot be generalized to all government agencies in all countries, the
expectation is that the study results may be generalized to some extent to the government
revenue agencies operating in other countries with similar economic and social
conditions as the Republic of Azerbaijan. The results of the study may also contribute to
leadership research by providing an empirical investigation of using performance
measurement systems as early warning systems for predicting and leading crises and
disruptive events.
Nature of the Study
According to Creswell (2005), in quantitative research, the researcher conducts an
objective, unbiased study by asking specific questions, collecting quantitative data, and
analyzing these data using statistical methods. In qualitative research, the researcher
conducts a subjective study by gathering qualitative data in text, audio, video, and other
formats, asking non-specific questions, and analyzing the themes in the data (Creswell,

6
2005). Creswell further indicated that the quantitative research method is required when
the relationship among variables needs to be studied, and the qualitative research method
is required when little-known or unknown problems need to be explored, or a central
phenomenon needs to be understood in detail. The purpose of this study was to explore
the relationship among the performance measurement indicators of future performance
and organizational crises; therefore, the quantitative research method was used.
Appropriateness of research method. McBurney and White (2009) argued that
in an ideal situation, the quantitative research, specifically a true experiment should be
selected as a research method because it offers a better degree of control over variables
and results that are more valid. However, carrying out true experiments on real-life
objects and in real-life situations is not always possible. Therefore, a degree of control
over research variables and settings may be lost in exchange for arriving at a better
description of reality (McBurney & White, 2009). The major rationale in selecting the
research method was that the research problem should match the selected research
method for the present study (Creswell, 2005).
The research problem for the present study consisted of the observations that early
warning signals preceding organizational crises were usually weak and that the
performance measurement systems used by organizations may not help them detect the
early warning signals if the performance measurement systems were not appropriately
designed and implemented. These general and specific problems were studied by
investigating the variables of performance results provided in performance measurement
reports, fulfillment of tax revenue forecasts, quality of the performance measurement
system of the organization, and capacity of the organization to forecast potential crises.

7
The quantitative research method was appropriate for achieving the goals of this study
because the study explored known variables by explaining the relationships among these
variables (Creswell, 2005).
Appropriateness of research design. Because of the different natures of the
research questions, two quantitative research designs were used in the study. The
correlational research design was used to explore if the performance results predict the
fulfillment of tax revenue forecasts. The survey research design was used to gather
information systematically from the participants and quantitatively describe the
population to which the participants belonged (Groves et al., 2009) and explore there is a
relationship between the quality of the performance measurement system of the
organization and the capacity of the organization to forecast crises. The experimental
design was not selected for the study because the researcher had no control over the
extraneous variables and did not give treatments to the participants (Creswell, 2005).
According to Creswell (2005), the correlational research designs are used to
predict the scores and explain the relationship among variables. Therefore, the
correlational research designs can be of two types: a prediction design and an explanatory
design (Creswell, 2005). The current study used the correlational prediction design to
answer the first research question because the first research question explored the
existence of a predictive relationship among scores of variables.
Sjoberg, Dyba, and Jorgensen (2007) argued that survey research is preferred
when the researcher cannot or does not want to control independent and dependent
variables, when the research has to be carried out in the natural setting of the phenomena
of interest, and when the research covers the events happening in current time or the

8
recent past. The present study did fit the situation described by Sjoberg et al. (2007).
The variables of the second research question were out of the control of the researcher,
and the crises, detection of the early crisis signals, and performance measurement process
had been taking place in the recent past in their natural setting. Based on the described
features of the phenomenon of interest, survey research was an appropriate design for the
present study.
Research population and instrument. The source of the data set for the
correlational design was the performance measurement results obtained from
performance measurement reports and the reports on the fulfillment of tax revenue
forecast by the MOT. The performance measurement reports provided numerical results
of the performance of the organization in the areas of tax return processing, tax audits,
cash operations control, enforced tax debt collection, and taxpayer services. The MOT
also produced statistical reports on fulfillment of tax revenue forecast by the organization
that also contained numerical data.
Data required for the survey design were gathered using a Likert-type scale
questionnaire distributed among 577 managerial-level employees of the MOT responsible
for making decisions in different functional areas. Searching for the existing survey
questionnaire for the purpose of the present study did not produce any results. A
questionnaire specifically developed for this research was used (see Appendix C). The
questionnaire contained questions about the quality of the performance measurement
system, crises the organization had faced, and the ability of the organizational leadership
to sense the early warning signals. The questionnaire did not contain questions that were
directly about the relationship among variables (de Vaus, 2002).

9
Research Questions and Hypotheses
The MOT uses the performance measurement system for aligning organizational
strategy with the performance goals of the departments and individual employees. The
system is based on balanced measures covering different fields of organizational
activities and produces a broad picture of the organization in a simple and visual format.
The final report of the performance measurement system shows results in three colors.
Green means good performance, yellow means a near-critical situation, and red means a
critical situation. The performance measurement system contains lead indicators, so it
may point to potential future disruptive events.
The MOT, as a major government revenue agency, is responsible for ensuring
timely inflow of tax revenues into the state budget. The amount of tax revenue that the
MOT ensures will enter the state budget is forecasted by the Parliament and the
government annually. The MOT reports to the government each year about tax revenue
forecast implementation. The major external indicator for assessing the performance of
the MOT is the forecast fulfillment, and under-achievement of the forecast is considered
a crisis in the MOT (A. A. Asadov, personal communication, December 10, 2009). The
research questions were focused on exploring whether the performance measurement
system was effective as an early warning system.
RQ1: How well do the performance results predict fulfillment of tax revenue
forecast?
H10: =0, no relationship exists between performance results and fulfillment of
tax revenue forecast.

10
H1a: 0, a relationship exists between performance results and fulfillment of tax
revenue forecast.
RQ2: What is the relationship between the quality of the performance
measurement system of the organization and the capacity of the organization to forecast
crises?
H20: =0, no significant correlation exists between the quality of the performance
measurement system and the detection of crises.
H2a: 0, a significant correlation exists between the quality of the performance
measurement system and the detection of crises.
The variables used in the study to test the hypotheses were performance results
provided in performance measurement reports, fulfillment of tax revenue forecasts,
quality of the performance measurement system of the organization, and capacity of the
organization to forecast potential crises. In the first research question, the performance
results provided in performance measurement reports constituted a predictor variable, and
the fulfillment of tax revenue forecasts constituted a criterion variable. The null
hypothesis for the first research question was tested using a bivariate regression method.
For the second research question, the null hypothesis was tested by measuring the
correlation, and therefore, in the second research question, the quality of the performance
measurement system of the organization and the capacity of the organization to forecast
potential crises were not defined as either predictor or criterion variables.
Theoretical Framework
Signal detection theory. Signal detection theory provided the broad theoretical
framework for the study. Signal detection theory has long been used in technical fields

11
such as radar and sonar signal processing and communication systems, and more
recently, in emerging applications using electronic sensors (Benjamin, Diaz, & Wee,
2009; Levy, 2008). From a pure mathematical statistics perspective, signal detection is
about selecting between the two alternative statistical decisions based on the likelihood
ratio exceeding or falling short of a certain value (Laming, 2010). The observer of the
state of nature is distanced from the state of nature and uses certain communication
channels to receive signals from the state of nature. The observers performance in terms
of understanding the likelihood of the signals representing the true state of nature and
making the right decisions (hits or correct rejections) or wrong decisions (misses or false
alarms) is limited to the capacity of the communication channel used (Laming, 2010;
Lerman et al., 2010).
For the present study, the communication channel was a performance
measurement system of a state revenue agency. A decision-maker of the organization
receives, through the performance measurement system, early warning signals about
possible events that may happen to the organization. The decision-maker needs to assess
whether the event is potentially disruptive to the organization and what the likelihood of
the event is and make relevant decisions. The decision makers ability to predict future
disruptive events is limited to the capacity of the performance measurement system used
as a communication channel.
Almost all decisions are made in the presence of uncertainty, and limitation of the
capacity of the communication channel to transmit useful information has important
implications for organizational leadership. Uncertainty implies that early warning signals
on which the decision is based are not strong and certain because useful information is

12
mixed with non-useful information. The result is the noise associated with the signal.
The objective is to detect the signal and extract useful information from the noise that is
important for crisis leadership (James & Woten, 2010; Mitroff, 2004).
Performance measurement theory. An appropriately developed performance
measurement system may provide sound indicators about the past and future performance
of the organization in all fields of the organizational activity (Bhasin, 2008). Historically,
the first systematic study of performance measurement in the organization was done by
Taylor (1911). Taylor, in his Principles of Scientific Management, tried to align the
interests of employees and employers by proposing performance measures that would
increase the productivity of the employer and result in a higher salary for the employee.
After Taylor, the development of performance measurement research can be
divided into two major phases. In the previous phase, historical accounting measures of
organizational performance were researched, whereas in the later phase, balanced
measures, including lead indicators, are being researched (Kaplan & Norton, 1992).
Improving the performance measurement system by including the lead indicators of
future performance of the organization also meant that the capacity of the performance
measurement system as a communication channel in predicting the crises would improve.
Balanced performance measurement will also form the theoretical framework of the
study together with signal detection theory.
The study as a bridge between the framework theories. The two theories
constituting the theoretical framework of the present study were established at different
times and developed in different paths until they merged into the context of
organizational leadership. The research field devoted to using performance measurement

13
as an early warning system is emerging (Andersen & Fagerhaug, 2002; Spitzer, 2007).
The increasing complexity of the environment of organizations, and the increased
possibility that organizations will face a crisis because of the changes in the environment
(Paraskevas, 2006), make environmental scanning, using different early warning
indicators, important for the organization. From this perspective, the study fits into the
research framework that attempts to combine developments in the fields of crisis
leadership, signal detection, and performance measurement research.
Controversies in the field. Performance measurement research is subject to
controversial ideas and concepts. One of the main sources of controversy is
implementation of performance measurement systems. According to Paranjape, Rossiter,
and Pantano (2006), examples of successful implementation of performance
measurement systems coexist with examples of failures. Certain researchers may argue
that failures in implementing performance measurement initiatives are the result of poor
implementation skills of the organizational leadership, whereas others may argue that
some internal flaws of performance measurement result in initiative failures (Paranjape et
al., 2006; Sptizer, 2007).
Examples of failures in implementing the performance measurement system
indicate that measuring organizational performance is difficult. Researchers do not
provide straightforward guidelines on developing performance measures that are
applicable across all organizations. Organizational leaders have to answer certain
questions about how to implement the performance measurement system in their
organization, including the design and selection of measures, their implementation in
practice, and updates of these measures based on organizational change (Paranjape et al.,

14
2006). Different answers to these questions are another source of controversy in
performance measurement research (Paranjape et al., 2006).
Elg and Kollberg (2009) argued that supporters of performance measurement
focused on the technical side of performance measurement design and implementation of
performance measurement systems by accepting performance measurement as something
objectively given. Critiques of performance measurement accept it as being constructed
by certain people and being incomplete (Elg & Kollberg, 2009). To address the existing
controversy, Elg and Kollberg suggested analyzing performance measurement in a
broader organizational context, by accepting performance measurement as a part of
organizational design. The present study may be helpful in resolving the controversy in
performance measurement research by providing empirical results on the implementation
of performance measurement in an organization.
Definition of Terms
Crisis: A crisis is an event that affects or has the potential to affect the whole
organization (Mitroff, 2004, p. 63). The result of a crisis may be a loss in the lives,
property, financial earnings, reputation, and general well-being of an organization
(Mitroff, 2004, p. 63). For the purposes of this study, the phrases a crisis and a
disruptive event are used interchangeably.
Performance measurement system: A performance measurement system is a
graphical and numerical information system (often referred to as a performance
dashboard or scorecard) used to monitor, assess, diagnose, and achieve desired
performance levels (Harbour, 2009, p. 10).

15
Early warning signals: Early warning signals are any events or indications that
predict a possible crisis (Kappelman, McKeeman, & Zhang, 2006). An organization
needs to have a signal detection mechanism to sense early warning signals and act upon
them (Mitroff, 2004).
Transition economy: Countries moving from centrally planned to marketoriented economies. These countries, which include China, Mongolia, Vietnam, former
republics of the Soviet Union, and the countries of Central and Eastern Europe, contain
about one-third of the world's population (Soubbotina, 2004, p. 132). Azerbaijan is a
former Soviet Union republic and a country with a transition economy.
Assumptions
The assumption was that participants would answer questions honestly. For this
purpose, the questionnaire explicitly stated that the survey is confidential, and the
answers would not affect the participant professionally or personally. The questionnaire
also included a statement that honest answers to the survey would produce better results
that may be used to improve the performance measurement system of the organization.
Scope and Limitations
This study was limited to subjects who agreed to participate voluntarily. This
study was also limited to the number of subjects surveyed and the amount of time
available to conduct the study. Another major limitation was the possible bias in the
answers of the survey participants because of possible fear of retaliation.
Delimitations
The study was confined to surveying 577 tax administration managers at different
decision-making levels. The study focused on organizational performance data of the

16
MOT of the Republic of Azerbaijan, including perceptions of managers of the
relationship between the performance measurement results of the organizational units of
the MOT and detecting disruptive events. The research was delimited with a government
organization operating in a transition economy.
Summary
Chapter 1 introduced the dissertation and the problem and purpose statements,
along with other elements of the research design. The problem is the inability of some
organizations to detect weak early warning signals that precede organizational crises
(Mitroff, 2004). The purpose of the study was to explore whether the performance
measurement system of the organization can be used as an early warning system
(Andersen & Fagerhaug, 2002). Chapter 2 examines the literature on crisis leadership,
signal detection and early warning as important parts of crisis leadership and how
performance measurement might serve as an early warning system.

17
Chapter 2: Review of the Literature
Crises are a part of everyday life. Though crises are low-probability events, the
consequences of a crisis for an organization can be devastating. One of the positive sides
of the issue is that weak early warning signals precede the crisis, and the organization
may sense these signals if it is equipped with appropriate detection mechanisms (Mitroff,
2004). According to the Organization for Economic Cooperation and Development
(OECD), tax administrations of countries can use performance measures in different tax
areas, such as costs of tax administration, staffing levels, the number of employees
devoted to taxpayer auditing, efficiency of tax audit activities, efficiency of tax arrears
collection, taxpayer satisfaction, and other areas (OECD, 2006). Each of these
performance measures can provide leaders with valuable information regarding possible
future problems.
A sudden increase in the costs of the tax administration compared with the tax
administrations of other countries may trigger political discontent in the country.
Understaffing of the tax administration compared with the tax administrations of other
countries may be a reason for unsatisfactory organizational performance and lack of
taxpayer satisfaction. The purpose of the research was to determine whether the
performance measurement system of a government revenue agency contained early
warning indicators to sense a crisis in advance.
The literature review starts with the history of performance measurement research
to reveal how developments in the field resulted in the inclusion of early warning
indicators in performance measurement systems. The crisis leadership concept and the
difference between proactive crisis leadership based on early warning signals and

18
reactive crisis management dealing with the results of the crisis are discussed further.
The latest developments in the leadership research are analyzed from the perspective of
crisis leadership, and the implications of the literature for the present research are
discussed. Relevant literature about the variables and the methodology of the study is
discussed further, followed by the research gap in the literature.
Title Searches, Articles, Research Documents, and Journals
The search for literature on performance measurement, crisis leadership, signal
detection, and early warning signals provided abundant results. The literature on using
performance measurement as an early warning system was narrow. According to Mitroff
(2004), the crisis leadership field is relatively young, and the literature on application of
early warning and signal detection theory to organizational leadership, especially to
performance measurement, may expand in the future.
The major source of information for the literature search was University of
Phoenixs online library, which provided access to databases of scholarly texts in
different areas. The databases included EBSCOhost, Gale PowerSearch, ProQuest, and
others. Other sources of information were government reports and reports of intergovernmental organizations. Peer-reviewed literature constituted the major source of
information, though Internet search engine results were used as hints for forming
appropriate search strings.
The keywords searched for included performance measurement, signal
detection, early warning signals, crisis leadership, crisis management, and public
sector modernization. Table 1 documents the distribution of key word searches
according to the sources.

19
Table 1
Distribution of Key Word Searches
Key Words Searched

Peer-

Popular

Germinal

Reviewed

Works

Works

Books

Articles
Performance measurement

26

Signal detection

Early warning signals

Crisis leadership

Crisis management

13

Public sector modernization

The literature review included the study of 91 references, of which 86% were
published within the last 5 years and about 90% represented founding theorists, empirical
research, peer-reviewed articles, books, and journals.
Literature Review
A literature review of performance measurement and crisis leadership research
may be too broad. A crisis may occur in any or all spheres of human activity in which
complex relationships exist among people, organizations, and technologies (Mitroff,
2004). The crisis may involve separate organizations, different industries, a country, a
group of countries, and even the whole world, like the financial crisis of 2008. The
literature review considered a particular form of crisis, namely the organizational crisis.
The organizational crisis is not the sole object of the literature review. The focus
of the review is the cross-section of research in the fields of performance measurement

20
and crisis leadership. The review will start with a discussion of historical development of
performance measurement research that resulted in performance measurement systems
that allow detection of early warning signals. Then modern performance measurement
systems will be discussed. That will be followed by a discussion of the signal detection
mechanisms capable of sensing early warning signals for predicting crises. The literature
review will be narrowed to discussing how performance measurement systems may serve
as early warning systems in the organization and the research gap in the field.
Historical overview. The history of performance measurement and related
research is very interesting because it is rich and goes hand in hand with the development
of management thinking. The history of measurement may be as old as the history of
humanity; however, systematic measurement of performance for management purposes
starts with the attempts of Frederick Taylor (1911). Taylor used different performance
measurement techniques to make employees work harder and be more productive
(Radnor & Barnes, 2007). From this perspective, productivity was the major
performance indicator for Taylors scientific management.
Scientific management represents the one-dimensional performance measurement
system that prevailed in performance research for a long time (Franco-Santos et al.,
2007). In one-dimensional measurement systems, the major indicators were financial
indicators used in both private and commercial organizations (Bhasin, 2008; FrancoSantos et al., 2007; Kneiding & Tracey, 2009). If profit was the major financial indicator
for the private organization, saving taxpayers money and accountability were the major
indicators for government organizations (Yang & Hsieh, 2007). Financial measures are

21
easily calculated and seem to be obvious indicators of efficient organizational
performance (Bhasin, 2008).
Financial measures remained dominant until the researchers started questioning
using only financial indicators for measuring performance (Franco-Santos et al., 2007;
Bhasin, 2008). One of the characteristics of the financial indicators is that they provide
the results of the past performance of the organization, and when the financial data
become available, making corrections to the past performance becomes impossible
(Marc, Peljhan, Ponikvar, Sobota, & Tekavcic, 2010). As strategic management became
popular, the managers of the organization wanted to manage future performance of their
organizations, and that required applying lead performance indicators (Beamon & Balcik,
2008; Bhagwat & Sharma, 2007).
Although past financial data relate to future performance of the organization,
researchers have argued that lead indicators, like customer and employee satisfaction, are
also indicators of future performance (Kaplan & Norton, 1992; Parida & Chattopadhyay,
2007). Kaplan and Norton (1992) argued that operational performance measures such as
customer satisfaction, internal processes, and human resources indicators need to be used
together with financial indicators to arrive at a balanced picture of the organizations
performance. The logic behind this approach was that if an organization succeeds in
terms of operational performance, then financial success would inevitably follow (Davila
& Wouters, 2007; Kaplan & Norton, 1992).
As researchers began to understand the shortcomings of using only financial
performance measures, the number of new research papers proposing new performance
measurement systems increased considerably in the last two decades of the 20th century

22
(Radnor & Barnes, 2007). Even after this period, the pace of the revolution has not
slowed down, because some of the proposed measurement systems did not work in
practice. Developments in information and communication technologies permitted
improving previous measurement systems and supplying leaders with more performance
measures (Van Aken, Letens, Coleman, Farris, & Goubergen, 2005).
Although the number of research papers in the field has increased, the most
widespread performance measurement concept is still the balanced scorecard made
popular by Kaplan and Norton (1992). These authors developed the balanced scorecard
concept further and transformed it from a tool for measuring only performance to a tool
for linking organizational resources with the organizational strategy through strategy
maps (Kaplan & Norton, 2007). The use of balanced performance measures that put
strategy at the center of the measurement process was a major step toward using
performance measures as early warning indicators.
Performance measurement in the first decade of the 21st century. After the
systematic account of performance measurement in organizational management was
started by Taylor (1911) at the beginning of the 20th century, performance measurement
research has been expanded. A qualitative shift took place in the 1990s, when the focus
of performance measurement changed from internal organizational productivity to
external customer satisfaction (Ratnayake, 2009). The shift in performance measurement
research had implications for the purpose of performance measurement, led to certain
controversies in the field, and was evident in attempts by public sector organizations to
reinvent themselves by focusing on outcomes for society rather than just on

23
organizational productivity. The next subsections will provide comparative discussion of
the relevant changes in performance measurement research.
The purpose of performance measurement in public organizations. One of the
major purposes of contemporary performance measurement is to provide a link between
organizational strategy and organizational outcomes (Antic & Sekulic, 2006; Kaplan &
Norton, 2007). From a performance measurement perspective, the measures of
organizational outcomes include financial and nonfinancial indicators to get a more
balanced picture of the organization (Antic & Sekulic, 2006; Burgess, Ong, & Shaw,
2007). The organization may have several strategic directions to meet the needs of and
create value for different stakeholders, including shareholders, customers, employees, the
government, the community, and suppliers. Financial and non-financial performance
indicators may be assigned to the goals of creating value for all stakeholder groups (Antic
& Sekulic, 2006).
The goals of the stakeholders may differ; therefore, the purpose of performance
measurement for different stakeholder groups may differ (Bromberg, 2009). The
shareholders may be interested in financial dimensions of the performance measurement
system, whereas the community may be interested in corporate social responsibility
indicators. Encompassing different purposes of performance measurement for different
stakeholder groups the major theme seems to be ensuring organizational accountability in
all fields by providing information and increasing transparency. Accountability is
especially important in the case of public organizations (Bromberg, 2009; Dubnick &
Frederickson, 2010; Halachmi & Holzer, 2010; Taylor, 2009).

24
Another purpose of performance measurement for a public organization is to
improve performance and organizational productivity. The stakeholders who expect
increased accountability from a public organization also expect better performance from
it. Bromberg (2009) argued that these two goals of performance measurement, namely
accountability and performance improvement, may be difficult to achieve at the same
time. Taylor (2009) argued that a performance measurement system might not produce
reliable information for performance improvement in line with accountability
improvement. Performance measures imposed in a public organization by stakeholders
may create an additional red tape for managers and reduce productivity; therefore,
performance measures need to meet the needs of both stakeholders and managers by
providing them with useful information (Ammons & Rivenbark, 2008; Bromber, 2009).
Accountability and performance improvement assume sustainability in
organizational operations. No accountability or performance improvement can be
required of an organization that does not function for any reasons. The stakeholders are
interested in smooth and uninterrupted functioning of the organization. From this
perspective, another purpose of performance measurement is to serve as a tool for crisis
leadership in predicting early warning signals of a crisis (Bhasin, 2008). The leaders and
researchers of the 21st century are challenged to develop and use multi-dimensional
performance measurement systems to serve different purposes of performance
measurement. The research focused on the third purpose of performance measurement as
an early warning system.
Performance measurement as an early warning system. Performance
measurement is closely related to early warning signal detection. Signal detection theory,

25
in fact, is concerned with measuring performance of the observer in discriminating
signals from noise. If the observer is replaced with the organization, then on the
organizational level, signal detection theory deals with measuring performance of the
organization in detecting the early warning signals of the crises. Andersen and
Fagerhaug (2002) argued that performance measurement systems function as early
warning systems in the organization because a balanced performance measurement
system that includes lead indicators may point to potential problems in terms of market
performance or business processes.
Crises may strike organizations in different fields of activity (Paraskevas, 2006).
Because all crises send early warning signals (Mitroff, 2004), the organization needs to
have detection tools in all activity areas. Balanced performance measures may serve as
early detection tools if the measures cover the four perspectives of finance, internal
processes, customers, and learning and development (human resources). The four
perspectives of the balanced scorecard cover both operational and financial performance
of the organization, and by tracking the indicators in these fields, the organizational
leader may sense the coming crisis (Fink et al., 2004; Spitzer, 2007). Fink et al.
suggested a modified version of the balanced scorecard that includes scenarios such as
tools for picking up weak early warning signals.
Andersen and Fagerhaug (2002) created a series of activities starting from
generating knowledge inside the organization, performing business processes based on
this knowledge, creating products and services based on these processes, and finally
arriving at financial results as these products and services are sold. According to
Andersen and Fagerhaug, the time difference between knowledge development and final

26
financial results may be considerable. If only financial results are taken as performance
measures, many wrong and unsound decisions made during the stages before the
financial results appear will go unnoticed and uncorrected (Parida & Chattopadhyay,
2007). Andersen and Fagerhaug argued that using performance measures on market
performance, business processes, and knowledge development in the organization
provided the leader with effective early warning indicators.
Pride and Chattopadhyay (2007) and Iskandarani and Reifschneider (2008)
reported the importance of performance measures as early warning indicators. Hunter,
Le Menestrel, and de Bettignies (2008), Kneiding and Tracey (2009), and Plaza-Ubeda,
Burgos-Jimenez, and Carmona-Moreno (2010) argued that key stakeholder attitudes
serve as early warning indicators of future organizational performance and reputation.
Bhasin (2008) also pointed out the importance of using performance indicators as early
warning indicators; however, he argued that only if all indicators are aligned could the
performance measurement system be effective in terms of strategy implementation. The
approach is not limited to theory. The performance management software developed by
Supply Chain Management (2006) contains a module on early warning indicators that,
based on considerable deviation of indicators from a historical mean, warns the user of
the possible problem.
Crisis leadership. The performance measurement system may allow
organizational leaders to deal with organizational crises proactively as part of crisis
leadership. No unified definition of crisis exists (Coombs, 2010). Braden, Cooper,
Klingele, Powell, and Robbins (2005) defined organizational crisis as an unexpected,

27
dramatic, and often unprecedented event that forces an organization into chaos and may
destroy the organization without urgent and decisive action (p. 1).
Virtually all crises send early warning signals, and according to Mitroff (2004),
the best form of crisis leadership is the ability to detect these signals and avoid the crisis.
From this perspective, crisis leadership is proactive. Crisis management, on the other
hand, is active and becomes visible after the crisis happens.
Signal detection. Mitroff (2004) argued that signal detection is the most important
element of crisis leadership, and the new organizational structure dealing with crises
needs to include signal detection mechanisms. If early warning signals preceding the
crisis are detected and acted upon, the organization may avoid (the best form of crisis
leadership) or mitigate the crisis (Mitroff, 2004). Although signal detection is used in
organizational crisis leadership, the roots of signal detection stem from technical fields
such as reading signals on the radar screen to understand whether enemy planes are
approaching (Abdi, 2010), or in the mathematical statistics and electronic
communications fields (Levy, 2008). The major theme of signal detection in all fields is
that almost all decisions are made in the presence of uncertainty (Levy, 2008), and signal
detection helps managers to understand the dimensions and characteristics of the error in
decision-making under uncertainty (Cook, Noyes, & Masakowski, 2007).
Signal detection theory usually involves differentiating decision-making stimuli
created by known processes from accidental stimuli because of chance. The former
stimuli are called signal and the latter are called noise (Goldstein, 2009). A classical
example of the essence of signal detection in a simple matrix format is shown in Table 2.
The radar operator may see on the screen of the radar dots indicating that some flying

28
objects are approaching the border (Cook et al., 2007). The dots may represent enemy
planes (signal) or a flock of birds (noise). The operator needs to decide whether the dots
indicate the enemy planes or not.
Table 2
Possible Choices in a Simple Signal Detection Case
Decision
Reality

Yes

No

Enemy planes (signal)

Hit

Miss

Other objects (noise)

False Alarm

Correct Rejection

In the case shown in Table 2, the objective is to maximize the proportion of hits
and correct rejections and decrease the proportion of false alarms and misses. The
easiness of the task and the strategy of the decision-maker will affect how the objective is
achieved (Abdi, 2010). If the noise level is lower, and distinguishing the signal from the
noise is easier, the proportion of hits and correct rejections will be higher. The
appropriate balance between hits and correct rejections is very important because if the
operator decides to say yes more often than no to increase the proportion of hits, he
or she will increase the proportion of false alarms as well. The result will be wasted
resources in terms of time, money, and fuel when planes are sent to fight non-existing
enemy planes (Cook, Noyes, & Masakowski, 2007).
Table 2 can be compared to Type I and Type II errors in statistics. When making
inferences about the population based on a sample from this population, the researcher
needs either to reject the null hypothesis or not reject the null hypothesis (Creswell,

29
2005). In this case, four outcomes are possible. The researcher may erroneously reject
the null hypothesis when the null hypothesis is true (Type I error) or may fail to reject the
null hypothesis when the null hypothesis is not true (Type II error). In other cases, when
the researcher rejects the null hypothesis when the null hypothesis is not true, and when
the researcher does not reject the null hypothesis when the null hypothesis is true, no
error is committed.
The signal detection theory example may successfully be applied to the early
warning systems of an organization, as well. Researchers have indicated the importance
of detecting early warning signals in dealing with organizational crises (Braden et al.,
2005; Garcia, 2006; Mitroff, 2004; Paraskevas, 2006). Then organizations need
appropriate detection mechanisms to distinguish early warning signals from false alarms.
As in the radar example, responding to false alarms may waste valuable organizational
resources, while missing the true signals may damage the organization even more
(Mitroff, 2004).
Early warning signals of a crisis are usually weak and are mixed with noise. The
source of this noise may be both internal and external. If the people in the organization
do not possess the necessary skills to detect early warning signals, or the organization
does not have a mechanism for transferring the signal to the right people (Mitroff, 2004),
the internal noise will be present. External noise is the result of the complexity of the
environment in which the organization operates. Complexity of the external environment
increases the number of interrelated factors that the leaders need to take into account in
making decisions and makes it difficult to identify the factor or set of factors causing the
crisis (Fink, Marr, Siebe, & Kuhle, 2005).

30
The organization needs to have mechanisms in place to scan the changes in
different environmental elements for improving organizational decision-making taking
into account early warning signals of the crises, (Mitroff, 2004). The complexity of both
the organization and the organizational environment makes signal detection even more
difficult (Lagadec, 2006; Paraskevas, 2006). In this case, removing all noise is not
advisable because the noise can carry some useful information (Aliev, 2007). Lagadec
argued that responding to contemporary crises with traditional reactive emergency
measures is not sufficient, and the new response strategy needs to address a very broad
circle of players.
Crisis response. When a crisis occurs, a timely response to the crisis usually
becomes very important (Garcia, 2006). Garcia argued that a slow response to a crisis
sometimes becomes more dangerous to the organization than the scale and severity of the
crisis. Decreasing crisis response time depends on how well the organization is prepared
for the crisis, including the presence of an early warning system, the designation of a
senior official to deal with the crisis preparation, and speedy authorization of
organizational decisions during initial crisis detection (Garcia, 2006). Successful
responsive crisis management depends on effective pro-active crisis leadership (Wooten
& James, 2008).
The recent oil spill of the Deepwater Horizon oil rig of British Petroleum (BP) in
the Gulf of Mexico indicates how early warning signs can help an organization avoid
disaster (Gulf oil spill, 2010). According to the preliminary investigation of the
Committee on Energy and Commerce of the U.S. House of Representatives (Waxman &
Barton, 2010), the explosion on the oil rig that caused the death of 11 employees and a

31
major oil spill in the Gulf was preceded by three major early warning signals. Fifty-one
minutes before the explosion, the amount of fluid pumped in started to outweigh the
amount of fluid coming out of the well; 41 minutes before the explosion, the well
continued to flow even though the pump was shut down; and 18 minutes before the
explosion, mud return increased, the pressure increased abruptly, and the pump was shut
down (Waxman & Barton, 2010). Nevertheless, company employees could not process
these signals appropriately, did not take enough preventive actions, and as a result, loss of
human lives and considerable damage to the environment occurred.
Crisis leadership is not easy. In theory, if crises are preceded by early warning
signals, a leader can detect these signals and prepare for the crisis in advance, manage
through the crisis, and deal with the following crisis in the same way. The theory may be
oversimplified when applied to real-life crises. Crises are not clearly identifiable through
space and time because the crisis may dynamically oscillate over considerable time,
sometimes scaling down, sometimes becoming severe (Boin & Rhinard, 2008;
Moynihan, 2008). If the conflict between the environment of the organization and
organizational values and belief system is not resolved, the crisis may continue until the
organization is obliterated (Braden et al., 2005).
The difficulty of crisis leadership and management is evident in examples of crisis
leadership failure. According to Jaques (2008), examples of badly managed crises
outnumber examples of well-managed crises in the crisis management literature. This
literature is still expanding for deeper analysis of the reasons and ways of better
managing the crises.

32
Stakeholder theory of crisis management. Alpaslan, Green, and Mitroff (2009)
connected crisis management to the stakeholder theory and argued that companies
adopting the stakeholder theory of corporate governance have more chances to engage in
proactive crisis leadership behaviors. The basic assumption of this approach is that all
crises can harm organizational stakeholders. Knowing how different groups of
stakeholders will behave during and before the crisis can help organizations prevent
crises or recover from them successfully (Alpaslan, et al., 2009). Stakeholders are
diverse, and diversity within the same stakeholder group is increasing because of
racial/ethnic majority growth, womens share growth, and other factors (Acquier, Gand,
& Szpirglas, 2008; Chin & Sanchez-Hucles, 2007). The performance measurement
system becomes even more important as a tool for predicting the behavior of different
organizational stakeholders.
Crisis communication. According to the research commissioned by Weber
Shandwick, a global public relations agency, 71% of 514 working Americans who
participated in the survey indicated that they expected their companys leadership to
communicate about the global financial crisis, but 54% of the participants said they did
not hear at all from their companys leadership about the crisis. The research concluded
that during hard times, employees need trustworthy information about their uncertain
future, and organizational leadership can fill this information gap and strengthen the
companys reputation. Rouse and Schuttler (2009), in the research report on the online
survey of 1,150 working American adults, indicated that leaders mostly did not change
their communication behaviors during the crisis, and even worse, many supervisors used
intimidating language during the crisis. Woodward and More (2010) also stressed the

33
importance of leadership communication during the crisis and afterwards in managing
relationships among the organization and stakeholders and the resulting reputation of the
organization.
Crisis communication and stakeholder theory of crisis management are closely
interrelated. According to Hunter et al. (2008), leadership communication affects the
feelings of the stakeholders about the organization. If the leader tries to take a defensive
position based on incomplete information instead of dealing with key stakeholder
concerns, the damage to the organizations reputation may increase (Coombs, 2007;
Hunter et al., 2008). Woodward and More (2010) argued the ability to communicate at
times of volatility is a fundamental competency for contemporary leaders (p. 130). The
implication of crisis communication literature for the current study was that the research
instrument should contain questions about the organization-stakeholder relationships and
leaderships communication during crisis.
Organizational complexity. Organizations are complex systems working in
complex environments; therefore, crises may be of different kinds that reflect
malfunctions in various elements of complex systems (Paraskevas, 2006). Leaders
cannot be prepared for all kinds of crises; however, organizations may cultivate the sense
of crisis leadership through their strategic plans in all organizational activities
(Hawabhay, Abratt, & Peters, 2009; Jaques, 2007). Wooten and James (2008) argued
that for a long time, research on crisis management focused on communication issues
during the crisis, and little attention was paid to the leadership skills required during and
before the crisis. Wooten and James indicated that leadership competencies are required

34
in all phases of crisis leadership, and one of these competencies is the ability to detect
early warning signals preceding the crisis.
Leadership in organizations as complex systems needs to consider the
interconnections among its elements and environment that requires reconsidering the
traditional reductionism and determinism approaches to analyzing the organization
(James, 2010). According to Scharmer (2009), traditional and modern industrial
structures or forms of thinking and operating (p. 3) were successful in the past; however,
they are not suitable for contemporary leadership and are the reason for failures of
leadership. Scharmer introduced Theory U, which aims to solve the core problems of
leadership practice and theory. If other leadership theories deal mainly with what leaders
do and how they do it, Theory U tries to address the source of leadership decisions as the
inner place from which leaders operate (Scharmer, 2009). According to Theory U, the
source of thinking is invisible to leaders, and therefore is called the blind spot
(Scharmer, 2009).
Scharmer (2009) approached crises from a different perspective in Theory U.
According to Scharmer, the crises of the contemporary world are not the results of the
failure of a leader, an organization, or a country. Crises happen because old social
structures are becoming broken, and simultaneously, norms or values are being lost. The
result is the chaos that produces crises in different areas of human life (Scharmer, 2009).
Scharmer (2009) argued that, when dealing with crises, experience is not useful
and coined the term presencing (a combination of the words presence and sensing),
which means that the leader needs to look at the future, see what the most potential future
state is, and change the current actions to bring that future into reality. Presencing is

35
described by Scharmer (2009) as illuminating the blind spot (p. 17) of leadership.
Lowe, Aparicio, Galbraith, Dorman, and Dellert (2009) argued that performance
measures could be used to help practitioners understand their blind spots by comparing
their performance results with those of others. Fried and Orellana (2006), on the other
hand, argued that the way the performance measurement system is used in an
organization could create blind spots because this system cannot represent the entire
reality of the organization.
To illuminate the blind spots created by the use of the performance measurement
system and ensure that organizational reality is expressed, organizational leaders need to
evaluate the effects of the performance measurement system on organizational practices
periodically and make appropriate changes to the system (Fried & Orellana, 2006). This
process of reevaluation has implications for the performance measurement system of the
organization as an early warning system. According to Scharmer (2009), an organization
may miss early warning signals because of its past successful experiences that are not
successful now. In some cases, organizational culture needs to change so that the
performance measurement system functions as an early warning system.
Organizational learning from crises. Dennis and Eliott (2009) identified three
types of relationships that differ between a crisis and learning: organizations can learn to
prevent a crisis; the learning process can trigger changes in the core beliefs by resulting
in a crisis within the organization, and finally, organizations can learn from the crisis.
According to Deverell (2009), a crisis can trigger different levels of organizational
learning based on the number of crises the organization has faced, single or double loop
learning that the organization engages in, and the presence of external critique and loss of

36
credibility. Eliott (2009) argued that one reason for the repetition of crises is the
organizations failure to learn from past crises. Eliott asserted that after the crises, the
policy is changed and improved most of the time, whereas, practice in the organization is
not affected, and the lessons learned from the crises are not transferred into the practicing
organizations. The implication of this discussion for the current study is that the research
instrument should contain questions about what the organization learned from the crisis
to assess the efficiency of the crisis leadership system of the organization.
Variables
Research question 1. The first research question of the present study explored
how well the performance results predict the fulfillment of tax revenue forecast or in a
broader sense is there a predictive relationship between the performance measurement
results and performance of the organization. The literature review provided mixed results
on this question. In the first research question, performance was a criterion or dependent
variable. For testing the relationship using quantitative techniques, performance should
be made operational or empirically testable (Neuman, 2003). Performance cannot be
observed directly; therefore, proxy variables should be used (Dant, Kacker, Coughlan, &
Emerson, 2007).
March and Sutton (1997), in their seminal article, argued that using organizational
performance as a dependent variable is problematic for several reasons. The first is that
performance improvements are quickly diffused across organizations in the industry, and
statistical techniques cannot reliably detect the relationship between the performance
improvement and the determinants of this improvement across organizations. The second
reason is that performance may affect its determinants, thus decreasing the predictive

37
capability of the statistical models. The third reason applies to situations in which
performance is made operational through surveys by asking questions about past events.
In this case, the retrospective bias can be observed because the current performance of
the organization may affect how the participants comprehend its past performance
(March & Sutton, 1997).
The study used the tax revenue collection as a proxy for organizational
performance to overcome the problem with the variables of the first research question.
The data about performance of the organization were not collected through a survey
among employees to avoid the retrospective bias mentioned by March and Sutton (1997).
The data came from tax revenue reports provided by government statistics and
represented a more objective basis for measurement than personal opinions.
Controlling confounding variables is very important for avoiding conclusions
about irrelevant relationships among variables. For instance, when testing the
relationship between the results of the performance measurement reports and fulfillment
of tax revenue forecasts, many other extraneous variables may affect this relationship.
The researcher might not be interested in measuring these extraneous variables; however,
neglecting these variables may negatively affect the outcome of the test. In this case, the
major issue is to identify the major extraneous variables that may affect the relationship
tested and put them into the equation for testing the relationship among variables.
Otherwise, the relationship observed after testing the relationship equation could be either
overestimated or underestimated (McDonald, 2009).
Although in an experimental setting the researcher can control the experiment and
account for all or most of the confounding variables, in management research, controlling

38
for confounding variables is not possible. The researcher simply does not have control
over all the variables that define the complex system of the organization and its
environment. Nevertheless, the most important confounding variables need to be
identified and considered in the data analysis (Varkevisser, Pathmanathan, & Brownlee,
2003). For the study, the growth rate of the gross domestic product (GDP) constituted a
major confounding variable.
The tax revenue level is significantly associated with the level of the GDP growth
rate (Arnold, 2008). Given that tax policy does not change significantly, the expectation
is that when the GDP level increases, the tax base increases, and the tax revenue level
would increase as well. Testing the effect of the results of the performance reports as a
predictor variable on the tax revenue forecast implementation as a criterion variable
would result in under or overestimated outcomes without considering the effect of GDP
growth on tax revenues. The quarterly GDP growth rate was considered a confounding
variable and controlled for in the analyses.
Research question 2. In the second research question, the relationship between
the quality of the performance measurement system of the organization and the capacity
of the organization to forecast a crisis was measured. Operationalization of the second
research questions variables was even more difficult because quantifying quality and
capacity was not straightforward. In general, the literature research disclosed a positive
relationship between the quality of the performance measurement system of the
organization and the capacity of the organization to forecast a crisis (Andersen &
Fagerhaug, 2002; Bhasin, 2008); however, the established relationship was theoretical
and required empirical testing.

39
The capacity to predict a crisis is a leadership competence (Peterson & van Fleet,
2008). Peterson and van Fleet used a questionnaire comprised of questions about
different leadership behaviors and competencies that can be expected from a leader
during a crisis. The participants rated their preference of the behaviors using a certain
scale. The summary of these answers was the basis for Peterson and van Fleets
empirical analysis of crisis leadership behaviors.
The research also used a topics-based questionnaire for the survey to identify the
relationship between the quality of the performance measurement system of the
organization and the capacity of the organization to forecast a crisis. For the performance
measurement section, each topic represented the characteristics of a good performance
measurement system identified during the literature research. For the crisis leadership
section, each topic represented the leadership competencies required during a crisis that
were also identified during the literature research. By answering the questions, the
participants rated how the performance measurement system and crisis leadership in their
organization fitted the common qualities and competencies identified during the literature
research.
Research Gap
The controversies in the field of performance measurement may indicate for the
need for additional research. Performance measurement history is rife with examples of
failures in implementing performance measurement systems (Paranjape et al., 2006).
Further research is required to determine whether the failures result from implementation
errors or from internal flaws in performance measurement logic (Sptizer, 2007). Even
successful performance measurement implementations are organization-specific, and

40
leaders need to answer certain questions before starting any performance measurement
initiative, beginning with the design and selection of performance indicators and
extending to updating these indicators based on organizational change (Paranjape et al.,
2006). The study may help resolve the controversy in performance measurement
research by providing empirical results about the implementation of performance
measurement in an organization.
When performance measurement is considered in the cross-section of crisis
leadership research, the research gap widens further. Using performance measurement as
an early warning system is a relatively new idea first suggested by Andersen and
Fagerhaug (2002) as a separate phrase. Application of early warning systems to
organizational management also happened in the relatively recent past and requires
further research (Mitroff, 2004). More research into using performance measurement as
an early warning system is needed to identify the common elements of both research
fields and propose successful combinations of those elements. Further research will have
to explore the sufficiency of existing performance measurement systems to serve as early
warning systems, the need for adding new dimensions to performance measurement, the
role of leadership in using the early warning signals provided by performance
measurement, and other issues.
Conclusion
The literature review indicated that research into using performance measurement
systems as early warning systems was relatively young. Though several authors noted
the ability of performance measurement systems to detect early warning systems (Fink et
al., 2004; Parida & Chattopadhyay, 2007), an Internet search for the phrase performance

41
measurement as an early warning system revealed that only Andersen and Fagerhaug
(2002) separately defined performance measurement as an early warning system. Mitroff
(2004) indicated that crisis leadership is a relatively young field and requires further
research. Early warning signal detection is a part of crisis leadership; therefore, the
literature gap exists in integrating crisis leadership research with performance
measurement research.
The results of the present research may contribute to the fields of performance
measurement and crisis leadership because the research questions about the capacity of a
performance measurement system to provide information about future disruptive events
will be explored. The first research question was devoted to determining whether the
performance measurement system provided useful information about the future
performance of the organization. The second research question focused on exploring
whether the performance measurement system can pick up the early warning signals of
the crises. The literature review of the research in the field suggested positive initial
answers to the research questions; however, the results of the research may have
determined whether the same situation exists in the real-life application of performance
measurement in an organization.
Summary
The literature review suggested that performance measurement, which includes
lead indicators, together with historical financial indicators, might point to potential
problems of an organization. Early warning about these problems may help prevent them
from turning into high-consequence crises. Mitroff (2004) even argued that detection of
early warning signals is the most important constituent of crisis leadership.

42
Performance measurement research has come a long way in devising
measurement systems that cover all aspects of organizations. Traditionally, performance
of organizations was measured based on financial indicators. Financial indicators are
historical indicators and can provide information about the past performance of the
organization. Later, researchers started proposing that using only financial indicators
does not provide enough information about future performance of the organization, which
is important in devising and implementing organizational strategy (Davila & Wouters,
2007; Kaplan & Norton, 1992). As a result, new performance measures were proposed to
provide a picture of future performance of the organization, which can be regarded as
early warning indicators.
Chapter 2 presented a review of the research literature on using performance
measurement as an early warning system. Chapter 3 presents the methodology of the
present study. It includes a discussion of the population and sample, sampling methods, a
description of the survey instrument, and data analysis. An explanation of reliability and
validity is also presented.

43
Chapter 3: Method
The purpose of the present quantitative descriptive survey research was to
measure the relationship between the application of a performance measurement system
and the ability of the decision-maker of a government revenue agency to detect
potentially disruptive events. If early warning signals preceding the crisis are detected
and acted upon, then the organization may avoid or mitigate the crisis (Mitroff, 2004).
The objective is to maximize the proportion of correct detections and correct rejections,
as well as, decrease the proportion of false alarms and false detections (Cook et al.,
2007).
Chapter 3 is devoted to explaining the methodology for establishing quantitative
relationships among variables. Chapter 3 describes the research design and research
questions and hypotheses. Chapter 3 also explains the instrument used in the study, data
collection, and analysis methods.
Research Method
The research method in this study was quantitative. According to Creswell
(2005), in quantitative research, the researcher conducts an objective, unbiased study by
asking specific questions, collects quantitative data, and analyzes these data using
statistical methods. In qualitative research, the researcher conducts a subjective study by
gathering qualitative data in text, audio, video, and other formats, asking non-specific
questions, and analyzing themes in the data (Creswell, 2005). Creswell further indicated
that the quantitative research method is required when the relationship among variables
needs to be studied, and the qualitative research method is required when a little or

44
unknown problem needs to be explored or a central phenomenon needs to be understood
in detail.
McBurney and White (2009) argued that in an ideal situation, quantitative
research, specifically true experiments should be selected as a research method because it
offers a better degree of control over variables and results that are more valid. However,
carrying out true experiments on real-life objects and in real-life situations is not always
possible. Therefore, a degree of control over research variables and settings may be
sacrificed to arrive at a better description of reality (McBurney & White, 2009).
The research problem for the present study consisted of the observations that early
warning signals preceding organizational crises are usually weak and that the
performance measurement systems used by organizations may not help them detect the
signals if the performance measurement systems are not appropriately designed and
implemented. These general and specific problems were studied by investigating the
variables of performance results provided in performance measurement reports,
fulfillment of tax revenue forecasts, quality of the performance measurement system of
the organization, and capacity of the organization to forecast potential crises. The
quantitative research method was appropriate for achieving the goals of this study
because the study explored known variables by explaining the relationships among them
(Creswell, 2005).
Research Design
Research design provides a frame of reference for interpreting the results of
research (De Vaus, 2002). Good research design provides a useful context for drawing
sound conclusions, selecting among alternative interpretations, and making conclusions

45
convincing (De Vaus, 2002). The study used two quantitative research designs: a
correlational design and a survey design to explore the research questions (Creswell,
2005). The experimental design was not selected for the study because the researcher
had no control over the extraneous variables and did not give treatments to the
participants (Creswell, 2005).
For the correlational design used to explore the first research question the source
of data for the analysis was the performance measurement results obtained from
performance measurement reports and data on the fulfillment of tax revenue forecast by
the Ministry of Taxes (MOT). The data obtained from performance measurement reports
and tax revenue forecast fulfillment reports were analyzed using regression methods. An
example of the performance measurement report of the MOT is shown in Appendix D.
Both sets of reports cover the period from 2005 to 2009. The year 2005 was selected as a
starting period because the performance measurement system of the MOT had been in
place since January 1, 2005. The analysis explored the predictive relationship between
the two reports.
For the second research question a survey research design was selected to gather
information systematically from the participants and describe the population to which the
participants belong (Groves et al., 2009). Sjoberg, Dyba, and Jorgensen (2007) argued
that survey research is preferred when the researcher cannot or does not want to control
independent and dependent variables, when the research has to be carried out in the
natural setting of the phenomena of interest, and when the research covers the events
happening in current time or the recent past, all of which were characteristics of the
present study. The variables of the study were out of the control of the researcher, and

46
the crises, detection of the early crisis signals, and performance measurement process had
been taking place in their natural setting from recent times until the current time. Based
on the described features of the phenomenon of interest, the survey research was an
appropriate design for the second research question.
Research Questions and Hypotheses
The MOT uses the performance measurement system for aligning organizational
strategy with the performance goals of the departments and individual employees. The
system is based on balanced measures covering different fields of organizational
activities and produces a broad picture of the organization in a simple visual format. The
final report of the performance measurement system shows results in three colors. Green
means good performance, yellow means a near-critical situation, and red means a critical
situation. The performance measurement system contains lead indicators; therefore, it
may point to potential future disruptive events.
The MOT, as a major government revenue agency, is responsible for ensuring
timely inflow of tax revenue into the state budget. The amount of tax revenue to the state
budget to be ensured by the MOT is forecast by the Parliament and government annually.
The MOT reports to the government each year about tax revenue forecast
implementation. The major external indicator for assessing performance of the MOT is
the forecast fulfillment, and non-fulfillment of the forecast is considered a crisis in the
MOT (A. A. Asadov, personal communication, December 10, 2009). The research
questions were aimed at exploring whether the performance measurement system was
effective as an early warning system.

47
RQ1: How well do the performance results predict fulfillment of tax revenue
forecast?
H10: =0, no relationship exists between performance results and fulfillment of
tax revenue forecast.
H1a: 0, a relationship exists between performance results and fulfillment of tax
revenue forecast.
RQ2: What is the relationship between the quality of the performance
measurement system of the organization and the capacity of the organization to forecast
crises?
H20: =0, no significant correlation exists between the quality of the performance
measurement system and the detection of crises.
H2a: 0, a significant correlation exists between the quality of the performance
measurement system and the detection of crises.
The variables used in the study to test the hypotheses were performance results
provided in performance measurement reports, fulfillment of tax revenue forecasts, the
quality of the performance measurement system of the organization, and the capacity of
the organization to forecast potential crises. In the first research question, the
performance results provided in performance measurement reports constituted a predictor
variable, and the fulfillment of tax revenue forecasts constituted a criterion variable. The
null hypothesis for the first research question was tested using a bivariate regression
analysis. For the second research question, the null hypothesis was tested by measuring a
correlation, and therefore, in the second research question, the quality of the performance

48
measurement system of the organization and the capacity of the organization to forecast
potential crises were not defined as predictor or criterion variables.
Population
The study used two datasets that were obtained from different populations.
Population for the correlational design consisted of the tax revenue and GDP data of
Azerbaijan and the performance measurement scores of the MOT. Azerbaijan gained
independence from the Soviet Union in 1991 and started transitioning from a totally state
controlled economy to a market economy. Before 1991, all enterprises and their incomes
belonged to the state and no income tax and value added tax were collected, therefore, the
population for the tax revenue and GDP data covered published statistical data starting
from 1991. Population of performance measurement scores were available for the period
starting from 2005 when the performance measurement system was established at the
MOT.
The population for the survey consisted of 577 managers of the major government
revenue agency of the Republic of Azerbaijan. These managers used the performance
measurement system of the organization regularly to report and track performance of
their units. The managers who constituted the population were in the frontline
operations, close to the taxpayers. If some problem occurred with the operations of the
organization, these managers would possess valuable information for detecting early
warning signals of the crises.
According to the report published by the organization (Ministry of Taxes, 2006),
88% of all employees of the agency are males. The percentage of females among
managers is even lower and equals 1% because only three female managers work for the

49
agency. Employees in the age group of 31-50 years constitute 66% of the employees.
The remaining employees are equally divided between the age groups of below 30 and
over 50 (17%). No information was provided about the age structure of the managers of
the agency; however, the assumption is that the age structure of the managers is similar to
that of all employees.
Sampling Frame
The target population is an abstract concept and needs to be operationalized by
defining a sampling frame (Neuman, 2003). According to Neuman (2003), a sampling
frame is a specific list that closely approximates all the elements in the population (p.
216). The data tables provided by the MOT and the State Statistics Committee of
Azerbaijan constituted the sampling frame for the multiple regression analysis used in the
correlational design. The telephone directory of the organization, containing all
managerial positions in the MOT, constituted the sampling frame for the survey study.
The relatively small size of the target population (577 individuals) for the survey
and the opportunity to computerize the data collection process through the online survey
allowed for inviting all target population members to participate in the survey. The
sample size was not limited, and oversampling was ensured. The suggested sample size,
calculated using the online sample size calculator by Creative Research Systems (2010),
was equal to 231, given a margin of error of 5%, a confidence level of 95%, a population
size of 577, and a response distribution of 50%.
According to Deutschlander (2009), a response rate of 10% to 40% in can be
expected in business surveys, whereas a higher response rate may be expected in surveys
conducted within an organization. The suggested sample size of 231 participants

50
constitutes approximately 40% of the target population. Maxfield and Babbie (2009)
argued that a response rate of 50% was acceptable for analysis and reporting.
Considering that the survey is conducted within an organization and that management
endorsed the survey, 50% response rate can be expected.
Sample
Sample for Research Question 1. The sample for regression analysis in the
framework of the correlational design was comprised of quarterly data covering the
period starting from the first quarter of 2005 and ending at the fourth quarter of 2011.
The data were in the form of 28 observations on the quarterly tax revenue, quarterly
performance report scores on audit, cash operations control, tax debts, tax return
processing, and taxpayer services, and quarterly gross domestic product (GDP) figure of
Azerbaijan. The 28 observations were not independent of one another because
performance in one period may be related to performance in the previous periods.
Besides, although the observations for each quarter were random variables, each variable
had only one sample size. Lack of independence among observations and one sample
size required application of time series analysis and restrictive statistical models
(Kirchgssner & Wolters, 2008).
Sample for Research Question 2. The survey was hosted on the internal website
of the MOT with a special link available to the participants only through the electronic
invitation letter sent by the deputy minister of taxes. The survey was hosted on the
website for three weeks from January 19, 2012, to February 9, 2012. The survey link was
distributed to 577 managers, of whom 444 participated in the survey. Some of the

51
participants (37 people) started the survey, however, did not complete it and quit without
submitting the answers.
An accidental, however, interesting issue was observed during the survey. One of
the respondents using the contact information shown on the questionnaire reported that
two questions of the survey repeated one another. The programmer, who wrote the
survey script and placed it on the internal server, admitted that he made a mistake when
transferring the questionnaire from a test server to the main server and questions 2.4 and
2.7 appeared on the questionnaire twice.
After the survey ended and the data obtained from the survey were inspected, the
observation was that some participants answered differently to the questions that were
mistakenly repeated word by word. For example, a participant may have selected the
first answer choice in the first of the repeated question, however, the third answer choice
in the subsequent repeated question. The number of these participants was equal to 63
and they were also excluded from data analysis. The basis for this exclusion decision
was the high probability that these participants did not even read the questions carefully
and answered the questions only for the sake of participation in the survey.
After the answers of 37 participants who quit the survey without submitting
answers and 63 participants who did not provide reliable answers were excluded from the
survey only answers of 344 participants who completed the survey were considered in
calculations. The response rate was equal to 60%. The number of participants was well
over the required sample size of 231 participants.

52
Informed Consent and Confidentiality
The Deputy Minister of Taxes granted permission for conducting the survey in the
organization by signing an authorization letter (Appendix A). The survey questionnaire
was placed on the internal website of the organization. All managers of the organization
received, through the internal electronic correspondence system of the agency, a
communication letter from the head of the organization inviting them to participate in the
survey. The letter contained a link that directed the potential participant to the informed
consent page of the questionnaire.
The informed consent page for this study (see Appendix B) introduced the topic
of study and the purpose of the survey to the potential participant. The informed consent
page also described the anonymity and confidentiality of the participants identity,
voluntary participation in the survey, and the right to quit the survey anytime without any
penalty. The informed consent page contained agreement and disagreement buttons at
the bottom of the text. The participant was directed to the first page of the questionnaire
only if he or she clicked on the agreement button. That indicated the participants
positive decision to participate.
The questionnaire for the survey did not contain questions about name, contact
details, or other sensitive information that may reveal the identity of the respondent, and
was anonymous. The informed consent form contained information about the
confidentiality of the participant. Only the researcher had access to the survey data. The
results of the survey were saved in a password-protected file on a compact disc and will
be kept in a secure place for 3 years after the study for the reviewers. After 3 years, the
disc will be burned.

53
Geographic Location
The study was limited to the managers working at the MOT on the territory of the
Republic of Azerbaijan. The headquarters of the MOT is in Baku, the capital of
Azerbaijan; however, the organization collects taxes and provides taxpayer services
throughout the entire country. The MOT has no foreign operations.
Instrumentation
Data for the survey were gathered using an online Likert-type scale questionnaire
distributed among the 577 managerial-level employees responsible for making decisions
in different functional areas. No existing survey questionnaire for the purpose of the
current study exists. As a result, a questionnaire specifically developed for this research
was used (Appendix C). The questionnaire contained questions about the quality of the
performance measurement system in predicting performance of the organization, which
crises the organization faced, and whether the performance measurement system could
detect the signals of these crises in advance.
The questionnaire was placed on the internal website of the MOT and
communicated to the managers of the organization in a letter from the head of the
organization. A three-week timeframe was allocated for completing the questionnaire.
To ensure that only managers had access to the survey, the hyperlink to the survey page
was sent only to those managers. This process could be controlled in the internal
electronic correspondence system of the organization. A special script was used to
prevent the same person from responding to the survey more than once.
The variables were quantified using a Likert-type scale questionnaire that
contained 23 questions in two parts and took, on average, approximately 30 minutes to

54
complete. The answers to six demographic questions, which were not in Likert-type
scale, were coded as one for the first answers choice and one additional unit for the each
next answer in order of appearance. Likert-type scale used in the second and third parts
of the questionnaire had five answer choices that were coded as five - strongly agree, four
- agree, three - neither agree nor disagree, two - disagree, and one - strongly disagree.
The questions were designed in such a way that reverse coding was not required. In
cases when multiple questions measured the same concept, summated scaling was used to
find the final score for the question (De Vaus, 2002).
The experts of the Performance Measurement Division of the MOT were asked to
comment on the grammar and clarity of the survey questions. These experts worked
daily with the performance measurement system of the organization and could provide
valuable feedback on improving the survey questions. The experts provided feedback
only on the grammar and clarity of the survey questions, and their possible biases do not
affect the research results.
Developing an Internet survey may require complex programming languages (De
Vaus, 2002). Therefore, a local programmer was hired to write a small program for
conducting the survey. This programmer had experience in developing Likert-type scale
questionnaires and placing them on the internal website of the MOT.
The program prepared by the programmer automatically coded the answers and
entered the answers in the database, thus making them available for data analysis.
Automatic coding eliminated manual data entry and coding stages and reduced coding
errors (De Vaus, 2002). Another positive aspect of using the program was that if invalid

55
data were entered or several answers were selected when only one answer was allowed,
the participant could be reminded of this error (De Vaus, 2002).
Validity and Reliability
According to Neuman (2003), validity is about truthfulness, and reliability is
about consistency. If the constructs that the researcher uses to describe the reality do not
fit the reality, the validity of the study suffers (Neuman, 2003). Achieving 100% validity
is not possible; however, the validity of the research can be improved by using certain
validation techniques (Creswell, 2005). Content validity indicates to what extent the
survey questions represent all the possible aspects of the construct being studied, in this
case performance measurement.
To ensure content validity, four performance measurement experts of the MOT
were asked whether the survey questions represented the performance measurement area
(Cooper & Schindler, 2003; Creswell, 2005). The experts suggested several changes for
improving the wording of the questions. After these changes, the experts unanimously
reported that the questions did cover all the facets of performance measurement.
Reliability is also important for increasing the usefulness of the research results.
Reliability indicates that repeated measurements produce similar results (Cooper &
Shindler, 2003; Neuman, 2003). Achieving perfect reliability is difficult, and the
researchers may use different ways to improve the reliability of the research (Neuman,
2003).
First, all constructs should be clearly identified, and each measure should be
linked to a single concept (Neuman, 2003). Otherwise, the measure may be associated
with different concepts by the respondents, and the results may include measurements of

56
other, unwanted concepts. For this study, the survey questions were discussed with a
professional statistician to ensure that ambiguity in measures was avoided.
Another way of increasing reliability is to measure information that is specific or
to increase the precision of measurement by increasing the level of measurement
(Neuman, 2003). The levels of measurement are nominal, ordinal, interval, and ratio
levels in the order of increased preciseness of measurement. More specific information
allows more specific measurement of the construct (Neuman, 2003). The reliability of
the present study was increased by making questions as specific as possible and using the
ordinal level of measurement that allows for both measuring the difference among
answers and ranking the answers (Neuman, 2003).
Using different indicators of the same variable also can increase reliability. In
this case, measuring different aspects of the variable can produce more stable results than
using a single indicator (Neuman, 2003). Using multiple-item Likert-type scale questions
in the survey may have increased reliability, since they offered multiple-indicator
measurement (De Vaus, 2002).
Cronbach's alpha was used to test the reliability of the research instrument.
Cronbach's alpha is used to test the internal consistency of the items of the questionnaire
in measuring the single construct (Andrew, Pedersen, & McEvoy, 2011). This coefficient
represents a correlation between responses and if the correlation between the responses to
the questions that measure the same construct is high then the internal consistency of the
questionnaire is high as well. Cronbach's alpha level of above .7 is considered acceptable
(Andrew et al, 2011).

57
The native language of the survey participants is Azerbaijani language; therefore,
the informed consent and survey questions were translated into Azerbaijani. The first
step in validating the survey was contacting Asaf Asadov, the advisor of the Minister of
Taxes of the Republic of Azerbaijan who is the initiator of the performance measurement
system in the organization. All questions were reviewed with the advisor for clarity of
meaning and sensitivity.
The review focused on not changing the questions but making some questions less
academic and understandable for average participant who had a bachelor degree and was
not well informed of complex management terms used in the leadership literature.
Besides, some sensitive questions asking directly assess the leadership performance of
the top management were modified to increase the chance of sincere answers to the
questions. The result of the review was a validated modified questionnaire that had a
more explicit explanation of terms like crisis leadership and provided examples for clarity
of meaning. The validated research instrument is provided in the Appendix C.
To validate the research instrument further a pilot study was conducted among the
employees of the department responsible for the functioning of the performance
measurement system in the organization. A total of 20 participants participated in the
pilot survey. The purpose of the pilot survey was to ensure the face validity of the
questionnaire. The participants of the pilot survey were asked to provide their
observation on the questions of the survey having unintended meaning, being sensitive,
and not very complex for comprehension. Though no essential changes were made to the
questionnaire, the feedback of the pilot survey participants helped to improve the
wording of the questionnaire.

58
The pilot survey participants followed the same procedure as the main survey
participants. They were provided with the link to the electronic survey placed on the
internal website of the organization. After clicking the link, they were taken to the
informed consent page and after clicking the agreement button, they were taken to the
questionnaire page. The pilot survey also helped to ensure that electronic survey
functions smooth without any technical problems and that the participants can access the
survey page only after indicating their agreement to participate on the survey on the
informed consent page.
Data Analysis
Research question 1. The data obtained from performance measurement reports
and tax revenue forecast fulfillment reports were analyzed using OLS (Ordinary Least
Squares) regression analysis methods. Both sets of reports were on a quarterly basis and
covered the 5-year period from 2005 to 2011, resulting in 28 observations per each
report. The 28 observations were not independent of one another because performance in
one period might be related to performance in the previous periods. Although the
observations for each quarter were random variables, each variable had only one sample
size. Time series issues had to be addressed in the analysis methods used to account for
the lack of independence and one sample size (Kirchgssner & Wolters, 2008).
Joint hypotheses test. The first research question explored the relationship
between the performance scores and the tax revenue. The performance scores data on
five processes of the Ministry of Taxes were used in data analysis. Therefore, the
estimated OLS model involved five variables in addition to a confounding variable and a
time trend variable. Running OLS on variables provides estimates for individual

59
variables, however, does not indicate he joint effect of the variables on the dependent
variable (the tax revenue). To test the joint effect of performance scores on the tax
revenue, the joint hypotheses test needs to be performed (Wooldridge, 2009).
Wooldridge (2009) suggested using F test for joint hypotheses testing. F test
involves the calculating the following statistic:

where, SSRr is the sum of squared residuals from the restricted model, SSRur is the sum
of squared residuals from the unrestricted model, q is the number of restrictions, n is the
number of observations, and k is the number of parameters in the unrestricted model.
The restricted model is the one without the performance score variables and the
unrestricted model is the one with all variables. If the Fq, n-k-1 statistic calculated is
statistically significant, then the null hypothesis that performance scores have no joint
effect on the tax revenue can be rejected.
A priori expectation was that the relationship between performance scores and tax
revenue might be either positive or negative. In either case, the tax revenue can be
statistically predicted by the performance scores. Therefore, a two-tailed hypotheses
testing was used in multiple regression analysis. The hypotheses were tested at 5% level
of significance.
Research question 2. The survey produced quantitative data that were analyzed
using descriptive and inferential statistics. Descriptive statistics were used to summarize
the patterns of the data. For descriptive statistics, frequency tables, graphs, and summary
statistics were used (Cooper & Schindler, 2003).

60
Using inferential statistics conclusions were drawn about the population based on
the sample of respondents who participated in the survey (Creswell, 2005). Shapiro-Wilk
test was used to test the normality of the data (Garca, Molina, Lozano, & Herrera, 2009).
The data distribution were not normal, therefore, a non-parametric Spearmans rank
correlation coefficient was used to test the association among variables (Doane &
Seward, 2009). A two-tailed hypotheses testing was used at 5% level of significance.
Summary
The purpose of the present quantitative descriptive survey study was to measure
the relationship between the application of the performance measurement system and the
ability of the decision-maker of the MOT to detect potentially disruptive events. The
population for the correlational research design was the tax revenue and GDP data of
Azerbaijan and the performance measurement scores of the MOT. The data were
analyzed using multiple regression analysis.
The managers working for the organization over the territory of Azerbaijan
constituted the population for the survey research design. The managers were asked to
participate in a Likert-type scale survey placed on the MOTs internal website. The data
collected from the survey were analyzed using descriptive and inferential statistics.
Chapter 1 identified the problem statement, purpose of the study, research
questions and hypotheses, and a theoretical framework. Chapter 2 provided a literature
review of the field of performance measurement and early warning signal detection and
the intersection of two fields of research. Chapter 3 provided the methodology of the
research for testing the hypotheses and exploring the research questions. Chapter 4

61
provides the results of the data analysis, and Chapter 5 focuses on interpreting the results
obtained in Chapter 4.

62
Chapter 4: Analysis and Results
The purpose of the present study was to measure the relationship between the
application of the performance measurement system and the ability of the decisionmakers of a government revenue agency to detect potentially disruptive events. The
current study proposed the following research questions and related hypotheses:
RQ1: How well do the performance results predict fulfillment of tax revenue
forecast?
H10: =0, no relationship exists between performance results and fulfillment of
tax revenue forecast.
H1a: 0, a relationship exists between performance results and fulfillment of tax
revenue forecast.
RQ2: What is the relationship between the quality of the performance
measurement system of the organization and the capacity of the organization to forecast
crises?
H20: =0, no significant correlation exists between the quality of the performance
measurement system and the detection of crises.
H2a: 0, a significant correlation exists between the quality of the performance
measurement system and the detection of crises.
Chapter 1 stated the general problem for the study was that if the government
revenue organization fails to predict early warning signals of the disruptions in the
activities of the organization, the results may be decreased revenue to the state budget,
poor taxpayer services, increased tax burden on taxpayers because of the poor tax
administration and many other negative effects (U.S. Government Accountability Office

63
(GAO), 2009a, 2009b, 2010). The specific problem was that performance measurement
systems are useful as early warning systems only if performance measures are balanced,
appropriately designed, and aligned with the strategy of the organization (Kaplan &
Norton, 2007; Parida & Chattopadhyay, 2007). Chapter 2 explained how the literature
covering the study problems and research questions evolved over time, the current status
of research and the existing literature gap. Chapter 3 described the research design,
research questions and hypotheses, the instrument used in the study, data collection, and
analysis methods. Chapter 4 reports the results of data analysis for each research
question.
Data collection procedure. The procedure of data collection was different for the
research questions. The correlational design for the first research question involved
running a multiple regression model on quarterly data covering the period starting from
the first quarter of 2005 and ending at the fourth quarter of 2011. Quarterly tax revenue
and performance report scores data were provided by the Economic Analysis Department
of the Ministry of Taxes (MOT). Quarterly GDP figures were retrieved from the website
of Azerbaijan State Statistics Committee. All variables were entered into MS Excel file
for further analysis.
The second research question was explored using the survey research design, and
the data were collected by the online questionnaire placed on the intranet of the MOT for
three weeks. Data collection for the second research question required more time as
compared with the time required for data collection for the first research question. The
process started with the programmer writing a script for the questionnaire and placing it
on the intranet by allocating a specific link to the questionnaire. The deputy minister

64
signed the survey participation invitation letter and sent the electronic letter with the link
to the questionnaire to the target population. Target population members participated in
the survey and in the end of the period allocated for the survey; all answers to the
questions were entered into an MS Excel file for further analysis.
The reliability of survey answers to the questions about the quality of
performance measurement and crisis leadership was measured using Cronbachs alpha by
a reliability test performed in SPSS software. The Cronbachs alpha for both sets of
questions were equal to .96. If Cronbachs alpha is close to one, then higher internal
consistency of the items measured by the survey questions can be assumed. According to
Gliem and Gliem (2003), the reasonable internal consistency may be assumed if the alpha
level is equal to or greater than .8. Therefore, the questionnaire provided a reliable
measurement of the constructs.
Results for Research Question 1
Descriptive statistics. Descriptive statistics calculations provided general
statistical properties of the data. In addition, graphical descriptions were used to
visualize the relationship among variables. Table 3 presents summary statistics of the
variables. In Table 3, tax revenues (Revenue variable) are represented in 10 million
AZN, gross domestic product (GDP) is represented in 100 million AZN, and other
variables are represented as percentages.

65
Table 3
Summary Statistics of the Variables
Variable

Mean

Median

Maximum

Std. Dev.

100.73

106.68

25.69

161.38

39.81

Audit

97.39

97.45

96.30

98.80

0.63

Cash

95.55

95.60

92.70

98.80

1.84

Debts

97.71

97.60

97.00

99.20

0.59

Returns

98.05

98.25

94.90

99.70

1.23

Services

99.50

99.15

98.50

102.00

0.81

GDP

81.08

91.03

21.33

154.86

35.61

Revenue

Minimum

Figure 1 provides a graphical representation of the dependent variables Revenue

and the confounding variable GDP. The mainly upward time trend is present in the data.
Furthermore, Revenue and GDP variables seem to be related. In other words, it appears
that as GDP figures increase the tax revenue also increases.
Revenue

GDP

180.0
160.0
140.0
120.0
100.0
80.0
60.0
40.0
20.0
0.0
Q1 Q3 Q1 Q3 Q1 Q3 Q1 Q3 Q1 Q3 Q1 Q3 Q1 Q3
20052005200620062007200720082008200920092010201020112011
Figure 1. Graphical representation of Revenue and GDP variables.

66
Figure 2 provides the graphical representation of the independent variables used
in the regression. One immediate observation is that time trend is not present in the
independent variables. The graph does not indicate a well-defined relationship among
independent variables. The Cash variable line appears to be below the lines of other
independent variables pointing to the possible relatively poor performance of the
Ministry of Taxes (MOT) in the area of cash operations control. However, the latter
relationship is not proven by analyses and is just suggested based on observations of the
graph.
Audit

Cash

Debts

Returns

Services

102.0

100.0

98.0

96.0

94.0

Q1 2005
Q2 2005
Q3 2005
Q4 2005
Q1 2006
Q2 2006
Q3 2006
Q4 2006
Q1 2007
Q2 2007
Q3 2007
Q4 2007
Q1 2008
Q2 2008
Q3 2008
Q4 2008
Q1 2009
Q2 2009
Q3 2009
Q4 2009
Q1 2010
Q2 2010
Q3 2010
Q4 2010
Q1 2011
Q2 2011
Q3 2011
Q4 2011

92.0

Figure 2. Graphical representation of independent variables.

An observation based on Figures 1 and 2 is that the variables without a time trend
(Audit, Cash, Debts, Returns, and Services) need to be compared with the variables with a
time trend (GDP). In this case, the variables with a time trend need to be de-trended to
prevent a spurious regression when the relationship among variables is found simply

67
because the variables are growing over time (Wooldridge, 2009). Wooldridge suggested
adding time trend variables in the regression for avoiding spurious regression. Therefore,
the variable t was added to the multiple regression model, representing the time trend
from the first quarter of 2005 to the fourth quarter of 2011.
Hypothesis results for research question 1. The hypotheses of the first research
question were tested by running a multiple regression analysis of quarterly data covering
the period starting from the first quarter of 2005 and ending at the fourth quarter of 2011.
Quarterly tax revenue constituted the dependent variable. Quarterly performance report
scores on audit, cash operations control, tax debts, tax return processing, and taxpayer
services were used as independent variables. In addition, the absolute quarterly GDP
figure of Azerbaijan was entered as a confounding variable.
Quarterly tax revenue and performance report scores data were provided by the
Economic Analysis Department of the MOT. Quarterly GDP figures were retrieved from
the website of Azerbaijan State Statistics Committee. All variables were entered into MS
Excel file for further analysis.
Multiple regression analysis was used to answer the research question and test the
hypotheses using the econometric software package Gretl. The dependent variable titled
the quarterly tax revenue was labeled as Revenue. The independent variables titled the
audit score, cash operations control score, tax debts score, tax returns processing score,
and taxpayer services score were labeled respectively as Audit, Cash, Debts, Returns, and
Services. The confounding variable titled the quarterly GDP was labeled as GDP. Also,
the time trend variable, labeled as Time, was introduced to consider the effect of time
trend on the relationship among dependent and independent variables.

68
The following multiple regression model was estimated by the ordinary least
squares (OLS) method using Gretl software:
Revenue = C + 1Audit + 2Cash+ 3Debts +4Returns + 5 Services + 6GDP + 7t
The results of the two-tailed hypotheses test are displayed in Table 4.
Table 4
Predictors of Tax Revenue
Tax revenue
Variable

Constant

794.29

Audit

17.37*

Cash

-4.39

Debts

-13.93*

Returns

-0.80

Services

9.95

GDP

0.88**

0.06

Adjusted R2

0.80

16.72

Note. N=28
* p < .05.
The adjusted R-squared was equal to 0.80 indicating that 80% of variation in the
dependent variable was explained by the model. The Durbin-Watson statistic was equal
to 1.89. Table 4 indicates that Audit, GDP, and Debts were significant at 5% significance
level.

69
For testing the joint effect of performance scores on the tax revenue, a joint
hypotheses test was performed using an F ratio as suggested by Wooldridge (2009).
Based on the suggestion of Wooldridge (2009) the following two-tailed hypotheses about
the joint effect of variables were tested using an F ratio:
H0: 1=2=3=4=5=0
Ha: At least one of 1, 2, 3, 4, or 5 is different from zero
The F statistic for the joint hypotheses test was equal to 2.91 with numerator
degrees of freedom equal to five and denominator degrees of freedom equal to 19. The
critical value of F(5;19) is equal to 2.74 at 5% significance level. The F statistics is
greater than its critical value; therefore, the null hypothesis is rejected and the variables
Audit, Cash, Debts, Returns, and Services are jointly significant at 5% level of
significance.
The observation that some variables were not significant partially may point to the
misspecification of the estimated original model. Ramsey Regression Equation
Specification Error Test (RESET) test was used to test the misspecification of the original
model (Wooldridge, 2009). The test results, F(2,18) = 2.03, p = 0.16, did not allow to
reject the null hypothesis of misspecification. Therefore, a new parsimonious multiple
regression model was estimated using the ordinary least squares (OLS) that excluded
partially nonsignificant variables tested in the original model:
Revenue = C + 1Audit + 2Debts +3GDP + 4t
The results of the two-tailed hypotheses test are displayed in Table 5.

70
Table 5
Predictors of Tax Revenue in a Parsimonious Model
Tax revenue
Variable
Constant

B
67.38

Audit

12.15*

Debts

-12.53

GDP

1.06*

-0.73

Adjusted R2
F

0.78
25.58

Note. N=28
* p < .05.
Omission of variables improved two of three model selection criteria reported by
Gretl. Schwarz criterion (253.95) and Hannan-Quinn criterion (249.33) were improved,
while Akaike criterion (247.29) did not improve. RESET test results were also improved,
F(2,21) = 4.10, p = 0.03, indicating that at 5% level of significance the null hypothesis of
misspecification of the new parsimonious model can be rejected.
In the new model, the adjusted R-squared was equal to 0.78 indicating that 78%
of variation in the dependent variable was explained by the model. The Durbin-Watson
statistic was equal to 1.58. Table 4 indicates that Audit and GDP were statistically
significant at 5% significance level, while Debts was not statistically significant.
Audit and GDP had positive signs and significantly predicted the increase in tax
revenues. On average, one point increase in the quarterly audit performance score

71
predicted 121.5 million AZN increase in quarterly tax revenues. On average, 100 million
AZN increase in quarterly GDP predicted 10.6 million AZN increase in quarterly tax
revenues.
Results for Research Question 2
Demographics of the sample. The participants were asked six demographic
questions about their age range, gender, work experience, experience of using the
performance measurement system, education level, and the department in which they
work. Most of the participants (35%) were between 40-49 years old. The rest were over
50 years old (26%), between 30-39 years (25 %) and between 20 and 29 years (14%).
Also, the majority of the sample (95%) were male participants.
The majority of the participants (70%) had more than 9 years of experience.
Managers with less than 9 years of experience constituted only 30% of participants. The
majority of the participants (52%) had more than 5 years of experience of working with
the performance measurement system. The participants who had less than 3 years of
using the performance measurement system constituted 36%.
The majority of the participants (78%) had a bachelor's degree or a 5-year
diploma that was the substitute of the bachelor's degree in Azerbaijan before the collapse
of the Soviet Union. Only 2% of participants reported that they had a doctorate degree.
The remaining group of participants (20%) had a master's degree.
The majority of the participants (45%) worked in regional tax departments.
Participants working in the central office, the audit department, and the Baku city tax
department were approximately equally distributed (approximately 17%). The smallest

72
groups of participants were from the large taxpayers' department and the tax crimes
department.
Descriptive statistics. Figure 3 shows the distribution of summated answers to
the question about the quality of the performance measurement system or QPM variable.
The data distribution seems to be skewed. Majority respondents selected more favorable
answers than non-favorable answers. The histogram suggests that QPM variable does not
have a normal distribution.

QPM
120

Frequency

100
80
60
40
20

0
23 26 28 31 33 36 39 41 44 47 49 52 54 57 60 62 65 67 70
Figure 3. Distribution of QPM variable.
Figure 4 indicates the distribution of summated answers to the question about the
crisis leadership or Leadership variable. The data distribution is skewed because the
majority of respondents selected answers that are more favorable. The histogram
suggests that Leadership variable does not have a normal distribution.

73

Leadership
140
120
Frequency

100
80
60
40
20
More

44

43

42

41

40

39

38

37

36

35

34

33

32

31

30

29

28

27

Figure 4. Distribution of Leadership variable.

In addition to the graphical description of data, Shapiro-Wilk test was also used to
test the normality of the data. For Shapiro-Wilk test, the following hypotheses were
tested (Park, 2008):
H0: p value .05 (The data follow a normal distribution)
Ha: p value < .05 (The data do not follow a normal distribution)
The confidence interval of 95% and the significance level of = .05 were used for
the tests. If the significance level reported is greater than or equal to the alpha level of
.05 then the null hypothesis of data following a normal distribution is not rejected.
Table 5 presents the results Shapiro-Wilk test. Table 5 indicates that for ShapiroWilk test, the significance levels reported by Gretl for both variables are less than
accepted =.05. Therefore, the null hypothesis is rejected. The test results support the
alternative hypothesis that the data do not follow a normal distribution.

74
Table 6
Normality Tests
Shapiro-Wilk
Statistic

df

Significance

QPM

0.89

344

< .001

Leadership

0.85

344

< .001

Because the data distribution is not normal, non-parametric tests should be used
for testing correlation hypotheses. The reason is that using parametric tests when the
normality assumption is violated may produce unreliable results (Doane & Seward,
2009). Non-parametric tests do not assume the normality of data and can have more
power (Doane & Seward, 2009). Therefore, the hypotheses of the second research
question were tested using Spearmans rank correlation that is a non-parametric test of
correlation.
Hypothesis results for research question 2. Both descriptive and inferential
methods were used to analyze the numerical data obtained from the survey. Statistical
properties of the data in the form of response frequencies, means, percentages, standard
deviations, and ranges of responses were derived using descriptive methods. Descriptive
analysis of the data was supported by graphical descriptions of data in the form of tables,
charts, diagrams, and graphs.
Inferential analysis was used to answer research questions and test the hypotheses
based on the sample drawn from the population using the econometric software package
Gretl. Answers to the second and third parts of the questionnaire were scored from one
(strongly disagree) to five (strongly agree). The "strongly agree" answer choice

75
represented the most favorable option. Each respondents score was summed across the
set of questions to get a final score of the respondent for that set of questions.
The quality of the performance measurement system of the organization and the
capacity of the organization to forecast potential crises were the two variables used in
testing the first set of hypotheses of the study. The variable titled the quality of the
performance measurement system of the organization was estimated by summing the
scores of the answers of each respondent across the questions 2.1 to 2.14. The variable
titled the capacity of the organization to forecast potential crises was estimated by
summing the scores of the answers of each respondent across the questions 3.1 to 3.9.
The variable titled quality of the performance measurement system of the
organization was labeled as QPM. The variable titled the capacity of the organization to
forecast potential crises was labeled as Leadership. The correlation between variables
QPM and Leadership was measured by correlating the summated answers to the
questions 2.1 to 2.14 with the summated answers to the questions 3.1 to 3.14.
The hypotheses of the second research question were tested with a correlational
analysis between two variables: the quality of the performance measurement system of
the organization and the capacity of the organization to forecast potential crises. A twotailed hypotheses test for the correlation between the quality of the performance
measurement system (M = 59.6, SD = 7.1) and the capacity of the organization to
forecast crises (M = 39.4, SD = 4.3) using Spearmans rank correlation test produced a
correlation coefficient of .78 with p < .001. Based on the correlation test, the null
hypothesis of no significant relationship between variables was rejected. The data

76
suggest that a significant relationship exists between the quality of the performance
measurement system and the capacity of the organization to forecast crises.
Summary
Chapter 4 provided the results of the data analysis under two major headings. For
the first research question, which involved multiple regression analysis, the data
collection and analysis procedures, descriptive statistics, and multiple regression results
were presented. The individual coefficients of performance scores on the audit and tax
debt collection processes were statistically significant. The coefficients of independent
variables were also jointly significant. Therefore, the null hypothesis of no relationship
between variables was rejected at 5% level of significance.
For the second research question, the results of the correlation analysis were
presented. The significant relationship between the quality of the performance
measurement system and the capacity of the organization to forecast crises was detected.
Therefore, the null hypothesis of no significant correlation between variables was
rejected at 5% level of significance.
Chapter 5 presents a detailed discussion of the results. Chapter 5 also explores
the implications of the research for the field of leadership. Recommendations for key
stakeholders and recommendations for future research are also provided.

77
Chapter 5: Conclusions and Recommendations
Though some early warning signals precede crises, these signals are usually weak
and without a proper detection mechanism. Thus, the organizational leaders may not
detect these signals on time (Mitroff, 2004). The purpose of the current quantitative
descriptive survey research was to measure the relationship between the application of
the performance measurement system and the ability of the decision-makers of a
government revenue agency to detect potentially disruptive events. The study employed
a quantitative research method applying a multiple regression analysis and Spearmans
correlation coefficient for testing the studys hypotheses.
Chapter 5 provides an interpretation of the findings presented in Chapter 4 and
puts these findings into the same framework with other chapters of the dissertation.
Leadership and social implications of the study findings will form a significant part of
Chapter 5. As a final point, Chapter 5 produces recommendations for key stakeholders
and future research.
Research Questions and Hypotheses testing
The first research question explored if the performance results predict fulfillment
of tax revenue forecast. The second research question explored the relationship between
the quality of the performance measurement system of the organization and the capacity
of the organization to forecast crises. Multiple regression analysis and correlation
analysis methods were used to test the hypotheses of the research questions. For the first
research question, data were gathered from the Ministry of Taxes (MOT) and the online
database of the State Statistics Committee of the Republic of Azerbaijan. For the second

78
research question, data were obtained from the online survey conducted among the
managers of the MOT.
Research question 1. The first research question explored if the performance
measurement results predict fulfillment of tax revenue forecast. The results of the joint
hypotheses testing indicated that performance scores on five organizational processes of
the MOT are jointly significant. The data support the hypothesis that performance results
predict the fulfillment of tax revenue forecast.
However, the analysis of the relationship of individual variables with the
dependent variable produced mixed results. Two key findings were noticed. First,
individually, only performance scores on audit and tax debt collection processes were
statistically significant. Second, the tax revenues were found to increase with the
increase in the performance scores on the audit process whereas to decrease with the
increase in the performance scores on the tax debt collection process.
Research question 2. The second research question explored if there is a
significant relationship between the quality of the performance measurement system of
the organization and the capacity of the organization to forecast crises. The hypotheses
test pointed to a significant correlation between the quality of the performance
measurement system and the capacity to detect crises. The correlation coefficient was
positive and relatively high indicating a strong correlation. Therefore, findings on the
second research question support the literature asserting that the quality of the
performance measurement system is related to the capacity of the organization to detect
crises. However, the findings should be considered from the perspective that a strong
correlation coefficient does not mean causation (Doaen & Seward, 2009).

79
Interpretation of the results. The study results seem to address the specific
research problem of the study. The hypotheses test points to a relationship between the
performance measurement system and capacity of the organization to predict crises.
However, a more detailed analysis of the relationship between variables reveals that the
overall positive relationship may hide the underlying differences. For example, for the
first research question, some performance scores were not positively associated with the
tax revenue, and some had no statistically significant relationship. There may a need for
additional research to analyze the reasons of deviations from the observed common
pattern.
Limitations
This study was limited to subjects who agreed to participate voluntarily. This
study was also limited to the number of subjects surveyed and the amount of time
available to conduct the study. Another major limitation was the possible bias in the
answers of the survey participants because of possible fear of retaliation.
The fact that the study was limited to a state organization in a particular country
with a different development history limits the application of study results to other
organizations. The performance measurement systems used across government
organizations may not be the same as no standard performance measurement system
exists. Understating of what constitutes the organizational crisis may also be different for
different organizations based on the socio-political paradigm of the country in which they
operate.
An honest response of the managers in the survey was one of the major
assumptions of the study. The data collection procedure was constructed in such a way

80
that maximum anonymity and confidentiality were ensured and communicated to the
participants. However, achieving 100% honesty is not possible. The fact that 63
participants answered differently to two questions that were repeated word-by-word
because of a programmer's mistake indicates that some participants participated in the
survey just to avoid possible penalty by management.
The organizational crisis for this study was defined as non-fulfillment of tax
revenue forecast by the revenue organization that may undermine the capability of the
government to carry out planned budget policy. This approach to defining the
organizational crisis was based on the current understanding of the crisis within the target
organization of the study. Other stakeholders in the society may define the organizational
crisis differently for the revenue organization. If a different approach to defining the
crisis in a revenue organization were adopted, then quantification of the crisis variable
and implications would be different. Therefore, quantification of the crisis variable in a
particular way should be considered as one of the limitations of the study.
The relationships established by the study findings are by no means causal
relationships. Both regression coefficients and correlation do not imply causality (Doane
& Seward, 2009). This limitation should be considered when interpreting the results of
the study.
Implications
This study contributes to the literature on the importance of measuring
performance and having an early warning system for predicting the signals of an
organizational crisis. The literature review in Chapter 2 presented an overview of the
development of the performance measurement and signal detection theories in

81
organizational management and leadership contexts. Chapter 2 also stressed the need for
combining the two theories to understand how the performance measurement framework
of the organization might be used as an early warning system of the organizational crisis.
While the literature on performance measurement and signal detection is abundant,
literature on the joint application of these theories is not well developed. The results of
this study relative to the relationship between the proxy variables of the performance
measurement system of the organization and those of the organizational crisis suggest
some insights into developing a performance measurement-based early warning system in
a government revenue organization.
The findings on the first research question overall supported previous literature,
which suggested that a properly designed performance measurement system might
predict the organizational crisis (Andersen & Fagerhaug, 2002; Iskandarani &
Reifschneider, 2008; Parida & Chattopadhyay, 2007). The statistical insignificance of
some variables seems to reinforce the assertion of the literature that performance
measures should be appropriately designed and constructed to predict the organizational
crisis (Bhasin, 2008; Mitroff, 2004). However, lack of relationship or negative
relationship may also be observed if the relevant processes are not important in predicting
the changes in the tax revenue in reality. For example, the observation that cash
operations control, tax returns processing, and taxpayer services processes have no
statistically significant relationship with tax revenues may be the result of poor
organization of these processes within the tax organization, or longer time may be
required for the effects of these processes on tax revenues to be distinguished.

82
The second research question was explored by measuring correlation between the
quality of the performance measurement system of the organization and the capacity of
the organization to forecast crises. The data were collected using a questionnaire. The
questions of the questionnaire were developed based on the literature review of
performance measurement and crisis leadership theories. Therefore, the participants were
assessing the quality of the performance measurement and crisis leadership systems of
the MOT through survey questions, which were then tested for correlation.
According to the signal detection theory, the stronger the signal detection
mechanism, the better the crisis forecast (Abdi, 2010; Mitroff, 2004). The positive
correlation in the case of the second research question indicates that on average the
survey participant who rated the quality of the performance measurement system of the
MOT higher, rated capacity of the organization to forecast crises higher, as well. The
positive correlation between the variables of the second research question does not imply
causation, however, a positive significant relationship between the quality of the
performance measurement system of the organization and the capacity of the
organization to forecast crises seems to be in line with the literature review. The
implication is that a performance measurement system with comprehensive indicators
may be related to the increased capacity of the organization to forecast organizational
crises.
From the beginning, the study was intended to be significant to leadership
research by combining leadership theory and decision-making theory with signal
detection theory and by providing an empirical investigation about the usefulness of this
combination. The study contributed to the combination of theories stated above, though

83
the scope of the study did not allow the ability to research all possible overlaps of
leadership and decision-making theories with signal detection theory. The contribution
was in terms of using performance measurement as a signal detection tool for sensing
early warning signals to guide the organizational decisions.
The results of the study may contribute to government performance measurement
as part of ongoing efforts to modernize the activities of the government sector and to
make government agencies as effective as private organizations (Currie, Humphreys,
Ucbasaran, & McManus, 2008; Wallis & McLoughlin, 2007). The study may be
generalized to other government organizations working under similar conditions. The
study also may be useful for organizations working in different countries but having
similar goals of modernizing.
Despite the limited generalization, the study provided some gap filling in the
performance measurement research focused on crisis leadership. Crisis leadership
concerns many organizations across countries (Mitroff, 2004). Therefore, the current
study may be useful to the research focusing on the cross section of performance
measurement and crisis leadership.
Recommendations for Key Stakeholders
The study results suggest that, on average, the manager perceives a positive
relationship between the performance measurement system and the capacity of the
organization to predict crises. Therefore, the major recommendation for the organization
leaders would be to develop the performance measurement system further in their
organizations. However, developing performance measurement should not be a sole
purpose. The crisis leadership in the organization should also be developed.

84
The organizational leaders need to consider that performance measurement is not
the only tool for sensing the organizational crises. If appropriate channels of
communication are created with the key stakeholders and frontline employees, the early
warning signals of crises may be detected in time. Another recommendation for the
organizational leaders would be to develop the communication channels with the key
organizational stakeholders further and introduce new ways of communication.
The last recommendation for the organizational leaders would be to expand the
performance measurement system beyond the measurement of the core organizational
processes. The major step towards this expansion would be integrating the taxpayers'
satisfaction survey into the performance measurement system. This would represent the
external assessment of the organization and would strengthen the early warning signal
detection potential of the performance measurement system.
Recommendations for Future Research
The limitations of the study and questions that arose during the research process
influenced recommendations for future research. The first recommendation for future
research would be using the cross sectional data from different organizations for studying
the performance measurement based early warning systems. The problem that different
organizations use different performance measurement system may be softened by using
the data on organizations that are using one of the widely accepted performance
measurement systems, for instance, the balanced scorecard. Then market performance of
these organizations before and after the recent economic crises may be compared to
performance of other organizations that do not use such performance measurement
systems.

85
Another recommendation would be using a mixed methodology study for
analysis. The observed underlying differences in the perception of managers towards the
performance measurement and crisis leadership relationship may be further explored by
applying qualitative research methods. As a result, a study in this direction would help
better understand the relationship between performance measurement and crisis
leadership.
Future research may also use the tendency among OECD (Organization for
Economic Cooperation and Development) member countries to benchmark performance
of their revenue organizations. The benchmarking process may result in a common
performance measurement system applied across government revenue organizations of
OECD member countries. If this benchmarking process were successful, then future
researchers would have a common database of performance data for different
organizations for performing valuable comparisons.
Future research may also use different methods of defining the crisis for the
organization. The present study used underachievement of tax revenue forecast as an
indicator of a crisis for a revenue organization. For future research, an external
assessment of organization by taxpayers, or the content study of newspaper articles about
the organization may be used as an indicator of a crisis. This would represent a more
customer-oriented approach to assessing organizational performance, representing a crisis
that means that the organization cannot meet the needs of its customers.
The final recommendation for future research would be using a research design
that allows exploring the causality among variables. Both regression and correlation
designs used in the study do not imply any causality. Exploring cause-effect

86
relationships between performance measurement scores and tax revenues and using a
longitudinal design for implying causality may be interesting for key stakeholders and
leadership research.
Summary
The quantitative research study explored the relationship between the application
of the performance measurement system and the ability of the decision-makers of a
government revenue agency to detect potentially disruptive events using correlations and
survey designs. Signal detection theory provided the broad theoretical framework for the
study and suggested that crisis leadership may be improved if the useful information
contained in the signals preceding the crises is extracted using different early warning
signal detection tools. Previous research suggested that an appropriately designed
performance measurement system might serve as an early warning signal detection tool
for organizational managers.
The first research question explored the relationship between performance scores
and tax revenues. The hypotheses test indicated that the performance scores jointly
predict the fulfillment of tax revenue forecast. The second research question explored the
relationship between the quality of the performance measurement system of the
organization and the capacity of the organization to forecast crises. The hypotheses test
indicated a strong positive correlation between variables.
Based on the study results the general recommendation for the organizational
leaders was to continue improving the performance measurement based early warning
systems by increasing the scope of the system and including performance indicators on
different areas of organizational performance. Another important recommendation was

87
to include managers of different levels in devising the performance measures so that the
performance measurement system can effectively detect the potential disruptive events.
Recommendations for future research were also suggested in exploring other avenues of
crisis leadership and performance measurement relationship by increasing the scope of
the research and including more stakeholders in the analyses.
Together, the results of both research questions added to the body of knowledge
on performance measurement and crisis leadership by providing an empirical study of the
joint application of the two theories. The research gap review indicated the lack of
literature on using the performance measurement system as a crisis leadership tool to
forecast organizational crises. The study indicated the need for improving performance
measures to increase the capacity of the organization to forecast organizational crises.
The study also showed that if performance measures are constructed inappropriately they
might be not relevant for crisis leadership purposes. The study reinforced the literature
assertion that comprehensive performance measures might be better predictors of
organizational performance overall.

88
References
Abdi, H. (2010). Signal detection theory. In B. McGaw, P. Peterson, and E. Baker (Eds.),
International encyclopedia of education (3rd ed.). (Vol. 7, pp. 407-410). New
York, NY: Elsevier.
Acquier, A., Gand, S., & Szpirglas, M. (2008). From stakeholder to stakesholder
management in crisis episodes: A case study in a public transportation company.
Journal of Contingencies and Crisis Management, 16(2), 101-114.
doi:10.1111/j.1468-5973.2008.00538.x
Aliev, T. (2007). Digital noise monitoring of defect origin. London, England: SpringerVerlag.
Alpaslan, C., Green, S., & Mitroff, I. (2009). Corporate governance in the context of
crises: towards a stakeholder theory of crisis management. Journal of
Contingencies and Crisis Management, 17(1), 38-49. doi:10.1111/j.14685973.2009.00555.x
Ammons, D., & Rivenbark, W.C. (2008). Factors influencing the use of performance data
to improve municipal services: Evidence from the North Carolina benchmarking
project. Public Administration Review, 68(2), 304318.
Andersen, B. & Fagerhaug, T. (2002). Performance measurement explained:
Implementing your state-of-the-art system. Milwaukee, WI: ASQC Quality Press.
Andrew, D., Pedersen, P.M., & McEvoy, C.D. (2011). Research methods and design in
sport management. Champaign, IL: Human Kinetics.

89
Antic, L. & Sekulic, V. (2006). New paradigm of business performance measurement in
contemporary business conditions. Economics and Organization, 3(1), 6977.
Retrieved from http://facta.junis.ni.ac.yu/eao/eao2006/eao2006-07.pdf
Beamon, B.M. & Balcik, B. (2008). Performance measurement in humanitarian relief
chains. International Journal of Public Sector Management, 21(1), 4-25.
doi:10.1108/09513550810846087
Benjamin, A. S., Diaz, M., & Wee, S. (2009). Signal detection with criterion noise:
Applications to recognition memory. Psychological Review, 116(1), 84-115.
doi:10.1037/a0014351
Bhagwat, R. & Sharma, M.K. (2007). Performance measurement of supply chain
management: A balanced scorecard approach. Computers & Industrial
Engineering, 53, 43-62. doi:10.1016/j.cie.2007.04.001
Bhasin, S. (2008). Lean and performance measurement. Journal of Manufacturing
Technology Management, 19(5), 670-684. doi: 10.1108/17410380810877311
Boin, A. & Rhinard, M. (2008). Managing transboundary crises: What role for the
European Union? International Studies Review,10, 126.
Braden V., Cooper J., Klingele, M., Powell, J., & Robbins, M. (2005). Crisis - A
leadership opportunity. John F. Kennedy School of Government, Harvard
University. Retrieved from
https://research.maxwell.af.mil/papers/ay2005/affellows/Powell.pdf
Bromberg, D. (2009). Performance measurement: A system with a purpose or a
purposeless system? Public Performance & Management Review, 33(2), 214-221.
doi: 10.2753/PMR1530-9576330202

90
Burgess, T.F., Ong, T.S., & Shaw, N.E. (2007). Traditional or contemporary? The
prevalence of performance measurement system types. International Journal of
Productivity and Performance Management, 56(7), 583602.
doi:10.1108/17410400710823633
Chin, J. L. & Sanchez-Hucles, J. (2007). Diversity and leadership. American
Psychologist, 62(6), 608-609. doi:10.1037/0003-066X62.6.608
Cook, M., Noyes, J., & Masakowski, Y. (Eds.). (2007). Decision making in complex
environments. Hampshire, England: Ashgate Publishing Company.
Coombs, W. T. (2007). Protecting organization reputations during a crisis: The
development and application of situational crisis communication theory.
Corporate Reputation Review, 10(3), 163-177.
Coombs, W.T. (2010). Parameters for crisis communication. In W.T. Coombs & S.J.
Holladay (Eds.), The handbook of crisis communication (pp. 17-53). Malden,
MA: Blackwell Publishing.
Creative Research Systems. (2010). Sample size calculator. Retrieved from
http://www.surveysystem.com/sscalc.htm
Creswell, J. W. (2005). Educational research: Planning, conducting, and evaluating
quantitative and qualitative research. Upper Saddle River, NJ: Prentice Hall.
Currie, G., Humphreys, M., Ucbasaran, D., & McManus, S. (2008). Entrepreneurial
leadership in the English public sector: paradox or possibility? Public
Administration, 86(4), 987-1008. doi:10.1111/j.1467-9299.2008.00736.x
Dant, R. P., Kacker, M., Coughlan, A. T., & Emerson, J. (2007). A cointegration analysis
of the correlates of performance in franchised channels. In G. Cliquet, M.

91
Tuunanen, G. Hendrikse, & J. Windsperger (Eds.), Economics and Management
of Networks (pp. 169-189). Heidelberg: Physica-Verlag HD. doi: 10.1007/978-37908-1758-4_9
Davila, T. & Wouters, M. (2007). Management accounting in the manufacturing sector:
Managing costs at the design and production stages. In C.S. Chapman, A.G.
Hopwood, & M.D. Shields (Eds.), Handbook of management accounting
research: Vol 2 (Ch. 15, pp. 831-858). Oxford, England: Elsevier.
DBM Consultants. (2009). Community perceptions survey: Prepared for Australian
Taxation Office. Retrieved from
http://www.ato.gov.au/download.asp?file=/content/downloads/cor00227095cps20
09.pdf
De Vaus, D. (2002). Surveys in social research (5th ed.).London, England: Routledge.
Deutschlander, C. (2009). Surveying fundamentals for business analysts. Vienna, VA:
Management Concepts.
Deverell, E. & Olsson, E. (2010). Organizational culture effects on strategy and
adaptability in crisis management. Risk Management 12, 116134.
doi:10.1057/rm.2009.18
Doane, D. & Seward, L. (2009). Applied statistics in business and economics (2nd ed).
New York, NY: McGraw Hill/Irwin.
Dubnick, M.J. & Frederickson, H.G. (2010). Accountable agents: Federal performance
measurement and third-party government. Journal of Public Administration
Research and Theory, 20(suppl 1), i143-i159. doi:10.1093/jopart/mup039

92
Elg, M., & Kollberg, B. (2009). Alternative arguments and directions for studying
performance measurement. Total Quality Management & Business Excellence,
20, 409-421. doi:10.1080/14783360902781899
Elliott, D. (2009). The failure of organizational learning from crisis a matter of life and
death?Journal of Contingencies & Crisis Management, 17(3), 157-168.
doi:10.1111/j.1468-5973.2009.00576.x
Fink, A., Marr, B., Siebe, A., & Kuhle, J. (2005). The future scorecard: combining
external and internal scenarios to create strategic foresight. Management
Decision, 43(3), 360-381. doi:10.1108/00251740510589751
Fink, A., Siebe, A. & Kuhle, J. (2004). How scenarios support strategic early warning
processes. Foresight: The Journal of Futures Studies, Strategic Thinking and
Policy, 6(3), 173-185. doi:10.1108/14636680410548392
Franco-Santos, M., Kennerley, M., Micheli, P., Martinez, V., Mason, S., Marr, B., Gray,
D., & Neely, A. (2007). Towards a definition of a business performance
measurement system. International Journal of Operations & Production
Management, 27(8), 784-801. doi:10.1108/01443570710763778
Fried, A. & Orellana, F. (2006). Lighting up blind spots while measuring knowledge
capital. The Electronic Journal of Knowledge Management, 4(1), 31-38.
Retrieved from http://www.ejkm.com/issue/download.html?idArticle=64
Garcia, H. (2006). Effective leadership response to crisis. Strategy and Leadership, 34(1),
4-10. doi:10.1108/10878570610637849
Gliem, J. & Gliem, R. (2003, October). Calculating, interpreting, and reporting
Cronbachs alpha reliability coefficient for Likert-type scales. Paper presented at

93
the Midwest Research-to-Practice Conference in Adult, Continuing, and
Community Education, The Ohio State University, Columbus, OH. Retrieved
from
https://scholarworks.iupui.edu/bitstream/handle/1805/344/Gliem+&+Gliem.pdf?s
equence=1
Goldstein, E. B. (2009). Sensation and perception (8th ed.). Belmont, CA: Cengage
Learning.
Groves, R., Fowler, F., Couper, M., Lepkowski, J., Singer, E., & Tourangeau, R. (2009).
Survey methodology (2nd ed.). New Jersey, NJ: Jon Wiley & Sons.
Gulf oil spill: BP says 'top kill' plug going to plan. (2010, May 27). BBC News.
Retrieved from http://www.bbc.co.uk/news/10167666
Halachmi, A. & Holzer, M. (2010). Citizen participation and performance measurement:
operationalizing democracy through better accountability. Public Administration
Quarterly, 34(3), 378-399.
Harbour, J.L. (2009). The basics of performance measurement (2nd ed.). New York, NY:
Taylor & Francis Group.
Hawabhay, B., Abratt, R., & Peters, M. (2009). The role of corporate communications in
developing a corporate brand image and reputation in Mauritius. Corporate
Reputation Review, 12, 3-20. doi:10.1057/crr.2009.4
Hunter, M., Le Menestrel, M., & de Bettignies, H. (2008). Beyond control: Crisis
strategies and stakeholder media in the Danone boycott of 2001. Corporate
Reputation Review, 11(4), 335-350. doi:10.1057/crr.2008.30

94
Iskandarani, M. & Reifschneider, F. (2008). Performance measurement in a global
program: motivation, new concepts and early lessons from a new system. Science
and Public Policy, 35(10), 745-755. doi: 10.3152/030234208X398495
James, E. & Wooten, L. (2010). Leading under pressure: From surviving to thriving
before, during and after a crisis. New York, NY: Routlege Press.
James, K. (2010). Incorporating complexity science theory into nursing
curricula.Creative Nursing, 16(3), 137-142. doi: 10.1891/1078-4535.16.3.137
Jaques, T. (2007). Issue management and crisis management: an integrated, non-linear,
relational construct. Public Relations Review, 33(2), 147-157.
doi:10.1016/j.pubrev.2007.02.001
Jaques, T. (2008). A case study approach to issue and crisis management: Schadenfreude
or an opportunity to learn? Journal of Communication Management, 12(3), 192203. doi:10.1108/13632540810899399
Kaplan, R., & Norton, D. (2007). Using the balanced scorecard as a strategic
management system. Harvard Business Review, 85(7/8), 150-161.
Kaplan, R.S. & Norton, D.P. (1992, January-February). The balanced scorecard:
Measures that drive performance. Harvard Business Review, 70, 71-79.
PMid:10119714
Kappelman, L.A., McKeeman, R., & Zhang, L. (2006). Early warning signs of IT project
failure: the dominant doze. Information Systems Management, 23(4), 31-36.
doi:10.1201/1078.10580530/46352.23.4.20060901/95110.4
Kirchgssner, G. & Wolters, J. (2008). Introduction to modern time series analysis.
Berlin, Germany: Springer.

95
Kneiding, C. & Tracey, P. (2009). Towards a performance measurement framework for
community development finance institutions in the UK. Journal of Business
Ethics, 86, 327-345. doi: 10.1007/s10551-008-9850-9
Lagadec, P. (2006). Crisis management in the twenty-first century Unthinkable
events in unthinkable contexts, in H. Rodriguez, E. L. Quarantelli, & R. R.
Dynes (Eds.). Handbook of Disaster Research (pp. 489-507), New York, NY:
Springer.
Laming, D. (2010). Statistical information and uncertainty: A critique of applications in
experimental psychology. Entropy, 12(4), 720-771. doi:10.3390/e12040720
Lerman, D. C., Tetreault, A., Hovanetz, A., Bellaci, E., Miller, J., Karp, H., & ...
Toupard, A. (2010). Applying signal-detection theory to the study of observer
accuracy and bias in behavioral assessment. Journal of Applied Behavior
Analysis, 43(2), 195-213. doi: 10.1901/jaba.2010.43-195
Levy, B. (2008). Principles of signal detection and parameter estimation. New York,
NY: Springer.
Lowe, M., Aparicio, A., Galbraith, R., Dorman, T., & Dellert, E. (2009). The future of
continuing medical education: Effectiveness of continuing medical education:
American college of chest physicians evidence-based educational guidelines.
CHEST, 135(3), 69s-75s.
Marc, M, Peljhan, D., Ponikvar, N., Sobota, A., & Tekavcic, M. (2010). Performance
measurement in large Slovenian companies: An assessment of progress.
International Journal of Management and Information Systems, 14(5), 129-139.

96
March, J. & Sutton, R. (1997). Organizational performance as a dependent variable.
Organization Science, 8, 698-706. doi:10.1287/orsc.8.6.698
Maxfield, M. & Babbie, E. (2009). Basics of research methods for criminal justice and
criminology (2nd ed.). Belmont, CA: Wadsworth.
McBurney, D. H., & White, T. L. (2009). Research methods (8th ed.). Belmont, CA:
Wadsworth.
Ministry of Taxes. (2006). Report on 6-year operation. Baku, Azerbaijan: The Ministry
of Taxes of the Republic of Azerbaijan.
Mitroff, I. (2004). Crisis leadership: Planning for the unthinkable. New York, NY: John
Wiley &Sons.
Moynihan, D. (2008). Learning under uncertainty: Networks in crisis management.
Public Administration Review 68(2), 350-361.
Neuman, W. L. (2003). Social research methods (5th ed.). Upper Saddle River, NJ:
Prentice Hall.
OECD (2006). Tax administration in OECD and selected non-OECD countries:
Comparative information series (2006). Retrieved from
http://www.oecd.org/dataoecd/43/7/37610131.pdf
Paranjape, B., Rossiter, M., & Pantano, V. (2006). Performance measurement systems:
successes, failures and future a review. Measuring Business Excellence, 10(3),
4-14. doi:10.1108/13683040610685748
Paraskevas, A. (2006). Crisis management or crisis response system? A complexity
science approach to organizational crises. Management Decision, 44(7), 892-907.
doi:10.1108/00251740610680587

97
Parida, A. & Chattopadhyay, G. (2007). Development of a multi-criteria hierarchical
framework for maintenance performance measurement (MPM). Journal of
Quality in Maintenance Engineering, 13(3), 241-258.
doi:10.1108/13552510710780276
Park, H.M. (2008). Univariate analysis and normality test using SAS, Stata, and SPSS.
Working Paper. The University Information Technology Services (UITS) Center
forStatistical and Mathematical Computing, Indiana University. Retrieved from
http://www.indiana.edu/~statmath/stat/all/normality/index.html
Peterson, T. & van Fleet, D. (2008). A tale of two situations: An empirical study of
behavior by not-for-profit managerial leaders. Public Performance &
Management Review, 31(4), 503-516. doi: 10.2753/PMR1530-9576310401
Plaza-Ubeda, J., Burgos-Jimenez, J., & Carmona-Moreno, E. (2010). Measuring
stakeholder integration: Knowledge, interaction and adaptational behavior
dimensions. Journal of Business Ethics, 93(3), 419-442. doi:10.1007/s10551-0090231-9
Radnor, Z. & Barnes, D. (2007). Historical analysis of performance measurement and
management in operations management, International Journal of Productivity
and Performance Management, 56(5/6), 384-396.
Ratnayake, R.M.C. (2009). Evolution of scientific management towards performance
measurement and managing systems for sustainable performance in industrial
assets: Philosophical point of view. Journal of Technology Management &
Innovation, 4(1), 152-161.

98
Rouse, R. A., & Schuttler, R. S. (2009). Crisis communication: A mixed method study of
supervisor leadership and employee performance during the 2008-2009 financial
crisis. University of Phoenix National Research Center study. Retrieved from
http://www.nrc-slci.com/Results.pdf
Scharmer, C. O. (2009). Theory U: Leading from the future as it emerges. San Francisco,
CA: Berrett-Koehler.
Sjoberg, D. I. K., Dyba, T., & Jorgensen, M. (2007). The future of empirical methods in
software engineering research. Proceedings of Future of Software Engineering
2007 (pp. 358-378). Los Alamitos, CA: IEEE Computer Society.
doi:10.1109/FOSE.2007.30
Soubbotina, T. (2004). Beyond economic growth: An introduction to sustainable
development (2nd ed.). Washington, DC: World Bank.
Spitzer, D. (2007). Transforming performance measurement: Rethinking the way we
measure and drive organizational success. New York, NY: AMACOM.
Supply Chain Consultants Introduces Zemeter Performance Management; Out-of-the-box
software provides daily view of supply chain through the use of Early Warning
Indicators and in-depth reporting. (8 May, 2006). PR Newswire. Retrieved from
Business Dateline. (Document ID: 1050209331)
Taylor, F. (1911). The principles of scientific management. Retrieved from
http://www.eldritchpress.org/fwt/ti.html
Taylor, J. (2009). Strengthening the link between performancemeasurement and decision
making.Public Administration, 87(4), 853-871. doi: 10.1111/j.14679299.2009.01788.x

99
U.S. Government Accountability Office. (2009a). 2009 tax filing season: IRS met many
2009 goals, but telephone access remained low, and taxpayer service and
enforcement could be improved (GAO Publication No. GAO-10-225). Retrieved
from http://www.gao.gov/new.items/d10225.pdf
U.S. Government Accountability Office. (2009b). Tax gap: Actions needed to address
noncompliance with S corporation tax rules (GAO Publication No. GAO-10195). Retrieved from http://www.gao.gov/new.items/d10195.pdf
U.S. Government Accountability Office. (2010). Tax gap: Actions needed to address
noncompliance with S corporation tax rules (GAO Publication No. GAO-10355). Retrieved from http://www.gao.gov/new.items/d10355.pdf
Van Aken. E., Letens, G; Coleman, G., Farris, J., & Goubergen, D. (2005). Assessing
maturity and effectiveness of enterprise performance measurement systems.
International Journal of Productivity and Performance Management, 54(5/6),
400-418. doi:10.1108/17410400510604557
Wallis, J. & McLoughlin, L. (2007). A diagnosis of leadership effectiveness inthe Irish
publicsector. Public Management Review, 9(3), 327-351.
doi:10.1080/14719030701425670
Waxman, H. & Barton, J. (2010). Memorandum on BPs internal investigation. House of
Representative Committee on Energy and Commerce. Retrieved from
http://energycommerce.house.gov/documents/20100525/Memo.BP.Internal.Invest
igation.pdf
Weber Shandwick (2008, October 13). Company leaders not communicating with
employees on financial crisis, according to new national research from Weber

100
Shandwick. Retrieved from http://www.webershandwick.com/Default.aspx/
AboutUs/PressReleases/2008/CompanyLeadersNotCommunicatingWithEmploye
esOnFinancialCrisisAccordingToNewNationalResearchFromWeberShandwick
Woodward, I. & More, E. (2010). Leadership communication at the edge. Australian
Journal of Communication, 37(1), 111-136.
Wooldridge, J. (2009). Introductory econometrics: A modern approach (4th ed.). Mason,
OH: South-Western Sengage Learning.
Wooten, L.P. & James, E.H. (2008). Linking crisis management and leadership
competencies: The role of human resource development. Advances in Developing
Human Resources 10(3), 352-379. doi:10.1177/1523422308316450
Yang, K. & Hsieh, J. (2007). Managerial effectiveness of government performance
measurement: Testing a middle-range model. Public Administration Review,
67(5), 861-879.

101
Appendix A: Authorization Letter

102

103
Appendix B: Informed Consent Form

104
Dear Participant,
I am a University of Phoenix student, and I will use this survey for completing my
dissertation entitled A Performance Measurement-based Early Warning System in a
Government Revenue Agency. The purpose of the study is to examine the relationship
between the performance measurement system and the ability of the organization to
detect early warning signals of the crises that may hurt operations of your organization.
You are asked to complete a survey that will take 20-30 minutes. The survey
contains questions about your perceptions of the performance measurement system of the
Ministry of Taxes. Other questions focus on your perceptions of the recent crises or
potential crises that, as a manager, you experienced at the Ministry of Taxes. You will
also be asked some demographic information as well.
There are no direct benefits to you from this study. However, you may contribute
to the research in the performance measurement field that, in turn, may be used to
improve the performance measurement system of your organization. The results of my
study will be published in the official journal of the Ministry of Taxes.
No risks or discomforts are anticipated from taking part in this study. If you feel
uncomfortable with a question, you can skip that question or withdraw from the study
altogether. If you decide to quit at any time before you have finished the questionnaire,
your answers will NOT be recorded.
Your responses will be kept completely confidential. No personal identification
data are required. Individual survey responses will not be identifiable as well. Your
answers to the questionnaire will be summarized automatically.

105
Your participation is voluntary; you are free to withdraw your participation from
this study at any time. If you do not want to continue, you can leave this website by
clicking on the I do not want to participate button at the bottom of the page. If you do
not click on the "submit" button at the end of the survey, your answers and participation
will not be recorded. You also may choose to skip any question(s) that you do not wish
to answer.
The results of the study will be used for scholarly purposes only. The results from
the study will be presented in educational settings, and the results might be published in a
professional journal.
If you have concerns or questions about this study, please contact Elnur Asgarli at
________@email.phoenix.edu or by phone +99450______________.
As a participant in this study, you should understand the following:
1. You may decline to participate or withdraw from participation at any time without
consequences.
2. Your identity will be kept confidential.
3. Elnur Asgarli, the researcher, has thoroughly explained the parameters of the research
study, and all of your questions and concerns have been addressed.
4. If the interviews are recorded, you must grant permission for the researcher, Elnur
Asgarli, to record digitally the interview. You understand that the information from
the recorded interviews may be transcribed. The researcher will structure a coding
process to assure that anonymity of your name is protected.
5. Data will be stored in a secure and locked area. The data will be held for a period of
3 years, and then destroyed.

106
6. The research results will be used for publication.
By clicking on the I understand and want to participate button below you
acknowledge that you understand the nature of the study, the potential risks to you as a
participant, and the means by which your identity will be kept confidential. By clicking
on I understand and want to participate button below this form you also indicate that
you are 18 years old or older and that you choose voluntarily to serve as a participant in
the study described.

I UNDERSTAND AND WANT TO

PARTICIPATE

I DO NOT WANT TO
PARTICIPATE

107
Appendix C: Research Instrument

108
This questionnaire will be used for collecting the data for the dissertation
research. The purpose of the research is to measure the relationship between the
application of the performance measurement system and the ability of the decisionmakers of the Ministry of Taxes (MOT) to detect potential disruptive events. You will be
asked three groups of questions. First, you will answer several general questions that will
be helpful in analyzing the effect of demographic data on the study results. Second, you
will be asked questions about the performance measurement system of the MOT that you
use regularly as managers. Third, you will be asked questions about the crisis leadership
situation within the MOT.
The questionnaire will contain several terms that may require explanation to avoid
confusion. The terms that may seem confusing will be explained here. However, if you
still think that certain terms are not stated clearly, you may contact me at
_______@email.phoenix.edu or (050)___________ for further clarification.
The event or situation that may negatively affect the operations of the Ministry of
Taxes: For the purposes of this study, the event or situation that may negatively affect the
operations of the Ministry of Taxes is the event or situation that may results in the
decrease of state budget revenues to be collected by the Ministry of Taxes. This type of
event may result in non-achievement of state budget forecast and weakening of one of the
sources of the government budget policy. Therefore, the Ministry of Taxes should always
assess and control the risk of this type of events. Development of technological potential
and human resources serves this purpose.
You may have experienced several events or situations that may negatively affect
the operations of the Ministry of Taxes during your employment at the MOT. Please,

109
answer questions based on your most recent experience of a disruptive event that took
place at the MOT.
Early warning signals: Early warning signals are any events or indications that
predict the possible crisis (Kappelman, McKeeman, & Zhang, 2006).
Performance indicators: Performance indicators refer to the criteria for assessing
the performance of the organizational units of the MOT in terms of specific tax processes
(tax return processing, tax audit, cash operations control, enforced tax debt collection,
taxpayer services).
Quantitative and qualitative performance measures: Quantitative performance
measures are calculated using the quantitative data entered by employees. Quantitative
performance measures at the MOT are expressed in percentages, figures, and in other
numeric formats. Qualitative performance measures are not expressed in numbers but are
comprised of descriptive information that can be gathered by the decision-maker through
personal interactions. For example, the management of the Ministry may send a letter to
local tax units to learn their opinions on a problematic situation and find out how they
assess the situation.

110
1. General questions
1.1.Please indicate your age range:
20-29
30-39
40-49
50+
1.2.Please indicate your gender:
Male
Female
1.3.Please indicate your years of work experience as a tax employee:
0-3
3-5
5-7
7-9
9+
1.4.Please indicate your years of experience of working with the performance
measurement system of the Ministry of Taxes as a manager:
0-3
3-5
5+
1.5.Please indicate your education level:
Bachelors degree or 5-year diploma
Masters degree

111
Doctorate degree
1.6.Please indicate your department:
Central Office of the Ministry of Taxes
Audit Department
Baku City Tax Department
Tax Crimes Department
Special Regime Tax Service Department
Regional Tax Department
2. Performance measurement system
2.1.Performance indicators used by the MOT reflect the underlying objectives of the
MOT.
Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree
2.2.Performance measurement information obtained from the performance
measurement system of the MOT is relevant to my work.
Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree

112
2.3.The performance measures used by the MOT cover all the necessary processes
(audit, cash control, taxpayer service, and others) going on in the MOT.
Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree
2.4.The Ministry of Taxes matches the internal assessment of its performance with
the externals assessment of this performance by the taxpayers by improving the
process of reviewing the complains of the taxpayers about the tax units.
Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree
2.5.Sufficient training is provided on performance measurement methodology.
Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree
2.6.The performance measurement guidelines provided by the Ministry of Taxes
helps to thoroughly understand the performance measurement system.

113
Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree
2.7.The performance measures used by the MOT are annually updated to ensure that
the performance measures are improved and are adjusted to the new work
processes:
Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree
2.8.The MOT balances the use of quantitative and qualitative performance measures
by considering alongside with quantitative information produced from the
performance measurement system, the qualitative information in the form of
suggestions and complaints of the taxpayers, suggestions of the local tax units and
other qualitative information on performance.
Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree

114
2.9.The performance measures used by the MOT are always initiated by the regional
departments who implement the job and are not imposed top-down from the
Central Office.
Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree
2.10. The performance measurement system used by the MOT allows for aligning
organizational goals with personal goals of employees.
Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree
2.11. The performance measures used by the MOT allow for identifying problematic
performance areas instead of just showing areas where the organization performs
well.
Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree

115
2.12. The MOT encourages managers who reveal problematic performance areas and
tries to improve those areas instead of concealing negative performance measures.
Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree
2.13. Instead of looking for someone to blame when the unsatisfactory performance
measure is reported, the MOT tries to understand the underlying process and
analyzes the appropriateness of the measure for the given process.
Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree
2.14. The employees whose performance is measured can complain about their
performance measurement results to the higher management without any barrier.
Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree

116
3. Crisis leadership during the event or situation that may negatively affect
the operations of the Ministry of Taxes
3.1.The MOT had a mechanism (a possibility of direct, online or video-conference
contact of taxpayers with the management, considering opinions of taxpayers in
tax legislation development, ant others) for sensing the approaching disruptive
events, giving enough time for crisis preparation.
Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree
3.2.The MOT was ready for unplanned disruptive events, and the damage of the
disruptive events for the organization was minimal.
Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree
3.3.When the employees in the lower level of the organizational hierarchy of the
MOT sensed the approaching disruptive events, the communication structure in
the MOT (for example through the performance measurement system) allowed
transferring the necessary information to the management of the organization and
this structure was effective.

117
Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree
3.4.When disruptive events happened, the top management of the MOT
communicated the situation clearly to the managers of all organizational units,
collects their opinion on the issue, and based on those opinions ensures relevant
changes to the legislation and internal regulations.
Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree
3.5.The MOT periodically contacts employees on ongoing processes in the Ministry
and provides communication on potential negative events that may happen in its
activities (for example, through weekly information hours with employees,
regular management communications and other ways).
Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree

118
3.6.The MOT systematically analyzed its processes and performance to identify the
critical situations in its activities and in which area the disruptive event may
happen (for example, regular internal audits, regular meetings with taxpayers and
frontline tax units, and through other ways).
Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree
3.7.During disruptive events, the management of the MOT provided a clear direction
to the staff dealing with the core operations of the MOT for resolving the situation
(for example, establishment of inter-departmental working groups, mobilization
of all departments whose participation is required to solve the problem and other
means).
Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree
3.8.During disruptive events, the management of the MOT, when required, changed
the routine procedures and guidelines to prevent and minimize the potential
damage for the employees and taxpayers most negatively affected by the
disruptive events.

119
Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree
3.9.After each disruptive event, the management of the MOT studies the event and
makes relevant improvements based on the suggestions of the taxpayers and tax
employees.
Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree

120
Appendix D: Sample Performance Measurement Report

121
Sample report on performance scores on the specific processes by departments of the
Ministry of Taxes

3.0

4.5

4.6

3.3
5.0

3.5
4.5

Specific tax processes

Cash
Tax
contr
debts
ol
3.7
3.4
4.0
4.5
4.3
4.3
4.2
4.0
4.6

4.0
4.4
4.0
4.0
4.6
4.6
4.6
4.0
4.6
4.6
4.6
4.6

4.0
4.0
5.0
4.0
4.0
5.0
4.0
4.0
5.0
3.3
4.0
4.0

4.5
3.7
3.3
3.7
3.7
3.5
4.0
4.0
4.0
4.0
4.0
4.0

4.4
4.0
4.5
3.4
3.7
4.6
4.3
4.4
4.2
4.3
3.9
3.6

Departments of Tax return

the Ministry
processing
Department 1
Department 2
Department 3
Department 4

4.0
4.0
4.0

Department
Department
Department
Department
Department
Department
Department
Department
Department
Department
Department
Department

4.0
4.0
4.0
4.0
4.0
4.0
4.0
4.0
4.0
4.0
4.0
4.0

5
6
7
8
9
10
11
12
13
14
15
16

4.8
3.7

Audit

4.3
4.4
3.6
2.1
3.3
3.4
3.3
3.8
4.1
4.0
3.5
4.3

4.6
4.6
4.4
4.6
4.6
4.6
4.6
4.1
4.6
3.7
4.4
4.6

Taxpayer
services
3.6 5.0 5.0
4.8 4.3 4.0
4.8 4.5 5.0
4.8
4.8
4.8
4.0
4.8
4.7
4.8
4.8
4.8
4.8
4.6
4.8

4.0
3.5
4.3
4.0
4.0
3.5
5.0
5.0
4.3
5.0
4.0
4.2

4.7
5.0
5.0
5.0
5.0
4.3
4.8
4.7
4.8
4.2
3.3
5.0

Avera
ge
score
4.1
4.2
3.9
4.5
4.3
4.2
4.3
3.9
4.2
4.2
4.3
4.3
4.4
4.2
4.0
4.3