Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
research-article2016
Article
Impact of Performance
Appraisal Justice on the
Effectiveness of
Pay-for-Performance Systems
After Civil Service Reform
Jungin Kim1
Abstract
The present study comprehensively examined how human resources (HR)
directors in six U.S. state governments with drastic levels of civil service reform
assess their states pay-for-performance effectiveness in terms of the performance
appraisal justice. It was found that the effectiveness of pay-for-performance is
influenced by perceived fairness of performance appraisal. More specifically, this
study found that perceived politicization of performance appraisal was negatively
and significantly associated with pay-for-performance effectiveness. This study also
found that perceived fairness of appraisal criteria was positively and significantly
associated with pay-for-performance effectiveness. In addition, this study shows
that the effectiveness of pay-for-performance may be affected by some demographic
characteristics (e.g., age and gender) and state differences. In particular, drastic
levels of civil service reform significantly and positively influenced the effectiveness
of pay-for-performance, but insignificantly affected performance appraisal fairness.
These results provide scholars and practitioners in the public sector with meaningful
strategies for civil service reform.
Keywords
HR directors, pay-for-performance effectiveness, performance appraisal justice
1The
Corresponding Author:
Jungin Kim, University of Suwon, 17, Wauan-gil, Bongdam-eup, Hwaseong-si, Gyeonggi-do, South Korea.
Email: jungink@suwon.ac.kr
Kim
149
Introduction
According to Folger and Konovsky (1989), perceived fairness, including distributive
justice and procedural justice, may strongly influence employees reactions to pay
increases. With the expansion of civil service reform, especially the adoption and
implementation of pay-for-performance, some researchers, such as McFarlin and
Sweeney (1992), have paid attention to the relationship between employees perceptions of fairness and their pay-for-performance. For instance, McFarlin and Sweeney
(1992) found that distributive and procedural justice might influence pay satisfaction
at the personal level and organizational commitment at the organizational level.
Similarly, Rubin and Kellough (2012) found that employees perceptions of procedural fairness are likely to be positively associated with a decrease in employee complaints regarding personnel systems.
Research Question 1: Do people assess pay-for-performance as more effective in
the process of civil service reform when they think that pay-for-performance, especially performance appraisal, has been implemented fairly?
Research Question 2: How do human resources (HR) directors, who played a
major role in the civil service reform process, assess the impact of fair performance
appraisal procedure on the effectiveness of pay-for-performance?
With regard to the research questions, the present study has three main purposes.
First, the study analyzes previous studies on performance appraisal justice, focusing
especially on the relationship between performance appraisal justice and the effectiveness of pay-for-performance in the public sector. Second, looking at U.S. state governments, it examines how HR directors who have experienced significant civil service
reform and implemented pay-for-performance assess their states performance appraisals in terms of performance appraisal justice. Finally, the study examines the extent to
which performance appraisal justice and the effectiveness of pay-for-performance are
influenced by the degree of civil service reform.
Through the theoretical and practical examination of HR directors assessment of
pay-for-performance in U.S. state governments, the present study aims to contribute to
the improvement of pay-for-performance systems in the public sector. As Meng and
Wu (2015) found that the employees perceived fairness of pay-for-performance could
significantly influence personal or organizational outcomes such as job engagement,
this study provides evidence for why scholars or practitioners should make efforts to
establish or find ways to improve the effectiveness of pay-for-performance in terms of
performance appraisal fairness in the process of civil service reform. More specifically, this study provides scholars and practitioners in the public sector with information on how state governments HR directors evaluate their states pay-for-performance.
Furthermore, it introduces lessons for designing or developing pay-for-performance
systems, such as improving procedural protections (e.g., due process) that need to be
considered when designing performance appraisals, in the process of civil service
reform.
150
Literature Review
Performance Appraisal Justice and Effectiveness of Pay-for-Performance
We define performance appraisal as the formal process of evaluating organizational
members (Erdogan, 2002, p. 556). Employees are concerned about how performance
appraisal is implemented within organizations. Performance appraisal is influenced by
social context (e.g., Levy & Williams, 2004; Murphy & Cleveland, 1995; Pichler,
2012). For example, the effectiveness of performance appraisal depends on social context: distal variables (e.g., technology, HR strategies, economic conditions), process
proximal variables (e.g., rater issues that directly affect how the appraisal process is
conducted), and structural proximal variables (e.g., multi-source feedback systems,
performance appraisal purpose, rater training) (Levy & Williams, 2004; Murphy &
Cleveland, 1995). Thus, we focus more on the subjective perception of performance
appraisal than on objective performance appraisal.
Performance appraisal fairness can be explained here from three viewpoints;
first, performance appraisal justice is attained when organization members perceive
clear performance appraisal criteria; second, when performance appraisal politics
does not appear in the process of performance appraisal, organization members perceive fair performance appraisal; and third, the payperformance link is crucial in
fostering performance appraisal justice. Accordingly, performance appraisal fairness is defined here as the formal process of rating organizational members performance (Erdogan, 2002).
The impact of feedback intervention (FI) on performance, suggested by Kluger and
DeNisi (1996), focused on the increased impact of both a positive and negative FI on
performance through the reinforcement of correct behaviors and the punishment of
incorrect behaviors. The extent to which FI by raters is associated with performance
might explain the effect of the performance appraisal on pay-for-performance. In particular, performance appraisal justice is considered one of the criteria for assessing the
extent to which performance appraisal is effective within an organization (Jacobs,
Kafry, & Zedeck, 1980).
Pay-for-performance was expected to increase employee performance and productivity because some previous studies (e.g., Deci, 1975) argued that pay could be a
particularly important motivator for employees.1 In general, most previous studies in
the public sector have mentioned concerns related to pay-for-performance, although
some research has reported its positive effect on performance improvement. Those
concerns were associated with the implementation of pay-for-performance, focusing
especially on the fairness of performance appraisals (Battaglio & Condrey, 2006;
Kellough & Lu, 1993; Pearce & Perry, 1983). That is, the effectiveness of pay-forperformance depends on contextual factors, including effective performance appraisals in the organization (Perry, Engbers, & Jun, 2009). Based on an intensive literature
review, Meng and Wu (2015), for instance, found that the critical reason why pay-forperformance may fail as a motivational factor is closely associated with issues in the
performance appraisal processes.
Kim
151
Pearce and Perry (1983) concluded that pay-for-performance has largely failed as a
motivator due to difficulties in the fair performance appraisal processes. In this regard,
Kellough and Lu (1993) argued that performance goals and evaluation criteria are
often diverse, conflicting, and complex to measure. In addition, there were some problems associated with measuring public sector employees performance that may be
attributable to the lack of objective and clear data because performance appraisal
depended on managers judgment of employees performance or attitude in the public
sector. Furthermore, standardized criteria for the comparison between employees performance and distributed pay awards may not always be available (Perry & Pearce,
1985), and such unclear appraisal processes can be subject to numerous biases
(Kellough & Lu, 1993). The unclear payperformance link could also negatively
influence employees behaviors or attitudes in the organization (Deckop, Mangel, &
Cirka, 1999; Koonmee, 2011).
As previous studies have found that performance appraisal criteria, managerial discretion, and the payperformance link are crucial in fostering appraisal fairness
(Deckop etal., 1999; Kellough & Lu, 1993; Koonmee, 2011; Meng & Wu, 2015;
Pearce & Perry, 1983; Perry & Pearce, 1985), this study examines three aspects influencing the perception of performance appraisal justice: performance appraisal criteria,
performance appraisal politics, and the link between merit and reward. That is, performance appraisal fairness is constructed based on the extent to which clear performance
appraisal criteria exist, the determination of performance rating is politicized, and pay
is linked to performance.
Perceived fairness of performance appraisal criteria. Vrooms (1964) expectancy theory
claims that a positive relationship is formed between clear performance appraisal and
employees performance by increasing employees expectancy (i.e., their effortperformance relation) and instrumentality (their performance-reward relation). When
performance appraisal criteria within an organization were determined clearly and
fairly, employees made efforts to attain greater performance. Accordingly, they were
rewarded fairly. Thus, clear performance appraisal criteria motivate employees to
work hard based on the belief that their efforts will be rewarded fairly.
More specifically, Longenecker (1997) argued that formal and specific performance appraisals are required to foster effective management development. According
to Longenecker, the biggest reason for ineffective performance appraisal is unclear
performance criteria. In addition, Folger, Konovsky, and Cropanzano argued in their
1992 study that the effectiveness of performance appraisal might suffer from the lack
of rational and unitary criteria for performance evaluation. Conflicts could arise
between raters and employees in the performance appraisal process due to a lack of
clarity regarding appraisal criteria. The leading criticism of performance appraisals
has been the absence of clear and objective appraisal criteria (Grint, 1993).
Based on qualitative research utilizing data from 120 managers of five U.S. organizations, Longenecker (1997) found that employee performance might improve when
appropriate performance appraisal criteria are devised and clear standards for evaluating employee performance are provided in the job description. In addition, employees
152
satisfaction with pay incentive was positively associated with clear performance
appraisal (Koonmee, 2011). That is, performance appraisal fairness with clear performance criteria correlates with the effectiveness of pay-for-performance. Therefore,
Hypothesis 1 can be suggested as follows:
Hypothesis 1: Clear performance appraisal criteria will be positively associated
with the effectiveness of pay-for-performance in the public sector.
Perceived politicization of performance appraisal. When political actions in an organization are not legitimated by formal authority (Mintzberg, 1983), politics within the
organization could lead to conflict among members. In this regard, political actions
always arise in the process of performance appraisal. Politics in the rating process
would produce misunderstandings between raters and ratees, because performance
appraisal produces political processes that concern two or more parties with different
interests (Folger etal., 1992). By reviewing most of the previous research on performance appraisal politics, Latham and Mann (2006) found that performance appraisal
was a stronger reflection of raters personal judgment than of ratees job performance.
Moreover, raters have a stronger influence on ratees because raters have more power
in the appraisal process (Kacmar & Carlson, 1997).
In particular, it is important that managers consider their political actions in the evaluation process,2 because managers experience office politics in considering the interpersonal relationships between them and their subordinates (Longenecker, Sims, & Gioia,
1987). Managers are influenced by politics when appraising subordinates, thus their
evaluation of subordinates performance could be intentionally and systematically biased
behind a mask of objectivity and rationality (Longenecker etal., 1987). In addition, as
managers, as raters in performance appraisal decision processes, are affected by extraneous variables that are not performance related (Bernardin & Beatty, 1984), they are
likely to be the primary source of bias and inaccuracy in their performance ratings.
Politics are an inevitable reality of organizational life. Moreover, Longenecker etal.
(1987) found that executive raters deliberately inflated or deflated employees performance appraisal if the performance could be improved. Raters positively or negatively
distorted their ratings to shock employees into improving their performance level. In
particular, politicization of performance appraisal tends to increase more easily when
performance rating is tied to a pay raise (Longenecker etal., 1987).
Thus, the politicization of performance appraisal is not positively related to organizational outcome. For example, employees are likely to decrease their performance
when they perceive the involvement of political actions in evaluating their performance (Latham & Mann, 2006). Employees may suspect unfairness, and may have
negative attitudes when they perceive any kind of favoritism or politics in the performance appraisal process. Accordingly, the positive impact of pay-for-performance on
performance can be attained when performance is objectively measured rather than
subjectively rated by supervisors (Rynes, Gerhart, & Parks, 2005).
The politics of performance appraisal also occurs in the public sector, and a negative relationship also exists between politicization of performance appraisal and the
Kim
153
effectiveness of pay-for-performance. Because hierarchically structured public organizations are composed of public managers and subordinates with different power distance, political actions occur when public managers evaluate their subordinates
performance. These performance appraisal politics force public employees to accept
unfair appraisal results from their managers. Gabris and Ihrke (2000) argued that public employees who perceive that their supervisors rate them fairly tend to recognize the
performance appraisal process as valid, and consider the pay-for-performance system
to be distributively fair. Based on such findings from previous studies, Hypothesis 2
can be suggested as follows:
Hypothesis 2: Politicization in performance rating will be negatively associated
with the effectiveness of pay-for-performance in the public sector.
Perceived fairness of the link between merit and reward.Vrooms (1964) expectancy
theory explains the extent to which the payperformance link is fairly perceived by
organizational members. When organizations increase the instrumentality by relating
employees performance with their rewards, employees try to attain a higher level of
performance because they perceive the link between merit and rewards as fair.
Furthermore, this study focuses on the raters justice perception in the process of performance appraisal to explain the link between performance and reward because performance appraisal justice is mainly attained through the accuracy of raters, the raters
cognitive processes, and feedback to employees (Latham & Mann, 2006). This study
deals with the fairness of performance appraisals conducted by supervisors and managers
among various raters. Taylor, Tracy, Renard, Harrison, and Carroll (1995, p. 502) argued
that managers assessments in pay-for-performance need to be examined, as managers
reactions to performance appraisal systems are even more important for system effectiveness than employees because of managers greater power to distort appraisal results and
sabotage its reward, developmental, and documentation purposes.
Multiple scholars (e.g., Judge, 1993; Latham & Mann, 2006) have noted that
employees viewed the appraisal fairness process as separate from the rating results.
Also, other studies (e.g., Dyer & Theriault, 1976; Longenecker, 1997) argued that
employees discriminated between the fairness of the evaluation process and the reward
received. In particular, such distinction is associated with the drawbacks of the raters
forced distribution of performance appraisal ratings. The empirical study by Schleicher,
Bull, and Green (2009), using an experiment with full-time MBA students, found that
raters tend to consider forced distribution of the performance appraisal task as less fair
under conditions of administrative purpose (e.g., promotion, pay raise, and termination) because raters are likely to evaluate employees with more flexibility and control
in the process of performance appraisal for administrative purposes. Thus, employees
perceived less appraisal justice when forced distribution of performance appraisal was
associated with pay-for-performance.
Longenecker (1997) found that the absence of a link between performance ratings
and employee performance rewards might decrease the fairness of pay-for-performance. When employees understand how pay criteria were fairly decided and
154
administered, they are more likely to be satisfied with pay decisions (Dyer & Theriault,
1976). Thus, employees consider the extent to which the performance appraisal process is fairly linked with performance appraisal ratings, and believe that such a link
will lead to fair economic compensation. The empirical study by Judge (1993), investigating the validity of the dimensions of the Pay Satisfaction Questionnaire, found
that a positive link between pay and performance was positively associated with pay
raise satisfaction. That is, when employees are satisfied with their pay raise based on
fair procedural appraisal, such as a payperformance linkage, they tend to be more
productive and motivated.
In a similar vein, it would be logical to assume that public employees could assess
separately the fairness of the performance appraisal process, the performance
appraisal rating, and the resulting pay allocation. The effectiveness of pay-for-performance might depend on how pay-for-performance was decided according to the link
between merit and evaluation in the public sector. The relation between pay discrepancies and motivation was discussed in some previous studies, including Nigro,
Nigro, and Kellough (2007) and Kellough and Selden (1997), which found that
poorly organized pay systems in the public sector could be negatively associated with
job satisfaction or public employee performance. Accordingly, Hypothesis 3 can be
suggested as follows:
Hypothesis 3: A positive link between merit and reward will be positively associated with the effectiveness of pay-for-performance in the public sector.
Kim
155
service reforms, this study takes a different approach from previous research on the
process of performance appraisal justice. In this regard, much is to be gained by studying HR directors perceptions of pay-for-performance. Survey items were adopted
from previous studies (Battaglio & Condrey, 2009; Condrey & Battaglio, 2007;
Kellough & Nigro, 2006).
Table 1 indicates that the six states varied in their levels of four elements of civil
service reform: compensation, pay-for-performance and performance appraisal, range
of grievable issues, and at-will employment. For example, Georgia state employees
were provided with bonuses for performance through the performance-based program
PerformancePlus (Walters, 2002). The 2010 Civil Service Reform Assessment survey
included six U.S. state governments that have experienced considerable civil service
reform, such as pay-for-performance, at-will employment, and implemented pay-forperformance: Colorado, Florida, Georgia, Kansas, Missouri, and South Carolina.
Georgia, for example, experienced extensive and comprehensive civil service reform,
by eliminating merit system protections for all employees after 1996 and decentralizing the states central personnel agency (Kellough & Nigro, 2006). Each states central
personnel offices provided HR directors contact information upon request; subsequently, online and postal surveys were administered.4
Of the 280 individuals included in the sample, 105 (37.5%) usable questionnaires
were returned. The number of respondents from each state was as follows: Colorado
(13), Florida (14), Georgia (21), Kansas (25), Missouri (7), and South Carolina (25).
HR directors who responded to the survey were 61.5% male and 38.5% female, and
average age was 54.06 years. In terms of respondents race, about 20.8% were nonWhite and about 79.2% were White. In addition, most HR directors had attained education beyond a 4-year degree.
Measures
First, a 5-point scale was used to analyze HR directors assessment of performance
appraisal fairness in terms of appraisal criteria, politicization of performance appraisal,
payperformance linkage, and effectiveness of pay-for-performance. Responses to
survey items were coded as 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree,
and 5 = strongly agree.
The effectiveness of pay-for-performance, as a dependent variable, was measured
by two items based on previous research (e.g., Kellough & Lu, 1993; Pearce & Perry,
1983)the effectiveness of pay-for-performance in making public employees productive and motivated. According to the study by Dess and Robinson (1984), the effectiveness of pay-for-performance was assessed based on subjective organization
performance measures.
Independent variables were identified as follows. Similar to previous research (e.g.,
Folger etal., 1992; Longenecker, 1997), the present study utilized relevant items
regarding performance appraisal based on Colquitt (2001). Clarity of appraisal criteria was measured by the extent to which the job description established clear standards
and expectations to evaluate performance. Politicization of performance appraisal
156
Total compensation
system
PerformancePlus
Flexible compensation
system
Florida
Georgia
Kansas
Missouri
South
Carolina
Pay-for-performance and
performance appraisal
Collective bargaining
agreements include range of
grievable issues
HR regulation 19-706
provides guidelines for
at-will employment
At-will employment
Note. PMP = performance management process; SHARP = statewide human resource and payroll system; EPMS = employee performance management system;
HR = human resources; OCGA= official code of Georgia annotated; CSR= code of state regulations; RSMo: Missouri revised statutes; PERforM: productivity,
excellence and results for Missouri.
Total compensation
system
Compensation
Colorado
State
Kim
157
was measured in terms of both office politics and favoritism, following the studies of
organizational politics (e.g., Latham & Mann, 2006). Payperformance linkage was
assessed using items drawn from previous studies (e.g., Dyer & Theriault, 1976;
Judge, 1993). The multiple items of the independent and dependent variables were
combined into a mean score.
As control variables, the present study used HR directors demographic characteristicsage, gender, race, private sector work experience, and educational attainment
because raters perception of performance appraisal justice could differ according to
their demographic characteristics. More specifically, control variables were measured
as follows: Age was measured on a 6-point Likert-type scale (1 = 24 years or younger,
2 = 25-34, 3 = 35-44, 4 = 45-54, 5 = 55-64, 6 = 65 years or older). Gender was coded
as a dummy variable, with female coded as 1 and male as 0. Minority was coded as a
dummy variable with minority coded as 1 and non-minority as 0. Private sector work
experience was coded as 1 if HR directors had private sector work experience.
Otherwise, it was 0. HR directors highest level of educational attainment was coded
1 = high school diploma, 2 = 2-year college degree, 3 = 4-year college degree, 4 =
masters degree, 5 = law degree, 6 = PhD or equivalent. And six states were measured
as a control variable.
Second, to test the factors influencing HR directors assessment of pay-for-performance presented in the hypotheses, the present study utilized the ordered logit regression model instead of ordinary least squares (OLS) regression because this research
used categorical dependent variables. This study used dependent and independent
variables, as shown in Table 2.5 These variables were selected based on an in-depth
literature review (e.g., Deckop etal., 1999; Kellough & Lu, 1993; Koonmee, 2011;
Meng & Wu, 2015; Pearce & Perry, 1983; Perry & Pearce, 1985). In addition to
ordered logit regression, ANOVA tests were conducted to assess the performance
appraisal justice and effectiveness of pay-for-performance across the six states.
Unlike the other empirical studies on performance appraisal justice in surveying
employees (e.g., Heslin & VandeWalle, 2011), the present study examined the perception of HR directors due to the importance of the raters perception of performance
appraisal justice. It could also be meaningful to examine HR directors perceptions of
the fairness of performance appraisals and the effectiveness of pay-for-performance,
because HR directors can help employees make decisions about pay-for-performance
implementation.
Findings
Perceived fairness and effectiveness of pay-for-performance.About 30.59% of survey
respondents believed that pay-for-performance increased productivity. More specifically, HR directors who responded to the survey in Georgia, Kansas, and Missouri
seemed to assess pay-for-performance as a poor tool for increasing productivity; however, many HR directors who responded to the survey in Florida and South Carolina
agreed that pay-for-performance effectively increases productivity. Such results may
be due to the fact that the state governments of Florida and South Carolina provide
158
Number
of items
Dependent variable
Pay-forperformance
effectiveness
Independent variables
Appraisal criteria
Politicization of
performance
appraisal
Payperformance
linkage
1
2
Control Variables
Age
Gender
Race
Private sector
experience
Education
1
1
1
1
1
Sample items
.76
.73
.67
clearer rules and guidelines than other states. Further systematic studies are required
for more articulated interpretations of why HR directors in these six U.S. state governments assessed the effectiveness of pay-for-performance in increasing productivity
differently. Fewer HR directors in Georgia (21.1%), Kansas (25%), and Missouri
(16.7%) agreed that the pay-for-performance system was a good way to motivate state
employees than in Florida and South Carolina (45.5% and 50%, respectively).
Performance appraisal criteria. Many HR directors agreed that pay-for-performance
provides the clear standards and expectations that are critical for fair performance
appraisal. Specifically, 81.3% of HR directors who responded to the survey agreed
159
Kim
Table 3. State Mean Difference in Performance Appraisal Justice and Effectiveness of
Pay-for-Performance.
State
Colorado
Florida
Georgia
Kansas
Missouri
South Carolina
Total
Appraisal
criteria
Politicization of
performance
appraisal
Pay
performance
linkage
Pay-forperformance
effectiveness
3.53
3.54
3.31
3.81
3.50
3.61
3.55
2.76
3.27
2.71
2.75
2.58
2.57
2.75
3.48
3.42
3.45
3.35
2.77
3.60
3.42
2.83
3.35
2.63
2.71
2.50
3.40
2.95
that their states provide clear standards and expectations for employee performance
appraisals.6
Politicization of performance appraisal. About 80% of survey respondents agreed that
office politics might not influence pay-for-performance. Approximately 45.5% of HR
directors in the Florida state government assessed favoritism as a problem for the payfor-performance program in their state agenciesa higher percentage than in other
states. Such results were similar to the findings from Battaglio and Condreys (2006)
study, which found that the reform of Florida state was relatively radical in that it
sought a privatization initiative targeted at outsourcing personnel management system
by Service First.
Payperformance linkage. HR directors in Florida (45.5%), Georgia (47.4%), and
Missouri (50%) agreed that pay raises in their work units are often unrelated to performance.7 Such an empirical finding is similar to concerns raised by scholars and
practitioners in previous studies (e.g., Kellough & Nigro, 2002).
Table 3 indicates the perception of HR directors regarding performance appraisal
justice and the effectiveness of pay-for-performance. Although they perceived a
high degree of appraisal criteria, politicization of performance appraisal, and pay
performance linkage, there were no significant differences among the six states;
however, HR directors in Florida (M = 3.35) and South Carolina (M = 3.40) perceived significantly higher levels of effectiveness of pay-for-performance than HR
directors in the other four states (Colorado, Georgia, Kansas, and Missouri). More
explanations are provided in the descriptive statistics (see Table 4).
Results of ANOVA and ordered logit regression. The simple correlation matrix showed that
the effectiveness of pay-for-performance was positively correlated with performance
appraisal criteria (r = .257, p < .05) and the payperformance linkage (r = .259, p < .05)
but insignificantly correlated with politicization of performance appraisal (r = .155,
160
1. Pay-for-performance
effectiveness
2. Appraisal criteria
3. Politicization of
performance appraisal
4. Payperformance
linkage
5. Age
6. Gender
7. Race
8. Private sector
experience
9. Education attainment
10. State
0.88
0.82
0.81
0.77
0.73
0.48
0.32
0.4
0.78
1.67
3.56
2.76
3.43
4.06
0.38
0.21
0.20
3.4
3.7
SD
2.95
1
1
2
0
0
0
1.67
1
1
5
6
5
1
1
1
5
5
Minimum Maximum
.001
.16
.09
.12
.05
.13
.26**
.26**
.15
.10
.047
.08.
.02
.19
.09
.28**
17
.11
.18
.08
.006
.35*** .54***
.31**
.12
.05
.09
.13
.17
.09
.23**
.18
.18
.09
.11
.06
.21** .46*** .07
.23**
.14
.04
.15
.08
.03
161
Kim
Table 5. ANOVA of Performance Appraisal Justice and Effectiveness of Pay-forPerformance.
Variables
Sum of squares
Appraisal criteria
Between groups
2.256
Within groups
54.953
Total
57.209
Politicization of performance appraisal
Between groups
3.813
Within groups
51.493
Total
55.306
Payperformance linkage
Between groups
3.329
Within groups
48.371
Total
51.700
Pay-for-performance effectiveness
Between groups
9.919
Within groups
53.888
Total
63.807
Mean square
0.451
0.687
0.657
0.763
0.652
1.170
0.666
0.605
1.101
1.984
0.700
2.835**
162
incremental changes in civil service reform would be ineffective in management flexibility and performance (Klingner, 2006). Nevertheless, the levels of civil service
reform varied across six states. Thus, the levels of the reforms across the six states
might significantly influence the effectiveness of pay-for-performance.
According to the ordered logit regression analyses, the value of log likelihood
explained the fitness of model through the Likelihood Ratio Chi-Square test of whether
all predictors regression coefficients in the model are simultaneously zero and in
nested models. In terms of performance appraisal fairness, performance appraisal criteria and politicization of performance appraisal significantly influenced HR directors
assessment of pay-for-performance effectiveness. More specifically, HR directors
assessed the effectiveness of pay-for-performance positively when they perceived that
job descriptions included clear standards or guidelines for employee performance
evaluation ( = 0.871, p < .01); however, politicization of performance appraisal was
negatively associated with HR directors assessment of pay-for-performance effectiveness ( = 0.904, p < .05). This implies that HR directors perceived effectiveness of
pay-for-performance would increase when they perceive a high level of clear performance appraisal and a low level of politicization of performance appraisal. Thus,
Hypotheses 1 and 2 were accepted. In addition, no statistically significant relationship
emerged between the payperformance linkage and pay-for-performance effectiveness. That is, Hypothesis 3 is rejected.
In particular, the proportional odds ratios in the ordered logit regression were interpreted as follows (e.g., Greene, 2008). For example, for a one-unit increase in the
clarity of appraisal criteria, the odds for cases in a group that is greater than neutral
versus less than or equal to neutral are significantly 2.39 times larger, given that
other variables are held constant. Conversely, for a one-unit increase in politicization
of performance appraisal, the odds for cases in a group that is greater than neutral as
opposed to less than or equal to neutral are significantly 0.4 times smaller.
Although the present study included demographic characteristics and the six states
in ordered logit regression, it found that age and gender were significantly associated
with HR directors assessment of pay-for-performance effectiveness. Younger HR
directors seemed to assess pay-for-performance more positively ( = 0.707, p < .1).
In addition, this study found that male HR directors tend to assess pay-for-performance effectiveness more favorably ( = 1.48, p < .01). Other demographic characteristics, including race, private sector work experience, and educational attainment,
were not statistically significant in explaining HR directors assessment of pay-forperformance effectiveness. Interestingly, this study found that HR directors who had
private sector work experience might assess pay-for-performance as being more effective than otherwise, although the relationship was not significant.
In addition, South Carolina was treated as the reference category in the ordered
logit regression. The present study found that HR directors in Colorado, Georgia,
Kansas, and Missouri assessed the effectiveness of pay-for-performance less positively than in South Carolina ( = 2.34, p < .01; = 2.23, p < .01; = 2.41, p < .01;
= 3.35, p < .01, respectively, see Table 6). South Carolina has experienced substantial civil service reform. More specifically, after passing the State Government
163
Kim
Table 6. Ordered Logit Regression of Pay-for-Performance Effectiveness.
Pay-for-performance effectiveness
Appraisal criteria
Politicization of performance
appraisal
Payperformance linkage
Age
Gender
Race
Private sector experience
Education attainment
Colorado
Florida
Georgia
Kansas
Missouri
_cut1
_cut2
_cut3
_cut4
_cut5
_cut6
_cut7
_cut8
Log likelihood
Coefficient
0.871***
.904**
0.155
0.707*
1.485***
0.157
0.096
0.104
2.341***
0.988
2.239***
2.411***
3.358***
7.785
7.199
6.386
5.01
3.441
2.562
.247
1.673
SE
Odds ratio
0.334
0.385
2.39
0.404
0.371
0.378
0.533
0.186
0.076
0.362
0.845
0.839
0.679
0.756
0.999
3.376
3.357
3.336
3.273
3.251
3.272
3.294
3.415
118.73005
1.168
0.492
0.226
0.853
1.218
0.9008
0.096
2.687
0.106
0.089
0.034
Note. The analysis of the present study did not include missing values, so the number of observations
utilized was 74 among 105 usable responses.
*p < .1. **p < .05. ***p < .01.
Accountability Act of 1993, the state experienced a substantial civil service reform,
the decentralization of the human resources management (HRM) function, and
empowerment of line managers regarding HR decisions (Hays, Byrd, & Wilkins,
2006). Thus, HR directors in South Carolina perceived the effectiveness of pay-forperformance positively.
164
etal., 2007) despite some side effects, such as dysfunctional competition among
employees. Although many scholars and practitioners (e.g., Kellough & Lu, 1993) have
warned of the negative effects of pay-for-performance, many public organizations have
still implemented a pay-for-performance system. Because those supporting the effectiveness of pay-for-performance believed that pay-for-performance is associated with
fair managerial control (Perry etal., 2009), the extent to which performance appraisal
was fair could determine the success of pay-for-performance.
The present study examined how HR directors in six U.S. state governments assess
their states pay-for-performance effectiveness in terms of performance appraisal
fairness by analyzing previous studies and the 2010 U.S. Civil Service Reform
Assessment Survey data. Although pay-for-performance was designed to increase
efficiency and productivity by adopting a market-based public service delivery model
in all levels of government (Peters, 1995), such a system has also had negative consequences. Which factors of pay-for-performance influence employees performance
positively or negatively? The present study assessed whether perceived fairness of
performance appraisal might influence the effectiveness of pay-for-performance.
More specifically, this study found that perceived unfairness in performance
appraisalespecially perceived unclear appraisal criteria and politicization of performance appraisalwas significantly associated with pay-for-performance effectiveness. In particular, the pay-for-performance system in the Justice Department
actually produced negative results like politicization of civil service and manipulation of employee appraisal (Bowman, 2010, p. 82).
So, how can unclearness and politicization of performance appraisal be controlled
to improve employees perception of fairness? The present studys results showed that
perceived effectiveness of pay-for-performance increases when job descriptions
clearly state the criteria for employee performance evaluation, which is consistent
with the findings of previous studies on the importance of clear job contents. To
improve performance appraisal criteria, Mesch and Rooney (2008), for instance,
emphasized a requirement for a more direct linkage between job responsibilities and
performance appraisal criteria. Chang and Hahn (2006) found that the existence of
sound and clear performance appraisal criteria contributes to improving the effectiveness of pay-for-performance. Similarly, the effectiveness of pay-for-performance
increases when the pay-for-performance system operates transparently (Bell, 2015).
Furthermore, Chiang and Birtch (2007) argued that performance appraisal criteria
need to be developed based on organizational culture to reduce conflict among employees. Also, such criteria should provide employees with clear goals for performance
(Longenecker, 1997). In particular, because ethical norms perish as a negative result of
civil service reform (Bowman & West, 2007), managers are required to adopt ethical
attitudes such as high levels of integrity and reliability to prevent politicization of
performance appraisal when evaluating employee performance.
OReilly and Pfeffer (2000) argued that organizations need to unlock hidden value
in all employees to achieve productivity and efficiency. To do that, organizations must
practice pay-for-performance carefully. That is, pay should reflect employees value,
and employees value should be determined fairly. In addition, the meaning of payment
Kim
165
166
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship,
and/or publication of this article: The paper was supported by The University of Suwon in 2015.
Notes
1. Empirical studies on pay-for-performance, however, have shown that the system has both
positive and negative effects on employee performance. That is, in terms of effectiveness,
many scholars have questioned whether pay-for-performance can help employees improve
their performance by financially motivating employees to work efficiently (Gabris &
Ihrke, 2000; Gabris & Mitchell, 1985; Montoya & Graham, 2007; Oh & Lewis, 2009;
Perry, 1992; Pynes, 2009; Rainey & Kellough, 2000).
2. According to the National Research Council (1991), pay-for-performance can be implemented inappropriately if rewards are assigned based on managers discretion. Managers
in Florida, for instance, can hire, fire, and reward employees at their discretion (Battaglio
& Condrey, 2006). Therefore, managerial discretion in employee evaluation is an ethical
issue because managers decisions can be cognitively biased, for example, by the halo
effect (Kellough & Lu, 1993). More discretion by managers would likely produce unstandardized and arbitrary criteria in performance appraisal, because they are likely to evaluate their subordinates based on their own subjective judgments rather than standardized
criteria.
3. However, the survey questionnaires do not include whether the respondents actually conducted the performance appraisal.
4. Survey questionnaires were sent by postal mail to those human resources (HR) directors
who preferred such a survey method or who did not provide an email address.
5. Table 1 presents survey items and Cronbachs alpha values.
6. About 69.2%, 66.7%, and 66.7% of HR directors in Colorado, Missouri, and South
Carolina state governments, respectively, also agreed that their states provide clear
appraisal standards.
7. Most HR directors in most of these states, including Colorado, Florida, Georgia, Missouri,
and South Carolina, disagreed that performance ratings of better than met expectations
are rotated among employees who deserve meaningful pay raises. In the case of Florida,
however, none of the HR directors who responded to the survey agreed or strongly agreed
with this. In addition, HR directors who responded to the survey, except for those in
Missouri, were less likely to claim that management had imposed quotas or limits on
167
Kim
the number of performance ratings above met expectations. In the case of HR directors
in Missouri who responded to the survey, 66.7% seemed to believe that there are quotas or
limits on the number of performance ratings although they feel that performance appraisals
are conducted fairly.
References
Battaglio, P. R., Jr., & Condrey, S. E. (2006). Civil service reform: Examining state and local
government cases. Review of Public Personnel Administration, 26, 118-138.
Battaglio, P. R., Jr., & Condrey, S. E. (2009). Reforming public management: Analyzing the
impact of public service reform on organizational and managerial trust. Journal of Public
Administration Research and Theory, 19, 689-707.
Bell, N. (2015). Performance-related pay and the crowding-out of motivation in the public sector: A randomized field experiment. Public Administration Review, 75, 230-241.
Bernardin, H. J., & Beatty, R. W. (1984). Performance appraisal: Assessing human behavior
at work. Boston, MA: Kent.
Bowman, J. S. (2010). The success of failure: The paradox of performance pay. Review of
Public Personnel Administration, 30, 70-88.
Bowman, J. S., & West, J. P. (2006). Ending civil service protections in Florida government:
Experiences in state agencies. Review of Public Personnel Administration, 26, 139-157.
Bowman, J. S., & West, J. P. (2007). Ending civil service protections in Florida government:
Experiences in state agencies. In J. S. Bowman & J. P. West (Eds.), American public service: Radical reform and the merit system (pp. 123-149). New York, NY: Taylor & Francis.
Bowman, J. S., West, J. P., & Gertz, S. C. (2006). Floridas service first: Radical reform in
the sunshine state. In J. E. Kellough & L. G. Nigro (Eds.), Civil service reform in the
states: Personnel policy and politics at the subnational level (pp. 145-170). Albany: State
University of New York Press.
Carnevale, D., & Housel, S. (2001). Human resource management reform in the states. In S.
Condrey & M. Robert (Eds.), Radical reform of the civil service (pp. 151-175). Lanham,
MD: Lexington Books.
Chang, E., & Hahn, J. (2006). Does pay-for-performance enhance perceived distributive justice
for collectivistic employees? Personnel Review, 35, 397-412.
Chiang, F. F. T., & Birtch, T. (2007). The transferability of management practices: Examining
cross-national differences in reward preferences. Human Relations, 60, 1293-1330.
Colquitt, J. A. (2001). On the dimensionality of organizational justice: A construct validation of
a measure. Journal of Applied Psychology, 86, 386-400.
Condrey, S. E., & Battaglio, P. R., Jr. (2007). A return to spoils? Revisiting radical civil service
reform in the United States. Public Administration Review, 67, 425-436.
Deci, E. L. (1975). Intrinsic motivation. New York, NY: Plenum.
Deckop, J. R., Mangel, R., & Cirka, C. C. (1999). Getting more than you pay for: Organizational
citizenship behavior and pay-for-performance plans. Academy of Management Journal, 42,
420-428.
Dess, G., & Robinson, R. R., Jr. (1984). Measuring organizational performance in the absence
of objective measures: The case of the privately-held firm and conglomerate business unit.
Strategic Management Journal, 5, 265-273.
Dyer, L., & Theriault, R. (1976). The determinants of pay satisfaction. Journal of Applied
Psychology, 61, 596-604.
Erdogan, B. (2002). Antecedents and consequences of justice perceptions in performance
appraisals. Human Resource Management Review, 12, 555-578.
168
Folger, R., & Konovsky, M. A. (1989). Effects of procedural and distributive justice on reactions to pay raise decisions. Academy of Management Journal, 32, 115-130.
Folger, R., Konovsky, M. A., & Cropanzano, R. (1992). A due process metaphor for performance appraisal. In B. M. Staw & L. L. Cummings (Eds.), Research in organizational
behavior (Vol. 14, pp. 129-177). Greenwich, CT: JAI Press.
Gabris, G. T., & Ihrke, D. M. (2000). Improving employee acceptance toward performance
appraisal and merit pay systems: The role of leadership credibility. Review of Public
Personnel Administration, 20, 41-53.
Gabris, G. T., & Mitchell, K. (1985). Merit-based performance appraisal and productivity: Do
employees perceive the connection? Public Productivity Review, 9, 311-327.
Greene, W. H. (2008). Econometric analysis (6th ed.). New Jersey, NJ: Prentice Hall.
Grint, K. (1993). Whats wrong with performance appraisals? A critique and a suggestion.
Human Resource Management Journal, 3(3), 61-77.
Hays, S. W., Byrd, C., & Wilkins, S. L. (2006). South Carolinas human resource management
system: The model for states with decentralized personnel structures. In J. E. Kellough &
L. G. Nigro (Eds.), Civil service reform in the states: Personnel policy and politics at the
subnational level (pp. 171-201). Albany: State University of New York Press.
Heslin, P. A., & VandeWalle, D. (2011). Performance appraisal procedural justice: The role of
a managers implicit person theory. Journal of Management, 37, 1694-1718.
Jacobs, R., Kafry, D., & Zedeck, S. (1980). Expectations of behaviorally anchored rating scales.
Personnel Psychology, 33, 595-640.
Judge, T. A. (1993). Validity of the dimensions of the Pay Satisfaction Questionnaire:
Implications for research and practice. Personnel Psychology, 46, 331-355.
Kacmar, K. M., & Carlson, D. S. (1997). Further validation of the perceptions of politics scales:
A multiple sample investigation. Journal of Management, 23, 627-658.
Kellough, J. E., & Lu, H. (1993). The paradox of merit pay in the public sector: Persistence of a
problematic procedure. Review of Public Personnel Administration, 13(2), 45-64.
Kellough, J. E., & Nigro, L. G. (2002). Pay for performance in Georgia state government:
Employee perspectives on GeorgiaGain after 5 years. Review of Public Personnel
Administration, 22, 146-166.
Kellough, J. E., & Nigro, L. G. (2006). Dramatic reform in the public service: At-will employment and the creation of a new public workforce. Journal of Public Administration Research
and Theory, 16, 447-466.
Kellough, J. E., & Selden, S. C. (1997). Pay-for-performance systems in state government:
Perceptions of state agency personnel managers. Review of Public Personnel Administration,
17(1), 5-21.
Klingner, D. E. (2006). Societal values and civil service systems in the United States. In J.
E. Kellough & L. G. Nigro (Eds.), Civil service reform in the states: Personnel policy
and politics at the subnational level (pp. 11-32). Albany: State University of New York
Press.
Kluger, A. N., & DeNisi, A. (1996). The effect of feedback interventions on performance:
A historical review, meta-analysis, and a preliminary feedback intervention theory.
Psychological Bulletin, 119, 254-284.
Koonmee, K. (2011). Fairness in the workplace: The relative effects of distributive and procedural justice on incentive satisfaction. The Business Review, 17(2), 160-166.
Latham, G. P., & Mann, S. (2006). Advances in the science of performance appraisal:
Implications for practice. In G. P. Hodgkinson & J. K. Ford (Eds.), International
review of industrial and organizational psychology (Vol. 21, pp. 295-337). Hoboken,
NJ: Wiley.
Kim
169
Levy, P. E., & Williams, J. R. (2004). The social context of performance appraisal: A review
and framework for the future. Journal of Management, 30, 881-905.
Longenecker, C. O. (1997). Why managerial performance appraisals are ineffective: Causes and
lessons. Career Development International, 2, 212-218.
Longenecker, C. O., Sims, H. P., & Gioia, D. A. (1987). Behind the mask: The politics of
employee appraisal. Academy of Management Executive, 1, 183-193.
McFarlin, D. B., & Sweeney, P. D. (1992). Distributive and procedural justice as predictors of
satisfaction with personal and organizational outcomes. Academy of Management Journal,
35, 626-637.
Meng, F., & Wu, J. (2015). Merit pay fairness, leader-member exchange, and job engagement:
Evidence from mainland China. Review of Public Personnel Administration, 35, 47-69.
Mesch, D. J., & Rooney, P. M. (2008). Determinants of compensation: A study of pay, performance, and gender differences for fundraising professionals. Nonprofit Management &
Leadership, 18, 435-463.
Mintzberg, H. (1983). Power in and around organizations. Englewood Cliffs, NJ: Prentice Hall.
Montoya, S., & Graham, J. D. (2007). Pay for performance in the U.S. federal civil service.
Policy Insight, 1(6), 1-4.
Moynihan, D. P. (2005). Goal-based learning and the future of performance management.
Public Administration Review, 65, 203-216.
Murphy, K. R., & Cleveland, J. N. (1995). Understanding performance appraisal: Social, organizational, and goal-based perspectives. Thousand Oaks, CA: SAGE.
National Research Council. (1991). Pay-for-performance: Evaluating performance appraisal
and merit pay. Washington, DC: National Academies Press.
Nigro, L., Nigro, F., & Kellough, J. E. (2007). The new public personnel administration (6th
ed.). Belmont, CA: Thomson Wadsworth.
Oh, S. S., & Lewis, G. B. (2009). Can performance appraisal systems inspire intrinsically motivated employees? Review of Public Personnel Administration, 29, 158-167.
OReilly, C. A., & Pfeffer, J. (2000). Hidden value: How great companies achieve extraordinary results with ordinary people. Boston, MA: Harvard Business School Press.
Pearce, J. L., & Perry, J. L. (1983). Federal merit pay: A longitudinal analysis. Public
Administration Review, 43, 315-325.
Perry, J. L. (1992). The merit pay reforms. In P. W. Ingraham & D. H. Rosenbloom (Eds.), The
promise and paradox of civil service reform (pp. 199-215). Pittsburgh, PA: University of
Pittsburgh Press.
Perry, J. L., Engbers, T. A., & Jun, S. Y. (2009). Back to the future? Performance-related pay,
empirical research, and the perils of persistence. Public Administration Review, 69, 39-51.
Perry, J. L., & Pearce, L. J. (1985). Civil service reform and the politics of performance
appraisal. In D. H. Rosenbloom (Ed.), Public personnel policy: The politics of civil service
(pp. 146-160). Port Washington, NY: Associate Faculty Press.
Peters, B. G. (1995). The public service, the changing state, and governance. In B. G. Peters
& D. J. Savoie (Eds.), Governance in a changing environment (pp. 288-320). Montreal,
Quebec: McGill-Queens University Press.
Pichler, S. (2012). The social context of performance appraisal and appraisal reactions: A metaanalysis. Human Resource Management, 51, 709-732.
Pynes, J. E. (2009). Human resources management for public and nonprofit organizations: A
strategic approach (3rd ed.). San Francisco, CA: Jossey-Bass.
170
Rainey, H. G., & Kellough, J. E. (2000). Civil service reform and incentives in the public service. In J. P. Pfiffner & D. A. Brook (Eds.), The future of merit: Twenty years after the
Civil Service Reform Act (pp. 127-145). Washington, DC: Woodrow Wilson Center Press.
Rubin, E. V. (2009). The role of procedural justice in public personnel management: Empirical
results from the department of defense. Journal of Public Administration Research and
Theory, 19, 125-143.
Rubin, E. V., & Kellough, J. E. (2012). Does civil service reform affect behavior? Linking
alternative personnel systems, perceptions of procedural justice, and complaints. Journal of
Public Administration Research and Theory, 22, 121-141.
Rynes, S. L., Gerhart, B., & Parks, L. (2005). Personnel psychology: Performance evaluation
and pay for performance. Annual Review of Psychology, 56, 571-600.
Schleicher, D. J., Bull, R. A., & Green, S. G. (2009). Rater reaction to forced distribution rating
system. Journal of Management, 35, 899-927.
Taylor, M. S., Tracy, K. B., Renard, M. K., Harrison, J. K., & Carroll, S. J. (1995). Due process
in performance appraisal: A quasi-experiment in procedural justice. Administrative Science
Quarterly, 40, 495-523.
Vroom, V. H. (1964). Work and motivation. San Francisco, CA: Jossey-Bass.
Walters, J. (2002). Life after civil service reform: The Texas, Georgia, and Florida experiences.
Washington, DC: IBM Endowment for The Business of Government.
Author Biography
Jungin Kim is an assistant professor in the Department of Public Administration at the
University of Suwon. Her research interests include human resources management and hman
resources development policies, organizational behavior, and organizational management. She
has had articles published in several journals, such as Public Administration Review (PAR).
Copyright of Public Personnel Management is the property of Sage Publications Inc. and its
content may not be copied or emailed to multiple sites or posted to a listserv without the
copyright holder's express written permission. However, users may print, download, or email
articles for individual use.