PERFORMANCE APPRAISAL FOR FIRE FIGHTER RECRUITS

EXECUTIVE DEVELOPMENT

BY: Hiram Keliipio

Honolulu Fire Department
Honolulu, Hawaii

An applied research project submitted to the National Fire Academy

as part of the Executive Fire Officer Program

February 2000
 2

ABSTRACT

The fire service has utilized performance appraisals (PA) to evaluate its personnel. The

problem was that the Honolulu Fire Department (HFD) has not developed an efficient and

effective system of evaluating the performance of Fire Fighter Recruits.

The purpose of this applied research project was to develop a performance appraisal report for

Fire Fighter Recruits. An action research methodology was used to answer the following research

questions:

1. What were the components of an effective performance appraisal system?

2. How have liked sized fire departments utilized these components?

3. What components could the HFD use in its appraisals?

4. What limitations/constraints were there in using parts of other successful systems?

The procedures utilized to complete this research project included a literature review of

textbooks, magazines, and trade journals, a review of the performance appraisals from like sized

departments, and an evaluation of the components that the HFD can included in creating a

performance appraisal for Fire Fighter Recruits.

The results of this research were that an effective PA included the objectives of the PA, the

process the PA followed, the establishing of standards and goals, a strategy for overcoming obstacles,

and the mitigation of legal issues. Results also showed that like sized departments were successful

with their PA because they prepared their recruits for the PA, they established clearly defined

requirements and standards, they ensured that the PA was comprehensive in content, the PA was done

on a regularly scheduled basis, two-way dialog was encouraged, and the individual was certain of his

standing with the department. Finally, results revealed that most of these components may be

utilized by the HFD and that limitations/constraints included rating problems, the methods by which
 3

the organization structurally formulated the PA system, the organizational purpose for the PA, the

procedural process, and the increased burden to the organization’s workload that resulted from legal

issues.

Recommendations were that the HFD’s current PA for Fire Fighter Recruits be discarded

and that a new PA be formulated that included generic components from effective PA systems

and specific components from the fire departments that were scrutinized. It was also

recommended that limitations/constraints were accounted for prior to the formulation of the PA

and that this accounting continued through the implementation and evaluation phases.
 4

TABLE OF CONTENTS

ABSTRACT.......................................................................................................................... 2

TABLE OF CONTENTS .................................................................................................... 4

INTRODUCTION................................................................................................................ 5

BACKGROUND AND SIGNIFICANCE .......................................................................... 6

LITERATURE REVIEW ................................................................................................... 8

PROCEDURES .................................................................................................................... 19

RESULTS ............................................................................................................................. 22

DISCUSSION ....................................................................................................................... 28

RECOMMENDATIONS..................................................................................................... 31

REFERENCES..................................................................................................................... 33

APPENDIX A: Recruit Performance Appraisal....................................................................... 35

 5

INTRODUCTION

The Honolulu Fire Department (HFD) was established in 1841. Through the years, it has

had a proud and colorful history. The HFD is the only fire department in the United States of

America to have been originally established by a sovereign monarch through a Royal Decree; it

is the only fire department in the nation to have fought a major conflagration under direct attack

by a foreign aggressor; and it remains the only fire department whose entire jurisdictional area is

completely surrounded by an ocean.

Today, the HFD is a major metropolitan fire department servicing a population base of

over one million people. To properly sustain this community, the HFD uses a workforce

consisting of 1,127 uniformed and civilian personnel to staff forty-four fire stations and four

administrative bureaus. The Department’s uniformed personnel are career, paid Fire Fighters.

The mission of the HFD is to respond to fires, emergency medical incidents, hazardous

materials incidents, and rescues on land and sea to save lives, property, and the environment. In

order to accomplish this mission, the HFD must overcome many challenges, some of which are

similar to those encountered by fire departments throughout the nation. One such challenge is to

develop a method or system through which the HFD appraises the performance of its workforce.

The HFD is committed to addressing the challenge of formulating, implementing and

evaluating a performance appraisal system for all ranks within the department. To date,

however, the HFD has not been able to commit a proper amount of resources to successfully

mitigate this issue.

The problem prompting this applied research project is that the HFD has not developed

an efficient and effective system of evaluating the performance of Fire Fighter Recruits. The
 6

purpose of this applied research project is to develop a performance appraisal report for Fire

Fighter Recruits.

Action research, including a literature review, is employed to answer the following

research questions:

4. What are the components of an effective performance appraisal system?

5. How have liked sized fire departments utilized these components?

6. What components could the HFD use in its appraisals?

7. What limitations/constraints are there in using parts of other successful systems?

The procedures used to complete this research include a literature of textbooks,

magazines, and trade journals, a review of the performance appraisals from like sized

departments, and an evaluation of the components that the HFD can includ in creating a

performance appraisal for Fire Fighter Recruits

BACKGROUND AND SIGNIFICANCE

Today, one hundred and fifty years since its inception, the HFD is experiencing growth

pains unlike any other time in its history. This is most evident in the changing demographics of

its workforce.

During the past five years, a dramatic increase occurred in the number of entry level

personnel (Fire Fighter Recruits) that joined the ranks of the HFD. This increase is attributed to

several factors. First, a large percentage of the workforce served a sufficient amount of years

and are able to retire. Second, several pay and benefit incentives created by the state legislature

encourage government employees to retire at the earliest possible time. Third, the State of

Hawaii’s stagnant economy is creating a monetary disincentive for personnel to remain on the
 7

job because the negotiated Collective Bargaining Agreement between the County and the Fire

Fighters Union is resulting in insignificant pay raises for the fire fighters. Fourth, the HFD

added five new fire companies to the suppression force.

The HFD has undertaken an aggressive program to fill these manpower vacancies as

quickly as possible. Two hundred fifty-one Fire Fighter Recruits were placed in service over the

past five years and an additional one hundred seventeen Recruits are scheduled to be placed in

service this year. Throughout this hiring frenzy, however, the HFD’s leaders have continually

articulated, in both their words and actions, the need to ensure that the recruit training process is

not just “passing through warm bodies” for the suppression force, but that every individual

completing recruit training receives proper preparation and training.

To this end, the Executive Chief Officers of the HFD have supported the recruit training

process with the manpower, equipment and facilities in sufficient quantities as needed to attain

this goal. With the increase in the numbers of recruits requiring training in such a brief period of

time, however, it is necessary for several portions of the recruit training program to be upgraded.

One of these components that requires revamping is the Fire Fighter Recruit Performance

Appraisal report.

In the past, the HFD Recruit Training Officers (RTOs), treated the PA as a necessary evil

required by the department as a simple way to report on the progress of the recruits. The PA was

dealt with as quickly as possible and was only given special time and effort in cases involving

recruits that were in jeopardy of being terminated from service. The result is that the PA is a

very ineffective tool.

Presently, the RTOs and the Executive Chief Officers understand how important a tool

the PA is in the training of the recruits. In addition, laws and regulations addressing employee
 8

rights in the workplace require that employers conduct fair appraisals of employees. As a result,

the PA reports are done on a regular basis, the RTOs invest a significant amount of time and

effort in the process, and the recruits benefit by gaining a clear understanding of their current

status and progress throughout the class.

It is apparent to HFD personnel involved in the recruit training process that an overhaul is

needed of the entire system through which the department’s recruits are appraised. A good

starting point would be with the PA report since it is the cornerstone of the system. The concern

is that the current PA report is outdated and, more importantly, that the department will be more

adversely affected, and suffer long term repercussions, should improperly trained recruits be sent

into the field.

This paper is being produced to satisfy the applied research project required by the

Executive Fire Officer Program – Executive Development Course at the National Fire Academy.

The issue of the PA reports relates to the unit in the course dealing with organizational culture

and the difficulties of employee evaluations and the benefits obtained through a positive

evaluation process. It also touches upon components in the units on labor relations, service

quality and legal issues.

LITERATURE REVIEW

A literature review was performed to identify existing research on Performance

Appraisals. The research questions are identified and summaries provided.

Research Question 1. Several major components are present in an effective PA.

Several authors stress the importance of setting clear objectives that the PA should attain.

Devill (1995) lists four key objectives for the PA. He says that the PA is a means for
 9

management to communicate the organizational goals and objectives to employees and it is a

system to motivate employees in order to improve individual performance, to equitably distribute

organizational rewards, such as salary increases and promotions, and for researchers to conduct

personnel management research. Devill sees the PA as a way through which the manager can

keep in touch with employees. He claims that this is the number one challenge for management.

Smither (1998) views the objective of the PA in relationship to the benefits it should

ultimately deliver to its targeted audience. For Smither, appraisals are based on the standards of

the customers’ goals and not the goals of the rater or subordinate. Appraisals are seen not as an

end but in the context of performance management. It links business objectives, day to day

performance, development and compensation. It also is linked to feedback, training, and

compensation. Because of this linkage, and not as part of the procedural process, he insists that

rater training is imperative.

The objective of the PA according to Gilley and Boughton (1996) is to serve as a

powerful tool to help managers communicate with employees regarding their performance. The

PA recognizes the employees’ strengths and achievements and defines performance goals and

action plans for the future. The PA helps to compare the manager’s expectations with employee

performance.

For Aurnhammer (1996), the formal PA serves three basic functions: it informs the

subordinate on how they are doing, it gives supervisors a more objective method to look at

performance and it identifies and assists in correcting deficient behaviors. The good PA, says

Aurnhammer, delivers constructive criticism in a proper manner, instructs its user on how to

handle negative reactions, and coaches and motivates employees to enhance performance.
 10

Aurnhammer believes that personnel are the most valuable resource in any organization.

To properly service this resource, the PA is based on specific goals the manager wants to achieve

and to develop behaviors an employer would like an employee to exhibit.

The objectives of the PA may include several varying components. Sachs (1991)

identifies these components. The PA, he claims, sets goals and responsibilities, it documents

behavior and performance, it prepares the employee to participate in the process, and it prepares

the evaluator for the process. As part of the objective, the supervisor must have an open mind

when it concerns the employee’s future goals, he must not discuss compensation and he must

follow through on plans for supervisor and employee feedback.

The PA, for Sachs, does not focus on deficiencies, but rather on how to correct the

deficiencies. A final objective for the PA is for the evaluator to compile a list of the employee’s

long and short term goals.

The second major component of the successful PA addresses the process through which

the PA is conducted. Haden (1995) focuses on six principles that he believes are necessary.

These are:

1. The PA is based on job analysis.

2. It measures behaviors rather than traits.

3. It is regularly reviewed with the employee.

4. It complies with applicable human rights legislation.

5. It can be empirically validated.

6. It accounts for known biases.

Taking it a step further, Haden emphasizes the importance of bringing into the process

the individuals conducting the appraisals. These personnel should receive proper training in the
 11

use of the appraisal system, be able to consistently observe employees on the job, have specific

written instructions on how to conduct the appraisal, know applicable human rights legislation,

and be familiar with possible biases.

For Clark (1999) the process of conducting the PA is accomplished in three steps. First,

the individual performs an appraisal on himself. Second, the employees who work with the

employee being appraised perform an appraisal on that employee. Third, the supervisor

performs an appraisal on the employee. Through all this, a performance guide check sheet is

used by those performing the appraisal. The check sheet covers job related items such as how

the employee performs on the fireground, company drills, station activities, and it measures

on-the-job attitudes and behavior. The check sheet includes room for comments. The Fire

Fighters are evaluated by each other and themselves as well as by their superiors.

Bramblette (1996) is certain that the PA is a requisite of modern management. The PA is

a valuable component in the communication process between the organization’s personnel.

Communication is the key to developing managers and employers within the organization. She

sees the process of conducting the PA to be successful provided that common errors are avoided.

She also proposes that the process doesn’t dwell on the individual’s negative characteristics.

This is accomplished by avoiding the halo effect, central tendency, critical effect, the leniency

effect, and the recency effect.

In order for the process to succeed, Aurnhammer (1996) insists that a PA is developed for

each rank. The PA should include the behaviors to be exhibited, it identifies goals to reach, and

it sets benchmarks for the employee’s progress.

Sachs (1991) states that emphasis must be placed on the PA interview. First, an overall

impression is given, followed with discussion on specific areas and incidents. Promising the
 12

employee anything that cannot be delivered is prohibited. Talk centers solely about job

performance and setting objectives for the future. At this time, goals for future assignments are

made by the supervisors and employees.

A final concern is voiced by London (1997). He proposes conducting the PA on an

annual basis with informal feedback encouraged throughout the year.

The next major component is articulated by Gilley and Boughton (1996). They insist that

the PA sets the organization’s standards of minimum requirements. It assists employees in

monitoring their performance and determines when they need help or guidance. It also helps to

establish performance standards, and through doing so, employees are sure that quality is being

met.

Although the PA assists managers by identifying goals to the employees, it helps in

other ways. It enhances the quality of the organization’s decision, it enhances the quality of

individual decisions, it provides a tool for organizational changes and development, and it affects

the employees’ view of and attachment to their organization.

Another major component of the successful PA is the ability to overcome obstacles.

Haden (1995) firmly believes in this concept. He relates that organizations must address these

issues in the design of the PA. These obstacles include the primacy effect, the contrast effect, the

order effect, recency, the inadequate information error, the inflexible appraisal period, central

tendency, leniency errors, strictness errors, systematic bias, the most recent performance error

and the halo effect.

Clark (1999) identifies a separate concern. Ideally, according to Clark, supervisors

and subordinates should see each other and work together everyday. When this is not possible,

to ensure that the PA is fair, a group of people should take part in the process.
 13

Security is an obstacle for Aurnhammer (1996). Specifically, personnel records,

including the PA, should be kept in a secured area with access restricted to those who have a

need to view them.

The final major component of the successful PA is the ability to mitigate legal

concerns. Haden (1995) emphatically states that a good PA complies with Equal Employment

Opportunity Commission (EEOC) Guidelines. An organization’s reason for complying with

these guidelines is that the guidelines promote equitable and non-discriminatory employment

practices. The guidelines:

1. Give specific instructions about how to conduct the appraisal and stress the fact

that personnel not receiving training in evaluation techniques may resort to vague

and subjective criteria and bias.

2. Allow job analysis to set the performance criteria.

3. Limit measurement to job related behavior and not become an appraisal of

personality traits.

4. Should be reviewed with employees to give employees the opportunity to modify

their behavior

5. Require that appraisers must be able to observe the employee on a daily basis.

Bramblette (1996), in addressing the legal aspects of the PA, states that the reason an

organization to institutes a regular PA is to comply with government regulations. Furthermore,

the PA may be used to promote or fine the employee and to defend the organization against

employee claims.
 14

In summary, the major components found in a successful PA are the objectives of the PA,

the process the PA follows, the establishing of standards and goals, a strategy for overcoming

obstacles, and the mitigation of legal issues.

Research Question 2: The Fire Departments that responded utilize simplified and

effective components in their appraisals. Though between them components differ, several basic

similarities are present.

The Los Angeles City Fire Department (LAFD) and the San Francisco Fire Department

(SFFD) begin the appraisal process by adequately preparing their probationary recruits. Time

and effort is expended in both jurisdictions is assuring that the recruits receive documents and

instructions explaining the PA. Communication between the supervisors and the recruits is

established early in the relationship. The process of open communication between the parties

continues throughout the appraisal period.

The PA documentation contains a complete listing of the requirements and standards that

the LAFD and the SFFD have established for their personnel. The standards are clearly defined

and completely articulated.

The PA is included in the documentation. It is complete and comprehensive and includes

a thorough explanation of each category for which the individual will be evaluated. Near the end

of the PA report, a section is included for both positive and negative comments from the rater.

The PA is performed on a regular and consistent basis. The LAFD appraises its recruits

monthly with a final report due at the end of the four month period. The SFFD rates their

personnel daily and sets specific months in which they also require that a PA be submitted.
 15

Both departments emphasize the need to establish and maintain two-way dialog between

the rater and the ratee. They claim that this allows the participants to analyze deficiencies, to

modify behaviors or training and to set goals.

Both organizations inform the recruits of what needs to be done, of how that individual is

performing, if adjustments are necessary, and exactly where the recruit stands in relationship to

the organization’s requirements.

In summary, the LAFD and the SFFD completely prepare their recruits for the PA, they

establish clearly defined requirements and standards, they ensure that the PA is comprehensive,

is done on a regularly scheduled basis, encourages two-way dialog, and that the individual is

aware of how he stands in relationship to the department’s requirements.

Research Question 3: The HFD’s utilizes an appraisal process that is basic and

antiquated. The PA is designed for the convenience of the rater and can be quickly completed

with minimum effort.

The HFD recruit receives very little preparation for the appraisal process. The HFD

“Recruit Procedures Manual” is issued to each recruit at an orientation held two days prior to the

commencement of the recruit training class. A brief description of the manual and the evaluation

process is given by the RTOs.

The Department’s requirements and standards are not clearly articulated in the early

stages of the recruits training. Instead, the recruit is informed of the requirements as the recruit

proceeds through training program.

The PA report contains categories of competencies that are important to the department.

A numerical rating is given by the RTO for each category. A small space is allotted to the RTO

for his comments recommendations.

 16

Although the PA is scheduled to be performed at two week intervals, there are times

when this does not occur. Depending upon the workload of the RTO staff, a one week delay is

not unusual.

Dialog is limited. Communication usually is passed from RTO to recruit. Continuing

feedback is not solicited and further interaction on rating issues rarely occurs.

In summary, the HFD utilizes a PA that is nearing obsolescence, provides minimum

orientation to the recruit, provides only a brief articulation of the department’s requirements and

standards, adheres to an irregular evaluation schedule, limits dialog between the parties involved,

and does not include a complete appraisal of the recruit’s standing within the PA process.

Research Question 4: There are several constraints and limitations which adversely

affect the PA process.

One of the most important constraints deals with the limitations of the raters and rating

systems employed. London (1997) discusses how organizational norms may influence the extent

to which employees are concerned about rating each other fairly. In this he believes that

subordinates who get favorable feedback from their manager feel obligated to rate the manager

favorably on upward feedback.

Murphy and Cleveland (1995) use as one of the strongest reasons to explain the

prevalence of rating problems the reality that in most organizations, valued rewards are not

available to raters to encourage them to rate accurately. When an organization is unwilling to

reward good raters and sanction poor ones, this assumption is correct. They claim that leniency

may not be an error but rather a behavior that allows the rater to obtain rewards and avoid

punishments.
 17

They further state that raters are reluctant to give low ratings due to the consequences for

the ratee, the consequences for the rater, for avoidance of negative reactions and to maintain the

organization’s image. They report that the raters have no problems in distinguishing between

good and bad performers, however, most raters learn quickly that the consequences of giving low

ratings to subordinates are often negative and immediate, and rewards for accurate ratings are

often uncertain. They conclude that rating inflation is not a rater error but rather an adaptive

behavior.

This concern focuses attention upon the relationship between raters and ratees and rating

inflation. This issue is resolved through the use of multiple raters. Ilgen and Pulakos (1999),

however, claim that by gathering multiple sources of PA information, the problem of rater

accountability and how multiple rater information is applied will sometimes increase.

Swan and Marguiles (1991) discuss rating errors not just by focusing on the individuals,

but by seeing the rater within the context of the organization’s structure. They list eight common

appraisal errors. They are:

1. Inadequately defined performance standards.

2. Over emphasis on recent performance.

3. Reliance on gut feelings.

4. The employee miscomprehending the performance standards.

5. Insufficient or unclear performance documentation.

6. Inadequate time allotted for discussions.

7. Too much talking by raters.

8. Lack of any follow up.

 18

Another major factor focuses upon the organization’s purpose for the PA and the manner

in which the PA proceeds. Murphy and Cleveland (1995) view the PA as being used as a basis

for administrative decisions, feedback, developing employees, organizational planning, human

resource planning and replacement, and to safeguard organizations against discrimination

lawsuits. The PA is used to improve current performance, to set objectives and to identify

training and development needs. As such, the PA carries a tremendous amount of organizational

baggage during its implementation.

A different factor is discussed by Ilgen and Pulakos (1999). They question the method

through which the PA is established. Specifically, they believe two questions must first be

answered:

1. What aspect of the incumbent’s behavior should be represented in measures of the

degree to which they contribute to organizational objectives?

2. Finding sources who can provide information about targeted aspects of

incumbents behaviors.

Smither (1998) discusses constraints to the effective PA as a procedural concern. He see

the challenges to the PA because of the following reasons:

1. The PA usually flows from the top down.

2. In most cases, it is done only on an annual basis.

3. The standards in the PA are set unilaterally.

4. The actual meeting is short in duration.

5. There is little or no feedback throughout the year.

6. Even when the normal work environment consists of people working in teams, the

PA focuses on the individual.

 19

7. Unless poor performance is truly abysmal, it is usually ignored.

8. The actual process is very political; there is no accountability to ensure that the

raters are fair.

9. Appraisals are completely unrelated to training decisions.

10. Everyone receives the same across the board salary increase, regardless of

performance

A final constraint is discussed by Murphy and Cleveland (1995). They acknowledge that

the single greatest influence in the development and use of performance assessment in the U.S.

has been the Uniform Guidelines on Employment Selection (1978) established by the EEOC.

For many organizations, however, the guidelines’ immediate impact is an increase in the

workload which proves to be extremely burdensome to the organizations.

In summary, research found that the constraints and limitations in using parts of other

successful systems include rating problems, the methods by which the organization structurally

formulates the PA system, the organizational purpose for the PA, the actual procedural process,

and the increased burden to the organization’s workload resulting from the EEOC’s role in the

PA process.

PROCEDURES

This research project employed action research methodologies to evaluate the

components of an effective performance appraisal system for fire recruits, to discover how liked

sized fire departments utilize these components, and to determine which of these components the

HFD could utilize in the evaluation of its fire recruits.

 20

Literature Review

The literature review commenced at the Learning Resource Center of the National

Emergency Training Center in August of 1999. Additional materials were subsequently obtained

from the University of Hawaii’s Library System and the State of Hawaii’s Public Library

System. Further literature reviews were conducted at the City and County of Honolulu’s

Municipal Library and Records Center and on various internet sites. Literature pertaining to

performance appraisals conducted in other fire departments were obtained directly from those

departments.

Due to the breath and complexity of the performance appraisal issue, the literature review

is not limited to only fire service based sources. Instead, the literature review targeted textbooks,

magazines and trade journals from all sources that addressed the performance appraisal issue.

These sources are summarized and included in the Literature Review section of this project.

Definition of Terms

Central Tendency. The tendency to give mid-range ratings on all performance

dimensions regardless of actual performance.

Executive Chief Officers: The Fire Chief, Deputy Fire Chief, and the three Assistant

Chief Officers of the Honolulu Fire Department.

First Impressions. The tendency to allow one’s first impression of the ratee to influence

ratings.

Halo. The tendency to allow perceptions of one performance dimension influence ratings

of other, unrelated performance dimensions.

 21

Lag Relationships. The process of filling out performance appraisals to justify decisions

that have already been made or to make it easier to implement a course of action already decided

upon.

Leniency. The tendency to give overly favorable ratings on all performance dimensions

regardless of actual performance.

Rating Inflation. The tendency to give a rating higher than a rating that is deserved.

Recency Effect. The tendency to allow a recent incident to influence judgments of

performance dimensions for the entire performance period.

Recruit Training Officers (RTOs): Personnel of the Honolulu Fire Department charged

with the responsibility of training the Fire Fighter Recruits.

Severity. The tendency to give overly negative ratings on all performance dimensions

regardless of actual performance.

Similarity. The tendency to give overly favorable ratings to ratees who are similar to the

rater on characteristics that are unrelated to performance.

Assumptions

In completion of this research project, three assumptions are made. First, that there does

not exist a document, or documents, from the HFD that explain the formulation and

implementation of the current fire recruit performance appraisal system. Second, that the authors

cited in the literature review performed proper and competent research. Third, that the

performance appraisal documents submitted by the other fire departments are true and accurate

representations of that department’s fire recruit performance evaluation system.

Limitations
 22

The limitations that affected this research project included the lack of an historical

perspective of the HFD fire recruit performance appraisal, the currency of the available

literature, the limited response from fire departments, and time constraints of the Applied

Research Program.

The fire recruit performance appraisal form currently used by the HFD remains shrouded

in mystery. The form exists; however, where it came from, who created it, and the date of its

implementation cannot be answered. Documentation addressing these questions does not exist.

Although the study of performance appraisals has received considerable attention, recent

studies relate how researchers based the foundation for their conclusions on primary research

that was conducted more than five years ago. As such, the field of available research became

limited in the effort to utilize current sources.

The HFD is a member of the Training Resource and Data Exchange (TRADE),

Region IX. Recruit performance appraisal documents were requested from twelve like-sized

departments within the region. Five responded, from which only two departments provided

documentation that, for various reasons, are applied in this project.

The time constraint imposed by the Fire Executive Officer Program requiring that the

project be completed within six months does not allow for a more complete search for literature

outside of the more readily available sources. Additional time would have allowed for a more

complete follow-up with the fire departments who failed to respond to the request for assistance

or to replace the non-respondents with other departments. It is also apparent that the

performance appraisal report is only one component in the entire recruit appraisal process. Time

constraints do not allow for an analysis of the entire process, but only this portion of it.
 23

RESULTS

Research Question 1: Results found that there is payoff in using a good PA. It gives the

manager a better understanding of the employee, it provides an opportunity to team build and

develop leadership skill, it tells the employee where he stands, it rewards for effective

performance, it gives a clear understanding of career path and employment development , and it

makes efficient use of resources (Aurnhammer, 1996).

Along with a distinctive payoff, another important result is in the actual process itself; the

supervisor and employee strengthen their relationship and become two adults working toward a

common, agreed upon goal (Sachs, 1991). The supervisor and employee benefit further from a

good PA because it gives a rating of the quality of the work performed, it helps to set new

performance objectives and goals, and it gives positive feedback.

The successful PA, by measuring human resource management techniques, focuses on

the amount and quality of employee productivity (Haden, 1995). This allows for employee

productivity to be measured and in turn can be used to determine salary, rank, and responsibility.

There is a question of dealing with the bias in the instruments used to assess knowledge and

skills.

The conducting of a successful PA impacts the development of employees and

strengthens of the organization’s culture (Devill, 1995). Organizations without an effective

appraisal system are defaulting on basic business principles and hence not developing a positive

organizational culture. The culture and the spirit of an organization are strongly influenced by

whether a manger reinforces good performance or simply assumes that it is “the employees job”.

Besides strengthening the organization’s culture, the successful PA serves as a vehicle for

motivating employees (Devill, 1995). The best way to motivate employees is to raise their level
 24

of personal and professional self esteem. Uplifting an employee’s self esteem through the PA

process builds a culture that achieves excellence. As such, the PA not only gives employees

needed feedback for organizational clarity and employee performance, but it also increases

motivation among employees and managers.

A fostering of communication results from the PA. Opening avenues of communication

through the PA allows management to keep the golden rule to “treat others in the same manner

that we would like to be treated”, yet management retains its right to point out incidents in which

goals haven’t been met (Aurnhammer, 1996). Also, communicating to the employee what is to

be evaluated in the process eliminates subjectivity.

Besides enhancing communication between managers and employees, the effective PA

improves employee performance (Devill, 1995). It assists managers and employees in focusing

on the goal at hand and emphasizes what has to be completed, by who, and by what time. The

PA serves as a reference for employee performance and as an overall organizational performance

baseline; it assists the organization in achieving objectives within a specific time frame.

The PA provides both the supervisor and the employee with a collaborative planning

session (Sachs, 1991). From this, they can take a look at past and current performances and

make plans for the future. Even better results are obtained if a follow through process for the

employee on career plans is added by using the employee’s declared career goals to evaluate

progress. Also, follow up is important, not only for the good performer, but especially with the

poor performer.

To the issue of employee enhancement is added the claim that the ideal PA also lets the

employee know how he stands (Bramblette, 1996). The PA gives guidance to the employee by

defining what is expected and it assists in defining specific areas of improvement of knowledge
 25

and skills. By focusing on the good, it strengthens the relationship between management and the

employee and it gives the employee a chance for expansion. In the process, it also identifies the

individual’s potential for advancement.

In the case where the PA meeting is viewed as a problem solving session to better the

situation, the result will have satisfied, motivated and productive employees. Otherwise, should

the PA be seen as a fault finding of past failures, then frustrated, nonproductive employees will

result.

Even if there should exist negative connotations for the PA, these too can be overcome

(Aurnhammer, 1996). To do so, it must be ensured that the process is ongoing; both positive and

negative aspects are included, and it is done in an atmosphere that encourages open dialogue.

When addressing the issue of improper rating scales, as long as the rating scales are based

on job analysis with input from people doing the job, then the rating scale will be clearest and

most acceptable to both raters and recipients (London, 1997). The fewest rating errors result

when scale formats include concrete behavioral descriptions that are understandable, observable,

and important to the job. If raters are willing to provide accurate ratings and accurate feedback,

the PA becomes a very powerful tool (Murphy and Cleveland, 1995).

Finally, sources of appraisals should include peers, direct reports, customers, supervisors

and self appraisals (Smither, 1998).

Research Question 2: Results found that the LAFD and the SFFD adequately prepare

their probationary recruits for the PA process. This is accomplished by providing sufficient time

and effort to the problem, by encouraging and maintaining open lines of communication between

raters and ratees, and by providing the recruits with complete documentation and instructions.
 26

Furthermore, the results found that the documents contain a complete listing of the

requirements and standards which are completely defined. Included in the documentation is the

PA form. The PA form includes an explanation of the evaluation’s categories. The form allows

for comments by the rater, both positive and negative. The PA is performed according to a

required schedule. The PA process encourages dialog between all parties involved.

As a result, the recruit is aware of the standards required by the respective departments,

understands what work must be performed, knows how he stands in relationship to the

standards, and is clear as to the behavior modification that is necessary on his part.

Research Question 3: Results found that the HFD has major deficiencies in its PA

process. Very little time and effort is focused on the PA. Recruits are given only a brief

overview; documentation from the recruit manual is brief and in need of revision.

Results also show that standards and requirements are not covered in sufficient detail as

to be considered adequate. The PA is performed on an inconsistent basis. Communication is

one-sided and dialog constrained. Finally, the recruit is uncertain as to his standing until after

the PA is performed.

Research Question 4: Results found that there are several constraints and limitations that

adversely affect the PA process.

Rating inflation is a primary issue. In fact, rating inflation is the norm in the United

States (Murphy and Cleveland, 1995). Most employees routinely receive positive appraisals

(even when their performance is poor) which in turn makes it difficult to fire a poor performer.

Rating inflation is due to the belief that it makes no sense to give low ratings except in extreme

circumstances. On the other hand, it makes perfect sense to give high ratings, regardless of the

true level of performance.

 27

Results also show that limitations will occur due to dramatic changes in the structure of

work and organizational forms that are taking place (Ilgen and Pulakos, 1999). Organizations

are becoming less bureaucratic, less mechanistic, and more organic. They will be flatter with

fewer layers of management. Decision making will fall to lower levels to allow for local control

and flexibility. The focus on appraisals is being changed because of the changing of roles. As an

example, the role of the supervisor as an employee appraiser may need to be reconceptualized

and reoperationalized to deal with the changing nature of work. Job responsibilities in the future

will be less rigidly programmed and more flexible. In addition, increased demand from external

customers will require closer monitoring and tailoring of services.

With the changing nature of work, a result is that the PA, when designed as intended,

serves as a control system in organizations (Murphy and Cleveland, 1995). An organization’s

top management can influence employees by establishing the type, format, frequency, and

purpose of the PA system. On the other end of the spectrum, when there is no apparent link

between ratings and administrative outcomes, the supervisor will have less power over the

subordinate.

Results reveal that the PA becomes an important legal issue when it is used in making

personnel decisions (Murphy and Cleveland, 1995). In addition, the appraisal process may

discriminate among employees with regards to merit, performance and the potential for

promotion (Devill, 1995). That is why the appraisal systems must be especially sensitive to

fairness (Smither, 1998). Not just fairness in terms of the legal issues involved, but to the

employees’ perception of fairness.

On a positive note, results show that the primary popularity in using the PA is due to the

need of managers to adjust to ever changing business environments (Tornow and London, 1998).
 28

It forces the individuals to be responsible for their continued ability to add value to the

organization. In this context, the PA becomes extremely valuable when individuals plateau in

their work because it gives them something to focus on (Sachs, 1991). Even if the employee

stays in one place, the PA reinforces the belief that the work he performs is essential.

Results show that fire departments are a special case (Sachs, 1991). In private industry,

unlike the Fire Departments, the PA determines raises and promotions. However, aside from the

pay and promotions issue, fire departments and private industry are seeing a trend in which their

personnel are changing teams more frequently. In this environment, individuals will have

considerable latitude over what they do to contribute to team goals. The team members, instead

of relying on direct supervision as in traditional bureaucracies, will share common goals and

enforce patterns of individual behavior within the team out of a sense of mutual accountability

that will reinforce the notion that they are their own internal customers.

Finally, results provide the newly created Recruit Performance Appraisal Report for the

Honolulu Fire Department. The Appraisal Policy, containing the purpose statement, goals, and

process, along with the Appraisal Form is included in its entirety in the Appendix.

DISCUSSION

Research Question 1: The oldest and most predominant purpose of the appraisal is for

making administrative decisions (Murphy and Cleveland, 1995). More recently, however, it is

used as a tool for employee development and feedback. In this form, the PA is a tool for the

managing rather than the measuring of subordinates. As a tool, it is used to motivate, direct, and

develop subordinates. This concept is contrary to the previously held delusion that the PA is
 29

used to give a good employee a pat on the back, or to put pressure on a bad employee to improve

performance (Sachs,1991).

Results showed that there are distinct payoffs for using a good PA. Likewise,

organizations and managers who do not follow the basic business principles of rewarding and

promoting the most qualified employee will have little support in the organization of the future

(Devill, 1995).

The basic components of a successful PA include clearly defined objectives for the PA, a

complete description of the process the PA will follow, the establishment of the organization’s

standards and goals, a strategy for overcoming obstacles, and a method through which legal

issues are identified and mitigated.

Of major concern is the problem with the rating methods used for the PA. To remedy

this problem, a set criteria must be established and discussed with all parties involved prior to

commencement of the process. In establishing the criteria, four question should be addressed

(Swan and Marguiles, 1991). First, is the information measured by the PA job relevant and is it

measured accurately? Second, does it provide a fair and accurate basis for comparing different

individuals? Third, is it reliable? Fourth, is the information gathered useful?

Organizations are furthered hampered by the fact that the process may take several years

before raters and employees become comfortable with the system. The organization, however,

should focus on the long term goal and deal with the short term challenges. In this interim

period, ratingless appraisals are used to avoid the defensiveness that often accompanies grading.

This researcher concurs with the findings of the authors concerning the major

components of the effective PA. Organizations that utilize these components are able to reach

their objectives and better their employees with minimum stress to the organization.
 30

This researcher also agrees with the findings of the authors that it is natural for people to

want to better themselves; to earn the intrinsic rewards of doing well and improving. As a result,

even the best performers should be give new goals to strive for or their interest in the job may

wane (Sachs, 1991).

Research Question 2: Recruits from the LAFD and the SFFD are competently prepared

for the PA. Through use of the PA, the recruit is aware of the standards and goals of the

respective departments, understands what work must be performed, knows how he stands in

relationship to the standards, and is clear as to required behaviors.

This researcher finds the PA systems utilized by the LAFD and the SFFD to be

professional and complete. This researcher is certain that those PA systems properly service

their organizations’ goals and the needs of the recruits.

Research Question 3: The PA currently used by the HFD is grossly deficient. The

portion of the PA that uses the generic Civil Service document is its worse component. That

generic form, the same one that is used for all employees of governmental entities, tends to lead

to “pencil whipping” because it doesn’t specifically address what’s significant to fires service

personnel for exceptional performance and provides little useful feedback (Aurnhammer, 1996).

This researcher agrees that more time and effort must be dedicated to the PA, that recruits

must be given a complete overview of the process, that the documents must be revised, that the

actual evaluation be performed on a timely basis, that communication between the rater and the

ratee be improved, and that the recruit is made aware of his standing with the department.

For the authors and this researcher, it is apparent that fire fighters desire to avoid conflict

in their professional lives. However, when the avoidance of conflict impacts the supervisor’s
 31

evaluation of employees, by practicing avoidance, the supervisor is doing a dis-service to the

employee and an injustice to the department, the other employees and the community.

Research Question 4: The changing nature of work is changing the way performance is

viewed and it is making it considerably more difficult to define, assess, predict and affect

performance (Ilgen and Pulakos, 1999). The key issues are fairness, politics, feedback, rater

training, appraising of supervisors, and the determination of what actually is measured (Smither,

1998). The PA, therefore, is shaped by a careful analysis of social conditions and the cultural

values of the organization.

Following the correct procedures is also emphasized (Bramblette, 1996). The PA

adversely affects an organization if it is late, if prior notification to the employee is not adequate,

if review of the job requirements and performance standards are not provided, if the strengths of

the employee are not noted, and if a plan for improvement is not included.

This researcher concurs with the finding of the authors, especially when focus is placed

upon the harm that an improperly implemented PA brings to an organization. Even if it is done

well, the PA can hurt the organization by overemphasizing the individual and under emphasizing

the system and team (Murphy and Cleveland, 1995). It often sends mixed messages as to what

is most and least important to job performance.

Finally, the PA is often a source of discontent for both raters and ratees. In fact,

subordinates have a perception that the rater is to blame if they have a negative rating (Murphy

and Cleveland, 1995).

 32

RECOMMENDATIONS

Research Question 1 asks what are the components of an effective performance appraisal

system. Results found that the effective PA includes the objectives of the PA, the process the PA

follows, the establishing of standards and goals, a strategy for overcoming obstacles, and the

mitigation of legal issues. This researcher recommends that these components be included in the

development of a PA for the HFD’s Fire Fighter Recruits.

Research Question 2 inquired into how liked sized departments utilize these components.

Results found that the LAFD and the SFFD completely prepare their recruits for the PA, they

establish clearly defined requirements and standards, they ensure that the PA is comprehensive in

content, the PA is done on a regularly scheduled basis, two-way dialog is encouraged, and the

individual is certain of his standing with the department. This researcher recommends that the

HFD incorporate into the PA for its Fire Fighter Recruits these components from the LAFD and

the SFFD.

Research Question 3 inquired as to what components could the HFD use in its appraisals.

Results found that the HFD’s PA is obsolete, it provides a minimum amount of information to

the recruit, it does not comprehensively cover the department’s requirements and standards, it is

not regularly delivered, it limits dialog and interaction, and it creates uncertainty for the recruit.

This researcher recommends that the HFD’s current PA for Fire Fighter Recruits be discarded

and that a new PA replace it.

Research Question 4 inquired into the limitations and constraints that are present in other

successful systems. Results found that the limitations/constraints in other systems include rating

problems, the methods by which the organization structurally formulates the PA system, the

organizational purpose for the PA, the actual procedural process, and the increased burden to the
 33

organization’s workload resulting from EEOC’s guidelines. This researcher recommends that

these limitations/constraints are accounted for prior to the formulation of the HFD’s PA for Fire

Fighter Recruits. Furthermore, this accounting should continue during the implementation and

evaluation phases of the PA.

 34

REFERENCES

Aurnhammer, Thomas W. (1996, August) Personnel evaluations – are we being

effective? Fire Chief Magazine. 102-106.

Bramblette, Pamela Marshall (1996, May) Development through appraisal interviews.

Responder Magazine. 4, 48.

Clark, Scott (1999, October) Performance evaluations for firefighters. The Voice

Magazine. 12-13.

Devill, Dan (1995, August) Performance evaluations. The Voice Magazine. 14-16.

Gilley, J. W., & Boughton, N. W. (1996). Stop managing, start coaching. Chicago, IL:

Irwin Professional Publishing.

Haden, D., & Wells, S. (1995, May/June) Evaluation bias in performance appraisals. The

Canadian Firefighter. 38-40.

Ilgen, D., & Pulakos, E. (1999) The changing nature of performance. San Francisco,

CA: Jossey-Bass Publishers.

London, Manuel (1997) Job feedback: giving, seeking, and using feedback for

performance improvement. Mahwah, NJ: Lawrence Erlbaum Associates.

Los Angeles Fire Department (1999) Field training and evaluation program. Los

Angeles, CA: Author.

Murphy, K. R., & Cleveland, J. N. (1995) Understanding performance appraisal.

Thousand Oaks, CA: Sage Publications.

Sachs. Randi Toler (1991) Productive performance appraisals. New York, NY:

American Management Association.

 35

San Francisco Fire Department (1997) Field training course for probationary

firefighters. San Francisco, CA: Author.

Smither, James W. (1998) Performance appraisal. San Francisco, CA: Jossey-Bass

Publishers.

Swan, W. S., & Margulies, P. (1991) How to do a superior performance appraisal. New

York, NY: John Wiley & Sons.

Tornow, W. W., & London, Manuel (1998) Maximizing the value of 360-degree

feedback. San Francisco, CA: Jossey-Bass Publishers.

Appendices Not Included. Please visit the Learning Resource Center on the
Web at http://www.lrc.fema.gov/ to learn how to obtain this report in its entirety
through Interlibrary Loan.