Intensive and Critical Care Nursing (2005) 21, 284—301

ORIGINAL ARTICLE

A human factors engineering conceptual

framework of nursing workload and patient safety
in intensive care units
Pascale Carayon ∗, Ayşe P. Gürses

Systems Engineering Initiative for Patient Safety, Center for Quality and Productivity Improvement and
Department of Industrial and Systems Engineering, University of Wisconsin-Madison, 610 Walnut Street
575 WARF, Madison, WI 53726, USA

Received 26 August 2004 ; received in revised form 10 December 2004; accepted 17 December 2004

KEYWORDS
Performance obstacles;
Nursing workload;
Quality of working life;
Patient safety;
Intensive care units
Summary In this paper, we review the literature on nursing workload in inten-
sive care units (ICUs) and its impact on patient safety and quality of working life of
nurses. We then propose a conceptual framework of ICU nursing workload that de-
fines causes, consequences and outcomes of workload. We identified four levels of
nursing workload (ICU/unit level, job level, patient level, and situation level), and
discuss measures associated with each of the four levels. A micro-level approach to
ICU nursing workload at the situation level is proposed and recommended in order
to reduce workload and mitigate its negative impact. Performance obstacles are
conceptualized as causes of ICU nursing workload at the situation level.
© 2005 Elsevier Ltd. All rights reserved.

Introduction by over 250% and now accounts for approximately

Intensive care constitutes a significant portion of
the American health care system. The number of
intensive care unit (ICUs) in the US is approximately
6000 (Angus et al., 2000). Every day, 55,000 pa-
tients are cared for in these 6000 ICUs (Halpern
et al., 1994). With a declining number of overall
acute care hospital beds over the past 20 years,
the number of intensive care beds has increased
∗ Corresponding author. Tel.: +1 608 2650503;
fax: +1 608 2631425.
E-mail addresses: carayon@engr.wisc.edu (P. Carayon),
apgurses@wisc.edu (A.P. Gürses).
10% of all hospital beds (American Hospital Associ-
ation, 1995; Lustbader and Fein, 2000). This per-
centage is expected to increase as the population
ages (Groeger et al., 1992). It has been estimated
that ICUs account for between 20 and 34% of acute
care hospitals in the US; this is double that of
other developed countries (Office of Technology As-
sessment, 1984; Jacobs and Noseworthy, 1990; So-
ciety of Critical Care Medicine, 1992; Cohen and
Chalfin, 1994; Chalfin et al., 1995). The cumula-
tive effect of this costly resource results in 1—1.5%
of the gross national product devoted to intensive
care medicine (Wagner et al., 1983). According to
Miranda et al. (2003), nursing staff budget consti-

0964-3397/$ — see front matter © 2005 Elsevier Ltd. All rights reserved.
doi:10.1016/j.iccn.2004.12.003
Human factors engineering conceptual framework
tutes approximately 50% of the total costs in an
ICU.
Every year, approximately 400,000 to 500,000
patients die in ICUs in the US (Angus et al., 1996).
Morbidity and mortality rates in ICUs vary widely
between institutions (Pronovost et al., 1999). Based
on data from England, Wales, and Northern Ireland,
the overall hospital mortality after adult intensive
care was estimated as 30.8%, ranging from 17.7 to
48.7% across hospitals (Intensive Care National Au-
dit and Research Center, 2001). Although some of
this variation is due to case mix (i.e. age, acute
severity, co morbidity, surgical status and reason for
admission), a significant variation in hospital mor-
tality is observed even after adjusting for differ-
ences in case mix (Knaus et al., 1986; Rowan et al.,
1993; Carmel and Rowan, 2001). This wide variation
in the organization and delivery of intensive care
makes it important to understand the impact of or-
ganizational factors on ICU outcomes. One cause of
this variation may be related to the ‘micro’ system
of the ICUs (Nelson et al., 2002). Nursing workload
is one important aspect of the design of a clinical
micro system.
There is substantial interest on the part of health
care purchasers for quality and safety in ICUs.
This is best illustrated by the Leapfrog Group’s
initiative. The Leapfrog Group is a coalition of
more than 140 public and private organizations
that provide health care benefits to approximately
34 million health consumers in all 50 states. It
was founded by the Business Roundtable, a na-
tional association of Fortune 500 CEOs, ‘‘in or-
der to help save lives and reduce preventable
medical mistakes by mobilizing employer purchas-
ing power to initiate breakthrough improvements
in the safety of health care and by giving con-
sumers information to make more informed hos-
pital choices’’ (http://www.leapfroggroup.com).
The Leapfrog Group has identified three initia-
tives that are assumed to improve patient safety.
One of these initiatives is specifically related
to intensive care: ‘‘Hospital ICU care should be
managed or co-managed by a physician certi-
fied (or eligible for certification) in critical care
medicine’’ (http://www.leapfroggroup.com). This
explicit focus of the Leapfrog Group on inten-
sive care demonstrates the importance given by
American businesses on improving ICU patient
care.
In this paper, we examine studies of patient
safety in ICUs and conclude that nursing work-
load is a major contributor to patient safety in
ICUs. We conducted a literature search of stud-
ies published after 1970 in MEDLINE and CINAHL
databases. We used the following keywords: nurs-
285
ing workload, patient safety, quality of working life,
job stressors, ICU nurse, intensive care unit, and
nurse/patient ratio. Based on the literature review,
we then propose a conceptualization of nursing
workload that includes various levels and measures.
Finally, we discuss possible causes of nursing work-
load in ICUs (performance obstacles) that can be
identified by studying workload at the situation
level.
Patient safety in ICUs
Research on patient safety and medical errors
shows that errors and the adverse events that
may result from those errors are frequent in ICUs
(Abramson et al., 1980; Merill and Boisaubin, 1981;
Donchin et al., 1995). Donchin et al. (1995) con-
ducted a study to evaluate human errors in a
medical—surgical ICU by using self-reports and di-
rect observations. They found that an estimated
number of 1.7 errors per patient per day occur in
ICUs. In this study, a human error was defined as a
deviation from standard conduct, as well as addi-
tion or omission of actions related to standard op-
erational instructions or routines of the unit. Ac-
cording to Donchin et al. (1995), patients in an ICU
receive on average 178 activities per day, and 1% of
these activities involve errors. Twenty-nine percent
of these errors have the potential to cause signifi-
cant deterioration in patient’s condition or death.
Bracco et al. (2000) conducted a prospective ob-
servational study of consecutive patients admitted
over 1 year to an 11-bed multidisciplinary ICU in a
non-university teaching hospital in order to identify
critical incidents and associated risk factors. A total
number of 1024 patients were included in the study.
These patients required 2801 days of treatment in
the ICU. The authors developed a list of 105 critical
incidents, based on published studies (Vitez, 1990;
Brennan et al., 1991; Giraud et al., 1993; Fasting
and Gisvold, 1996). After a critical incident was
identified, it was analyzed and its cause was classi-
fied as equipment-, patient- or human-related. If
the cause was related to a technical equipment
failure, it was classified as equipment-related. If
the cause was related to a human error, such as a
health care provider’s actions not going as intended
or the intended action not being appropriate, the
critical incident was identified as human-related. If
none of the above was identified, the critical inci-
dent was classified as patient-related. The number
of critical incidents detected during the study pe-
riod (1 year) was 777. Thirty-one percent of these
incidents were human-related, 2% were equipment-
related, and 67% were patient-related. The authors
286
also found that human errors prolonged ICU stay by
425 patient days over a 1-year period.
Several studies have examined various types of
errors in ICUs. Giraud et al. (1993) conducted a
prospective, observational study to examine ia-
trogenic complications and to identify the factors
that favor the occurrence of these complications.
The authors define an iatrogenic complication
as ‘‘an adverse event that was independent of
the patient’s underlying disease’’. A total of 382
patients for 400 consecutive admissions provided
data. Thirty-one percent of the admissions had
iatrogenic complications. The risk of mortality
among patients with iatrogenic complications
was significantly higher than the risk of mortality
among patients without iatrogenic complications.
Forty-four percent of all iatrogenic complications
were associated with either human errors (insuf-
ficient surveillance, inadequate experience) or
equipment-related problems (equipment failure,
inadequate equipment). The relationship of the
occurrence of iatrogenic complications with prog-
nostic indices, nursing workload, and length of stay
was examined. Patients with two or more organ
failures and patients older than 65 years were found
to have a higher risk of developing major iatrogenic
complications. Nursing workload was measured
with the OMEGA system. The OMEGA system mea-
sures nursing workload based on 47 diagnostic and
therapeutic items (Le Gall et al., 1990). In addition
to the OMEGA system, the authors measured nursing
workload subjectively in order to take into account
non-objectively measurable factors, such as expe-
rience, fatigue, and availability of help from others
in the unit. At the end of each 12-h shift, nurses
were asked to rate the level of their workload using
the following four categories: 1 = minor, 2 = normal,
3 = heavy, 4 = excessive. Results showed that nurses
rated workload as ‘‘excessive’’ 302 times (4.7%),
‘‘heavy’’ 1494 times (23.3%), ‘‘normal’’ 4263 times
(66.5%) and ‘‘minor’’ 349 times (5.4%). The non-
parametric Kruskal—Wallis test was used to test the
association between the occurrence of iatrogenic
complications, and nursing workload and length of
stay. Results revealed that the frequency of iatro-
genic complications was positively associated with
both the subjectively rated nursing workload and
the OMEGA scores. Length of stay was also found to
be significantly higher for patients who developed
iatrogenic complications than for patients who did
not. Based on their findings, the authors recom-
mended improving the organization of workload
and training, and increasing the use of noninvasive
monitoring.
Tissot et al. (1999) detected 132 medication
preparation and administration errors in a to-
P. Carayon, A.P. Gürses
tal of 2009 observed events (6.6%) in a medi-
cal ICU. Tissot and colleagues used an adaptation
of the ‘‘disguised-observation technique’’ devel-
oped by Barker and McConnell (1962). For 6 h ev-
ery day, two observers (pharmacy residents) ob-
served two randomly selected nurses giving med-
ication to patients. Based on these observations,
the researchers classified errors into 6 categories:
wrong drug preparation, dose error, wrong adminis-
tration technique, physicochemical incompatibility
errors occurring at simultaneous administration of
two or more medicines via the same route, wrong
rate, and wrong time. They identified 10 possible
causes of medication errors, including high nursing
workload and fatigue.
Research on patient safety and medical errors
in ICUs shows that errors are frequent in ICUs and
that they can result in negative patient outcomes.
System-related human errors seem to be particu-
larly prevalent in ICUs. Several factors may lead to
poor outcomes in ICUs, such as inadequate commu-
nication between nurses and physicians (Donchin et
al., 1995), impaired access to information (Leape
et al., 1995), and high nursing workload (Giraud et
al., 1993; Tissot et al., 1999). This paper focuses
on nursing workload and its impact on quality of
working life of nurses and quality and safety of care
provided to ICU patients.
Workload in ICUs
ICU practice is filled with high workload situations
(Crickmore, 1987; Malacrida et al., 1991; Oates and
Oates, 1996). Nurses must continuously respond to
the needs of patients and families, and routinely
interact with the most intense emotional aspects of
life. Research shows that nursing workload is one of
the most important determinants of patient safety
and quality of care in ICUs.
An analysis of the Australian Incident Monitor-
ing Study data for ICUs shows that a nursing staff
shortage may lead to compromised quality of care
(Beckmann et al., 1998). The two main causes of
nursing staff shortage were inappropriate staffing
for current patient load (81% of the incidents)
and inability to respond to increased unit activ-
ity (19% of the incidents). Insufficient nursing staff
was found to be associated with the occurrence
of the following incidents: drug administration or
documentation problems, inadequate patient su-
pervision, incorrect ventilator or equipment setup,
and self-extubation. Undesirable patient outcomes
associated with insufficient nursing staff include
major physiological change, patient or relative dis-
satisfaction, and physical injury. An undesirable pa-
Human factors engineering conceptual framework
tient outcome was reported in 37% of the incidents
associated with insufficient nursing staff.
Tarnow-Mordi et al. (2000) examined the rela-
tionship between mortality rates and the workload
of hospital staff in one adult ICU in the United
Kingdom. Measures of workload for a specific pa-
tient’s stay included occupancy per shift, peak oc-
cupancy, ICU nursing requirement per shift (i.e. the
highest number of nurses required for the ICU ac-
cording to the recommendations of the UK Inten-
sive Care Society; i.e. 0.5 nurse per patient per
shift for patients who are spontaneously breath-
ing and need simple monitoring only), ICU nursing
requirement during patient’s first shift, ICU occu-
pancy during patient’s first shift, the ratio of oc-
cupied to appropriately staffed beds per shift, and
the ICU nursing requirement per occupied bed per
shift. Results demonstrated that patients exposed
to high ICU workload were more likely to die than
those exposed to low workload. The three measures
of workload most strongly associated with mortality
were peak occupancy, average nursing requirement
per occupied bed per shift, and the ratio of occu-
pied to appropriately staffed beds. Explanations for
the association between high workload and mor-
tality include insufficient time for clinical proce-
dures to be done appropriately, inadequate training
or supervision, errors, overcrowding and conse-
quently nosocomial infections, limited availability
of equipment, and premature discharge from the
ICU.
The UK Neonatal Staffing Study Group (2002)
conducted a study to determine whether work-
load was associated with risk-adjusted outcomes in
neonatal intensive care units (NICUs). They studied
13,515 infants consecutively admitted to 54 ran-
domly selected NICUs. Results showed that mor-
tality was significantly positively associated with
the percentage maximum occupancy on admission
and negatively associated with the nurse-to-infant
ratio in all types of NICUs. As the maximum oc-
cupancy in the unit increases, nurse staffing be-
comes more inadequate, which, in turn, results
in an increase in the risk of mortality. Based on
this study, the authors suggest that service in
NICUs should be improved by reducing nursing work-
load.
Research also shows that workload is one of
the most important job stressors among ICU nurses
(Crickmore, 1987; Malacrida et al., 1991; Oates and
Oates, 1996). The practice of medicine is inherently
stressful. For example, Malacrida et al. (1991) stud-
ied the quality and frequencies of stressors in an
ICU environment by using two different data collec-
tion methods: the Nursing Stress Scale (NSS) ques-
tionnaire, and a computer-aided self-observation
287
method developed by the authors for this study.
In the computer-aided self-observation method, 16
ICU nurses were asked to record their experiences
on a computer placed in the ICU, immediately af-
ter having experienced a stressful event. These
same 16 ICU nurses and 31 other nurses working
at the same hospital were then asked to fill out
the NSS questionnaire. Results obtained by both
data collection methods showed that the most im-
portant stressors for nurses were related to deal-
ing with death and high workload. In this study,
staffing and scheduling problems, not having suf-
ficient time to complete nursing tasks and not hav-
ing sufficient time to provide emotional support to
patients were all considered as workload-related
problems.
In addition to research on working conditions and
stress, there is considerable evidence for the re-
lationship between nursing working conditions and
job satisfaction (Bratt et al., 2000; Darvas and
Hawkins, 2002). Behavioral consequences of job
dissatisfaction in nursing, such as low morale, ab-
senteeism, turnover, and poor job performance,
can potentially threaten patient care quality and
organizational effectiveness (Cavanagh, 1992). Sev-
eral nursing studies have examined the outcomes
or consequences of job satisfaction. Studies have
found positive associations between job satisfac-
tion and job performance (McCloskey and Mc-
Cain, 1987), and patient satisfaction and quality
of care (Tarnowski-Goodell and Van Ess Coeling,
1994).
High workload seems to be related to sub-
optimal patient care (Keijsers et al., 1995; Aiken
et al., 2002), seems to influence care provider’s
decision to perform various procedures (Griffith et
al., 1999) and may lead to reduced patient satis-
faction (Anderson and Maloney, 1998). High work-
load may also lead to poor nurse-patient com-
munication (Llenore and Ogle, 1999), impaired
nurse—physician collaboration (Baggs et al., 1999),
nurse burnout and job dissatisfaction (Aiken et al.,
2002).
Evans and Carlson (1992) suggest strate-
gies such as improving working conditions and
nurse—physician collaboration to reduce workload
and to deal with the nursing shortage crisis. In this
paper, we present a framework for conceptualizing
nursing workload, its work-related causes, its
effects on the quality of working life of nurses,
the quality and safety of care they provide, and
associated economic outcomes. Our ultimate
goal is to provide a human factors engineering
framework for identifying work system factors that
can be modified to reduce workload in order to
improve quality of working life of ICU nurses and
288
improve the quality and safety of care provided to
ICU patients.
Conceptual framework of ICU nursing
workload
Fig. 1 shows the proposed conceptual framework
of ICU nursing workload. Given our human fac-
tors engineering approach, one can define vari-
ous work-related factors that contribute to nursing
workload (i.e. causes). We also define different lev-
els of nursing workload (see next section). Nurs-
ing workload, in turn, can have a number of
intermediate consequences (such as insufficient
compliance of nurses with hand hygiene poli-
cies or inadequate patient supervision), which can
lead to various outcomes. The consequences of
nursing workload provide mechanisms for under-
standing and specifying how nursing workload can
contribute to various outcomes. The consequences
mediate the relationship between nursing work-
load and outcomes. Outcomes include patient out-
comes (quality of care and patient safety), family
outcomes (satisfaction with the care provided),
nurse outcomes (quality of working life), and eco-
nomic outcomes. The conceptual framework de-
fines a ‘chain of events’ from causes to workload,
to consequences and outcomes. Table 1 provides
a summary of the research evidence that pro-
vides support for various aspects of the conceptual
framework.
Levels and measures of nursing
workload in ICUs
Studies have conceptualized nursing workload in
ICUs differently and have used various instruments
to measure nursing workload. We can categorize
the workload measures into four groups: (1) work-
load measures at the ICU (unit) level, (2) workload
at the job level, (3) workload at the patient level,
and (4) workload at the situation level (see Table 2).
These measures can be organized into a hierar-
chy of measures. In an ICU (unit level), there are
various nursing jobs. When performing their job,
Figure 1 Conceptual framework of ICU nursing workload.
P. Carayon, A.P. Gürses
nurses encounter various situations and various pa-
tients. Therefore, the situation- and patient-levels
of workload are embedded in the job-level of work-
load, and the job-level of workload is embedded in
the unit-level of workload. In addition, measures
at a lower level, e.g., patient level, can be aggre-
gated to measure workload at a higher level, e.g.,
unit level. We provide more information about each
of these four workload measures below.
Workload at the ICU (unit) level
The most widely used measure to assess work-
load at the ICU level is the nurse/patient ratio.
Studies have examined the relationship between
nurse/patient ratio and clinical and economic out-
comes of abdominal aortic surgery (Pronovost et
al., 1999), complications and resource use after
hepatectomy (Dimick et al., 2001), complications
and resource use among patients who had oe-
sophagectomy (Amaravadi et al., 2000), and infec-
tion rates (Archibald et al., 1997; Grundmann et
al., 2002).
ICU-level measures of workload include a modi-
fied nurse/patient ratio measure that considers ed-
ucation and tenure of nurses. For example, Robert
et al. (2000) conducted a nested case-control study
to identify the risk factors associated with the ac-
quisition of nosocomial primary bloodstream in-
fections (BSIs) in a surgical intensive care unit
(SICU). The authors compared 28 case-patients (pa-
tients with BSI) with 99 randomly selected control-
patients who stayed in the SICU for at least 3 days.
The study failed to show any significant associ-
ation between the overall nurse/patient ratio in
the unit and the acquisition of BSIs. The authors
also studied nursing staff composition as a possible
risk factor for the acquisition of BSIs. They defined
‘‘regular staff’’ as nurses permanently assigned to
the SICU, and ‘‘pool staff’’ as agency nurses or
nurses from the hospital pool service. The authors
then compared nursing staff composition (regular
nurse/patient ratio versus pool nurse/patient ratio)
between the case- and control-patients. Results in-
dicated that having a low regular nurse/patient ra-
tio and a high pool nurse/patient ratio significantly
increase the risk of BSIs.

Table 1 Causes, consequences and outcomes of nursing workload in ICUs
Author Title
Amaravadi ICU nurse/patient ratio is
et al. associated with
(2000) complications and resource
use after esophagectomy
Anderson A descriptive, correlational
and Maloney study of patient satisfaction,
(1998) provider satisfaction, and
provider workload at the
army medical center
Archibald Patient density,
et al. (1997) nurse/patient ratio and
nosocomial infection risk in a
pediatric cardiac intensive
care unit
Ball and Realising the potential of
McElligot critical care nurses: an
(2003) exploratory study of the
factors that affect and
comprise the nursing
contribution to the recovery
of critically ill patients
Human factors engineering conceptual framework
Causes Workload Consequences Outcomes
Staffing level of nurses Workload at the unit A nurse/patient ratio of less than 1:2
level (nurse/patient was found to be related to an
ratio) increased risk of several postoperative
pulmonary and infectious
complications, a 39% increase in
length of stay, and a 32% increase in
direct hospital costs ($4,810) among
patients who had esophagectomy.
Not investigated Nursing workload at the Patient satisfaction was found to be
unit level significantly higher in ICUs than in
(nurse/patient ratio) general care units because of the high
nurse/patient ratio in ICUs.
Staffing level of nurses Nursing workload at the The nosocomial infection rate in the
unit level (nursing pediatric cardiac intensive care unit
hours /patient day was negatively correlated with the
ratio) nursing hours/patient day ratio.
Geography (layout of Changes in nursing • Decreased vs. increased risk to
the ICU) attributes: patients
Activity level in the • patient centered • Timely vs. delayed patient
unit (e.g., admissions, versus task-orientated progression
transfers) care
Skill-mix • proactive care vs. • Potential for recovery vs.
Patient dependency progress hindered deterioration of the patient
• vigilance vs. failure • Family satisfaction with the care
to appreciate cues provided
• coping with
unpredictable vs.
ineffective coping
• emotional support vs.
emotional tension
289

Table 1 (Continued )
Author Title
Beckmann Problems associated with
et al. nursing staff shortage: An
(1998) analysis of the 3600 incident
reports submitted to the
Australian Incident
Monitoring Study (AIMS-ICU)
Dang et al. Postoperative complications:
(2002) Does intensive care unit staff
nursing make a difference?
Dimick et Effect of nurse/patient ratio
al. (2001) in the intensive care unit on
pulmonary complications and
resource use after
hepatectomy
290
Causes Workload Consequences Outcomes
Staffing level of nurses Nursing workload at the Incidents that were Nursing staffing level in the ICU was
Inability to respond to unit level (staffing level most frequently found to be negatively associated with
increased unit activity in the unit was associated with the possibility of a major physiological
subjectively assessed insufficient nursing change in patient’s condition and the
by nurses as sufficient staff: possibility of physical injury of a
or insufficient) • drug administration patient and positively associated with
or documentation patient or family satisfaction.
problems
• inadequate patient
supervision
• incorrect set up of
ventilator or equipment
• self-extubation
Staffing level of nurses Workload at the unit The level of nurse staffing was
level: negatively associated with the risk of
• Low-intensity complications in patients undergoing
staffing: 1:3 or greater abdominal aortic surgery.
on the day and night
shifts
• Medium-intensity
staffing: 1:3 or greater
on either the day or
the night shift, but not
both
• High-intensity
staffing: 1:2 or fewer
on the day and night
shifts
Staffing level of nurses Nursing workload at the As nurse/patient ratio decreases,
P. Carayon, A.P. Gürses
unit level postoperative pulmonary
(nurse/patient ratio) complications and use of resources
increase. A nurse/patient ratio less
than 1/2 was related to a 14% increase
in direct hospital costs.

Giraud et Iatrogenic complications in
al. (1993) adult intensive care units: A
prospective two-center study
Grundmann Risk factors for the
et al. transmission of
(2002) methicillin-resistant
staphylococcus aureus in an
adult intensive care unit:
Fitting a model to the data
Gunnarsson Mobile computerized
et al. tomography scanning in the
(2000) neurosurgery intensive care
unit: Increase in patient
safety and reduction of staff
workload
Jakob and Intensive care 1980—1995:
Rothen Change in patient
(1997) characteristics, nursing
workload and outcome
Human factors engineering conceptual framework
Nursing workload at the As the level of nursing The occurrence of major iatrogenic
situation level workload (both at the complications is positively associated
(subjectively situation level and the with increased morbidity and
measured) patient level) mortality.
Nursing workload at the increases, the risk of
patient level (OMEGA) major iatrogenic
complications
increases.
Nursing workload at the Insufficient compliance Exposure to relative nursing staff
unit level (daily nurse with hand hygiene deficit was the only factor associated
staffing level/daily bed policies with the potential transmission of
occupancy) methicillin-resistant staphylococcus
aureus.
Use of mobile Nursing workload at the A reduction of 60% to 75% in staff
computerized situation level (time workload is estimated when mobile CT
tomography (CT) head spent by nurses on a is used as opposed to conventional CT.
scanning versus single scanning
conventional head CT procedure = Mean
scanning in a number of
neurosurgery ICU nurses × Mean time for
transportation and
scanning)
• Increase in the Nursing workload at the Although there has been a significant
number of patients unit level (Swiss increase in nursing workload during
• Number of severely Society of Intensive the last 16 years, ICU mortality rate
sick patients Care Medicine grading and length of stay decreased. This
• Age of patients system) Nursing may be due to improved patient
admitted to this ICU workload at the patient treatment or care.
during the last 16 years level (modified version
of TISS)
291
Table 1 (Continued )
Author Title
Malacrida Computer-aided
et al. self-observation
(1991) psychological stressors in an
ICU
Morales et Hospital mortality rate and
al. (2003) length of stay in patients
admitted at night to the
intensive care unit
Moreno et Mortality after discharge
al. (2001) from intensive care: The
impact of organ system
failure and nursing workload
use at discharge
Pronovost Organizational Staffing level of nurses
et al. characteristics of intensive
(1999) care units related to
outcomes of abdominal aortic
surgery
Robert et The influence of the
al. (2000) composition of the nursing
staff on primary bloodstream
infection rates in a surgical
intensive care unit
292
Causes Workload Consequences Outcomes
Nursing workload at the High workload is one of the most
job level (as measured important stressors among ICU nurses.
by Nursing Stress Scale
and a computer-aided
self-observation
method)
Admissions to the ICU Nursing workload at the No significant association was found
situation level between workload and the mortality
(situation = patient rate, and between workload and
admission) length of stay.
Workload rated as
‘‘heavy’’ if 3 or more
admissions during night
shifts
Workload rated as
‘‘regular’’ if less than 3
admissions during night
shifts
Organ dysfunction or Workload at the patient Patients who have a high degree of
failure level (NEMS) organ dysfunction or failure before
being discharged from ICU require
more nursing workload and have a
higher probability of dying after being
discharged from ICU.
Workload at the unit A nurse/patient ratio less than 1/2
level (nurse/patient was related to increased ICU days and
ratio) hospital length of stay.
P. Carayon, A.P. Gürses
Staffing level of nurses Workload at the unit No significant association exists
Regular nursing staff level (nurse/patient between the overall nurse/patient
versus pool nursing ratio, regular ratio in the unit and acquisition of
staff nurse/patient ratio, blood stream infections. A low regular
pool nurse/patient nurse/patient ratio and a high pool
ratio) nurse/patient ratio significantly
increase the risk of blood stream
infections.

Schaufeli Personnel Workload at the job level
and Le (scale derived from
Blanc Karasek’s job content
(1998) instrument (1985)
Ipatients /Opatients ,
Iteam /Oteam )
Workload at the patient
level (NEMS)
Stillwell et The impact of do-not Do-not-resuscitate Workload at the patient
al. (1997) resuscitate orders on nursing (DNR) order level (Medicus Systems
workload in an ICU Corporation Interact 2000
Workload and Productivity
System)
Tarnow- Hospital mortality in relation Staffing level of nurses Workload at the unit level
Mordi et al. to staff workload: A 4-year High patient volume (ICU occupancy per shift,
(2000) study in an adult intensive (number of beds ICU occupancy during a
care unit occupied exceeds the patient’s first shift, peak
number appropriately occupancy during a
staffed) patient’s stay, average
occupancy per shift, ICU
nursing requirement per
shift, ICU nursing
requirement during
patient’s first shift, ratio
of occupied to
appropriately staffed
beds)
Human factors engineering conceptual framework
Perceived workload was positively
related with burnout and negatively
related with perceived unit
performance. Burnout was a mediator
between workload and unit
performance. An objective measure of
workload was positively related to
burnout, but burnout was not related
to objective unit performance.
This study did not find any difference
in the number of hours of nursing care
required for a patient in the ICU
before and after a DNR order is
written. This finding is in contradiction
with the belief that patients with DNR
orders require less nursing care.
Consequences of As ICU workload increases, the risk of
excessive workload: mortality increases. Peak occupancy,
• insufficient time for average nursing requirement per
clinical procedures to occupied bed per shift, and the ratio
be done appropriately of occupied to appropriately staffed
• inadequate training beds are related to increased
or supervision mortality risk.
• errors
• nosocomial infections
as a result of
overcrowding
• limited availability of
equipment
• premature discharge
from the ICU
293

Table 1 (Continued )
Author Title
Thorens et Influence of the quality of
al. (1995) nursing on the duration of
weaning from mechanical
ventilation in patients with
chronic obstructive
pulmonary disease
Tissot et al. Medication errors at the
(1999) administration stage in an
intensive care unit
UK Patient volume, staffing, and
Neonatal workload in relation to
Staffing risk-adjusted outcomes in a
Study random stratified sample of
Group UK neonatal intensive care
(2002) units: a prospective
evaluation
294
Causes Workload Consequences Outcomes
Workload at the patient
level (ICU nursing
requirement per
occupied bed per shift)
Staffing level of nurses Workload at the unit The duration of mechanical
Education level of ICU level: index of nursing ventilation is negatively correlated
nurses (certified versus is a ratio based on the with the nursing index.
noncertified) number and the
qualifications of the
nurses working in the
unit.
Work overload The authors identified ten possible
causes of medication errors, including
high nursing workload fatigue.
Staffing level of nurses Percentage maximum Risk adjusted mortality increases as
occupancy on the maximum occupancy on admission
admission increases and the nurse/infant ratio
Nurse/infant ratio decreases.
P. Carayon, A.P. Gürses
Human factors engineering conceptual framework
Nurse/patient ratio and other ICU-level work-
load measures can be used to measure the overall
nursing workload in an ICU. This information can
be used to compare ICUs and their patient out-
comes. Important research using ICU-level work-
load measures has demonstrated strong evidence of
the relationship between high nursing workload and
negative patient outcomes. This type of workload
measure is at a macro-level, and therefore does not
consider contextual and organizational character-
istics of a particular ICU, such as the type of work
performed by the nurses and other staff involved
in patient care as well as ancillary tasks. Improve-
ment solutions that can be derived from studies
using ICU-level workload measures would typically
involve either increasing the number of nurses or
decreasing the number of patients. This type of im-
provement strategy is probably not feasible to im-
plement given the current nursing shortage and the
ever-expanding ICU patient population. Therefore,
we need another type of approach for examining
and dealing with workload.
Workload at the job level
Workload can be conceptualized as a working con-
dition that is a stable characteristic of the job.
One example of a study using a job-level mea-
sure of workload is that conducted by Schaufeli
and Le Blanc (1998). They examined the impact
of workload on burnout and performance among
ICU nurses. They used three different subjective
measures and one objective measure to measure
nursing workload. One of the subjective mea-
sures was a scale derived from Karasek’s Job Con-
tent instrument (Karasek, 1985). Results of the
study showed that perceived workload was pos-
itively related with burnout and negatively re-
lated with unit performance evaluated by ICU
nurses.
There is a large body of research that links
job-level workload (a working condition) to vari-
ous nursing outcomes, such as stress (Crickmore,
1987; Malacrida et al., 1991) and job dissatisfac-
tion (Freeman and O’Brien-Pallas, 1998).
Job-level workload is one of many working condi-
tions that characterize a job. Therefore, job-level
workload is a stable job characteristic that does
not vary much over time unless some change is
introduced in the work organization that will af-
fect the characteristics of the job (Carayon et al.,
2001). Measures of job-level workload can be use-
ful to compare and contrast ICU nursing workload
to workload in other nursing jobs and to examine
how ICU nursing workload affects quality of work-
ing life of nurses and quality and safety of care.
295
This approach can provide information on work-
load in ICU nursing as compared to other jobs.
When using this approach to reduce nursing work-
load, the main improvement strategy consists of
changing the work organization, which is difficult
to accomplish in ICUs. In addition, some of the
key elements of an ICU nursing job, e.g. patient
requirements and patient care tasks, cannot be
changed.
Workload at the patient level
Several measures have been developed to deter-
mine nursing requirements and resource allocation
in ICUs. These measures estimate nursing workload
based on the condition of the patient. The most
widely used patient-level workload measure is the
Therapeutic Intervention Scoring System-76 (TISS-
76) developed originally by Cullen et al. (1974)
and expanded by Keene and Cullen (1983). TISS-76
measures nursing workload based on several thera-
peutic variables related to patient’s condition. The
original TISS-76 has been criticized for the following
reasons (Miranda et al., 1996, 1997):
1- Scoring with TISS-76 is time consuming.
2- Users find filling out TISS-76 instrument cumber-
some and boring.
3- TISS-76 does not have items related to indirect
patient care activities of ICU nurses such as con-
tact with family or maintaining supplies.
4- TISS does not have items related to activities
of the nursing staff that are not related to pa-
tient care (e.g., meetings, trainee supervision,
research).
5- TISS-76 does not have high inter-rater reliability.
A simplified version of TISS with 28 items (TISS-
28) was created to address the first two criticisms.
TISS-28 explains 86% of the variation in TISS-76
(Miranda et al., 1996). In 1994, the Foundation for
Research on Intensive Care in Europe (FRICE) began
a large prospective survey of ICUs in Europe. TISS-
28 was found to be too long and time consuming
for use in this multi-center multinational epidemi-
ological study (Miranda et al., 1997). As a result,
FRICE developed and validated a new therapeutic
index called the Nine Equivalents of Nursing Man-
power Use Score (NEMS). NEMS was derived from
the TISS-28 and includes 9 items. NEMS was com-
pared with TISS-28 in several studies and found to
be highly associated with TISS-28. However, NEMS
was found to have a poor discriminative power of
workload At the individual patient level (Miranda et
al., 1997; Rothen et al., 1999).
Although TISS measures nursing workload based
on therapeutic interventions related to severity of
296 P. Carayon, A.P. Gürses

Table 2 Levels and Measures of Nursing Workload in ICUs

Levels of Workload Measures of Workload
ICU/Unit Level Nurse/patient ratio (Dimick et al., 2001; Pronovost et al., 1999)
Nursing hours/patient day ratio (Archibald et al., 1997)
Regular nurse/patient ratio (Robert et al., 2000)
Pool nurse/patient ratio (Robert et al., 2000)
Index of nursing = ratio calculated based on both the number and the qualifications
of the nurses working in the unit (Thorens et al., 1995)
Patient census (Anderson and Maloney, 1998)
Daily bed occupancy = measure of the entire patient load passing through the ICU
during a single day (Grundmann et al., 2002)
Daily nurse staffing level/daily bed occupancy = ratio of less than 1 is considered as
a relative staff deficit (Grundmann et al., 2002).
Occupancy per shift = highest number of ICU beds occupied each shift during a pa-
tient’s stay (Tarnow-Mordi et al., 2000)
Peak occupancy = highest occupancy per shift during a patient’s stay (Tarnow-Mordi
et al., 2000)
Ratio of occupied /appropriately staffed beds per shift (Tarnow-Mordi et al., 2000)
ICU nursing requirement per shift = total number of ICU nurses required in an ICU
per shift (Tarnow-Mordi et al., 2000)
Number of nursing days per year (Jakob and Rothen, 1997)
Number of patients per nurse per year (Jakob and Rothen, 1997)
Work Utilization ratio (WUR) = ratio between the produced workload and the avail-
able workload which can be calculated by dividing total NEMS points used during
1 year by the number of nurses × 200 × 46/3 where 200 is the annual number of
working days for each nurse, 46 is the maximal number of NEMS points a nurse can
perform in a day, and 3 is the number of 8-h nursing shifts (Moreno and Miranda,
1998).
Job Level Overall perceived workload associated with a job - Karasek’s Job Content Instrument
(Karasek, 1985)
Ipatients /Opatients = ratio of investments to outcomes evaluated by ICU nurses, in the
relationship with their patients (Schaufeli and Le Blanc, 1998)
Iteam /Oteam = ratio of investments to outcomes evaluated by ICU nurses, in the rela-
tionship with their colleagues (Schaufeli and Le Blanc, 1998)
Perceived quantitative workload - Caplan et al. (1975)
Patient Level Time spent by nurses on a single scanning procedure = mean number of
nurses × mean time for transportation and scanning (Gunnarsson et al., 2000)
Therapeutic Intervention Scoring System-28 (TISS-28) = simplified version of TISS-76
with 28 items (Miranda et al., 1996).
Nine Equivalents of Nursing Manpower Use Score (NEMS) = NEMS is the simplified
version of TISS-28. It quantifies the nursing manpower use in ICUs based on nine
items (Miranda et al., 1997).
Nursing Activities Score (NAS) = modified version of TISS-28 with five additional items
and 14 sub-items describing nursing activities in an ICU (e.g., monitoring, care of rel-
atives, administrative tasks). As opposed to the weights in TISS-28, which represent
the severity of the patient’s condition, the weights of NAS represent the calculated
percentage of nursing time dedicated to the performance of the activities listed
(Miranda et al., 2003).
PRN 80 = Canadian information system for the management of nursing staff in hos-
pitals and homes for the elderly. This instrument lists 214 indicators or tasks that
nurses complete on behalf of patients during a 24-h period. Each indicator or task
is assigned a point value that represents the time to complete a specific nursing
intervention as well as the number of times the task will be completed on a shift
(EROS Equipe de Recherche Operationnelle en Sante, 1981).
Time Oriented Score System (TOSS) = method for direct quantification of nurses’
workload required by ICU patients (Italian Multicenter Group of ICU Research, 1991)
OMEGA = ICU-specific activity scoring system used in the French ICUs since 1990. The
Omega score is calculated during each patient’s ICU stay and scored from 1 to 10
(Le Gall et al., 1990).
Human factors engineering conceptual framework 297

Table 2 (Continued )
Levels of Workload Measures of Workload
UK Intensive Care Society Nursing Dependency Schedule = quantifies the qualitative
description of nurse dependency based on the guidelines of the UK Intensive Care
Society. The number of nurses required to take care of a patient is between 0.5 and
2 (Standards Subcommittee Intensive Care Society, 1983).
Patient Intensity for Nursing Index (PINI) = A multidimensional measurement instru-
ment for measuring nursing intensity in ICUs. It consists of three dimensions; depen-
dency, severity, and complexity with time (hours of nursing care) (Prescott et al.,
1989; Soeken and Prescott, 1991).
Patient nursing condition as measured by the number and types of nursing diag-
noses: Nursing diagnoses are determined by the National Conferences on the Clas-
sification of Nursing Diagnosis to describe patients’ nursing conditions (Gebbie and
Levin, 1975). Nursing diagnosis has the advantage of being related to nursing theory
as opposed to medical diagnosis generated by physicians (Halloran, 1985).
Swiss Society of Intensive Care Medicine (SGI) Grading System = category I = one
nurse/patient per shift, category II = one nurse/two patients/shift, category
III = one nurse/three patients/shift, category Ia = 1.3 nurses/patient/shift (Jakob and
Rothen, 1997)
Medicus Systems Corporation InterAct 2000 Workload and Productivity System
(Interact, 2000)
Situation level Subjective workload scales, such as NASA-TLX (Hart and Staveland, 1988)
Time spent by nurses on a single scanning procedure = mean number of
nurses × mean time for transportation and scanning (Gunnarsson et al., 2000)

illness, many nursing activities are not necessar-
ily related to illness severity. In response to this
criticism of TISS, Miranda et al. (2003) recently de-
veloped a new instrument called the Nursing Ac-
tivities Score (NAS). NAS is a modified version of
TISS-28 with an additional five new items and 14
sub-items. The weights of NAS measure time con-
sumed by nursing activities at the patient level.
As opposed to the weights in TISS-28, which rep-
resent the severity of the patient’s condition, the
weights of NAS represent the calculated percent-
age of nursing staff’s time (one period of 24 h)
dedicated to the performance of the activities in-
cluded in the instrument. The sum of the weights
of the individual items scored reflects the amount
of time that the nursing staff in an ICU spends
on performing tasks during a particular day. NAS
was validated in a study of 99 ICUs in 15 coun-
tries. Results indicate that NAS explains 81% of the
nursing time, whereas TISS-28 explains only 43%
(Miranda et al., 2003).
The nursing workload measures in the patient-
level category are based on a patient’s condition.
They are not helpful to understand the impact of
different ‘‘situations’’ that are likely to occur in an
ICU on nursing workload. We, therefore, introduce
the situation level of workload measures to under-
stand the impact of various situations on nursing
workload.
Workload at the situation level
Situation-level workload is that workload explained
by what happens in a clinical microsystem and is
temporally bound. First, the characteristics of the
clinical microsystem can contribute to situation-
level workload. For example, a nurse who goes into
an isolation patient’s room to perform some nursing
activities discovers unexpectedly that one of the
supplies s/he needs is not available in the room.
The nurse, then, leaves the room, removes his/her
isolation gown, goes to the supply room, finds the
right supply, takes it back to the unit, puts the iso-
lation gown on, and enters the patient’s room with
the supply necessary to perform the activities. In
this example, additional workload in the form of
extra tasks and increased time is related to inade-
quate stocking of isolation rooms. This micro-level
characteristic is related to the design of the clini-
cal microsystem. A second important characteristic
of situation-level workload is its narrow temporal
boundaries. Situation-level workload is evaluated
either for a specific event, such as intrahospital pa-
tient transport, or over a short time period, such as
a shift.
We propose that workload measures developed
in the human factors engineering literature can
be used to assess situation-level workload. Very
few studies conducted in ICUs have used the work-
298
load measurement instruments developed by hu-
man factors engineers. Among the many tools de-
veloped by human factors engineers for measur-
ing workload, subjective workload rating scales
have emerged as a primary source of informa-
tion about situational loading (Casali and Wier-
wille, 1983). Tsang (2001) identify three different
ways to measure workload subjectively: (1) using
a single one-dimensional rating scale for measur-
ing the overall workload level versus using a multi-
dimensional rating scale to measure different di-
mensions of workload; (2) subjects’ rating of work-
load experienced immediately after they complete
the task or retrospectively after experiencing all
the task conditions; and (3) absolute or relative
ratings. An absolute rating is based on the task
by itself. For relative ratings, subjects compare
the task to a single standard or to multiple task
conditions.
Subjective measures have high acceptance be-
cause subjects can express their opinions. Further-
more, they are easy to use. They also have high
transferability to new system or new task condi-
tions because their unit of measurement is not task
dependent (Nisbett and Wilson, 1977). Subjective
measures are reliable and have concurrent validity
with performance measures (Gopher and Braune,
1984). One weakness of subjective measures is that
subjects may not remember everything accurately
if they rate their workload level long after complet-
ing the tasks. Subject’s bias and past experience
may also affect subjective workload ratings (Tsang,
2001).
Two of the most widely subjective workload
measures are the NASA Task Loading Index (NASA-
TLX) (Hart and Staveland, 1988) and the Subjective
Workload Assessment Technique (SWAT) developed
by the USAF Armstrong Aeromedical Research Lab-
oratories (Reid et al., 1981). Both scales are mul-
tidimensional. The scales differ from each other in
many respects such as the way they were derived,
the number and type of dimensions, the degree to
which individual differences are represented, and
the scaling method used to combine the individ-
ual scale ratings into an overall value of workload
(Hendy et al., 1989).
The NASA-TLX was developed from the NASA
Bipolar Scale (Hart and Staveland, 1988), and con-
sists of six scales: mental demand, physical de-
mand, temporal demand, performance, effort and
frustration level. Each scale is divided into 20 equal
intervals with the verbal anchors Low and High
(Hendy et al., 1989). SWAT consists of three scales,
i.e. mental effort load, time load, and psychologi-
cal stress load, with three levels for each scale ([1]-
low, [2]-medium, [3]-high) (Reid et al., 1981).
P. Carayon, A.P. Gürses
These subjective workload measures can be used
to evaluate ICU nursing workload with regard to a
specific event, such as intrahospital patient trans-
port, or with regard to a short time period, such as
after a shift. We argue that the strength of study-
ing workload at the situation level is the ability to
identify specific characteristics of the microsystem
that contribute to workload. This information can
then be used in redesign and improvement efforts.
Conclusion
We suggest that focusing on ICU nursing workload
at the situation level can provide the information
necessary to identify specific causes of workload.
Using human factors engineering (HFE) knowledge,
and HFE approaches and measurement methods of
workload can provide direction for specific inter-
ventions that can be implemented in ICUs in order
to improve nursing quality of working life, and qual-
ity and safety of patient care.
According to this ‘‘micro’’ approach, one needs
to examine the factors in a nurse’s immediate work
system and her/his clinical microsystem that add
unnecessary workload, increase the stress level,
and hinder performance. These factors are known
as performance obstacles (Peters et al., 1980;
Brown and Mitchell, 1988; Park and Han, 2002).
There may also be work system factors that can
facilitate the ICU nurse in performing her job;
these factors are called performance facilitators
(Carayon et al., 2005). We propose that the con-
cept of performance obstacles and facilitators can
be used to identify the sources of nursing workload,
and their impact on quality of working life, and
quality and safety of care in ICUs (Carayon et al.,
2005). The performance obstacles and facilitators
are the causes of workload (see Fig. 1). In a man-
ner inspired by the Balance Theory of Job Design
of Smith and Carayon (1989, 2000), one can iden-
tify the performance obstacles that significantly in-
crease unnecessary workload in an ICU and build
up the performance facilitators, and therefore re-
design some aspects of the ICU work system.
A similar approach has been proposed by Ball and
McElligot (2003). They conducted an exploratory
study to identify the factors that affect and com-
prise the contribution of ICU nurses to the recovery
of critically ill patients. Based on data collected via
various methods (e.g., interviews, observations),
they developed a conceptual model of the con-
textual factors in an ICU that affect individual at-
tributes of ICU nurses and consequently the patient
outcomes. Contextual factors in an ICU belonged
to four categories: geography (layout of the unit),
Human factors engineering conceptual framework
activity (the overall activity level in the unit such
as the number of admissions and transfers), skill
mix (permanent versus agency nursing staff expe-
rience level), and patient dependency (e.g., the
degree of confusion and agitation of the patient,
and number of turns required). Positive individual
attributes included patient centered care, proac-
tive management, vigilance, coping with unpre-
dictable situations, and providing emotional sup-
port to patients and families. Negative individual
attributes were task orientated care, failure to ap-
preciate cues about the condition of the patient, in-
effective coping with unpredictable situations, and
increased emotional tension. Therefore, it is im-
portant to take into account both the contextual
factors and individual attributes of nurses when ex-
amining workload of nurse staffing in an ICU.
In this paper, we have presented an overview of
nursing workload in ICUs, its causes, consequences
and outcomes, in particular with regard to patient
safety. We have proposed a conceptual framework
of ICU nursing workload that defines levels of work-
load (unit, job, patient, and situation). Much re-
search has been conducted at the ICU/unit level
and at the patient level; some research has been
conducted at the job level; but little is known about
workload at the situation-level. There is a need
to focus on situation-level workload and identify
those performance obstacles and facilitators that
contribute to workload. This information can help
us to design and target interventions aimed at re-
ducing or eliminating specific causes of workload.
This approach can be expanded to address nurs-
ing workload in other care settings (Carayon et al.,
2003; Institute of Medicine, 2004). We would like to
encourage researchers and practitioners to further
investigate the various levels of nursing workload
and outcomes, in particular situation-level work-
load, and the means of reducing ICU nursing work-
load.
Acknowledgements
Funding was provided by the Agency for Healthcare
Research and Quality (Grant # P20 HS11561-01; PI:
Pascale Carayon, and Dissertation Grant # 1 R36
HS014517).
References
Abramson NA, Wald KS, et al. Adverse occurrences in intensive
care units. J Am Med Assoc 1980;244:1582—4.
Aiken LH, Clarke SP, et al. Hospital nurse staffing and patient
mortality, nurse burnout, and job dissatisfaction. J Am Med
Assoc 2002;288(16):1987—93.
299
Amaravadi RK, Dimick JB, et al. ICU nurse-to-patient ratio
is associated with complications and resource use after
esophagectomy. Intensive Care Med 2000;26:1857—62.
American Hospital Association. American Hospital Association
Statistics-1973—1994. Chicago, Illinois: AHA; 1995.
Anderson FD, Maloney JP. A descriptive, correlational study
of patient satisfaction, provider satisfaction, and provider
workload at an Army medical center. Mil Med 1998;163:90—4.
Angus DC, Kelly MA, et al. Current and projected work-
force requirements for the care of the critically ill pa-
tient and patients with pulmonary disease; can we meet
the requirements of an aging population. J Am Med Assoc
2000;284:2762—70.
Angus DC, Linde-Zwirble WT, et al. The effect of managed care
on ICU length of stay. J Am Med Assoc 1996;276:1075—82.
Archibald LK, Manning ML, et al. Patient density, nurse-
to-patient ratio and nosocomial infection risk in a pe-
diatric cardiac intensive care unit. Pediatr Infect Dis J
1997;16(11):1045—8.
Baggs JD, Schmitt MH, et al. Association between nurse-
physician collaboration and patient outcomes in three inten-
sive care units. Crit Care Med 1999;27(9):1991—8.
Ball C, McElligot M. Realising the potential of critical care
nurses’: an exploratory study of the factors that affect and
comprise the nursing contribution to the recovery of critically
ill patients. Intensive Crit Care Nurs 2003;19:226—38.
Barker K, McConnell W. The problems of detecting medication
errors in hospitals. Am J Hosp Pharm 1962;19:360—9.
Beckmann U, Baldwin I, et al. Problems associated with nursing
staff shortage: an analysis of the first 3600 incident reports
submitted to the Australian Incident Monitoring Study (AIMS-
ICU). Anaesth Intensive Care 1998;26:396—400.
Bracco D, Favre JB, et al. Human errors in a multidisciplinary in-
tensive care unit: A 1-year prospective study. Intensive Care
Med 2000;27(1):137—45.
Bratt MM, Broome M, et al. Influence of stress and nursing lead-
ership on job satisfaction of pediatric intensive care unit
nurses. Am J Crit Care 2000;9(5):307—17.
Brennan TA, Leape LL, et al. Incidence of adverse events and
negligence in hospitalized patients. Results of the Harvard
medical practice study I. N Eng J Med 1991;324(6):370—6.
Brown KA, Mitchell TR. Performance obstacles for direct and
indirect labour in high technology manufacturing. Int J Prod
Res 1988;26(11):1819—32.
Caplan RD, Cobb S, et al. Job Demands and Worker Health: Main
Effects and Occupational Differences. Ann Arbor, MI: Institute
for Social Research; 1975.
Carayon P, Alvarado C. et al., Reducing workload and increas-
ing patient safety through work and workspace design. Pa-
per commissioned by the Institute of Medicine Committee
on the Work Environment for Nurses and Patient Safety,
Center for Quality and Productivity Improvement (CQPI),
University of Wisconsin-Madison; 2003. http://www.engr.
wisc.edu/centers/cqpi/reports/pdfs/r185.pdf.
Carayon P, Gürses AP, et al. Performance obstacles of health care
providers. In: Korunka C, Hoffmann P, editors. To be published
in the Organizational Psychology and Health Care, 4. Munich:
Hampp Publishers; 2005.
Carayon P, Haims MC, et al. Psychosocial work factors and work
organization. In: Karwowski W, editor. The International En-
cyclopedia of Ergonomics and Human Factors. London: Taylor
and Francis; 2001. p. 111—21.
Carayon P, Smith MJ. Work organization and ergonomics. Appl
Ergonom 2000;31:649—62.
Carmel S, Rowan K. Variation in intensive care unit outcomes: A
search for the evidence on organizational factors. Curr Opin
Crit Care 2001;7:284—96.
300
Casali JG, Wierwille WW. A comparison of rating scale,
secondary-task, physiological, and primary task workload es-
timation techniques in a simulated flight task emphasizing
communications load. Human Factor 1983;25:623—41.
Cavanagh SJ. Job satisfaction of nursing staff working in hospi-
tals. J Adv Nurs 1992;17:704—11.
Chalfin DB, Cohen IL, et al. The economics and cost-
effectiveness of critical care medicine. Intensive Care Med
1995;21:952—61.
Cohen IL, Chalfin DB. Economics of mechanical ventilation: Sur-
viving the 90’s. Clin Pulm Med 1994;1:100—7.
Crickmore R. A review of stress in the intensive care unit. Inten-
sive Care Nurs 1987;3:19—27.
Cullen DJ, Civetta JM, et al. Therapeutic intervention scoring
system: A method for quantitative comparison of patient
care. Crit Care Med 1974;2(2):57—60.
Dang D, Johantgen ME, et al. Postoperative complications: Does
intensive care unit staff nursing make a difference? Heart
Lung 2002;31(3):219—28.
Darvas JA, Hawkins LG. What makes a good intensive care unit:
a nursing perspective. Aust Crit Care 2002;15(2):77—82.
Dimick JB, Swoboda SM, et al. Effect of nurse-to-patient ra-
tio in the intensive care unit on pulmonary complica-
tions and resource use after hepatectomy. Am J Crit Care
2001;10(6):376—82.
Donchin Y, Gopher D, et al. A look into the nature and causes
of human errors in the intensive care unit. Crit Care Med
1995;23:294—300.
EROS Equipe de Recherche Operationnelle en Sante. PRN 80:
La mesure du niveau des soins infirmiers requis. Montreal:
Bibliotheque Nationale du Quebec et du Canada; 1981.
Evans SA, Carlson R. Nurse-physician collaboration: Solv-
ing the nursing shortage crisis. J Am Coll Cardiol
1992;20(7):1669—73.
Fasting S, Gisvold SE. Data recording of problems during anaes-
thesia: Presentation of a well-functioning simple system.
Acta Anaesthesiol Scand 1996;40:1173—83.
Freeman T, O’Brien-Pallas LL. Factors influencing job sat-
isfaction on specialty nursing units. Can J Nurs Admin
1998;11(3):25—51.
Gebbie K, Levin MA. In: Proceedings of the first national confer-
ence on the classification of nursing diagnoses; 1975.
Giraud T, Dhainaut JF, et al. Iatrogenic complications in adult in-
tensive care units: A prospective two-center study. Crit Care
Med 1993;21(1):40—51.
Gopher D, Braune R. On the psychophysics of workload:
Why bother with subjective measures? Human Factor
1984;26:519—32.
Griffith CH, Wilson JF, et al. Housestaff workload and procedure
frequency in the neonatal intensive care unit. Crit Care Med
1999;27(4):815—20.
Groeger JS, Strosberg MA, et al. Descriptive analysis of
critical care units in the United States. Crit Care Med
1992;20:846—83.
Grundmann H, Hori S, et al. Risk factors for the transmission
of methicilin-resistant staphylococcus aureus in an adult in-
tensive care unit: Fitting a model to the data. J Infect Dis
2002;185:481—8.
Gunnarsson T, Theodorsson A, et al. Mobile computerized to-
mography scanning in the neurosurgery intensive care unit:
Increase in patient safety and reduction of staff workload. J
Neurosurg 2000;93:432—6.
Halloran EJ. Nursing workload, medical diagnosis related groups,
and nursing diagnoses. Res Nurs Health 1985;8:421—33.
Halpern NA, Bettes L, et al. Federal and nationwide intensive
care units and healthcare costs: 1986—1992. Crit Care Med
1994;22:2001—7.
P. Carayon, A.P. Gürses
Hart SG, Staveland LE. Development of NASA-TLX (Task Load In-
dex): Results of emprical and theoretical research. In: Han-
cock PA, Meshkati N, editors. Human Mental Workload. North-
Holland: Elsevier Science Publishers B.V.; 1988. p. 139—83.
Hendy KC, Campbell EL, et al. Review of workload: Measure-
ment and prediction. In: Proceedings of the Human Factors
Association of Canada 22nd Annual Conference; 1989.
Institute of Medicine. Keeping Patients Safe: Transforming the
Work Environment of Nurses. Washington, DC: The National
Academies Press; 2004.
Intensive Care National Audit and Research Center, Intensive
Care National Audit and Research Center: Annual Report from
the Case Mix Program Database. London; 2001.
Interact. Workload and Productivity System. Evanston, Ilinois:
Medicus Systems Corporation; 2000.
Italian Multicenter Group of ICU Research. Time oriented score
system (TOSS): A method for direct and quantitative assess-
ment of nursing workload for ICU patients. Intensive Care
Med 1991;17:340—5.
Jacobs P, Noseworthy TW. National estimates of intensive care
utilization and costs: Canada and the United States. Crit Care
Med 1990;18:1282—6.
Jakob SM, Rothen HU. Intensive care 1980—1995: Change in pa-
tient characteristics, nursing workload and outcome. Inten-
sive Care Med 1980-;23:1165—70.
Karasek RA. Job content instrument: questionnaire and users
guide, Revision 1.1. LA. University of Southern California;
1985.
Keene AR, Cullen DJ. Therapeutic intervention scoring system:
Update 1983. Crit Care Med 1983;11:1—3.
Keijsers GJ, Schaufeli WB, et al. Performance and burnout in
intensive care units. Work and Stress 1995;9(4):513—27.
Knaus WA, Draper EA, et al. An evaluation of outcome from
intensive care in major medical centers. Ann Internal Med
1986;104:410—8.
Le Gall JR, Loirat P, et al. The patients in management of in-
tensive care. In: Miranda DR, Williams A, Loirat P, editors.
Guidelines for Better Use of Resources. Dordrecht: Kluwer;
1990. p. 11—53.
Leape LL, Bates DW, et al. Systems analysis of adverse drug
events. J Am Med Assoc 1995;274(1):35—43.
Llenore E, Ogle KR. Nurse-patient communication in the inten-
sive care unit: a review of the literature. Aust Crit Care
1999;12(4):142—5.
Lustbader D, Fein A. Emerging trends in ICU management and
staffing. Crit Care Clin 2000;16(4).
Malacrida R, Bomio D, et al. Computer-aided self-observation
psychological stressors in an ICU. Int J Clin Monitor Comput
1991;8:201—5.
McCloskey JC, McCain BE. Satisfaction, commitment and profes-
sionalism of newly employed nurses. Image: J Nurs Scholar-
ship 1987;19(1):20—4.
Merill JM, Boisaubin EV. Adverse occurences in the intensive care
unit. J Am Med Assoc 1981;245:1214.
Miranda R, Nap, et al. Nursing Activities Score. Crit Care Med
2003;31(2):374—82.
Miranda DR, de Rijk A, et al. Simplified therapeutic intervention
scoring system: The TISS-28 items- results from a multicenter
study. Crit Care Med 1996;24:64—73.
Miranda DR, Moreno R, et al. Nine equivalents of nursing man-
power use score (NEMS). Intensive Care Med 1997;23:760—5.
Morales IJ, Peters SG, et al. Hospital mortality rate and length of
stay in patients admitted at night to the intensive care unit.
Crit Care Med 2003;31(3):858—63.
Moreno R, Miranda DR. Nursing staff in intensive care in Eu-
rope: the mismatch between planning and practice. Chest
1998;113(3):752—8.
Human factors engineering conceptual framework
Moreno R, Miranda DR, et al. Mortality after discharge from
intensive care: the impact of organ system failure and
nursing workload use at discharge. Intensive Care Med
2001;2001(27):999—1004.
Nelson EC, Batalden PB, et al. Microsystems in health care: Part
1. learning from high-performing front-line clinical units.
Joint Comm J Qual Safety 2002;28(9):472—93.
Nisbett RE, Wilson TD. Telling more than we can know: Valid
reports on mental processes. Psychol Rev 1977;84:231—59.
Oates PR, Oates RK. Stress and work relationships in the neona-
tal intensive care unit: Are they worse than in the wards? J
Paediatr Child Health 1996;32:57—9.
Office of Technology Assessment. Health technology case study
28: Intensive care units (ICUs), clinical outcomes, costs and
decision making. Washington DC: U.S. Congress, Office of
Technology Assessment; 1984.
Park KS, Han SW. Performance obstacles in cellular manufactur-
ing implementation - Empirical investigation. Human Factors
Ergonom Manufact 2002;12(1):17—29.
Peters LH, O’Connor EJ, et al. The behavioral and affective con-
sequences of performance-relevant situational variables. Or-
ganizat Behav Human Perf 1980;25:79—96.
Prescott PA, Soeken KL, et al. Measuring patient intensity: a re-
liability study. Evaluat Health Profess 1989;12(3):255—69.
Pronovost PJ, Jenckes MW, et al. Organizational characteristics
of intensive care units related to outcomes of abdominal aor-
tic surgery. J Am Med Assoc 1999;281(14):1310—7.
Reid GB, Shingledecker CA, et al. Development of multidimen-
sional subjective measures of workload. In: Proceedings of
the IEEE International Conference on Cybernetics and Soci-
ety; 1981.
Robert J, Fridkin S, et al. The influence of the composition of
the nursing staff on primary bloodstream infection rates in
a surgical intensive care unit. Infect Control Hosp Epidemiol
2000;21(1):12—7.
Rothen HU, Kung V, et al. Validation of ‘‘nine equivalents of nurs-
ing manpower use score’’ on an independent data sample.
Intensive Care Med 1999;25:606—11.
Rowan KM, Kerr JH, et al. Intensive Care Society’s APACHE II
study in Britain and Ireland-II: Outcome comparisons of inten-
sive care units after adjustment for case mix by the APACHE
II method. Br Med J 1993;307:977—81.
301
Schaufeli W, Le Blanc P. Personnel. In: Miranda DR, Ryan DW,
Schaufeli WB, Fidler V, editors. Organisation and Manage-
ment of Intensive Care: A Prospective Study in 12 European
Countries. Springer; 1998. p. 169—205.
Smith MJ, Carayon-Sainfort P. A balance theory of job design for
stress reduction. Int J Industr Ergonom 1989;4:67—79.
Society of Critical Care Medicine. Critical Care in the United
States. Anaheim, CA: Society of Critical Care Medicine;
1992.
Soeken KL, Prescott PA. Patient intensity for nursing index: The
Measurement model. Res Nurs Health 1991;14:297—304.
Standards Subcommittee Intensive Care Society. Standards for
Intensive Care Units. London: Biomedica Ltd; 1983.
Stillwell SB, Woletz J, et al. The impact of do-not-resuscitate
orders on nursing workload in an ICU. Am J of Crit Care
1997;6(5):400—5.
Tarnow-Mordi W, Hau C, et al. Hospital mortality in relation to
staff workload: A 4-year study in an adult intensive-care unit.
Lancet 2000;356:185—9.
Tarnowski-Goodell T, Van Ess Coeling H. Outcomes of nurses’ job
satisfaction. J Nurs Admin 1994;24(11):36—41.
Thorens J, Kaelin RM, et al. Influence of the quality of nursing on
the duration of weaning from mechanical ventilation in pa-
tients with chronic obstructive pulmonary disease. Crit Care
Med 1995;23(11):1807—15.
Tissot E, Cornette C, et al. Medication errors at the adminis-
tration stage in an intensive care unit. Intensive Care Med
1999;25:353—9.
Tsang PS. Mental workload. In: Karwowski W, editor. Interna-
tional Encyclopedia of Ergonomics and Human Factors, 1.
New York: Taylor & Francis; 2001. p. 500—3.
UK Neonatal Staffing Atudy Group. Patient volume, staffing,
and workload in relation to risk-adjusted outcomes in
a random stratified sample of UK neonatal intensive
care units: a prospective evaluation. Lancet 2002;359:99—
107.
Vitez TS. A model for quality assurance in anesthesiology. J Clin
Anesthesiol 1990;2:280—7.
Wagner DP, Wineland TD, et al. The hidden costs of treating
severely ill patients: Charges and resource consumption in
an intensive care unit. Health Care Financ Rev 1983;5:81—
6.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.