Research and Theory for Nursing Practice: An International Journal, Vol. 25, No.

2, 2011

The Impact of Nursing Surveillance

on Failure to Rescue

Leah L. Shever, PhD, RN

University of Michigan Hospital & Health System, Ann Arbor

Greater amounts of nursing surveillance is thought to decrease failure to rescue but

studies to date have used nurse staffing levels as a proxy for nursing surveillance. The
purpose of this nursing effectiveness study was to examine the unique treatment effect
of nursing surveillance on failure to rescue. Data were abstracted from 9 electronic
clinical data repositories including the nursing documentation system that used the
Nursing Interventions Classification (NIC) to record nursing care. Nursing surveillance
was quantified as “high use” when the subjects received it an average of 12 times per
day or more. Propensity scores were used to match subjects who had received high-
dose nursing surveillance with subjects who received low-dose nursing surveillance
(average of less than 12 times a day). The results indicate that when nursing surveil-
lance is performed an average of 12 times a day or greater, there is a significant ( p 5
.0058) decrease in the odds of experiencing failure to rescue (odds ratio [OR] 5 0.52)
compared to when surveillance was delivered an average of less than 12 times a day.
Additional unique variables included in this study are robust levels of nurse staffing
based on hourly data, medical treatments, pharmaceutical treatments, and nursing
treatments. The use of propensity scores helped determine the unique contribution
of nursing surveillance on failure to rescue in this observational study.

Keywords: failure to rescue; nursing surveillance; nursing effectiveness

research; propensity scores

H
ospitalization can be a scary event because of not only the health condition
requiring hospitalization but also fear of complications and adverse events
that too often occur during hospitalization. In To Err is Human: Building a Safer
Health System (Institute of Medicine [IOM], 2000) and Crossing the Quality Chasm:
A New Health System for the 21st Century (IOM, 2001), the IOM exposed that the current
U.S. health care system often does a poor job of keeping patients safe. In a more recent
report, the IOM asserts that nurses are the health care providers most likely to both
prevent and identify complications and therefore activate the appropriate responses in
a timely manner. This rapid response is sometimes referred to as “rescuing” the patient
from death following a complication (IOM, 2004). Not recognizing complications or

© 2011 Springer Publishing Company 107

DOI: 10.1891/1541-6577.25.2.107
108 Shever

not activating appropriate interventions is seen as failure to rescue. Failure to rescue

is a patient safety indicator that many hospitals are tracking and addressing. One
example of how this phenomenon has been addressed is the rise of rapid response
or medical emergency teams (Institute for Healthcare Improvement, 2007).
Generally, failure to rescue has been defined as the death of a patient after a com-
plication. The original development of this quality indicator was specifically patient
death after surgery and within 30 days of discharge and was it compared to two other
commonly used hospital quality indicators: complications and death (Silber, Williams,
Krakauer, & Schwartz, 1992). In this study, researchers included any death after an
operation as failure to rescue with the rationale that patients had been judged healthy
enough to survive surgery and therefore a complication must have occurred. With this
definition, researchers concluded that failure to rescue was a more sensitive metric
to care provided by board-certified anesthesiologists (Silber et al., 1992).
Aiken, Clarke, Cheung, Sloane, and Silber (2003) and Aiken, Clarke, Sloane,
Sochalski, and Silber (2002) more closely examined the relationship between nurse
staffing and failure to rescue. Previously, nurse staffing had been measured using
administrative datasets and therefore a common measure employed to capture
nurse staffing was nurse-to-hospital bed ratio. There are many problems with the
nurse-to-hospital bed ratio like the fact that it does not take into account the number
of patients in the hospital beds, nor does it detect any fluctuation in the number of
nursing staff working on a given day.
Rather than use the administrative datasets as the source for nurse staffing,
Aiken et al. (2003) and Aiken et al. (2002) sent out surveys to a random sample
of nurses employed in study hospitals. In this study, failure to rescue was defined
consistently with Silber and colleagues’ (1992) definition of any death following surgery
(Aiken et al., 2002). The results indicated that adding one patient to a nurse’s work-
load increased the odds of failure to rescue by 7% after controlling for patient and
hospital characteristics (Aiken et al., 2002). Also, a 10% increase in the proportion
of registered nurses (RNs) with higher degrees decreased the risk of failure to rescue
by 5% after controlling for hospital and patient characteristics (Aiken et al., 2003).
In a study that explored nursing-sensitive quality measures, Needleman, Buerhaus,
Mattke, Stewart, and Zelevinsky (2001) examined failure to rescue with a more restricted
list of documented complications (e.g., cardiac arrest, deep vein thrombosis [DVT]/
pulmonary embolus [PE]) in medical and surgical patients. Researchers felt that this
newer definition of failure to rescue was more appropriate and that assuming a com-
plication occurred without proof (i.e., documentation) lacks face validity. Additionally,
researchers believed that in order to apply failure to rescue to medical as well as surgi-
cal patients, the included complications had to be more specific because of medical
patients often having complex comorbid conditions (Needleman & Buerhaus, 2007).
Using this newer definition with fewer designated complications, this study showed
that among medical patients, a higher proportion of RN hours was associated with lower
rates of failure to rescue. However, the number of RN hours per patient day (HPPDs)
was not a significant predictor of failure to rescue in medical patients (Needleman et
al., 2001, 2002). Among surgical patients, a greater number of RN HPPDs was associ-
ated with lower rates of failure to rescue (Needleman et al., 2001, 2002).
The Impact of Nursing Surveillance on Failure to Rescue 109

Because, in large part, of previous studies that demonstrated an association

between higher nurse staffing levels and decreased failure to rescue (Aiken et al., 2002;
Needleman et al., 2002; Silber et al., 2000; Silber, Rosenbaum, Schwartz, Ross,
& Williams, 1995), the concept of nursing surveillance has been hypothesized to
be an important factor in the detection and quick treatment of complications and
therefore preventing failure to rescue (Aiken et al., 2002; Clarke & Aiken, 2003).
Aiken et al. (2002) have hypothesized that round-the-clock surveillance by nurses
is the key to early detection and prompt treatment of a complication. As a result,
the Department of Nursing Care Quality at the National Quality Forum (NQF) has
deemed failure to rescue to be an appropriate quality care indicator influenced by
nursing care (NQF, 2006). Failure to rescue is 1 of 15 nursing care performance
measures considered to be nursing sensitive (NQF, 2006).
Some failure to rescue studies have attempted to measure the concept of surveil-
lance by measuring nurse staffing, but no studies to date have examined the actual
nursing treatment of surveillance. Specifically, no other study has examined higher
levels of surveillance versus lower levels of surveillance and how they impact failure
to rescue. In addition, Clarke and Aiken (2003) note that one of the disadvantages of
most failure to rescue studies is that they are comprised of administrative discharge
abstracts that contain little information about the treatments patients received while
hospitalized, which may impact failure to rescue.

STUDY PURPOSE

The purpose of this study was to examine the unique contribution of nursing sur-
veillance, as documented in the patient’s medical record, on failure to rescue in
older, hospitalized patients. The research question for this study was “What is the
unique contribution of the nursing treatment surveillance on failure to rescue?”
Specifically, “What is the impact of high use of surveillance (delivered 12 times/
day) on failure to rescue compared to not high use (,12 times/day)?” The defini-
tion of failure to rescue was death that occurred after a documented complication
during hospitalization. Surveillance is defined as “purposeful and ongoing acqui-
sition, interpretation, and synthesis of patient data for clinical decision making”
(Dochterman & Bulechek, 2004, p.687).

STUDY DESIGN AND METHODS

The data for this study came from a large, nursing effectiveness study conducted
at one Midwestern tertiary care hospital (Titler, 2000). It was an observational
study that built a clinical effectiveness database from nine electronic, clinical, and
administrative data repositories from a 4-year period (July 1, 1998 to June 31, 2002)
on three older patient populations: those undergoing a hip procedure, those with
congestive heart failure, and those at risk for falling. Both the larger study and this
study were approved by the institution’s Institutional Review Board (IRB).
110 Shever

The data for the larger study came from nine clinical and administrative electronic
data repositories, which are essentially storage areas for electronic data, at one
institution. Of the electronic records, 10% were examined every 3 months to validate
key variables or data elements. Accuracy rates for each quarter of the final year of
data abstraction were 95.8%, 97%, 98.5%, and 99% (Titler, 2000). The larger study
stored the clinical data repositories in a structured query language (SQL) server—a
large electronic data storage area. Patient identifiers were scrambled to create a
unique subject number to maintain patient confidentiality. The subject identifiers
linked the relational databases that were then built. Numerous periodic data checks
were done to ensure that the information in the relational databases matched the
information stored in the nine corresponding electronic data repositories.
This study used data from a subset of the nine data repositories. Patient char-
acteristics and clinical conditions were extracted from medical record abstracts
(MRAs). The context of care variables were taken from the census repository and
nurse staffing systems. The MRA was the source for medical treatments, both the
number and the type. The pharmacy repository provided the pharmaceutical treat-
ments. The nursing information system housed the nursing treatments. Failure to
rescue measures were taken from sources in the MRA.

SETTING AND SAMPLE

The setting for this study was a large academic tertiary care hospital in the
Midwest. One of the unique benefits of conducting nursing effectiveness research
at this hospital is that the hospital has a centralized nursing information system
that incorporated standardized nursing language using the Nursing Interventions
Classification (NIC; Dochterman & Bulechek, 2004). The nurses at this institution
had been using this system and NIC to document the care they delivered elec-
tronically for numerous years prior to the study. Nurses were able to customize,
or select, nursing treatments for each patient in a couple of different ways. Any
nursing treatment could be chosen from an alphabetical list or by patient condi-
tions such as their medical diagnosis. Some nursing units also had preset groups
of treatments for specific patient populations. The caregiver could easily cus-
tomize the treatments for the patient by adding individual treatments to a preset
group or deleting treatments from a group. At the time of this study, there were
more than 250 direct care nursing treatments in the hospital’s electronic nursing
documentation system.
The inclusion criteria used for this study included hospitalizations to one Midwestern
tertiary care hospital over a 4-year period for patients 60 years or older upon admis-
sion and at risk of falling or received the nursing intervention of Fall Prevention.
Patients were determined to be at risk of falling based on a fall risk assessment that
was completed upon admission or when the patient received the nursing treatment
of Fall Prevention. Patients at risk for falling were selected with the rationale that
they would be highly sensitive to nursing care and specifically, to the nursing treat-
ment surveillance (NQF, 2006).
The Impact of Nursing Surveillance on Failure to Rescue 111

VARIABLE DEFINITIONS

Dependent Variable
Failure to rescue was defined as death that occurred during hospitalization follow-
ing a complication as recorded from diagnostic codes in the MRA. Operationally,
the discharge disposition, located in the MRA, was equivalent to death and the MRA
complications included cardiac arrest, respiratory arrest, cerebral vascular accident
(CVA), myocardial infarction, pneumothorax, DVT, PE, and tissue or organ injuries.
This definition is more restrictive than Silber’s in that a medical complication had
to be documented before the death for an event to be counted as failure to rescue. Part
of the rationale for the stricter criteria is that this definition was applied to both medi-
cal and surgical patients; therefore, Silber’s rationale that patients judged as healthy
enough to survive surgery and therefore would not likely die without a complication
occurring, even if one was not documented, could not be applied. The definition was
broader than Needleman’s, Agency for Healthcare Research and Quality’s (AHRQ),
or NQF’s in that it included more complications as documented in the MRA rather
than just five or six complications. This decision was made after considering a more
recent study by Silber et al. (2007) that compared the three most commonly used
definitions of failure to rescue from a medical complication (i.e., death after surgery
[original definition], death occurring after a possible five complications [Needleman
et al., 2001, 2002], and death occurring after a possible six complications [AHRQ,
2007]). This study found that when the three definitions were applied to the same
sample, the latter two had 40% fewer deaths than the original definition of failure to
rescue. The researchers assert that defining failure to rescue where there are only
five or six complications included in the definition may be an unreliable measure
(Silber et al., 2007).

Independent Variables
Independent variables used in this study and their definitions can be found in Table 1.
There were four main variable types: patient characteristics, clinical conditions,
context of care variables, and treatments. Patient characteristics were defined as
preexisting qualities or attributes a person possessed prior to, or at the time of
admission, and included age, ethnicity, gender, employment status and site admit-
ted from. Clinical conditions describe the patient’s extent of compromised health
status and included the patient’s primary medical diagnosis, comorbid conditions,
severity of illness, and previous hospitalizations during the study period. The context
of care variables refer to the environment where the patient received care during
their hospital stay (e.g., HPPDs, skill mix, etc.) Treatments included the number and
type of medical, pharmaceutical, and nursing treatments.

Treatment Variable of Interest: Nursing Surveillance

As stated previously, surveillance is defined in NIC as “purposeful and ­ongoing
acquisition, interpretation, and synthesis of patient data for clinical decision ­making”
(Dochterman & Bulechek, 2004, p.687). Activities nurses perform to deliver ­surveillance
TABLE 1. Conceptual and Operational Definitions of Independent Variables 112
Patient Characteristics
Variable Name Variable Definition and Coding Source Variable Type and Operational Definition
Age Age when patient was admitted to hospital Continuous; measured in years
Ethnicity Race: a group of people united by certain Dichotomous; 0 5 White, 1 5 all others (includes
  characteristics   the categories of African American, Hispanic,
  Native American/Alaskan Native, Asian/Pacific
  Islander, and other)
Gender The behavioral, cultural, and psychological Dichotomous; 0 5 female, 1 5 male
  traits typically associated with one’s sex
Employment status Activity pursued as a livelihood Categorical; 1 5 retired, 2 5 working/employed,
  3 5 homemaker, 4 5 not retired/not employed
Site admitted from The site from which the patient was admitted Categorical; 1 5 hospital, 2 5 care facility,
  to the hospital   3 5 home/other routine admission
Clinical Conditions
Primary medical The primary medical diagnoses came from the Dichotomous; 0 5 no, the diagnosis (i.e., as
  diagnosis   ICD-9-CM codes (Public Health Service and   represented by a particular CCS category) is not
  Health Care Financing Administration, 1994)   the primary diagnosis, 1 5 yes, the diagnosis
  found in MRA diagnostic codes and have been   (i.e., as represented by a particular CCS category)
  classified into CCS categories (HCUP, 2002)   is the primary diagnosis
Elixhauser comorbid Clinical conditions that exist before admission Each of 30 comorbid medical conditions is treated
  conditions   are not related to the principal reason for   as a dichotomous variable: 0 5 no, the condition
  hospitalization and are likely to be significant   was not present at time of admission, 1 5 yes,
  factors influencing mortality and resource use   the condition was present at the time of
  (Elixhauser, Steiner, Harris, & Coffey, 1998)   admission
Severity of illness A rating assigned to each hospitalization Integral; 1 5 mild, 2 5 moderate, 3 5 major,
  retrospectively to measure organ system loss   4 5 severe
  of function or physiological decompensation.
Shever

  Coded using the APR-DRGs (3M Health

  Information Systems, 1993)
Previous The number of previous hospitalizations that Integral; 0 5 no previous hospitalizations, 1 5 1
  hospitalizations during   the patient experienced during the study   previous hospitalization, 2 5 2 previous
  the study period   period   hospitalizations, 3 5 3 previous hospitalizations,
  4 5 4 or greater previous hospitalizations
Context of Care Variables
Average CGPR RN For an entire hospitalization, the average number Continuous; for each 1 hour of the hospitalization,
  of all hourly CGPR RN values (Budreau,   calculate:
  Balakrishnan, Titler, & Hafner, 1999) for the Total number of RN hours for a 1 hour time period
  hospitalization; the hourly CGPR RN values Total number of patient hours for that same hour
  serve as the building blocks for this variable And then calculate:
  and are calculated by dividing the total Sum of hourly CGPR RN values for the entire
  RN hours for a 1-hour period by the total      hospitalization
  patient hours for that same 1 hour time period Total hours of hospitalization
Nursing skill mix Proportion of RNs to all nursing direct Continuous; the average of the hourly RN values
  caregivers for a specified period   were obtained by dividing the total number of
  RNs for all hours by the total number of hours for
  the hospitalization; the average of the total care-
The Impact of Nursing Surveillance on Failure to Rescue

  giver hours was obtained by dividing the total

  number of caregivers for all hours by the total
  number of hours for the hospitalization
CGPR RN dip The extent to which the minimum amount of Continuous; average CGPR RN minus the average
  proportion   RN care falls below the average of all the hourly   of the three lowest hourly CGPR RN values for
  CGPR values for the entire hospitalization; this   the hospitalization; the larger this value is, the
  represents the variability in the amount of RN   more the CGPR RN fell below the average for the
  care that is available, specifically the extent to   hospitalization
  which the amount of RN care available drops
  below the average amount of RN care
  available for the hospitalization
113

(Continued)
TABLE 1. Conceptual and Operational Definitions of Independent Variables (Continued) 114
Patient Characteristics
Variable Name Variable Definition and Coding Source Variable Type and Operational Definition
Number of units The sum of the number of units on which Integral; 1 5 1 unit, 2 5 2 units, 3 5 3 units,
  resided on   treatment was provided to an individual   4 5 4 units, 5 5 5 units
  patient during the course of the hospitalization
Treatments
Number of medical Medical procedures performed during a Continuous; a count on the number of medical
  treatments   hospitalization to diagnose and treat a given   treatments that were performed during the
  patient based on a physician’s judgment and   course of a hospitalization, this is not the number
  knowledge to promote or maintain health,   of unique medical treatments
  cure diseases, or palliate incurable diseases;
  coded using ICD-9-CM codes (Public Health
  Service and Health Care Financing
  Administration, 1994) from the MRA and
  regrouped into multilevel CCS categories
  (HCUP, 2002)
Types of medical Any procedure that, based on a physician’s Dichotomous; 0 5 no treatment (i.e., as
  treatments   judgment and knowledge, is necessary to   represented by a particular CCS category) was
  promote or maintain health, cure diseases,   not received during hospitalization, 1 5 yes, the
  or palliate disease processes that are   treatment (i.e., as represented by a particular
  incurable; coded using ICD-9-CM codes   CCS category) was received at least once during
  (Public Health Service and Health Care   hospitalization
  Financing Administration, 1994) from the
  MRA and regrouped into multilevel CCS
  categories (HCUP, 2002)
Number of unique The count per hospitalization of unique generic Continuous; a count of the number of unique
  medications   drug names for drugs administered at least   medications delivered during a hospitalization
Shever

  hospitalization; medication types were coded

  using the AHFS’s three-level system types
  were coded (McEvoy, 2000)
Pharmaceutical Medications used in the care of patients Dichotomous; 0 5 no medication from the AHFS
  treatments   during a hospitalization; medication types   class was administered during the hospitalization;
  were coded using the AHFS’s three-level   1 5 yes, at least one medication from the AHFS
  system (McEvoy, 2000)   class was administered at least once during the
  hospitalization
Number of unique The number of unique nursing treatments Continuous; a count of the unique nursing
  nursing treatments   delivered during the hospitalization; captured   treatments delivered during the hospitalization
  using NIC (Dochterman & Bulechek, 2004)
Nursing treatments Any treatment nursing personnel performed to Categorical; (multi-level; Reed et al., 2007)
  enhance patient/client outcomes; captured A. NIC used in .95% of hospitalizations; divided
  using NIC (Dochterman & Bulechek, 2004) into quartiles:
1 5 1%25% (lowest use rates, includes 0 use)
2 5 26%50% quartile
3 5 51%75% quartile
4 5 76%100% quartile (highest use rates)

B. NIC used in 95% and 5% of hospitalizations;

  divided into thirds:
The Impact of Nursing Surveillance on Failure to Rescue

0 5 NIC not used

15 1%33% lowest third
25 34%67% middle third
35 68%100% top third

C. NIC used in ,5% of the hospitalizations

05 did not receive the NIC
15 did receive the NIC
Note. AHFS 5 American Hospital Formulary Service; APR-DRGs 5 All Patient Refined Diagnosis Related Groups;
CCS 5 Clinical Classification Software; CGPR 5 caregiverpatient ratio; HCUP 5 Healthcare Cost and Utilization Project;
ICD-9-CM 5 International Classification of Diseases, Ninth Revision, Clinical Modification; MRA 5 medical record abstract;
NIC 5 Nursing Interventions Classification; RN 5 registered nurse.
115
116 Shever

include determining a patient’s health risks (e.g., orthostatic hypotension, physical

impairments, cognitive impairments), monitoring the patient’s ability to perform
activities of daily living (e.g., ability to ambulate independently), monitoring the
patient’s behavior patterns (e.g., using the call light, waiting for assistance to ambu-
late when appropriate), monitoring the patient’s elimination patterns (e.g., toileting
every 2 hours), and others (Dochterman & Bulechek, 2004; Titler, 1992). To measure
surveillance at the patient level, the number of times it was documented as deliv-
ered during the hospital stay was divided by the number days the patient was in the
hospital. This resulted in the average number of times per day it was delivered to a
patient. The documentation of surveillance was captured in the patient’s electronic
health record using the NIC.

PROPENSITY SCORES

Propensity scores analysis is a useful statistical approach when a researcher is inves-

tigating a treatment effect in an observational study (Rosenbaum & Rubin, 1983).
Traditionally, case-control methods have been used in intervention studies to minimize
bias between subjects who receive a specified treatment (cases) and those who do
not receive the specified treatment (controls) by matching on selected variables that
may contribute to the treatment bias, known as confounders (e.g., age, comorbid
conditions, medical treatments, etc.) The limitation in case-control studies is that
matching is possible on only a limited number of variables. In contrast, propensity
score methods calculate a propensity score for each subject through regression
methods and can thus accommodate many more variables (potential treatment
confounders) than is possible using case-control methods.
Rosenbaum and Rubin (1983) first proposed propensity scores as a method for
establishing causal inference in observational studies. Propensity scores are used
to reduce the estimation bias of the treatment effect between subjects that received
treatment and those subjects that did not (Austin, Grootendorst, & Anderson, 2007;
D’Agostino, 1998). The propensity score, calculated for each subject, can be used in one
of three ways: to match subjects on, to stratify subjects into like groups, or covariance-
adjustment in a regression (D’Agostino, 1998; Qin, Titler, Shever, & Kim, 2008).
A conceptual model was used to guide the researcher through the multiple steps
necessary to use propensity scores (see Figure 1). The first step is to dichotomize
the treatment variable of interest: surveillance. Surveillance was divided into two
groups based on the average number of times per day it was delivered during a
hospitalization: 0 5 delivered less than 12 times a day (low surveillance use) and
15 delivered 12 times a day or more (high surveillance use), which averages to
delivering the treatment once every 2 hours. This decision was based on expert
opinion as well as review of the distribution of the average number of times per
day that surveillance was delivered.
The dichotomous treatment variable (i.e., high surveillance use and low surveil-
lance use) first acted as the dependent variable and a logistic regression was run.
Currently, there is not consensus on the criteria for how variables are selected to
 What is the treatment variable of interest?
(must be dichotomous)

Propensity Scores-(Generated from those independent

Potential Confounders variables thought to be confounders.)

Choose one of three methods

Patient Characteristics
Yes Methods
Clinical Conditions Stratification Covariance Matching
Adjusted
Regression
Context of Care Is the
Is the Regression Regression
Is the
variable
related to
Treatments
both the
Medical
The Impact of Nursing Surveillance on Failure to Rescue

treatment and
Pharmacy
outcome variables? Outcome
Nursing
• Failure to Rescue

No Yes

Independent variables available for second step of Do not include

propensity scores analysis. No variable in the
- Is the variable related to the outcome variable? analysis.

Figure 1. Model for nursing effectiveness research using propensity scores. © Leah Shever
117
118 Shever

enter the logistic regression (Austin et al., 2007; Shah, Laupacis, Hux, & Austin, 2005).
However, one study has shown that there is less bias and greater precision when
variables that effect both the treatment assignment and dependent variable are
selected (Austin et al., 2007). This is reflected by the question in the triangle in
Figure 1 that asks whether the variable is related to both the treatment and outcome
variables. The author therefore selected variables related to the treatment variable
(i.e., surveillance use) and the outcome variable (i.e., failure to rescue) for the first
step of variable selection used to generate the propensity scores. Variables that are
related to both the treatment of interest and the outcome variable will henceforth
be referred to as confounders (Austin et al., 2007).
Confounders were selected for this analysis based on previous research on
failure to rescue, nursing surveillance, as well as clinical knowledge and expert
opinion. A propensity score was calculated for each hospitalization based on the
confounders in Table 2. More details are available upon request from the author
on the rationale for variable selection.
There are three ways that the propensity scores can then be used: stratifica-
tion, covariance adjustment, or matching. In this study, the propensity scores
were used to match a subject in the “treatment” group (i.e., high surveillance
use) with a subject in the “control” group (i.e., low surveillance use) as depicted
by the bold boxes and arrows in Figure 1. A hospitalization that received sur-
veillance an average of 12 times a day or more was matched, on its propensity
score, to a hospitalization that received surveillance an average of less than
12 times a day.
As Figure 1 indicates, after matching subjects on their propensity scores, the
matched subjects are then used for the main regression. Variables included in the
main regression included the dichotomous surveillance treatment variable (i.e., high
or low surveillance use) and other independent variables related to the dependent
variable. As Figure 1 indicates, if a variable was used to calculate the propensity
scores, it was not used again in the main regression to avoid correlations between
the propensity scores and the variable (Qin et al., 2008). Variables were selected
to enter the main regression if they were thought to impact the outcome of failure
to rescue. Decisions were again based on previous research of failure to rescue
and expert opinion. The independent variables used in the main regression are
displayed in the results of the main regression in Table 3. More detail is available
from the author upon request.

DATA ANALYSIS

Propensity scores calculation using logistic regression and the process for match-
ing were done using SAS software, version 9.1.3 (SAS Institute, 2006). Specifically,
Proc gmatch in SAS was used to perform the one-to-one matching (SAS Institute,
2006). Descriptive statistics were also generated for the independent and dependent
variables to provide more information about the variables.
The Impact of Nursing Surveillance on Failure to Rescue 119

TABLE 2. Variables Used as Confounders

Patient Characteristics
Age (Silber et al., 1992)
Site where patient was admitted from (Silber et al., 2000; Silber et al., 2002;
  Silber et al., 1995)
Clinical Conditions
Primary medical diagnoses:
• Diseases of the heart • Cancer, other primary
• Cerebrovascular • Cancer of lymphatic and hematopoietic tissue
• Complications • Cancer of bronchus, lung
• Respiratory • Maintenance chemotherapy, radiotherapy
• Diseases of arteries, arterioles • Gastrointestinal hemorrhage
• Chronic obstructive pulmonary

Severity of illness (Silber et al., 2000; Silber et al., 2002; Silber et al., 1995)
The number of comorbid medical conditions (Aiken et al., 2003; Aiken et
  al., 2002; Needleman et al., 2002; Silber et al., 2000; Silber et al., 1995;
  Silber et al., 1992)
Treatments
The number of medical treatment received during hospitalization
Medical treatments:
• Blood transfusion • Tracheostomy, temporary and permanent
• Respiratory intubation and • Other OR heart procedures
mechanical ventilation • Heart valve procedures
• Cancer chemotherapy • CABG
• PTCA • Extracorporeal circulation auxiliary to
• Insertion, revision, replacement, open-heart procedure
removal of cardiac pacemaker • Other OR therapeutic nervous system
• Peripheral vascular bypass procedures

Pharmaceutical treatments:
• Sympathomimetic (adrenergic) • Vasodilating agents
agents • Thrombolytic agents
• Blood derivatives • Hemorrheologic agents
• Hypotensive agents • General anesthetics

Nursing treatments:
• Blood products administration • Airway management
• Dying care • Artificial airway management
• Respiratory monitoring • Mechanical ventilation
• Bleeding precautions

Note. CABG 5 coronary artery bypass graft; OR 5 operating room; PTCA 5

­ ercutaneous transluminal coronary angioplasty.
p
TABLE 3. Results of Regression Using Propensity Scores Examining High Surveillance Use on Failure to Rescue
120

Standard
Variable Names Estimate Error p value Odds Ratio
Surveillance (12 times/day) .6391 0.2318 .0058 0.528
Context of Care
Average CGPR RN (mean RN HPPD  9.14; .3601 0.4681 .4417 0.698
  best staffing]
Average CGPR RN (mean RN HPPD  6.64) .5139 0.4380 .2408 0.598
Average CGPR RN (mean RN HPPD  5.60) .0620 0.4011 .8772 0.940
Average CGPR RN (mean RN HPPD  4.06;
  worse staffing]
CGPR RN dip proportion (per 0.2 unit) 1.9972 0.7096 .0049 1.491 (per 0.2 unit)
Skill mix (per 0.1 unit) 7.2096 1.7617 ,.0001 2.056 (per 0.1unit)
Treatments
Medical treatments
Other OR procedures on vessels other 0.3898 0.2680 .1458 1.477
  than head and neck
Laminectomy, excision intervertebral disc 0.2557 0.4839 .5972 1.291
Other OR gastrointestinal therapeutic 0.6513 0.4032 .1062 1.918
  procedures
Gastrostomy, temporary and permanent 20.3981 0.4306 .3552 0.672
Oophorectomy, unilateral and bilateral 211.8656 571.2 .9834 ,0.001
Amputation of lower extremity 1.3517 0.4222 .0014 3.864
Colorectal resection 0.3762 0.5079 .4589 1.457
Hysterectomy, abdominal and vaginal 29.3032 610.0 .9878 ,0.001
Spinal fusion 0.4207 0.5602 .4527 1.523
Shever
Pharmaceutical treatments
Number of pharmaceutical treatments 0.0539 0.00721 ,.0001 1.055
Nursing treatments
Number of unique nursing treatments 20.00245 0.0183 .8976 0.998
Neurologic monitoring
  High use (68%–100%) 7.56 use rate 0.1662 0.5155 .7472 1.181
  Medium use (34%–67%) 4.46 use rate 0.4530 0.3945 .2508 1.573
  Low use (1%–33%) 1.96 use rate 0.5537 0.2527 .0285 1.740
Surgical preparation 0.90 use rate 20.5180 0.3606 .1509 0.596
Hemodynamic monitoring 1.48 use rate 20.0348 0.3844 .9279 0.966
Aspiration precautions 2.54 use rate 20.2707 0.3533 .4435 0.763
Note. CGPR 5 caregiverpatient ratio; HPPD 5 hour per patient day; OR 5 operating room; RN 5 registered nurse.
The Impact of Nursing Surveillance on Failure to Rescue
121
122 Shever

RESULTS

There were 10,187 hospitalizations comprised of 7,851 unique patients as some patients
were hospitalized multiple times over the study period. Patients were mostly White
(93.5%), retired (74.4%), and admitted from home (64.4%). The mean age of patients
was 73.7 years, females accounted for 52.6% of the sample, and 53.2% were married.
This patient group, defined primarily by a nursing treatment, was medically diverse. The
most common primary medical diagnoses included diseases of the circulatory system
(28.5%); neoplasms (13.8%); injury, including fractures, or poisoning (11.5%); diseases
of the respiratory system (7.5%); and diseases of the digestive system (7.4%).
Patients who had experienced one or more documented complications were
the starting pool for failure to rescue. In this sample, 1,058 hospitalizations (10.4%
of the sample) experienced a complication. The most frequent first complications
consisted of 366 cardiac complications,129 tissue organ injuries, 110 pneumotho-
raxes, 91 cardiac arrests, 75 other specified vein complications, 62 respiratory
complications, 56 iatrogenic cerebrovascular infarction or hemorrhage, 44 other
CVAs, 37 respiratory arrests, and others. Of those 1,058 hospitalized patients
who experienced a medical complication, 168 resulted in death (i.e., failure
to rescue).
The one-to-one matching method resulted in a total sample size of 10,004 hos-
pitalizations, with 5,002 hospitalizations composed of patients who received sur-
veillance an average of 12 times a day or more, and 5,002 hospitalizations where
patients received surveillance an average of less than 12 times a day. There were
31 hospitalizations where patients received surveillance 12 times a day or more
that experienced failure to rescue. In the group that received surveillance less than
12 times a day, 135 hospitalizations resulted in failure to rescue.
Table 3 displays the results of the regression analysis that examined the effect of
high surveillance use on failure to rescue. High surveillance use was significantly
(p 5 .0058) and inversely associated with failure to rescue. Patients who received
surveillance an average of 12 times a day or more were almost half as likely
(OR 5 0.53) to experience failure to rescue compared to patients who received
surveillance less than 12 times a day. The results associated with high surveillance
use are after controlling for other variables believed to be confounders. This is the
proportion of variability explained by high surveillance use on failure to rescue after
using propensity scores to control for treatment bias.
Other variables included in the main regression included context of care and some
treatment variables. The “average caregiver–patient ratio” (CGPR) RN (i.e., HPPDs)
was not significant. The “CGPR dip proportion” (i.e., falling below the average unit
staffing level) was significantly (p 5 .0049) and positively associated with failure to
rescue. The results indicate that when staffing fell 20% below the unit average for a
patient’s hospital stay, it was associated with a 50% increased odds (OR 5 1.49) of
experiencing failure to rescue. Nursing skill mix was also significantly (p , .0001)
and positively associated with failure to rescue. The results indicate that when the
proportion of RNs to all total caregivers increased by 10%, the odds of experiencing
failure to rescue doubled (OR 5 2.06; see Table 3).
The Impact of Nursing Surveillance on Failure to Rescue 123

Nine medical treatments were included in the main regression but only “amputa-
tion of lower extremity” was significantly (p 5 .0014) and positively associated with
failure to rescue so that patients who received this medical treatment were 3.9 times
more likely (OR 5 3.86) to result in failure to rescue than patients who did not receive
this medical treatment. The “number of unique pharmaceutical treatments” received
during a hospital stay was significantly and positively associated with failure to
rescue. For each additional medication received during a hospital stay, the odds of
experiencing failure to rescue increased by 6% (OR 5 1.06; see Table 3).
The “number of unique nursing treatments” received during a hospital stay and
four specific nursing treatments were included in the second regression step (see
Table 3) but only the low use of “neurologic monitoring” was significant (p , .05).
Patients who received neurologic monitoring approximately two times a day (use
rate 5 1.96) were 74% more likely (OR 5 1.74) to experience failure to rescue than
patients who did not receive this nursing treatment.

DISCUSSION

Previous research has attempted to capture nursing surveillance by measuring

nurse staffing levels (e.g., nurse-to-hospital bed ratios, nurse-to-patient ratios). This
study is the first to measure actual nursing surveillance as nurses recorded it in the
patient’s electronic health record. It is the next step to a more full understanding of
how nursing activities, not just nurse staffing, impact patient outcomes.
When considering the results associated with surveillance use, it is helpful to
consider the statistical method used to arrive at the results. Propensity scores were
calculated based on variables the researcher believed to be confounders, those
variables thought to influence the dependent variable (i.e., failure to rescue), and the
amount of surveillance received by the patient. Hospitalizations were then matched on
those propensity scores, thereby creating two groups that were essentially equivalent
in terms of those confounders (Qin et al., 2008). Even though randomization was
not possible in this observational study, the treatment effect of surveillance is more
clearly understood because of the use of propensity scores, which helped create two
equivalent groups that received different “doses” of nursing surveillance.
After creating two equivalent groups based on propensity scores and control-
ling for other variables believed to be related to failure to rescue (e.g., nurse
staffing variables, invasive medical treatments, number of unique medications
received), patients who received high surveillance use (12 times/day) had
reduced odds of failure to rescue by approximately 50% (OR 5 0.52) compared
to when surveillance was delivered less than 12 times a day. By checking on the
patient more frequently (i.e., providing more surveillance), the nurse is likely to
detect changes in the patient’s condition more rapidly and therefore, decrease
the amount of time between complication onset and appropriate treatment.
A complication that is caught early and treated appropriately is more likely to
result in a positive patient outcome compared to complications that go unde-
tected or untreated.
124 Shever

Somewhat surprising were the nonsignificant findings related to the average

CGPR RN (as described in Table 1, this represents the HPPDs) and failure to rescue.
This is important from a systems and policy perspective because many hospitals
and even states (e.g., California) have mandated nurse-to-patient ratios to maintain
minimum HPPDs. These mandated ratios have huge implications that include, but
are not limited to, financial strain on institutions, the closing of units or institutions
due to inability to maintain ratios, which thereby limit access to care for people in
the community, and numerous nursing workforce issues like the use of contract or
agency staff to maintain the ratios.
As Burns and Grove (2001) point out in their chapter on outcomes research, there
is not an established set of methodologies in place for conducting outcomes effec-
tiveness research, and therefore, new strategies should be sought out by research-
ers to address the research questions related to effectiveness of intervention(s).
Although not as robust as case-control designs that better control for treatment
bias, propensity scores are useful in observational studies when trying to control
for treatment effect.
There were limitations to this study. First, data were collected from one large
academic medical center in the Midwest, which limits the generalizability of the
findings. As this study was conducted at one large tertiary care hospital, patients
may have been hospitalized at other local care facilities and that information was
not collected. This may have been important especially for the independent variable
“previous hospital stays during the study period.” Another related consideration is
that patients were not followed past discharge. Adverse outcomes and even death
could have occurred after discharge and that information was not captured in this
study (Forster, Murff, Peterson, Gandhi, & Bates, 2003).
Another limitation is the use of clinical data for research. Although the patient’s
medical record is considered a legal document, is used to plan and evaluate care for
the patient, and is used for billing purposes, there are, sometimes, inconsistencies
in how information is put into the documentation system and there are limitations
on what can be abstracted (Burns & Grove, 2001). Specifically for this study, it would
have been helpful to know the code status of patients (e.g., full code versus do not
resuscitate) in relation to the patient’s mortality.

CONCLUSION

This study makes important contributions to a more full understanding of failure

to rescue. This study is the first to measure nursing surveillance delivered as a
nursing treatment and captured using a standardized language compared to using
staffing levels as a proxy, which have been used previously. The use of propensity
scores helped determine the unique treatment effect of surveillance on failure
to rescue. Another distinctive aspect of this study was the inclusion of medical,
pharmaceutical, and nursing treatments. Lastly, variables used to measure nurse
staffing were more robust based on hourly information than any used in previous
failure to rescue research.
The Impact of Nursing Surveillance on Failure to Rescue 125

Nursing surveillance done an average of 12 times a day or greater decreased the

odds of experiencing failure to rescue compared surveillance delivered an average of
less than 12 times a day. The use of propensity scores helped determine the unique
contribution of surveillance on failure to rescue in this observational study. More
studies like this one are needed to demonstrate the impact of nursing interventions
on patient outcomes.

REFERENCES
Agency for Healthcare Research and Quality. (2007). National quality measures clearinghouse:
Failure to rescue. Retrieved January 20, 2007, from http://www.qualitymeasures.ahrq
.gov/summary/summary.aspx?doc_id=8902&string=Failure+AND+Rescue
Aiken, L. H., Clarke, S. P., Cheung, R. B., Sloane, D. M., & Silber, J. H. (2003). Educational
levels of hospital nurses and surgical patient mortality. Journal of the American Medical
Association, 290(12), 1617–1623.
Aiken, L. H., Clarke, S. P., Sloane, D. M., Sochalski, J., & Silber, J. H. (2002). Hospital nurse staff-
ing and patient mortality, nurse burnout, and job dissatisfaction. Journal of the American
Medical Association, 288(16), 1987–1993.
Austin, P. C., Grootendorst, P., & Anderson, G. M. (2007). A comparison of the ability of different
propensity score models to balance measured variables between treated and untreated
subjects: A Monte Carlo study. Statistics in Medicine, 26(4), 734–753.
Budreau, G., Balakrishnan, R., Titler, M., & Hafner, M. J. (1999). Caregiver-patient ratio: Capturing
census and staffing variability. Nursing Economics, 17(6), 317–324.
Burns, N., & Grove, S. (2001). The practice of nursing research: Conduct, critique, & utilization
(4th ed.). Philadelphia: W.B. Saunders Co.
Clarke, S. P., & Aiken, L. H. (2003). Failure to rescue. American Journal of Nursing, 103(1), 42–47.
D’Agostino, R. B., Jr. (1998). Propensity score methods for bias reduction in the comparison of a
treatment to a non-randomized control group. Statistics in Medicine, 17(19), 2265–2281.
Dochterman, J. M., & Bulechek, G. M. (Eds.). (2004). Nursing interventions classification (NIC)
(4th ed.). St. Louis, MO: Mosby.
Elixhauser, A., Steiner, C., Harris, D. R., & Coffey, R. M. (1998). Comorbidity measures for use
with administrative data. Medical Care, 36(1), 8–27.
Forster, A. J., Murff, H. J., Peterson, J. F., Gandhi, T. K., & Bates, D. W. (2003). The incidence
and severity of adverse events affecting patients after discharge from the hospital. Annals
of Internal Medicine, 138(3), 161–167.
Healthcare Cost and Utilization Project. (2002). Clinical classifications software (CCS) for ICD-
9-CM. Rockville, MD: Author.
Institute for Healthcare Improvement. (2007). Establish a rapid response team. Retrieved
September 25, 2007, from http://www.ihi.org/IHI/Topics/CriticalCare/IntensiveCare/
Changes/EstablishaRapidResponseTeam.htm
Institute of Medicine. (2000). To err is human: Building a safer health system. Washington, DC:
National Academy of Sciences.
Institute of Medicine. (2001). Crossing the quality chasm: A new health system for the 21st cen-
tury. Washington, DC: Free Press.
Institute of Medicine. (Ed.). (2004). Keeping patients safe: Transforming the work environment
of nurses. Washington, DC: National Academy Press.
McEvoy, G. K. (2000). American hospital formulary service (AHFS) drug information 2000. Bethesda,
MD: American Society of Health System Pharmacists, Inc.
126 Shever

National Quality Forum. (2006). Nursing care performance measures. Retrieved April 29, 2007,
from http://www.qualityforum.org/nursing/#measures
Needleman, J., Buerhaus, P., Mattke, S., Stewart, M., & Zelevinsky, K. (2001). Nurse staffing and
patient outcomes in hospitals (Final Report Submitted Under Contract No. 230-99-0021).
Boston: Harvard School of Public Health.
Needleman, J., Buerhaus, P., Mattke, S., Stewart, M., & Zelevinsky, K. (2002). Nurse-staffing levels
and the quality of care in hospitals. New England Journal of Medicine, 346(22), 1715–1722.
Needleman, J., & Buerhaus, P. I. (2007). Failure-to-rescue: Comparing definitions to measure
quality of care. Medical Care, 45(10), 913–915.
Public Health Service and Health Care Financing Administration (Ed.). (1994). ICD-9-CM:
International classification of diseases, 9th revision, clinical modification. Washington, DC:
Public Health Service.
Qin, R., Titler, M. G., Shever, L. L., & Kim, T. (2008). Estimating effects of nursing intervention
via propensity score analysis. Nursing Research, 57(6), 444–452.
Reed, D., Titler, M. G., Dochterman, J. M., Shever, L. L., Kanak, M., & Picone, D. M. (2007).
Measuring the dose of nursing intervention. International Journal of Nursing Terminologies
& Classifications, 18(4), 121–130.
Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observa-
tional studies for casual effects. Biometrika, 70, 41–55.
SAS Institute. (2006). SAS Software (Version 9.1.3) [Computer software]. Cary, NC: SAS
Institute, Inc.
Shah, B. R., Laupacis, A., Hux, J. E., & Austin, P. C. (2005). Propensity score methods gave
similar results to traditional regression modeling in observational studies: A systematic
review. Journal of Clinical Epidemiology, 58(6), 550–559.
Silber, J. H., Kennedy, S. K., Even-Shoshan, O., Chen, W., Koziol, L. F., Showan, A. M., et al. (2000).
Anesthesiologist direction and patient outcomes. Anesthesiology, 93(1), 152–163.
Silber, J. H., Kennedy, S. K., Even-Shoshan, O., Chen, W., Mosher, R. E., Showan, A. M., et al. (2002).
Anesthesiologist board certification and patient outcomes. Anesthesiology, 96(5), 1044–1052.
Silber, J. H., Romano, P. S., Rosen, A. K., Wang, Y., Even-Shoshan, O., & Volpp, K. G. (2007). Failure-
to-rescue: Comparing definitions to measure quality of care. Medical Care, 45(10), 918–925.
Silber, J. H., Rosenbaum, P. R., Schwartz, J. S., Ross, R. N., & Williams, S. V. (1995). Evaluation
of the complication rate as a measure of quality of care in coronary artery bypass graft
surgery. Journal of the American Medical Association, 274(4), 317–323.
Silber, J. H., Williams, S. V., Krakauer, H., & Schwartz, J. S. (1992). Hospital and patient char-
acteristics associated with death after surgery. A study of adverse occurrence and failure
to rescue. Medical Care, 30(7), 615–629.
3M Health Information Systems. (1993). All patient refined diagnosis related groups (APR-DRGs).
Wallingford, CT: Author.
Titler, M. G. (1992). Interventions related to surveillance. Nursing Clinics of North America,
27(2), 495–515.
Titler, M. G. (2000). Nursing interventions and outcomes effectiveness in 3 older populations (No. R01
Report No. NR05331-02). Rockville, MD: National Institute of Nursing Research.

Correspondence regarding this article should be directed to Leah L. Shever, PhD, RN, University
of Michigan Hospital & Health System, 300 North Ingalls, Room NI 5A07, Ann Arbor, MI 48109-
5446. E-mail: sheverl@med.umich.edu
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.