Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
qxd
4/20/06
4:21 PM
Page 272
C H A P T E R
Outcomes Research
Chapter Overview
The Theoretical Basis of Outcomes Research 273
Donabedians Theory of Quality Health Care 273 The Agency for Health Services Research 281 The Agency for Health Care Policy and Research 281 Advanced-Practice Nursing Outcomes Research 292 Practice-Based Research Networks 293
294
Le a r n i n g O utco m e s
After completing this chapter, you should be able to: 1. Discriminate between traditional quantitative research and outcomes research 2. Explain the theoretical basis of outcomes research. 3. Explain the importance of outcomes research. 4. Identify nursing-sensitive patient outcomes. 5. Describe some of the unique methodologies of outcomes research.
Ke y Te r m s
Clinical decision analysis, p. 302 Consensus knowledge building, p. 298 Cost-benet analysis, p. 306 Cost-effectiveness analysis, p. 305 Costs, p. 306 Design, p. 298 Efciency, p. 305 Geographical analyses, p. 305 Health, p. 273 Interdisciplinary teams, p. 304 Intermediate end points, p. 307 Measurement error, p. 297 Medical, p. 283 Multilevel analysis, p. 315 Opportunity costs, p. 306 Outcomes research, p. 273 Out-of-pocket costs, p. 306 Population-based studies, p. 302 Practice pattern proling, p. 299 Prospective cohort study, p. 301 Quality, p. 273 Research tradition, p. 294 Retrospective cohort study, p. 301 Sampling error, p. 297 Small area analyses, p. 305 Standard of care, p. 276 Standardized mortality ratio (SMR), p. 301 Structures of care, p. 277 Variance analysis, p. 313
272
Ch09-X2640.qxd
4/20/06
4:21 PM
Page 273
C H A P T E R
Outcomes Research
273
STUDY TOOLS Be sure to visit http://evolve.elsevier.com/Burns/understanding for additional examples and selftests. Also, a review of this chapters concepts and practice exercises can be found in Chapter 9 of the Study Guide for Understanding Nursing Research: Building an Evidence-Based Practice, 4th edition.
Outcomes research focuses on the end results of patient care. In order to explain the end
results, nurse researchers also must understand the processes used to provide patient care. The strategies used in outcomes research are, to some extent, a departure from the accepted scientic methodology for health care research, and they incorporate evaluation methods, epidemiology, and economic theory. The ndings of outcome studies continue to have a powerful impact on the provision of health care and the development of health policy. The momentum propelling outcomes research comes not from scholars but from policy makers, insurers, and the public. These groups increasingly demand that providers justify interventions and systems of care in terms of improved patient lives and that costs of care be considered in the evaluation of treatment outcomes (Hinshaw, 1992). A major shift has occurred in published nursing studies, with the number of studies using traditional quantitative or qualitative methods being dwarfed by the number of outcomes studies. A large number of nursing and multidisciplinary journals focused on outcomes research have been initiated. A listing of these journals is presented in Table 9-1. This chapter provides an explanation of the theoretical basis of outcomes research, a brief history of the emerging endeavors to examine outcomes, the importance of outcomes research designed to examine nursing practice, and methodologies used in outcomes research.
Ch09-X2640.qxd
4/20/06
4:21 PM
Page 274
274
C H A P T E R
Outcomes Research
Table 91
Best Practice Clinical Effectiveness in Nursing Effective Health Care Evidence-Based Mental Health Evidence-Based Practice Health Education Research Health Promotion Practice Health Technology Assessment International Journal of Health Care Quality Assurance Incorporating Leadership in Health Services Journal of Care Management Journal of Clinical Ethics Journal of Clinical Outcomes Management Journal for Health Care Quality Journal of Health Services and Research Policy Journal of Integrated Care Pathways Journal of Nursing Care Quality Journal of Nursing Quality Assurance Journal of Quality in Clinical Practice Journal of Rehabilitation Outcomes Measurement Nursing and Health Policy Review Outcomes Management Outcomes Management for Nursing Practice Patient Satisfaction Management Quality in Health Care Quality in Primary Care Quality and Safety in Health Care Quality Management in Health Care
Donabedian (1987) identies three objects of evaluation in appraising quality: structure, process, and outcome. A complete quality assessment program requires the simultaneous use of all three concepts and an examination of the relationships among the three. However, researchers have had little success in accomplishing this theoretical goal. Studies designed to examine all three concepts would require sufciently large samples of various structures, each with the various processes being compared and large samples of subjects who have experienced the outcomes of those processes. The funding and the cooperation necessary to accomplish this goal are not yet available.
Evaluating Outcomes
The goal of outcomes research, the evaluation of outcomes as dened by Donabedian, is not as simplistic as it might immediately appear. Donabedians theory requires that identied outcomes be clearly linked with the process that caused the outcome. To accomplish this linking, the researcher must dene the process and justify the causal links with the selected outcomes. The identication of desirable outcomes requires dialogue between the subjects of care and the providers of care. Although the providers of care may delineate what is
Ch09-X2640.qxd
4/20/06
4:21 PM
Page 275
C H A P T E R
Outcomes Research
275
Plan, institution, system Organized team Several practitioners* Individual practitioner Physical-psychological function
Psychological function
Social function Individual Aggregate: case load Individual Aggregate: target population, community Person
Patient
*Of the same profession or of different professions Figure 9-1. Level and scope of concern as factors in the denition of quality. (From Donabedian, A. [1987]. Some basic issues in evaluating the quality of health care. In L. I. Rinke [Ed.], Outcome measures in home care [Vol. 1, pp. 328]. New York: National League for Nursing.)
Care by practitioners and other providers Technical Knowledge, judgment skills Interpersonal Amenities Care implemented by patient Contribution of provider Contribution of patient and family Care received by community Access to care Performance of provider Performance of patient and family Figure 9-2. Various levels at which the quality of health care can be assessed. (From Donabedian, A. [1988]. The quality of care: How can it be assessed? Journal of the American Medical Association, 260[12], 1744. Copyright 1988, American Medical Association.)
Ch09-X2640.qxd
4/20/06
4:21 PM
Page 276
276
C H A P T E R
Outcomes Research
achievable, the subjects of care must clarify what is desirable. The outcomes also must be relevant to the goals of the health care professionals, the health care system of which the professionals are a part, and society. Outcomes are time dependent. Some outcomes may not be apparent for a long period after the process that is purported to cause them, whereas others may be apparent immediately. Some outcomes are temporary and others are permanent. Thus, the selection of an appropriate time frame for determining the selected outcomes must be established. A nal obstacle to outcomes evaluation is one of attribution. This requires assigning the place and degree of responsibility for the outcomes observed. A specic outcome often is influenced by a multiplicity of factors. Lewis (1995) points out that health care represents only one dimension of a complex situation. Patient factors, such as compliance, predisposition to disease, age, propensity to use resources, high-risk behaviors (e.g., smoking), and lifestyle, also must be taken into account. Environmental factors such as air quality, public policies related to smoking, and occupational hazards also must be included. Responsibility for outcomes may be distributed among providers, patients, employers, insurers, and the community. As yet, little scientic basis has been established for judging the precise relationship between each of these factors and the selected outcome. Many of the influencing factors may be outside the jurisdiction or influence of the health care system or of the providers within it. One solution to this problem of identifying relevant outcomes is to dene a set of closely related outcomes specic to the condition for which care is being provided. Critical pathways and care maps may be useful in dening at least related outcomes. However, related outcomes do not provide the degree of evidence of examining the desired outcomes.
Evaluating Process
The process of clinical management has been, for most health care professionals, an art rather than a science. Understanding the process sufciently to study it must begin with much careful reflection, dialogue, and observation. Clinical management has multiple components, many of which have not yet been clearly dened or tested. Bergmark and Oscarsson (1991, pp. 139140) suggest the following questions as important to consider in evaluating process: (1) What constitutes the therapeutic agent? (2) Do practitioners actually do what they say they do? (3) Do practitioners always know what they do? Current outcomes studies are using process variables that are easy to identify. Answers to questions such as those posed by Bergmark and Oscarsson are more difcult to dene and initially will require observation, interviews, and the use of qualitative research methodologies. Three components of process of particular interest to Donabedian are standards of care, practice styles, and costs of care.
Standards of Care
A standard of care is a norm by which quality of care is judged. Clinical guidelines, critical paths, and care maps dene standards of care. According to Donabedian (1987), a practitioner has legitimate responsibility to apply available knowledge in the management of a dysfunctional state. This management consists of (1) the identication or diagnosis of the
Ch09-X2640.qxd
4/20/06
4:21 PM
Page 277
C H A P T E R
Outcomes Research
277
dysfunction, (2) the decision whether to intervene, (3) the choice of intervention objectives, (4) the choice of methods and techniques to achieve the objectives, and (5) the skillful execution of the selected techniques. Donabedian (1987) recommends the development of criteria to be used as a basis for judging the quality of care. These criteria may take the form of clinical guidelines or care maps based on previous validation of the contribution of the care to outcomes. The clinical guidelines published by the Agency for Healthcare Research and Quality (AHRQ) establish norms on which the validity of clinical management can be judged. However, the core of the problem, from Donabedians perspective, is clinical judgment. Analysis of the process of making diagnoses and therapeutic decisions is critical to the evaluation of the quality of care. The emergence of decision trees and algorithms is a response to Donabedians concerns and provides a means of evaluating the adequacy of clinical judgments.
Practice Styles
The style of practice is another dimension of the process of care that influences quality; however, it is problematic to judge what constitutes goodness in style and to provide justication for the decisions. Moreover, diverse styles of interpersonal relationships are possible. Most studies examining practice styles are conducted with physicians as subjects. A few studies of practice styles of nurses, however, are beginning to appear in the literature (Bircumshaw & Chapman, 1988; Fullerton, Hollenbach, & Wingard, 1996).
Costs of Care
A third dimension of the examination of quality of care is cost. Maintaining a specied level of quality of care necessarily has cost consequences. Providing more and better care is likely to increase costs but also is likely to produce savings. Economic benets can be obtained by preventing illness, preventing complications, maintaining a higher quality of life, or prolonging productive life. A related issue is who bears the costs of care. Some measures purported to reduce costs have instead simply shifted costs to another party. For example, in certain instances a hospital can reduce its costs by discharging a particular type of patient early, but total costs will increase if the necessary community-based health care raises costs above those incurred by keeping the patient hospitalized longer. In this case, the third-party provider may experience higher costs. In many cases, the costs are shifted from the health care system to the family as out-of-pocket costs. Studies examining changes in costs of care must consider total costs.
Evaluating Structure
Structures of care are the elements of organization and administration that guide the
processes of care. The rst step in evaluating structure is to identify and describe the elements of the structure. Various administration and management theories may be used to identify the elements of structure to be studied. Examples of such elements are leadership, tolerance of innovativeness, organizational hierarchy, decision-making processes, distribution of power, nancial management, and administrative decision-making processes.
Ch09-X2640.qxd
4/20/06
4:21 PM
Page 278
278
C H A P T E R
Outcomes Research
The second step is to evaluate the impact of various structure elements on the process of care and on outcomes. This evaluation requires comparing different structures that provide the same processes of care. In the evaluation of structures, the unit of measure is the structure. The evaluation requires access to a sufciently large sample of like structures with similar processes and outcomes, which can then be compared with a sample of another structure providing the same processes and outcomes. For example, a researcher may decide to compare various structures providing primary health care, such as the private physician ofce, the health maintenance organization (HMO), the rural health clinic, the communityoriented primary care clinic, and the nurse-managed center. Another researcher may wish to examine surgical care provided within the structures of a private outpatient surgical clinic, a private hospital, a county hospital, and a teaching hospital associated with a health science center. In each of these studies, the focus will be the impact of structure on processes of care and outcomes of care. See Figures 9-3 through 9-6 for examples of frameworks for outcomes studies with this focus.
Disease
Clinical indicators: Measurements of a patients physical and biomedical status used to infer the degree of disease
Treatment modifiers: Factors that alter outcome associated with treatment alternatives
External controls: Nonclinical factors that affect availability or use of treatment alternatives
Treatment alternatives
Economic outcomes: Total costs of medical care associated with treatment alternatives balanced against clinical or humanistic outcomes
Figure 9-3. The conceptual model: Economic, clinical, and humanistic outcome (ECHO) model. (From Kozma, C. M., Reeder, C. E., & Schulz, R. M. [1993]. Economic, clinical, and humanistic outcomes: A planning model for pharmacoeconomic research. Clinical Therapeutics, 15[6], 1125.)
Ch09-X2640.qxd
4/20/06
4:21 PM
Page 279
C H A P T E R
Outcomes Research
279
EXTERNAL ENVIRONMENT TASK ENVIRONMENT Epidemiology Personal behavior Inputs (patients) (Resources) CORE Structure Process Health outcomes (Physiological health) (Quality of life)
Environmental Biomedical Figure 9-4. A systems perspective of health services research. (From Anderson, R. M., Davidson, P. L., & Ganz, P. A. [1994]. Symbiotic relationships of quality of life, health services research and other health research. Quality of Life Research, 3[5], 367.)
HEALTH SERVICES SYSTEM Personnel Facilities and equipment Range of services Organization Management and amenities Continuity Accessibility Financing Population eligible Governance
Structure
Provision of care
Problem recognition Diagnosis Management Reassessment Persons Utilization Acceptance and satisfaction Understanding Participation
Process
Receipt of care
Outcome
Figure 9-5. The health services system. (From Vivier, P. M., Bernier, J. A., & Stareld, B. [1994]. Current approaches to measuring health outcomes in pediatric research. Current Opinions in Pediatrics, 6[5], 531.)
Ch09-X2640.qxd
4/20/06
4:21 PM
Page 280
280
C H A P T E R
Outcomes Research
Symptom amplification
Personality motivation
Values preferences
Symptom status
Functional status
Psychological supports
Nonmedical factors
Figure 9-6. Relationships among measures of patient outcome in a health-related quality-of-life conceptual model. (From Wilson, I. B., & Cleary, P. D. [1995]. Linking clinical variables with health-related quality of life: A conceptual model of patient outcomes. Journal of the American Medical Association, 273[1], 60. Copyright 1995, the American Medical Association.)
Ch09-X2640.qxd
4/20/06
4:21 PM
Page 281
C H A P T E R
Outcomes Research
281
Ch09-X2640.qxd
4/20/06
4:21 PM
Page 282
282
C H A P T E R
Outcomes Research
STRUCTURE OF CARE System characteristics Organization Specialty mix Financial incentives Workload Access/convenience Provider characteristics Age Gender Specialty training Economic incentives Beliefs/attitudes Preferences Job satisfaction Patient characteristics Age Gender Diagnosis/condition Severity Comorbid conditions Health habits Beliefs/attitudes Preferences
PROCESS OF CARE Technical style Visits Medications Referrals Test ordering Hospitalizations Expenditures Continuity of care Coordination Interpersonal style Interpersonal manner Patient participation Counseling Communication level
OUTCOMES Clinical end points Symptoms and signs Laboratory values Death Functional status Physical Mental Social Role General well-being Health perceptions Energy/fatigue Pain Life satisfaction Satisfaction with care Access Convenience Financial coverage Quality General
AU: article title? AU: Copyrighted here versus Copyright elsewhere ok?
Figure 9-7. The Medical Outcomes Study (MOS) conceptual framework. (From American Medical Association. [1989]. Journal of the American Medical Association, 262, 925930. Copyrighted 1989, American Medical Association.)
areas: the effectiveness, efciency, quality, and outcomes of health services; clinical practice, including primary care; health care technologies, facilities, and equipment; health care costs, productivity, and market forces; health promotion and disease prevention; health statistics and epidemiology; and medical liability. (Gray, 1992, p. 40)
A National Advisory Council for Health Care Policy, Research, and Evaluation also was established by Congress. The Council was required to include (1) health care researchers; (2) health care professionals (specically including nurses); (3) professionals from the elds of business, law, ethics, economics, and public policy; and (4) persons representing the interests of consumers. The budget for the AHCPR increased to $1.9 million in 1988, $5.9 million in 1989, and $37.5 million in 1990. The AHCPR initiated several major research efforts to examine medical outcomes. Two of the most signicant, described next, are the Medical Treatment Effectiveness Program (MEDTEP) and a component of MEDTEP referred to as Patient Outcomes Research Teams (PORTs) (Greene, Bondy, & Maklan, 1994).
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 283
C H A P T E R
Outcomes Research
283
A major task of PORTs was to disseminate their ndings and change the practice of health care providers to improve patient outcomes. A framework for dissemination was developed that identied the audiences for disseminated products, the media involved, and the strategies that foster assimilation and adoption of information (Goldberg, Cummings, Steinberg, Ricci, Shannon, Soumerai, et al., 1994). A Cost of Care Workgroup, consisting of a representative from each PORT, was convened in 1994 with the following four goals: (1) to determine the best methods for estimating the cost of certain conditions using claims data, (2) to evaluate methods for estimating the cost of care using billing information and patient interview data, (3) to examine methods for determining the indirect cost of care, and (4) to evaluate methods for comparing the cost of care internationally (Lave et al., 1994). In 1992, the National Center for Nursing Research (NCNR) sponsored a Conference on Patient Outcomes Research: Examining the Effectiveness of Nursing Practice. In the keynote speech, Hinshaw, then director of the NCNR, made the following suggestions:
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 284
284
C H A P T E R
Outcomes Research
From a nursing perspective, particular clinical conditions need to be identied that are more specic to nursings focus on prevention, health promotion, symptom management, and the amelioration of the effects of acute and chronic illnesses. We are all familiar with clinical conditions that are central to our practice, such as skin integrity, pain, urinary incontinence, nausea and vomiting, nutritional decits, confusion, restricted mobility, depression, fatigue, and illnessrelated stress. It will be particularly important in our research programs that we begin to both dene and rene the patient outcomes specic to interventions focused on such clinical conditions. (Hinshaw, 1992, p. 9)
Examining the impact of nursing on overall hospital outcomes requires inclusion of nursing data in the large databases used to analyze outcomes. The cost of adding new variables to these databases is high. Nursing professionals are competing with the voices of others who wish to add their own relevant variables. However, with the force of the American Nurses Association (ANA), the voice of the profession is being heard. The NCNR, now the National Institute for Nursing Research (NINR), developed a partnership with the AHCPR to fund outcomes studies of importance to nursing. Calls for proposals jointly supported by the AHCPR and the NINR are announced each year. (These calls for proposals can be found on the NINR home page on the World Wide Web: http://www.nih.gov/ninr/.) With a growing budget and strong political support, proponents of the AHCPR were becoming a powerful force demanding change in health care, because of the demand for health care reform that existed throughout the government and among the public. The role of the AHCPR, however, was the focus of considerable controversy. For example, a subset of spinal orthopedic surgeons strongly opposed the guidelines for the treatment of back pain. From 1994-1998, this group of physicians assailed members of Congress with visits, letters, and telephone calls. The AHCPR was under attack by powerful forces and was in danger of being eliminated or of having its budget greatly reduced. In fact, for the 1997 scal year budget, Congress cut AHCPR funding by $35 million, thereby ending the funding of MEDTEP and new PORTs. The following year, however, Congressional members recognized that the opposition to the agency had been based on special interests and that the contested PORT ndings were valid (Deyo, Psaty, Simon, Wagner, & Omenn, 1997; Fardon, Garn, & Saal, 1997). The intimidation of agencies and researchers by special-interest groups has serious implications for both scientists and society. Fardon, Garn, and Saal, in a letter to The New England Journal of Medicine (1997), express concern about this problem.
Harassment of researchers and funding agencies is a substantial disincentive to pursuing certain research on medical care or health risks. In effect, special-interest groups with money and power want to dene acceptable questions and shape the range of acceptable answers. Eliminating public, peer-reviewed funding would slow the production of objective knowledge, force investigators to seek funding that may not be free of conflict of interest, and leave patients, physicians, and insurers without essential scientic evidence. University faculty members are governed by nancial conflict-of-interest rules intended to prevent them from conducting research in which they or their relatives might have a nancial stake. Thus, the elimination of public research support and the intimidation of independent investigators are inimical to larger social interests. Professional societies, universities, and the government need to weigh in quickly and heavily against strategies and specic cases of intimidation and vengeful budget cuts. Inquiry may be warranted concerning the extent to which special-interest groups block or delay the publication
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 285
C H A P T E R
Outcomes Research
285
of unwanted ndings. Journals may need to make a special effort to avoid relying on otherwise highly qualied reviewers and editorialists who have nancial conflicts of interests, especially consultants to rms whose products receive negative evaluations. Journals may also need to set up defenses against potential threats of withholding advertising. When funding agencies come under attack from groups with narrow interests, prompt and unambiguous responses from universities and professional organizations are needed. Self-interested attacks must be pointed out to politicians, who may otherwise be unable to distinguish self-interested parties from disinterested ones (Deyo et al., 1997, pp. 1315-1316).
The AHCPR operated without authorization from 1995 until December 6, 1999, receiving operating funds through congressional appropriations. The reauthorization act changed the name of the AHCPR to the Agency for Healthcare Research and Quality (AHRQ). The AHRQ is designated as a scientic research agency. The term policy was removed from the agency name, to avoid the perception that this body determined federal health care policies and regulations. The word quality was added to the agencys name, establishing the AHRQ as the lead federal agency on quality of care research, with a new responsibility to coordinate all federal quality improvement efforts and health services research. The new legislation eliminated the requirement that the AHRQ develop clinical practice guidelines. However, the AHRQ still supports these efforts through EvidenceBased Practice Centers and the dissemination of evidence-based guidelines through its National Guideline Clearinghouse. The new legislation denes the AHRQs mission as follows:
Meet the information needs of its customerspatients and clinicians, health system leaders, and policymakersso that they can make more informed healthcare decisions. Build the evidence base for what works and doesnt work in healthcare and develop the information, tools, and strategies that decisionmakers can use to make good decisions and provide high-quality healthcare based on evidence. Develop scientic knowledge in these areas but will not mandate guidelines or standards for measuring quality. (One Hundred Sixth Congress of the United States, 1999, pp. 23)
The proposed budget for the AHRQ in scal year 2004 was $279 million. The United States is not the only country making demands for improvements in quality of care and reductions in costs. Many countries around the world are experiencing similar concerns and addressing them in relation to their particular government structure. Thus, the movement into outcomes research and the application of the approaches described in this chapter constitute a worldwide phenomenon.
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 286
286
C H A P T E R
Outcomes Research
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 287
C H A P T E R
Outcomes Research
287
Table 92
AU: PFRDR: caps/punctuation in all borrowed material in tables are per copy
Patient falls Patient falls with injury Pressure ulcers% of patients with documented ulcer (stage IIV on day of prevalence study. Also have Hospitalacquired ulcer% of patients with documented ulcer (stage IIV) on day of prevalence study Nurse satisfaction Nursing Hours Per Patient Day (HPPD)RN, LPN/LVN, UAPnumber of productive hours worked by nursing staff with direct patient care responsibilities Staff mixthe total number of productive hours worked by each skill mix category (RN, LPN, UAP)/total staff hours Type of unit (critical care, step down, medical, surgical and combined) Number of staffed beds designated by the hospital Agency stafftotal number of productive hours worked by contract staff Urban vs. rural category New Indicators Undergoing Pilot Testing Pediatric pain Peripheral intravenous infiltration Restraint use Patient aggression
LPN, licensed practical nurse; LVN, licensed vocational nurse; RN, registered nurse; UAP, unlicensed assistive personnel. From American Nurses AssociationNational Center for Nursing Quality. Retrieved on XXX 00, 200X, from http://nursingworld.org/quality/prtdatabase.htm
If nursing care could be compared among hospitals, nursing would have the evidence to justify claims of patient harms from the changes in RN stafng. ANA entered a new area of research and asked questions that had not previously been studied. No one knew what indicators were sensitive to the nursing care provided to patients or what relationships existed between nursing inputs and patient outcomes. They had to persuade hospitals to participate in the study at a time when hospitals had a severe case of data paranoia (fear of providing data to anyone outside the hospital because of how third parties might interpret the data). Every hospital had a different way of measuring the indicators selected by ANA. Persuading them to change to a standardized measure of the indicators for consistency across hospitals was a major endeavor (Jennings, Loan, DePaul, Brosch, & Hildreth, 2001; Rowell, 2001). Nurse researchers and cooperating hospitals instituted the mechanisms required for data collection and began multiple pilot studies. These pilot studies identied multiple obstacles to the project. They learned that not only must the indicators be measured consistently, but that data collection must be standardized. As studies continued, indicators were amplied and continue to be tested. As this testing continues, further alterations in the indicators occur (Anonymous, 1997; Campbell-Heider, Krainovich-Miller, King, Sedhom, & Malinski, 1998; Jennings et al., 2001). The ANA proposes that all hospitals collect and report on the 10 nursing-sensitive quality indicators. The ANA is working to ensure that these indicators are included in data collected by accrediting organizations and by the federal government, and that the data be shared with key groups. The ANA also is encouraging state nurses associations to lobby state legislatures to include the nursing-sensitive quality indicators into regulations or state law. In 1998, the ANA provided funding to develop a national database to house data collected using nursing-sensitive quality indicators. This database, named The National Database of Nursing Quality Indicators (NDNQI), is a program of the National Center for
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 288
288
C H A P T E R
Outcomes Research
Nursing Quality, funded by the ANA. The database is housed at the University of Kansas Medical Center Research Institute (KUMCRI) and of the University of Kansas School of Nursing. In 2001, data from nursing-sensitive quality indicators were being collected from more than 120 hospitals in 24 states across the United States. By 2005, that number had increased to 767 hospitals in the 50 states and the District of Columbia. The National Center for Nursing Quality analyzes the data quarterly and provides feedback reports to all participating hospitals. Condential benchmarking reports are provided to allow hospitals to compare their results with those of other hospitals (Rowell, 2001). In 1997, the ANA appointed members of an Advisory Committee on Community-Based Non-Acute Care Indicators to identify the rst core set of indictors for non-acute care settings. Some of the members had helped develop the Acute Care Indicators, giving the committee some continuity of the work. The committee began by selecting a theoretical base for its work: Evans and Stoddarts (1990) determinants of health model and also Donabedians model of quality. As its work progressed, the committee chose to synthesize a model to guide the identication and testing of indicators (Figure 9-8). The committee followed the acute care groups process in selecting the indicators. A hired contractor conducted the literature review, while the committee conducted focus groups and interviews with key stakeholders such as consumers of care, registered nurses, policy makers, regulators, payers, facility administrators, and purchasers. Owing to budget constraints, the ANA limited development to 10 indicators (see Table 9-3). The committee requests that all nurses and nursing organizations join with the ANA to continue to expand this work (Head, Mass, & Johnson, 2003; Sawyer et al., 2002). (For current information on the ANAs Safety & Quality Initiative, visit the following website: http://nursingworld.org/quality.) A number of studies on the effects of nursing staff mix on patient outcomes have been published recently (Blegen, Goode, & Reed, 1998; Buerhaus & Needleman, 2000; Cho, Ketean, Barkauskas, & Smith, 2003; Hall, Doran, Baker, Pink, Sidani, OBrien, et al., 2001; Houser, 2000; Needleman, Buerhaus, Mattke, Steward, & Zelevinsky, 2002a, 2002b). These studies are nding a signicant effect of stafng mix on patient outcomes.
Health outcomes
Nurse
Figure 9-8. Model used to guide identication and testing of community-based non-acute care indicators by the American Nurses Association (ANA) Advisory Committee (1997). (In Sawyer, L.M., Berkowitz, B., Larrabee, J.H., Marino, B.L., Martin, K.S., et al: Expanding American Nurses Association Nursing Quality Indicators to community-based practices. Outcomes Management, 6(2), p. 53.)
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 289
C H A P T E R
Outcomes Research
289
Table 93
Pain management (symptom severity)The treatment and prevention of pain and discomfort. Effectiveness is related to level of functioning and activities of daily living and includes measures of frequency, intensity and duration of pain symptoms. Consistency of communication (strength of therapeutic alliance)Consistent RN/advanced practice registered nurses (APRN) provider identified in the data/record. Staff mix (utilization of services)Total number of direct care hours or total number of encounters provided by RN or APRN staff who have client care responsibilities (per episode/encounter/case as appropriate to the setting) and (RNs, LPNs, UAPs caring for clients)The percent of registered nursing care hours as a total of all nursing care hours; secondary measurepercent of APRNs. Client satisfactionThe degree to which the care received met client expectations regarding nursing care, pain management, patient education and overall care. Prevention of tobacco use (risk reduction)The number of clients attending educational sessions per year provided and/or coordinated by RNs about the risks of tobacco use (includes: coordination of educational sessions/programs either with individuals or groups). Cardiovascular prevention (risk reduction)The number of clients attending educational sessions per year provided and/or coordinated by RNs about risks of cardiovascular disease. Care giver activity (protective factors)The existence or frequency of primary care giver involvement. Identification of primary care giver (protective factors) ADL/IADL (level of function)The degree to which the normal physical or entire action of a system occurs (physical or psychological). Psychosocial interaction (level of function)The degree to which the normal action of a system occurs.
ADL/IADL, activities of daily living/instrumental activities of daily living; LPN, licensed practical nurse; LVN, licensed vocational nurse; RN, registered nurse; UAP, unlicensed assistive personnel. From American Nurses Association. Retrieved on April 1, 2006, from http://nursingworld.org/readroom/Go to nursing sensitive indicators for community-based nonacute care settings.
The most signicant of these studies, conducted by Needleman et al. (2002a, 2002b), used discharge and stafng data from 799 hospitals in 11 states to estimate nurse stafng levels for RNs, LPN/LVNs, and aides, as well as the frequency of a wide range of complications developed by patients during their hospital stay. The data cover 6 million patients discharged from hospitals in 1997. These investigators found that low levels of RN stafng among a hospitals nurses were associated with higher rates of serious complications such as pneumonia, upper gastrointestinal bleeding, shock, and cardiac arrest, including deaths among patients with these three complications, as well as sepsis or deep vein thrombosis. These complications occurred 3% to 9% more often than in hospitals with lower levels of RN stafng.
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 290
290
C H A P T E R
Outcomes Research
comes can only be fully achieved in the presence of other features in the practice environment, then it will be the presence or absence of these features and not solely stafng levels that will produce the desired patient results. Without a clear understanding of the circumstances under which stafng affects outcomes, we lack the capacity to improve patient outcomes if efforts are directed only at changing stafng levels. (p. 11)
Standing, Anthony, and Hertz (2001) conducted a triangulated study of outcomes after delegation to unlicensed assistive personnel (UAP), funded by the National Council of State Boards of Nursing. This report describes the qualitative analysis of interviews of RNs who described a delegation with a positive outcome and a delegation with a negative outcome. Negative outcomes after delegation ranged from family or client upsets, to fractures or other injuries, to death. In some cases, the UAP performed activities that had not been delegated to them. Negative outcomes were most frequently due to the UAPs not receiving or following directions, or not adhering to established policy. Positive outcomes included enhanced client well-being as indicated by increased socialization and other measures, prevention of poor client outcomes, and enhanced unit functioning.
Stetler, Morsi, and Burns (2000) have worked to develop a comprehensive, in-depth prole of nursing-sensitive outcomes of hospital nursing care at the unit level and to use the information in routine quality monitoring. They used a prevention framework based on the work of Stetler and DeZell (1989). The framework describes the nurses role in preventing complications in a nosocomial hospital environment; treatment consequences; a patients health status, disease state, or evolving condition; and the patients inability to care for themselves safely. From a safety perspective, the framework classies outcomes as positive or negative. Outcomes are further classied in terms of preventability, impact, severity, and a holistic view of patient safety. Positive behaviors protect or rescue patients from potential or actual negative events. These actions are categorized as (1) detection/reporting, (2) detection/ prevention, and (3) facilitation of resolution/prevention.
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 291
C H A P T E R
Outcomes Research
291
Other work has included development of a model of nursing effectiveness to use in studies of patient outcomes.
Irvine, Sidani, and Hall (1998) have developed The Nursing Role Effectiveness Model (Figure 9-9) to guide the examination and explanation of the links between nursing processes and patient outcomes. The model is based on Donabedians theory of quality care. Roles are dened as
positions in organizations that have attached to them a set of expected behaviors. Professional roles are complex because they consist of components that are based on normative expectations concerning standards of practice that have been established by external regulatory bodies and secondly, on normative expectations that have evolved over time that are unique to the organization. (p. 59)
The Nursing Role Effectiveness Model has three major components: structure, the nurses role, and patient/health outcomes. Structure has three subcomponents: nurse, organizational, and patient. Nurse variables that influence quality of nursing care include factors such as experience level, knowledge, and skill level. Organizational components that can affect quality of nursing care include staff mix, workload, and assignment patterns. Patient characteristics that can affect quality of care include health status, severity, and morbidity. Nurses role has three subcomponents: nurses independent role, nurses dependent role, and nurses interdependent role. Independent role functions include assessment, diagnosis, nurse initiated interventions, and follow-up care. The patient/health outcomes of the independent role are clinical/symptom control, freedom from complications, functional status/self-care, knowledge of disease and its treatment, satisfaction and costs. The dependent role functions include execution of medical orders and physician-initiated treatments. It is the dependent role functions that can lead to patient/health outcomes of adverse events. Interdependent role functions include communication, case management, coordination of care, and continuity/monitoring and reporting. The interdependent role results in team functioning and affects the patient/health outcomes of the independent role. The Propositions of The Nursing Role Effectiveness Model were stated as follows (Irvine, Sidani, & Hall, 1998):
Nursings capacity to engage effectively in the independent, dependent, and interdependent role functions is influenced by individual nurse variables, patient variables, and organizational structure variables. (p. 61) Nurses interdependent role function depends upon the nurses ability to communicate and articulate her/his opinion to other members of the health care team. (p. 61) Nurse, patient, and system structural variables have a direct effect on clinical, functional, satisfaction, and cost outcomes. (p. 61) Nurses independent role function can have a direct effect on clinical, functional, satisfaction, and cost outcomes. (p. 61) Medication errors and other adverse events associated with nurses dependent role function can ultimately affect all categories of patient outcome. (p. 62) Nursings interdependent role function can affect the quality of interprofessional communication and coordination. The nature of inter-professional communication and coordination can influence other important patient outcomes and costs such as risk-adjusted length of stay, risk-adjusted mortality rates, excess home care costs following discharge, unplanned visits to the physician or emergency department, and unplanned re-hospitalization. (p. 62)
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 292
292
C H A P T E R
Outcomes Research
Structure
Patient/Health Outcomes Clinical/symptom control Freedom from complications Functional status/self-care Knowledge of disease and its treatment satisfaction costs
Nurses Dependent Role Organizational Staff mix Workload Assignment pattern Execution of medical orders Physician-initiated treatments Adverse events
Nurses Interdependent Role Patient Health status Severity Morbidity Communication Case management Coordination of care Continuity/monitoring and reporting
Team functioning
Figure 9-9. The Nursing Role Effectiveness Model. (From Irvine, D., Sidani, S., & Hall, L. M. [1998]. Linking outcomes to nurses roles in health care. Nursing Economics, 16[2], 59.)
Sidani and Irvine (1999) subsequently modied The Nursing Role Effectiveness Model to evaluate the nurse practitioner role in acute care settings, providing a guide for nurse practitioners interested in studying their roles. These investigators selected variables relevant to acute care nurse practitioner (ACNP) practice to operationalize the components of the model, and proposed relationships among the elements of the three components. Using this framework, the research group examined the organizational factors influencing nurse practitioners role implementation in acute care settings (Irvine et al., 2000). They also conducted a study examining the practice patterns of ACNPs (Sidani et al., 2000). Practice patterns of ACNPs also have been examined by Rosenfeld, McEvoy, and Glassman (2003). Organizational changes resulting from implementing the APNs role in acute care were studied by Cummings, Fraser, and Tarlier (2003). In 2002, Doran, Sidani, Keatings, and Doidge conducted an empirical test of The Nursing Role Effectiveness Model. These investigators found that The Nursing Role Effectiveness Model was effective in guiding the evaluation of outcomes of nursing care. They noted that for the most part the hypothesized relationships among the variables were supported. However, further work is needed to develop an understanding of how nurses engage in their coordinating role functions and how we can measure these role activities (p. 30).
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 293
C H A P T E R
Outcomes Research
293
allow comparisons with other practitioners. Nurse-sensitive outcomes that document effectiveness of specic interventions have yet to be identied, however (Kleinpell-Nowell & Weiner, 1999). Computerized electronic patient records make the study of clinical practice considerably easier, but the use of these electronic records varies widely among APNs. APNs are very familiar with Current Procedural Terminology (CPT) codes and International Classication of Diseases-9 (ICD-9) codes. However, most are unfamiliar with standardized nursing languages, such as Nursing Outcomes Classication (NOC) and Nursing Interventions Classication (NIC), which tend to be used in electronic databases. Still, a few studies of APN practice are beginning to appear in the literature.
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 294
294
C H A P T E R
Outcomes Research
translate research ndings into primary care practice (McCloskey, Grey, Deshefy-Longhi, & Grey, 2003, p. 39). Initial data gathering has been primarily descriptive and examines the characteristics of practitioners participating in the network. A series of studies is planned to determine how APRN practices operate, how APRN services are determined and billed, and what clinical outcomes are obtained
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 295
C H A P T E R
Outcomes Research
295
11. Determine the clinical signicance of improvement, as well as the statistical signicance. 12. Determine the cost-benet ratio and cost-effectiveness of the treatments under evaluation. 13. Use decision analyses to synthesize information about patients outcomes and preferences for various types of outcomes. 14. Disseminate information to both patients and health care providers about which persons would and which would not benet from the procedure. 15. Conduct a clinical trial to evaluate the effects of the intervention. 16. Incorporate ndings into treatment guidelines. 17. Modify provider and patient behavior so that proven, effective treatment is given to persons who are most likely to benet. The PORTs recognized the need to allow diversity in research strategies, measures, and analyses to facilitate methodological advances (Fowler, Cleary, Magaziner, Patrick, & Benjamin, 1994). Creative flexibility often is necessary to develop ways to answer new questions. Finding ways to determine the impact of a condition on a persons life is difcult. Interpreting results also can be problematic, because clinical signicance is considered as important as statistical signicance. This issue requires a judgment by the research team about what constitutes clinical signicance in that particular area of study. This section describes some of the sampling issues, research strategies, measurements, and statistical approaches being used by researchers in outcomes studies. The descriptions provided are not sufcient to guide the researcher in using the approaches described but rather provide a broad overview of a variety of methods being used. For additional information, refer to the citations for each topic. Outcomes studies cross a variety of disciplines; thus, the emerging methodology is being enriched by a cross-pollination of ideas, some of which are new to nursing research.
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 296
296
C H A P T E R
Outcomes Research
lacking in credibility (Orchard, 1994). Using this argument, traditionalists claim that the ndings of most outcomes studies are not valid and should not be used as a basis to establish guidelines for clinical practice or to build a body of knowledge. Slade, Kuipers, and Priebe (2002) suggest that
research questions are designed so that they can be answered by Randomized Controlled Trials (RCTs). Specically, the use of RCTs involves the identication of an intervention which is given to patients in the experimental group, but not the control group. This encourages the asking of particular types of research questions, typically of the form Does intervention X work for disorder Y? However, one might argue that the RCT methodology limits the questions that can be asked, and hence can restrict the potential ndings from research. Furthermore, if different questions were being asked, the RCTs would not always be the best methodology to employ. The question Which patients with condition Y does intervention X work for? may prove to have more clinical relevance, and answering this question may involve asking the question How does intervention X work?, a question which cannot be answered just by using RCTs. (pp. 1213)
Clinical data
Administrative data
Large administrative databases Insurance claims databases Tumor or disease registries Vital statistics databases
Figure 9-10. Types of databases emanating from patient care encounters. (From Lange, L. L., & Jacox, A. [1993]. Using large databases in nursing and health policy research. Journal of Professional Nursing, 9[4], 204.)
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 297
C H A P T E R
Outcomes Research
297
1991). An example is the Medicare database managed by the Health Care Financing Administration (HCFA). These large administrative databases can be used to determine the incidence or prevalence of disease, geographic variations in medical care utilization, characteristics of medical care, outcomes of care, and complementarity with clinical trials. Wray and colleagues (1995) caution, however, that analyses should be restricted to outcomes specic to a particular subgroup of patients, rather than one adverse outcome of all disease states. Problems with the quality of data in the large databases are well recognized. The data have been gathered and entered by hundreds of people in a variety of settings. Few quality checks on the data are performed, and within the same data sets, records may have different lengths and structures. Missing data are common. Sampling and measurement errors are inherent in all large databases. Sampling error is a result of the way in which cases are selected for inclusion in the database; measurement error emerges from problems related to the operational denition of concepts. Thus, reliability and validity of the data are concerns (Davis, 1990; Lange & Jacox, 1993). Large databases are used in outcomes studies to examine patient care outcomes. The outcomes that can be examined are limited to those recorded in the database and thus tend to be rather general. Existing databases can be used for analyses such as assessment of nursing care delivery models, variation in nursing practices, or evaluation of patients risk of hospitalacquired infection, hospital-acquired pressure ulcer, or falls. Lange and Jacox (1993) identify the following important health policy questions related to nursing that should be examined through the use of large databases:
1. 2. 3. 4. What is standard nursing practice in various settings? What is the relationship between variations in nursing practice and patient outcomes? What are the effects of different nursing staff mixes on patient outcomes and costs? What are the total costs for episodes of treatment of specic conditions, and what part of those are attributable to nursing care? 5. Who is being reimbursed for nursing care delivery? (Lange & Jacox, p. 207)
To examine these questions, nurses must develop the statistical and methodological skills needed for working with large databases. Large databases contain patient and institutional information from huge numbers of patients. They exist in computer-readable form, require special statistical methods and computer techniques, and can be used by researchers who were not involved in the creation of the database. Regrettably, nursing data are noticeably missing from these large databases and hence from funded health policy studies using them. A nursing minimum data set has been repeatedly recommended for inclusion in these databases (Werley, Devine, Zorn, Ryan, & Westra, 1991; Werley & Lang, 1988; Zielstorff, Hudgings, Grobe, & the National Commission on Nursing Implementation Project Task Force on Nursing Information Systems, 1993). This minimum data set would comprise a set of variables necessary and sufcient to describe an episode of illness or the care given by a provider. The ANA has mandated the formation of a Steering Committee on Databases to Support Clinical Nursing Practice. The following nursing classication schemes are being used in national databases:
The North American Nursing Diagnosis Association (NANDA) classication The Omaha System: Applications for Community Health Nursing classication The Home Health Care Classication The Nursing Interventions Classication (NIC)
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 298
298
C H A P T E R
Outcomes Research
The Nursing Outcomes Classication (NOC) Temple (1990) expressed the following concerns regarding the use of large data sets, rather than controlled trials, to assess effectiveness of treatments:
We have traveled this route before with uncontrolled observations. It has always been hoped, and has often been asserted, that uncontrolled databases can be adjusted in some way that will allow valid comparisons of treatments. I know of no systematic attempt to document this. Outcomes researchers counter these criticisms by pointing out that experimental studies lack external validity and are not useful for application in clinical settings. They claim that the ndings of clinical trials are not being used by clinicians because they are not representative of the patients seeking care. (p. 211)
Consensus knowledge building Practice pattern proling Prospective cohort studies Retrospective cohort studies Population-based studies Clinical decision analysis Study of the effectiveness of interdisciplinary teams Geographical analyses Economic studies Ethical studies Dening and testing of interventions
senting a variety of constituencies. Initially, an extensive international search of the literature on the topic of concern, including unpublished studies, studies in progress, dissertations, and theses, is conducted. Several separate reviews may be performed, focusing on specic questions about the outcomes of care, diagnosis, prevention, or prognosis. Because meta-analytic methods often cannot be applied to the literature pertinent to PORTs, systematic approaches to critique and synthesis have been developed to identify relevant studies and gather and analyze data abstracted from the studies (Powe et al., 1994). The results are dispersed to researchers and clinical experts in the eld, who are asked to carefully examine the material and then participate in a consensus conference. The consensus
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 299
C H A P T E R
Outcomes Research
299
conference yields clinical guidelines, which are published and widely distributed to clinicians. The clinical guidelines also are used as practice norms to study process and outcomes in that eld. Gaps in the knowledge base are identied and research priorities determined by the consensus group. Preliminary steps in this process may include conducting extensive integrative reviews and seeking consensus from a multidisciplinary research team and locally available clinicians. A review can be accomplished by establishing a website and conducting dialogue with experts via the Internet. The review may then be published in Sigma Theta Taus online journal, Knowledge Synthesis in Nursing, and then dialogue related to the review may be conducted over the Internet. The Delphi method also has been used to seek consensus (Vermeulen, Ratko, Erstad, Brecher, & Matuszewski, 1995).
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 300
300
C H A P T E R
Outcomes Research
for a particular nurse, group of nurses, or group of nursing units. Proling can be used for quality improvement, assessment of provider performance, and utilization review. Methods of improving outcomes are not addressed by proling, although this process can identify problem areas. It can be used to determine how and by whom performance should be changed to improve outcomes. Proling also can identify outliers (persons with extreme scores or values), allowing more detailed examination of the reasons for the disparity of data. The databases currently being used for proling are not ideal, because they were developed for other purposes. Outcomes that can be examined are limited to broad outcomes, such as morbidity and mortality rates, complication rate, readmission rate, and frequency of utilization of various services (Lasker et al., 1992; McNeil et al., 1992). Table 9-4 lists examples of the large database measures that might be used in proling.
Preventive
Portion of population in specific age and sex groups receiving recommended tests or procedures
National recommendation
Diagnosis
% of population diagnosed (and under care) for specific chronic conditions by age and sex Medications Average number of new prescriptions per person per year Surgery Rate of surgical procedures per year: total, inpatient, and ambulatory (if applicable) Hospital readmissions within 3 months of discharge
Treatment
Average number of new prescriptions for antibiotics per person per year Cesarean section rate for all deliveries % of readmissions for some condition % of readmission identifying a complication
Outcomes
Reproduced in part from Steinwachs, D. M., Weiner, J. P., & Shapiro, S. (1989). Management information systems and quality. In N. Goldfield & D. B. Nash (Eds.), Providing quality care: The challenge to clinicians (pp. 160180). Philadelphia: American College of Physicians.
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 301
C H A P T E R
Outcomes Research
301
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 302
302
C H A P T E R
Outcomes Research
Direct recall: the subject accesses the memory without having to think or search memory resulting in correct information. Indirect recall: the subject accesses the memory after thinking or searching memory, resulting in correct information. Limited recall: access to the memory does not occur but information that suggests the contents of the memory is accessed, resulting in an educated guess. No recall: neither the memory nor information relevant to the memory may be accessed, resulting in a wild guess.
Population-Based Studies
Population-based studies also are important in outcomes research. Conditions must be
studied in the context of the community, rather than of the medical system. To avoid selection bias with this method, all cases of a condition occurring in the dened population are included, rather than only patients treated at a particular health care facility. Efforts may be made to include people with the condition who have not received treatment. Community-based norms of tests and survey instruments obtained in this manner provide a clearer picture of the range of values than does evaluation of the limited spectrum of patients seen in specialty clinics. Estimates of instrument sensitivity and specicity are more accurate. This method is useful in elucidating the natural history of a condition or in identifying the long-term risks and benets of a particular intervention (Guess, Jacobsen, Girman, Oesterling, Chute, Panser, et al., 1995).
possible diagnostic and management courses of action, assessing the probability and value of various outcomes, and then calculating the optimal course of action. Decision analysis is based on the following four assumptions: (1) decisions can be quantied; (2) all possible courses of action can be identied and evaluated; (3) the different values of outcomes, viewed from the perspective of the nurse, patient, payer, and administrator, can be examined; and (4) the analysis allows selection of an optimal course of therapy. To perform the analysis, the researchers must dene the boundaries of the clinical in terms of a logical sequence of events over time. All possible courses of action are then determined. These courses of action usually are represented in a decision tree consisting of a starting point, available alternatives, probable events, and outcomes. Next, the goals and objectives of problem resolution are dened. Researchers calculate the probability of occurrence of each path of the decision tree and ensure that an outcome exists for each potential path. Each outcome is assigned a value. These values may be expressed in terms of money, morbidity incidents, quality-of-life measures, or duration of hospital stay. (Figure 9-11 displays a simplied decision tree for breech delivery in obstetrics.) Researchers can then identify an optimal course of action according to which decision maximizes the chances of the most desirable outcomes (Crane, 1988; Keeler, 1994; Sonnenberg, Roberts, Tsevat, Wong, Barry, & Kent, 1994). Studies analyzing clinical decisions have primarily used questionnaires and interviews. However, determining the clinical decisions of practitioners is not an easy task. Much of
A a s h c n c p
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 303
C H A P T E R
Outcomes Research
303
ECV + T.O.L.
ECV + C/S
Selected T.O.L.
Schedule C/S
C/S
(.10)
Cephalic, C/S
Cephalic, C/S
Cephalic, C/S
Figure 9-11. Simplied decision tree for breech delivery. Numbers in parentheses refer to estimated probability of event. (From Keeler, E. B. [1994]. Decision analysis and cost-effectiveness analysis in womens health care. Clinical Obstetrics and Gynecology, 37[1], 208.)
patient care involves the clinician and the patient alone. The underlying theories of care and the processes of care are hidden from view. Thus, it is difcult for clinicians to compare their specic approaches to care. Among physicians, care delivered by other physicians is rarely observed (OConnor et al., 1993). Studies have found that physicians have difculty recalling their decisions and providing rationales for them (Chaput de Saintonge & Hattersley, 1985; Kirwan, Chaput de Saintonge, Joyce, Holmes, & Currey, 1986). Unsworth (2001) describes the following strategies for studying clinical decision making:
semistructured interviews audio-assisted recall (the clinician listens to an audiotape of the clinician-client dialogue and uses this to aid recall of her or his reasoning processes) video-assisted recall (the clinician uses video footage to prompt recall of reasoning processes) the clinician writing notes as he or she solves a problem the think-aloud method (the clinician provides verbal commentary during interaction with the client) the clinician presents the reasoning about a clinician-client session afterward from memory use of a head-mounted video camera with video-assisted recall.
ED: JGENZ: if this is not direct quote, please re-code as Bulleted list indent and cap 1st word of entries
AU: if this list is not a direct quote, suggest rewording here as follows: contemporaneous note taking (the clinician problem)
Chaput de Saintonge and associates (1988) propose a strategy for analyzing clinical decisions using paper patients. The techniques seem to parallel the decisions made by
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 304
304
C H A P T E R
Outcomes Research
practitioners in the clinical setting. In their study, 10 common clinical variables used to evaluate the status of patients with rheumatoid arthritis were collected at two points for 30 patients participating in a clinical trial, at the time of entry and 1 year later. Data for 20 of the patients were duplicated throughout the table to check the consistency of responses, making 50 responses in all. The variables were presented to rheumatologists on a single sheet of paper labeled before and after a year. Physicians were asked to indicate the extent of change in each patients condition using a visual analogue scale (VAS) with the ends labeled greatest possible deterioration and greatest possible improvement. They also were asked whether they considered the change clinically important. Then they were asked to indicate the relative importance of each variable, rating the variables on a scale of 1 to 100. Regression analyses were performed in which each VAS variable was used as the dependent variable. With increasing VAS values, judgments of clinical importance changed from not important to important. This change occurred over a 5-mm length of the scale or less. (VAS scales traditionally are 100 mm in length.) The researchers designated the midpoint of this transition zone as the threshold value of clinical importance. Consistency of responses was tested to correlate responses with those of duplicate cases. The researchers then developed a consensus model by weighing each physicians responses on the basis of the correlation. The VAS scores were multiplied by the correlation coefcient. These VAS scores were then used as the dependent variable in another regression analysis. This method is useful in identifying the variables important in the making of clinical decisions and the consistency with which practitioners make their decisions.
Part of the communication process consists of regularly scheduled face-to-face meetings. The assumption is that collaborative team approaches provide more effective care than that delivered using team approaches or noncollaborative multidisciplinary approaches (parallel care). Interdisciplinary teams are becoming more common as health care changes. Examples are hospice care teams, home health teams, and psychiatric care teams. Studying the effectiveness of interdisciplinary teams is difcult, however. The characteristics that make team care more effective have not been identied. Studies usually focus on the evaluation of a single team, rather than on conducting comparison studies. The outcomes of team care also are multidimensional, requiring the use of multiple dependent variables. Evaluation studies examining team care often examine only posttreatment data without baseline data. Comparison of groups will not reveal any evidence that the groups were similar in terms of important variables before the intervention. Involvement of family
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 305
C H A P T E R
Outcomes Research
305
members with the team has not been examined. Clearly, this is an important focus of research requiring more rigorous designs than have previously been used.
Geographical Analyses
Geographical analyses are used to examine variations in health status, health services, patterns of care, or patterns of use by geographical area and sometimes are referred to as small area analyses. Variations may be associated with sociodemographic, economic, medical, cultural, or behavioral characteristics. Locality-specic factors of a health care system, such as capacity, access, and convenience, may play a role in explaining variations. The social setting, environment, living conditions, and community also may be important factors. The interactions between the characteristics of a locality and of its inhabitants are complex. The characteristics of the total community may transcend the characteristics of persons within the community and may influence subgroup behavior. High education levels in the community commonly are associated with greater access to information and receptiveness to ideas from outside the community. Regression analyses commonly are used to develop models using all of the risk factors and the characteristics of the community. Results often are displayed through the use of maps (Kieffer, Alexander, & Mor, 1992). After the analysis, the researcher must determine whether differences in rates are due to chance alone and whether high rates are too high. From a more theoretical perspective, the researcher must then explain the geographical variation uncovered by the analysis (Volinn, Diehr, Ciol, & Loeser, 1994). Geographical information systems (GISs) can provide an important tool for performing geographic analyses. A GIS uses relational databases to facilitate processing of spatial information. The software tools in a GIS can be used for mapping, data summaries, and analysis of spatial relationships. GISs have the capability of modeling data flows so that the effect of proposed changes in interventions applied to individuals or communities on outcomes can be modeled (Auffrey, 1998).
Economic Studies
Many of the problems studied in health services research address concerns related to the efcient use of scarce resources and, thus, to economics. Health economists are concerned with the costs and benets of alternative treatments or ways of identifying the most efcient means of care. The economists denition of efciency is the least-cost method of achieving a desired end with the maximum benet to be obtained from available resources. If available resources must be shared with other programs or other types of patients, an economic study can determine whether changing the distribution of resources will increase total benet or welfare. To determine the efciency of a treatment, the economist conducts a cost-effectiveness analysis. This technique uses a single measure of outcomes, and all other factors are expressed in monetary terms as net cost per unit of output (Ludbrook, 1990). Cost-effectiveness analyses compare different ways of accomplishing a clinical goal, such as diagnosing a condition, treating an illness, or providing a service. The alternative approaches are compared in terms of costs and benets. The purpose is to identify the strategy that provides the most value for the money. Tradeoffs between costs and benets are unavoidable, however (Oster,
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 306
306
C H A P T E R
Outcomes Research
1988). Stone (1998) describes the methodology for performing a cost-effectiveness analysis. It is time for nurses to take a more active role in conducting cost-effectiveness research. Nurses are well positioned to evaluate health care practices and have the incentive to conduct the studies. Nursing practice is seldom a subject of cost-effectiveness analyses. Nevertheless, such knowledge would enable nurses to rene their practice, substituting interventions that maximize nurses time to the best advantage, in terms of the patients health, for interventions that offer less gain (Siegel, 1998). As Lieu and Newman (1998) point out:
[C]ost effective does not necessar[il]y mean cost-saving (Doubilet, Weinstein, & McNeil, 1986). Many health interventions, even preventive ones, do not save money (Tengs, Adams, Pliskin, Safran, Siegel, Weinstein, et al., 1995). Rather, a service should be called cost-effective if its benets are judged worth the costs. Recently, a consensus panel supported by the National Institutes of Health published recommendations that dene standards for conducting costeffectiveness analysis (Gold, Siegel, Russell, & Weinstein, 1996). Cost-effectiveness analysis is only one of several methods that can be used for the economic evaluation of health services (Drummond, Stoddart, and Torrance, 1987). Although these methods are useful, an intervention cannot be cost-effective without being effective (Lieu & Newman, 1998, p. 1043).
To examine overall benets, a cost-benet analysis is performed. With this method, the costs and benets of alternative ways of using resources are assessed in monetary terms, and the use that produces the greatest net benet is chosen. The costs included in an economic study are dened in exact ways. The actual costs associated with an activity, not prices, must be used. Cost is not the same as price. In most cases, price is greater than cost. Costs are a measure of the actual use of resources, rather than the price charged. Charges are a poor reflection of actual costs. Costs typically included in a cost-benet analysis are costs to the provider, costs to third-party payers (e.g., insurance), out-of-pocket costs, and opportunity costs. Out-of-pocket costs are those expenses incurred by the patient or family members, or both, that are not reimbursable by the insurance company. Examples are costs of buying supplies, dressings, medications, and special food, transportation expenses, and unreimbursable care expenses. Opportunity costs are lost opportunities that the patient, family member, or others experience. For example, a family member who must stay at home to care for the patient may lose the opportunity to earn more money. A teenager who needs to drop out of school for a semester to care for a parent may lose the opportunity to advance her education. A husband might have been able to take a better job if the family could have moved to another town, rather than staying in place to enable a member to receive specic medical care. Opportunity costs often are not included in the consideration of overall costs. This omission results in an underestimation of costs and an overestimation of benet. For example, caring for an acutely ill patient at home is cost-effective if out-of-pocket costs and opportunity costs are not considered. However, the total costs of providing the care, regardless of who pays or who receives the money, must be included. In performing such a study, it is important to state whose costs are being considered and who is to weigh the benets against the consequences. Allred, Arford, Mauldin, and Goodwin (1998) critiqued the seven nursing studies between 1992 and 1996 in which cost-effectiveness analyses were performed. They found
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 307
C H A P T E R
Outcomes Research
307
these studies to be equivalent in quality to those from other disciplines. They concluded that more emphasis must be placed on cost-effectiveness analyses in nursing research, and they provided guidelines for conducting these studies. Stone (1998) has described the recommended guidelines for journal reports of costeffectiveness analyses. Cost-effectiveness studies should be used as aids in decision making, rather than as the end decision. If a cost-effectiveness study is conducted to inform those who make resource allocation decisions, a standard reference case should be presented to allow the decision makers to compare a proposed new health intervention with existing practice.
Ethical Studies
Outcomes studies often result in the development of policies for the allocation of scarce resources. Ethicists take the position that moral principles, such as justice, must be considered as constraints on the use of costs and benets, to choose treatments that maximize the benet per unit cost. Value commitments are inherent in choices about research methods and about the selection and interpretation of outcome variables, and these commitments should be acknowledged by researchers.
The choices researchers make should be documented and the reasons for those choices should be given explicitly in publications and presentations so that readers and other users of the information are enabled and expected to bear more responsibility for interpreting and applying the ndings appropriately. (Lynn & Virnig, 1995)
Veatch (1993) proposes that analysis of the implications of rationing decisions in terms of the principles of justice and autonomy will provide more acceptable criteria than outcomes predictors alone. As an example, Veatch performs an ethical analysis of the use of outcome predictors in decisions related to early withdrawal of life support. Ethical studies should play an important role in outcomes programs of research.
Measurement Methods
The selection of appropriate outcome variables is critical to the success of a study (Bernstein & Hilborne, 1993). As in any study, evidence of validity and reliability of the methods of measurement must be evaluated. Outcomes selected for nursing studies should be those most consistent with nursing practice and theory (Harris & Warren, 1995). In some studies, rather than selecting the nal outcome of care, which may not occur for months or years, measures of intermediate end points are used. Intermediate end points are events or markers that act as precursors to the nal outcome. It is important, however, to document the validity of the intermediate end point in predicting the outcome (Freedman & Schatzkin, 1992). In early outcomes studies, researchers selected outcome measures that could be easily obtained, rather than those most desirable for outcomes studies. Table 9-5 identies characteristics important to evaluate in selecting methods of measuring outcomes. In evaluating a particular outcome measure, the researcher should consult the literature for previous studies that have used that particular method of measurement, including the publication describing development of the method of measurement. Information related to the measurement can be organized into a table such as Table 9-6, allowing easy comparison of several methods of measuring a particular outcome.
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 308
308
C H A P T E R
Outcomes Research
Leidy, 1991 Nelson, Landgraf, Hays, Wasson, & Kirk, 1990 Stewart et al., 1989 Lohr, 1988 Bombardier & Tugwell, 1987 Feinstein et al., 1986 Kirshner & Guyatt, 1985 Deyo, 1984
Comprehensiveness
Deyo, 1984
Reliability
Nelson et al., 1990 Spitzer, 1987 Guyatt et al., 1987 Deyo, 1984
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 309
C H A P T E R
Outcomes Research
309
Responsiveness
Stewart & Archbold, 1992 Leidy, 1991 Jaeschke, Singer, & Guyatt, 1989 Guyatt et al., 1989 Bombardier & Tugwell, 1987 Guyatt et al., 1987 Deyo & Centor, 1986 Deyo, 1984
From Harris, M. R., & Warren, J. J. (1995). Patient outcomes: Assessment issues for the CNS. Clinical Nurse Specialist, 9(2), 82.
Outcomes researchers are moving away from classical measurement theory as a means of evaluating the reliability of measurement methods. They are interested in identifying change in measures over time in a subject, and instruments developed through the use of classical measurement theory are often not sensitive to these changes. The magnitude of change that can be detected also is important to determine. In addition, measures may detect change within a particular range of values but may not be sensitive to changes outside that range. The sensitivity to change of many commonly used outcome measures has not been examined (Deyo & Carter, 1992; Felson, Anderson, & Meenan, 1990). Studies must be conducted specically to determine the sensitivity of measures before they are used in outcomes studies. As the sensitivity of a measure increases, statistical power increases, allowing smaller sample sizes to detect signicant differences. Creative methods of collecting data on instruments for large outcomes studies must be explored. In a busy ofce or clinic setting, the typical strategy of having clerks or other staff administer questionnaires or scales to patients is time intensive and costly and may result in lost data. Greist and colleagues (1997) recommend using the computer and the telephone to collect such data. Computers containing the instrument can be placed in locations convenient to patients, so the instrument can be completed with a minimum of staff involvement. Another option is telephone interviews using the computer. The traditional telephone interview using interviewers to ask questions is costly. The same interactive voice response (IVR) technology used in voice mail, however, can be used in telephone interviewing by computer. IVR allows the patient to respond to yesno and multiple-choice questions by pressing numbers on the keypad or by saying yes or no or a number from 0 to 9. Patients can record answers in their own voice.
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 310
310
C H A P T E R
Outcomes Research
Table 96 Characteristic
AU: throughout table: should first reference in entry align with first decriptor in column 1? [versus characteristic itself]
Characteristics of the Katz Activities of Daily Living (ADL) Scale: A Proposed Outcome Instrument
Reference(s)
Applicability Purpose is to objectively evaluate results of treatment in chronically ill and aging populations Predicts service utilization in elderly population Used in case-mix adjustments Scale discriminates well on disability in elderly population, norms easily referenced Ratings judgment based on direct observation and caregiver reports, known differences in observed vs. reported ratings Practicality Brief, 6 items with 3 levels of dependency Can be used by clinicians and non-clinicians Measures performance (not ability) Aggregate score represents increasing level of dependency Comprehensiveness Includes bathing, dressing, toileting, transfer, continence, and eating Does not explain etiology of level of performance Reliability Performance may be influenced by motivational, social, and environmental factors High internal consistency reported Validity Content and construct validity assessments are acceptable
Katz et al., 1970 Winner et al., 1990 Fries, 1990 Spector, 1990 Spector, 1990 Burns, 1992 Katz et al., 1970 Spector, 1990 Katz et al., 1970 Spector, 1990 Katz et al., 1970 Kane & Bayer, 1991
Responsiveness No published reports that quantify relationship of scale change to minimal clinically important change
From Harris, M. R., & Warren, J. J. (1995). Patient outcomes: Assessment issues for the CNS. Clinical Nurse Specialist, 9(2), 85.
Measuring the frequency and nature of care activities of various staff has been problematic in studies of the process of care. Strategies commonly used are chart review, time and motion studies, work sampling, and retrospective recall. None of these is a satisfactory indicator of the actual care that occurs (Hale, Thomas, Bond, & Todd, 1997). Holmes, Teresi, Lindeman, and Glandon (1997) recommend the use of barcode methodology to measure service inputs. Scanning the barcodes captures what care is provided, for whom, by whom, and at what time. Barcoded service sheets and a portable barcode reader are used with an accompanying database management system.
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 311
C H A P T E R
Outcomes Research
311
values is a result of interindividual variation. Patient change, however, results in changes within the subject. With classical measurement theory analysis, a measure that did not vary between individual subjects is considered to have zero (or poor) reliability. This measure, however, may be an excellent measure of change over time if individual subjects change on that measure (even if group averages do not change much). Thus, it is inappropriate to assess the reliability of difference scores according to the internal consistency of measures (Collins & Johnston, 1995). Some outcomes researchers use measures obtained from individual subjects as indicators of characteristics of a group. The data from the measures are aggregated to reflect the group. In this case, the researcher must assess the extent to which the responses represent the group. Although the group mean usually is expected to serve this purpose, it may not adequately represent the group. Verran, Mark, and Lamb (1992) describe techniques for examining the psychometric properties of instruments used to describe group-level phenomena. Items of the instrument should be assessed for content validity to determine how well they measure group-level concepts. Reliability and validity must be assessed at the aggregated level, rather than at the individual level. Commonly, multiple outcomes measures are used in outcomes studies. Researchers wish to evaluate all relevant effects of care. However, quantity of measures is not necessarily evidence of the quality of the measures. Researchers should select the measures most relevant to the treatment, avoiding measures that are closely correlated. Interpreting the results of studies in which multiple outcomes have been used can be problematic. For example, Felson and colleagues (1990, p. 141) ask which is the better therapy, the one that shows a change in 6 outcome measures out of 12 tested or the one that shows a change in 4 of the 12 measures? What if the 4 that demonstrate change with one therapy are not the same as the 6 that show a change in another therapy? If multiple comparisons are made, it is important to make statistical adjustments for them; the risk of a Type I error is greater when multiple comparisons are made. Some researchers recommend combining various measures into a single summary score (DesHarnais, McMahon, & Wroblewski, 1991; Felson et al., 1990). Such global composite measures have not been widely used, however. The various measures used in such an index may not be equally weighted and may be difcult to combine. Also, the composite index value may not be readily interpretable by clinicians. The focus of most measures developed for outcomes studies has been the individual patient. However, a number of organizations are now developing measures of the quality of performance of systems of care. In 1990, the Consortium Research on Indicators of System Performance (CRISP) project began to develop indicators of the quality of performance of integrated delivery systems. From the perspective of CRISP, the success of a health system is associated with its ability to decrease the number of episodes of diseases in the population. Therefore, the impact of the delivery system on the community is considered an important measure of performance. CRISP has developed a number of indicators now in use by consortium members, who pay to participate in the studies (Bergman, 1994). The JCAHO also is applying outcomes data to quality management efforts in hospitals using the IMSystem (Information Management System) (McCormick, 1990; Nadzim, Turpin, Hanold, & White, 1993). The National Committee for Quality Assurance, the organization that accredits managed care plans, has developed a tool (HEDISHealth plan Employer Data and Information Set) for comparing managed care plans. Comparisons
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 312
312
C H A P T E R
Outcomes Research
involve more than 60 measures, including patient satisfaction, quality of care, and nancial stability (Guadagnoli & McNeil, 1994). Researchers at the Henry Ford Health Systems Center for Health System Studies in Detroit have evolved 80 performance indicators to evaluate health systems (Anderson, 1991).
Outcomes researchers also are questioning the method of analysis of change. Collins and Johnston (1995) suggest that the recommended analysis method of regressing pretest scores on outcome scores and basing the analysis of change on residual change scores is overly conservative and tends to understate the extent of real change. Serious questions remain about the conceptual meaning of these residual change scores. For some outcomes, the changes may be nonlinear or may go up and down, rather than always increasing. Thus, it is as important to uncover patterns of change as it is to test for statistically signicant differences at various time points. Some changes may occur in relation to stages of recovery or improvement. These changes may occur over weeks, months, or even years. A more complete picture of the process of recovery can be obtained by examining the process in greater detail and over a broader range. With this approach, a recovery curve can
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 313
C H A P T E R
Outcomes Research
313
be developed, which provides a model of the recovery process and can then be tested (Collins & Johnston, 1995; Ottenbacher, Johnson, & Hojem, 1995).
Variance Analysis
Variance analysis is used to track individual and group variance from a specic critical
pathway. The goal is to decrease preventable variance in process, thus helping patients and their families achieve optimal outcomes. Some of the variance is due to the presence of comorbid conditions. Keeping a patient with comorbidity on the desired pathway may require utilization of more resources early in treatment. Thus, it is important to track both variance and comorbidity. Studies examining variations from pathways may facilitate the tailoring of existing critical pathways for a specic comorbid condition. Variance analysis also can be used to identify at-risk patients who may benet from the services of a case manager. Variance analysis tracking is expressed through the use of graphics, and the expected pathway is plotted on the graph. The care providers plot deviations (negative variance) on the graph, allowing immediate comparison with the expected pathway. Deviations may be related to the patient, the system, or the provider (Tidwell, 1993).
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 314
314
C H A P T E R
Outcomes Research
Table 97
Patient A Time 1 Time 2 Time 3 Patient B Time 1 Time 2 Time 3 Patient C Time 1 Time 2 Time 3
M1, Moving unassisted from bed to chair; M2, moving unassisted from bed to another room; M3, moving unassisted up stairs. From Collins, L. M., & Johnston, M. V. (1995). Analysis of stage-sequential change in rehabilitation research. American Journal of Physical Medicine and Rehabilitation, 74(2), 167.
After performing LTA by means of a computerized program designed for that purpose, the investigators presented their results in a table (Table 9-A). Data for only two points in time are shown, although the program can handle up to such ve points.
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 315
C H A P T E R
Outcomes Research
315
TABLE 9-A A Hypothetical Latent Transition Model of Recovery from Neglect Following Stroke Latent Status F Total Marginal Proportions Time 1 proportions Time 2 proportions C S S and C
0.0 0.27
0.40 0.25
0.30 0.30
0.30 0.18
Time 2 Latent Status Time 1 Latent Status Time 1 to Time 2 Transition Proportions with Rows Functional (F) Cognitive limitation (C) Sensory limitation (S) S and C 0 0.46 0.30 0.0 0 0.54 0.0 0.10 0 0.0 0.70 0.30 0 0.0 0.0 0.60 F* C S S and C
From Collins, L. M., & Johnston, M. V. (1995). Analysis of stage-sequential change in rehabilitation research. American Journal of Physical Medicine and Rehabilitation, 74(2), 168. *No patients were functional at Time 1.
The rst line of the table contains the estimate of the proportion of patients in each of the four stages at Time 1. In this example, 30% of the sample had both S and C limitations, 30% had S limitations, and 40% had C limitations, and none was functional. At Time 2, the proportion in each functional limitation appears to have declined, except that for S limitations, which is unchanged, and 27% are now in the functional stage. The bottom half of the table is a matrix of transition probabilities that reveals patterns of change. Of patients who started with S, 30% improved; however, the overall percentage at S remained the same because 30% of the patients who started at S and C moved to the S category. Of patients who initially had C problems alone, 46% moved to the functional category. A third set of quantities estimated by the full LTA model but not shown in the table are the relationships between items and stage memberships. This relationship indicates the probability that when a subject moves from one category to another, each item will also change to reflect the new stage membership. Thus, this relationship is a determination of the effectiveness of the test items or clinical symptoms as indicators of stage membership.
Multilevel Analysis
Multilevel analysis is used in epidemiology to study how environmental factors (aggregatelevel characteristics) and individual attributes and behaviors (individual-level characteristics) interact to influence individual-level health behaviors and disease risks. For example, the risk that an adolescent will start smoking is associated with the following variables: (1) attributes of the child (e.g., self-esteem, academic achievement, refusal skills), (2) attributes of the childs
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 316
316
C H A P T E R
Outcomes Research
family (e.g., parental attitudes toward smoking, smoking behavior of parents), (3) general characteristics of the community (e.g., ease of minors access to cigarettes, school policies regarding smoking, city smoking ordinances, social norms of students for smoking), and (4) general social factors (e.g., geographical region, economic policies that influence the price of cigarettes). The researchers might ask, Does smoking status covary with the level of restriction of smoking in public places after we have controlled for the individual-level variables that influence smoking risks? (Von Korff, Koepsell, Curry, & Diehr, 1992).
Outcomes research was developed to examine the end results of patient care. The scientic approaches used in outcomes studies differ in some important ways from those used in traditional research. The theory on which outcomes research is based was developed by Donabedian (1987). Quality is the overriding construct of the theory. The three major concepts of the theory are health, subjects of care, and providers of care. The goal of outcomes research is the evaluation of outcomes as dened by Donabedian, whose theory requires that identied outcomes be clearly linked with the process that caused the outcome. Outcomes research programs are complex and may consist of multiple studies using a variety of designs whose ndings must be merged in the process of forming conclusions. The researcher must consider the measures sensitivity to change and the magnitude of change that can be detected.
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 317
C H A P T E R
Outcomes Research
317
Statistical approaches used in outcomes studies include new approaches to examining measurement reliability, strategies to analyze change, and the analysis of improvement. Strategies must be developed in nursing to disseminate the ndings from outcomes studies to the various constituencies needing the information.
R E F E R E N C E S
Allred, C. A., Arford, P. H., Mauldin, P. D., & Goodwin, L. K. (1998). Cost-effectiveness analysis in the nursing literature, 19921996. The Journal of Nursing Scholarship, 30(3), 235242. American Medical Association (1989). Journal of the American Medical Association, 22, 125 930. Copyrighted 1989, American Medical Association. Anderson, H. J. (1991). Sizing up systems: researchers to test performance measures. Hospitals, 65(20), 3334. Anonymous. (1997). A report on the implementation of a nursing report card for acute care settings executive summaryJanuary 1997. New Mexico Nurse, 42(2), 11. Auffrey, C. (1998). Geographic information systems as a tool for community health research and practice. Nursing Research Methods [Online]. Available at: http://www.nursing.uc.edu/nrm/AUFFREY51598.htm Barton, A. J., Gilbert, L., Erickson, V., Baramee, J., Sowers, D., & Robertson, K. J. (2003). A guide to assist nurse practitioners with standardized nursing language. Computers, Informatics, Nursing: CIN, 21(3), 2003. Blegen, M. A., Goode, C. J., & Reed, L. (1998). Nurse stafng and patient outcomes, Nursing Research, 47(1), 4350. Buerhaus, P. I., & Needleman, J. (2000). Pollicy implications of research on nurse stafng and quality of patient care. Policy, Politics, & Nursing Practice, 1(1), 515. Bergman, R. (1994). Are my outcomes better than yours? Hospital Health Network, 68(15), 113116. Bergmark, A., & Oscarsson, L. (1991). Does anybody really know what they are doing? Some comments related to methodology of treatment service research. British Journal of Addiction, 86(2), 139142. Bernstein, S. J., & Hilborne, L. H. (1993). Clinical indicators: The road to quality care? Joint Commission Journal on Quality Improvement, 19(11), 501509. Bircumshaw, D., & Chapman, C. M. (1988). A study to compare the practice style of graduate and non-graduate nurses and midwives: The pilot study. Journal of Advanced Nursing, 13(5), 605614. Buerhaus, P. I., & Needleman, J. (2000). Policy implications of research on nurse stafng and quality of patient care. Policy, Politics, & Nursing Practice, 1(1), 515. Campbell-Heider, N., Krainovich-Miller, B., King, K. B., Sedhom, L., & Malinski, V. (1998). Empowering staff nurses to participate in the American Nurses Associations call for quality indicators research. Journal of the New York State Nurses Association, 29(3/4), 2127. Chaput de Saintonge, D. M., & Hattersley, L. A. (1985). Antibiotics for otitis media: Can we help doctors agree? Family Practice, 2(4), 205212. Chaput de Saintonge, D. M., Kirway, J. R., Evans, S. J., & Crane, G. J. (1988). How can we design trials to detect clinically important changes in disease severity? British Journal of Clinical Pharmacology, 26(4), 355362. Cho, S., Ketean, S., Barkauskas, V. H., & Smith, D. G. (2003). The effects of nurse stafng on adverse events, morbidity, mortality, and medical costs. Nursing Research, 52(2), 7179. Clinton, J. J. (1993). Financing medical effectiveness research: Role of the agency for health care policy and research. Annals of the New York Academy of Sciences, 703, 295297. Collins, L. M., & Horn, J. L. (1991). Best methods for the analysis of change: Recent advances, unanswered questions, future directions. Washington, DC: American Psychological Association.
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 318
318
C H A P T E R
Outcomes Research
Collins, L. M., & Johnston, M. V. (1995). Analysis of stagesequential change in rehabilitation research. American Journal of Physical Medicine and Rehabilitation, 74(2), 163170. Crane, V. S. (1988). Economic aspects of clinical decision making: Applications of clinical decision analysis. American Journal of Hospital Pharmacy, 45(3), 548553. Cummings, G. G., Fraser, K., & Tarlier, D. S. (2003). Implementing advanced nurse practitioner roles in acute care: An evaluation of organizational change. Journal of Nursing Administration, 33(3), 139145. Davis, K. (1990). Use of data registries to evaluate medical procedures: Coronary artery surgery study and the balloon valvuloplasty registry. International Journal of Technology Assessment in Health Care, 6(2), 203210. DesHarnais, S., McMahon, L. F., Jr., & Wroblewski, R. (1991). Measuring outcomes of hospital care using multiple risk-adjusted indexes. HSR: Health Services Research, 26(4), 425445. Deyo, R. A. (1984). Measuring functional outcomes in the therapeutic trials for chronic disease. Controlled Clinical Trials, 5(3), 223240. Deyo, R. A., & Carter, W. B. (1992). Strategies for improving and expanding the application of health status measures in clinical settings. Medical Care, 30(5), MS176MS186. Deyo, R. A., Psaty, B. M., Simon, G., Wagner, E. H., & Omenn, G. S. (1997). The messenger under attack intimidation of researchers by special interest groups. The New England Journal of Medicine, 336(16), 11761180. Deyo, R. A., Taylor, V. M., Diehr, P., Conrad, D., Cherkin, D. C., Ciol, M., et al. (1994). Analysis of automated administrative and survey databases to study patterns and outcomes of care. Spine, 19(18S), 2083S2091S. Donabedian, A. (1976). Benets in medical care programs. Cambridge: Harvard University Press. Donabedian, A. (1978). Needed research in quality assessment and monitoring. Hyattsville, MD: U.S. Department of Health, Education, and Welfare, Public Health Service, National Center for Health Services Research: NCHSR Ofce of Scientic and Technical Information. [Available from National Technical Information Service, Springeld, VA.] Donabedian, A. (1980). Explorations in quality assessment and monitoring. Ann Arbor, MI: Health Administration Press. Donabedian, A. (1982). The criteria and standards of quality. Ann Arbor, MI: Health Administration Press. Donabedian, A. (1987). Some basic issues in evaluating the quality of health care. In L. T. Rinke (Ed.), Outcome measures in home care (Vol. I, pp. 328). New York: National League for Nursing. [Original work published 1976.] Donohue, R. (2003). Nurse practitionerclient interaction as resource exchange in a womens health clinic: An exploratory study. Journal of Clinical Nursing, 12(5), 717725. Doran, D. I., Sidani, S., Keatings, M., & Doidge, D. (2002). An empirical test of the Nursing Role Effectiveness Model. Journal of Advanced Nursing, 38(1), 2939.
Doubilet, P., Weinstein, M. C., & McNeil, B. H. (1986). Use and misuse of the term cost effective in medicine. The New England Journal of Medicine, 314(4), 253255. Drummond, M. F., Stoddart, G. L. & Torrance, G. W. (1987). Method for the economic evaluation of health care programs. New York: Oxford University Press. Evans, R. G., & Stoddart, G. L. (1990). Producing health, consuming health care. Social Science and Medicine, 31(12), 13471363. Fardon, D. F., Garn, S. R., & Saal, J. A. (1997). Intimidation of researchers by special interest groups [Letter; comment]. The New England Journal of Medicine, 337(18), 13151316. Feinstein, A. R., Josephy, B. R., & Wells, C. K. (1986). Scientic and clinical problems in indexes of functional disability. Annals of Internal Medicine 105(3), 413420. Felson, D. T., Anderson, J. J., & Meenan, R. F. (1990). Time for changes in the design, analysis, and reporting of rheumatoid arthritis clinical trials. Arthritis and Rheumatism, 33(1), 140149. Fowler, F. J., Jr., Cleary, P. D., Magaziner, J., Patrick, D. L., & Benjamin, K. L. (1994). Methodological issues in measuring patient-reported outcomes: The agenda of the work group on outcomes assessment. Medical Care, 32(7 Suppl.), JS65JS76. Freedman, L. S., & Schatzkin, A. (1992). Sample size for studying intermediate endpoints within intervention trials or observational studies. American Journal of Epidemiology, 136(9), 11481159. Freund, D. A., Dittus, R. S., Fitzgerald, J., & Heck, D. (1990). Assessing and improving outcomes: Total knee replacement. HSR: Health Services Research, 25(5), 723726. Fullerton, J. T., Hollenbach, K. A., & Wingard, D. L. (1996). Research exchange. Practice styles: A comparison of obstetricians and nurse-midwives. Journal of NurseMidwifery, 41(3), 243-250. Gold, M. R., Siegel, J. E., Russell, L. B., & Weinstein, M. C. (Eds.). (1996). Cost-effectiveness in health and medicine. New York: Oxford University Press. Goldberg, H. I., Cummings, M. A., Steinberg, E. P., Ricci, E. M., Shannon, T., Soumerai, S. B., et al. (1994). Deliberations on the dissemination of PORT products: Translating research ndings into improved patient outcomes. Medical Care, 32(7 Suppl.), JS90JS110. Gottman, J. M., & Rushe, R. H. (1993). The analysis of change: Issues, fallacies, and new ideas. Journal of Consulting and Clinical Psychology, 61(6), 907910. Gray, B. H. (1992). The legislative battle over health services research. Health Affairs, 11(4), 3866. Greene, R., Bondy, P. K., & Maklan, C. W. (1994). The national medical effectiveness research initiative. Diabetes Care, 17(Suppl. 1), 4549. Greeneld, S., Nelson, E. C., Zubkoff, M., Manning, W., Rogers, W., Kravitz, R. L., et al. (1992). Variations in resource utilization among medical specialties and systems of care: Results from the Medical Outcomes study. JAMA, 267(12), 16241630. Greist, J. H., Jefferson, J. W., Wenzel, K. W., Kobak, K. A., Bailey, T. M., Katzelnich, D. J., et al. (1997). The telephone assessment program: Efcient patient
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 319
C H A P T E R
Outcomes Research
319
monitoring and clinician feedback. M.D. Computing: Computers in Medical Practice, 14(5), 382387. Guadagnoli, E., & McNeil, B. J. (1994). Outcomes research: Hope for the future or the latest rage? Inquiry, 31(1), 1424. Guess, H. A., Jacobsen, S. J., Girman, C. J., Oesterling, J. E., Chute, C. G., Panser, L. A., et al. (1995). The role of community-based longitudinal studies in evaluating treatment effects: Example: Benign prostatic hyperplasia. Medical Care, 33(4 Suppl.), AS26AS35. Guyatt, G., Walter, S., & Norman, G. (1987). Measuring change over time: assessing the usefulness of evaluative investments. Journal of Chronic Disease. 4(2) 171178. Hale, C. A., Thomas, L. H., Bond, S., & Todd, C. (1997). The nursing record as a research tool to identify nursing interventions. Journal of Clinical Nursing, 6(3), 207214. Hall, L. M., Doran, D. I., Baker, G. R., Pink, G., Sidani, S., OBrien, P., et al. (2001). A study of the impact of nursing staff mix models & organizational change strategies on patient, system & caregiver outcomes. Available at: http://www.nursing.utoronto.ca/ lmcgillishall/research/completed-lead.html Harris, C. W. (Ed.). (1967). Problems in measuring change. Madison: University of Wisconsin Press. Harris, M. R., & Warren, J. J. (1995). Patient outcomes: Assessment issues for the CNS. Clinical Nurse Specialist, 9(2), 8286. Head, B. J., Maas, M., and Johnson, M. (2003). Validity and community-health-nursing sensitivity of six outcomes for community health nursing with older clients. Public Health Nursing, 25(5), 385398. Herrmann, D. (1995). Reporting current, past, and changed health status: What we know about distortion. Medical Care, 33(4 Suppl.), AS89AS94. Hinshaw, A. S. (1992). Welcome: Patient outcome research conference. In Patient Outcomes Research: Examining the Effectiveness of Nursing Practice: Proceedings of a Conference Sponsored by the National Center for Nursing Research, September 1113, 1991 (NIH Publication No. 933411). Washington, DC: U.S. Department of Health and Human Services, Public Health Service, National Institutes of Health. Holmes, D., Teresi, J., Lindeman, D. A., & Glandon, G. L. (1997). Measurement of personal care inputs in chronic care settings. Journal of Mental Health and Aging, 3(1), 119127. Houser, J. L. (2000). A model for evaluating the context of nursing care delivery. Unpublished doctoral dissertation. Greeley: University of Northern Colorado. Ingersoll, G. L., McIntosh, E., Williams, M. (2000). Nursesensitive outcomes of advanced practice. Journal of Advanced Nursing, 32(5), 12721282. Irvine, D., Sidani, S., & Hall, L. M. (1998). Linking outcomes to nurses roles in health care. Nursing Economics, 16(2), 5864, 87. Irvine, D., Sidani, S., Porter, H., OBrien-Pallas, L., Simpson, B., Hall, L. M., et al. (2000). Organizational factors influencing nurse practitioners role implementation in acute care settings. Canadian Journal of Nursing Leadership, 13(3), 2835. Jennings, B. M., Loan, L. A., DePaul, D., Brosch, L. R., & Hildreth, P. (2001). Lessons learned while collecting
ANA indicator data, Journal of Nursing Administration, 31(3), 121129. Keeler, E. B. (1994). Decision analysis and costeffectiveness analysis in womens health care. Clinical Obstetrics and Gynecology, 37(1), 207215. Kelly, K. C., Huber, D. G., Johnson, M., McCloskey, J. C., & Maas, M. (1994). The Medical Outcomes Study: A nursing perspective. Journal of Professional Nursing, 10(4), 209216. Kieffer, E., Alexander, G. R., & Mor, J. (1992). Area-level predictors of use of prenatal care in diverse populations. Public Health Reports, 107(6), 653658. Kirwan, J. R., Chaput de Saintonge, D. M., Joyce, C. R. B., Holmes, J., & Currey, H. L. F. (1986). Inability of rheumatologists to describe their true policies for assessing rheumatoid arthritis. Annals of Rheumatic Diseases, 45, 156161. Kleinpell-Nowell, R., & Weiner, T. (1999). Measuring advanced practice nursing outcomes. AACN Clinical Issues, 19(3), 356368. Kozma, C. M., Reeder, C. E., & Schuiz, R. M. (1993). Economic, clinical, and humanistic outcomes: A planning model for pharmacoeromic research. Clinical Therapeutics, 15(6), 1125. Lange, L. L., & Jacox, A. (1993). Using large data, bases in nursing and health policy research. Journal of Professional Nursing, 9(4), 204211. Lasker, R. D., Shapiro, D. W., & Tucker, A. M. (1992). Realizing the potential of practice pattern proling. Inquiry, 29(3), 287297. Lave, J. R., Pashos, C. L., Anderson, G. F., Brailer, D., Bubloz, T., Conrad, D., et al. (1994). Costing medical care: Using Medicare administrative data. Medical Care, 32(7), JS77JS89. Lee, T. H., & Goldman, L. (1989). Development and analysis of observational data bases. Journal of the American College of Cardiology, 14(3, Suppl. A), 44A47A. Lewis, B. E. (1995). HMO outcomes research: Lessons from the eld. Journal of Ambulatory Care Management, 18(1), 4755. Lieu, T. A., & Newman, T. B. (1998). Issues in studying the effectiveness of health services for children: Improving the quality of healthcare for children: An agenda for research. HSR: Health Services Research, 4(33), 1041. Loegering, L., Reiter, R. C., & Gambone, J. C. (1994). Measuring the quality of health care. Clinical Obstetrics and Gynecology, 37(1), 122136. Ludbrook, A. (1990). Using economic appraisal in health services research. Health Bulletin, 48(2), 8190. Lynn, J., & Virnig, B. A. (1995). Assessing the signicance of treatment effects: Comments from the perspective of ethics. Medical Care, 33(4), AS292AS298. Mark, B. A. (1995). The black box of patient outcomes research. The Journal of Nursing Scholarship, 27(1), 42. Mattson, M. E., & Donovan, D. M. (1994). Clinical applications: The transition from research to practice. Journal of Studies on Alcohol, 12(Suppl.), 163166. McCloskey, B., Grey, M., Deshefy-Longhi, T., & Grey, L. (2003). APRN practice patterns in primary care. NP The Nurse Practitioner: The American Journal of Primary Health Care, 28(4), 3944.
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 320
320
C H A P T E R
Outcomes Research
McCormick, B. (1990). Outcomes in action: The JCAHOs clinical indicators. Hospitals, 64(19), 3438. McDonald, C. J., & Hui, S. L. (1991). The analysis of humongous databases: Problems and promises. Statistics in Medicine, 10(4), 511518. McNeil, B. J., Pedersen, S. H., & Gatsonis, C. (1992). Current issues in proling quality of care. Inquiry, 29(3), 298307. Mitchell, J. B., Bubolz, T., Pail, J. E., Pashos, C. L., Escarce, J. J., Muhlbaier, L. H., et al. (1994). Using Medicare claims for outcomes research. Medical Care, 32(7 Suppl.), JS38JS51. Mitchell, P. H., Ferketich, S., Jennings, B. M., & American Academy of Nursing Expert Panel on Quality Health Care. (1998). Quality health outcomes model. The Journal of Nursing Scholarship, 30(1), 4346. Moritz, P. (1991). Innovative nursing practice models and patient outcomes. Nursing Outlook, 39(3), 111114. Moses, L. E. (1995). Measuring effects without randomized trials? Options, problems, challenges. Medical Care, 33(4), AS8AS14. Nadzim, D. M., Turpin, R., Hanold, L. S., & White, R. E. (1993). Data-driven performance improvement in health care: The Joint Commissions Indicator Measurement System (IMSystem). The Joint Commission Journal on Quality Improvement, 19(11), 492500. Needleman, J., Buerhaus, P., Mattke, S., Stewart, M., & Zelevinsky, K. (2002a). Nurse stafng and quality of care in hospitals in the United States. Policy, Politics, & Nursing Practice, 3(4), 306308. Needleman, J., Buerhaus, Pl., Mattke, S., Steward, M., & Zelevinsky, K. (2002b). Nurse-stafng levels and the quality of care in hospitals. The New England Journal of Medicine, 346(22), 17151722. OConnor, G. T., Plume, S. K., & Wennberg, J. E. (1993). Regional organization for outcomes research. Annals of the New York Academy of Sciences, 703, 4451. One Hundred Sixth Congress of the United States, 1999, S. 580, An Act to Amend title IX of the Public Health Service Act to revise and extend the Agency for Health Care Policy and Research, pp. 23. Orchard, C. (1994). Comparing healthcare outcomes. British Medical Journal, 308(6942), 14931496. Oster, G. (1988). Economic aspects of clinical decision making: Applications in patient care. American Journal of Hospital Pharmacy, 45(3), 543547. Ottenbacher, K. J., Johnson, M. B., & Hojem, M. (1995). The signicance of clinical change and clinical change of signicance: Issues and methods. American Journal of Occupational Therapy, 42(3), 156163. Powe, N. R., Turner, J. A., Maklan, C. W., & Ersek, M. (1994). Alternative methods for formal literature review and meta-analysis in AHCPR Patient Outcomes Research Teams. Medical Care, 32(7), JS22JS37. Power, E. J., Tunis, S. R., & Wagner, J. L. (1994). Technology assessment and public health. Annual Review of Public Health, 15, 561579. Rettig, R. (1990). History, development, and importance to nursing of outcomes research. Journal of Nursing Quality Assurance, 5(2), 1317.
Riesenberg, D., & Glass, R. M. (1989). The Medical Outcomes Study. JAMA, 262(7), 943. Rosenfeld, P., McEvoy, M. D., & Glassman, K. (2003). Measuring practice patterns among acute care nurse practitioners. Journal of Nursing Administration, 33(3), 159165. Rovine, M. J., & Von Eye, A. (1991). Applied computational statistics in longitudinal research. San Diego: Academic Press. Rowell, P. (2001). Lessons learned while collecting ANA indicator data: The American Nurses Association responds. Journal of Nursing Administration, 31(3), 130131. Sawyer, L. M., Berkowitz, B., Haber, J. E., Larrabee, J. H., Marino, B. L., Martin, K. S., et al. (2002). Expanding American Nurses Association Nursing Quality Indicators to community-based practices. Outcomes Management, 6(2), 5361. Schmitt, M. H., Farrell, M. P., & Heinemann, G. D. (1988). Conceptual and methodological problems in studying the effects of interdisciplinary geriatric teams. The Gerontologist, 28(6), 753764. Sidani, S., & Irvine, D. (1999). A conceptual framework for evaluating the nurse practitioner role in acute care settings. Journal of Advanced Nursing, 30(1), 5866. Sidani, S., Irvine, D., Porter, H., OBrien-Pallas, L., Simpson, B., Hall, L. M., et al. (2000). Practice patterns of acute care nurse practitioners. Canadian Journal of Nursing Leadership, 13(3), 612. Siegel, J. E. (1998). Cost-effectiveness analysis and nursing researchIs there a t? The Journal of Nursing Scholarship, 30(3), 221222. Slade, M., Kuipers, E., & Priebe, S. (2002). Mental health services research methodology, International Review of Psychiatry, 14(1), 1218. Sledge, C. B. (1993). Why do outcomes research? Orthopedics, 16(10), 10931096. Sochalski, J. (2001). Quality of care, nurse stafng, and patient outcomes. Policy, Politics, & Nursing Practice, 2(1), 918. Sonnenberg, F. A., Roberts, M. S., Tsevat, J., Wong, J. B., Barry, M., & Kent, D. L. (1994). Toward a peer review process for medical decision analysis models. Medical Care, 32(7), JS52JS64. Standing, T., Anthony, M. K., & Hertz, J. E. (2001). Nurses narratives of outcomes after delegation to unlicensed assistive personnel, Outcomes Management for Nursing Practice, 5(1), 1823. Steinwachs, D. M., Weiner, J. P., & Shapiro, S. (1989). Management information systems and quality. In N. Goldfield & D.B. Nash (Eds.). Providing quality care: The challenge to clinicians (pp. 160180). Philadelphia: ACP. Stetler, C., & DeZell, A. D. (1989). In M. L. Etheredge (Ed.), Collaborative care: Nursing case management (pp. 6777). Chicago: American Hospital Association. Stetler, C., Morsi, D., & Burns, M. (2000). Physical and emotional patient safety: A different look at nursingsensitive outcomes. Outcomes Management for Nursing Practice, 4(4), 159166. Stewart, A. L., Greeneld, S., Hays, R. D., Wells, K., Rogers, W. H., Berry, S. D., et al. (1989). Functional
Ch09-X2640.qxd
4/20/06
4:22 PM
Page 321
C H A P T E R
Outcomes Research
321
status and well-being of patients with chronic conditions. JAMA, 262(7), 907913. Stone, D. W. (1998). Methods for conducting and reporting cost-effectiveness analysis in nursing. The Journal of Nursing Scholarship, 30(3), 229234. Swaen, G. M. H., & Meijers, J. M. M. (1988). Influence of design characteristics on the outcomes of retrospective cohort studies. British Journal of Industrial Medicine, 45(9), 624629. Tanenbaum, S. J. (1994). Knowing and acting in medical practice: The epistemological politics of outcomes research. Journal of Health Politics, Policy and Law, 19(1), 2744. Temple, R. (1990). Problems in the use of large data sets to assess effectiveness. International Journal of Technology Assessment in Health Care, 6(2), 211219. Tengs, T. O., Adams, M. E., Pliskin, J. S., Safran, D. G., Siegel, J. E., Weinstein, M. C., et al. (1995). Five hundred life-saving interventions and their costeffectiveness. Risk Analysis, 15(3), 369390. Tidwell, S. L. (1993). A graphic tool for tracking variance & comorbidities in cardiac surgery case management. Progress in Cardiovascular Nursing, 8(2), 619. Turk, D. C., & Rudy, T. E. (1994). Methods for evaluating treatment outcomes: Ways to overcome potential obstacles. Spine, 19(15), 17591763. Unsworth, C. A. (2001). Using a head-mounted video camera to study clinical reasoning. The American Journal of Occupational Therapy, 55(5), 582588. U.S. Congress, Ofce of Technology Assessment. (1994). Identifying health technologies that work: Searching for evidence (Publication No. OTA-H-608). Washington, DC: U.S. Government Printing Ofce. Veatch, R. M. (1993). Justice and outcomes research: The ethical limits. The Journal of Clinical Ethics, 4(3), 258261. Vermeulen, L. C., Jr., Ratko, T. A., Erstad, B. L., Brecher, M. E., & Matuszewski, K. A. (1995). A paradigm for consensus: The University Hospital Consortium Guidelines for the use of albumin, nonprotein colloid, and crystalloid solutions. Archives of Internal Medicine, 155(4), 373379. Verran, J. A., Mark, B. A., & Lamb, G. (1992). Psychometric examination of instruments using aggregated data. Research in Nursing & Health, 15(3), 237240. Vivier, P. M., Bernier, J. A., and Starfield, B. (1994). Current approaches to measuring health outcomes in pediatric research. Current Opinions in Pediatrics, 6(5), 531.
Volinn, E., Diehr, P., Ciol, M. A., & Loeser, J. D. (1994). Why does geographic variation in health care practices matter? (and seven questions to ask in evaluating studies on geographic variation). Spine, 19(18S), 2092S2100S. Von Eye, A. (Ed.). (1990a). Statistical methods in longitudinal research, Vol. I: Principles and structuring change. Boston: Academic Press. Von Eye, A. (Ed.). (1990b). Statistical methods in longitudinal research, Vol. II: Time series and categorical longitudinal data. Boston: Academic Press. Von Korff, M., Koepsell, T., Curry, S., & Diehr, P. (1992). Multi-level analysis in epidemiologic research on health behaviors and outcomes. American Journal of Epidemiology, 135(10), 10771082. Wennberg, J. E., Barry, M. J., Fowler, F. J., & Mulley, A. (1993). Outcomes research, PORTs, and health care reform. Annals of the New York Academy of Sciences, 703, 5262. Werley, H., Devine, E., Zorn, C., Ryan, P., & Westra, B. (1991). The nursing minimum data set: Abstraction tool for standardized, comparable, essential data. American Journal of Public Health, 81(4), 421426. Werley, H., & Lang, N. (1988). Identication of the nursing minimum data set. New York: Springer. White, K. L. (1993). Health care research: Old wine in new bottles. The Pharos of Alpha Omega Alpha Honor Medical Society, 56(3), 1216. Wilson, I. B., & Cleary, P. D. (1995). Linking clinical variables with health-related quality of life: A conceptual model of patient outcomes. Journal of the American Medical Association, 273(1), 60. Copyright 1995, the American Medical Association. Wood, L. W. (1990). Medical treatment effectiveness research. Journal of Occupational Medicine, 32(12), 11731174. Wray, N. P., Ashton, C. M., Kuykendall, D. H., Petersen, N. J., Souchek, J., & Hollingsworth, J. C. (1995). Selecting disease-outcome pairs for monitoring the quality of hospital care. Medical Care, 33(1), 7589. Zielstorff, R., Hudgings, C., Grobe, S., & the National Commission on Nursing Implementation Project (NCNIP) Task Force on Nursing Information Systems. (1993). Next generation nursing information systems: Essential characteristics for professional practice. Washington, DC: American Nurses Association.