Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
This manuscript has been reproduced from the microfilm master. UMI
films the text directly from the original or copy submitted. Thus, some
thesis and dissertation copies are in typewriter face, while others may be
from any type of computer printer.
The quality of this reproduction is dependent upon the quality of the
copy submitted.
Also, if
UMI
A Bell & Howell Information Company
300 North Zed) Road, Ann Arbor MI 48106-1346 USA
313/761-4700 800/521-0600
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
DISSERTATION
By
Susan M. Meyer, B.S., M.B.A.
****
Dissertation Committee:
Approved by
Professor David A. Collier, Advisor
Professor Sharon B. Schweikhart
Professor Michael E. W alker
Advisor
Business Administration Graduate Program
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
UMI
Reproduced with permission o f the copyright owner. Further reproduction prohibited without permission.
Copyright by
Susan M. M eyer
1998
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
ABSTRACT
The 1995 pilot criteria of the Malcolm Baldrige National Quality Award (MBNQA)
in Health Care have generated great interest from the health care sector in the award itself
and in using the criteria as a self-assessment tool. The MBNQA in Health Care is a model
of Total Quality Management specifically designed for health care organizations.
This
study evaluates the causal linkages and performance relationships among the seven
categories o f the Baldrige model in U.S. hospitals.
A questionnaire
is developed
to empirically
measure
the 28
dimensions
community hospitals across the U.S., and 228 responded (28 percent).
Twenty-three research hypotheses test relationships among the categories o f the
Baldrige Health Care model. Following is a sum m ary o f the major findings:
1. Baldrige theory that Leadership drives the System which causes Results is supported in
hospitals. Results include Organizational Performance Results (Baldrige Category 6.0)
and custom er satisfaction (Category 7.0).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
5. The results o f this study provide the first comprehensive evaluation of the causal
influences and performance relationships proposed by the Baldrige Health Care model.
The development of a measurement model and empirical validation of Baldrige theory
contribute to this field of research.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
ACKNOWLEDGMENTS
My
appreciation also to Sharon B. Schweikhart for serving on my committee. She has been a
great source of knowledge and is a wonderful colleague.
I extend my gratitude to each o f my committee members fo r their many hours of
guidance and assistance. Each member shared with me their knowledge and expertise in
their field, and they showed how bringing together faculty from across a variety of
departments brings great value to the student.
Finally, my deepest gratitude to family and friends who supported and encouraged
me in this endeavor.
myself. Also, my appreciation to those faculty, staff, and students who have been helpful
and supportive throughout my years at The Ohio State University.
iv
Reproduced with permission o f the copyright owner. Further reproduction prohibited without permission.
VITA
June
10,
1065
1988
1992
1994-1998
PUBLICATIONS
Refereed Journals:
Collier, D.A. and M eyer, S.M., "A service positioning matrix." International Journal o f
Operations and Production Management (forthcoming).
Meyer, S.M. and Collier, D.A., "Contrasting the original Malcolm Baldrige National
Quality Award with the new Health Care Award." Quality Management in Health Care
(forthcoming).
Christensen, B.V., M anion, R.M., Iacarella, C.L., M eyer, S.M ., Cartland, J.L., BruhnDing, B.J., and W ilson, R.F. (1998), "Vascular com plications after angiography with and
without use of sandbags." Nursing Research, 47(1): 51-53.
Chambers, J.W ., Voss, G.S., Snider, J.R., Meyer, S.M ., Cartland, J.L., and W ilson,
R.F. (1996), "Direct in vivo effects of nitric oxide on coronary circulation." Am erican
Journal o f Physiology, 271: H 1584-1593.
Meyer, S.M. (1994), "Book review: Blooming, by S. A. Toth." Journal o f College and
Adult Reading a nd Learning, 3: 59-60.
Johnson, T.H ., D as, G.S., McGinn, A.L., C hristensen, B.V., Meyer, S.M., and Wilson.
R.F. (1994), "Physiologic assessment of the coronary collateral circulation in transplanted
human hearts." Journal o f Heart and Lung Transplantation, 13: 840-846.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Christensen, B.V., M eyer, S.M ., Iacarella, C.L., Kubo, S.H., and W ilson, R.F. (1994),
"Coronary angioplasty in cardiac transplant recipients: A qualitative angiographic long term
follow-up study." Journal o f Heart and Lang Transplantation, 13: 212-220.
Refereed Proceedings:
Meyer, S.M. and W ard, P.T. (1998), "Proactiveness and capabilities in U.S. hospitals."
Midwest Decision Sciences Institute 1998 Proceedings.
Collier, D.A. and Meyer, S.M. (1997), "A consumer benefit package - servicescape
positioning matrix." National Decision Sciences Institute 1997 Proceedings.
Meyer, S.M., W ard, P.T., Leong, G.K., and Butler, T.W . (1996), "How location, size,
and strategy influence hospital performance." National Decision Sciences Institute 1996
Proceedings.
Meyer, S.M. (1996), "Contrasting the original Malcolm Baldrige National Quality Award
with the new Health Care Award." Midwest Decision Sciences Institute 1996 Proceedings.
FIELDS OF STUDY
Major Field: Business Administration
Concentration: Operations Management
Minor Field: Psychology
Concentration: Quantitative Methods
vi
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
TABLE OF CONTENTS
Page
A b s tra c t..................................................................................................................................... ii
A c k n o w le d g m e n ts ................................................................................................................ iv
V ita .............................................................................................................................................. v
L ist
o f T a b le s .................................................................................................................... x
L ist
o f F ig u re s ....................................................................................................................xii
Chapters:
1.
2.
L iteratu re
R e v ie w .................................................................................................. 24
vii
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
3.
4.
62
64
69
71
74
M e th o d o lo g y ..............................................................................................................85
4.1
4.2
4.3
R e se a rc h d e s ig n ............................................................................................ 85
Q u e stio n n a ire d ev elo p m en t........................................................................88
P ilot te st o f q u estio n n aire........................................................................ 92
4.3.1
R eliability analysis o f pilot test data......................................95
4 .3 .2 V alid ity o f q u estio n n aire....................................................... 99
4.3 .3 C onclu sio n s from pilot te st...................................................102
4 .4 M ain stu d y d ata co llectio n .................................................................... 102
4.4.1 S am p le s e le c tio n .................................................................... 102
4 .4 .2 D ata p re p a ra tio n ...................................................................... 104
4.4.3 M issin g d a ta ............................................................................. 105
4.5 M ain study d ata descrip tio n................................................................. 106
4.5.1
M ain study scale reliab ility ................................................ 107
4 .5 .2 O rganizational data d escrip tio n ........................................... 108
4.6 Sum m ary o f pilot test and data collection.......................................... 110
5.
viii
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
129
132
139
140
148
150
162
6.
6.1
6.2
B ib lio g ra p h y ...........................................................................................................................207
Appendix A
Appendix B
227
Appendix C
Appendix D
ix
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
LIST OF TABLES
Table
Page
1.1
1.2
1.3
2.1
3.1
3.2
4.1
4.2
4.3
4 .4
4.5
4.6
Cronbachs A lpha Analysis for Dimension 1.1 Senior Executive and Health
C are S ta ff L e a d e rs h ip ..................................................................................... 117
4.7
4.8
4.9
4.10
M ain
4.11
4.12
S tu d y
S c a le s ............................................................................................ 121
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
4.13
4.14
4.15
5.1
5.2
5.3
5.4
5.5
5.6
6.1
6.2
6.3
xi
Reproduced with permission o f the copyright owner. Further reproduction prohibited without permission.
126
LIST OF FIGURES
Page
Figure
1.1
3.1
C o rre la tio n
3.2
3.3
3.4
S p e c ific C ausal
3.5
3.6
3.7
4.1
5.1
5.2
5.3
5.4
5.5
5.6
5.7
5.8
5.9
H y p o th e se s..........................................................................................78
H ypotheses................................................................................ 81
xii
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
177
5.10
6.1
6.2
xiii
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
CHAPTER 1
INTRODUCTION
"What is needed is new machinery, a new approach. For health care, a way o f
interjecting an overarching drive fo r excellence fro m all elem ents o f the system.
Do such alternatives exist?
Yes.
1.1 Introduction
The proportion of the U.S. gross domestic product (GDP) attributed to health care
expenses rose dram atically from 6 percent in 1965 to 14 percent in 1994.
After
remaining steady for several years, in 1998, the proportion o f the G D P consumed by
health care expenses shows signs o f increasing even further. W hile the rate of increase in
health care costs slow ed through the 1990s, the ever increasing costs and the aging
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
population of the U.S. provide a bleak outlook for the next century. Actuarial estimates
indicate that between 25 and 50 percent of the GDP, depending on future policies, will go
to health care expenses by 2065 (Washawsky, 1994). As com petitive pressures heighten,
health care organizations are acquired, merge, and close. In 1995 alone, 59 non-profit
hospitals were acquired by for profit firms (Burda and W eissenstein, 1997). Prior to
1995, an average of ju st nine hospitals per year converted from non-profit status to for
profit.
In 1987, the U .S. C ongress asked the N ational Institute o f Standards and
Technology (NIST) to develop and manage the M alcolm Baldrige N ational Quality
Award (MBNQA). The aw ards primary focus was to prom ote quality awareness and
practices, and recognize and publicize the quality achievem ents o f U.S. companies.
Since 1988, the M BNQA has been awarded to companies that demonstrate high levels of
business excellence and quality achievement.
categories: (1) manufacturing companies; (2) service com panies; (3) small businesses.
These three categories o f companies are evaluated on the same criteria (NIST, 1995a).
In 1995, two new sets o f criteria, specifically designed for health care and
education organizations, were introduced (NIST, 1995b; NIST, 1995c). These criteria
were pilot tested, and forty-six health care organizations com pleted applications for the
Health Care Award pilot study.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
health care organizations with a framework for developing and managing their quality
systems. Failure to em pirically validate the Baldrige fram ew ork may indicate a need to
restructure the award criteria.
The study is undertaken for several reasons. First, the MBNQA in Health Care
criteria are derived from the criteria of the original award (NIST, 1995b p. 2). This study
addresses w hether the o rig in al industrial m odel is transferable to health care
organizations. Second, there is no empirical evidence that the criteria linkages proposed
by the M BNQA in H ealth Care exist in health care organizations with high levels o f
quality performance, one of the premises of the award. This study shows whether those
linkages are validated. Third, this study clarifies the direction o f influence am ong the
criteria o f the M BN Q A in H ealth Care. M any of the linkages in the criteria are bi
directional, as shown in Figure 1.1, and the true direction o f influence among them is
unclear. This study validates the direction o f influence among the criteria based on
empirical evidence.
There is little research presented in the health care literature to em pirically
support the applicability o f TQM in hospitals. M otwani et al. (1996) review the health
care literature on TQM and find few articles based on em pirical studies of more than one
hospital. O f the articles they review, 25 discuss TQ M m ethodology, thirteen are case
studies, and ju st three are em pirical m ulti-site studies.
increased the num ber o f multi-site studies, as discussed in C hapter 2, but there is still a
great need for large-scale studies on this topic.
The current state o f the health care m arketplace also provides an im petus for
undertaking this study. Health care costs in the U.S. have increased dramatically in the
past two decades. In m ore recent years, as pressure to cut costs have hit this industry,
some hospitals have successfully balanced spending and incom e to m aintain th eir
com petitiveness in the m arketplace. Others have struggled to survive, or have closed,
3
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
organizations where their efforts and resources should be invested to maintain the quality
o f their m edical care w hile also maintaining their fiscal viability. This research show s
w here effort and resources are best invested in hospitals.
The focus o f the M BNQA in H ealth C are is to prom ote aw areness o f the
im portance o f quality im provem ent in all U.S. health care organizations. This study
focuses on quality m anagem ent in U.S. hospitals. Q uality is a multi-dimensional concept
and is defined in the health care industry in a num ber o f ways including conform ance to
standards, patient satisfaction, satisfactory medical outcom es, continuous im provem ent,
high productivity, access to care, appropriateness o f care, as well as other dim ensions
(Joint Com mission on Accreditation of Healthcare O rganizations, 1993).
The structure o f the M BNQA in H ealth C are is a model of Total Q uality
M anagem ent (TQ M ) for health care organizations. TQ M is a management approach to
building quality consciousness into the daily operations o f an organization. A TQ M
m anagem ent approach is com prised of both philosophical and structural elem ents.
Philosophical elem en ts include concepts and attitu d es that are necessary fo r an
organization to foster in order to successfully im plem ent TQM . Structural elem ents are
specific program s, m ethods, and human resource developm ents that are also considered
necessary to successful TQ M implementation.
A TQM
Some o f the
philosophical and structural elem ents com m only associated with TQM are given in
Tables 1.1 and 1.2, respectively. Each TQM elem ent is briefly defined in Tables 1.1 and
1.2 to illustrate how these concepts apply to the health care industry. For exam ple, the
philosophical elem en t o f custom er focus included in Table 1.1 addresses custom er
satisfaction in an industrial model of TQM, while in a health care setting, custom er focus
4
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
includes not only custom er satisfaction, but also the outcom es of medical treatm ents on
patients.
The philosophical elements listed in Table 1.1 include the core values associated
with TQM and their relevance to health care organizations. Elements such as a systems
view, im plem enter involvem ent, and process optim ization exemplify the organizationwide approach that TQM encompasses. The practical m eaning of these philosophical
elements is that every department and employee in a health care organization m ust be
included in efforts to implement TQM and to ensure continuous improvement.
The structural elem ents of TQM listed in Table 1.2 include the more practical
components of TQ M implementation. Health care organizations can use these structural
elements as building blocks as they transform themselves into organizations w hose focus
and daily operations are quality driven, including benchmarking themselves against other
health care o rg anizations, reengineering their m ost troublesom e processes, and
developing teams to address problems and opportunities throughout the organization.
The original M BNQA was developed by industry and academic experts who
hypothesized causal linkages among the criteria of the award. The award criteria have
been updated each year since the award's inception in 1988 and are now m ore focused
and aligned (NIST, 1995a). The MBNQA in Health C are was similarly designed by
industry and academ ic experts and is based on the sam e core concepts as the original
award. The Health Care Award utilizes a seven-part framework that is very sim ilar to the
original MBNQA, as shown in Figure 1.1. The seven main categories of the M BNQA in
Health Care criteria are given below. They are identified within the Baldrige framework
which groups them by Leadership, System, and Results:
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Gamp.
Category
Leadership:
1.0 Leadership
System:
Results:
The MBNQA in Health Care framework has four distinct elem ents, the Driver,
the System, M easures of Progress, and the Goal.
Leadership (Category 1.0) which "sets directions creates values, goals, and systems, and
guides the pursuit of health care value and organizational perform ance improvement"
(NIST, 1995b p. 7).
Planning (3.0), Human Resource D evelopm ent and M anagem ent (4.0), and Process
M anagement (5.0). The System "com prises the set of well-defined and well-designed
processes for meeting the organization's patient and performance requirem ents" (NIST,
1995b p. 7). Measures of Progress consist of Category 6.0 O rganizational Performance
Results. These measures "provide a results-oriented basis for channeling actions to
delivering ever-improving health care value and organizational perform ance"
(NIST,
1995b p. 7). Finally, the Goal is evaluated by Category 7.0 Focus on and Satisfaction of
Patients and O ther Stakeholders. The Goal includes "the delivery o f ever-im proving
health care value to patients and success in the health care marketplace" (NIST, 1995b p.
7).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
The M BNQA in H ealth Care is structured such that organizations applying for the
award earn points in each o f the seven categories. The num ber o f points that can be
earned in each category indicates that category's relative importance in the overall
implementation of the quality initiatives that are pursued in an organization, according to
the experts who developed this award structure. The points awarded in each o f the seven
main categories of the Baldrige Health Care criteria, as well as the 28 subcategories o f
which they are comprised, are shown in Table 1.3.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
T here are several aspects of quality definition and quality m easurem ent that are
prevalent and necessary in the application o f TQM in health care organizations, but
which are absent from the industrial m odel o f TQM. Manufacturing generally consists of
a num ber o f repetitive sequences in w hich custom ers have little or no involvem ent. As
the am ount o f custom er contact in production increases, and as processes m ove from
being manufacturing-based to service-based, w orker skills, processes, and organizational
goals m ust change (Chase, 1983). Evaluating w orker performance in environm ents with
high w orker discretion, measuring variable o r non-repeatable processes, and determining
the correct measures of quality perform ance all become more difficult in service-based
organizations.
An appropriate model o f TQM for health care organizations m ust account for the
aspects that differentiate this high contact service from the manufacturing environment.
M eyer and C ollier (1998) report that while many quality management concepts and tools
used in other industries are applicable in hospitals, there remain num erous differences
betw een health care and other industries that m ust be addressed.
T hese differences
M ultiple customers
In a typical industrial model o f TQM , the term custom er refers only to the
buyers or end-users of products and services. Health care organizations m ust please
many custom ers including patients, patient families, communiues, insurers or third-party
payers, governments, investors, and health science students. Patients are the recipients of
the health care services, but the other custom er groups also have a great interest in either
the services or the health care organization itself.
8
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Patient fam ilies are often present during portions o f the health care service
delivery and in some instances make decisions regarding the services to be delivered.
Communities are concerned with the well-being of the local population and want services
such as educational and screening programs and indigent care. Insurers or third-party
payers are concerned with the costs of medical care. They also want various types of
information from the health care organization to ensure that proper medical procedures
are performed. Federal and state governments bear much o f the cost o f medical care in
the U.S. through funding o f Medicare and Medicaid. G overnm ents also own facilities
such as veterans' hospitals. Investors, both private and charitable, have an interest in the
mission and performance of their organizations. Finally, health science students are
considered customers because they receive training at health care facilities as part o f their
education.
There are tradeoffs between the requirem ents for satisfying each o f these
custom er groups. Generally, patients and health science students prefer programs and
activities which increase spending by health care organizations while governm ents and
third-party payers prefer measures to decrease spending. Likewise, while students and
governments put a great deal of emphasis on education and research, patients may have
little immediate interest in these.
For health care organizations, the implications of providing services in this system
of multiple custom ers are great. Measuring and m anaging the satisfaction o f each of
these customers, and making tradeoffs among them, is a com plex task. The organization
must determine whether some or all of these customers' satisfaction is important enough
to measure and evaluate. The health care organization m ust also determine the internal
performance standards and measures that accurately reflect the interests of each custom er
group.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
custom ers, either the ultim ate consum er or industrial business-to-business custom ers.
But in most cases the end-users of health care services, the patients, do not pay for the
services and may have little interest in their cost.
Government pays for a majority of the health care costs in the U.S., and the health
care industry has somewhat unlimited access to revenue based on government supported
health care for the elderly (M edicare) and poor (Medicaid). Hospitals, which account for
approximately 40 percent o f spending and 50 percent of em ploym ent in the health care
industry, provide a good exam ple o f the multiple sources o f revenue that are available.
Using general community (non-government) hospitals as an example, 40 percent o f their
cumulative revenue com es from government reim bursem ent for Medicare patients, 14
percent from M edicaid, 34 percent from private insurers, 7 percent self-paid by the
patient, and 5 percent paid by other sources (Standard and Poor, 1995).
The implication o f m ultiple revenue sources is increased complexity for health
care organizations. Each revenue source requires different information and forms, has
different operating rules, and the payments from these sources can vary considerably
based on government policies and agreements with insurers. In addition, this system o f
reimbursement provides opportunity for fraud, as payers do not always receive accurate
information on the health care services provided.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
treating a health problem. Service (process) quality, which is discussed in the following
section, is how the service is delivered (Collier, 1994 p. 32). For example, the clinical
quality of a kidney transplant procedure is the physiological function of that kidney over
time, while service quality includes the ease o f the hospital check-in process and the
friendliness o f the staff.
C redence
characteristics are those w hich custom ers cannot evaluate because they are eith e r
unqualified to do so or do not have the opportunity to because the quality o f the services
cannot be seen (Zeitham l, 1981). Both of these definitions apply to medical procedures
such as surgery, w here the patient generally does not observe the surgery and cannot
accurately assess the physiological outcome of the surgery.
C linical q u ality involves very personal and private issues regarding an
individual's health. These quality issues cannot be approached in the same manner as the
design and m anagem ent of the tolerances for an autom obile. As explained in C ollier
(1994, p. 43), the patient participates in the service delivery, but does not know if he
received a 68 percent or 98 percent good kidney transplant. Being unable to accurately
evaluate clinical quality, the patient and patient's fam ily focus on som ething they can
evaluate - service (process) quality.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
m easures include processing and response tim es, cost per tran sactio n , errors per
transaction, etc. External measures of service quality involve the custom er's perception
of service which is often influenced by non-medical encounters. For exam ple, if patients
have difficulty parking near the health care facility, that experience m ay lead to a
negative perception o f the entire service system.
Service quality is an important driver of patient satisfaction with both the medical
and non-medical aspects of the service delivery. This challenges health care management
to properly m easure and evaluate service quality, clinical quality, and overall patient
satisfaction, as well as the satisfaction o f other customer groups.
organizations m ay influence how they com pete in the m arketplace and how they use
TQM to improve their organization.
A m ajor implication of allowing for profit, non-profit, governm ent, and teaching
organizations to apply for the same award is that there is no consistent set o f performance
measures on w hich all o f these organizations can be evaluated. For profit health care
organizations m ay focus on quality and profit measures, while governm ent organizations
may be m ore concerned with com m unity health and cost control.
The ability or
appropriateness o f com paring the perform ance of these very d iffe re n t types o f
12
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
organizations may create challenges to the design of the M BNQA in Health Care and the
perform ance m easures that are used in the aw ard criteria to evaluate health care
organizations.
C ost of failure
The custom er (patient) cost of a service failure can be critical in health care
organizations. Errors in service delivery can result in negative consequences or even
death. Few other industries have this extensive exposure to such high costs o f service
failure. Efforts to reduce failure costs or to reduce the occurrence o f failures translates
into efforts to fool-proof key processes in health care facilities. Service failure can result
from health care provider or process errors, inventory stockouts, unpredictable patient
actions, and physiological changes in patient conditions.
Health care provider errors occur when physicians, nurses, or other staff perform
incorrect procedures, adm inister incorrect drug dosages, serve the wrong m eal, etc.
These errors are caused by poor process design and m anagem ent and errors in hum an
judgm ent. Some health care providers are financially protected from such errors by
m alpractice insurance, but health care facilities often bear the brunt of such errors
because they may also be financially responsible and their reputation can be m arred by
these service delivery errors.
The cost of an inventory stockout can be m easured or estim ated in m any
industries. The cost includes lost sales and, in some instances, loss o f customer goodwill.
The cost of an inventory stockout in the health care industry is not as easily m easured or
estim ated. In some instances, where other products can be substituted, the cost o f a
stockout is a minimal loss o f service quality.
available, the result could be com plete service failure or, at worst, death. For this reason,
m ost health care organizations carry high levels of safety stock. Large facilities often
13
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
have both central and department stocking locations, so that backup supplies are nearly
always available. Hence, inventory m anagem ent is different in the health care industry
from many other industries because the cost o f a stockout can be so great.
Health care delivery failures can also occur when patients take unpredictable
actions or have unexpected physiological changes. Patient actions such as falling after
getting out of bed unassisted may result in service failure and can be the responsibility of
the health care organization. The health care industry has additional service failures that
results from patient physiology such as an allergic reaction to a drug.
A gain, the
organization does not have com plete control over some failures, but may be held
responsible for them.
The potentially high cost of a service failure requires health care providers to have
higher perform ance standards than many other industries. They m ust attem pt to fool
proof their processes to ensure that failures are minimized.
P h y sician s practice
defensive medicine, for example, ordering extensive tests although one inexpensive test
may be adequate to diagnose a m ajority o f patients showing a p articu lar sym ptom .
Hospitals use standardized procedures for some processes, such as preparing a patient for
surgery, so that no necessary procedures are omitted.
organizations carry excess inventory to guard against stockouts. These attem pts to guard
against the high cost of service failure can be very costly themselves.
O ther authors have also discussed problem s associated with trying to apply the
industrial model of TQM in the health care environment. Donabedian (1993) points out
the following deficiencies in the industrial model:
14
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
consum ers make their own decisions on the types of products and services they purchase.
T his transfer o f decision-m aking pow er is the result o f the trust patients have in
physicians' abilities and because m edical care has many credence characteristics that
patients are not fully able to evaluate, as described above.
T hese
providing more extensive treatm ent may be favorable to patients. However, tests and
treatm ents are expensive, and third party payers may limit m edical care due to its cost
and questionable value.
15
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
m anufacturing organizations.
In sum m ary, this section includes an overview o f som e of the most im portant
differences in the application o f TQM in industrial and health care environm ents.
Generally, health care organizations are more complex environm ents in which to evaluate
quality, creating com plexity in the developm ent and im plem entation of an appropriate
model of TQM.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
owned organizations w hich are common in the health care industry. N on-profit and
government-owned health care organizations likely pursue M easures o f Progress (see
Section 1.1 for description) that differ from those o f for profit organizations. Non-profit
and government-owned organizations are not eligible to apply for the original award.
The second difference is in Category 7.0 where the title in the original award is Customer
Focus and Satisfaction and the health care title is Focus on and Satisfaction o f Patients
and Other Stakeholders. This change is made to recognize that the health care industry
serves many custom ers including patients, patient fam ilies, com m unities, insurers or
third-party payers, governments, and health science students.
There are additional differences within the dim ensions (subcategories) o f the
criteria of the original M BN Q A and the Health Care Award criteria. Within Category 5.0
Process Management, D im ension 5.3 Patient Care Support Services Design and Delivery
has been redefined to apply to patient support services such as laboratory and radiology
services. In the original criteria, support services are defined as accounting, finance,
inform ation services, etc.
M anagement has been added to the Health Care criteria to address business support
services.
In Category 6.0 Organizational Performance Results, three items have been added
in the Health Care criteria.
Results, D im ension 6.3 C om m unity Health Services R esults, and D im ension 6.5
Accreditation and A ssessm ent Results. These three new dim ensions address im portant
goals for health care organizations, but are not as applicable to the industries targeted by
the original MBNQA. In addition, Dimension 6.2 C om pany Operational and Financial
Results and Dimension 6.3 Supplier Performance Results in the original award have been
combined in Dimension 6.4 Administrative, Business, and Supplier Results in the Health
17
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Care criteria. In total, the original award criteria address 24 separate dim ensions w hile
the Health Care criteria include 28 dimensions.
The N IST has two main goals in establishing separate criteria for the health care
industry. First, the H ealth Care Award criteria focus on delivery of ever-im proving value
to patients and other stakeholders, contributing to im proved health care quality. Second,
the Health Care criteria encourage improvement o f overall organizational effectiveness,
use o f resources, and capabilities.
organization follows the guidelines presented in the M BNQA in Health Care criteria, it
w ill be successful at providing both efficient and effective health care serv ices.
Efficiency includes costs, timeliness, and cycle tim e. Effectiveness refers to efficacy,
appropriateness, risk and availability of service (NIST, 1995b p. 25).
successfully balanced their spending and income while others struggle to survive. This
study helps to clarify where hospitals should invest their efforts and resources in order to
maintain quality medical care and Fiscal viability.
18
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
agencies are virtually free from competition, perhaps reducing their interest in or need for
TQ M (A nschutz, 1995). In contrast, manufacturing and service firms see the M BNQA as
the best fram ew ork for applying TQM in their organizations (Knotts, 1993). Validation
o f the fram ew ork presented by the M BNQA in H ealth Care may provide evidence that
this TQM fram ew ork is applicable to both public and private organizations.
19
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1.
2.
3.
4.
5.
6.
7.
8.
9.
20
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1.
2.
3.
4.
Top management com m itm ent - leadership to make TQM effective and foster its
integration into the institutional fabric of the organization
5.
6.
7.
8.
21
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Categories/Items
Point Values
Leadership:
1.0
1.1
1.2
1.3
Leadership_______________________________________________ 90
Senior Executive and Health Care Staff Leadership
45
Leadership System and Organization
25
Public Responsibility and Citizenship
20
System:
2.0
2.1
2.2
2.3
5.0
5 .1
5.2
5.3
5.4
5.5
5.6
6.0
6.1
6.2
6.3
6.4
6.5
7.0
7.1
7.2
7.3
7.4
7.5
Human Resource
H um an Resource
Employee/Health
Em ployee/Health
D evelopm ent
4.4 Employee/Health
Results:
22
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Process
Management
Focus on and
Satisfaction of
Human Resource
Development and
Management
Leadership
Strategic
Planning
to
u>
Organizational
Performance
Information
Results
Analysis
Source: NIST (1 9 9 5 b )
Driver
. System
Results
Figure 1.1: Malcolm Baldrige National Quality Award in Health Care Pilot Criteria Model
CHAPTER 2
LITERATURE REVIEW
In this chapter, Total Quality M anagement (TQM) models from the management
literature, including m odels that are sp ecifically designed for the m anufacturing
environment, are discussed first. Some practitioners and researchers have attem pted to
apply these m odels in the health care environm ent.
transferring TQ M from the manufacturing environm ent to the health care environm ent
are discussed in Chapter I. Additional discussion is included in this chapter, along with a
more in-depth analysis o f the relevant literature. Next, quality m odels from the health
care literature are discussed. Finally, research from the health care literature that is
relevant to the concepts addressed by the criteria o f the M alcolm B aldrige National
Quality A w ard in Health Care is discussed.
conducted in hospitals, review of the health care literature is focused on research that has
been done in the hospital environm ent.
As discussed in Chapter 1, there has been little em pirical research on the
application of TQ M in the health care environment. Bigelow and A m dt's (1995) review
of the literature shows that most TQM research in the health care environm ent can be
classified in three general categories: (1) descriptive or definitional, (2) implementation
issues, and (3) case studies. Bigelow and A rndt identify five em pirical studies that
evaluate TQ M in practice. However, these studies report only very general information
24
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
on TQM in practice (e.g., questions such as, "Does your hospital have a TQM
program?"), and none of them provides empirical validation o f the effectiveness of TQM
programs in hospitals.
processes for meeting the com panys custom er and perform ance requirem ents. The
Driver and System influence the two results criteria. Business R esults and Customer
Focus and Satisfaction.
directional and bi-directional arrow s which indicate the hypothesized influence of the
categories on one another (NIST, 1995a).
25
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
The original M BNQA was designed in 1987 for manufacturing organizations, and
a separate category for award applications was established for service organizations in
1990. This introduction signaled a recognized need for service organizations to have a
structured fram ew ork by which TQM could be analyzed and measured. The M BNQA
criteria for service organizations are the criteria used for m anufacturing organizations.
There is also a separate award category for small businesses. In 1994, a pilot program
was undertaken to add two more categories to the Baldrige A w ard, health care and
education.
The basic frameworks for both o f the new categories are sim ilar to the
original M BNQA (NIST, 1995a, 1995b, 1995c), although unlike the service and small
business categories, the health care and education criteria were specifically designed for
the two industries.
W ilson (1997), W ilson and C ollier (1998a), and Handfield and G hosh (1995)
report analyses o f em pirical studies of the original Baldrige criteria in m anufacturing
environm ents.
He uses structural
equation m odeling to show that Leadership has a positive causal influence on each of the
System categories and two o f the System categories. Inform ation and A nalysis and
Process M anagem ent, have positive causal influences on both of the results categories.
In general, W ilson reports findings that support the theory o f the B aldrige m odel of
TQM , that leadership and management o f system s w hich focus on q u ality lead to
enhanced results. W ilson's results are described in further detail in C hapter 3.
W ilson and C ollier (1998a) report that the original Baldrige m odel explains 46
percent of the variance in customer satisfaction and 39 percent of the variance in financial
results in the m anufacturing environment. These results indicate that the B aldrige model
of TQM is closely tied to both customer satisfaction and firm perform ance, and that the
26
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
im plem entation o f the concepts presented in the Baldrige criteria generally results in
improved firm performance.
H andfield and Ghosh (1995) report results that are sim ilar to W ilson (1997).
Leadership has a positive causal influence on each o f the Baldrige System categories.
Process M anagem ent and Strategic Planning are found to influence the Baldrige-defined
C ustom er S atisfaction category while H um an R esource M anagement influences the
Business Results category.
A fram ework sim ilar to the original M BNQA is developed by Flynn et al. (1994),
but unlike the M BN Q A which is purported to be appropriate for any industry, Flynn et
al.'s fram ew ork is specifically designed fo r m anufacturing firms.
M BNQAs Leadership category. Top M anagem ent Support is the driver in Flynn et al.'s
framework of organizational TQM. The interm ediate criteria in this framework include
six categories th at are sim ilar to the B aldrige criteria: Quality Inform ation, Process
M anagem ent, P roduct Design, W orkforce M anagem ent, Supplier Involvem ent, and
Custom er Involvem ent. The outcome m easures in this framework are hierarchical in
nature, where continuous improvement o f m anufacturing capability leads to custom er
satisfaction, w hich in turn leads to com petitive advantage for the firm. In testing their
framework, Flynn et al. use two performance measures: (1) percent o f products shipped
without rework; and (2) perception of quality program 's contribution to plant's distinctive
competence. Flynn et al. test their framework by developing a scale to m easure each of
the categories in their model.
First, the two performance m easures are not broad enough in scope to
27
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
In sum m ary, the TQM models presented in the m anagem ent literature have been
designed to apply to either m anufacturing environm ents or all types o f firms. These
models do not account for the com plexity and variety o f issues that are present and
pervasive in the health care industry. As a result, TQM m odels were not widely used in
the past in the health care industry, and quality models more specific to this industry have
been developed as discussed below.
28
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
To date, no m ultiple-
hospital studies have adequately evaluated the causal relationships among the constructs
of a comprehensive quality program.
The M BN Q A in Health Care, shown in Figure 1.1, is one framework o f TQ M that
has been introduced in the health care industry. This model is described in C hapter 1.
Other models for evaluating quality in health care organizations are discussed here.
D onabedian (1980) presents a simple model that measures and evaluates quality
at three levels in a health care organization. The low est level of m easurem ent is the
structure level that is defined as the resources available to provide a particular service.
For example, the scope of services offered by a hospital laboratory is a measure o f quality
at this level. The middle level of measurement is the process level that is defined as the
extent to which professionals perform according to accepted standards. For exam ple, the
interpretation of the results of a laboratory test is a measure of quality at the m iddle level.
The highest level o f quality assessment is the outcom e level at which the change in the
health, knowledge, or behavior of patients is observed. The diagnosis o f a patient as a
result o f laboratory tests is an example o f a quality measure at this level. T he focus of
this quality model is measuring all of the relevant aspects of quality in health care.
The Q uality-Profitability Model introduced by Harkey and Vraciu (1992) is an
adaptation of G arvins model which finds strong associations between product quality and
profitability (G arvin, 1988).
29
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
approach predicts that quality will lead to profitability through both m arket gains and
reduced costs. M arket gains are achieved as improved quality leads to increased patient
and physician satisfaction, building of reputation and loyalty, increased m arket share and
potentially higher prices. At the same tim e, quality drives cost reductions because it
im proves productivity and lowers the cost o f treating each patient. Em pirical evidence
presented in this study shows that while quality is a significant predictor o f profitability,
environm ental variables are the only other significant variables in this m odel.
The
percentages o f m arket share insured by M edicare and m anaged care plans have
significant negative effects on profitability in this analysis.
A fram ew ork of TQM which includes categories for knowledge, leadership, tools,
and daily w ork application is presented by Batalden and Stoltz (1993). The building of
organizational know ledge includes the specific knowledge o f health care professionals,
knowledge about system s, and understanding process variations. Leadership is derived
from a h o sp ital's m ission and the integration o f the mission into the values o f the
workforce. Tools and methods that are used to develop TQM include statistical analysis,
planning, understanding processes, and collaboration between departm ents. Finally, the
integration o f TQM into the daily work of hospital employees gives them opportunities to
test new m odels or processes for delivering health care and to review and evaluate
im provem ents. A case study is used to evaluate this model, and the application appears
to support B atalden and Stolz's framework. Participants in the case study observed that
"one im provem ent seem ed to lead to another."
A nother approach is presented by N elson et al. (1996) who introduce the Clinical
V alue C om pass, a four-dim ensional approach to managing health care, quality, and
value.
N elson et al. believe that the follow ing four dimensions should be explored,
satisfaction. C osts include direct medical costs (e.g., ambulatory care, inpatient services,
30
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
m edications) and indirect social costs (e.g., days lost from w ork, caregiver costs).
Functional health status is defined as physical function, m ental health, and social/role
function.
satisfaction with the health care delivery process and patient's perceived health benefit
from care received. Health care organizations are encouraged to identify key measures of
outcomes and costs for each o f their clinical processes.
The aim o f the Clinical Value Compass is to accelerate clinical improvements and
to use a variety of methods in making improvements (Nelson et al., 1996). This approach
is unique in that it accounts for three types of outcomes from m edical care, or in effect,
three types of quality: functional, clinical, and satisfaction-related. Under this approach.
Nelson et al. advocate using measurement systems to quantify the quality of processes as
they occur, not as add-ons, so that front line staff who deliver patient care also measure
the processes that they use and their outcomes.
R esearchers studied the original MBNQA criteria in hospitals prior to the
introduction o f the 1995 criteria that are specifically d esig n ed for health care
organizations. Jennings and W estfall (1994) develop a m easurem ent instrument that
illustrates how the Baldrige criteria can be used to identify a hospitals strengths and
weaknesses. Their resulis indicate that there tend to be sim ilarities across hospitals in the
areas of quality m anagem ent that are strengths and those that are weaknesses. They find
that most hospitals have strong leadership, management of process quality, and customer
focus. W eaknesses included em ployee involvement in planning processes, design of
quality systems, and cross-departm ental cooperation. The m easurem ent instrument used
in Jennings and W estfall's study is based on the original M BN Q A criteria and includes
one scale for each o f the seven main criteria categories. The m easurem ent instrument
31
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
used in the study presented here is based on the MBNQA in Health Care criteria and
includes a scale for each of the 28 dimensions that comprise the seven main categories.
Shortell et al. (1995) and Carman et al. (1996) use the original (i.e., non-health
care, industrial) MBNQA criteria to m easure the extent of TQM im plem entation in
hospitals.
These researchers (both articles are from the same team o f researchers)
develop a m easurem ent instrum ent m uch like Jennings and W estfall's (1994) that
includes one scale for each of the seven m ain categories of the original M BNQA. While
Shortell e t al. and Carman et al. use m easures o f the MBNQA constructs to judge
whether the studied hospitals have high o r low levels of TQM implementation, but there
is no evaluation o f the constructs them selves or the relationships am ong them. The
Baldrige constructs and the relationships among them are addressed in the current study.
C arm an et al.'s (1996) study uses data from both hospital em ployee perceptions
and researchers observations from visits to each of the participating hospitals. These
two sets o f data are found to be highly correlated. The performance measures used in this
study are: change in patient satisfaction (over one year); change in adjusted cost per
admission (over two years); change in m arket share (over two years); change in average
length of stay (over three years); and change in labor productivity, as measured by full
time equivalency personnel per adjusted adm ission (over three years). The researchers
use em ployee perceptions to analyze how well the Baldrige constructs predict hospital
performance.
C arm an et al. develop three m easures that they term "Q uality Im provem ent
outputs" and base them on their "conceptualization of the Baldrige schema" (1996 p. 54).
The first is a sum o f scales (from Shortell et al., 1995) that m easure the following
constructs: leadership, strategic quality planning, education, em powerm ent, information
and analysis employed, and m anagement o f the quality improvement process. It should
be noted that som e o f these scales do not directly correspond to B aldrige categories.
32
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Carman et al.'s second m easure of outputs is a scale for "quality results," and the third
measure is a scale for "the custom er satisfaction score" (1996 p. 54).
Results o f
regression analysis show that none of the output m easures significantly predict the
performance measures. Regression paths weights and significance levels are not reported
making further interpretation of the results impossible.
In a study by Sanders (1997), each of the 92 responding hospitals report that they
have a quality program, but one-third of them have im plem ented quality initiatives in
only select departments. O f those with a hospital-wide program , 95 percent view their
quality program as perm anent. In contrast, in hospitals w ith quality programs in only
select departments, only 33 percent see the programs as perm anent.
(1)
33
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
valuable in improving health care processes; (3) D ata useful for quality im provem ent are
abundant in health care (i.e., health care organizations docum ent extensively);
(4)
Q uality improvement m ethods can be fun (i.e., teams enjoy exploring new m ethods and
making informed changes); (5) Costs of poor quality are high (i.e., the industrial quality
experts report extensive w aste, rework, complexity, and process variation in the health
care organizations); (6) Involving doctors is difficult; (7) Training needs arise often
(e.g., training leaders in fundam ental quality concepts); (8) Nonclinical processes are
easiest to address first (i.e., clinical quality is considered "doctors turf"); (9) H ealth care
organizations may need a broader definition o f quality (e.g., "M any hospitals ... apply the
term [quality] exclusively to hands-on medical care" p. 156); and (10) In health care, as
in industry, the fate of quality improvement is first o f all in the hands of leaders.
Berwick et al. discuss two important shortcom ings o f the NDP. First, only a few
health care organizations attem pted to improve clinical processes (patient m edical care
processes), and no organization measured outcomes in terms o f improved patient health
status. This is a shortcom ing because the NDP provides no results that support using
quality management to improve patient health care. Second, given the limited duration of
the project (eight months), no organization addressed changing its organizational culture.
Therefore, none of the health care organizations undertook a "transformation ... through
the adoption o f m anagem ent m ethods that allow quality im provem ent activities to
flourish" (Berwick et al., 1990 p. 26).
In summary, while the NDP provides evidence that quality management practices
are useful for improving processes in health care organizations, most of the participating
organizations addressed problem s that have previously been addressed in non-health care
settings. The next im portant step for health care organizations is to address processes
that are unique to the health care environment.
34
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2.2.1.1 Leadership
The Leadership category addresses "senior executives' and health care s ta ff
leaders personal leadership and involvement in creating and sustaining a focus on patient
care and organizational m ission, clear values and expectations, ... quality health care
services and performance excellence, ... public responsibilities and citizenship" (NIST,
1995b p. 13).
Description of concept
Leadership in health care organizations includes the governing board (board o f
directors), senior executives, faculty leaders (physicians), and staff leaders such as highranking nurses and physicians. One of the responsibilities that the MBNQA in H ealth
Care assigns to health care organization's leadership is to effectively com municate and
reinforce the organization's mission to the entire staff. The staff includes all paid staff,
independent practitioners such as physicians and nurse practitioners, and unpaid staff
such as volunteers and health profession students.
The Baldrige Leadership category also includes a responsibility to the public
served by the organization in areas such as preparedness for natural disasters (e.g.,
35
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
earthquakes, m ajor industrial accidents), ethical conduct, and public disclosure of health
care performance.
Previous research
There are two main sources of hospitals m ost influential leadership, hospital
executives and hospital boards.
literature as different from their non-health care counterparts. Hospital adm inistrators
tend to be more introverted and creative than business executives (G riest, 1990).
Twenty-seven percent o f surveyed hospital executives say they chose their profession to
satisfy their interest in medicine, rather than an interest in management. T hey are a
highly educated group, with 93 percent of top hospital administrators having a graduatelevel degree.
Non-profit hospital governing boards are more involved in the strategic decision
making processes of their organizations than the boards o f non-health care businesses in
the private sector (Judge and Zeithaml, 1992). This study provides only a com parison o f
non-profit hospitals with non-health care businesses and does not include analysis o f for
profit hospital boards. Studying board involvem ent in strategic planning in for profit
hospitals would have shown whether non-profit status or the health care industry itself is
the differentiating factor.
Alexander and Morlock (1985) report on a 1979 study conducted by the American
Hospital Association that finds hospital participation in multi-hospital arrangem ents is
positively related to increased formal control by the governing board over the hospital
chief executive officer, particularly in privately ow ned hospitals.
There is anecdotal evidence in the literature that good leadership leads to better
quality outcomes in hospitals (Batalden and Stoltz, 1993). In this case study, one hospital
36
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
found that C Q I im plem entation is accelerated by the strong support o f top hospital
administrators.
D im ension
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
A health care organization's methods for addressing ethical issues is also part of
the public responsibility that the M BNQA in H ealth Care criteria emphasize. M edical
ethics guidelines are generally set by health care organizations and professional groups.
Researchers studying the guidelines o f the A m erican College o f Physicians Ethics
Manual find that 98 percent o f 400 responding physicians report actions that do not meet
those guidelines (Green et al., 1996). Because a hospital often bears responsibility for its
physicians' actions (although the physicians are often not employed by the hospital), it is
im portant that e th ic s be addressed on an organization-w ide basis.
M ethods for
developing an ethics com m ittee are discussed by Kluge (1996) and Dalgo and A nderson
(1995).
Description of concept
Sources o f inform ation and data to be analyzed in health care organizations
include medical outcom es, financial, accounting, m arketing, and com m unity health
services. Data should be standardized so that data, codes, and terminology transfer across
departments and are com patible with both internal and external databases. For exam ple.
M edicare has standard form ats for collection and reporting of data that participating
health care providers m ust use. Users o f inform ation and data are internal and external
custom er groups, including other departm ents, patients, insurers, other payers, and
accreditation groups.
38
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Previous research
Many businesses find that information system s are critical to their grow th and
success. Hospital adm inistrators and managers benefit from training on the m ethods and
benefits of utilizing the information that is available to them for identification of process
improvement potential and for decision support (Skibicki, 1994). Hospital adm inistrators
see integration of system s across separate facilities as the most important priority for
information system s in the next few years (Bergman, 1994). In addition, many hospitals
have implemented or are moving toward a computer-based patient record.
Setting up a functionally appropriate inform ation system in a hospital requires
extensive planning. Information systems must meet both internal and external needs and
reporting requirem ents. Requirements include standardization to external requirem ents,
linkages between internal systems, comprehensive population-based inform ation, and
confidentiality of patient records. These and other goals require a systematic approach to
information system development (DesHamais et al., 1994).
Information system s can enhance quality initiatives in health care organizations
by providing perform ance feedback data on which to base future decisions.
Information
systems can also be used for tracking the progress o f quality programs. One hospital uses
its information system to track process improvement teams and for sim ulation m odeling
of design processes (Goldberg et al., 1995). The process improvement teams use the
information system to gather and link data that are needed for making process changes.
They also use the system to evaluate the results of the changes that they im plem ent. Set
up and evaluation o f inform ation systems is evaluated in Dimensions 2.1 and 2.3,
respectively, of the M BNQA in Health Care criteria.
Dimension 2.2 of the MBNQA addresses the use of benchmarking for com petitive
com parisons.
reengineering their adm issions process after benchm arking against industry leaders
39
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
(Collins, 1994). They find that understanding their current process and being flexible
with job functions and allocation of resources are critical to the success o f such efforts.
The M BNQA addresses benchm arking from the standpoint o f in teg ratin g such
information in the planning processes of an organization.
Description of concept
Only for profit firms are eligible to apply for the original MBNQA. In contrast,
for profit, non-profit, and government owned health care organizations can apply for the
MBNQA in Health Care. (Hereafter, the term "non-profit" refers to both non-profit and
government owned facilities.)
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
professionals.
Using the M BNQA in H ealth Care criteria for ev alu atin g quality
m anagem ent in these various types o f organizations assumes that the B aldrige m odel of
TQM is appropriate for all of them.
Previous research
The basic concept of developing strategies in health care organizations is similar
to strategic developm ent in other industries.
corporate com petencies and resources are evaluated, and values and aspirations are
accounted for in strategy developm ent (Andrews, 1980). But A ndrew s presents a fourth
dom ain for health care organizations to include in their strategic p lanning, societal
obligations which take on much greater w eight than in most other industries. The dual
focus betw een financial performance and societal obligations results in a m ore complex
set of perform ance measures on which health care organizations are evaluated at the back
end, m aking the strategic planning process more complex at the front end.
T he focus of Dimension 3.2 o f the criteria of the MBNQA in H ealth C are is on
the deploym ent o f strategic plans. Top m anagem ent involvement in visioning, driving
decision-m aking downward, interactive planning, team building, and m anager training
are the keys to successfully deploying a TQM strategy in health care organizations, much
like in other industries (Gelinas and M anthey, 1995).
Several classification schemes have been developed to define coherent groups of
hospitals based on their strategies. A sim ple classification is presented by Zelm an and
Parham (1990) that identifies two strateg ic groups, G eneralists and Specialists.
G eneralists, who offer a wide variety o f health care services, survive in the m arketplace
by relying on excess capacity, resource variety, and their ability to accom m odate
changing m arket conditions.
utilizing their capacity to maximize efficiency. Generalists tend to be less efficient than
41
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
variation in strategies because hospitals position them selves in the middle of these two
extrem e strategies.
benefits o f each extreme strategy w ithout the success or risk o f moving away from the
median. Calem and Rizzo show that hospitals may be m ost viable in the marketplace if
they choose a strategy that is "m iddle of the road" rather than either generalist or
specialist.
Pegels and Sekar (1989), Nath and Sudharshan (1994), and Butler et al. (1996)
provide additional hospital strategy classification schemes.
42
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
H ospitals with a proper fit between their environm ent and strategy have the
highest perform ance, and hospitals can improve their perform ance by changing their
strategy to achieve that fit (Lamont, 1993; Zajac, 1989). G oes and Meyer (1990) study
hospitals strategic changes longitudinally and find th at their strategic changes are
infrequent, especially am ong the best performing hospitals.
Recent research on hospital strategies indicates that pursuing multiple strategies
may be more appropriate than pursuing a single strategy (Ashm os et al., 1996). The
reason, in part, is that hospitals may compete with specific medical services provided by
other hospitals or other types of facilities, but they m ay not compete with the other
organizations as a w hole.
addressed with different strategies, resulting in multiple strategies for hospitals which
provide multiple services.
Some specific environm ental and organizational situations influence strategy
development and deploym ent in health care organizations. A California hospital was
forced, by market competition, to become involved in a health maintenance organization
(HMO) that prepaid the hospital a fixed amount for each patient assigned to its care
(Eckholm, 1995). G iven the fixed payments, the hospital had to contain its costs to treat
HMO patients w ithin the paym ent provided by the H M O .
potential resulted in the hospital outsourcing some procedures, reducing the num ber of
tests and referrals to specialists, and the departure of som e specialists who w ould not
43
Reproduced with permission o f the copyright owner. Further reproduction prohibited without permission.
work under the new conditions. The strategy o f the hospital, although designed to ensure
the survival of the institution, effectively chased many physicians away.
Solovy (1995) show s how strategies differ by region in the U.S.
Because
hospitals generally draw customers from their immediate geographic locale, they m ust
compete within their regional marketplace. Environm ental and organizational factors
influence hospitals' likelihood to merge and cluster (Olden and Luke, 1996).
This
strategic response to market threats is undertaken to protect against revenue losses and
closure. The environm ental factors that m ost strongly predict a hospital's likelihood to
merge with other hospitals are a large number o f other hospitals in the im m ediate region,
greater penetration o f HMOs into the region, and a low ratio of registered nurses-perpopulation. The organizational factors that are associated with a greater likelihood of
merging are for profit ownership, non-governm ent ownership, and size (indicated by
number of beds).
It is also w orth noting one application o f a strategic planning tool (G raf, 1996).
An Im portance-Perform ance-A w areness (IPA ) map is presented as a tool for more
efficiently m atching internal capabilities with external market needs in the health care
environment.
maintain an environm ent conducive to performance excellence ..." (NIST, 1995b p. 20).
44
Reproduced with permission o f the copyright owner. Further reproduction prohibited without permission.
Description o f concept
The M BN Q A in Health Care criteria define human resources to include paid staff
(employees and contractors), independent practitioners (physicians, physician assistants,
and nurse practitioners not em ployed by the organization), v o lu n teers, and health
profession students (medical, nursing, and ancillary) (NIST, 1995b). T he staff o f health
care organizations is more complex than that o f many other organizations where multiple
classifications such as the ones noted here are not necessary. Interestingly, the MBNQA
in Health Care criteria define health profession students as both staff and custom ers of the
organization in which they train. Some authors consider physicians to have a dual role as
both custom ers and staff (B erw ick et al., 1990), w hile o th er au th o rs and most
practitioners consider physicians to be custom ers (Nelson et al., 1992). Physicians are
defined as staff in the Baldrige criteria.
Previous research
Q uality team s, consisting o f the sta ff o f an o rg an izatio n , are an essential
component o f TQ M . Quality teams are form ed in health care organizations in much the
same way that they are formed in other organizations (G oldberg and Pegels, 1984).
Zmud and M cLaughlin (1989) discuss the intluence of top m anagem ent on team s, saying
that m anagem ent should signal the significance o f the effort to have successful teams,
provide feedback and motivation for w orkers, and provide recognition and rewards for
team performance.
Quality teams have been successful in some health care organizations and not in
others.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
custom er needs com ing first and being involved in decisions regarding their jo b s are
important components of TQ M . Conversely, the failure of quality team s in health care
organizations is due to several causes such as failure to recognize the 'systems' nature of
hospitals, failure to incorporate physicians into plans, and failure to allow quality teams
to deal with non-work issues.
Job design and jo b assignm ents in health care organizations are im portant to
productivity measures and staff effectiveness. Labor statistics show that the proportion
of professional and technical workers in the health care industry increased from 1983 to
1989 (Anderson and W ootton, 1991). The wide-spread use o f sophisticated medical
technology, more specialized services, and more acute (m ore seriously ill) patients has
increased demand for highly skilled workers. Although in m any industries technology
replaces the need for skilled workers, in the health care industry, advanced technologies
generally require a greater num ber o f highly skilled physicians, nurses and technicians.
A t the same time, reengineering practices are shifting less technical and non-medical
duties to lower skilled (and low er paid) workers.
A typical patient in one Indiana hospital interacts with an average of 28 different
hospital w orkers per day, including nurses, physicians, cleaning and food service
personnel, laboratory technicians, etc. (Ferret, 1996).
personal interactions indicates a com plexity not found in m ost other industries. The
personnel who interact w ith patients m ust be trained for their job, with various medical
and non-medical skills, but they also need customer service skills and interpersonal skills
in order to ensure a satisfactory and appropriate interaction with patients.
Lathrop (1991) reports on a three-year study by Booz-Allen and Hamilton which
reveals that the extent of compartm entalization of services in U.S. hospitals leads to poor
service for patients and results in high costs. Lathrop suggests that hospitals undertake a
46
Reproduced with permission o f the copyright owner. Further reproduction prohibited without permission.
fundam ental restructuring and design services around patients, terming the result the
'patient-focused hospital.'
Tidikis and Strasen (1994) report that while 80 to 95 percent of hospital staff are
licensed, only 20 percent o f the tasks that these individuals perform require licensure.
M anthey (1991) finds nursing assistants to be o f lim ited value unless their skills are 75
percent of those of registered nurses. These contrasting views of the personnel needs o f
hospitals exem plify the confusion over the type of w orkers and job design needed to
m aximize productivity and patient satisfaction. T idikis and Strasen (1994) argue for a
less trained workforce w hile Manthey wants higher skill levels. The most appropriate job
design depends on the needs o f a particular health care organization and the services that
it offers, but clearly this issue is critical to maximizing staff productivity.
D im ension 4.4 of the MBNQA in H ealth C are criteria addresses the w ork
environment and how it affects employees' well-being. W illiams et al. (1994) report that
hospital m anagem ent, physicians, and union representatives all agree that the w ork
environment should be a priority of occupational health services.
Description of concept
Process m anagem ent is evaluated by six dim ensions in the MBNQA in H ealth
Care criteria. The design and delivery of the following processes are evaluated: patient
47
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
care services, support services, com m unity services, adm inistrative and business
operations managem ent, and supplier management.
Patient care services are "those services provided directly to the patient for the
purpose o f prev en tio n , m aintenance, h ealth prom otion, scree n in g , d iag n o sis,
treatment/therapy, rehabilitation, or recovery" (NIST, 1995b p. 24). Support services are
"those which support the organization's delivery o f health care services to patients" (e.g.,
laboratory results, housekeeping, m edical records) (NIST, 1995b p. 26). C om m unity
services are "population-based services supporting the general health o f the com m unity
served" (e.g., educational, im m unization, and population screening (cholesterol)
programs; indigent care) (NIST, 1995b p. 27). Administrative and business operations
managem ent includes "finance and accounting, information services, plant and facilities
m anagement, m aterials management, planning and marketing..." (NIST, 1995b p. 27).
Supplier m anagem ent refers to the m anagem ent o f "outside providers of goods and
services" (NIST, 1995b p. 28).
Previous research
D im ension 5.1 of the M BNQA in H ealth C are criteria advocates the need to
include patient input in the design and redesign of services (medical and non-m edical).
Jacono (1990) discusses differences in patient and patient family needs in hospitals and
nurses' perceptions o f those needs. Jacono's results show that nurses are not adequately
able to predict the needs of patients and their families, indicating that the needs of
patients and patient families are not always apparent to hospital employees. A formalized
ongoing patient and patient family feedback mechanism instills the benefits of continuous
quality im provem ent into service design for the organization (B lack in g to n and
M cLauchlan, 1995; Health Care Strategic M anagem ent, 1994).
A m ethodology for
48
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Louden (1991)
This becom es
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
(1991) predicts that more partnerships between buyers and users of medical products will
be formed in the future. Just-in-time and stockless relationships are exam ples o f these
'value-added' partnerships.
Description o f concept
As described in C hapter 1, no single m easure is adequate to evaluate the
performance o f a hospital. Therefore, most hospitals use a set of perform ance measures
on which they can broadly evaluate their perform ance both internally and externally in
regards to their standing in the marketplace. For profit, non-profit, governm ent, and
teaching hospitals should each use perform ance m easures that are most appropriate for
their organization.
Previous research
T here is little agreem ent on the sp ecific perform ance m easures that are
appropriate for evaluating the outcomes and perform ance of hospitals. However, there is
general agreem ent that no one measure accurately reflects the perform ance o f these
complex organizations. Anderson (1991) reports 81 performance m easures (developed
by N erenz and Z ajac) that evaluate financial perform ance, quality perform ance,
community service, and customer satisfaction. A nderson proposes that hospitals choose
50
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
perform ance m easures from this com prehensive list that most adequately reflect their
m ission.
For exam ple, the financial loss of an em pty hospital bed (estim ated at
approximately $36,500 per year in 1987) m ust be balanced against the com m unity benefit
of excess capacity and availability o f hospital beds (Gaynor, 1995). H ere, financial
performance may suffer at the expense of improved service to the community.
The m ost reasonable and accurate com parisons of hospitals include evaluation of
a variety of perform ance measures. HCIA's annual selection of the top 100 hospitals in
the U.S. is based on eight performance criteria. Data from these top hospitals indicate
that organizations with good quality perform ance also tend to have good financial
performance (M orrissey, 1994). The performance criteria consist of financial m easures
(expense per discharge, cash-flow margin, and long-term debt to total assets), operations
measures (average length of stay, investment in capital assets, and charge per discharge),
and clinical outcome measures (mortality and complication rates).
Public dem and for information on hospital performance has been grow ing in
recent years (Jaklevic, 1996a).
publishing mortality rates, lengths of patient stays, and prices (Greene, 1996). M ichigan
hospitals are reporting mortality rates and lengths of stays (Jaklevic, 1996b). Health care
managers and professionals in both states are concerned with the publics ability to
correctly interpret such information.
Some studies evaluate hospitals based on their bond rating, using it as a proxy for
financial perform ance (Pallarito, 1996), because it is difficult to compare the financial
performance o f for profit and non-profit hospitals. But others believe that bond ratings
are too easily m anipulated and are not appropriate measures o f perform ance (Prince,
1994).
There are several medical outcome measures that can be used to evaluate hospital
performance. However, differences in outcomes across hospitals are the result o f several
51
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
factors.
D esH arnais et al. (1988) develop a m odel o f four factors that influence
outcom es.
adm ission;
(2) the patient's physiological reserve, including age, sex, and health
condition;
(3) the patient's social condition, such as living arrangem ent and financial
situation;
T he rate o f patient
readm issions to hospitals and complication rates are also used to evaluate the quality of
medical care provided by hospitals (DesHarnais et al., 1991). In a study o f 8000 patients,
500 surgeons, and fifteen hospitals, D esH arnais et al. find that features associated with a
hospital are better predictors of medical outcom es, as measured by adjusted mortality and
m orbidity rates, than surgeon ch aracteristics.
expenditure per patient day is the best predictor o f patient outcom es, w hile surgeon
q u a lific a tio n s (m easured as the num ber o f residencies co m p leted ) and surgeon
com m itm ent (m easured as proportion of a physicians work com pleted at the studied
hospitals) are also significant predictors.
A ttem p ts to use outcom e m easures such as m ortality, read m issio n , and
com plication rates to evaluate hospitals are ham pered when a m edical procedure or
illness is associated with either a small num ber o f patients or a low rate o f poor outcomes
(Luft and H unt, 1986). A small number of patients or a low rate of poor outcomes makes
it difficult to achieve statistically significant findings. For exam ple, a sam ple size of
52
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
conclusions about the com plication rates resulting from the two treatm ent methods in a
smaller sample, they do not achieve statistically significant results because the population
o f patients from which to sample is limited.
Although there is little em phasis on financial results in the Baldrige Health Care
criteria. D im ension 6.4 addresses m easures o f productivity, co st reduction, asset
utilization, etc. Eagle et al. (1994) discuss a case study w here a quality program is
introduced in a hospital anesthesia departm ent. They find that there are measurable
financial benefits, but that the human resources costs o f im plem enting the program far
outweigh the financial benefits. However, participants in the study conclude that their
five-year investigation of this quality program provided many unmeasurable benefits
such as improved education, research, and patient and practitioner satisfaction.
A dm inistrative perform ance is addressed in Dim ension 6.4 o f the M BNQA in
Health Care. Administrative expenses account for an average of 24.8 percent o f hospital
costs in the U.S., adding approximately $250 billion per year to national health care costs
(W oodhandler et al., 1993). Although administrative costs vary by state, they compare
poorly with Canada's rate o f 9 to 11 percent of hospital costs. The ability of a hospital to
control its adm inistrative costs may be indicative of its ability to control other costs
because some types of costs, such as labor and nonlabor costs, are significantly correlated
(Healthcare Financial M anagement, 1995).
Lack of evidence o f the success o f quality program s is a common barrier to
implementation, according to health care administrators (G ustafson and Jundt, 1995).
Gustofson and Jundt's review o f the literature indicates that there is little evidence of
financial benefit from TQ M , but that the application of TQM principles is generally
53
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
TQM im plem entation in manufacturing firms (e.g., Easton and Jarrell, 1998; W ilson,
1997; Hendricks and Singhal, 1996; Handfield and Ghosh, 1995).
The literature does report that other m easurable organizational im provem ents,
such as custom er satisfaction (which is discussed in the following section), appear to
benefit health care organizations. This lack o f focus on financial benefits may be more
evident in non-profit and government owned hospitals in which profitability and financial
performance are likely addressed in a different manner than in for profit hospitals. But
based on the literature, the lack of financial benefits from quality programs appears to
transcend all organizational types in the health care industry.
54
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Description of concept
Much of the health care literature approaches custom er satisfaction as pertaining
only to patients. Patient fam ilies are sometimes defined as custom ers when associated
with a pediatric patient, but m ost o f the other stakeholders in this environm ent are
virtually ignored with a few exceptions that are discussed in this section. The MBNQA
in Health Care criteria define the term custom ers as patients and other stakeholders,
including patient families, com m unities, insurers or third-party payers, employers, health
care providers, and professional students in medical fields (NIST, 1995b).
Satisfying all custom er groups is extremely complex because there are trade-offs
in the requirements of each group (Gopalakrishnan and M clntrye, 1992). In hospitals,
patients, physicians, and students generally prefer increased spending, while insurers and
third-party payers, including governm ents, prefer decreased spending. Likewise, while
physicians, students, and governm ents want to further the know ledge, education, and
training of the medical profession, patients may have little immediate interest in these.
Previous research
Improving custom er satisfaction has become more im portant to hospitals as a
means of attracting patients in com petitive markets. Basic concepts o f customer service
had not been broadly applied in this industry in the past because patients were not defined
55
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
as custom ers.
custom er satisfaction, hospitals find that better screening in the hiring process and
m entoring new em ployees help to improve the custom er service skills o f their employees
(M ontague, 1995).
The B aldrige Health C are model requires m easurem ent o f patient and other
stakeholder satisfaction. Hospitals must have knowledge and an understanding of these
custom er groups perceptions o f their organization. Tenner and DeToro (1992) use a twodimensional framework to illustrate the relationship between the approach used to gather
custom er feedback and the level of understanding achieved from that approach. At the
lowest level, m anagem ent relies on custom er com plaints for feedback. This reactive
approach to gathering inform ation results in a very low level o f understanding o f
custom er perceptions. At an intermediate level, the organization actively listens to the
custom er and reacts to their needs.
understanding com es from using proactive approaches such as personal interviews with
custom ers, focus groups, and specially designed surveys. The organization achieves a
deeper understanding of custom er expectations and perceptions at this level.
Patient satisfaction with hospital services has been m easured using a variety of
instrum ents.
m easures custom er perception o f quality as a gap betw een their expectations and their
perceptions of organizational performance (Parasuraman et al., 1988). SERVQUAL was
originally designed to evaluate customer perceptions based on ten dimensions of quality tangibles, reliability, responsiveness, competency, courtesy, com m unication, credibility,
security, access, and understanding. SERVQUAL was later reduced to five dimensions
that are believed to encom pass the dimensions of quality that are necessary to adequately
56
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
assess custom er's perception o f quality. Those five dim ensions are tangibles, reliability,
responsiveness, assurance, and empathy.
Evaluating SERV Q UA L using hospital patients shows that the five-dim ension
instrum ent may include dimensions that are not necessary in this industry and lack
quality dimensions that are unique to health care (Bowers et al., 1994). Q ualitative and
quantitative analyses reveal that dimensions for caring and com m unication should be
added to SERV Q U A L if it is to accurately measure the quality perceptions o f hospital
patients. The caring dim ension must go beyond courtesy from service providers to a
level of personal involvem ent by hospital caregivers that approaches love, according to
patients in Bowers et al.'s study. The communication dim ension is in the ten dim ensions
of the original SERV QU A L structure, but was dropped in the five-dimension version.
Additionally, the dimensions of tangibles, security, credibility, competence, and courtesy
(from the original model with ten dimensions) are not relevant in perceptions o f hospital
patients, and Bowers argues they should be dropped from the instrument w hen used to
evaluate patient satisfaction.
In a sim ilar study, Vandamme and Leunis (1993) report that the reliability
dimension of the SERVQUAL instrument is not important for measuring quality in health
care organizations. A dditionally, the low variability in patient perception scores (m ost
questions have an average score between 6.0 and 6.8 on a 7 point Likert scale) m ake
analysis of the survey data difficult. Babakus and M angold (1992) also find that several
SERVQUAL items are redundant and can be dropped. C ronbachs coefficient alphas (an
indication o f internal consistency among items) for Babakus and Mangold's final scales
are between .50 and .80, generally regarded as acceptable levels for new scales o r new
uses (in this case the instrum ent is new to the health care environm ent) for existing
scales.
57
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
stakeholder groups vary greatly. Sobczak et al. do not provide em pirical support for the
measures used to evaluate the satisfaction o f each o f these stakeholder groups, but
recognize their som etim es conflicting needs. This results in hospitals having to create
and measure satisfaction in a variety of ways. Fouler et al. identify nineteen internal and
external custom er groups which hospitals executives view as pow erful stakeholders.
Physicians top the list of powerful stakeholders and patients are judged to be second to
physicians, as shown in Table 2.1.
Different definitions of internal and external customers have been used by other
authors (Nelson et al., 1992). Here, internal custom ers are defined as physicians and
hospital employees, and external customers are patients, patient fam ilies, and third party
payers.
perceptions of physicians and employees, the two internal custom er groups, are highly
correlated. In a related study by Nelson et al. (1989), hospital em ployee perceptions and
patient perceptions o f quality are also highly correlated. These significant correlations
show that quality perceptions are consistent across a variety o f sources, providing
evidence that these instruments are reliable. These studies, along with Licata's (1995)
study described above, indicate that different types of customer groups often have similar
impressions of quality for a particular hospital.
satisfaction o f all o f the customer groups discussed here, but perhaps the opinions of
multiple custom er groups are redundant.
In recent years, there has been a strong push toward m easuring and reporting
customer satisfaction by managed care organizations, employers, and others. There has
also been a trend tow ard using standardized questions and m easurem ents so that these
results are com parable across health care organizations.
59
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2.3 Summary
Based on the literature review provided here, two main conclusions can be drawn.
F irst, the B aldrige H ealth C are A w ard criteria have not been ev alu ated as a
comprehensive causal model w ith com plex interactions am ong the categories. Second,
some of the concepts included in the seven categories (made up o f 28 dim ensions) of the
Baldrige criteria have been addressed in the health care literature, but not in enough depth
to fully evaluate the dom ain o f any o f the B aldrige c ate g o rie s o r dim ensions
(subcategories). The next step is to comprehensively study the Baldrige Aw ard criteria in
the health care environment.
60
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
61
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
CHAPTER 3
Category number
Category name
1.0
Leadership
2.0
Information Analysis
3.0
Strategic Planning
4.0
5.0
Process Management
6.0
7.0
62
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
O rganizational Performance Results (6.0) and Focus on and Satisfaction o f Patients and
Other Stakeholders (7.0).
A hospital with good Leadership (1.0) is expected to have a good System (2.0,
3.0, 4.0, 5.0) while a hospital w ith poor Leadership should have a poor System.
Hospitals with good and poor System s are expected to achieve good and poor results (6.0,
7.0) in a sam e manner. Schematics o f these proposed correlation relationships are shown
in Figure 3.1. The relationships addressed in the first set o f research hypotheses address
association among the Baldrige constructs, and no predictive influence is hypothesized.
The second set of hypotheses (presented in Section 3.3) tests aggregate causal
relationships among the award criteria. Leadership (1.0) is the driver o f the System (2.0,
3.0, 4.0, 5.0), which in turn cau ses R esults in the two perform ance categories,
O rganizational Performance Results (6.0) and Focus on and Satisfaction o f Patients and
O ther Stakeholders (7.0). A m odel showing the causal paths tested by the aggregate
causal hypotheses is shown in Figure 3.2. The structural equation m odel that is used to
test the second set of hypotheses is shown in Figure 3.3.
The third set of hypotheses (presented in Section 3.4) addresses m ore specific
relationships among the seven m ain Baldrige categories.
predicted to have a positive causal influence on each o f the four System categories in the
B aldrige m odel:
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
R esource D evelopm ent and M anagem ent (4.0); and Process M an ag em en t (5.0).
Leadership is also predicted to have a positive causal influence on each o f the two results
categories: O rganizational Performance Results (6.0); and Focus on and Satisfaction of
Patients and O ther Stakeholders (7.0).
predicted to have a positive causal influence on each o f the two perform ance categories.
Finally, O rganizational Performance Results (6.0) is predicted to have a positive causal
influence on Focus on and Satisfaction o f Patients and Other Stakeholders (7.0).
model showing the causal paths tested by these hypotheses is shown in Figure 3.4. The
structural equation model that is used to test this set of hypotheses is show n in Figure 3.5.
All research hypotheses addressed in this study are described in further detail below.
D efinitions o f the terminology used throughout this document are provided here.
These entities are used to describe the Baldrige model and the m ethodology used here to
test it.
3.2
Entity
Definition
Category
Construct
Dimension
Scale
Item
Correlation hypotheses
The first set of research hypotheses addresses correlation relationships among
constructs o f the Baldrige criteria. Leadership (1.0) is expected to be correlated with the
64
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
System (2.0, 3.0, 4.0, 5.0), and the System is expected to be correlated with Results (6.0,
7.0). The terminology used here reflects that used by the Baldrige criteria which refer to
the System as including the following Baldrige categories: Inform ation and Analysis
(2.0); Strategic Planning (3.0); Human Resource Developm ent and M anagement (4.0);
and Process Management (5.0) (NIST, 1995b).
The Baldrige Health Care model specifies linkages between Leadership (1.0) and
the System, as shown in Figure 1.1 (NIST, 1995b). The B aldrige model also specifies
linkages between the System and both results categories (6.0, 7.0), also shown in Figure
1.1 (NIST, 1995b). These linkages are grounded in the Baldrige theory that health care
organizations that successfully im plem ent Total Quality M anagem ent (TQM) have good
leadership, a good quality managem ent system, and good performance.
In this study, good Leadership is expected to be found in hospitals that also have a
good System. This relationship im plies that other hospitals have both poor Leadership
and a poor System. Likew ise, good System m anagem ent is expected to be found in
hospitals with good results, and poor System management should be found in hospitals
with poor results. These relationships are associative in nature, and there is no causal or
predictive relationship implied in the first set of hypotheses.
The number(s) of the Baldrige criteria category(s) referenced in each hypothesis
given below is included in parentheses. In these hypotheses, the mathematical notation
for subscript 2 refers to the System. Figure 3.1 shows a schematic of these hypotheses.
H ,:
Hospital Leadership (1.0) and System management (2.0, 3.0, 4.0, 5.0) are
positively correlated. [<j>12 > 0]
H 2:
Hospital System managem ent (2.0, 3.0,4.0, 5.0) and perform ance (6.0, 7.0)
are positively correlated. [<j)26 > 0; <f>27 > 0]
65
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
studies defines and evaluates quality m anagem ent system s and perform ance using
different measurements and methods, but the results all indicate that quality m anagem ent
systems are positively related to firm performance.
The premises contained in this first set of hypotheses have not been studied in the
hospital environment. This study will be the first com prehensive attempt to confirm or
disconfirm these associative relationships among the m ajor entities o f the B aldrige
framework.
The constructs included in the first set o f hypotheses. Leadership (1.0), the
System (2.0, 3.0, 4.0, 5.0) and Results (6.0, 7.0), are m easured using the questionnaire
described in Section 4.2. The constructs included in these hypotheses include the seven
Baldrige categories (1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0) that are multidimensional and include
a total of 28 dimensions (subcategories). A scale is developed to measure each of the 28
dim ensions.
For exam ple, the Leadership (1.0) category consists o f the follow ing
dim ensions:
66
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Z zX
y y
(3.1)
For exam ple, each hospital's score for the Leadership (1.0) construct is calculated
using the Baldrige dimension weights given in Table 3.1 as shown below:
67
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Y l.O
y,o
Z I . 1 ^ 'I . I
^ 1 . 2 ^ 1 .2
(3.2)
Z 1 . 2 ^ 1.2
Equation 3 .1 is used to calculate a score for each hospital on each o f the Baldrige
constructs. These construct scores (y^) for Leadership (1.0), Organizational Performance
Results (6.0), and Focus on and Satisfaction o f Patients and O ther Stakeholders (7.0)
provide the data necessary to test Hypotheses 1 and 2.
D eterm ining each h o sp ital's score for the System requires an additional
calculation. The System is made up o f the following Baldrige categories: Information
and A nalysis (2.0); Strategic Planning (3.0); Human Resource D evelopm ent and
M anagem ent (4.0); and Process M anagem ent (5.0). Each Baldrige category is assigned a
proportional weighted value in the Baldrige Award criteria (NIST, 1995b). The category
point values are shown in Table 3.1, and the proportions that indicate each category's
Baldrige weight relative to the total point value o f the System are calculated as shown
below:
Point
Value
Proportional
Value
75
75/410 = .183
55
55/410 = .134
140
140/410 = .341
140
140/410 = .341
Total
410
1.0
Construct
68
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
The following equation is then used to calculate each hospital's score for the
System:
^System
(3.3)
The scores for Leadership (1.0), the System (2.0, 3.0, 4.0, 5.0), O rganizational
Perform ance R esults (6.0), and Focus on and Satisfaction o f Patients and O th er
Stakeholders (7.0) are used to calculate the correlation coefficient that are used to test
Hypotheses 1 and 2.
69
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
The aggregate causal hypotheses given below address the directional influences
shown in Figure 3.2. At the end of each hypothesis statem ent, the structural equation
model path that is evaluated by the hypothesis is identified in brackets.
H3:
Leadership (1.0) has a positive influence on the System (2.0, 3.0,4.0, 5.0).
[Yu > 0 ]
H4:
H5:
The System (2.0, 3.0, 4.0, 5.0) has a positive influence on Focus on and
Satisfaction o f Patients and Other Stakeholders (7.0). [B3. > 0]
70
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
The studies cited above measure and evaluate the constructs of leadership, system
management, and firm performance using a variety o f methods and provide evidence that
leadership and system m anagem ent are influential in determ ining firm perform ance. In
particular, W ilson (1997) finds that the three relationships addressed by the aggregate
causal hypotheses presented above are supported in the manufacturing environm ent. His
results show that Leadership (1.0) has a directional influence on the System (2.0, 3.0, 4.0,
5.0). He also finds that the System has a positive influence on Business R esults, the
category in the original Baldrige criteria that is analogous to Organizational Perform ance
Results (6.0) in the H ealth Care Award criteria. W ilson also finds that the System has a
significant positive influence on Customer Satisfaction, the category that is analogous to
Focus on and Satisfaction of Patients and O ther Stakeholders (7.0) in the H ealth C are
Award criteria.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
H6:
H7:
H8:
Leadership (1.0) has a positive influence on Human Resource D evelopm ent and
[721
> 0]
H9:
Leadership (1.0) has a positive influence on Process Management (5.0). [y41 > 0]
H 10:
H P:
H 13:
Hu:
Information and Analysis (2.0) has a positive influence on Process M anagem ent
(5.0). [B41 > 0]
H 15:
H 16:
Inform ation and Analysis (2.0) has a positive influence on Focus on and
Satisfaction of Patients and Other Stakeholders (7.0). [B61 > 01
72
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
H i7:
Hu:
H 19:
H20:
H21:
H22:
H23:
These specific causal hypotheses are evaluated using a structural equation model.
The model used to test these hypotheses is shown in Figure 3.5. The data used to test
these hypotheses are obtained from the questionnaire described in Section 4.2. Principal
components analysis is performed, as described above in Section 3.2 to obtain scores for
each o f the 28 B aldrige dimensions (see Table 3.1). The structural m odel show n in
Figure 3.5 is estim ated using these dim ension scores. This m ethodology is described in
further detail in C hapter 4.
73
Reproduced with permission o f the copyright owner. Further reproduction prohibited without permission.
Previous studies provide evidence that many of the causal paths hypothesized
here are statistically significant in the manufacturing environment. Results from Wilson
(1997) and Handfield and Ghosh (1995) are shown in Table 3.2. The paths tested in the
two cited studies are listed in the left colum n o f the table. Each row represents one
hypothesis and the direction of causation is indicated by the colum n headings, "From"
and "To".
Som e of the Baldrige category nam es are abbreviated, and the category
T able 3.2 documents that m any o f the paths tested here in health care
3.5 Summary
Three sets of research hypotheses are used to test the relationships theorized by
the MBNQA in Health Care model (NIST, 1995b). The first set o f hypotheses evaluates
74
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
correlation relationships between the m ain entities of the B aldrige m odel using two
hypotheses. T he second set of hypotheses tests aggregate causal relationships among the
Baldrige model main entities. These three hypotheses evaluate the relationships among
Leadership (1.0), the System (2.0, 3.0, 4.0, 5.0) and Results (6.0, 7.0). T he third set of
research hypotheses tests specific causal relationships among the seven Baldrige criteria
categories. This third set includes 18 hypotheses. Correlation analysis is used to test the
first set of hypotheses. Structural equation modeling is used to test the second and third
sets of hypotheses.
75
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Cateeories/Items
Point
Value
Proportion
W eight
1,0
1.1
1.2
1.3
Leadership
Senior Executive and Health Care Staff Leadership
Leadership System and Organization
Public Responsibility and Citizenship
90
45
25
20
.50
.28
.22
2,0
2.1
2.2
2.3
75
20
15
40
.27
.20
.53
3,0
Strategic Planning
Strategy Development
Strategy Deployment
55
35
20
.64
.36
140
20
45
.14
.32
4.4
50
25
.36
.18
5.0
5.1
5.2
5.3
5.4
5.5
5.6
Process M anagement
Design and Introduction o f Patient Health Care Services
Delivery of Patient H ealth Care
Patient Care Support Services Design and Delivery
Community Health Services Design and Delivery
Administrative and Business Operations Management
Supplier Performance M anagement
140
35
35
20
15
20
15
.25
.25
.14
.11
.14
.11
6,0
6.1
6.2
6.3
6.4
6.5
250
80
40
30
90
10
.32
.16
.12
.36
.04
7,0
.12
.12
.12
.40
.24
3.1
3.2
4.0
4.1
4.2
4.3
7.1
7.2
7.3
7.4
7.5
76
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Hypothesis tested
Handfield and
G h osh (1995)
Tested in
this study?
From:
To:
Leadership (1.0)
Information (2.0)
.728
.550
yes
Leadership (1.0)
Strategy (3.0)
.295
.778
yes
Leadership (1.0)
.379
.363
yes
Leadership (1.0)
P ro cess(5.0)
.229
.383
yes
Leadership (1.0)
Performance (6.0)
NS
yes
Leadership (1.0)
Custom er (7.0)
NS
yes
Information (2.0)
Strategy (3.0)
.556
yes
Information (2.0)
.187
.259
yes
Information (2.0)
P ro ce ss(5.0)
.160
.486
yes
Information (2.0)
Performance (6.0)
.245
yes
Information (2.0)
Customer (7.0)
.267
yes
Strategy (3.0)
Information (2.0)
.216
no
Strategy (3.0)
NS
no
Strategy (3.0)
Performance (6.0)
NS
Strategy (3.0)
Customer (7.0)
NS
.297
yes
Performance (6.0)
NS
.141
yes
Custom er (7.0)
NS
Process (5.0)
Process (5.0)
Performance (6.0)
.193
P rocess(5.0)
Custom er (7.0)
.455
Performance (6.0)
Custom er (7.0)
Wilson (1997)
yes
yes
.231
no
yes
.502
yes
NS
yes
Notes: Paths with weighted values are significant at p < .01 or p < .05.
NS denotes paths that are tested but not significant.
Blanks denote paths that are untested.
Table 3.2: Literature Support for Tested Paths
77
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
-s:
,= c
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
System
Process
Management
5.0
Human Resource
Developm ent and
Management
Leadership
1.0
VO
h3
7.0
4.0
Strategic
Planning
3.0
Organizational
Performance Results
6.0
Information and
Analysis
2.0
Note: Each path is labeled with the hypothesis that tests its signficance.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Strategic
Planning
Information
& Analysis
Human
Resource
Process
Management
Customer
Satisfaction
Leadership
System
Performance
Results
I
Figure 3.3: Structural Equation Model to Test Aggregate Causal Hypotheses
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Process
Management
5.0
Leadership
Human Resource
Development and
Management
4.0
Strategic
Planning
3.0
Informationa and
Analysis
2.0
Organizational
Performance Results
6.0
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Process
Management
Human Resource
Development and
Management
63
^ Focus on and
Satisfaction of Patients
and Other Stakeholders
Leadership
Strategic
Planning
Organizational
Performance Results
Information and
Analysis
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Process
Management
5.0
Leadership
Human Resource
Development and
Management
Customer Focus
and Satisfaction
4.0
Strategic
Planning
3.0
Information and
Analysis
2.0
Note: + denotes significant paths
Business Results
6.0
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Process
Management
5.0
I
Human Resource
Development and
Management
Leadership
Customer Focus
and Satisfaction
7.0
4.0
Strategic
Planning
3.0
oo
Information and
Analysis
Business Results
CHAPTER 4
M ETHODOLOGY
In this chapter, the research m ethodology used to test the perform ance
relationships in the pilot criteria of the M alcolm Baldrige N ational Q uality Award
(MBNQA) in Health Care is presented. The empirical methodology em ployed uses data
gathered from U.S. hospitals. D evelopm ent o f the questionnaire used in this study is
described in detail in this chapter. The results of a pilot test are discussed. Sample
selection for the main study is described, and the procedures used to co llect data for the
main study are presented.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
em pirical test, each of the seven categories of quality management included in the award
criteria is measured in a large sample o f hospitals. The seven categories o f the Baldrige
A w ard criteria are:
P lanning;
1.0 Leadership;
3.0 Strategic
4.0 H um an R eso u rce D evelopm ent and M anagem ent; 5.0 Process
M anagem ent;
Satisfaction of Patients and O ther Stakeholders. The seven categories are further broken
down into 28 dimensions, as shown in Table 1.3. The Baldrige criteria are designed to be
applicable to all health care organizations, including hospitals, clinics, nursing homes,
and health maintenance organizations (HMOs) (NIST, 1995b).
G eneral acute care h o sp ita ls in the U.S., also co m m o n ly referred to as
com m unity hospitals, com pose the population studied here.
defined as those that provide a wide variety of medical services (e.g., maternity services,
surgery, cancer treatments). They contrast with specialty hospitals (e.g., psychiatric and
rehabilitation hospitals) which provide limited types of services. Acute care hospitals are
defined as those which provide short-term patient care.
average patient length of stay o f less than 30 days, as defined by the American Hospital
Association (1997). This contrasts with long term care that is provided by nursing homes
and sim ilar types o f facilities with average patient length o f stay of 30 days or more.
Therefore, the hospital population studied here includes hospitals that provide a wide
variety of medical services and that serve patients who are hospitalized an average of less
than 30 days.
This study is conducted at the facility level so that each hospital (i.e., each
facility) is counted separately in the sample. Many hospitals are m em bers of health
system s, m anaged care system s, etc. that affiliate them w ith o th e r health care
organizations. Regardless o f the num ber of other organizations w ith which they are
86
Reproduced with permission o f the copyright owner. Further reproduction prohibited without permission.
First, there are more than 5000 general acute care hospitals in the U .S.,
providing an ample population from which to draw a random sample (AHA, 1997).
Second, hospitals account for over 40 percent of em ploym ent and 50 percent of spending
in the U.S. health care industry (Standard and Poor, 1995). These percentages are higher
than those of any other type of health care organization (e.g., clinics, nursing homes).
A third reason for using general acute care hospitals in this study is that these
hospitals provide a w ide variety o f medical services and are very complex organizations
(see Section 1.2.1). The Baldrige criteria must account for this complexity and the broad
range of issues that these hospitals face. Additionally, general acute care hospitals range
in size from 2 beds to more than 1000 beds (AHA, 1997). This size range creates a
variety of complex environm ents in which to evaluate the Baldrige criteria.
Finally, hospitals have been at the forefront o f publicly debated issues regarding
health care. Costs, patient relations, and quality o f care are a few of the issues that are
highly visible to both health care providers and the public.
hospitals as an exam ple of all that is right and wrong with health care in the U.S. m akes
them a good target for study. General acute care hospitals are referred to as h o sp ita ls
throughout this document.
The study presented here provides a rigorous evaluation o f the constructs o f the
Baldrige Health Care Award criteria. It is also the first test o f the relationships implied in
the Baldrige model in a large sample o f U.S. hospitals. The data used here to study the
Baldrige criteria are obtained from a questionnaire that is mailed to hospitals (see
Appendix A for the final version of the questionnaire). The questionnaire is designed to
87
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission o f the copyright owner. Further reproduction prohibited without permission.
grounded in the 1995 pilot criteria o f the M BNQA in Health C are, and none has
evaluated the 28 dimensions of the Health Care Award criteria.
The pilot criteria of the MBNQA in Health Care were designed by quality and
health care experts and were closely modeled after the original M BNQA criteria. The
original criteria were designed, and have since been updated, by hundreds o f quality
experts (NIST, 1995a). The Health Care Award criteria are proposed to be appropriate
for all types o f health care organizations, and thus, the critical com ponents o f quality
m anagem ent for health care organizations are broadly defined and described in the
criteria. Developing a questionnaire for this study requires interpreting the criteria as
they apply to hospitals.
A scale is developed to operationalize the measurement of each of the 28 Baldrige
Health Care criteria dimensions. Each scale is developed based on a thorough review and
understanding of the Baldrige criteria (NIST, 1995b). The number of items in each scale
was determined by ensuring that the content o f each Baldrige dim ension is adequately
addressed. The num ber of items included in each scale range from two item s to seven
items.
The developm ent of a scale for Dimension l .l Senior Executive and H ealth Care
Staff Leadership is used as an example here to illustrate the process used to develop each
o f the measurement scales. The description of this dimension in the published aw ard
criteria was thoroughly reviewed (NIST, 1995b p. 13). The award criteria define the
domain of D im ension 1.1 to include both Areas to Address" and N otes" (that are
published in the award criteria). A number o f items (questions) are developed to address
the critical com ponents o f Dimension 1.1. Initially, the six items show n in T able 4.1
w ere deemed to capture the domain o f D im ension 1.1. One item was subsequently
dropped from this scale based on the results o f the pilot test, as discussed in Section 4.31.
89
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Each of the six items included in the initial scale for D im ension 1.1 is grounded in
the criteria of the Baldrige Health Care Award. Table 4.2 shows the source o f each item,
quoting the precise wording from the aw ard criteria that is used to form ulate each
questionnaire item. This grounding ensures that the wording and concept o f each item is
directly traceable to the award criteria. This sam e procedure is used to develop the other
27 scales included in the questionnaire used in this study. The source o f each item
included in the 28 measurement scales is given in Appendix B.
Each item included in the questionnaire is directly traceable to the Baldrige
Health Care criteria, with the exception o f the item s used in the scale that measures
D im ension 5.6 Supplier Perform ance M anagem ent.
developed by Flynn et al. (1994). Three items from Flynn et al.'s four-item scale are used
here.
These three items capture the content dom ain of D im ension 5.6.
No other
previously published studies provide scales that are adequate for this study, although
some scales are patterned after those used by W ilson (1997).
M ost studies that include the use o f a multi-scale survey designed to measure the
dim ensions of TQM have been conducted in m anufacturing environm ents.
With the
exception of Flynn et al.'s supplier m anagem ent questions, previously published scales
are either not transferable to the health care environm ent or do not adequately address the
content domain of the dimensions o f the B aldrige Health Care criteria.
Studies from
m anufacturing tend to focus on product developm ent, supplier involvem ent in product
developm ent, and product quality perform ance. These concepts d iffer significantly from
those addressed in hospitals.
A total of 120 item s, that m ake up the scales to m easure the 28 Baldrige
dim ensions, are included in the questionnaire used for the pilot test. The questionnaire
also contains questions regarding demographic and environmental inform ation.
90
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
The questionnaire is designed so that respondents indicate their opinion for each
item on a seven-point Likert-type scale. Most of the items use a Likert scale on w hich a
score of one (1) indicates the respondents hospital never engages in the indicated
activity, and a score o f seven (7) indicates that the respondents hospital always engages
in the indicated activity. This type o f scale is unipolar because one end o f the scale
represents the com plete absence of the item being evaluated while the other end o f the
scale represents a maximum value for the item.
Some of the scales that measure the dim ensions o f the results categories are
designed to use a Likert-type scale on which a score o f one (1) indicates the respondents
hospital performs significantly worse than its competitors on that particular item, a score
o f four (4) indicates the respondent's hospital performs the same as its competitors, and a
score of seven (7) indicates that the respondents hospital perform s significantly better
than its competitors. This type a scale is a bipolar scale with a defined m idpoint that
represents a neutral point between two opposing poles.
Seven-point Likert scales are used throughout the questionnaire because research
indicates that seven-point scales are most easily com pleted by respondents while also
providing the most reliable data.
frequent use of the 'uncertain midpoint for three- and five-point scales than for sevenpoint scales, indicating that shorter scales may be more difficult for respondents to
complete comfortably. Additionally, Matell and Jacoby (1972) reported that the time to
complete a set o f questions is essentially the same across scales with two through seven
points. However, com pletion time increases for scales w ith eight or more points. Thus,
there is no practical cost, m easured by the time it takes a respondent to com plete a
questionnaire, to using scales o f up to seven points, but there is a time cost to using
longer scales.
questionnaire used in this study contains 120 items used to evaluate the Baldrige criteria
91
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Both Lissitz and G reen (1975) and Jenkins and T ab er (1977) find that
m easurem ent reliability increases from two- through five-point scales, but is equivalent
for scales of seven or more points.
Each item included in the questionnaire was scrutinized by a form er Baldrige
exam iner to help ensure content validity o f the questionnaire, i.e., that the items capture
the content domain of the Baldrige Health Care criteria. Each item was also evaluated by
a researcher with extensive ex perience using surveys and scales for em pirical
m easurem ent for readability and precision in measuring a single concept. Finally, the
author o f this study, who worked as a hospital administrator for seven years, confirmed
that the content and wording of each item is appropriate for hospitals. These steps help to
establish content validity and ground the questionnaire in the 1995 pilot criteria of the
M BNQA in Health Care.
Ohio hospitals are included so that a portion of the sam ple is randomly
92
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
identified as the Director o f Quality, Quality Manager, or Vice President o f Quality were
obtained for each hospital in the pilot test sample (HCIA, 1995). A packet containing a
cover letter (shown in Appendix C), a questionnaire, and a business-reply envelope was
m ailed to one individual at each hospital.
following categories: for profit, non-profit, and governm ent ow ned. Data on hospital
size and ownership are obtained for each of the 156 hospitals in the pilot test sample from
a published source (AHA, 1997; HCIA, 1995).
To analyze non-respondent bias by size, each hospital is assigned to one of three
size categories. Data on num ber o f hospital beds are obtained from the AHA (1997).
The first category includes hospitals with fewer than 100 beds, the second category
consists of hospitals with betw een 100 and 350 beds, and the third category includes
hospitals with more than 350 beds. The number of hospitals in each o f these categories,
as well as the actual and expected number of responses in each category, are shown in
T able 4.3. The expect response rate for each hospital size category is calculated by
m ultiplying the proportion o f the sam ple in each size category by the total num ber of
respondents (n = 51). For exam ple, the proportion of hospitals with fewer than 100 beds
in the sample is 0.32 (50 o f 156 hospitals).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
A nalysis o f non-respondent bias by hospital size results in a significant C hisquare test statistic (C hi-square = 10.11, df = 2, p = .006). T he significant C hi-square
statistic indicates that the actual response rate by size category is significantly different
from the expected response rate. Therefore, hospital size and the probability o f response
are not statistically independent. A visual review o f the actual and expected num ber o f
responses in each category, presented in Table 4.3, show s that there is a higher than
expected response rate from small hospitals (few er than 100 beds). This high response
rate o f small hospitals is addressed in the selection o f the main study sample, as described
in Section 4.4.1.
N on-respondent bias is also analyzed by hospital ownership. Data on hospital
ow nership are obtained from HCIA (1995). The num ber o f sample hospitals defined as
for profit, non-profit, and government owned is show n in Table 4.4. The actual and
expected num ber o f responses in each of the ow nership categories is also given in Table
4.4. The C hi-square test statistic for non-respondent bias by hospital ownership is not
significant (C hi-square = 3.52, d f = 2, p = .17). This result indicates that the actual
response rate in each ow nership category is not significantly different than expected.
Reliability analysis of the pilot test data is presented in Section 4.3.1. D iscussion
o f questionnaire validity is presented in Section 4.3.2. Based on preliminary analysis o f
the pilot test data, several things are learned. Frist, a double mailing to each sam ple
hospital resulted in an adequate response rate (33 percent).
respond at a higher than expected rate, that is, hospitals with fewer than 100 beds have a
response rate o f 50 percent (see Table 4.3). Finally, there is no bias in response rate by
hospital ownership (see Table 4.4).
94
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
questionnaire scores are highly related to (correlated with) true scores (Suen, 1990). A
true score (the mean of unbiased estimates of the construct being measured under all
possible conditions) is equal to the observed score plus m easurem ent error (Suen, 1990).
Thus, reliable scales can be consistently measured and are close approximations of true
scores.
Several methods can be used for estimating the reliability o f measurement scales.
The four most common m ethods are estimating reliability from: (1) the coefficient of
correlation between scores on repetitions of the same test (com m only called the test-retest
method); (2) the coefficient o f correlation between scores on parallel forms of a test; (3)
the coefficient of correlation between scores on com parable halves o f a test (split-half
reliability); and (4) the intercorrelations among the com ponents o f a test (Ghiselli et al.,
1981). The fourth method is by far the most commonly used method in the management
literature, where reliability is estimated by measuring intercorrelations using Cronbach's
coefficient alpha.
Cronbachs coefficient alpha is a lower-bound (conservative estimate) o f the
internal consistency o f m easurem ent of the items in a scale (Suen, 1990). Alpha ranges
from 0 to 1.00, where 0 indicates no correlation among scale item measures and 1.00
indicates perfect correlation among scale item m easures.
average variance in scale item measures relative to the variance in a scales true score.
For example, an alpha value o f .80 indicates that 80 percent o f the variance in scale item
95
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
measures represent the true score and 20 percent (1 - .80) of the variance is measurement
error.
Cronbachs alpha is calculated for each of the 28 scales in the questionnaire used
in the pilot test. The alpha value for each scale is show n in Table 4.5, as well as the
num ber of items included in each scale. Alpha values range from .59 to .96. Nunnally
(1978) proposes that new scales should have an alpha value o f at least .60 and that
existing scales should have an alpha value of at least .70 to be considered adequately
reliable. New scales are those that have not previously been empirically tested. Existing
scales are those which have been tested and found to be reliable. An alpha level of .60
indicates that 40 percent (1 - .60) or less of the variance in the scale item measures is
error. Flynn et al. (1990) suggest adopting N unnally's criteria for scale reliability for
em pirical research in the Operations M anagement field. The scales tested here are all
new scales. (Only three of Flynn et al.'s (1994) four-item supplier management scale are
used, so the scale is evaluated as a new scale.)
The number of responses used to calculate Cronbach's coefficient alpha for each
scale is shown in Table 4.5. Only questionnaires that include responses for every item in
a particular scale are used to calculate the alpha value.
The scales are further evaluated to determ ine if any items can be dropped (to
shorten the questionnaire). SPSS 8.0 software provides users with the resulting alpha
value if any of the items in a scale is dropped.
Executive and Health C are Staff Leadership consists o f six items in the pilot test
questionnaire.
dropping Item D from the scale improves the alpha value to .8148. A thorough review of
the Baldrige description of Dimension 1.1 and the rem aining items ensures that dropping
this item does not compromise the validity of this scale.
96
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
One item is dropped from each o f six scales (Dim ensions 1.1, 1.2, 4.3, 6.4, 7.1,
and 7.4) to achieve the alpha values shown in Table 4.5. Further im provem ents in alpha
values are possible for som e scales by dropping additional items, but because each scale
is developed to measure the conceptual domain o f a M BNQA in Health C are dimension,
no items are dropped that compromise the content validity of the scales. Scale validity is
discussed in Section 4.3.2.
Tw enty-seven o f the scales included in the pilot test questionnaire are reliable
using Nunnally's (1978) and Flynn et al.'s (1990) criterion of an alpha value o f .60 or
greater. The pilot test scale for Dimension 1.3 Public Responsibility and C itizenship
does not meet the criterion for new scales. This problem is addressed by adding one
more item to the scale for Dimension 1.3 in the questionnaire used in the main study.
Reliability is further assessed by evaluating the dimensionality o f each scale. The
goal is to develop scales that each m easure a single concept or dom ain, that o f a
particular Baldrige dim ension. Principal com ponents analysis is first perform ed on each
scale.
components that can be calculated for a particular scale is equal to the num ber of items in
the scale. The com ponents for a scale account for decreasing proportions o f the variance
in that scales items. The first com ponent accounts for the highest portion o f the scale
item variance, and subsequent components account for the remaining variance.
Carm ines and Z eller (1979) suggest the following criteria for establishing scale
unidim ensionality:
(1) The first com ponent in each scale should ex p lain a large
proportion of the variance in the items (at least 40 percent); (2) Subsequent com ponents
should explain fairly equal proportions o f the rem aining variance; (3) M ost o f the items
should have large loadings on the first com ponent (at least .30); (4) M ost o f the items
should have larger loadings on the first component than on subsequent com ponents.
97
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Principal com ponents analysis is perform ed to reduce each scale to a single score
(based on the first com ponent for each scale).
component of each o f the 28 scales evaluated here varies from 49 to 96 percent, meeting
Carmines and Zeller's first criterion. The variance explained by the first com ponent for
each scale is show n in Table 4.7.
The scree plot and additional variance explained by subsequent com ponents are
also evaluated for each scale. The scree plot show s the eigenvalue for each of the
com ponents obtained from principal com ponents analysis of a scale.
An eigenvalue
represents the am ount of variance accounted for by a component (Hair et al., 1995). For
a unidim ensional scale, the scree plot is expected to show a large drop in the magnitude
of the eigenvalues from the first eigenvalue (first plotted point) to the second eigenvalue.
Points subsequent to the first point should be relatively horizontal to one another with
only a small drop in magnitude among each o f them.
Figure 4.1 shows the scree plot for D im ension 1.1 Senior Executive and Health
Care Staff Leadership. The evaluation o f this scree plot is provided as an exam ple here,
and the sam e process is used to provide evidence o f unidim ensionality in the other 27
scales used in this study. As shown in Figure 4.1, the first com ponent explains a large
proportion of scale variance as indicated by the large first eigenvalue.
The second
through fifth eigenvalues are much sm aller and fairly equivalent in m agnitude. Thus,
com ponents subsequent to the first one have low explanatory power.
Each of the 28
scales meets Carm ines and Zeller's second criterion o f subsequent com ponents explaining
fairly equal proportions of the variance rem aining after the first com ponent is removed.
The loadings of each of the item s on each o f the scales used in this study are
reviewed to evaluate Carmines and Zeller's third and fourth criteria. The loading o f each
item should be at least 0.30 on the first com ponent in principal com ponents analysis, and
the first co m p o n en t loading should be g reater than the loadings on subsequent
98
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
com ponents. For example. Table 4.8 shows the com ponent loadings for each item used
in the scale for Dimension 1.1 Senior Executive and Health Care Staff Leadership. The
loadings are all greater than .30 on the first com ponent, ranging from .705 to .881,
satisfying Carmines and Zeller's third criterion. Additionally, all loadings on subsequent
com ponents are less than those on the first component, upholding Carmines and Zeller's
fourth criterion.
Based on the reliability analysis performed on the pilot test data, the following is
learned: (1) The reliability o f six scales is improved by dropping one item from each
scale (see Table 4.5). (2) Tw enty-seven of the 28 scales developed for this study have
adequate reliability (see T able 4.5).
D imension 1.3 is addressed by adding one item to that scale. (3) U nidim ensionality o f
each measurement scale, based on Carm ines and Zeller's criteria is verified (see Tables
4 .6 ,4 .7 and 4.8).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
num ber o f items (questions) to adequately m easure a hospitals perform ance on that
dimension. The content validity of the questionnaire is supported by this grounding on
the Baldrige criteria. The source of each item included in the questionnaire is given in
Appendix B, showing that each item in the questionnaire can be directly traced to the
Baldrige Health Care criteria (NIST, 1995b).
Content validity o f the questionnaire is further demonstrated by having a number
of experts thoroughly review its content. These experts include: David A. Collier, Ph.D.,
former Baldrige Award examiner; Ted. W. G race, M.D., Director, Wilce Student Health
C enter, T he O hio State U niversity;
A dm inistration, W ilce Student H ealth C enter, The Ohio State U niversity, w hose
background includes ten years as a hospital vice president of quality; and Gail B. Marsh,
Administrator for quality in operations. The Ohio State University Medical Center.
Criterion-related validity is the ability of a set o f measures to correspond to, relate
to, or predict scores on another variable called a criterion. In this study, the predictors
include five o f the constructs in the Baldrige Health Care criteria including C ategory 1.0
Leadership and the four System categories: Inform ation and Analysis (2.0); Strategic
Planning (3.0); Human Resource D evelopm ent and M anagem ent (4.0); and Process
M anagement (5.0). Criterion-related validity is evaluated using the results categories as
the predicted criteria. The results categories are O rganizational Perform ance Results
(6.0) and Focus on and Satisfaction of Patients and O ther Stakeholders (7.0).
To provide evidence o f criterion-related validity, multiple regression is used to
predict each of the results categories using the Leadership and System categories as the
independent variables. Scores for each of the Baldrige categories are obtained using the
methodology described in detail in Sections 5.1.1 and 5.2.1. Baldrige Categories 1.0, 2.0,
3.0, 4.0, and 5.0 are used to predict Baldrige Category 6.0 Organizational Perform ance
Results, resulting in R2 = 0.34 (p < .01). This m eans that the Baldrige Leadership and
100
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
System categories account for 34 percent of the variance in the criterion, Baldrige
C ategory 6.0. This result provides evidence of criterion-related validity because the
predictors (Categories 1.0, 2.0, 3.0, 4.0, and 5.0) explain a significant portion o f the
variance in the criterion (Category 6.0).
A similar multiple regression is performed using Baldrige C ategories 1.0, 2.0, 3.0,
4.0, and 5.0 as independent variables to predict Category 7.0 Focus on and Satisfaction of
Patients and Other Stakeholders. This regression analysis shows that R 2 = 0.50 (p < .01),
m eaning 50 percent o f the variance in the criterion, Baldrige C ategory 7.0 is accounted
for by the independent variables. Again, this result provides evidence o f criterion-related
validity because a significant portion o f the variance in the criterion (C ategory 7.0) is
explained by the predictors (Categories 1.0, 2.0, 3.0,4.0, and 5.0).
Construct validity of a m easurem ent scale is developed w hen that scale is deemed
an appropriate operational definition o f an abstract variable o r construct that cannot be
m easured directly. Flynn et al. (1990) use job satisfaction as an exam ple o f a construct
which cannot be directly assessed, but can be inferred from scores on sum m ated scales
w hich measure job satisfaction through evaluations o f w orker turnover, and other jobrelated phenomena. Scale unidim ensionality (discussed in Section 4.3.1) is useful for
supporting construct validity because it insures that a single construct or dim ension is
measured by each scale (Flynn et al., 1990).
The dimensions o f quality m anagem ent addressed in the B aldrige health care
criteria cannot be measured directly. Therefore, a scale has been developed to measure
each of the 28 dimensions. For exam ple. Dimension 1.1 Senior Executive and Health
C are Staff Leadership cannot be m easured directly, so a scale o f five item s is developed
to provide a measure o f the conceptual domain of this dim ension. This sam e process of
scale development is used for each o f the 28 Baldrige dim ensions. Only with repeated
use and evaluation can scales be deem ed to have construct validity (Suen, 1990). The
101
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
study presented here provides the initial steps toward building a valid m easurement tool
for the Baldrige Health Care model.
described the selection of the main study sample and the collection and preparation o f
data in the main study. T he analysis of the main study data is described in detail in
C hapter 5.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
individuals who are identified by name, it is necessary for the appropriate individual to be
identified from published sources.
The second reason that hospitals with few er than 60 beds are not included in the
study sam ple is that these hospitals, some w ith as few as five beds, likely have quality
m anagem ent system s that are much less com plex than those o f larger hospitals. A small
hospital's ability to organize and evaluate its quality management system is unlikely to
match the requirem ents implied by the M BNQA in Health Care criteria. For exam ple,
the information system capabilities described in the Baldrige Award criteria, although not
specific, w ould likely require a large investm ent in hardware and softw are.
Sm all
The
remaining 814 hospitals were used for the main study sample.
T able 4.9 shows the num ber of hospitals in both the study sam ple and the
population by the bed size categories used by the American Hospital A ssociation (AHA,
1997). The percentage of sample and population hospitals in each category is also given.
103
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
These percentages verify that the proportion o f hospitals in each size category in the
study sample is similar to that found in the population.
The questionnaire described in Section 4.2 was edited with the changes described
in Section 4.3.1.
Specifically, six items are dropped and one item is added to the
demographic and environmental questions. The questionnaire used in the main study is
shown in Appendix A. Table 4.10 defines each o f the items included in the questionnaire
as it corresponds to a Baldrige dimension scale. The scale sections and identifying letters
shown in Table 4.10 correspond to the sections and letters used in the questionnaire
shown in Appendix A.
The questionnaire w as m ailed to an identified D irector o f Q uality, Q uality
M anager, or Vice President o f Quality at each o f the main study hospitals. Included in
the mailed packet were a cover letter describing the study (shown in A ppendix C ), the
questionnaire (shown in Appendix A), and a business reply envelope in which to return
the completed questionnaire. Six weeks after the initial mailing, a second questionnaire
was sent to all non-responding hospitals. Two hundred twenty-eight hospitals com pleted
and returned the questionnaire for a response rate o f 28 percent.
Non-respondent bias is analyzed by hospital size, as measured by number o f beds.
Table 4.11 shows the actual and expected number o f responses in each of the A H A size
categories. The Chi-square test statistic for non-respondent bias by hospital size is not
significant (Chi-square = 3.41, d f = 5, p = .64). This result indicates that the actual
response rate is about the same across size categories.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
portion o f self-reported perform ance data and are not included in further analysis. The
rem aining 220 responses are used in the analysis in this study. Som e o f the analyzed 220
responses are also m issing a m inim al num ber of data points, and a description o f the
m ethodology used to com pensate for these missing data points is provided in Section
4.4.3 below.
A statistical comparison is made between edited set of 220 responses that are used
in the analysis in this study and the eight responses that are rem oved from the data set.
This com parison ensures that no bias is introduced into the study data set by removing
the eight responses from analysis. The Levene test of hom ogeneity-of-variance shows no
significant differences in the variances o f number of beds or occupancy rate for the two
sam ples (H air et al, 1995). T -tests show that there is no significant difference between
the two data sets for the average num ber o f beds per hospital or average occupancy rate.
The results of these analyses are show n in Table 4.12. The results indicate that removing
the eight incomplete questionnaires from the study data set likely did not bias the data set.
The sam ple size for each data set in Table 4.12 differs from the total sample sizes o f 220
for the edited data set and
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
In sum m ary, the following guidelines are used for questionnaires with missing
data points:
1.
Scale missing one data point -> average item score inserted
2.
3.
a.
b.
Scale with more than five items -> average item score inserted
An exam ple is provided here to illustrate the negligible effect on the main study
data set o f replacing m issing data points with scale m eans. For Dimension l .l Senior
Executive and H ealth Care Staff Leadership, replacing m issing data points with means
increased the num ber o f usable questionnaires from 219 to 220 (only one questionnaire
contains a missing responses in Dimension 1.1 items). To analyze the difference between
the data sets containing 219 and 220 responses, principal com ponent scores are calculated
for both data sets. The variance explained by the first principal component in each data
set is 55.627 percent (n = 219) and 55.641 percent (n = 220), a negligible difference.
The benefit o f replacing missing data points is that the effective num ber of
responses is higher than when eliminating all questionnaires with missing data points.
The methodology used here for replacing missing data points is used by Wilson (1997).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
in Appendix D, and the items included in the scale used to measure each dim ension are
given.
For each item , the percent of responses at each point on the Likert scale is
reported as well as the percent o f questionnaires w ith no response. The item loadings
reported in Appendix D are discussed in Section 4.51.
Responsibility and Citizenship had a Cronbach's coefficient alpha value o f .57 in the pilot
test. An alpha value o f .60 is generally considered acceptable for new scales (Nunnally,
1978; Flynn et al., 1990). The reliability of each o f the 28 scales used in this study is re
evaluated based on the
220
Based on the m ain sam ple data set, C ro n b a c h 's alpha values for the 28
measurement scales range from .74 to .93, as show n in Table 4.13. These values all
indicate adequate reliability (Nunnally, 1978; Flynn et al., 1990). Also reported in Table
4.13 are the number o f items in each scale and the num ber of responses (cases) used to
calculate each alpha value. Only questionnaires that included responses for every item
for a particular scale are used to calculate the scales alpha value.
Although dropping items would improve some alpha values, no items are dropped
to improve the alpha values given in Table 4.13, ensuring the content validity o f each
measurement scale. For example, dropping item e (shown in Appendix D) from the scale
for Dimension 1.1 w ould improve the alpha value from .79 to .81. However, any benefit
from a minimal im provem ent in this already adequately reliable scale is offset by the
need to maintain the content validity of the scale.
107
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
D im ension 1.3 Public Responsibility and Citizenship was not reliable based on
the pilot test data, as described in Section 4.3.1. One item was added to this scale in the
questionnaire sent to the main sample hospitals. The scale is found to be adequately
reliable in the main study sample, with a Cronbach's alpha value of .76.
Scale unidim ensionality, which was tested and confirmed for each scale in the
pilot test, is re-evaluated in the main sample data set by examining C arm ines and Zeller's
(1979) criteria for establishing scale unidim ensionality (discussed in detail in Section
4.3.1). The percentage of variance explained by the first principle com ponent o f each
measurement scale is given in Table 4.14, addressing Carmines and Zeller's first criterion
that the first com ponent of each scale explain more than 40 percent of variance in the
items. The loadings of each item on the first principal component of its respective scale
are given in the far right column of each table in Appendix D. These loadings document
that the scales m eet Carmines and Zeller's third criterion of item loadings greater than
.30. The second and fourth criteria (a large eigenvalue for the first principal com ponent
and sm all, fairly equal eigenvalues for subsequent principal com ponents) are also
evaluated and upheld in the main study data set.
108
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
governm ent owned (n = 215). One hundred nineteen hospitals (56 percent) report that
they operate independently, while 93 (44 percent) are affiliated with or ow ned by a multi
hospital system (n = 2 1 2 ). Finally, the follow ing information is reported for teaching
activities: 23 hospitals (11 percent) engage in m ajor teaching activities;
124 hospitals
(60 percent) participate in m inor teaching activities such as resident training or other
allied health professional training; and 60 hospitals (29 percent) have no involvem ent in
teaching activities (n = 207).
The descriptive organizational data reported here show the variety o f hospitals in
the sam ple data set. The variation in the sam ple data set is sim ilar to that found in the
population of general acute care hospitals.
The ability of each o f the organizational variables described above to predict
hospital performance is tested. The effect o f organizational variables on perform ance is
evaluated to test w hether certain types o f hospitals (e.g., for profit, non-teaching, etc.)
have b etter average perform ance than the other hospitals studied here.
Baldrige
C ategories 6.0 Organizational Performance Results and 7.0 Focus on and Satisfaction of
Patients and Other Stakeholders are used as the measures of perform ance in this analysis,
based on data obtained from the study questionnaire. The methodology used to calculate
these measures for each hospital is described in detail in Sections 5.1.1 and 5 .2 .1.
The organizational variables o f num ber of beds, hospital ow n ersh ip , system
membership, and teaching status are used to predict Baldrige Categories 6.0 and 7.0 using
regression analyses. All data are self-reported by the responding hospitals. N um ber of
beds is measured as a continuous variable. Hospital ownership is a categorical variable
w ith the following classifications: (1) proprietary, (2) non-profit, and (3) governm ent.
H ospital system m em bership is defined by two categories: (1) independent, and (2)
affiliated with or owned by a m ulti-hospital system . Teaching status is a categorical
109
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
variable: (1) m ajor, (2) limited (residents only, or allied health), and (3) none. All
categorical variables are dummy coded for this analysis.
The results, which are shown in Table 4.15, show that performance o f the sample
hospitals is not substantively explained by any o f these organizational variables, although
two o f the tested relationships are statistically significant.
N um ber o f beds is a
W ith num ber o f beds as the independent variable and C ategory 6.0 as the
dependent variable, R 2 = .02 (p = .02), meaning that number of beds explains about two
percent o f perform ance in Category 6.0. Likew ise, hospital ownership explains about
three percent o f Category 7.0 (R 2 = .03, p = .03). Although these results are statistically
significant, the explanatory power of the independent variables is m inim al and not
remarkable enough for further consideration.
In general, the results of testing the predictive power of the organizational
variables studied here (and shown in Table 4.15) indicate that there are no substantive
differences in the performance of the types o f hospitals defined by these organizational
variables.
Baldrige H ealth Care dimensions. Each dim ension is measured by a set o f questions,
referred to as a scale. The questionnaire was pilot tested on a group o f hospitals and
110
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
found to be reliable using several methods o f analysis. The validity o f the questionnaire
is also thoroughly reviewed and supported.
The questionnaire used in this study was improved based on the results of the
pilot test and was mailed to the main study sample of 814 hospitals throughout the U.S.
Two hundred tw enty-eight questionnaires were completed and returned for a response
rate of 28 percent. There is no respondent bias by hospital size. The data obtained from
the questionnaire are prepared for analysis by excluding eight qu estio n n aires with
missing perform ance-related data (Section 4.4.2) and addressing m issing data points in
the questionnaires that remain in the analyzed data set (Section 4.4.3). Scale reliability is
meticulously evaluated in the main study data set for the 28 scales that m easure the 28
dimensions o f the M BNQA in Health Care. The reliability of the 28 scales is found to be
adequate (Nunnally, 1978; Flynn e ta l., 1990).
Ill
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Item
Survey questions
a.
b.
c.
d.
e.
f.
112
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Item
Criteria reference
a.
b.
c.
d.
e.
f.
Table 4.2: Sources for Pilot T est Dimension 1.1 Scale Items
113
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Sample
Actual
Responses
Expected
Responses
n< 1 0 0
50
25
16
100<n<350
56
13
18
n>350
50
13
16
Total
156
51
51
Number of beds
Pearson Chi-square
Value
df
10.11
.006
114
Reproduced with permission o f the copyright owner. Further reproduction prohibited without permission.
Ownership
Sample
Actual
Responses
Expected
Responses
For profit
41
13
Non-profit
97
34
32
Government
18
156
51
51
Total
Pearson Chi-square
Value
df
3.52
.17
115
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Number
Constructs / Dimensions______________ o f Cases
Number
o f Items
C ronbachs
Alpha
Leadership
l.l Senior Executive and Health Care Staff Leadership
1.2 Leadership System and Organization
1.3 Public Responsibility and Citizenship
51
51
51
5*
3
3
.81
.69
.56
49
50
50
5*
3
.88
Strategic Planning
3.1 Strategy Developm ent
3.2 Strategy D eploym ent
50
50
5
4
.93
51
51
50
4
4
4*
.84
.71
.8 8
51
.84
.94
4
3
3
4
3
.90
.89
.73
.82
.65
.96
.82
.72
2 .1
2 .2
Process M anagement
51
5.1 Design and Introduction of Patient Health Care
Services
51
5.2 Delivery o f Patient Health Care
5.3 Patient C are Support Services Design and Delivery 51
5.4 C om m unity Health Services Design and Delivery
49
5.5 A dm inistrative and Business Operations Management51
49
5.6 Supplier Performance Management
O rganization Performance Results
Patient Health Care Results
Patient C are Support Services Results
6.3 C om m unity Health Services Results
6.4 A dm inistrative, Business, and Supplier Results
6.5 A ccreditation and Assessment Results
6 .1
6 .2
7.1
7.2
7.3
7.4
7.5
44
48
50
47
49
2
2
7*
2
.90
.89
.88
.8 6
.77
.89
.79
.89
.75
.62
* one question was dropped from scale to achieve a higher value of C ronbachs alpha
Table 4.5: Pilot Test Scale Reliability
116
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Alpha if
Item Deleted
.7452
Item B
.7521
Item C
.7421
Item D
.8148
Item E
.7965
Item F
.7887
Initial scale
(6
.8061
.8148
1.1
R e p ro du ced with permission o f the copyright owner. Further reproduction prohibited without permission.
Variance 1
explained
Constructs / Dimensions
Leadership
Senior Executive and Health Care Staff Leadership
Leadership System and Organization
1.3 Public Responsibility and Citizenship
62%
64%
57%
69%
84%
66%
Strategic Planning
3.1 Strategy Development
3.2 Strategy Deployment
79%
74%
1.1
1 .2
2 .1
2 .2
4.1
4.2
4.3
4.4
69%
58%
73%
63%
5.1
5.2
5.3
5.4
5.5
5.6
Process Management
Design and Introduction of Patient Health Care Services
Delivery of Patient Health Care
Patient Care Support Services Design and Delivery
C om m unity Health Services Design and Delivery
Administrative and Business Operations Management
Supplier Performance Management
81%
78%
82%
66%
70%
69%
7.1
7.2
7.3
7.4
7.5
118
Reproduced with permission o f the copyright owner. Further reproduction prohibited without permission.
53%
74%
96%
49%
79%
71%
62%
72%
58%
59%
Component
1
Item A
.853
-.068
-.234
-.383
.258
Item B
.881
-.199
-.216
-.082
-.361
Item C
.749
.446
-.287
.390
.071
Item E
.723
-.542
.295
.288
.116
Item F
.705
.412
.556
-.145
-.056
Table 4.8: Loadings for Dimension 1.1 Senior Executive and Health Care Staff
Leadership
119
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Number
of beds
Sample
n (% )
6-24
278 (5)
25-49
922 (18)
(15)2
1139(22)
50-99
12 1
Population
n (%)
100-199
263 (32)
1324(26)
200-299
227 (30)
718(14)
300-399
8 3 (1 0 )
354 (7)
400-499
55 (7)
195 (4)
> 500
65 ( 8 )
264 (5)
1 Population:
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Constructs / Dimensions
Leadership
Senior Executive and Health Care Staff Leadership
Leadership System and Organization
1.3 Public Responsibility and Citizenship
1.1
1 .2
Strategic Planning
3.1 Strategy Development
3.2 Strategy Deployment
II. a-e
II. f-h
II. i-n
III. a-e
III. f-i
4.1
4.2
4.3
4.4
5.1
5.2
5.3
5.4
5.5
5.6
Process M anagement
Design and Introduction of Patient Health Care Services
Delivery of Patient Health Care
Patient Care Support Services Design and D elivery
Community Health Services Design and D elivery
Administrative and Business Operations M anagem ent
Supplier Performance Management
7.1
7.2
7.3
7.4
7.5
I. a-e
I. f-h
I. i-1
121
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
IV.
IV.
IV.
IV.
a-d
e-h
i-1
m-q
V. a-e
V. f-i
V .j-l
I. m-o
V. m-p
V. q-s
VI. a-g
VI. h-i
VI. j-k
VI. 1-r
VI. s-t
VII. a-e
vn. f-i
vn. j-n
vn. o-r
VI. u-w
Sample
Actual
Responses
Expected
Responses
ndO O
1 21
32
34
100-199
263
72
74
200-299
227
59
64
300-399
83
27
23
400-499
55
20
15
>500
65
18
18
Total
814
228
228
Number o f beds
Pearson Chi-square
Value
df
fi
3.41
.64
122
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Sample
size
Number
of beds
220
8
Occupancy
rate
220
8
Number of
responses
Mean
Standard
deviation
df
205
234
171
1.16
211
.25
163
138
180
60.6
16.4
.60
184
.55
64.7
9.6
123
Reproduced with permission o f the copyright owner. Further reproduction prohibited without permission.
Number
Constructs / D im ensions_______________ of Cases
N um ber C ronbachs
o f Item s
Alpha
Leadership
Senior Executive and Health C are Staff Leadership
Leadership System and O rganization
1.3 Public Responsibility and C itizenship
218
217
217
5
3
4
.81
.77
.76
218
219
215
5
3
6
.91
.84
.90
Strategic Planning
3.1 Strategy Development
3.2 Strategy Deployment
216
216
5
4
.93
.89
217
217
218
4
4
4
.8 6
218
.87
.91
1.1
1 .2
2.1
2 .2
Process Management
219
5.1 Design and Introduction o f Patient Health Care
Services
5.2 Delivery of Patient Health Care
218
5.3 Patient Care Support Services Design and Delivery 219
5.4 Com m unity Health Services D esign and Delivery
218
5.5 Administrative and Business O perations Management217
5.6 Supplier Performance M anagem ent
218
Organization Performance Results
6 . 1 Pauent Health Care Results
6 . 2 Padent Care Support Services Results
6.3 Com m unity Health Services Results
6.4 Administrauve, Business, and Supplier Results
6.5 Accreditadon and A ssessm ent Results
7.1
7.2
7.3
7.4
7.5
204
216
218
196
214
4
3
3
4
3
7
.80
.89
.8 6
.81
.74
.8 6
.82
.83
.77
.85
.78
.79
.83
.84
.89
.87
.82
2
2
124
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Variance 1
explained
Constructs / Dimensions
Leadership
Senior Executive and Health Care Staff Leadership
Leadership System and Organization
1.3 Public Responsibility and Citizenship
1.1
1 .2
56%
68%
59%
75%
76%
67%
Strategic Planning
3.1 Strategy Development
3.2 Strategy Deployment
78%
76%
4.1
4.2
4.3
4.4
70%
64%
75%
66%
5.1
5.2
5.3
5.4
5.5
5.6
Process Management
Design and Introduction o f Patient Health Care Services
Delivery of Patient Health Care
Patient Care Support Services Design and Delivery
Community Health Services Design and Delivery
Administrative and Business Operations Management
Supplier Performance M anagem ent
74%
71%
72%
66%
70%
74%
7.1
7.2
7.3
7.4
7.5
125
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
50%
82%
87%
46%
82%
71%
68%
70%
72%
74%
Simple regression
O rganizational factor
(independent variable)
Outcome factor
(dependent variable)
N um ber o f beds
Performance (6.0 ) 1
.0 2
194
.03
Customer (7.0 ) 2
.0 1
198
.1 2
Performance (6.0)
.0 2
.17
Customer (7.0)
.03
.03
Performance (6.0)
< .0 1
.54
Customer (7.0)
< .0 1
.87
Performance (6.0)
< .0 1
.75
Customer (7.0)
< .0 1
.99
Ownership
System/Independent
Teaching status
Note:
1
2
R2
df
126
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
p-value
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2.0
0)
3
(0
>
c
Q)
g>
in
Factor Number
Figure 4.1 Scree Plot of Eigenvalues for Dimension 1.1 Senior Executive and Health Care Staff Leadership
CHAPTER 5
RESULTS
In this chapter, the results o f testing the research hypotheses are presented and
discussed.
perform ance relationships and causal linkages proposed by the pilot criteria o f the
M alcolm Baldrige National Q uality Award in H ealth C are (N IST, 1995b). First the
results o f testing tw o hypotheses which measure the degree o f association betw een
Leadership (1.0), the System (2.0, 3.0, 4.0, 5.0), and Results (6.0, 7.0), are reported. The
process of developing m easures o f these Baldrige constructs, necessary for testing these
hypotheses, is described in detail.
Second, an aggregate causal model in which L eadership (1.0) has a positive
causal influence on the System (2.0, 3.0, 4.0, 5.0), and the System has a positive causal
influence on Results (6.0, 7.0), is tested. Structural equation modeling is used to test the
three aggregate model hypotheses, and the processes o f building the measurement model
and the structural m odel necessary for structural equation m odeling are described in
detail. W hile the First two hypotheses test the existence o f relationships o f association,
they do not address a direction of causation between the B aldrige constructs. The three
aggregate causal hypotheses test the direction o f causation and the basic theory that
Leadership drives the System which causes Results. Finally, eighteen hypotheses that
128
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
address specific causal paths between the seven main constructs in the Baldrige Health
Care model are tested using structural equation modeling.
component consists o f the dependent causal relationships that link the model constructs,
or in this case, link the Baldrige model categories.
All o f the hypotheses tested in this study require the developm ent o f the
measurement model for the Baldrige Health Care model. To ensure a valid test o f the
research hypotheses, the measurement model m ust accurately reflect the content and
intent of the M B N Q A in H ealth Care criteria.
described in Section 5.1.1. The results of testing the research hypotheses are reported in
Sections 5.2.2 (correlation hypotheses), 5.3.2 (aggregate causal hypotheses), and 5.4.2
(specific causal hypotheses).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
model for this study. The outcome of m easurem ent model developm ent is a score for
each of the 28 dimensions included in the Baldrige model (listed in Table 1.3) for each
hospital in the m ain study data set.
measurement model are used in further analysis to test the research hypotheses.
Figure 5.1 provides an exam ple o f the relationships betw een constructs,
dimensions, and item s that are discussed here. This portion o f the m odel includes the
Leadership (1.0) construct. Dimension 1.1 Senior Executive and H ealth C are Staff
Leadership and the five questionnaire item s that operationalize m easurem ent o f the
dimension, Dimension 1.2 Leadership System and Organization and the three items that
measure it, and Dimension 1.3 Public Responsibility and Citizenship and the four items
that measure it. Figure 5.1 provides an exam ple by showing the m easurem ent model
com ponents for one o f the seven Baldrige constructs.
components exist for each of the six rem aining constructs (2.0, 3.0, 4.0, 5.0, 6.0, 7.0).
The following entities are shown in Figure 5 .1:
Entity
Definition
Item
Dimension
Construct
Baldrige category
M easurement path
The data used for measurement model development are the item scores obtained
from the questionnaire described in Section 4.2. The data set is the main study data set
described in Section 4.5. Structural equation modeling allows the m easurem ent and
structural models to be estim ated sim ultaneously. However, for this study the large
130
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
num ber of variables (115) precludes this option. Therefore, as a first step, principal
com ponents analysis is performed on the item s influenced by each dim ension, and the
first principal com ponent score for each set o f items is used as the dim ension score.
An exam ple is provided here to illustrate the methodology used to build the
m easurem ent m odel.
Executive and Health Care Staff Leadership score is obtained using this process. The
sam e process is used to obtain each hospital's score for the remaining 27 dim ensions in
the Baldrige Health Care model.
Item scores are obtained from the questionnaire that was m ailed to hospital
quality managers. The questionnaire is shown in Appendix A, and all responses are on a
seven-point L ikert-type scale. Principal com ponents analysis is used to reduce each
hospital's item scores to a single score for each o f the Baldrige 28 dim ensions. Principal
components analysis is a data reduction technique which is useful when the objective is
to sum m arize m ost of the original inform ation, such as that contained in the five item
scores for D im ension 1.1, in a smaller num ber o f scores (Hair et al., 1995). In this case,
the first com ponent score for each dimension is obtained. For the exam ple used here, the
first component score for the five items that make up the scale for D im ension 1.1 Senior
Executive and Health Care Staff Leadership is obtained for each hospital in the study.
Figure 5.2 show s the process used to build the m easurem ent m odel.
The
measurement scale for Dimension 1.1 Senior Executive and Health C are Staff Leadership
includes five items. The items are the m easured variables (MV) in this m odel, and thus
are designated M V Xin Figure 5.2. The first principal component score for Dim ension l.l
is designated
The loading or weight o f each item on the first principal com ponent is
designated t x in the figure, labeling the paths from the dimension to each m easured
variable. A dditionally, the variance in each item that is not accounted for by the first
131
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
principal component is captured by the error term (ex) for each item. The path loading
CO for each item used in this study is given in Appendix D.
In the same m anner, principal component scores are obtained for the remaining 27
Baldrige dimensions. A m inimum of two items and a m axim um of seven items are used
to measure each of the 28 dim ensions of the Baldrige model.
H ,:
Hospital Leadership (1.0) and System managem ent (2.0, 3.0, 4.0, 5.0) are
positively correlated. [c()l2 > 0 ]
H 2:
Hospital System managem ent (2.0, 3.0, 4.0, 5.0) and performance (6.0, 7.0)
are positively correlated. [<j)26 > 0; <j)27 > 0J
132
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Leadership construct have proportional weights of 0.50 (45 o f the 90 points assigned to
the Leadership construct), 0.28 (25/90), and 0.22 (20/90), respectively (see Table 3.1 for
MBNQA points and weights). These weights are used to reduce the three dim ension
scores to a single Leadership construct score.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
X z y Ay
(3.1)
In the exam ple provided here, the construct score for Leadership (1.0) is
calculated using Equation 3.2:
Yl 0 =
(3.2)
The principal component scores for Dimensions 1.1, 1.2, and 1.3 are inputs that
allow the calculation of a Leadership (1.0) construct score for each hospital. Figure 5.3
illustrates this process. The Leadership construct (y10) is the weighted com posite of
D im ensions 1.1 (Au ), 1.2 (A12), and 1.3 (A13). Each of the paths is labeled in Figure 5.3
to reflect the relative weight assigned to that dimension by the Baldrige criteria.
sim ilar process is used to calculate the scores o f each of the six rem aining B aldrige
constructs for each hospital in the study.
134
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
calculate the total System score for each hospital in the main study data set:
^System
(3.3)
A detailed explanation of the logic used to generate Equation 3.3, as well as the
procedure used to com pute each category w eight in the equation, is provided in Section
3.2.
indicates the strength of association between the constructs being evaluated and can range
from -1 to +1, w here -1 indicates a perfect negative relationship (as one gets larger, the
other gets sm aller), 0 indicates no relationship, and +1 indicates a perfect positive
relationship (both get larger or smaller at the sam e rate) (Hair et al., 1995).
The correlation coefficient for the relationship between L ead ersh ip and the
System is r = 0.75 (p < .01). This coefficient indicates that the relationship between
Leadership and the System is positive and statistically significant, and hospitals w ith high
Leadership construct scores tend to have high System scores w hile hospitals w ith low
135
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
The 90 percent
confidence interval for this correlation coefficient is (.68, .80). Therefore, Hypothesis 1
is supported.
H [:
Hospital Leadership (1.0) and System management (2.0, 3.0, 4.0, 5.0) are
A scatter plot of the relationship between Leadership and the System is shown in
Figure 5.4. The variables in Figure 5.4 are standardized z-scores (mean 0, standard
deviation 1), and each point represents an individual hospital. The scatter plot shows a
strong, positive linear relationship between Leadership and the System.
To test Hypothesis 2, two relationships are analyzed. The first is between the
System and O rganizational Perform ance Results (6.0).
The 90
136
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
for the relationship between the System and Focus on and Satisfaction of Patients and
O ther Stakeholders is 0.69 (p < .01), and the 90 percent confidence interval is (.61, .75).
Again, this relationship is positive and statistically significant. Therefore, Hypothesis 2
is supported.
H2:
Hospital System management (2.0, 3.0, 4.0,5.0) and perform ance (6.0, 7.0)
are positively correlated. [<)):6 > 0; <j)27 > 0]
137
Reproduced with permission o f the copyright owner. Further reproduction prohibited without permission.
in an organization (i.e. inform ation systems, strategic planning, hum an resources, and
medical and non-medical processes) is empirically validated in the studied U.S. hospitals.
Also, correlations show the strong association between the System and Results (r = .57, p
< .01 for Category 6.0; r = .69, p < .01 for Category 7.0). A gain, these correlations
indicate that an association betw een the System and results in U.S. hospitals is
empirically validated.
The results of Hypotheses 1 and 2 are important for several reasons. First, these
results provide evidence for hospital administrators that leadership, management systems,
and perform ance outcom es are highly interrelated in hospitals.
outcomes, both organizational (Category 6.0) and custom er satisfaction (Category 7.0),
are likely to have good system s o f m anagem ent.
In the remainder of this chapter, causal influences within the Baldrige Health Care
model are investigated. The relationships of association that are identified by the first
138
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
two hypotheses provide the initial step in validating the Baldrige model in the hospital
environment.
5.3
hypotheses. These hypotheses test the most aggregate level of causal influences within
the Baldrige H ealth C are m odel, and they hy p o th esize the direction o f causation.
Baldrige theory purports that Leadership (1.0) is the driver o f the System (2.0, 3.0, 4.0,
5.0) which causes R esults (6.0, 7.0) for hospitals.
positive influence on the System , and the System has a positive influence on Results.
The three hypotheses represent an aggregate and recursive causal network. These causal
relationships are tested by the following hypotheses:
H 3:
Leadership (1.0) has a positive influence on the System (2.0, 3.0, 4.0, 5.0).
[Yu > 0 ]
H4:
The System (2.0, 3.0, 4.0, 5.0) has a positive influence on Organizational
Performance Results (6.0). [B3i > 0]
H5:
The System (2.0, 3.0, 4.0, 5.0) has a positive influence on Focus on and
Satisfaction o f Patients and Other Stakeholders (7.0). [B31 > 0]
139
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
obtained by com puting principal component scores for the questionnaire items on each
dimension and then sum m ing the dimension scores according to the Baldrige w eighting
scheme (NIST, 1995b). This process is described in m ore detail above in Sections 5.1.1
and 5.2.1.
Hypotheses 3 through 5 test the significance o f the causal paths shown in Figure
3.2 in which each tested path is labeled with the corresponding hypothesis number. The
proposed relationships are that Leadership (1.0) has a positive causal influence on the
System (2.0, 3.0, 4.0, 5.0), and the System has a positive causal influence on
perform ance.
Organizational Perform ance Results (6.0) and Focus on and Satisfaction of Patients and
Other Stakeholders (7.0).
The m ethodology used to test H ypotheses 3 through 5 is structural equation
modeling. Developm ent of the structural model is described in the following section.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
specifies the causal paths among the constructs specified in the m easurem ent model.
Figure 5.7 show s the location of a structural path relative to the m easurem ent model
components shown in Figure 5.1. The structural path in Figure 5.7 is a directional causal
path from Leadership (1.0) to the System (2.0, 3.0, 4.0, 5.0). The category components
o f the System, Categories 2.0,3.0, 4.0, and 5.0, are not shown in Figure 5.7 (due to space
limitations). The dimensions and item s associated with the System categories are not
shown.
Figure 3.3 shows that the model tested here is a recursive system in which the
direction o f influence is always from left to right but never from right to left. That is,
Leadership exerts a causal influence on the System, and the System does not influence
Leadership, i.e., there is no reciprocal causation. For example, the System (endogenous
variable) is dependent only on Leadership (exogenous variable), and O rganizational
Perform ance R esults (endogenous variable) is dependent only on the previously
accounted for System (endogenous variable).
The structural equation model shown in Figure 3.3 is estimated using RAMONA
structural equation modeling software which is in the Systat software package (Systat,
1995). The input for model estim ation is a correlation matrix of the four constructs
evaluated here (Leadership; System; O rganizational Performance Results; Focus on and
Satisfaction o f Patients and O ther S takeholders).
standardized, the correlation matrix and the covariance matrix are identical. Generalized
least squares is the estim ation m ethod used.
estim ation m ethod in most structural equation m odeling softw are packages, but
generalized least squares is more com putationally robust and is useful when maximum
likelihood does not converge well, as happened in this analysis. Both o f these estimation
methods are valid under the same assumptions of normality. The difference between the
141
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
tit measures and path estim ates when using generalized least squares and m axim um
likelihood is minimal (Browne, 1974).
The results o f estim ation of the m odel show n in Figure 3.3 (w hich tests
Hypotheses 3 through 5) indicate that the model contains no redundant parameters and no
boundary estimates (variance estimates of zero), both o f which would indicate problem s
in the model. Following is a discussion of model Fit.
The sam ple discrepancy function is an indication o f the fit o f the m odel to the
sam ple correlation m atrix (S), with a minimum value o f zero (Browne and C udeck,
1993). W hen estim ating a structural model using generalized least squares, the m odel
param eters are determ ined so that the following equation is minimized (B row ne and
Mels, 1994 p. 84):
F gls = AutS-KS-Sou)!2
(5.1)
For the aggregate causal model (shown in Figure 3.3), the sample discrepancy function is
.023.
The population discrepancy function is an estim ator o f the fit of the m odel to the
population (rather than the sample) correlation m atrix. This function is an estim ate
because the population correlation matrix is not know n. To calculate this function, the
sample discrepancy function is adjusted to account for sam ple size and the degrees o f
142
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
freedom in the m odel. The population discrepancy function is calculated using the
following equation (Browne and Mels, 1994 p. 86):
(5.2)
The population discrepancy function for the m odel estim ated here (show n in
Figure 3.3) is .009. The model has three degrees o f freedom , and sample size is 220.
The root mean square error of approximation (RM SEA) is a measure o f m odel fit
that is not dependent on sample size. Many other fit m easures (e.g., chi-square, goodness
o f fit index) are highly dependent on sample size.
population discrepancy function (F0) divided by the num ber o f degrees o f freedom in the
model, i.e., the square root of F0/d f (Steiger and Lind. 1980). In other words, RM SEA is
a measure o f the fit per degree o f freedom in the m odel. The following guidelines are
used to determine model fit using RMSEA (Browne and Mels, 1994):
RMSEA
< .0 5
.05 - .10
>
.10
Conclusion
Good m odel fit
Reasonable m odel fit
Poor model fit
143
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
The RMSEA for the m odel tested here is .056, indicating good to reasonable fit of the
model. This measure of model fit is reported in Table 5.1.
The chi-square test statistic for the aggregate causal model is 5.049 (p = .17), as
reported in Table 5.1. A chi-square value close to the degrees o f freedom in the model
indicates that the sam ple correlation matrix and the reproduced correlation matrix
resulting from model estim ation do not differ significantly. There are three degrees of
freedom in this model. The significance level of the chi-square statistic indicates the
probability that the difference between the sample and reproduced m atrices is due only to
sam pling variation (H air et al., 1995).
reported here (p = .17) indicates that the sample and reproduced m atrices are not
significantly different. It is important to note that the chi-square statistic is very sensitive
to sample size, and sam ples larger than 200 (this study has a sam ple size o f 220) usually
result in a significant chi-square statistic regardless of the model being estim ated (Hair et
al., 1995). Therefore, it is recommended that less reliance be place on the chi-square
statistic and more on the RM SEA, which indicates good to reasonable m odel fit for the
model estimated here.
An analysis of the residual correlation matrix is also perform ed. The residual
m atrix is the difference betw een the sample correlation m atrix and the reproduced
correlation matrix which results from model estim ation.
particularly problematic, it appears to indicate that the relationship betw een these two
constructs may not be fully accounted for by the specified model. G enerally, residuals
less than .10 are considered adequately small. The magnitudes o f the residuals for this
model are reported in Table 5.1.
144
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Residuals can be reduced or eliminated by adding paths in the m odel, but doing so
changes the specified model. Because the goal here is to determine the path estimates in
the structural model shown in Figure 3.3, no changes are made to the m odel simply to
reduce correlation residuals. This analysis is strictly confirmatory structural equation
modeling, and the goal is to test the Baldrige theory o f causation and the direction of
causation am ong the Baldrige constructs.
To confirm the model estimation results from RAMONA softw are, the model is
also estim ated using LISREL software, another structural equation m odeling software
package.
M odel fit and path estim ates from LISREL confirm those obtained from
RAMONA.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
5.2. The location o f each of the error terms is shown in Figure 3.3. The variance of .396
for the System erro r term indicates that approxim ately 60 p ercent (1 - .396) of the
variance in the System is accounted for by the model and the rem aining 40 percent is not
explained by the m odel. Likewise, the model accounts for 45 percent of the variance in
Category 6.0 and 60 percent of the variance in Category 7.0. The 90 percent confidence
intervals and standard errors for these variances are also reported in Table 5.2. All of the
error term variances are statistically significant at p < .01 w hen t-tests are performed,
indicating that these parameters are significantly different from zero.
The results o f testing the aggregate causal hypotheses are given below. These
hypotheses are tested by evaluating the significance o f the ag g reg ate model paths
reported in Table 5.2. Figure 5.8 show s the aggregate causal model w ith the paths tested
by Hypotheses 3 through 5 labeled with their path estimates.
H3:
Leadership (1.0) has a positive influence on the System (2.0, 3.0, 4.0, 5.0).
[yu > 0 ]
H4:
The System (2.0, 3.0, 4.0, 5.0) has a positive influence on O rganizational
Performance Results (6.0). [B2i > 0]
Supported (p < .01)
H5:
The System (2.0, 3.0,4.0, 5.0) has a positive influence on Focus on and
Satisfaction o f Patients and O ther Stakeholders (7.0). [B3I > 0 |
Empirical support for Hypotheses 3 through 5 shows that the hypothesized causal
influences are observed in the studied U.S. hospitals.
through 5 provide support for the aggregate-level Baldrige theory that Leadership is the
146
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
driver of the System, and im proving the design and m anagem ent o f the System leads to
better Results (i.e., Baldrige C ategories 6.0 and 7.0).
The causal and directional influences identified here indicate that im proving
hospital leadership should result in an improved Baldrige-defined System. These results
support those of Batalden and Stoltz (1993) who report that strong support by hospital
adm inistrators of quality initiatives helps to accelerate im plem entation o f C ontinuous
Quality Improvement (i.e., TQ M ). Additionally, the causal and directional influence of
the System on Results that is em pirically validated here supports Gustafson and Jundt's
(1995) review of the health care literature which reveals that TQM has been tied to
improved quality perform ance and customer satisfaction in the health care environment.
The causal paths am ong the constructs in the aggregate causal m odel all have
relatively high weights, as docum ented in Table 5.2. T he w eights are standardized so
that they are directly com parable. For example, a path w eight of .80 indicates a stronger
influence than a path weight o f .50. The path weights estim ated here (approximately .7 to
.8) are relatively high, indicating that Leadership is a good predictor of the System and
the System is a good predictor o f both Results categories. The 90 percent confidence
intervals are also analyzed to determine the precision of these estimates.
The error term variances in Table 5.2 show that 60 percent of the variance in the
System, 45 percent of the variance in Organizational Performance Results, and 60 percent
o f the variance in Focus on and Satisfaction o f Patients and O ther Stakeholders are
accounted for by the m odel.
variance accounted for by the aggregate causal model indicate strong predictive pow er
w ithin the model.
Leadership, and hospital R esu lts (both organizational perform ance and cu sto m er
satisfaction) are largely based on the performance of the System.
147
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
H6:
H7:
Leadership (1.0) has a positive influence on Strategic Planning (3.0). [y2I > 0]
H8:
Leadership (1.0) has a positive influence on Human Resource D evelopm ent and
M anagement (4.0). [y31 > 0]
H9:
Leadership (1.0) has a positive influence on Process Management (5.0). [y41 > 0]
H I0:
Hn :
Hp :
148
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
H 13:
Inform ation and Analysis (2.0) has a positive influence on Human Resource
Developm ent and Management (4.0). [B31 > 0]
Hu:
Inform ation and Analysis (2.0) has a positive influence on Process M anagem ent
(5.0). [B41 > 0 ]
H 15:
H 16:
H 17:
H lg:
H 19:
H20:
H21:
H22:
H23:
Reproduced with permission o f the copyright owner. Further reproduction prohibited without permission.
This
method uses a correlation m atrix o f the 28 Baldrige dim ension scores obtained from the
measurement model described in Section 5.2.1 as input for structural model estimation.
theoretically sound because it is designed by hundreds of quality and health care experts
and is based on the original M BNQA (industrial) m odel w hich has been studied
empirically (e.g., W ilson and Collier, 1998a; W ilson, 1997; H andfield and Ghosh, 1995).
Additionally, the m easures used in the analyses perform ed here are highly reliable (see
Section 4.5.1 and Table 4.13).
150
Reproduced with permission o f the copyright owner. Further reproduction prohibited without permission.
The methodology used here separates the Baldrige System into its com ponents.
Categories 2.0, 3.0, 4.0, and 5.0, and estimates the specific causal model by allowing the
correlation terms betw een the Baldrige constructs to be estim ated based on optim al
combinations o f each constructs dimensions. This contrasts with the aggregate causal
model in which the seven-construct Baldrige model was sim plified based on the Baldrige
weighting scheme to its four m ajor components: Leadership, the System, and two Results
categories. Analysis o f the aggregate causal model is described in Sections 5.3, 5.3.1,
and 5.3.2.
Figure 5.9 shows an exam ple of the location o f a structural path relative to the
Baldrige constructs and dim ensions. The structural path in Figure 5.9 is a directional
causal path from the Leadership (1.0) construct to the Inform ation and Analysis (2.0)
construct. In the complete model estimated here, there are sim ilar causal paths among the
rem aining Baldrige constructs, but due to space lim itations. Figure 5.9 shows ju st a
portion of the full model.
The structural m odel shown in Figure 3.5 is estim ated using maximum likelihood,
the first method employed by m ost researchers and the default estimation method in most
structural equation m odeling softw are packages.
estim ated above, the m odel estim ation procedure m inim izes the sample discrepancy
function. Following is the equation used to calculate the sam ple discrepancy function
using maximum likelihood estimation:
F m l = lo g IIm lI
'
lo g
151
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
(5.3)
The sample discrepancy function for the specific causal model is 3.991. The population
discrepancy function, calculated using Equation 5.2 above, is 2.475. The results o f model
estim ation show that the m odel contains no redundant param eters and no boundary
estimates.
Model fit data for the specific causal model are shown in Table 5.3. The RM SEA
of .086 indicates reasonable fit of the model. Seventy-four param eters are estim ated in
this model with 332 degrees of freedom.
The chi-square test statistic is 874 (p < .01). The significance level indicates that
the sample correlation m atrix is significantly different from the reproduced correlation
matrix (which results from model estimation). However, H a ire t al. (1995) point out that
most models have significant chi-square values when estim ated with a sample of more
than 200 observations (this study has 220 observations).
dividing chi-square by the degrees of freedom in the model to measure what H air et al.
(1995) term the 'norm ed chi-square.' For this model, the norm ed chi-square is 2.63
(874/332). Carmines and M clver (1981) use an upper threshold for the norm ed chisquare of 2.0 to 3.0 to conclude that a model represents the observed data. It should be
noted that, like the original chi-square statistic, the normed chi-square value is also based
on sample size and additionally, is a guideline with no statistical basis.
The sample correlation matrix contains 406 correlation term s that represent the
relationships among the 28 Baldrige dimensions. The residual matrix is the difference
between the sample correlation matrix and the reproduced correlation matrix that results
from model estimation. The 406 terms in the residual correlation matrix are categorized
by magnitude in Table 5.3, and the proportion of residuals in each category is reported.
152
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Tw o hundred tw enty-eight (71 percent) of the 406 residuals are less than .05.
An
additional 20 percent o f residuals are in the range from .05 to .10. Taken together, 91
percent of the residuals are less than .10, a level that is generally considered to be
adequately small. In other words, this model is able to predict 91 percent o f the measured
variable correlations within . 10.
The remaining 9 percent of residuals reported in Table 5.3 are larger in magnitude
and may indicate underestim ated (i.e., these large residuals are all positive) correlation
relationships in the model. M ost of these larger residuals are very close to . 10 and are not
viewed as problem atic because residuals around .10 are generally considered to be
adequately small. O f the 9 residuals (2.5 percent) that are greater than .20, six involve a
correlation relationship with Dimension 7.5 Patient/Stakeholder Satisfaction Com parison.
In other words, the sam ple of hospital data analyzed here has six Baldrige dim ensions
that have higher correlations with Dimension 7.5 than this model would predict. Patient
and stakeholder satisfaction is evaluated in a num ber o f ways in the Baldrige criteria, as
indicated by the C ategory 7.0 dimensions shown below . None o f correlations involving
Dimensions 7.1 through 7.4 (that also address patient and stakeholder satisfaction) have
large residuals.
correlations between Dimensions 7.1 through 7.4 and other Baldrige dimensions.
7.2
7.3
7.4
7.5
153
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
R esiduals can be reduced or elim inated by adding paths b etw een pairs of
dim ensions w hich show large residuals, but doing so changes the specified m odel.
Similar to the structural equation model analysis reported in Section 5.3.2 above, the goal
here is to estim ate the paths specified in the structural model shown in Figure 3.5 and to
provide support for the Baldrige Health Care A w ard m odel. Therefore, no changes are
made in the model.
M odel fit and path estim ates from R A M O N A softw are are confirm ed using
LISREL softw are. Because the path estim ates from the constructs to the dim ensions in
the specific causal m odel are all greater than .50, it can be concluded that each dimension
is a good indicator o f the construct that it m easures. Likewise, the variance estim ates for
the error term s o f the dimensions in the m odel indicate that a majority o f the variance in
each of the dim ensions is accounted for by the construct that it measures.
significant positive causal influence on each o f the Baldrige System categories (2.0, 3.0,
4.0, 5.0) and O rganizational Performance R esults (6.0). Additionally, Inform ation and
Analysis (2.0) has a significant positive causal influence on each of the o ther System
constructs (3.0, 4.0, 5.0) and O rganizational Perform ance Results (6.0).
Finally, the
influence of Hum an Resource Development and M anagem ent (4.0), Process M anagement
154
Reproduced with permission o f the copyright owner. Further reproduction prohibited without permission.
(5.0), and O rganizational Perform ance Results (6.0) on Focus on and Satisfaction of
Patients and O ther Stakeholders (7.0) are all significant.
The results o f testing the specific causal hypotheses are given below. These
hypotheses are tested by analyzing the significance of each of the m odel paths reported in
Table 5.4. Figure 5.10 shows the specific causal model with each o f the tested paths
labeled with the path estimate.
H6:
[Yu > 0]
H7:
Leadership (1.0) has a positive influence on Strategic Planning (3.0). [y;i > 0]
H9:
Leadership (1.0) has a positive influence on Process M anagem ent (5.0). [y41 > 0]
Not supported
H 12:
H 13:
155
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
H 14:
H 15:
H 16:
H 17:
H lg:
H 19:
Human Resource D evelopm ent and Management (4.0) has a positive influence on
Organizational Performance Results (6.0). [B53 > 0]
Not supported
H 20:
Human Resource D evelopm ent and Management (4.0) has a positive influence on
Focus on and Satisfaction o f Patients and Other Stakeholders (7.0). [B63 > 0]
H 22:
H 23:
It should be noted that the significance of each o f the specific causal hypotheses is
conditional upon the presence of the other paths in the tested model. Removing a path or
156
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
adding a new path to the specific causal model may change the significance o f these
hypothesis tests.
The results of testing the specific causal hypotheses provide empirical support for
many of the relationships included in the Baldrige theory, while other Baldrige theorized
relationships are not supported by this empirical analysis. Specifically, H ypotheses 6
through 10, 12 through 15, 20, 22, and 23, as discussed in detail below, are supported
based on the statistical significance tests of the causal paths that they address. The causal
influences that these hypotheses address are em pirically supported in the study sam ple of
U.S. hospitals.
Hypotheses 6 through 11 address the causal influence o f Leadership (1.0) on each
o f the other categories in the Baldrige H ealth C are m odel.
supported, except H ypothesis 11 which tests the influence o f Leadership (1.0) on Focus
on and Satisfaction o f Patients and Other Stakeholders (7.0), one of the two R esults
categories. The results of these hypothesis tests indicate that Leadership is an overall
driver of systems and processes in hospitals. However, its effect on customer satisfaction
(Category 7.0) may be only indirect through the other categories in the Baldrige m odel on
which Leadership has a direct causal influence.
The strong influence of Leadership on each o f the System categories (2.0, 3.0,
4.0, 5.0), as indicated by the path weights reported in Table 5.4, confirms the results from
the aggregate causal model, where Leadership (1.0) has a significant causal influence on
the System as a w hole (H ypothesis 3, see Section 5.3.2).
standardized and their relative weights can be compared. For example, the path estim ate
from Leadership to Inform ation and Analysis (2.0) is .775 (p < .01), approximately twice
the weight of the estim ate of the path from Leadership to Human Resource D evelopm ent
and M anagement (4.0) which is .381 (p < .01). This m eans that the causal influence of
Leadership on Inform ation and Analysis is about twice as strong its influence on H um an
157
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Resources. The 90 percent confidence intervals o f these path estimates, shown in Table
5.4, indicate that both are fairly precise, and the relative relationship betw een the two
paths holds even if the estim ates fluctuate within their respective intervals.
It should be noted that the 90 percent confidence interval for the causal influence
o f Leadership on O rganizational Performance R esults (6.0) ranges from .114 to .743.
The path estim ate o f .429 (p < .05) appears m oderately strong, but the real influence o f
Leadership on this results category is not precisely estim ated in this model based on the
data that is analyzed. The confidence interval indicates that, based on random ly selected
samples o f 220 hospitals from the studied population, 90 percent of the estim ates for the
path from Leadership to Organizational Perform ance Results would be between . 114 and
.743.
H ypotheses 12 through 14 address the causal influence o f Inform ation and
Analysis (2.0) on the other System categories. Strategic Planning (3.0), Human R esource
D evelopm ent and M anagem ent (4.0), and Process M anagem ent (5.0). Each o f these
hypotheses is supported. These hypotheses are the first tests of w ithin-System causal
influences, and their significance provides evidence that the System categories (2.0, 3.0,
4.0, 5.0) are interlinked.
Taken together. Hypotheses 6 through 14 provide several insights for hospital
administrators. Strong support of quality initiatives from senior level m anagem ent (i.e..
Leadership) has long been cited as the critical starting point in an organizations quest for
a quality-driven culture. Here, Leadership has a positive causal influence on each o f the
System categories. These results provide em pirical validation for the role o f hospital
leadership as the critical driver of a hospitals im plem entation of quality-focused system s
and processes.
The significant influence of Leadership on O rganizational Perform ance Results
indicates that senior m anagem ent may have a direct role in improving quality outcom es.
158
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
These results support Batalden and Stoltz's (1993) findings that strong support by senior
adm inistrators is an accelerator in the im plem entation o f quality initiatives in hospitals.
Leadership's role in the quality m anagem ent system s o f hospitals is both direct, as
indicated by the significant path from Leadership to Organizational Perform ance Results,
and indirect as seen by its influence on R esults through the System categories o f the
Baldrige model.
The significant influence of Information and Analysis (2.0) on Strategic Planning
(3.0), Human Resource Development and M anagem ent (4.0), and Process M anagem ent
(5.0) (H ypotheses 12 through 14) supports Baldrige theory that, An effective health care
system needs to be built upon a fram ew ork o f measurement, inform ation, data, and
analysis (N IST, 1995b p. 4). Analysis extracts meaning from data and enables health
care organizations to make informed decisions and develop appropriate perform ance
indicators. Inform ation systems are most effective when used both w ithin and across
hospital departm ents, and this is reflected by the significant influence of Inform ation and
Analysis on each o f the other System categories. As shown in Figure 1.1, the published
Baldrige Health Care Award model includes unspecified directional influences within the
System. The results of Hypotheses 12 through 14 provide evidence that inform ation
systems are the critical interlinking factor within the System.
Hypotheses 15 through 22 address the causal influences of the System categories
on the two R esults categories. Hypothesis 15, which tests the influence o f Information
and Analysis (2.0) on Organizational Perform ance Results (6.0), is supported. Like the
influence of Leadership (1.0) on Category 6.0 described above, the relationship between
Categories 2.0 and 6.0 is both direct and indirect. Information and A nalysis has a direct
causal influence on Organizational Perform ance Results, with a path w eight estimate of
.522 (p < .05), and an indirect effect through its influence on other System categories. It
should be noted that the 90 percent confidence interval for the causal influence of
159
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Category 2.0 on Category 6.0 ranges from .091 to .953, indicating that the model is not
able to estimate this path precisely based on the model estim ated and the sample data
used.
Hypothesis 20 is supported, indicating a significant causal influence o f Human
Resource Development and M anagem ent (4.0) on Focus on and Satisfaction o f Pauents
and O ther Stakeholders (7.0), hereafter referred to as custom er satisfacuon. As described
in Secuon 1.2.1, hospital custom ers (pauents, padent families, etc.) may have difficulty
evaluating clinical quality (technical aspects of medical care), but they can more readily
evaluate service quality. Thus, custom er interactions with hospital personnel likely has a
strong influence on their satisfaction.
hospital staff to interact with them in a way that goes beyond courtesy to a relationship
that approaches love. The very personal interactions between patients and hospital
staff may be strong determinants o f padent satisfaction.
Developing better w ork system s, improved staff training, and m easuring and
promoting staff well-being, all components of Baldrige Category 4.0, help to increase the
satisfaction of hospital staff, w hich in turn improves patient (and other custom ers)
satisfaction.
customers of personal insurance firms and find that employees' self-perceived capabilities
are closely tied to their satisfaction. These results indicate that training employees and
improving job design can improve employee satisfaction.
Hypothesis 22, which addresses the causal influence of Process Management (5.0)
on custom er satisfaction (7.0) is also supported. Similar to human resources, as described
above, process m anagem ent is often visible to patients in the design and delivery of
medical services (e.g., x-rays, m inor surgery) and non-m edical services (e.g., billing,
food service). Estimation o f the specific causal model show s that Process M anagement
has a stronger influence on custom er satisfaction than the other Baldrige categories.
160
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Table 5.4 shows that the path from Process M anagem ent to customer satisfaction has a
w eight of .772 (p < .01), while the path from Human Resources to custom er satisfaction
is .262 (p < .05). The significant link between processes and customer satisfaction that is
documented here provides motivation for hospitals to design and manage their processes
from the patients perspective. Patient focus is one o f the core values o f the Baldrige
Health Care criteria (NIST, 1995b); quality and value are considered its key com ponents.
A dditionally, Process M anagem ent addresses the design and delivery o f com m unity
health services and supplier management. These aspects of process m anagem ent are less
visible, but may indirectly influence custom er satisfaction.
Finally, H ypothesis 23 is supported, supporting validation of a causal influence
from Organizational Perform ance Results (6.0) to custom er satisfaction (7.0). This link
is not unexpected, as C ategory 6.0 measures internal performance while C ategory 7.0
addresses external m easures of performance. The com ponents of Category 6.0, including
the outcom es associated w ith patient health care services, support serv ice s, and
com m unity services, serve as m anagem ents o b jectiv e m easures o f a h o s p ita l's
performance, while m easures o f customer satisfaction provide management w ith a gauge
o f perceptive m easures from outside the organization.
Im proving O rg an izatio n al
161
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
5.5 Summary
In summary, 23 research hypotheses are addressed in this study using correlation
analysis and structural equation m odeling. First, a measurement m odel is developed,
using principal components analysis to reduce the item scores obtained from the study
questionnaire to the 28 Baldrige dim ension scores.
H ypotheses 1 and 2 are tested by first using the B aldrige-defined w eighting
schem e (shown in Table 1.3) to calculate scores for each o f the seven m ain Baldrige
constructs, and second by further w eighting the System constructs (2.0, 3.0, 4.0, 5.0) to
calculate a single System score for each hospital in the study. H ypotheses 1 and 2 are
supported, as Leadership (1.0) is significantly correlated with the System (2.0, 3.0, 4.0,
5.0) and the System is significantly correlated with hospital Results as m easured by both
O rganizational Performance Results (6.0) and Focus on and Satisfaction o f Patients and
O ther Stakeholders (7.0). These significant findings provide evidence that the most
general relationships im plied by the B aldrige Health C are model and theory are
em pirically validated in hospitals. H ypotheses 1 and 2 test relationships o f association
only, and do not imply causation or a direction of influence am ong the B aldrige
constructs.
H ypotheses 3 through 5, the aggregate causal hypotheses, test causal and
directional influences among the B aldrige constructs. These hypotheses are tested using
structural equation modeling, with the input being a correlation matrix o f the Baldrige
construct and System scores. The aggregate causal model (shown in Figure 5.8) is found
to have reasonable model fit, and H ypotheses 3 through 5 are supported, revealing
significant causal paths from Leadership (1.0) to the System (2.0, 3.0, 4.0, 5.0) and from
the System to each of the two R esults categories. Organizational Perform ance Results
(6.0) and Focus on and Satisfaction o f Patients and Other Stakeholders (7.0).
162
Reproduced with permission o f the copyright owner. Further reproduction prohibited without permission.
These results support the Baldrige theory that L eadership is the driver o f the
System, and good m anagement of the System leads to im proved results. The significant
relationships among these constructs are positive and causal, indicating that changes in
one construct can lead to the same directional change in the constructs that it influences.
M easures of model fit for the aggregate causal model are reported in Table 5 .1, and path
estim ates are reported in Table 5.2.
Structural equation modeling is used to test H ypotheses 6 through 23, the specific
causal hypotheses. The model uses a correlation matrix o f the 28 Baldrige dimensions as
input for model estimation, and the results are shown in Figure 5.10. Overall model fit is
reasonable (see Table 5.3), and numerous causal paths among the Baldrige constructs are
found to be statistically significant (see Table 5.4).
The results of estim ating the specific causal model support Baldrige theory that
Leadership (1.0) has a positive causal influence on each o f the System categories (2.0,
3.0, 4.0, 5.0) and Information and Analysis (2.0) has a positive causal influence on each
o f the other System categories (3.0, 4.0. 5.0).
Inform ation and A nalysis (2.0) have positive causal influences on O rganizational
Perform ance Results (6.0).
Process M anagement (5.0), and Organizational Performance Results (6.0) have positive
causal influences on Focus on and Satisfaction of Patients and O ther Stakeholders (7.0).
The results of testing the specific causal hypotheses provide empirical support for
som e o f the relationships included in the Baldrige theory.
relationships are not supported by this empirical analysis. In total, the results indicate
that the categories within the Baldrige Health Care m odel are highly related and that
specific causal influences am ong them explain much o f the performance of hospital
systems as well as hospitals' internal and external outcome measures.
163
Reproduced with permission o f the copyright owner. Further reproduction prohibited without permission.
Item
Value
.056
5.049 (p = . 17)
Degrees of freedom
Residuals 1
< .05
.0 5 -.1 0
>.10
(60%)
3 (30%)
I ( 1 0 %)
The matrix used to estim ate the aggregate causal model contains 10 correlation terms.
The total number and proportion o f residuals that fall into each range is listed.
1
Table 5.1: Overall M odel Fit for Aggregate Causal M odel (see Figure 5.8)
164
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Point
estimate
90% confidence
interval
Standard
error
t-value
Leadership->System 2
.724
(.580, .867)
.087
8.28**
System->Performance 3
.697
(.594, .800)
.063
1 1 . 1 2 **
S y s te m -^ u s to m e r 1
.810
(.695, .924)
.070
11.62**
Leadership<->Leadership
1 .0 0 0
(.855, 1.170)
.096
10.46**
e(Syst)<->e(Syst)
.396
(.263, .595)
.098
4.02**
e(Perf)<->e(Perf)
.554
(.392, .782)
.116
4.76**
(.236, .670)
.126
3.15**
Path
Note:
165
Reproduced with permission o f the copyright owner. Further reproduction prohibited without permission.
Item
Value
.086
874.1 (p = .00)
Degrees o f freedom
332
74
< .05
.0 5 -.1 0
.1 0 - .20
.20 - .30
> .3 0
228 (71%)
82 (2 0 %)
27 (7%)
7 (2%)
2 (0.5%)
1 The matrix used to estimate the specific causal model contains 406 correlation terms.
The total num ber and proportion of residuals that fall into each range is listed.
Table 5.3: O verall Model Fit for Specific Causal Model (see Figure 5.10)
166
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Path
Point
estimate
90% confidence
interval
Standard
error
t-value
L eadership'-^nform ation 2
.775
(.708, .841)
.040
19.21**
Leadership->Strategy 3
.470
(.330, .610)
.085
5.51**
Leadership->Human Res .4
.381
(.239, .524)
.087
4.40**
Leadership->Process 5
.364
(.224, .503)
.085
4.30**
Leadership->Performance 6
.429
(.114, .743)
.191
2.24*
-.055
(-.311, .2 0 2 )
.156
0.35
Information->Strategy
.460
(.321, .599)
.085
5.43**
Information->Human Res.
.547
(.409, .685)
.084
6.52**
Information->Process
.610
(.477, .742)
.081
7.54**
Information->Performance
.522
(.091, .953)
.262
1.99*
Information->Customer
-.347
(-.703, .008)
.216
1.61
Strategy->Performance
-.154
(-.407, .108)
.160
0.97
Strategy->Customer
.013
(-.185, .211)
.120
0.11
Human Res.->Performance
.132
(-.128, .392)
.158
0.84
Human Res.->Customer
.262
(.070, .453)
.116
2.25*
-.216
(-.604, .171)
.236
0.92
Process->Customer
.772
(.462, 1.082)
.188
4.10**
Performance->C ustomer
.301
(.169, .432)
.080
3 7 7 **
Leadership->Customer 7
Process->Performance
167
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Point
estimate
90% confidence
interval
Standard
error
e( 1 -1 )<->e( 1.1 )
.413
(.343, .497)
.047
8 . 8 8 **
e ( 1 .2 )<->e( 1 .2 )
.259
(.206, .325)
.036
7.18**
e( 1 .3)<->e( 1.3)
.329
(.269, .403)
.404
8.16**
e( 2 . 1 )<->e( 2 . 1 )
.687
(.583, .810)
.069
1 0 .0 1 **
e( 2 .2 )<->e(2 .2 )
.458
(.383, .547)
.049
e(2.3)<->e(2.3)
.194
(.145, .258)
.034
5.73**
e(3.1)<->e(3.1)
.162
(.123, .214)
.027
5.93**
e(3.2)<->e(3.2)
.128
(.091, .179)
.026
4.83**
e(4.1)<->e(4.1)
.233
(.191, .285)
.028
8 . 2 1 **
e(4.2)<->e(4.2)
.259
(.213, .314)
.030
8.51**
e(4.3)<->e(4.3)
.194
(.156, .241)
.026
7.61**
e(4.4)<->e(4.4)
.282
(.234, .340)
.032
8.73**
e(5.1)<->e(5.1)
.282
(.232, .342)
.033
8.44**
e(5.2)<->e(5.2)
.270
(.221, .329)
.032
8.31**
e(5.3)<->e(5.3)
.453
(.381,.538)
.048
9.53**
e(5.4)<->e(5.4)
.648
(.550, .763)
.064
10.05**
e(5.5)<->e(5.5)
.467
(.394, .555)
.049
9.58**
e(5.6)<->e(5.6)
.648
(.550, .763)
.064
10.05**
e( 6 . 1 )<->e(6 . 1 )
.523
(.435, .630)
.059
8.85**
e( 6 .2 )<->e(6 .2 )
.302
(.234, .391)
.047
6.38**
e(6.3)<->e(6.3)
.630
(.529, .750)
.067
9.43**
e(6.4)<->e(6.4)
.599
(.501, .715)
.064
9.28**
e(6.5)<->e(6.5)
.608
(.510, .725)
.065
9.33**
e(7.1)<->e(7.1)
.323
(.267, .391)
.038
8.61**
e(7.2)<->e(7.2)
.194
(.152, .248)
.029
6.77**
e(7.3)<->e(7.3)
.272
(.221, .333)
.034
8.07**
e(7.4)<->e(7.4)
.604
(.512, .713)
.061
9.90**
.703
(.597. .827)
e(7.5)<->e(7.5)
Note: ** path significant at p < .01
1 variance for error term of Dimension 1.1
.070
1 0 . 1 0 **
Path
t-value
25**
Table 5.5: Error Term Variances for Dimensions in Specific Causal Model
168
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Point
estimate
90% confidence
interval
Standard
error
e( 2 .0 )<->e( 2 .0 ) 1
.400
(.309, .517)
.062
6.40**
e(3.0)<->e(3.0)
.232
(.178, .303)
.038
6.19**
e(4.0)<->e(4.0)
.233
(.179, .302)
.037
6.30**
e(5.0)<->e(5.0)
.153
(.107,.218)
.033
4.64**
e( 6 .0 )<->e( 6 .0 )
.523
(.423, .648)
.068
7.73**
e(7.0)<->e(7.0)
.256
(.190, .346)
.047
5.48**
Path
t-value
Table 5.6: Error Term V ariances for Categories in Specific Causal Model
169
Reproduced with permission o f the copyright owner. Further reproduction prohibited without permission.
a.
CN
CN
CN
170
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
CN
mv
Senior Executive
and Health Care
Staff Leadership
MY,
MV<
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
MV,
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Senior Executive
and Health Care
Staff Leadership
Leadership
System and
Organization
^1.1
X 1.2
'
N>
Leadership
Public
Responsibility
and Citzenship
^1.3
173
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Figure 5.5: Scatter Plot of MBNQA System and Organizational Performance Results
174
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1-
0i
System
Ii
-3 I
-3
-1
1-------------------------------------------------------1
-2
-1
175
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Leadership
,0
Structural path
System
Process
Management
Information
and Analysis
Human Resource
Development & Mgm
Strategic
Planning
176
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Leadership
System
Results
Process
Management
5.0
Leadership
Human Resource
Development and
Management
4.0
Strategic
Planning
3.0
Organizational
Performance Results
6.0
Information and
Analysis
2.0
Figure 5.8: Structural Equation Modeling Results for Aggregate Causal HypoUieses
Leadership
1.0
Structural path
Information and
Analysis 2.0
Measurement paths
2.3
2.2
178
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Process
Management
5.0
-.22
-.06
Human Resource
Development and
Management
4.0
Leadership
y xy
.0 1 /
Strategic
Planning
3.0
.26 *
Organizational
Performance Results
6.0
z , r 7
-.35
. 78 *
Information and
Analysis
. 52 *
2.0
Note:
Figure 5.10: Structural Equation Modeling Results for Specific Causal Hypotheses
CHAPTER 6
Award criteria have been distributed by NIST, an indication of the health care industrys
interest in both the Baldrige Award and in using the criteria for self-assessment purposes.
In 1998, the B aldrige Health Care Award m ay be awarded for the first tim e pending
funding from public and private sources.
This research study, using data gathered from U.S. hospitals, em pirically tests the
theory and perform ance relationships in the pilot criteria of the M alcolm B aldrige
National Q uality Award in Health Care. These criteria are purported to be the foundation
of T otal Q uality M anagem ent (TQM) for the health care environm ent.
E m pirical
validation of the Baldrige model provides hospitals with a framework for developing and
managing their quality systems and improving performance.
This study is undertaken for several reasons. First, the MBNQA in H ealth Care
criteria are derived from the criteria o f the original Baldrige Award (i.e. the industrial
award). This study addresses the appropriateness o f transferring the industrial model to
the health care environm ent. Second, this study exam ines whether the Baldrige theory o f
180
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
causal relationships am ong the Health Care criteria categories are em pirically supported
in hospitals. Third, this study clarifies the direction o f influence am ong the criteria of the
MBNQA in Health Care.
This study is im portant for several reasons. There is little em pirical research in
the health care literature that addresses the relationships among the m ajor components o f
TQM . A dditionally, the current state o f the U.S. health care industry, which has seen
dramatic increases in costs in the past two decades, provides an im petus for undertaking
this study. U nder cost-cutting pressures, som e health care organizations successfully
balance income and spending, maintaining their competitiveness, w hile others struggle to
survive. It is not clear w here these organizations should place their efforts and resources
to maintain quality m edical care and fiscal viability. This research shows where these
efforts and resources are best invested in these organizations.
The goal o f the M BN Q A in H ealth Care is to prom ote aw areness of the
importance of quality im provem ent in U.S. health care organizations. The structure of
the award is based on a m odel of TQ M for health care organizations.
TQM is a
management approach to building quality consciousness and quality perform ance into an
organizations daily operations. The M BNQA in Health Care has been developed by
quality and health care experts who hypothesize causal influences and linkages among
the award criteria categories.
Over 90 percent o f U.S. hospitals report some level o f engagem ent in TQM
activities (Sanders, 1997). The practitioner literature confirms that hospitals are finding
benefits from im plem entin g TQ M .
proliferation of TQ M in the health care industry is to provide em pirical support for the
performance relationships among the m ajor components of the Baldrige framework. This
study is the first m ajor em pirical investigation of this type.
181
Reproduced with permission o f the copyright owner. Further reproduction prohibited without permission.
Group
Category
Leadership:
1.0 Leadership
System:
Results:
6.1 Overview
The objective o f this study is to em p irically investigate the perform ance
relationships implied by the model presented in the 1995 pilot criteria o f the M BNQA in
Health Care (NIST, 1995b). Research hypotheses are developed to evaluate three levels
182
Reproduced with permission o f the copyright owner. Further reproduction prohibited without permission.
of relationships within the framework of the Baldrige model, including general measures
of association, aggregate-level causal influences, and specific causal influences among
the model constructs.
Section 6.2.
This study is the first major empirical investigation o f the causal linkages in the
criteria of the M BNQA in Health Care. No previous studies have evaluated the Baldrige
Health Care criteria in this manner. However, several studies, as discussed in Section
2.2, have used the original MBNQA criteria (from the industrial award) to m easure the
extent o f TQM implementation in hospitals (Carman et al., 1996; Shortell et al., 1995;
Jennings and W estfall, 1994).
Several studies have em pirically validated perform ance relationships in the
original Baldrige Award model as well as other models of TQM in the m anufacturing
environment. W ilson (1997) and Wilson and C ollier (1998a) report that m any o f the
causal linkages am ong the Baldrige model constructs are em pirically validated in
manufacturing firms. M ost importantly, these studies document that TQM system s can
be linked to perform ance outcomes in m anufacturing firms, including both custom er
satisfaction and financial performance. Handfield and Ghosh (1995) find sim ilar results
in their evaluation of the Baldrige criteria in the manufacturing environment. Flynn et al.
(1994) test a sim ilar model of TQM and find results that support leadership as the driver
in quality m anagem ent systems and tie quality management systems to perform ance in
manufacturing firms.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
documentation of these results in manufacturing firms. The study presented here tests the
Baldrige model of quality m anagem ent and organizational perform ance that is specific to
the health care industry.
First,
relationships of association am ong the m ajor com ponents o f the Baldrige m odel.
Leadership (1.0), the System (2.0, 3.0, 4.0, 5.0), and R esults (6.0, 7.0), are studied to
determine if these com ponents are positively correlated, as B aldrige theory proposes.
Schematics of the proposed correlation relationships are show n in Figure 3.1. Hypothesis
1 tests the correlation betw een Leadership and the System , w hile H ypothesis 2 tests the
correlation between the System and Results.
evaluate directional causal influences from Leadership (1.0) to the System (2.0, 3.0, 4.0,
5.0) (Hypothesis 3) and from the System to Results Categories 6.0 (Hypothesis 4) and 7.0
(Hypothesis 5). The hypothesized aggregate causal relationships are shown in Figure 3.2.
Specific causal hypotheses test the causal influences am ong the seven categories
in the Baldrige Health C are m odel (the seven categories are listed above in Section 6.1).
Leadership (1.0) is proposed to have a causal influence on each o f the other six constructs
in the Baldrige model (H ypotheses 6 through 11). Inform ation and Analysis (2.0) is
hypothesized to influence the other System constructs (3.0, 4.0, 5.0) (H ypotheses 12
through 14) and the tw o R esults categories (6.0, 7 .0 ) (H y p o th eses 15 and 16).
Hypotheses 17 through 22 test the influence of each o f the rem aining System categories
(3.0, 4.0, 5.0) on the tw o R esults categories (6.0, 7.0).
Organizational Perform ance R esults (6.0) on Focus on and Satisfaction of Patients and
O ther Stakeholders (7.0), both Results categories, is evaluated by H ypotheses 23. A
184
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
m odel showing the causal paths tested by the specific causal hypotheses is show n in
Figure 3.4.
6.2.1 Methodology
This study empirically investigates the causal linkages in the pilot criteria of the
M alcolm Baldrige National Q uality Award in Health Care. A mailed questionnaire is
used to obtain measures for each o f the 28 Baldrige dim ensions (subcategories of the
seven main categories) o f the aw ard criteria. Each of the 28 dim ensions is m easured
using a scale, or set o f questions, on which hospital quality m anagers indicated their
perceptions about their organizations. Two-hundred tw enty-eight general acute care
hospitals completed and returned the questionnaire, and 220 o f those responses are used
in the analysis reported here (the remaining eight were m issing performance data). The
28 m easurem ent scales used to m easure the dim ensions o f the Baldrige Health C are
Award are both reliable and valid based on numerous analyses.
Measures of each of the 28 Baldrige dimensions are calculated by using principal
com ponents analysis to obtain the first component score for each scale. These dim ension
scores are used to test the specific causal hypotheses.
aggregate causal hypotheses, the seven Baldrige construct scores are calculated by
proportionally weighting each dim ension score relative to its B aldrige point value.
A dditionally, a score for the Baldrige-defined System is calculated by proportionally
weighting each of the System categories (2.0, 3.0, 4.0, 5.0) relative to its Baldrige point
value.
The results o f testing the research hypotheses are reported and discussed in detail
in C hapter 5.
(correlation hypotheses), 6.2 (aggregate causal hypotheses), and 6.3 (specific causal
hypotheses). A discussion o f the findings of this study is provided below.
185
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
The results o f
Hypothesis 1 show that Leadership (1.0) and the System (2.0, 3 .0 ,4 .0 , 5.0) are positively
correlated (r = .75, p < .01). Hypothesis 2 shows that the System and O rganizational
Performance Results (6.0) are positively correlated (r = .57, p < .01), and the System and
Focus on and Satisfaction o f Patients and O ther S takeholders (7.0) are positively
correlated (r = .69, p < .01). The results o f these correlation analyses and o f testing
Hypotheses 1 and 2 are reported in Table 6.1.
The relationships of association revealed by these hypotheses support the Baldrige
theory that there are three major interrelated com ponents in the Baldrige model o f TQM :
Leadership, System, and Results (NIST, 1995b). The relationships documented here are
associative in nature and do not address causation among the Baldrige constructs. These
correlations reveal that associations between Leadership and System management, and
System management and Results are found in U.S. hospitals.
The correlation between Leadership and the System (r = .75) supports Baldrige
theory that Leadership is strongly tied to systems and processes throughout hospitals.
These system s and processes include information system s and the analysis and use of
data and inform ation, strategic planning processes and how they are com m unicated
throughout the organization, human resources m anagem ent, and the m anagem ent o f
medical and non-medical processes.
The measured correlation between the System and Focus on and Satisfaction o f
Patients and O ther Stakeholders (r = .69) and the System and Organizational Performance
Results (r = .57) provide evidence of the link betw een the System and Results, an
important component o f a model of TQM for hospitals.
186
Reproduced with permission o f the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
estim ates are standardized. The estimate o f the path from Leadership to the System
im plies that an im provem ent o f one standard deviation in a hospitals Leadership score
w ill cause an im provem ent o f approximately .724 standard deviation in that h o sp ital's
System score. The path estim ate from the System to Organization Performance R esults is
.697 (p < .01), and the path estim ate from the System to Focus on and Satisfaction o f
Patients and O ther Stakeholders is .810 (p < .01).
statistically significant when evaluated with t-tests, resulting in the support o f H ypotheses
3 through 5 as reported in Table 6.2. These results are sim ilar to those found by W ilson
(1997) who studied the m anufacturing environm ent. The similarity to these previous
results indicates that the M BNQA causal model seem s to be applicable in a variety of
organizational types including m anufacturing and health care, supporting M B N Q A
efforts to develop a truly universal model o f q u ality m anagem ent and organizational
performance.
The variance o f the error terms for the endogenous variables in the model shown
in Figure 6.1 (System , Organization Performance Results, Focus on and Satisfaction of
Patients and O ther Stakeholders) are used to determ ine the proportion o f variance in the
endogenous variables that is explained by the m odel.
System error term indicates that approximately 60 percent ( I - .396) of the variance in the
System is accounted for by the model. Likewise, the model accounts for 45 percent of
the variance in C ategory 6.0 and 60 percent o f the variance in Category 7.0.
These
percentages are im portant because they reveal that the aggregate causal model used here
to test the B aldrige theory that Leadership drives the System which causes R esults
explains a large proportion of the variance in each o f the endogenous variables in the
model.
The conclusion from the aggregate causal hypotheses tests is that the aggregate
causal influences im plied by the pilot criteria o f the M BNQA in H ealth C are are
188
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
empirically supported in U.S. hospitals. The Leadership constructs influence shows that
senior m anagem ent is a driving force in the implem entation of TQM in hospitals. Senior
management guides the development of systems and processes within hospitals including
information system s, strategic planning, human resources, and medical and nonm edical
processes.
The results obtained here show that hospital system s (inform ation, strategic,
human resource, and process) influence both organizational performance (e.g., medical
and efficiency outcom es) and custom er satisfaction (patients and other stakeholders).
Evidence of these linkages between what hospitals do on a daily basis (systems) and their
outcom es (results) is im portant for hospitals as they seek to ju stify th eir TQ M
implementation efforts.
The associations identified here are important to researchers and the developm ent
of the Baldrige Health Care criteria because they provide empirical support for validation
of the basic prem ises o f the Baldrige theory and framework. The fram ew ork includes
Leadership (1.0) as the driver of System (2.0, 3.0, 4.0, 5.0), and the System as creating
Results in both outcom e categories (6.0, 7.0).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
and 23 shows that the influence o f Hum an Resource D evelopm ent and M anagement
(4.0), Process Management (5.0), and Organizational Performance Results (6.0) on Focus
on and Satisfaction of Patients and O ther Stakeholders (7.0) are significant. Figure 6.2
shows the results of path estimation for the specific causal model.
M odel estimation of the specific causal structural model shown in Figure 3.5 is
used to test Hypotheses 6 through 23. The input for model estim ation is a correlation
matrix o f the 28 Baldrige dimension scores. Dimension score developm ent is described
in Section 5.1.1.
The estim ated m odel has reasonable fit with RM SEA = .086, as
Reproduced with permission o f the copyright owner. Further reproduction prohibited without permission.
im plem entation (e.g., Flynn et al., 1994; Batalden and Stoltz, 1993).
The analysis
reported here supports validation o f the causal and directional influence o f leadership on
inform ation systems, strategic planning, human resources, and process m anagem ent The
results o f this study provide em pirical validation of the role o f leadership as the critical
driver in the Baldrige model.
L eadership (1.0) also has a positive causal influence on O rganizational
Perform ance R esults (6.0). This causal influence may be due in part to Leaderships
seem ingly prom inent role in a h o sp itals quality m anagem ent system .
The role of
Leadership in the quality m anagem ent systems o f hospitals may be not only direct, as
indicated by the significant path from Leadership (1.0) to O rganizational Performance
R esults (6.0), but Leadership also likely has an indirect influence through the System
categories o f the Baldrige model.
T hese results provide em p irical validation o f the critical role o f hospital
adm inistrators as the drivers of quality m anagem ent initiatives. H ospital administrators
should invest their efforts into creating a foundation for a quality-driven culture in order
to m axim ize the benefits of their quality initiatives.
construct described above, Information and Analysis has no direct influence on Focus on
and Satisfaction of Patients and O ther Stakeholders (7.0).
191
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Human Resource Developm ent and M anagem ent (4.0) has a direct significant
influence on custom er satisfaction (7.0). As described in Section 1.2.1, patients, patient
families, and other hospital custom er groups may find it difficult to evaluate clinical
(technical) quality, and they can more readily evaluate service (process) quality. Because
o f this, their interaction w ith hospital staff likely dom inates their judgment o f their level
of satisfaction. Baldrige Category 4.0 advocates that hospital staff should be effectively
treated as though they are internal customers so that job design and training result in
greater satisfaction o f the staff. This, in turn, leads to improved quality in interactions
with external custom ers (patients, patient families, etc.) whose own satisfaction will be
enhanced. The circular effect of customer and em ployee satisfaction is described by
Schlesinger and H eskett (1994) in their service-profit chain which they develop by
analyzing successful service organizations.
services and policies that improve employee satisfaction (and thus, employee loyalty and
retention) motivate em ployees to improve service quality and service value, resulting in
increased custom er satisfaction. The theory of the service-profit chain is support by the
results of this study in w hich managing a hospital's human resources with a focus on
improving employee satisfacuon is found to significantly influence customer satisfaction.
Like human resources, process management is often visible to patients and other
custom er groups in the design and delivery of m edical and non-medical services. The
posidve causal influence of Process Management (5.0) on custom er satisfaction (7.0) that
is identified here gives validation to designing hospital processes from a p atie n t's
perspecdve. Padent focus is one of the core values defined by the Baldrige Health Care
criteria (NIST, 1995b). The Baldrige dimensions o f process management include the
design and delivery of padent health care services, support services, com m unity health
services, and the m anagem ent of administrative and business operations and suppliers.
The complex set of processes addressed in this category are both visible (e.g., health care
193
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
and services, com m unity services) and hidden (e.g., administrative management, supplier
m anagem ent) from patients and other custom er groups.
designing and m anaging all processes for efficiency and effectiveness with a focus on
patients.
Finally, Organizational Performance Results (6.0) positively influences custom er
satisfaction (7.0). This link supports Baldrige theory (as well as the general theory of
TQM) that improving an organizations internal performance, as measured by C ategory
6 .0 ,
performance, as m easured by Category 7.0. Category 6.0 includes the more objective
measures of a hospitals perform ance such as medical outcom es, costs, and efficiency
measures while Category 7.0 accounts for perceptive measures of performance such as
customer satisfaction and the hospitals ability to gather and use market information.
These results support sim ilar research in other industries where internal and
external perform ance have been shown to be interlinked.
interlinking internal and external performance measures for credit card processing, and
Collier and W ilson (1997) show how internal and external performance are linked fo r a
telephone repair service.
Performance Results (6.0) to custom er satisfaction (7.0) that is identified in this study is
evidence of a sim ilar interlinking o f internal and external performance m easures for
hospitals. These results provide impetus for hospitals to focus on improving hum an
resources and process m anagem ent, both of which have direct causal influence on
customer satisfaction, and to strive for improved internal performance outcomes that also
help to create improved custom er satisfaction.
Overall, many of the specific causal influences in the Baldrige Health Care m odel
are em pirically validated, and a few are not. However, more empirical studies o f this
194
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
type should be com pleted before drawing final conclusions o f the importance of each of
the Baldrige constructs in hospitals.
Strategic Planning (3.0) is the only B aldrige category that does not have a
statistically significant direct causal influence on at least one of the two R esults
categories (6.0, 7.0). W hile this does not negate the presence of this construct in the
Baldrige Health C are m odel, it may be indicative o f the difficulty some hospitals have in
developing and deploying strategic plans.
hospitals' m ission statem ents receive little or no attention in strategic planning, and
hospitals are generally uncertain about what should be included in mission statem ents.
Further, Calem and Rizzo (1995) report that hospitals tend to pursue 'middle of the road'
strategies that neither carry the risks nor offer the rew ards o f more specific strategies.
The current study supports these previous findings that strategic planning often plays a
relatively weak role in many hospitals.
The results obtained here provide em p irical validation for m any o f the
relationships im plied by the MBNQA in Health C are model. Each of the categories of
the Baldrige H ealth C are model is found to be associated with the other com ponents
through causal and directional relationships. W ilson (1997) and W ilson and C ollier
(1998a), w ho evaluate the original Baldrige A w ard m odel in the m anufacturing
environment, present results with some differences from those reported here. They find
that Inform ation and A nalysis (2.0) and Process M anagem ent (5.0) have significant
causal influences on both of the results categories (6.0, 7.0). The study reported here
finds that Inform ation and Analysis (2.0) has a significant influence on C ategory 6.0
while Process M anagem ent (5.0) has a significant influence on Category 7.0. O ther
differences include the direct influence o f L ead ersh ip (1.0) on O rg an izatio n al
Performance Results (6.0), that is validated in hospitals but not in manufacturing, and the
195
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1.
performance of the Baldrige-defined System (2.0, 3.0, 4.0, 5.0) in U.S. hospitals (r = .75,
p < .01). Additionally, hospital Systems are significantly associated with perform ance as
196
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
measured by O rganizational Performance R esults (6.0) (r = .57, p < .01) and Focus on
and Satisfaction o f Patients and Other Stakeholders (7.0) (r = .69, p < .01).
2.
Leadership (1.0) has a positive causal influence on the System (2.0, 3.0, 4.0, 5.0)
in U.S. hospitals (y = .724, p < .01). The System has a positive causal influence on
O rganizational Perform ance Results (6.0) ((3 = .697, p < .01) and Focus on and
Satisfaction of Patients and Other Stakeholders (7.0) (|3 = .810, p < .01). The Baldrige
theory that Leadership drives the System which causes Results is supported based on the
data set and statistical analyses used here.
3.
H ospital Leadership (1.0) has a direct causal influence on Inform ation and
Analysis (2.0) (y
= .775, p < .01), Strategic Planning (3.0) (y = .470, p < .01), Human
Resource Developm ent and Management (4.0) (y = .381, p < .01), Process M anagement
(5.0) (y = .364, p < .01), and Organizational Performance Results (6.0) (y = .429, p <
.05).
Organizational Perform ance Results (6.0) and Focus on and Satisfaction o f Patients and
O ther Stakeholders (7.0).
influences perform ance in each of these other categories in the Baldrige Health Care
model.
4.
Inform ation system s are highly influenced by hospital leadership. The causal
influence of Leadership (1.0) on Information and Analysis (2.0) ((3 = .775) is about twice
as strong as Leadership's influence on the other System categories ((3 = .470, .381, and
.364 for C ategories 3.0, 4.0, and 5.0, respectively).
hospital adm inistrators recognize the critical role o f information systems in hospitals in
providing systems of measurement, information, and data analysis.
197
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
5.
positively influences Strategic Planning (3.0) ((3 = 460, p < .01), Human Resource
Development and M anagement (4.0) ((3 = .547, p < .01), and Process Management (5.0)
(P = .610, p < .01). These relationships identify Information and Analysis as influential
in a variety of other areas o f hospital management and clarify the direction and strength
o f causation within the Baldrige System. These results confirm those o f Wilson (1997)
and Wilson and Collier (1998a).
6.
Perform ance Results (6.0) in hospitals (p = .522, p < .05). This positive significant
relationship indicates that effective use of measurement, inform ation, and data are key
assets in the performance of U.S. hospitals, particularly their internal systems and results
measures.
7.
Human Resource D evelopm ent and M anagem ent (4.0) has a positive causal
8.
Process M anagem ent (5.0) has a positive causal influence on Focus on and
Satisfaction of Patients and O ther Stakeholders (7.0) in U.S. hospitals (P = .772, p < .01).
198
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
These results indicate that the design and delivery o f medical and non-medical processes
is critical to custom er satisfaction and should be managed from the patient's perspective.
9.
on and Satisfaction o f Patients and Other Stakeholders (7.0) ((3 = .301, p < .01). This
significant relationship supports Baldrige theory that improving internal perform ance
(Category 6.0) can lead to improved external performance (Category 7.0).
10.
Overall, the results of this study provide the first comprehensive evaluation o f the
theory and perform ance relationships proposed by the Baldrige Health Care m odel and
criteria (N IST, 1995b). The m easurement m ethods used here to obtain data in U.S.
hospitals are both reliable and valid. These results provide support for m uch o f the
Baldrige theory of quality management in the health care environment.
11.
A com prehensive measurement model for the MBNQA in Health Care criteria has
been developed, tested, and evaluated in this study. The questionnaire developed here is
valid and reliable for measuring the 28 dim ensions of the MBNQA in Health C are model
in U.S. hospitals (NIST, 1995b).
199
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Some possible extensions o f this study are briefly described here. First, because
the data used in the analyses reported here are based on hospital quality m anagers
perceptions, a logical extension o f this study is to test the research hypotheses based on
either researchers perceptions o f hospital performance or more objective measures of the
Baldrige constructs. Carman et al. (1996) report that researchers perceptions of hospital
perform ance on the original B aldrige criteria are highly co rrelated w ith hospital
managers perceptions. Regardless, if the same or similar results to those presented here
are achieved using perform ance-related data from secondary sources, it would enhance
the credibility and reliability of the results o f this study.
Second, this study should be repeated in another sample o f hospitals and in health
care organizations other than general acute care hospitals. This w ould reveal whether the
results of this study are unique to eith er the sample studied here or to the hospital
environment. The causal relationships among the Baldrige constructs that are identified
here (see F igures 6.1 and 6.2) may not be consistent across all types o f health care
organizations (i.e., hospitals, nursing hom es, clinics, health m aintenance organizations,
etc.).
The relationships between the construct Organizational Perform ance Results (6.0)
and the six other constructs in the Baldrige model (1.0, 2.0, 3.0, 4.0, 5.0, 7.0) should be
further evaluated.
The data used in this study indicate that Leadership (1.0) and
Information and Analysis (2.0) have positive causal influences on Category 6.0. Baldrige
theory also proposes that Strategic Planning (3.0), Human Resource D evelopment and
M anagement (4.0), and Process M anagem ent (5.0) influence Organizational Performance
Results (6.0).
Future analysis should include determ ining which of the dim ensions of Category
6.0 are m ost influenced by the other constructs in the Baldrige m odel. Category 6.0 is
conceptually broad, and its intent is to m easure all facets o f internal hospital
200
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
performance. Its domain includes patient health care results, support service results,
com m unity health service resu lts, adm inistrative resu lts, su p p lier results, and
accreditation results. For this study, the five dimensions that make up Category 6.0 are
measured using 20 items (from the questionnaire shown in A ppendix A). The domain of
Category 6.0 is so broad that it is difficult to measure each of its com ponents adequately
within the space constraints o f a m ailed questionnaire.
Category 6.0 may be influenced by Baldrige System categories in relationships that are
not identified in this study.
Assessment Results separately from the other dimensions o f C ategory 6.0 may identify
other Baldrige dimensions that influence it such as those associated with staff licensure
and process management. The current compilation o f ail organizational performance
results into a single construct may confound and negate identifying significant causal
influences.
The Baldrige health care criteria have become well know n and widely utilized by
hospitals for self-assessment purposes. This study provides evidence that many of the
associative relationships and causal influences im plied by the Baldrige Health Care
model are observed in U.S. hospitals.
201
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Hypothesis
Correlation
tested
Correlation
result
Hypothesis
supported
Leadership 1<->System 2
.75**
Yes
System<->Performance 3
.57**
Yes
System<->Customer 4
.69**
Yes
202
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Hypothesis
Causal path
Path
estimate
Hypothesis
supported
Leadership->System 2
.724**
Yes
System->Performance 3
.697**
Yes
S y ste m -^ u sto m e r 4
.810**
Yes
203
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Causal path
Path
estimate
Hypothesis
supported
Leadership 1->Information 2
.775**
Yes
Leadership->Strategy 3
.470**
Yes
Leadership->Human Resources 4
.381**
Yes
Leadership->Process 5
.364**
Yes
10
Leadership->Perform ance 6
.429*
Yes
11
Leadership->Customer 7
12
Information->Strategy
.460**
Yes
13
Information->Human Resources
.547**
Yes
14
Information->Process
.610**
Yes
15
Information->Performance
.522*
Yes
16
Information->Customer
-.347
No
17
Strategy->Performance
-.154
No
18
Strategy->Customer
.013
No
19
Human Resources->Performance
.132
No
20
Human Resources->Customer
.262*
Yes
21
Process->Performance
22
Process->Customer
772**
Yes
23
.301**
Yes
Hypothesis
Note:
-.055
-.216
No
No
204
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Leadership
- System
- Results
Process
Management
5.0
Leadership
to
ocn
Human Resource
Development and
Management
4.0
Strategic
Planning
3.0
Organizational
Performance Results
6.0
Information and
Analysis
2.0
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Process
Management
5.0
-.06
Leadership
-.22
Human Resource
Development and
Management
4.0
Strategic
Planning
3.0
Organizational
Performance Results
-.15
6.0
-.35
Information and
Analysis
II,s: .52*
2.0
Note:
BIBLIOGRAPHY
AI-Assaf, A.F. and Schm ele, J.A. (1993), The Textbook o f Total Quality in Healthcare,
St. Lucie Press, D elray Beach, FL.
Alexander, J.A. and M orlock, L.L. (1985), M ulti-institutional arrangements:
Relationships between governing boards and hospital ch ief executive officers, Health
Services Research, Vol. 19 No. 6 Part 1, pp. 675-699.
A H A (American Hospital A ssociation) (1997), The AHA Guide to the Health Care
Field, AHA, Chicago, IL.
Al-Haider, A.S. and Wan, T.T.H . (1991), "Modeling organizauonal determinants to
hospital mortality," Health Service Research, Vol. 26 No. 3, pp. 303-323.
Anderson, H.J. (1991), "Sizing up systems: Researchers to test performance measures,"
H ospitals, Vol. 65 No. 20, pp. 33-34.
Anderson, K. and W ootton, B. (1991), "Changes in hospital staffing patterns," M onthly
Labor Review, Vol. 114 No. 3, pp. 3-9.
Andrews, K.R. (1980), The Concept o f Corporate Strategy, Richard D. Irwin,
Homewood, IL.
Anschutz, E.E. (1995), "A point o f view: The public sector challenge," National
Productivity Review, Vol. 15 No. 1, pp. 1-6.
Appleby, J. (1995), "Creating healthier communities: 25 pioneering models,"
Healthcare Forum, Vol. 38 No. 3, pp.43-54.
Ashmos, D.P., Duchon, D., Hauge, F.E., and McDaniel, R.R. (1996), "The nature of
internal complexity in unusual organizations: A study o f environm entally sensitive and
insensitive hospitals," A cadem y o f M anagement Best Paper Proceedings.
Babakus, E. and M angold, W .C. (1992), "Adapting the SERV Q U A L scale to hospital
services: An em pirical investigation," Health Services Research, Vol. 26 No. 6 , pp.
767-786.
Bailey, S. (1998), Statem ent to Subcommittee on Health o f the House Ways and Means
Committee in W ashington, D.C., M arch 12, 1998. Reprinted in On Q, Vol. 8 No. 3, p.
17.
207
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Baird, J. (1981), "Quality circles may substantially improve hospital employees' morale,"
M odem Healthcare, Vol. 11 No. 10, pp. 70-74.
Batalden, P.B. and Stoltz, P.K. (1993), "A framework for the continual improvement o f
health care: Building and applying professional and im provem ent knowledge to test
changes in daily work," The Joint Commission Journal on Quality Improvement, Vol.
19 No. 10, pp. 424-447.
Bergman, R. (1994), "Leadership survey shows reform forces drive computerization,"
Hospitals and Health N etworks, Vol. 18 No. 6 , p. 116.
Berwick, D.M., Godfrey, A.B., and Roessner, J. (1990), Curing Health Care, JosseyBass Inc., San Francisco, CA.
Bigelow, B. and Arndt, M. (1995), "Total Quality M anagement: Field of dreams?"
Health Care M anagem ent Review, Vol. 20 No. 4, pp. 15-25.
Blackington, S.M. and M cLauchlan, T. (1995), "Continuous Quality Improvement in the
neonatal intensive care unit: Evaluating parent satisfaction," Journal o f Nursing Care
Quality, Vol. 9 No. 4, pp. 78-85.
Blane, D. (1991), "Beyong customer-supplier partnerships: A vision o f the future,"
Hospitals M aterial M anagem ent Quarterly, Vol. 12 No. 4, pp. 76-79.
Bowers, M.R., Swan, J.E., and Koehler, W.F. (1994), "W hat attributes determine quality
and satisfaction with health care delivery?" Health Care M anagem ent Review, Vol. 19
No. 4, pp. 49-55.
Browne, M.W. (1974), "Generalised least squares estim ators in the analysis of
covariance structures," South African Statistical Journal, Vol. 8 No. 1, pp. 1-24.
Browne, M.W. and Cudeck, R. (1993) Alternative ways o f assessing model fit, in
Testing Structural Equation Models, eds. K.A. Bollen and J.S. Long, SAGE Publications,
Newbury Park, CA.
Browne, M.W. and Mels, G. (1994), RAMONA User's G uide, The Ohio State University
Dept, of Psychology, Columbus, OH.
Burda, D. and W eissenstein, E. (1997), "Hospital closings up for first time since 1988,"
M odem Healthcare, Vol. 26 No. 48, pp. 2-3.
Butler, T.W ., Leong, G.K ., and Everett, L.N. (1996), "The operations management role
in hospital strategic planning," Journal o f Operations M anagement, Vol. 14 No. 2, pp.
137-156.
Calem, P.S. and Rizzo, J.A. (1995), "Competition and specialization in the hospital
industry: An application of Hotelling's location model," Southern Economic Journal,
Vol. 61 No. 4, pp. 1182-1198.
208
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Carman, J.M., Shortell, S.M., Foster, R.W., Hughes, E.F.X., Boerstler, H., OBrien, J.L.,
and O'Connor, E.J. (1996), "Keys for successful implementation of Total Quality
Management in hospitals," Health Care M anagem ent Review, Vol. 21 No. 1, pp. 48-60.
Carmines, E.G. and M clver, J. (1981), "Analyzing models with unobserved variables:
Analysis of covariance structures," in Social M easurement: Current Issues, eds. G.
Bohmstedt and E. Borgatta, SAGE Publications, Beverly Hills, CA.
Carmines, E.G. and Zeller, R.A. (1979), Reliability and Validity Assessment, Sage
Publications, Beverly Hill, CA.
Chase, R.B. and Tansik, D.A. (1983), "The custom er contact model for organization
design," M anagem ent Science, Vol. 29 No. 9, pp. 1037-1050.
Christensen, B.V., Iacarella, C.L., Meyer, S.M., M anion, R.V., Bruhn-Ding, B.J., and
Wilson, R.F. (1998), "Sandbags do not prevent vascular complications after
angiography," Nursing Research, Vol. 47 No. 1, pp. 51-53.
Collier, D.A. (1991), "A service quality process map for credit card processing,"
Decision Sciences, Vol. 22 No. 2, pp. 406-419.
Collier, D.A. (1994), The Service/Quality Solution, Irwin Professional Publishing, Buss
Ridge, IL, and American Society of Quality Control's Quality Press, Milwaukee, WI.
Collier, D.A. and W ilson, D.D. (1997), "The role o f automation and labor in determining
customer satisfaction in a telephone repair service process," Decision Sciences, Vol. 28
No. 3, pp. 689-708.
Collins, L.W. (1994), "TQM information systems: An elusive goal," Joint Commission
Journal on Quality Improvement, Vol. 20 No. 11, pp. 607-613.
Counte, M.A., Glandon, G.L., Oleske, D.M., and Hill, J.P. (1992), "Total Quality
Management in health care organizations: An analysis of employee impacts," H ospital
and Health Service Administration, Vol. 37 No. 4, pp. 503-518.
Counts, C.S. and Prowant, B.F. (1994), "Disaster preparedness: Is your unit really
ready?" Anna Journal, Vol. 21 No. 4, pp. 155-161, 201.
Dalgo, J.T. and A nderson, F. (1995), "Developing a hospital ethics committee," Nursing
Management, Vol. 26 No. 9, pp. 104-106.
Deloitte and Touche (1993), The Comparative Performance o f U.S. Hospitals: The
Sourcebook, HCIA, Chicago, IL.
DesHamais, S., Chesney, J.D., Wroblewski, R.T., Fleming, S.T., and McMahon, L.F.
(1988), "The risk-adjusted mortality index: A new measure o f hospital performance,"
Medical Care, Vol. 26 No. 12, pp. 1129-1148.
DesHamais, S., M cM ahon, L.F., and W roblewski, R. (1991), "Measuring outcomes o f
hospital care using m ultiple risk-adjusted indexes," Health Services Research, Vol. 26
No. 4, pp.425-445.
209
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
DesHamais, S., M arshall, B and J. Dulski, J. (1994), "Information m anagem ent in the
age of m anaged competition," Joint Commission Journal on Quality Im provem ent., Vol.
20 No. 11, pp. 631-638.
Donabedian, A. (1980), "The definition o f quality and approaches to its management," in
Explorations in Quality Assessment and M onitoring, Health A dm inistration Press, Ann
Arbor, MI: Vol. 1, pp.95-99.
Donabedian, A., "M odels of quality assurance, Lenard S. Rosenfeld M em orial Lecture,
School o f Public Health, University of North Carolina at Chapel H ill, N C, February 26,
1993.
Eagle, C.J., Davies, J.M ., and D. Pagenkopf, D. (1994), "The cost o f established quality
assurance programme: Is it worth it?," Canadian Journal o f Anaesthesia, Vol. 41 No. 9,
pp. 813-817.
Easton, G.S. and Jarrell, S.L. (1998), "The effects of Total Quality M anagem ent on
corporate perform ance: An empirical investigation," Journal o f Business, Vol. 71 No.
2, pp. 253-307.
Eckholm, E. (1995), "How capitation is transforming hospitals - and their staffs."
Medical Econom ics, Vol. 72 No. 10, pp. 45-55.
Ferret, A. (1996), "Patient FIRST: A focussed initiative to redesign service for
tomorrow," Presentation, Midwest Decision Sciences Institute M eetings, South Bend,
IN, April 20.
Flood, A.B., Scott, W.R., Ewy, W., and Forrest, W.H. (1982), "Effectiveness in
professional organizations: The impact o f surgeons and surgical staff organizations on
the quality of care in hospitals," Health Services Research, Vol. 17 No. 4, pp. 341-366.
Flynn, B.B., Sakakibara, S., Schroeder, R.G., Bates, K.A., and Flynn, E.J. (1990),
"Empirical research methods in operations management," Journal o f O perations
Management, Vol. 9 No. 2, pp. 250-284.
Flynn, B.B., Schroeder, R.G., and S. Sakakibara, S. (1994), "A fram ew ork for quality
management research and an associated measurement instrument," Journal o f
Operations Management, Vol. 11 No. 4, pp. 339-366.
Fottler, M .D., Blair, J.D., Whitehead, C.J., Laus, M.D., and Savage, G.T. (1989),
"Assessing key stakeholders: Who matters to hospitals and why?" H ospital and Health
Services Administration, Vol. 34 No. 4, pp. 525-546.
Garvin, D.A. (1988), Managing Quality, Free Press, New York.
Gaynor, M. (1995), "Uncertain demand, the structure of hospital costs, and the cost of
empty hospital beds," Journal o f Health Economics, Vol. 14 No. 3, pp. 291-317.
210
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Gelinas, L.S. and M anthey, M. (1995), "Improving patient outcom es through system
change: A focus on the changing roles of healthcare organization executives," Journal
o f Nursing Adm inistration, Vol. 25 No. 5, pp. 55-63.
General Accounting Office (1991), "Management practices: U.S. companies improve
performance through quality efforts," Report to the Honorable Donald Ritter, House of
Representatives.
Ghiselli, E.E., Cam pbell, J.P., and Zedeck, S. (1981), M easurem ent Theory fo r the
Behavioral Sciences, W .H. Freeman and Company, New York.
Gibson, C.K., Newton, D.J., and Cochran, D.S. (1990), "An em pirical investigation of
the nature of hospital mission statements," Health Care M anagem ent Review, Vol. 15
No. 3, pp.35-45.
Goes, J.B. and Meyer, A.D. (1990), "Patterns of strategic change, environmental change,
and performance: A longitudinal study of California hospitals," Academ y o f
M anagement Best Paper Proceedings, pp. 85-89.
Goldberg, A.M. and Pegels, C.C. (1984), Quality Circles in H ealth Care Facilities,
Aspen Systems Corporation, Rockville, MD.
Goldberg, E Diamond, H.S., and Rongaus, A.M. (1995), "Total Q uality M anagement o f
a medical residency," Am erican Journal o f Medical Quality, Vol. 10 No. 1, pp. 10-13.
Gopalakrishnan, K.N. and M clntrye, B.E. (1992), "Hurdles to quality health care,"
Quality Progress, Vol. 25 No. 4, pp. 93-95.
Graham, N.O. (1995), Quality in Health Care: Theory, Application, a nd Evolution,
Aspen Publishers, G aithersburg, MD.
Graf, L. (1996), "Strategic analysis for resource allocation decisions in health care
organizations," Journal o f M anagerial Issues, Vol. 8 No. 1, pp. 92-107.
Green, M.J., Mitchell, G., Stocking, C.B., Cassel, D.K., and Siegler, M. (1996), "Do
actions reported by physicians in training conflict with consensus guidelines on ethics?"
Archives o f Internal M edicine, Vol. 156 No. 3, pp. 298-304.
Greene, J. (1995), "Tax-exem pts feeling the heat," M odem H ealthcare, Vol. 25 No. 47,
pp. 46-52.
Greene, J., (1996) "Florida 2nd state to issue 'report cards'," M odem Healthcare, Vol.
26 No. 10, p. 80.
Griest, D.L. and Belles, D.R. (1990), "Health care executives: A personality profile,"
Hospitals, Vol. 64 No. 3, pp.74-75.
Gustafson, D.H. and Jundt, A.S. (1995), "Findings of innovation research applied to
quality management principles for health care," Health Care M anagem ent Review, Vol.
20 No. 2, pp. 16-33.
211
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Hair, J.F., Anderson, R.E., Tatham, R.L., and Black, W.C. (1995), M ultivariate Data
Analysis, Fourth Edition, Prentice Hall, Upper Saddle River, NJ.
Handfield, R.B. and Ghosh, S. (1995), "An em pirical test of linkages between the
Baldrige criteria and financial performance," Proceedings o f Decision Sciences Instirure
Annual Meetings, p. 1713.
Harkey, J. and Vraciu, R.A. (1992), "Quality o f health care and financial perform ance:
Is there a link?" Health Care Management Review, Vol. 17 No. 4, pp. 55-64.
HCIA (Health Care Investment Analysts, Inc.) (1996), Profiles o f U.S. H ospitals, HCIA,
Baltimore, MD.
Health Care Strategic Management, (1994), "Hospitals increase satisfaction and reduce
costs by redesigning around patients," Vol. 12 No. 6 , p. 18.
Health Care Strategic Management, (1997), "HCIA releases list of top 100 hospitals,
updates performance benchmarks for 1997," Vol. 15, No. 1, pp. 6-7.
Healthcare Financial Managment, (1995), "From increasing revenues to controlling
costs: Benchmark data for strategic planning," Special Report Supplement, pp. 1-16.
Hemmes, M. (1991), "CEO support key to success o f health care recycling," H ospitals,
Vol. 65 No. 14, p. 60.
Hendricks, K.B. and Singhal, V.R. (1996), "Quality awards and the marekt value o f the
firm: An empirical investigation," M anagem ent Scinece, Vol. 42 No. 3, pp. 415-436.
Heskett, J.L. and Schlesinger, L.A. (1994), "Putting the service profit chain to w ork,
Harvard Business Review, Vol. 72 No. 2, pp. 164-174.
Honnas, R. and Zlotnick, C. (1995), "Quality improvement in action: D evelopm ent o f a
tool," Journal o f Nursing Care Quality, Vol. 9 No. 4, pp. 72-77.
Houston, K.L., Clute, C., Ryan-Crepin, K Kimball, M and Matthews, R. (1994),
"TQM brings collaboration to patient transport process," Journal o f Nursing Care
Quality, Vol. 9 No. 1, pp. 21-29.
Hutton, J.D. and Richardson, L.D. (1995), "Healthscapes: The role of the facility and
physical environment on consumer attitudes, satisfaction, quality assessments, and
behaviors," Health Care M anagement Review, Vol. 20 No. 2, pp. 48-61.
Jacono, J. (1990), "Comparison of perceived needs o f family members between RNs and
family members of critically ill patients in IC and neonatal ICUs," H eart and Lung, Vol.
19 No. 1, pp. 72-78.
Jaklevic, M.C. (1996a), "Marketers gear up for quality ratings," M odem H ealthcare,
Vol. 26 No. 14, p.49.
Jaklevic, M.C. (1996b), "Michigan association reports on hospital performance,"
M odem Healthcare, Vol. 26 No. 22, p.23.
212
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Jenkins, G.D. and Tabor, T.D. (1977), A monte carlo study o f factors affecting three
indices of composite scale reliability," Journal o f A pplied Psychology, Vol. 62 No. 4,
pp. 392-398.
Jennings, K. and W estfall, F. (1994), "A survey-based benchm arking approach for health
care using the Baldrige quality criteria," The Joint Commission Journal on Quality
Improvement, Vol. 20 No. 9, pp. 500-509.
Joint Commission Journal on Quality Improvement, (1995), "Using patient input in a
cycle for performance improvement," Vol. 21 No. 2, pp. 87-96.
Joint Commission on Accreditation of Healthcare Organizations, (1993), Implementing
Quality Improvement: A Hospital Leader's Guide, O akbrook Terrace, IL.
Joreskog, K.G. (1969), "A general approach to confirmatory maximum likelihood factor
analysis," Psychometrika, Vol. 34 No. 2 Part 1, pp. 183-202.
Joss, R. and Kogan, M. (1995), Advancing Quality: Total Quality M anagement in the
Nation Health Service, Open University Press, Bristol, PA.
Judge, W.Q. and Zeitham l, C.P. (1992), "An empirical com parison between the boards
strategic role in nonprofit hospitals and in for-profit industrial firms," Health Services
Research, Vol. 27 No. 1, pp. 47-64.
Kalayjian, A.S. (1994), "Mental health outreach program following the earthquake in
Armenia: Utilizing the nursing process in developing and managing the post-natural
disaster plan," Issues in M ental Health Nursing, Vol. 15 No. 6 , pp. 533-50.
Ketchen, D.J., Thom as, J.B., and Snow, C.C. (1993), "Organizational configurations and
performance: A comparison o f theoretical approaches" Academ y o f Management
Journal, Vol. 36 No. 6 , pp. 1278-1313.
Kluge, E.H. (1996), "The roles and functions of hospital-based ethics committees,"
Canadian Medical Association Journal, Vol. 154 No. 7, pp. 1094-1095.
Knotts, U.S. (1993), "What does the U.S. business com munity really think about the
Baldrige Award," Quality Progress, Vol. 26 No. 5, pp. 49-53.
Lachance, K.R., Santos, A.B., and Bums, B.J. (1994), "The response of an assertive
community treatment program following a natural disaster," Community M ental Health
Journal, Vol. 30 No. 5, pp. 505-515.
Lamont, B.T. (1993), "Porter's generic strategies, discontinuous environments, and
performance: A longitudinal study of changing strategies in the hospital industry,"
Health Services Research, Vol. 28 No. 5, pp. 623-640.
Lathrop, J.P. (1991), The patient-focused hospital, Healthcare Forum, Vol. 43 No. 4,
pp. 16-21.
213
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Licata, J.W. (1995), "Diagnosing perceived quality in the medical service channel,"
Journal o f Health Care M arketing, Vol. 15 No. 4, pp. 42-49.
Lichtman, D.M. and Appleman, K.A. (1995), "Measures of effectiveness: A
methodology of integrating planning, measurement, and continuous improvem ent,"
Military Medicine, Vol. 160 No. 4, pp. 189-193.
Lissitz, R.W. and G reen, S.B. (1975), "Effect o f the number of scale points on reliability:
A monte carlo approach," Journal o f A pplied Psychology, Vol. 60 No. 1, pp. 10-13.
Louden, T.L. (1991), "M arketing takes m ore than ju st selling product," H ealth Industry
Today, Vol. 54 No. 11, p. 16.
Luft, H. and Hunt, S. (1986), "Evaluating individual hospitals quality through outcome
statistics," JAMA, Vol. 255 No. 20, pp. 2780-2784.
Manthey, M. (1991), "Staffing and productivity," Nursing Management, Vol. 22 No.
1 2 , pp. 2 0 .
Marszalek-Gaucher, E. (1993), Total Quality in Healthcare: From Theory to Practice,
Jossey-Bass Publishers, San Francisco, CA.
Matell, M.S. and Jacoby, J. (1971), "Is there an optimal number o f alternatives for Likert
scale items? Study I: Reliability and validity," Educational and Psychological
Measurement, Vol. 31 No. 3, pp. 657-674.
Matell, M.S. and Jacoby, J. (1972), "Is there an optimal number of alternatives for Likert
scale items? Effects o f testing time and scale properties," Journal o f A pplied
Psychology, Vol. 56, pp. 506-509.
McDermott, R.E. (1994), "The role o f com munity hospitals in the perform ance o f local
economies," Journal o f Health Care Finance, Vol. 21 No. 2, pp.55-65.
McLaughlin, C.P. and Kaluzny, A.D. (1994), Continuous Quality Im provem ent: Theory,
Implementation and Applications, Aspen Publishers, Gaithersburg, MD.
Meyer, S.M. and Collier, D.A. (1998), "Contrasting the original M alcolm Baldrige
National Quality Award with the new Health Care Award." Quality M anagem ent in
Health Care, Vol. 6 No. 3.
Miles, R.E. and Snow, C.C. (1978), Organizational Strategy, Structure, a n d Process,
McGraw-Hill, New York.
Montague, J. (1995), "Customer service: M aking hospitals more hospitable," H ospital
and Health Networks, Vol. 69 No. 18, p.64.
Morrisey, J. (1994), "Top 100 hospitals set new success standards," M o d em Healthcare,
Vol. 24 No. 46, pp. 74-76.
Motwani, J., Sower, V.E., and Brashier, L.W. (1996), "Implementing TQ M in the health
care sector," Health Care Management Review, Vol. 21, No. 1, pp. 73-82.
214
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Murray, M., Stockard, R.R., Blaylock, B Stanfa, D.K., and McKee, M. (1994),
"Product evaluation and process improvement," Journal o f Nursing Care Q uality, Vol. 9
No. 1, pp. 16-20.
Nath, D., and Sudharshan, D. (1994), "M easuring strategy coherence through patterns of
strategic choices," Strategic Management Journal, Vol. 15 No. I, pp. 43-51.
National Institute o f Standards and Technology, (1995a), Malcolm Baldrige National
Quality Award: 1995 Award Criteria. G aithersburg, MD.
National Institute o f Standards and Technology, (1995b), Malcolm Baldrige National
Quality Award: 1995 Health Care Pilot Criteria. Gaithersburg, MD.
National Institute o f Standards and Technology, (1995c), Malcolm Baldrige National
Quality Award: 1995 Education Pilot Criteria. G aithersburg, MD.
Nelson, E.C., H ays, R.D., Larson, C., and Batalden, P.B. (1989), "The patient judgm ent
system: Reliability and validity," Quality Review Bulletin, Vol. 15 No. 6 , pp. 185-191.
Nelson, E.C., Larson, C.O., Hayes, R.D., N elson, S.A., W ard, D., and Batalden, P.B.
(1992), "The physician and employee judgm ent system: Reliability and validity o f a
hospital quality measurement method," Quality Review Bulletin, Vol. 18 No. 9, pp. 284292.
Nunnally, J.C. (1978), Psychometric Theory, M cGraw-Hill, New York.
Olden, P.C. and Luke, R.D.(1996), "Revolutionary change in health care m arkets:
Covariates o f local hospital clustering in three time periods," Presention, A cadem y of
M anagement A nnual Meetings, Cincinnati, OH, Aug. 12.
Pallarito, K. (1996), "Not-for-profit system a model for fiscal fitness," M odern
Healthcare, Vol. 26 No. 2, pp.48, 51.
Parasuraman, A., Zeithaml, V.A., and Berry, L.L. (1988). "SERVQUAL: A multiple
item scale for measuring customer perceptions o f service quality," Journal o f Retailing,
Vol. 64 No. 1, pp. 12-40.
Pegels, C.C. and Sekar, C. (1989), "Determining strategic groups using m ultidim ensional
scaling," Interfaces, Vol. 19 No. 3, pp. 47-57.
Prince, T.R. (1994), "Bond ratings, debt insurance, and hospital operating perform ance,"
Health Care M anagem ent Review, Vol. 19 No. 1, pp. 64-73.
Sanders, N.R. (1997), "Health care organizations can learn from the experiences of
others," Q uality Progress, Vol. 30 No. 2, pp. 47-49.
Schlesinger, L.A. and Heskett, J.L. (1991), "Breaking the cycle of failure in services,"
Sloan M anagem ent Review , Vol. 32 No. 3, pp. 17-28.
215
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Schlesinger, L.A. and Zomitsky, J. (1991), "Job satisfaction, service capability, and
customer satisfaction: An exam ination of linkages and management implications,"
Human Resource Planning, Vol. 14 No. 2, pp. 141-149.
Schweikhart, S.B. and Strasser, S. (1993), "Service recovery in health service
organizations," H ospital and Health Services Administration, Vol. 38 No. I, pp.3-23.
Shortell, S.M., O 'Brien, J.L., Carman, J.M., Foster, R.W., Hughes, E.F.X., Boerstler, H
and O'Connor, E.J. (1995), "Assessing the im pact of continuous quality improvement/
Total Quality M anagement: Concept versus implementation, Health Services Research,
Vol. 30 No. 2, pp. 377-401.
Shortell, S.M. and Zajac, E.J. (1990), Perceptual and archival measures o f M iles and
Snows strategic types: A comprehensive assessment of reliability and validity."
Academy o f M anagem ent Journal, Vol. 33 No. 4, pp. 817-832.
Skibicki, R. (1994), "Educating executives to integrate information m anagem ent with
quality improvement: A case study," Joint Commission Journal on Quality
Improvement, Vol. 20 No. 11 pp. 623-630.
Sobczak, C., Chaillet, J.R., Bradford, D., and Makleff, R. (1991), "Quality measurement
and management in an HMO setting," Topics in Health Care Financing, Vol. 18, No. 2,
pp.67-74.
Solovy, A. (1995), "Predicting the unpredictable," Hospital and Health Networks, Vol.
69 No. 1, pp. 26-29.
Standard and Poor Corporation, (1995), Standard and Poor's Industry Survevs, New
York, NY.
Steiger, J.H. and Lind, J. (1980), Statistically based tests for the num ber o f common
factors," Presentation, Psychometric Society annual meeting, Iowa City, LA, May.
Suen, H.K. (1990), Principles o f Test Theories, Lawrence Erlbaum Associates, Inc.
Hillsdale, NJ.
Systat 6.0 for DOS, copyright SPSS Inc. 1995.
Tenner, A.R. and DeToro, I.J. (1992), Total Quality Management: Three Steps to
Continuous Im provem ent, Addison-Wesley, Reading, MA.
Tidikis, F. and Strasen, L. (1994), "Patient-focused care units improve service and
financial outcomes," Healthcare Financial Management, Vol. 48 No. 9, pp. 38-44.
Vandamme, R. and Leunis, J. (1993), "Development of a multiple-item scale for
measuring hospitals service quality," International Journal o f Service Industry
Management, Vol. 4 No. 3, pp. 30-49.
Washawsky, M.J. (1994), "Projections of health care expenditures as a share o f the GDP:
Actuarial and m acroeconomic approaches," Health Services Research, Vol. 29, No. 3,
PP. 293-313.
216
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
W illiams, N., Sobti, A., and Aw, T.C. (1994), "Comparison o f perceived occupational
health needs among managers, em ployee representatives and occupational physicians,"
Occupational Medicine, Vol. 44 No. 4 , pp. 205-208.
Wilson, D.D. (1997), "An empirical study to test the causal linkages implied in the
Malcolm Baldrige National Quality Award," Unpublished dissertation.
Wilson, D.D. and Collier, D.A. (1998a), "An empirical investigation o f the Malcolm
Baldrige National Quality Award causal model," working paper, Florida State
University.
Wilson, D.D. and Collier, D.A. (1998b), "The Malcolm Baldrige National Quality
Award weighting scheme and firm performance," working paper, Florida State
University.
W oodhandler, S., Himmelstein, D.U., and Lewontin,, J.P. (1993) "Adm inistrative costs
in U.S. hospitals," New England Journal o f Medicine, Vol. 329 No. 6 , pp.400-403.
Zajac, E.J. (1989), "Changing generic strategies: Likelihood, direction, and performance
implications," Strategic M anagement Journal, Vol. 10 No. 5, pp. 413-430.
Zeithaml, V.A. (1981), Marketing in Services. American M arketing Association,
Chicago, IL.
Zelman, W.N., and Parham, D.L. (1990), "Strategic, operational, and marketing concerns
of product-line management in health care," Health Care M anagem ent Review, Vol. 15
No. 1, pp. 29-35.
Zmud, R.W. and McLaughlin, C.P. (1989), "That's not my job: M anaging secondary
tasks effectively," Sloan M anagem ent Review, Vol. 30 No. 2, pp. 29-36.
217
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
APPENDIX A
218
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Instructions
P le a s e in d ic a te y o u r o p in io n on all q u e s tio n s b y c ir c lin g a n u m b er or f illin g in the blan k a s in d ica te d . T h ere
are n o righ t o r w r o n g a n s w e r s . Y o u r r e s p o n s e s are strictly confidential and a b so lu te ly n o in fo r m a tio n that
y o u su b m it w ill b e id e n tifie d w ith y o u o r y o u r h o sp ita l.
Copyright @ 1097 The Ohio Stale University
Please indicate ownership o f your hospital:
________Proprietary
________ Non-profit
________Major
________Limited
_________Ciovl.
None
beds
Independent
* Please note these Baldrige Award definitions to be used throughout the following questions:
1-inplovees include physicians, staff, volunteers and professional students who contribute to die delivery o f patient services.
Senior executives include your top executive (President, CHO) and the individuals who report directly to this person.
I.
to
vo
A lw a y s
S om n(inie.s
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
j.
to
to
o
A lw a y s
S o m e tim e s
N o t at all
S o m e tim e s
A lw a y s
5
5
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
N o t at all
A lw a y s
.Som e tim e s
i.
j.
N o t at all
A lw a y s
a. Our strategies address both short term uid long term planning.
2
2
h. Short and long term decisions and actions are aligned widi our
strategic plans.
i.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
IV. The following statements concern your hospitals Human Rest urce Management.
Please indicate how o ften the following occur in your hospital
N o t at in
SoniL'tiniL's
to
to
to
A lw a y s
d. We motivate employees by improved job design (such as crosstraining, job rotation, etc.).
i.
j.
2
2
T
4m
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
V.
u>
i.
a. New and revised health care services are reviewed mid tested
before they are introduced to patients.
to
to
A lw a y s
S o m e tim e s
N o t al all
Please indicate how often these occur in support services (lab tests, housekeeping, medical records):
N o t al all
j.
S o m e tim e s
A lw a y s
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Please indicate how often these occur in administrative services (accounting, materials management, hospital
administration):
S o m e tim e s
N o t at all
A lw a y s
Please indicate h o w o f te n these occur with suppliers (all outside providers f goods and services):
S o m e tim e s
N o t at all
to
to
A lw a y s
Please indicate vour position relative to vour competitors on each o f the fo l w ing:
.S ig n ific a n tly
A bou t
w o rse
th e sa u te
S ig n ific a n tly
better
i.
j.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1.
S ig n ific a n tly
A bout
w o rse
th e .same
belter
n. Asset utilization
6
6
6
N)
to
Lt)
S ig n ific a n tly
Please indicate the decree o f emphasis that van place on each o f the following:
l e a s e n o t e :
T h e t e r m . C u s t o m e r s , r e f e r s t o b o t h p a t ie n t s a n d o t h e r
No
M o d era te
lix lr c in c
s t a k e h o l d e r s ( s u c h u s th ir d p a r t y p a y o r s a n d i n v e s t o r s ) .
em p h a sis
e m p h a sis
e m p h a sis
d.
i.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
lixtrcine
emphasis
7
Moderate
emphasis
4
No
emphasis
j- Measuring customer feedback accurately
k. Measuring customer feedback on a reguktr basis
i.
to
to
os
Iligh/Hxcellcut
A v era g e
APPENDIX B
I.
Leadership
1.1
b.
c.
d.
e.
1.2
227
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1.3
EL
Derived from 5.4a (1): how key community health needs are
determined...
Derived from 5.4b: how assessments, measurements...are used to
maintain high performance.
Derived from 5.4 Note (1): Community health services could
include...unique health services provided a t a financial loss,
...service in free clinics...
b.
c.
d.
e.
228
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2.2
2.3
III.
Strategic Planning
3.1
229
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
3.2
IV.
4.2
h.
230
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
4.3
4.4
V.
Process Management
5.1
Derived from Flynn, et. al., 1994 and 5.1b: how patient health
care service designs are reviewed and/or tested to ensure safe,
effective, trouble-free initiation.
Derived from Flynn, et. al., 1994 and 5.1a (2): how patient needs
and desires...are translated into...patient health care processes...
Derived from 5.1 Note (5): A measurement plan should spell out
what is to be measured.
Derived from 5. la (3): how all delivery requirements associated
with the services are addressed early in design by all appropriate
organizationl units/departments...
Derived from 5. lc: how service design...[is} evaluated and
im proved..
231
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
5.2
5.3
5.5
Derived from 5.3a (3): how key requirements are addressed early
in design...
Derived from 5.3b (2): how assessments, measurements and/or
observations are...used to maintain high performance.
Derived from 5.3c: how patient care support services are
evaluated a nd improved...[usingJ information from patients...
5.6
Derived from 5.2c: how patient health care services are evaluated
and im proved to achieve...efficiency., and Note (3): Efficiency
includes costs, timeliness...
Derived from 5.2c: how patient health care services are evaluated
and im proved to achieve... effectiveness...; and Note (3):
Effectiveness refers to...appropriateness, risk...
Derived from 5.2b (1): how health care service delivery is
explained to set realistic patient expectations...
Derived from 5.2 Note (5): ...maintenance o f process performance
using measurements and/or observations to decide whether or not
corrective action is needed.
232
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
VI.
f.
g.
6.2
i.
6.3
233
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
6.4
6.5
234
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
VII.
d.
e.
7.2
235
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
7.3
k.
1.
m.
n.
7.4
Derived from 7.3a: how the organization determ ines patient and
other stakeholder satisfaction [including J processes and
measurement scales used...and how objectivity and validity are
ensured.
Derived from 7.3a: how the organization determ ines patient and
other stakeholder satisfaction [includingI processes and
measurement scales used...[and] frequency o f determinations...
Derived from 7.3a (3): how stakeholder satisfaction measurements
capture key information that reflects stakeholders' likely future
market behavior, such as repurchase intentions...
Derived from 7.3b: how patient and stakeholder satisfaction
relative to that fo r competitors...is determined.
Derived from 7.3c: how the organization evaluates and improves
irs overall processes and measurement scales fo r determining
patient and. stakeholder satisfaction...
236
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
APPENDIX C
COVER LETTER FOR QUESTIONNAIRE MAILING
FirstName LastName
Company
Addressl
Address2
City, State PostalCode
Dear Participant:
A research team at The Ohio State University is conducting a study o f the criteria o f
iheM alcolm Baldrige N ational Q uality A w ard in H ealth Care. A pilot test o f these
criteria was com pleted in 1995 by the National Institute o f Standards and Technology,
and this prestigious award may be given in 1998 pending C ongressional funding. T he
Baldrige Award criteria provide a model o f T otal Q uality M anagem ent (TQ M ) that
hospitals can use to plan and design their service system s, regardless o f their interest in
applying for the award.
This research project investigates the pilot criteria o f the Baldrige Health Care A w ard,
and we will determine the validity and relative im portance o f the award criteria in the
health care environment. You have been selected to participate because you are likely
familiar with your hospital's quality management practices.
We greatly appreciate your participation in this project. In return, you will receive a
summary o f the questionnaire results which you m ay use fo r benchm arking your TQ M
practices against other hospitals. Please try to answ er all questions as each has been
asked w ith a p articular research objective in m ind. T his q u estio n n aire tak es
approxim ately 20 m inutes to com plete. All o f y o u r responses w ill be strictly
confidential, and I will be the only individual with access to the raw data.
This research project is supported by a grant from the A m erican Society for Q uality
(ASQ). If you have any q u estio n s, please c o n tac t me at (614) 292 -4 1 3 6 o r
m eyer.240@ osu.edu. Thank you very much for your participation in this research
project.
Sincerely:
Susan M. Meyer
Dept, o f M anagement Sciences
The Ohio State University
237
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
APPENDIX D
QUESTIONNAIRE SCALE ITEM S, RESPONSES, AND
PRINCIPAL COM PONENT LOADINGS
NA
Load
*
5.0
3.6
11.8
24.1
34.1
20.9
0.5
.869
1.4
4.1
7.7
20.9
36.4
29.1
.5
.871
0.9
4.5
8.6
15.0
21.4
27.3
22.3
.750
0.5
0.5
0.9
3.6
5.9
29.1
59.5
.662
0.9
0.5
6.4
12.7
30.5
49.1
.517
1,1
NA
Load
0.9
1.4
3.2
19.1
24.1
34.5
16.8
.775
0.5
0.9
3.6
8.6
22.7
42.7
20.9
.854
0.5
0.9
3.6
10.5
16.4
43.6
23.6
0.9
.850
1.2
* Load refers to the loading of the item on the first principal component on the indicated
Baldrige dimension (scale).
238
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
NA
Load
1-8
3-6
5.5
19.5
30.5
29.5
9.1
0.5
.737
0.9
1.8
1.8
10.5
21.4
35.9
27.3
0.5
.707
1.4
0.9
2.7
9.1
18.6
40.9
26.4
.23
0-5
0 .5
0.5
2.7
10.5
37.7
47.7
.788
i.
NA
Load
7.7
7.7
13.6
19.1
20.0
23.2
8 .6
.880
6.8
9.1
14.1
22.3
21.8
15.5
10.5
.917
4.5
9.5
19.1
20.5
21.4
15.5
9.5
.902
2.7
5.5
14.1
23.2
23.6
23.2
7.3
0.5
.876
16.4
15.0
17.7
23.6
11.8
10.5
5.0
.755
2.1
M a n a g e m e n t o f Inform ation
a n d Data
239
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
NA
Load
2.3
13.2
13.6
21.8
19.5
19.1
10.5
.918
1.8
5.9
6.8
21.8
25.5
23.6
14.5
.901
8.2
9.1
10.9
25.0
18.6
20.5
7.7
.799
2.2
P erfo rm a n ce C o m p a riso n s
and B en ch m ark in g
Please indicate how often the follow ing occur in your hospital:
Scale anchors: Not at all ( I ) - Sometimes (4) - Always (7) ( NA indicates no response)
1
NA
Load
1.4
4.5
7.7
24.1
26.4
26.8
9.1
.881
0.9
3.6
9.5
19.1
29.5
28.2
9.1
.867
0.5
5.5
4.5
17.7
31.8
32.3
7.3
0.5
.793
4.5
11.4
13.6
31.4
21.4
14.1
2.7
0.9
.773
m. We compare performance
measurements with our competitors
(such as comparing cost per DRG).
1.8
3.6
7.3
13.6
25.0
31.4
17.3
.723
0.9
0.5
8.2
18.2
30.5
26.8
14.5
0.5
.854
2.3
A n alysis and U se o f
O r g a n iz a tio n -L e v e l D a ta
240
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Please indicate how often the follow ing occur in your hospital:
Scale anchors: N ot at all (I) - Sometimes (4) - Always (7) (NA indicates no response)
3.1
S tra teg y D e v e lo p m e n t
NA
Load
.886
0.5
1.4
0.9
9.1
15.9
36.8
34.5
0.9
2.3
1.8
5.9
15.0
36.8
37.7
0.5
.887
1.4
0.5
2.7
10.9
25.5
34.1
24.5
0.5
.914
0.9
2.7
6.4
15.0
24.1
31.4
19.1
0.5
.846
0-9
3.6
5.0
16.8
27.3
27.3
18.2
0.9
.875
Please indicate how often the follow ing occur in your hospital:
Scale anchors: N ot a t all (I) - Sometimes (4) - Always (7) (NA indicates no response)
1
NA
Load
1.4
4.5
10.9
19.1
24.5
23.2
15.9
0.5
.833
3.6
5.5
7.3
16.8
27.3
24.1
15.0
0.5
.881
0.5
2.7
5.5
13.6
25.5
33.6
18.2
0.5
.920
i.
3.2
2.3
5.5
13.2
21.8
32.7
20.0
1.4
.856
3.2
f.
S tra teg y
D e p lo y m e n t
241
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
NA
Load
1.4
4.5
4.5
15.0
28.2
34.1
12.3
.872
3.2
5.5
10.9
18.6
29.5
25.0
7.3
.879
0.5
3.2
3.2
14.1
27.3
39.5
11.8
0.5
.809
1-8
4.1
8.6
19.1
31.8
27.3
6.8
0.5
.802
4.1
NA
Load
0.9
2.7
15.9
38.6
30.0
11.8
.726
0.9
3.6
5.5
24.1
40.0
20.9
5.0
.850
8.6
10.5
12.3
22.3
24.1
11.8
10.5
.837
5.0
9.5
14.5
25.0
26.4
15.0
3.6
0.9
.813
4 .2
S y ste m s
242
Reproduced with permission o f the copyright owner. Further reproduction prohibited without permission.
E m p lo y e e /H e a lth C are S ta ff
E d u c a tio n , T r a in in g , and
D e v e lo p m e n t
NA
Load
i.
1-8
3.6
5.9
15.9
31.4
28.2
12.7
0.5
.833
j.
2.7
7.3
16.4
25.9
22.7
16.4
8 .6
.863
2.7
5.0
16.4
28.6
25.9
15.0
6 .4
.915
I.
5.0
10.9
15.0
26.8
23.2
14.1
5 .0
.860
NA
Load
0-9
4.1
11.8
24.1
31.8
19.5
7.7
.855
5.0
13.6
15.0
17.7
22.3
17.3
9.1
.830
o.
0-9
3.2
8.6
1 1.8
25.9
33.2
15.9
0.5
.794
9.1
12.7
14.5
24.1
18.6
14.5
6 .4
.822
11-4
8.2
14.1
17.3
20.0
17.3
11.8
.790
243
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Please indicate how often the following occur in patient services (services with direct
patient contact):
Scale anchors: Not at all ( I) - Sometimes (4) - Always (7) (NA indicates no response)
5.1
NA
Load
1-8
4.1
5.0
20.9
19.5
30.5
18.2
.797
0.9
3.6
4.1
19.1
28.6
32.7
10.9
.845
c.
2.7
3 .6
7.7
18.6
29.1
29.1
9.1
.916
d.
3.2
4.5
9.1
18.2
27.3
27.7
10.0
.897
e.
3.6
4.1
10.9
16.4
25.9
30.9
8.2
.876
Please indicate how often the following occur in patient services (services with direct
patient contact):
Scale anchors: N ot at all (I) - Sometimes (4) - Always (7) (NA indicates no response)
5.2
NA
Load
f.
2.7
1.8
5.5
16.4
21.4
39.1
13.2
.886
g.
2.7
2.7
6.8
12.7
25.5
35.9
13.6
.913
h.
0.5
3.2
10.0
23.6
42.7
20.0
.690
i.
Measurements/observations of patient
services w ill indicate when corrective
actions are needed.
0.5
0.9
3.6
14.5
29.1
38.6
12.3
0 .5
.870
244
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Please indicate how often the following occur in support services (lab tests, housekeeping,
medical records):
Scale anchors: Not at all (1) - Sometimes (4) - Always (7) (NA indicates no response)
j.
5 .3 P atien t C are S u p p o rt
S erv ices D esig n a n d D e liv ery
1-4
2.3
7.3
2 3 .2
34.1
22.3
1-4
1.8
3.6
17.3
25.5
1-8
3.6
10.5
NA
Load
9.5
.822
36.8
13.6
.908
15.0 42.3
26.8
.836
Please indicate how often the follow ing occur in your hospital:
Scale anchors: Not at all (I ) - Sometimes (4) - Always (7) (NA indicates no response)
1
NA
Lind
1.4
1.4
2.3
10.0
16.8
44.5
23.6
.854
2.7
2.7
8.6
15.5
20.0
30.9
19.1
0.5
.852
1.4
0.9
2.3
4 .5
7.7
26.8
56.4
.720
5 .4
C om m un ity H ealth S e r v ic e s
D esign and D e liv e r y
Please indicate how often the following occur in administrative services (accounting,
materials management):
Scale anchors: Not at all (1) - Sometimes (4) - Always (7) (NA indicates no response)
1
NA
Loud
5.9
8.6
11.4
23.2
23.2
21.8
5.5
0.5
.798
10.0
14.5
15.5
27.3
15.5
14.1
3.2
.812
10.5
11.4
14.5
25.5
17.3
14.5
6.4
.880
4.5
9.1
15.5
22.3
22.3
15.9
10.0
0.5
.860
5 .5
245
Reproduced with permission o f the copyright owner. Further reproduction prohibited without permission.
Please indicate how often the following occur with suppliers (all outside providers o f
goods and services):
Scale anchors: Not at all (I) Sometimes (4) - Always (7) (NA indicates no response)
-
NA
Load
0.9
3.6
10.9
28.2
44.5
11.4
0.5
.791
0.9
4.5
8.2
22.7
27.3
25.5
10.5
0.5
.893
3.6
8.2
13.6
30.0
22.7
18.2
3.6
.878
5.6
S u p p lie r P erform an ce
M anagem ent
Please indicate your position relative to your competitors on each o f the following:
Scale anchors: Significantly worse (I) - About the same (4) - Significantly better (7)
(NA indicates no response)
6.1
NA
Load
0.9
8.6
25.9
25.0
29.1
10.0
0.5
.466
0.5
6.4
39.5
28.2
21.4
2.3
1.8
.669
25.0
6.8
2.7
.727
0.5
0.5
2.7
38.6
23 2
0.5
2.7
35.9
31.4
22.7
4.1
2.7
.819
0.5
3.2
34.1
31.4
21.8
7.3
1.8
.857
1.4
7.3
35.0
26.4
25.9
2.7
1.4
.721
1.4
4.5
37.3
28.6
20.5
3.6
4.1
.679
Please indicate your position relative to your competitors on each o f the following:
Scale anchors: Significantly worse (1) - About the same (4) - Significantly better (7)
(NA indicates no response)
6 .2
h. Effectiveness (appropriateness,
availability) o f support services
i.
NA
Load
6.8
28.2
32.7
26.8
4.1
1.4
.904
0.5
1.4
10.9
25.5
29.5
25.5
5.5
1.4
.904
246
Reproduced with permission o f the copyright owner. Further reproduction prohibited without permission.
Please indicate your position relative to your competitors on each o f the following:
Scale anchors: Significantly worse ( I) - About the same (4) - Significantly better (7)
(NA indicates no response)
1
NA
Load
0.9
1.4
5.5
16.4
27.3
29.1
19.1
0.5
.933
0.9
2.7
7.3
214
19.1
31.8
16.4
0.5
.933
6 .3
C om m unity H ealth S e r v ic es
R e s u lt s
Please indicate your position relative to your competitors on each o f the following:
Scale anchors: Significantly worse ( I) - About the same (4) - Significantly better (7)
(NA indicates no response)
6 .4
1.
m. Employee productivity
n. Asset utilization
o. Worker turnover (low turnover =
better, high = worse)
P- Personnel costs per patient
q-
r.
NA
Load
0.9
1.4
9.5
38.6
23.6
19.1
2.7
4.1
.557
0.9
9.1
28.6
29.1
25.9
4.1
2.3
.753
1.4
4.5
25.9
35.5
26.4
2.7
3.6
.672
1.8
3.2
11.4
29.5
24.1
22.3
5.0
2.7
.359
0.9
16.4
34.1
27.7
14.4
4.5
2.3
.815
0.5
13.6
33.6
29.1
14.1
3.6
5.5
.748
0.5
6.4
10.9
32.3
14.1
2.7
3.2
.683
Please indicate your position relative to your competitors on each o f the following:
Scale anchors: Significantly worse (I) - About the same (4) - Significantly better (7)
(NA indicates no response)
6.5
s.
t.
A ccreditation a n d
A ssessm en t R e su lts
NA
Load
0.9
33.2
24.1
28.2
13.2
0.5
.911
0.9
34.1
17.7
30.5
15.0
1.8
.911
247
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Please indicate the degree o f emphasis that you place on each o f the following:
Scale anchors: No emphasis (I) Moderate emphasis (4) Extreme emphasis (7)
(NA indicates no response)
-
NA
Load
0.5
1.8
3.6
20.0
27.7
33.6
12.3
0.5
.728
0.9
4.1
7.3
19.1
26.8
28.2
13.2
0.5
.827
1.8
2.7
5.9
19.5
30.5
25.0
13.2
1.4
.863
16.8
9.5
16.4
23.2
16.4
11.8
4.5
1.4
.758
0-5
2.3
7.3
18.6
29.5
32.7
7.7
1.4
.778
7.1
Please indicate the degree o f emphasis that you place on each o f the following:
Scale anchors: No emphasis (I) - Moderate emphasis (4) - Extreme emphasis (7)
(NA indicates no response)
1
NA
Load
1.4
4.1
10.5
18.2
34.1
25.5
5.5
0.9
.775
1.8
4.1
7.3
16.4
23.6
29.5
16.8
0.5
.848
3.2
6.8
16.8
27.3
29.5
0.5
.887
0.5
1.4
10.5
12.7
18.6
32.7
15.9
OO 7
0.9
.803
7.2
P a tie n t/S ta k e h o ld e r
R e la tio n sh ip M a n a g e m e n t
f.
248
Reproduced with permission o f the copyright owner. Further reproduction prohibited without permission.
Please indicate the degree o f emphasis that you place on each o f the following:
Scale anchors: Mo emphasis (J) - Moderate emphasis (4) - Extreme emphasis (7)
(NA indicates no response)
1
NA
Load
0.5
2.7
4.5
13.6
19.5
37.7
20.5
0.9
.899
0.9
0.5
4.5
8.2
15.9
38.6
30.9
0.5
.861
2.7
2.3
9.1
13.2
21.8
32.3
17.7
0.9
.855
3.2
4.1
12.7
20.9
15.0
30.0
12.7
1.4
.767
0.9
3.2
4.1
14.5
15.0
39.1
22.3
0.9
.813
7.3
P a tie n t/S ta k e h o ld e r
S a tis fa c tio n D e te r m in a tio n
j.
P a tie n t/S ta k e h o ld e r
S a t is f a c t io n R e s u lts
0.9
1.8
15.0
26.8
22.3
NA
Load
44.5
10.0
0.9
.810
50.0
14.1
2.3
.827
0.5
10.9
0.5
0.9
12.7
33.6
41.4
10.0
0.9
.909
0.9
10.9
28.6
47.3
11.4
0.9
.838
Please indicate your position relative to your competitors on each o f the following:
Scale anchors: Significantly worse (I) - About the sam e (4) - Significantly better (7)
(NA indicates no response)
1
NA
Load
0.5
0.9
2.3
20.0
31.4
31.4
11.4
2.3
.862
0.5
0.9
5.0
41.4
26.4
17.7
3.6
4.5
.818
1.8
30.9
33.2
26.8
5.5
1.8
.906
7.5
P a tie n t/S ta k e h o ld e r
S a tis fa c tio n C o m p a r is o n
249
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
IMAGE EVALUATION
TEST TARGET ( Q A - 3 )
| |
1.0
2.8
|3D
U_
H-
IM
2.2
1 3.6
2.0
l.l
JJL
1.25
1.4
1.6
150mm
A P P L I E D A IIVUG E . I n c
r -= = =
Phone: 716/482-0300
Fax: 716/268-5989
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.