PRACTICE APPLICATIONS

Topics of Professional Interest

Minimizing False-Positive Nutrition Referrals

Generated from the Malnutrition Screening Tool

HE JOINT COMMISSION SPECies that each hospital has

dened criteria that identify
when nutritional plans are
developed and assess their patients
according to dened time frames.1 The
Academy of Nutrition and Dietetics
(Academy) denes nutrition screening
as the process of identifying characteristics known to be associated with
nutrition problems, with a goal of identifying individuals who are malnourished or at nutritional risk and are in
need of intervention and/or education
from a registered dietitian nutritionist
(RDN).2 Many hospitals apply nutrition
screening as part of the admission
database process typically completed
by registered nurses to determine patients requiring a referral to an RDN
for a complete nutrition assessment
and development of the nutrition care
plan. An effective nutrition screening
process is essential to help prioritize
hospital resources, including RDN
time, on patients at highest need for
services.
A wide variety of nutrition screening
questions are employed in hospitals in
the United States based on patient
population needs and/or multidisciplinary input at hospitals.3 Several validated nutrition screening tools exist,
and a majority of hospitals opt to use
only one tool for all of their adult inpatient populations. Popular screening
tools
include
the
Malnutrition
Screening Tool (MST),4 the Malnutrition

This article was written by Wendy

Phillips, MS, RD, FAND, division
director of clinical support, Morrison,
Healthcare, St George, UT (at the
time of the study, she was clinical
nutrition director, University of Virginia
Health System, Crozet); and Sunitha
Zechariah, division director of clinical
support, Morrison Healthcare, Evans,
GA (at the time of the study, she was
clinical nutrition manager, University
Health System, Augusta, GA).
http://dx.doi.org/10.1016/j.jand.2016.05.014

2016 by the Academy of Nutrition and Dietetics.

Universal Screening Tool (MUST),5 and

the Nutrition Risk Screening-2002
(NRS-2002).6 The MST is described in
detail in this article and is considered a
quick and easy screening tool with
two questions. The MUST and the NRS2002 are considered comprehensive
nutrition screening tools, with the
MUST including a ve-step screening
tool with measures of disease severity,
weight loss, and body mass index
(BMI).5 The NRS-2002 includes measurement of BMI, disease severity,
weight loss, and dietary intake.6
Tools such as these are developed
with a goal of predicting nutritional
status or predicting poor clinical outcomes related to malnutrition. Due to a
lack of a consistent denition of
malnutrition or a gold standard against
which to compare the validity of
nutrition screening tools, most have
been developed and validated with
assessment by a clinician or using a
standardized assessment tool as the
reference method.7 The chosen reference standard is assumed to be superior to the tool being validated. The
Subjective Global Assessment (SGA),8
developed in 1982 within a surgical
population, is an assessment tool, is
completely based on clinicians evaluations, and is often regarded as the
gold standard against which to
compare other screening tools. Validity
of the SGA was demonstrated by correlation of the clinical classications in
the tool with objective measurement of
nutritional status and with three measures of hospital morbidity: incidence
of infections, use of antibiotics, and
length of stay (LOS).8
Ease of use is the main deciding
factor for the choice of nutrition
screening tool.3 As discussed in a systematic review of nutrition screening
tools in hospital settings in 2012,7
there is no consensus on a single best
nutrition screening or assessment tool
to use for all categories of hospitalized
patients. Several nutrition screening
tools were evaluated by the Academy

in 2009 for validity and reliability as

part of their Evidence Analysis Library
(EAL) process,9 and this can be a helpful reference for clinicians.
As mentioned, the MST4 is commonly
used due to its simplicity and ability to
be completed without additional calculations. The MST was validated using
the SGA as the reference standard.
The MST includes questions about an
adult hospitalized patients appetite
and weight changes (see Table 1).
Scores are allocated based on a patients
or caregivers response to the questions.
In cases where the patient responds
yes to the question about losing
weight without trying, then the nurse
proceeds to the second question to ask
the amount of weight lost. Points are
assigned based on the total amount of
weight lost. If the patient verbalizes
having lost weight, but is unsure how
much weight has been lost, he or she
would be assigned 2 points. If the patient answers no or unsure to the rst
question, then the score for that section
should be generated and the appetite
question should be asked next. It is not
necessary to ask or score the second
weight loss question investigating the
amount of weight lost in cases where
the patient answers no or unsure to
the rst question. The weight loss and
appetite scores are then totaled. In cases
where a patient scores 2 or more points
on the screening tool, he or she is
considered at nutritional risk and a
referral is sent to an RDN to complete a
more in-depth assessment and to
determine whether nutrition interventions are warranted. Patients
who are uncertain whether they have
lost weight recently, or know that they
have lost weight but are unsure how
much, are assigned 2 points (not 4
points, as may be misinterpreted from
Table 1). Ferguson and colleagues,4 authors of the MST tool, assigned a nal
value of 2 points to these patients
because this had the highest sensitivity
and specicity at predicting the score
on the SGA. In other words, patients

JOURNAL OF THE ACADEMY OF NUTRITION AND DIETETICS

PRACTICE APPLICATIONS
Table 1. Malnutrition Screening Tool
Malnutrition Screening
Tool item

Score

Have you lost weight

recently without trying?
No

Unsure

If yes, how much weight

(kilograms) have you lost?
1-5

6-10

11-15

>15

Unsure

Have you been eating poorly

because of a decreased
appetite?
No

Yes

Total
who were unsure of weight loss were
shown to be more likely to be either
malnourished or at risk for malnutrition
than those who had not lost weight.
The MST was evaluated in four crosssectional studies for patients in acutecare hospital settings and ambulatory
oncology centers.4 The tool was determined to accurately identify patients at
nutritional risk and in need of RDN
assessment and intervention at least
93% of the time (ie, true positives).
In addition, the MST accurately identied adequately nourished patients as
not at risk (ie, true negatives) 93% of
the time. These are measures of sensitivity and specicity, respectively.
Other nutrition screening tools have
higher specicity and sensitivity,9 as
described in the Academys EAL, but
these tools have either not been validated for use in acute-care hospital
populations or are labor intensive to
administer when compared with the
MST.9 Because nurses complete the
nutrition screening tool in most acutecare hospitals, and have limited time to
complete it, it is benecial to use the
tool that is quick and easy to administer. Of the 11 tools evaluated by the
Academys EAL workgroup,9 the MST
was shown in 2009 to be both valid
and reliable for identifying nutrition
2

problems in acute-care settings, and its

simplicity and rapidity has allowed
many hospitals to adopt this tool for
malnutrition screening. The simplicity
of the tool is veried by a high interrater reliability of 93% to 97%. The
high interrater reliability indicates that
most nurses will assign the same score
to a patient regardless of which nurse
is using the tool at that time.
Subsequent to the original validation
of the MST in 1999 by Ferguson and
colleagues4 in a generalized hospital
population and an ambulatory oncology
population, it was compared for use in
different populations. Amaral and colleagues10 evaluated the MST in predicting outcomes and nutritional status
in oncology inpatients in Portugal between March and June 2005. They
compared the screening value of MST,
MUST, and NRS-2002 in identifying patients at risk for malnutrition and to
explore their ability to predict a high
LOS. The NRS-2002 was chosen as the
reference method instead of the SGA
based on previous evidence that the
NRS-2002 is a strong predictor of LOS
and has shown the highest agreement
with other screening and nutrition
assessment tools in hospitalized patients. In the study by Amaral and colleagues,10 patients were less likely to be
identied as malnourished or at risk for
malnutrition using the MST than they
were when the MUST or NRS-2002 was
used, and the MUST showed the most
agreement with the chosen reference
standard, the NRS-2002. Patients who
were identied as malnourished using
the MUST and NRS-2002 were more
likely to have a longer LOS, but this effect
was not seen when the MST was used.
Because the NRS-2002 was used as the
reference standard and, therefore,
assumed to be the superior tool, failure
of the MST to predict those with a longer
LOS indicated that the MST was less
likely to be predictive of malnutrition
and therefore had a lower sensitivity.
The results obtained by this group were
likely different from the validation
study by Ferguson and colleagues4 due
to a different patient population and a
different reference standard.
In 2006, Neelemaat and colleagues11
compared ve nutrition screening
tools, including the MST, on their ability
to predict malnutrition in a generalized
hospital population. The reference
standard used was a classication of
moderate or severe malnutrition based

JOURNAL OF THE ACADEMY OF NUTRITION AND DIETETICS

on the patients BMI and degree of

weight loss during the previous 6
months. The MST was shown to be 76%
sensitive and 90% specic. Again, the
difference in results from the original
validation study are likely due to a
different patient population and a
different reference standard.
The MST was again compared for use
in the inpatient oncology patient population between July 2011 and March
2013 in a hospital in London.12 The
reference standard this time was the
Patient-Generated SGA, which is a
modied version of the SGA. In this
study, the MST had a sensitivity of 66%
(95% CI 25 to 75) and a specicity of
83% (95% CI 86 to 98) for hospitalized
oncology patients when compared
against the reference standard of the
Patient Generated-SGA. The positive
predictive value was 91%, and the
negative predictive value was 49%. This
indicates that whereas most patients
who were malnourished or at risk for
malnutrition were identied correctly,
a large number of false-positive patient
referrals were also generated. The results of this study were similar to the
study by Amaral and colleagues10 in an
inpatient oncology population, despite
using a difference reference standard.
Consistent with the original validation study4 and these studies,10-12 MST
identies malnourished patients fairly
quickly; however, also consistent with
these studies, it generates falsepositive referrals. In statistics, false
positives are known as type 1 errors,
which detect an effect that is not present, whereas a false negative fails to
detect an effect that is present. A
screening referral is classied as false
positive in cases where the patient
triggers positively on the nutrition
screen, but the RDN did not identify a
nutrition diagnosis and/or intervention
for the patient upon completing a full
nutrition assessment. A patient would
be considered a false negative if he or
she was at nutritional risk, yet the
screening tool failed to identify him or
her and the RDN was not alerted about
this patient through the nutrition
screening tool. Ferguson and colleagues4 acknowledged that the false
positives from the MST could be
reduced by changing the cutoff value
for identifying a patient as at risk for
malnutrition as 3 points or more, but
were concerned that this may increase
false negatives such that patients who
--

2016 Volume

Number

PRACTICE APPLICATIONS
are at risk for malnutrition would not
be referred to an RDN for further
assessment.
False-positive referrals can signicantly add to the workload of an RDN,
which can have several adverse effects
along with a decline in job satisfaction.
Time spent assessing patients who
triggered positively on the nutrition
screen yet who are not actually in need
of nutrition intervention and monitoring reduces an RDNs productive
hours to assess and intervene with patients who are actually at nutritional
risk. In theory, one may argue that it
takes just a few minutes to investigate a
patient triggered by the nutrition
screening tool for evidence of whether
the patient is at nutritional risk. However, in practice, RDNs spend as much as
15 to 20 minutes per false positive
referral to investigate whether there
was any credible weight loss and/or loss
of appetite. For instance, consider a
patient who was triggered with an MST
score of 2 points due to unsure weight
loss. For an RDN to determine whether
this unsure weight loss is true, rst an
RDN has to spend several minutes in the
electronic medical record to explore for
further documentation. In cases where
there is lack of documentation, RDN
visits the patient for fact nding. In
cases where the patient is cognitive and
oriented, he or she may be able to
articulate the amount of weight loss or
lack thereof. In cases where the patient
is nonverbal and/or incoherent and in
the absence of a caregiver, an RDN has
to nd the nurse providing care for the
patient and discuss whether there is
any evidence for weight loss. In cases
where an RDN has 5 false referrals a day,
this extensive, investigative work can
lead him or her to lose about 75 to 100
minutes (1.15 to 1.30 hours) in a day.
This burden is exacerbated during
weekends because frequently only one
RDN provides care for the entire facility.
This loss of productivity can increase
RDNs exasperation in time and labor
wasted in investigating false positive
referrals and decrease their availability
for value-added activities such as
participation
in
multidisciplinary
rounds and quality assurance/performance improvement activities. In
addition, RDNs compensate the lost
time by spending longer hours to complete their work leading to a decline in
job satisfaction and increase in RDN
turnover in acute care. Although some
--

2016 Volume

Number

surveys have shown that RDN jobs are

the least stressful in health care,13 these
surveys do not specically report on
RDN jobs in acute-care hospitals, and
the less stressful trend will soon reverse
if RDNs continue to experience long
work hours in acute care.
Our study was initiated due to RDNs
reporting high levels of frustration with
the increased frequency of false referrals from the MST tool. We set out to
examine whether there was any evidence in RDNs report of increased false
referrals from the MST or whether this
is a perceived notion. If our study
showed credible evidence of increased
false referrals, our goal was to develop
strategies and practical solutions to
minimize them, allowing RDNs to focus
on value-added activities that benet
patients and the multidisciplinary team.
Any strategy that decreases the likelihood of false-positive referrals also
increases the likelihood of false negatives. However, this can be mitigated,
because most hospital policies and
procedures include additional nutrition
screening measures carried out by
RDNs to identify high-risk patients who
may not have triggered positively on
the nutrition screening tool. This may
include patients with pressure ulcers or
receiving nutrition therapy such as
enteral or parenteral nutrition, patients
meeting a specic LOS at the facility, or
patients who have specic diagnoses
such as congestive heart failure.

EVALUATION PROCESS
Data were collected between August
and October 2013, with each hospital
collecting data for 31 days during this
period. Five acute-care hospitals that
utilize the MST for nutrition screening
participated in this study to identify the
prevalence of inappropriate referrals, or
false positives, generated from the MST.
All hospitals had similar screening and
prioritization policies to determine
when a patient would be assessed and
reassessed by an RDN. Average adult
census, excluding psychiatric, pediatrics, obstetrics, and labor and delivery
oors ranged from 220 to 435 patients
during the data collection period. Institutional review board approval was not
required because these were department
performance
improvement
projects and no condential data identifying individual patients were shared
at any time. All nutrition screening was

completed by registered nurses and all

tracking of appropriateness of the referrals was performed by RDNs. Exclusion criteria were patients younger than
age 18 years, those admitted to a psychiatric ward, or pregnant. All patients
for whom the MST had not been
completed were also excluded; most
often these were patients who were
mechanically ventilated and sedated
with no family present and therefore
the usual weight and dietary intake
were unknown. These exclusions were
similar to the original validation study
completed by Ferguson and colleagues.4
Although the original validation study
for the MST used the SGA classication
as the gold standard to determine risk of
malnutrition, this study used the RDNs
expertise to determine whether nutrition interventions were needed and/or a
nutrition diagnosis was identied as the
indication of whether the trigger was a
false positive. This method of assessment by a nutrition and dietetics practitioner
to
determine
whether
nutritional status is consistent with
validation methods used in other
studies.7 All RDNs at each hospital were
utilizing the Academy/American Society
for Parenteral and Enteral Nutrition
consensus statement criteria to determine presence of and degree of malnutrition,14 thus creating a consistent,
objective method for nutrition assessment. In addition, all of the participating
hospitals have the same nutrition
assessment standards and used evidenced based information from the
same hospital-approved diet manual.
Consistent with the original validation study,4 patients were considered
positive triggers in cases where they
scored 2 points or more on the MST
tool. Patients were further categorized
in this study based on scores of 2
points, 3 points, or 4 or more points on
the tool (see Table 2 for sample size
from each category). A nutrition

Table 2. Number of patients and

percentage of false triggers on the
Malnutrition Screening Tool (MST) for
each score category
MST Category

Sample size (n)

False triggers (%)

681

392

257

36

26

17

JOURNAL OF THE ACADEMY OF NUTRITION AND DIETETICS

PRACTICE APPLICATIONS
screening referral triggered by the MST
was classied as false positive in cases
where an RDN completed a full nutrition assessment and did not identify
the patient to be at nutritional risk and
therefore did not assign a nutrition
diagnosis and/or necessary nutrition
intervention for the patient. Data were
organized, tabulated, and reported via
descriptive results using Microsoft
Excel.15 Data were statistically analyzed
for signicance using a Pearson c2 test
with SPSS.16

DISCUSSION
Of the 1,330 patients screened using
MST in the ve acute-care hospitals,
71% were accurately recognized as
malnourished and 29% were inaccurately triggered as malnourished or as
at risk for malnutrition. Therefore, 29%
of the total patients studied were
considered false-positive triggers. This
total patient population was further
analyzed based on the MST score. Of
the 1,330 total patients, 681 patients
received an MST score of 2. Of these
681 patients, 242 patients were false
triggers (36%). Of the 681 patients who
scored 2, 211 patients (31%) were triggered due to an unsure of weight loss
response. This was 16% of the overall
sample size. A total of 392 patients
received a score of 3, of which 102
(26%) were false triggers. A total of 257
patients received a score of 4, of
which 44 were false triggers (17%) (see
the Figure and Table 2). It was found
that the likelihood of false triggers

MST 4
17%

from the screening tool was higher as

the score decreased (P<0.001).

PRACTICE IMPLICATIONS
This study showed a lower specicity
of the MST than the original validation
study4 (93% specic in the original
study vs 71% specic in the current
study). This is likely due to the different
method used to classify the patient as a
false-positive trigger, because the
original study compared the screening
questions with the SGA score and this
study used RDNs expertise in determining the need for intervention and/
or documentation of a nutrition diagnosis as an indicator of whether the
patient was malnourished. Therefore,
individual clinician judgment played a
greater role in this study. Because
multiple clinicians at ve different
hospitals participated, a limitation of
this study is the likelihood for differences in clinician judgment. One may
argue that clinicians judgement may
vary, which may insert bias into the
evaluation process and hence is not an
ideal validation method/reference
standard. Although variations are
possible, it certainly is minimal
because all of the clinicians across the
ve hospitals were trained using the
same policies and procedures, had
similar standards of practice, and used
evidence-based information from the
same approved diet manual. In addition, clinician expertise is essential and
the predominant way in identifying
and treating malnourished patients
in actual clinical practice. Without

MST 2
36%

MST 3
26%
Figure. Percentage of patients who triggered as false positives on the Malnutrition Screening
Tool (MST) in each score category. Overall, 29% of patients triggered as false positive.
4

JOURNAL OF THE ACADEMY OF NUTRITION AND DIETETICS

clinicians and their skill in applying

reference criteria to evaluate a patient
and determine appropriate nutrition
information, all scientically validated
tools will serve no purpose in real-life
hospital settings. Hence, this method
of validation reects real-world application of the MST and, therefore, is
relevant to clinical practice.
Ferguson and colleagues4 suggested
that patients with a higher MST score
should be prioritized to receive nutrition care rst. This study conrmed
that recommendation because patients
with higher scores on the MST had a
reduced incidence of false-positive
triggers and were, therefore, more
likely to be malnourished and in need
of RDN attention and intervention.
With increasing cuts and constraints
on stafng, it is imperative to maximize RDNs time with those patients
who benet the most from nutrition
interventions. Identifying which patients generate false-positive referrals
from the MST at their own facility can
help clinicians customize clinical
nutrition policies and procedures to
help RDNs use their time effectively.
For example, in cases where a performance improvement study reveals a
high frequency of false-positive triggers with MST score of 2, then the
nutrition screening policy may need to
be changed to indicate that patients
with an MST score of 3 or more will be
given the highest priority by RDNs
before addressing others.
Another limitation of this study is
that the incidence of false negatives
was not tracked nor quantied.
However, patients not identied
through the MST may be referred to
an RDN through a physician consultation request, prioritized to be seen
by an RDN based on disease diagnoses
associated with high nutritional risk
(such as gastrointestinal obstruction
or short bowel syndrome), due to the
presence of a pressure ulcer, and/or
due to the method of feeding such as
enteral or parenteral nutrition.
Another strategy to combat inaccurate screening using MST is to mandate
periodic training on its proper use to
nursing and other staff who conduct
the nutrition screen. Staff should be
trained to include caregivers for
gathering pertinent information when
necessary before completing the
questions and assigning a score on the
MST tool. Because patients, nursing,
--

2016 Volume

Number

PRACTICE APPLICATIONS
and other clinicians in the United
States are not accustomed to using the
metric system, it is best to list the unit
of measure for the amount of weight
lost in pounds as well as kilograms
when building the MST into the electronic medical record. This will reduce
confusion and the burden on the clinicians in converting the measures as
patients/caregivers primarily report
weight in pounds.
We also propose that the authors of
the MST consider editing the MST tool
to remove the response unsure from
the question, If yes, how much weight
have you lost? (Table 1). This would
reduce the confusion from having the
answer of unsure of weight loss in
two different places and reduce the
likelihood of patients being assigned a
score of 4 rather than 2. This would be
especially important for those hospitals
that alter their nutrition screening
policy to prioritize rst the patients
with 3 points or more on the MST.

nutrition screen, including gathering

pertinent information from both patients and caregivers when available,
will be benecial to reduce the burden
on RDNs while optimizing provision of
standardized care and improved clinical outcomes.

References
1.

The Joint Commission. www.jointcommission.

org. Accessed March 10, 2016.

2.

Identifying patients at risk: ADAs denitions for nutrition screening and nutrition
assessment. Council on Practice (COP)
Quality Management Committee. J Am
Diet Assoc. 1994;94(8):838-839.

3.

Patel V, Corkins MR, DiMaria-Ghalili RA,

et al. Nutrition screening and assessment in hospitalized patients: A survey
of current practices in the United
States. Nutr Clin Pract. 2014;29(4):483490.

4.

Ferguson M, Capra S, Bauer J, Banks M.

Development of a valid and reliable
malnutrition screening tool for adult
acute
hospital
patients.
Nutrition.
1999;15(6):458-464.

5.

Elia M. The MUST report. Nutritional

Screening of Adults: A Multidisciplinary
Responsibility.
Malnutrition
Advisory
Group (MAG). Redditch, UK: BAPEN;
2003.

CONCLUSIONS
The MST is a validated screening tool
that provides a simple and quick
method of screening patients who are
at nutritional risk.4 However, the MST
may also generate inaccurate referrals
burdening RDNs existing workload.
Enhancing the patient prioritization
process by customizing the policies and
procedures based on departmental
performance-improvement data, and
training staff to correctly complete the

6.

Rasmussen HH, Holst M, Kondrup J.

Measuring nutritional risk in hospitals.
Clin Epidemiol. 2010;2:209-216.

7.

Van Bokhorts-de van der Schueren MAE,

Guaitoli PR, Jansma EP, de Vet HCW.
Nutrition screening tools: Does one size
t all? A systematic review of screening
tools for the hospital setting. Clin Nutr.
2014;33(1):39-58.

8.

Baker JP, Detsky AS, Wesson DE,

Wolman SL, Stewart S, Whitewell J, et al.
Nutritional assessment: A comparison of

clinical judgment and objective measurements. N Engl J Med. 1982;306(16):

969-972.
9.

NSCR: Nutrition screening tools (20092010). Evidence Analysis Library of the

Academy of Nutrition and Dietetics.
http://www.andeal.org/topic.cfm?menu
3584&cat4305. Accessed March 10,
2016.

10.

Amaral TF, Antunes A, Cabral S, Alves P,

Kent-Smith L. An evaluation of three
nutrition screening tools in a Portuguese
oncology centre. J Hum Nutr Diet.
2008;21(6):575-583.

11.

Neelemaat F, Meijers J, Kruizenga H, van

Ballegooijen H, van Bokhorstde van der
Schueren M. Comparison of ve malnutrition screening tools in one hospital
inpatient sample. J Clin Nursing.
2011;20(15-16):2144-2152.

12.

Shaw C, Fleuret C, Pickard JM,

Mohammed K, Black G, Wedlake L. Comparison of a novel, simple nutrition
screening tool for adult oncology inpatients and the Malnutrition Screening
Tool against the Patient-Generated Subjective Global Assessment. Support Care
Cancer. 2015;23(1):47-54.

13.

Top 10 least stressful healthcare jobs.

http://www.hospitalcareers.com/blog/leaststressful-healthcare-jobs/. Accessed March
27, 2016.

14.

White JV, Guenter P, Jensen G, Malone A,

Schoeld M. Consensus statement: Academy of Nutrition and Dietetics and
American Society for Parenteral and
Enteral Nutrition: Characteristics recommended for the identication and documentation
of
adult
malnutrition
(undernutrition). JPEN J Parenter Enteral
Nutr. 2012;36(3):275-283.

15.

Excel 2013 [computer software]. Redmond, WA: Microsoft; 2013.

16.

IBM SPSS Statistics for Windows [computer program]. Version 22.0. Armonk,
NY: IBM Corp; 2013.

DISCLOSURES
STATEMENT OF POTENTIAL CONFLICT OF INTEREST
No potential conict of interest was reported by the authors.

FUNDING/SUPPORT
No funding/support was reported for this article.

--

2016 Volume

Number

JOURNAL OF THE ACADEMY OF NUTRITION AND DIETETICS