ACADEMIC EMERGENCY MEDICINE December 2001, Volume 8, Number 12
EDUCATIONAL ADVANCES
Reliability of the Visual Analog Scale for
Measurement of Acute Pain
POLLY E. BIJUR, PHD, WENDY SILVER, MA, E. JOHN GALLAGHER, MD
Abstract. Objective: Reliable and valid measures
of pain are needed to advance research initiatives on
appropriate and effective use of analgesia in the
emergency department (ED). The reliability of visual
analog scale (VAS) scores has not been demonstrated
in the acute setting where pain fluctuation might be
greater than for chronic pain. The objective of the
study was to assess the reliability of the VAS for mea-
surement of acute pain. Methods: This was a pro-
spective convenience sample of adults with acute pain
presenting to two EDs. Intraclass correlation coeffi-
cients (ICCs) with 95% confidence intervals (95% CIs)
and a Bland-Altman analysis were used to assess re-
liability of paired VAS measurements obtained 1 min-
ute apart every 30 minutes over two hours. Results:
E FFECTIVE pain management in the emer-
gency department (ED) is a major challenge
to emergency physicians. The First International
Symposium on Pain Research in Emergency Med-
icine identified the development of valid and reli-
able measures of acute pain as a necessary first
step in a research agenda designed to guide deci-
sions about analgesia in the ED.1 Without valid
(accurate) and reliable (reproducible) instruments,
any true effect of treatment can be obscured by
measurement error, or ineffective treatments may
be erroneously considered therapeutic.
The visual analog scale (VAS) is a valid and re-
liable measure of chronic pain intensity.25 Three
studies have demonstrated the validity of the VAS
for acute pain measurement among ED patients to
measure acute pain.68 However, little work has
From the Department of Emergency Medicine, Albert Einstein
College of Medicine (PEB, WS, EJG), Bronx, NY.
Received February 28, 2001; revision received May 29, 2001;
accepted June 1, 2001.
Presented at the SAEM annual meeting, San Francisco, CA,
May 2000.
Address for correspondence: Dr. Polly Bijur, Albert Einstein
College of Medicine, Kennedy Center Room 920, 1410 Pelham
Parkway South, Bronx, NY 10461. Fax: 718-430-8821; e-mail:
bijur@aecom.yu.edu
1153
The summary ICC for all paired VAS scores was 0.97
[95% CI = 0.96 to 0.98]. The Bland-Altman analysis
showed that 50% of the paired measurements were
within 2 mm of one another, 90% were within 9 mm,
and 95% were within 16 mm. The paired measure-
ments were more reproducible at the extremes of pain
intensity than at moderate levels of pain. Conclu-
sions: Reliability of the VAS for acute pain measure-
ment as assessed by the ICC appears to be high.
Ninety percent of the pain ratings were reproducible
within 9 mm. These data suggest that the VAS is suf-
ficiently reliable to be used to assess acute pain. Key
words: pain; pain measurement; reproducibility of re-
sults. ACADEMIC EMERGENCY MEDICINE 2001;
8:11531157
been done to assess the reliability of the VAS for
measurement of acute pain. The few studies that
have explicitly assessed the reproducibility of VAS
measures of pain focused on chronic or postopera-
tive pain, and most examined the correlation be-
tween repeat VAS measures. A study of a mechan-
ical version of a VAS (a tool with a 10-cm ruler and
a marker that the patient moves to the point in-
dicating his or her intensity of pain) used by pa-
tients with rheumatoid arthritis found a correla-
tion of 0.88 between two measures taken two hours
apart.5 Studies that examined the correlation be-
tween a vertically oriented VAS for pain with a
horizontally oriented VAS found correlations of
0.99 and 0.91 when they were given within 10
minutes of each other2,3 to patients with a variety
of rheumatic diseases.
Although calculation of the Pearson product
moment correlation coefficient has often been used
to assess VAS reliability,25 this method has been
justly criticized as providing an inflated esti-
mate.911 A more appropriate formulation of the
correlation coefficient12 and an analysis of the ac-
tual difference between repeated measures have
both been recommended.9
Our primary goal in this study was to assess
the reliability of the VAS in acute pain using meth-
odologically appropriate statistical techniques.
1154 VAS
METHODS
Study Design. An observational prospective co-
hort design was used to assess the reliability of
VAS pain measurements. The study was approved
by the institutional review boards of the two hos-
pitals that provided patients. Written informed
consent forms in English or Spanish were com-
pleted by all participants.
Study Setting and Population. All English- and
Spanish-speaking patients 18 years of age or older
who presented to the ED with acute pain as a com-
ponent of their chief complaint were eligible for in-
clusion. Acute pain was operationally defined as
pain of recent onset or exacerbation of existing
pain of sufficient severity to bring the patient to
the ED. Patients with altered mental status and
decreased visual acuity that precluded scoring of
the VAS were excluded. Data were collected on a
convenience sample of patients by trained research
associates during the hours of 8:00 AM to 8:00 PM,
six days a week over a period of nine months.
Study Protocol. After providing written informed
consent, patients were asked to rate pain intensity
by placing a mark on a 100-mm VAS. The VAS was
horizontally positioned with the extremes labeled
least possible pain and worst possible pain. One
minute later, patients were asked to rate their pain
severity again on a fresh VAS without reference to
the first measurement. A minute was chosen as the
time interval between paired ratings under the as-
sumption that most pain would not change within
a 1-minute period. This procedure was repeated at
times 0 and 1 minute, 30 and 31 minutes, 60 and
61 minutes, 90 and 91 minutes, and 120 and 121
minutes for a maximum of two hours (five paired
readings) or until the patient left the ED. The first
measurement in each of the five 1-minute time in-
tervals is referred to as time 0 and the second mea-
surement as time 1.
Data Analysis. Reliability of the VAS was as-
sessed in two ways. Following classic measure-
ment theory,13 correlations between VAS measures
taken 1 minute apart were calculated. However,
intraclass correlation coefficients (ICCs) were used
rather than Pearson productmoment correlations
because the ICC can reach 1.0 only when the two
measurements are identical (y intercept of 0, slope
of 1.0).12 In contrast, a Pearson productmoment
correlation value of 1.0 requires only that the mea-
surements be perfectly correlated, but not neces-
sarily identical. Intraclass correlation coefficients
of 0.75 and above are considered to be excellent.12
The magnitude of correlation coefficients is af-
fected by the range of scores included in the sam-
Bijur et al. RELIABILITY OF VAS FOR ACUTE PAIN
ple, increasing as the range of scores increases. To
adjust for this, ICCs were calculated separately for
VAS scores above and below 50 mm. Multiple re-
gression analyses were performed for each of the
time periods to assess whether differences between
1-minute measures of pain on the VAS were asso-
ciated with sex, age, and location of the pain cat-
egorized as abdominal/genitourinary, extremity,
and other.
The Bland-Altman analysis consists of calculat-
ing the difference between paired pain scores 1
minute apart. The distribution of the differences is
examined and the differences are plotted against
the average of the two scores under the assump-
tion that the mean is the best measure of the true
pain score. In our data the differences between the
1-minute scores were not normally distributed.
Therefore, the median and the ranges of differ-
ences between measures made 1 minute apart that
included 50%, 90%, and 95% of the differences
were calculated as recommended for differences
that are not normally distributed.14
The number of VAS differences greater than or
equal to 10 mm per patient was calculated in order
to assess whether the most discordant VAS scores
taken 1 minute apart were due to individual pa-
tients who could not appropriately perform the
task of rating their pain on the VAS.
SPSS (Version 9, SPSS Inc., Chicago, IL) pro-
vided all descriptive and inferential statistics re-
ported.
RESULTS
Ninety-six patients participated in the study, pro-
viding 432 paired measures on the VAS obtained 1
minute apart. Mean age of the population was 37
years (range 1971 years), 55% of whom were fe-
male. The most common locations of pain were the
abdomen/genitourinary tract and extremities.
The ICCs between VAS scores 1 minute apart
were between 0.95 and 0.98 with very narrow con-
fidence intervals at all five time points (Table 1).
As expected, the correlations are lower in each of
the two subsets of VAS scores dichotomized at 50
mm. However, all ICCs remain in the range con-
sidered to represent excellent reliability.12 There
were no significant associations between sex, age,
and location of pain and the difference between the
VAS scores.
The Bland-Altman plot (Fig. 1) displays the dif-
ferences between VAS ratings over a 1-minute time
interval plotted against the average of the two VAS
ratings. The differences between ratings 1 minute
apart ranged from 48 mm to 47 mm. The median
difference was 0. This suggests that time 1 ratings
of pain were not systematically lower or higher
than time 0 ratings. Fifty percent of the ratings
ACADEMIC EMERGENCY MEDICINE December 2001, Volume 8, Number 12 1155

TABLE 1. Relationship between Time 0 and Time 1 Pain Measures

Intraclass Correlation for VAS Ratings in the Following Ranges:
Intraclass
Correlation 049 mm 50100 mm

Time (Min) n r 95% CI n r 95% CI n r 95% CI

0 and 1 96 0.95 (0.93, 0.97) 35 0.72 (0.51, 0.84) 61 0.85 (0.76, 0.91)
30 and 31 95 0.98 (0.97, 0.99) 36 0.95 (0.91, 0.98) 59 0.89 (0.82, 0.93)
60 and 61 92 0.97 (0.96, 0.98) 38 0.97 (0.95, 0.99) 54 0.84 (0.73, 0.90)
90 and 91 77 0.98 (0.97, 0.99) 35 0.89 (0.80, 0.94) 42 0.94 (0.90, 0.97)
120 and 121 72 0.96 (0.94, 0.97) 36 0.81 (0.67, 0.90) 36 0.84 (0.71, 0.92)

TOTAL 432 0.97 (0.96, 0.98) 180 0.88 (0.84, 0.91) 252 0.87 (0.84, 0.90)

taken 1 minute apart were within 2 mm of each
other, 90% were within 9 mm, and 95% were
within 16 mm.
It can be seen from Figure 1 that the 1-minute
ratings of pain were more reproducible at the ex-
tremes of pain intensity than at moderate levels of
pain. For the 88 one-minute differences associated
with average pain of 20 mm or less, 95% were be-
tween 9 and 4 mm. Similarly 95% of the 130 dif-
ferences associated with average pain 80 mm and
above were between 7 and 4 mm.
In order to determine whether the most discor-
dant ratings came from a small group of patients
who may not have understood or were unable to
use the VAS, the number of VAS differences of 10
mm or more were analyzed separately. Only two
patients had three VAS differences 1 minute apart
that were 10 mm or higher, ten had two VAS dif-
ferences 10 mm or higher, and 20 had a single VAS
difference 10 mm or higher. Sixteen of the discor-
dant ratings were at baseline, 11 at 30 minutes,
six at 60 minutes, five at 90 minutes, and eight at
120 minutes.
DISCUSSION
The VAS is generally regarded as a valid and re-
liable tool for chronic pain measurement.25 Al-
though it appears to be equally valid in acute pain
measurement,68 to the best of our knowledge, its
reliability has not been assessed in this setting. We
wished to examine the reliability of the VAS in
acute pain, using two different methodologies that
would be less likely to inflate its reliability than
the traditional Pearson correlation coefficient.9,12
The ICCs (Table 1) suggest that, in the aggre-
gate, the reliability of the VAS for acute pain mea-
surement in the ED setting is high.
An examination of the Bland-Altman plot, how-
ever, suggests that the VAS may not be as reliable
as the ICCs would indicate. Although 50% of the
ratings are within 2 mm of each other, and 90%

Figure 1. Bland-Altman plot: differences between paired visual analog scale (VAS) scores obtained 1 minute apart
by the average of the two scores. Dotted lines indicate the interval that includes 95% of differences between time
0 and time 1 VAS scores; solid lines indicate the interval that includes 90% of differences between time 0 and time
1 VAS scores.
1156 VAS
are within 9 mm, there remain about 5% of paired
measurements whose differences encroach upon
the threshold of the minimum clinically significant
difference in pain (approximately 13 mm). This is
consistent with the findings of DeLoach et al., who
reported that 95% of the differences between acute
postoperative pain measured 3 minutes apart were
within 18 mm.15 The magnitude of measurement
error found in this study and that of DeLoach et
al. can be used to explain the consistent finding of
discrepancies between changes in VAS scores and
verbal descriptions of pain in a small proportion of
patients (e.g., a VAS difference indicating a 15-mm
decrease in pain described as a little more
pain).68 some of the larger differences in pain measured 1
Because most treatment studies select patients
with substantial pain, it is of particular interest
that the correlation of VAS scores taken 1 minute
apart was high, 0.87, for initial VAS scores of 50
mm or higher. Similarly, the greater reproducibil-
ity of pain ratings among patients with high rather
than moderate levels of pain suggests that even
those in extreme pain can make reproducible as-
sessments of acute pain severity. The finding that
the reproducibility was greatest for the lowest and
highest pain intensities may reflect a floor and ceil-
ing effect. Alternatively, it may simply be easier to
quantify the absence of significant pain and the
presence of intense pain, than to assess moderate
pain.
LIMITATIONS AND FUTURE QUESTIONS
Although the high ICCs and the finding that 90%
of the paired ratings were within 9 mm of one an-
other suggest that the VAS is a reliable measure
of acute pain for the majority of ED patients, about
5% of the pain ratings did differ, on at least one
occasion, by more than what is generally consid-
ered to represent a clinically significant difference
in pain, approximately 13 mm. Differences in re-
producibility on the VAS over a short time frame
can reflect several sources of error, including: 1)
difficulty in translating the subjective experience
of pain into a distance measure on a quantitative
scale; 2) inability to make an accurate mark on the
VAS due to motor, cognitive, or visual impairment;
and 3) lack of sufficient effort to appropriately com-
plete the task due to pain or cultural and other
behavioral characteristics. However, all of these
patient-related factors would be expected to affect
measurements over the entire two-hour time pe-
riod, not just in one or two half-hour periods when
the VAS scores were obtained. Our data indicate
that this is not the case, since only two patients
had more than two ratings with pain differences of
more than 9 mm.
It is also possible that pain can vary over the
Bijur et al. RELIABILITY OF VAS FOR ACUTE PAIN
course of a minute, and that lack of agreement be-
tween VAS scores 1 minute apart is not measure-
ment error, but rather due to fluctuating pain,
such as might occur with intermittent small bowel
obstruction. Due to the episodic nature of pain ex-
perienced in the ED setting, it is plausible that
some pain measurements obtained in studies com-
paring different treatment modalities will reflect
rapidly changing levels of pain rather than treat-
ment effects. In small studies, this phenomenon
might bias conclusions. In large trials, however,
randomization should distribute patients with rap-
idly fluctuating pain equally among treatment
groups. We cannot rule out the possibility that
minute apart simply reflect random variation,
rather than true change in the experience of pain.
The actual measurement properties of a VAS for
pain can be best studied with an experimental de-
sign in which the pain stimulus is standardized by
the investigators. However, the purpose of this
study was to assess the reliability of the VAS in
the ED in order that conclusions drawn from treat-
ment studies of acute pain using this instrument
would be informed by an estimate of its measure-
ment error. In this natural setting pain does fluc-
tuate. Although most pain ratings are within 9 mm
of each other, any study of pain in the ED must
take this fluctuation into account. This limitation
is not apparent from the ICC, but does become ev-
ident when the Bland-Altman plot, which graphi-
cally displays all the data, is inspected.
There were no significant differences in relia-
bility by location, age, or gender. However, the lack
of detail about type of pain other than location lim-
its the ability to assess whether there are any dif-
ferences in reproducibility of the VAS by type of
pain. A comparison of VAS scores and verbal rat-
ings from this data set reported elsewhere8 with
Todd et al.s findings on a group of patients with
trauma,6 and Kellys Australian patients with a
mix of different pain etiologies7 indicated similar
minimum clinically significant differences despite
major differences in the populations studied and
casemixes. This suggests that our findings may be
generalizable to settings with varying casemixes
and populations, although it is possible that mea-
sures of certain types of pain may be more or less
reproducible than what we found. Similarly, our
experience in an urban setting with Spanish- and
English-speaking patients may not be generaliza-
ble to all settings.
CONCLUSIONS
The findings from this study indicate that the VAS
is a highly reliable instrument for measurement of
acute pain. For 90% of paired measurements, dif-
ACADEMIC EMERGENCY MEDICINE December 2001, Volume 8, Number 12
ferences between scores obtained 1 minute apart
were 9 mm or less. The clinical significance of this
finding is that if a VAS were used to measure
change in pain in individual patients, change of 10
mm or more would be likely to indicate a true
change in the experience of pain for most patients.
For research purposes the findings from this study
suggest that mean differences between groups of
patients smaller than 10 mm are within the error
of the method, and should be interpreted with cau-
tion.
For about 5% of measurements, there appeared
to be a clinically significant difference between two
VAS ratings made by the same patient 1 minute
apart.68 The pattern of this finding suggests that
a change of this magnitude in VAS score may rep-
resent clinically significant fluctuation in acute
pain occurring in a small proportion of patients
over a 1-minute interval.
References
1. Ducharme J. Proceedings from the First International Sym-
posium on Pain Research in Emergency Medicine: Foreword.
Ann Emerg Med. 1996; 27:399403.
2. Downie WW, Leatham PA, Rhind VW, Wright V, Branco JA,
Anderson JA. Studies with pain rating scales. Ann Rheum Dis.
1978; 37:37881.
1157
3. Scott J, Huskisson EC. Vertical or horizontal visual ana-
logue scales. Ann Rheum Dis. 1979; 38:560.
4. McCormack HM, Horne DJL, Sheather S. Clinical applica-
tions of visual analogue scales: a critical review. Psychol Med.
1988; 18:100719.
5. Gaston-Johansson F. Measurement of pain: the psychomet-
ric properties of the Pain-O-Meter, a simple, inexpensive pain
assessment tool that could change health care practices. J Pain
Symptom Manage. 1996; 12:17281.
6. Todd KH, Funk KG, Funk JP, Bonacci R. Clinical signifi-
cance of reported changes in pain severity. Ann Emerg Med.
1996; 4:4859.
7. Kelly AM. Does the clinically significant difference in visual
analog scale pain scores vary with gender, age, or cause of
pain? Acad Emerg Med. 1998; 11:108690.
8. Libman M, Berkoff D, Lahn M, Bijur P, Gallagher EJ. In-
dependent validation of the minimum clinically important
change in pain scores as measured by visual analog scale [ab-
stract]. Acad Emerg Med. 2000; 7:550.
9. Bland JM, Altman DG. Statistical methods for assessing
agreement between two methods of clinical measurement. Lan-
cet. 1986; 1:30710.
10. Bland JM, Altman DG. Measurement error and correlation
coefficients. BMJ. 1996; 313:412.
11. Porter AMW. Misuse of correlation and regression in three
medical journals. J R Soc Med. 1999; 92:1238.
12. Fleiss JL. The Design and Analysis of Clinical Experi-
ments. New York: John Wiley & Sons, 1986.
13. Crocker L, Algina J. Introduction to Classical and Modern
Test Theory. New York: Harcourt Brace Jovanovich, 1986.
14. Bland JM, Altman DG. Measuring agreement in method
comparison studies. Stat Methods Med Res. 1999; 8:13560.
15. DeLoach LJ, Higgins MS, Caplan AB, Stiff JL. The visual
analog scale in the immediate postoperative period: intrasub-
ject variability and correlation with a numeric scale. Anesth
Analg. 1998; 86:1026.