Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
have considerable experience in validating blood pressure 3. Validation measurements. Observer and device measure-
measuring devices. ments are recorded on subjects in two phases. In the
first phase, 15 subjects are recruited; devices passing
In setting about its objective, the ESH Working Group this primary phase proceed to the secondary phase, for
recognized the urgent imperative to provide a simplified which a further 18 subjects are recruited.
protocol that does not sacrifice the integrity of the earlier 4. Analysis. An analysis of the recorded measurements is
protocols. When the AAMI and BHS protocols [2–5] were carried out after each phase.
published, the relevant committees did not have evidence 5. Reporting. The results are presented in tabular and
from previous studies on which to base their recommenda- graphical forms.
tions. The ESH Working Group has had the advantage of
being able to examine and analyze the data from 19
validation studies performed according to the AAMI and Observer training and assessment
BHS protocols at the Blood Pressure Unit in Dublin [7– Consideration must first be given to the technique of blood
23]. A critical assessment of this database of evidence has pressure measurement, which should be as follows
permitted a rationalization and simplification of validation throughout the validation procedure.
procedures without losing the merits of the much more
Blood pressure measurement technique
complicated earlier protocols. The basic recommendations
of the simplified International Protocol have been pre- A standard mercury sphygmomanometer, the components
sented at meetings of the ESH Working Group, and the of which have been carefully checked before the study, is
proceedings of these meetings have been published in used as a reference standard. It is appreciated that terminal
order to invite comment and discussion [24–27]. digit preference is a problem with conventional mercury
sphygmomanometry, and care should be taken to reduce
The International Protocol has been drafted in such a way this in the observer training session. The Hawksley
as to be applicable to the majority of blood pressure random-zero sphygmomanometer only disguises digit pre-
measuring devices on the market. The validation procedure ference, and its accuracy has been questioned [7,28];
is therefore confined to adults over the age of 30 years (as therefore, its use is not recommended in validation studies.
these will constitute the majority of subjects with All blood pressures should be recorded to the nearest
hypertension), and does not make recommendations for 2 mmHg.
special groups, such as children, pregnant women and the
elderly, or for special circumstances, for example exercise. Blood pressure should be measured with the arm supported
It is anticipated that the relative ease of performance of the at heart level [29]; the level of the manometer does not
International Protocol will encourage manufacturers to affect the accuracy of measurement, but it should be at eye
submit blood pressure measuring devices for validation in level and within 1 m of the observer. The quality of the
order to obtain the minimum approval necessary for a stethoscope is also crucial to performing the evaluation
device to be used in clinical medicine, and that, in time, procedure. Stethoscopes with badly fitting earpieces and
most devices on the market will be assessed according to poor-quality diaphragms preclude precise auscultation of
the protocol for basic accuracy. It does not preclude the Korotkoff sounds. A well-maintained quality stetho-
manufacturers of devices from subjecting their products to scope is recommended.
more rigorous assessment and validation.
Observer training
Validation procedure The first prerequisite for this validation test is to ensure
Summary that the observers have adequate auditory and visual acuity,
The validation team should consist of four persons and that they have achieved the required accuracy as laid
experienced in blood pressure measurement: two observers out below. It is, however, possible that observers who fulfil
and a supervisor (generally nurses), and an ‘expert’ (a these criteria at the outset of the study will not do so at the
doctor overseeing the entire procedure). If the doctor can end, and if this happens the observers must be re-assessed
be present throughout the entire validation procedure, he/ for accuracy. To avoid this, analysis should be performed as
she can take over the role of supervisor, thereby reducing the study proceeds to detect any drift in agreement
the number of personnel to three. The validation between the observers.
procedure consists of the following steps
Observers may be trained in the following ways:
1. Observer training and assessment. Two observers are
trained in accurate blood pressure measurement. 1. By fulfilling the test requirements of the CD-ROMs
2. Familiarization session. The validation team becomes produced by the BHS or the Société Française
familiar with the workings of the device and the d’Hypertension Artérielle as described in Appendix B
accompanying software. [30,31].
Protocol for validating BP measuring devices in adults ESH Working Group 5
2. By formal training and assessment is described in test device recommends different cuff sizes, the appro-
Appendix B [32, 33]. priate cuff/bladder should be used, but no other part of the
3. By using an audio-visual method for validation, such as apparatus should be changed. It is important to ensure,
the Sphygmocorder [34,35] as discussed in Appendix when assessing auscultatory devices, that the same
B. microphone(s) are used throughout the validation test.
Table 1 Blood pressure ranges for entry blood pressure (BPA) BPB Device detection blood pressure, observer 3. This
SBP DBP blood pressure is measured to allow the test
instrument to determine the blood pressure
Low 90^129 40^79
Medium 130^160 80^100 characteristics of the subject; more than one
High 161^180 101^130 attempt may be needed with some devices; this
For the primary phase, ¢ve of the 15 subjects should have a systolic blood pressure
measurement is not included in the analysis. If
(SBP) in each of the ranges. Similarly, ¢ve of the 15 subjects should have a diastolic the device fails to record a measurement after
blood pressure (DBP) in each of the ranges. For the secondary phase,11 of the 33 three attempts, the subject is excused.
subjects (including the ¢rst 15 subjects) should have SBP and DBP in each of the
ranges.It is recommended that recruitment should commence by targeting subjects
BP1 Observers 1 and 2 with the mercury standard.
likely to have pressures in the low-systolic and high-diastolic ranges, then progres- BP2 Supervisor with the test instrument.
sing to complete the high-systolic and low-diastolic ranges so that it will be easy to BP3 Observers 1 and 2 with the mercury standard.
complete the recruitment with the remaining medium ranges.
BP4 Supervisor with the test instrument.
BP5 Observers 1 and 2 with the mercury standard.
blood pressure at the time of the validation procedure BP6 Supervisor with the test instrument.
(BPA), rather than that at the time of recruitment for BP7 Observers 1 and 2 with the mercury standard.
validation.
3. Documentation must be provided for data omitted for
legitimate technical reasons. Once a subject has been
Observer measurement included, the data for that subject should not be
Measurements can be either assessed live using two excluded from the study if blood pressure values are
observers or recorded and later re-assessed using the obtainable; if blood pressure measurements using
Sphygmocorder [34,35]. Measurements made simulta- either the reference method or the test instrument
neously by two observers must be checked by the are unavailable, data entry for that individual may be
validation supervisor. If the systolic and diastolic measure- excluded, with an accompanying explanation. Addi-
ments are no more than 4 mmHg apart, the mean value of tional individuals must then enter into the study to
the two observer measurements for both systolic and ensure a sample size of 33.
diastolic blood pressures is used. Otherwise, the measure-
ment must be taken again.
Analysis
When the Sphygmocorder is used, two observers should For a detailed discussion on the statistical methods used in
assess the recording separately. If their opinion differs, they the protocol, see Appendix D. A software program has been
should re-assess the values together until agreement is designed specifically to analyze the data (Société Française
reached. Further references to ‘observer measurement’ d’Hypertension Artérielle, Paris).
indicate either the mean of two observer measurements or
the agreed measurement using the Sphygmocorder. At least Accuracy criteria
30 s should be allowed between each measurement to The BHS protocol introduced the concept of classifying
avoid venous congestion, but not more than 60 s or the differences between test and control measurements
variability may be increased. according to whether these lay within 5, 10 or 15 mmHg, or
were over 15 mmHg apart. The final grading was based on
Procedure the number of differences falling into these categories.
This protocol seeks to keep this concept but expand its
1. The subject is introduced to the observers, and the flexibility.
procedure is explained. Arm circumference, sex, date
of birth and current date and time are noted. The Differences are always calculated by subtracting the
subject is then asked to relax for 10–15 min (in order observer measurement from the device measurement.
to minimize anxiety and any white-coat effect, which When comparing and categorizing differences, their
will increase variability). absolute values are used. A difference is categorized into
2. Nine sequential same-arm measurements using the one of four bands according to its rounded absolute value
test instrument and a standard mercury sphygmoman- for SBP and DBP:
ometer are recorded as follows:
0–5 mmHg These represent measurements consid-
BPA Entry blood pressure, observers 1 and 2 each with ered to be very accurate (no error of
the mercury standard. The mean values are used clinical relevance).
to categorize the subject into a low, medium or 6–10 mmHg These represent measurements consid-
high range separately for SBP and DBP (Table 1). ered to be slightly inaccurate.
Protocol for validating BP measuring devices in adults ESH Working Group 7
The following information should be provided with blood Method(s) of blood pressure measurement
pressure measuring devices, and the final report should The basic method of pressure detection (e.g. auscultatory
acknowledge that such information is available, and or oscillometric) should be stated, and if more than one
although this need not be presented in detail, any method is used, the indications for changing methods and
deficiencies should be listed in the report. the means of denoting this on the recording should be
stated. With Korotkoff sound-detecting devices, whether
Basic information phase IV or phase V is being used for the diastolic end-
The information provided in operational manuals is often point must be disclosed. If data are derived from recorded
deficient. Without appropriate specifications and opera- measurements, such as mean pressure, the method of
calculation must be stated.
Fig. 1
Computer analysis
Some automated systems are compatible with personal
computer systems. The exact requirements for linking with
computer systems and their approximate cost should be
stated. If the automated system is dependent on its own
computer for plotting and analysis, this should be made
clear, and the cost of the computer facility, if it is an
optional extra, should be stated.
mechanical blood pressure measuring systems. European Committee for of factors may make it difficult or impossible to perform
Standardization; 1997.
simultaneous comparison on the same arm.
43 O’Brien E. Modi¢cation of blood-pressure measuring devices and the
protocol of the British Hypertension Society. Blood Press Monit 1999;
4: 53^54.
Devices, for example, that deflate at a rate of more than
44 European Commission for Standardisation. European Standard EN 540.
Clinical investigation of medical devices for human subjects. European 5 mmHg per second do not permit accurate measurement
Commission for Standardisation; 1993. by an auscultating observer, leading to inaccurate compar-
45 West NW, Sheridan JJ, Littler WA. Direct measurement of blood pressure.
In: O’Brien E, O’Malley K, editors: Handbook of hypertension. Vol. 14.
ison between the test and reference devices [4]. At fast
Blood pressure measurement. Amsterdam: Elsevier; 1991. pp. 315^333. deflation rates, an auscultating observer will tend to
46 Parati G, Casadei R, Gropelli A, Di Rienzo M, Mancia G. Comparison of underestimate SBP and overestimate DBP by recording
¢nger and intra-arterial blood pressure monitoring at rest and during
laboratory testing. Hypertension 1989; 13: 647^655. the first definite pressure phase at which Korotkoff sounds
47 O’Brien E, Atkins N, Mee F, O’Malley K. Comparative accuracy of six are audible as the systolic value and the last definite phase
ambulatory devices according to blood pressure levels. J Hypertens
1993; 11: 673^675.
of audible sounds as the diastolic. The device may have a
facility for slowing the rate of deflation so that the
simultaneous comparison can be performed, but this is
not permissible as any modification of the usual operational
Appendix A. Comparison with previous mode may alter its accuracy.
protocols
Our approach to simplifying previous validation procedures
Other factors that may preclude simultaneous same-arm
has concentrated on the following areas:
testing are confusion of noise from the device with
Korotkoff sounds, failure of the inflating mechanism to
Elimination of pre-validation phases reach the required pressure, sudden deflation before DBP
The main validation procedure of the existing BHS can be confirmed and uneven deflation, making accurate
protocol has five phases: (i) before-use device calibration; auscultation impossible. The most important objection to
(ii) the in-use (field) phase; (iii) after-use device calibra- simultaneous comparisons is that true simultaneous
tion; (iv) static device validation; and (v) report of the measurement cannot be achieved with oscillometric
evaluation (4). Phases (i)–(iii) were originally introduced devices, which now constitute virtually all automated
to identify intra-device variability, but if a device has devices available for blood pressure measurement.
fulfilled the general requirements of the European Union
directives [40–42] or the AAMI standard [5], it is not Simultaneous opposite-arm comparisons are not permitted
necessary to subject these devices to phases (i), (ii) or (iii) because the blood pressure difference between the arms is
of the BHS protocol. These pre-validation phases are thus a variable rather than a constant factor, and the measure-
not included in the present protocol, thereby resulting in ments are not truly simultaneous. To overcome the
considerable reduction in time and labor. problems associated with simultaneous measurements in
either the same or opposite arms, sequential testing is
Improving observer recruitment and training advocated in this protocol.
The most fallible component of blood pressure measure-
ment is the human observer, and consideration must be
Minimizing observer error during validation
given to the role of the education and certification of The role of the supervisor has been modified from that in
observers. CD-ROMs are available to facilitate the training the BHS protocol [4] so that he or she observes the result
and assessment of observers [30,31]. of each paired measurement made by observers 1 and 2,
and if either the SBP or DBP values are more than 4 mmHg
The Sphygmocorder, a device that provides an audio apart, the supervisor will simply state that the measure-
recording of Korotkoff sounds with a video recording of a ment must be taken again, without giving a reason, so that
mercury column, has been designed to provide objective neither observer will be biased when re-taking the blood
evidence of validation blood pressures [34,35]. The pressure. In this way, errors will be minimized. Experience
Sphygmocorder removes the expensive need to employ has shown, for example, that errors of 10 mmHg can be
two observers and a supervisor throughout the validation made by observers simply misreading the mercury column.
procedure and has greatly facilitated device validation. Another change in the protocol has been to use the mean of
the two observers’ results rather than analyzing the results
for each observer separately, these mean values being
Use of simultaneous or sequential comparisons
The basis of device evaluation is the comparison between referred to simply as ‘observer measurements’.
blood pressure measured by the device being tested and
measurements made by trained observers using a mercury Reduction in the number of subjects recruited
sphygmomanometer and stethoscope to auscultate the Reducing the number of subjects required for validation
Korotkoff sounds. With most automated devices, a number would greatly simplify the procedure, and there are now
12 Blood Pressure Monitoring 2002,Vol 7 No 1
sufficient data from the many validation studies performed fail, a tolerance factor whereby one of the above targets is
to review the number of subjects required [7–23]. not achieved for five measurements is allowed.
The first AAMI protocol required a sample of 85 subjects, Algorithm integrity and design modi¢cation
the paired measurements being averaged to give a total of 85 The first BHS protocol emphasized the importance of
paired comparisons [2]. The BHS protocols [3,4] and the manufacturers indicating by a change in model number any
revised AAMI protocol [5] did not average the values, modifications made to blood pressure measuring devices
leaving 255 sets of measurements for analysis. In the current [3]. The revised BHS protocol, published in 1993, went
protocol, reducing the number of paired measurements to 99 further by stipulating not only that manufacturers must
(which allows for easy conversion to equivalent percentage indicate clearly all modifications to the technological and
values) brings the sample size back to the original AAMI software components of automated devices by changing
recommendation but reduces the number of subjects to 33. the device number, but also that modified devices must be
Reducing the number of subjects results, of course, in some subjected to renewed validation [4]. These stipulations
loss in measurement independence, but an analysis of 19 were influenced by consequences that had resulted from
validation studies has shown that reducing the number of changes made by manufacturers to the algorithms of
subjects recruited from 85 to 33 is possible without affecting devices for measuring ambulatory blood pressure [43].
the accuracy of the validation (Appendix D) [7–23]. Manufacturers have, however, from time to time expressed
the view that the BHS stipulations were unreasonable, in
The subjects should be at least 30 years of age in order to that they obliged the manufacturer to go to the unneces-
ensure those recruited are representative of the age range sary expense of re-evaluating a device that had undergone
in which most hypertensive patients lie. some design modifications without any alteration of the
algorithm. Moreover, the stipulation might inhibit bene-
Relaxing the range of blood pressures ficial modifications to device design, which need not
Experience has shown that recruiting subjects at the involve adjusting an algorithm previously shown to have
extremes of high and low pressure is impractical. Further- fulfilled the accuracy criteria of the protocol. This
more, as blood pressure variability is greater at these stipulation remains in principle in the present protocol
extremes, sequential comparisons may become unreliable. but can be waived if a manufacturer of a device that has
The relaxation of these requirements to those shown in previously fulfilled the accuracy criteria of the protocol can
Table 1 above, with an equal number of subjects being provide the following: (1) independent evidence that the
recruited to each range, facilitates the validation procedure algorithm in the modified device is identical to that in the
without unduly affecting the results. originally validated model; (2) evidence that the proposed
modifications cannot alter the performance of the algo-
rithm; a system of model numbering that (3) acknowledges
Eliminating ‘hopeless’devices a common algorithm and (4) denotes the features of the
Our data support dividing the validation process into two
modification [43].
phases. In the primary phase, three pairs of measurements
are performed in 15 subjects in the stipulated pressure
ranges, any device failing this phase being eliminated from Intra-subject variability
further testing. Devices passing this phase then proceed to The influence of intra-subject variability is substantial and
a secondary phase, a further 18 subjects (to provide a total can disadvantage devices, particularly when sequential
of 33) being recruited, in whom comparisons must fulfil the measurements differ by over 10 mmHg, as happens
criteria shown in Table 2. These alterations do not especially in the higher pressure ranges. Two simple
substantially alter the results of the validation studies measures to cope with this problem have been incorpo-
examined, but by eliminating ‘hopeless’ devices at an early rated into this protocol.
stage, the validation process is simplified and unnecessary 1. Exclusion of subjects with extremely high and extremely low
testing avoided [7–23]. pressures. Not only do measurements in these ranges
tend to vary considerably, but also large differences,
Expression of validation results which would be substantial in the mid-range pressures,
In this protocol, the BHS grading system and AAMI are in practice unlikely to affect treatment at these
assessment according to the mean and standard deviation extremes.
of the differences have been abandoned in favour of a 2. Tolerance for comparative differences over 15 mmHg. It must
straightforward pass/fail system. Moreover, a degree of be accepted that sequential measurements may vary
tolerance in deciding the pass/fail category has been quite considerably in some subjects, especially at high
incorporated into the protocol. Ideally 65, 80 and 95 of pressures, and that these are not errors. An analysis of
the 99 measurements should lie within 5, 10 and 15 mmHg previous studies has shown that sequential SBP
respectively, but because a device might only marginally measurements typically lie within 5, 10 and 15 mmHg
Protocol for validating BP measuring devices in adults ESH Working Group 13
of each other 75, 93 and 97% of the time, respectively. Observer assessment
The mean difference is typically 1 mmHg, with a As as alternative to self-assessment, observers can be tested
standard deviation of around 5 mmHg. formally as in the BHS protocol [4].
Audio-visual techniques
Training observers as described above is a labour–intensive
procedure, and even when observers are instructed to a
Diagram of observer assessment procedure high degree of accuracy, there is the problem of maintain-
ing that accuracy throughout the study [32,33].
14 Blood Pressure Monitoring 2002,Vol 7 No 1
A need has been recognized, therefore, for an electronic device performance in tracking fast beat-by-beat blood
audio visual system to measure blood pressure in validation pressure changes (46).
studies that is not dependent on observers but will
nevertheless retain the traditional auscultatory methodol-
ogy using the mercury sphygmomanometer. An example of Appendix D. Statistical considerations
such is the Sphygmocorder which was developed for this Sample size
purpose and, since it was first described in 1995 [34], a The AAMI published its first protocol for the validation of
number of improvements have been made to the system blood pressure measuring devices in 1987 [2]. The
[35]. This system is being developed for commercial accuracy component of the protocol basically consisted of
distribution. a comparison of the mean of three test device measure-
ments with simultaneous observer measurements, measur-
ing blood pressure with a mercury sphygmomanometer, on
Appendix C. Intra^arterial comparison each of eighty-five subjects. The selection of 85 subjects
The ESH Working Group agrees with the stipulations of was made on the ability to detect a somewhat arbitrary
the previous BHS protocol that intra-arterial comparisons error of 5 7 8 mmHg at a significance level of 0.05 and a
should not be recommended for general validation, while power of 0.98. The calculation was based on independent,
acknowledging that intra-arterial comparisons may in some rather than paired, samples for comparison, thereby
instances give information that cannot be obtained non- allowing for the fact that devices and observers may not
invasively [4]. If, however, intra-arterial comparisons are to measure blood pressure on exactly the same heart beat
be performed, they should be confined to centers with even when using simultaneous readings.
proven expertise in the technique, and the requirements of
EN 540, Clinical investigation of medical devices for human A blood pressure measuring device could pass the AAMI
subjects,, which requires among other stipulations that the protocol, but still be inaccurate. The BHS protocol
World Medical Declaration of Helsinki is fulfilled, that the identified two difficulties [3,4]. The first was that only
Ethics Committee be provided with information to assess average measurements were used in the analysis whereas
whether the risks to subjects, who cannot be expected to individual measurements would be identified in practice.
derive any direct therapeutic benefit, can be justified by The second was that, in using means and standard
the collective benefit, that provisions have been made to deviations, the percentage of measurements required to
compensate subjects in the event of injury, and that full be reasonably accurate, that is lying within 5 mmHg, was
informed consent is obtained from all subjects [44]. insufficient. Paradoxically, few outlying measurements are
permitted in the normal model whereas a more relaxed
A comparison between blood pressure measuring systems approach may be necessary in practice as variability can be
that utilize indirect measurement and those using the considerable in some subjects and may make a truly
direct intra-arterial measurement of blood pressure is not accurate reading appear otherwise.
recommended in this protocol. Apart from ethical con-
siderations, there are several reasons for this. Systolic and When the first BHS protocol was published in 1990 [3], the
diastolic blood pressure values obtained by the direct requirement to take three simultaneous measurements on
technique differ from measurements obtained by indirect each of 85 subjects was therefore retained, but the
methods [4,46]. Clinical practice derives from data measurements were no longer averaged, thus giving 255
obtained by the indirect rather than the direct technique. pairs of measurements for comparison. The accuracy
There is considerable beat-to-beat variation in blood criteria were based on the percentage of measurements
pressure, which is not reflected in indirect readings. Blood lying within 5, 10 and 15 mmHg. Furthermore, the
pressures measured directly and indirectly from the same possibility of device-induced bias was highlighted with a
artery are rarely (if ever) identical. Discrepancies in SBP as recommendation that bracketing sequential measurements
great as 24 mmHg and in DBP as much as 16 mmHg have be used as an alternative to simultaneous measurement. A
been observed when blood pressure has been measured by grading system was introduced to describe accuracy [3].
both techniques on the same arm at the same time. In
addition, these differences are random, displaying no In its revised protocol in 1993, the AAMI also recom-
schematic pattern [4,45]. mended that measurements no longer be averaged; it also
permitted the sequential technique when simultaneous
It is, however, recognized that valuable information on measurements were not feasible [5]. The 5 7 8 mmHg
device performance may derive from intra-arterial compar- accuracy criteria were retained.
isons in certain circumstances, such as validating devices
that analyse beat-by-beat blood pressure non-invasively, It has proved extremely difficult to recruit 85 subjects
but the International Protocol would need to be modified within the pressure range requirement of the previous
procedurally to allow intro-arterial comparisons and to test protocols; in practice, more than 100 subjects have been
Protocol for validating BP measuring devices in adults ESH Working Group 15
needed to fulfil the pressure range stipulations. A number preferable to a mean and standard deviation method of
of factors were considered in reducing sample size. validation.
1. The original statistical criteria were based on 85
Pressure ranges
measurements [2] whereas later protocols used 255
Prior to the introduction of the 1993 BHS protocol [4],
[3–5].
there was no specific recommendation on the range of
2. For grading results, percentage values are conceptually
blood pressure required for validation. As a consequence,
the most appropriate.
these varied greatly from one validation to the next. As
3. The more practical the study is, the more easily and
most devices fared worse in the high pressure ranges, this
more often it will be performed.
reduced the reliability and comparability of results [47].
Having performed 19 studies [7–23], we were able to re-
analyze the data from these studies to check the validity of To redress this problem, specific ranges were introduced.
new proposals. Taking all factors into consideration, the In particular, at least eight subjects in both the hypotensive
most appropriate sample size was 99 measurements, which and severe hypertensive ranges had to be recruited. The
provides more than the 85 measurement pairs required in reasoning behind the inclusion of the hypotensive range
the previous BHS and AAMI protocols [4,5] but with a was not only to assess accuracy in subjects with hypoten-
sample size of only 33 rather than 85 subjects. Although sion, but also to give some indication of accuracy for
there is some loss of measurement independence, the devices measuring ambulatory blood pressure during sleep
results compare well with independent measurements when the values can fall to low levels. Subjects with severe
[7–23]. hypertension were included because such levels are quite
common in hypertension clinics.
The selection of 33 subjects is based on two factors. First,
In practice, however, these two groups have proved
each subject has three measurements, each of which is
extremely difficult to find. The prevalence of persistent
used individually. This gives 99 sets of measurements,
hypotension is very low, and whereas severely hypertensive
which is larger than the 85 sets of measurements accepted
patients were to be found in specialist clinics, the
as the minimum necessary in the AAMI and BHS protocols
validation study had to be performed before blood
[4,5].
pressure-lowering drugs were prescribed, which was often
ethically impractical. Furthermore, during the resting
In comparing the variance of all 99 differences (total
laboratory phase of the validation procedure, blood
variance) with the 33 differences obtained from the mean
pressures in such subjects tended to fall below the
differences for each subject (the between-subject var-
required level required. Next and importantly, blood
iance), the F-test consistently yields a significantly lower
pressure in these subjects tended to be highly variable,
variance for the 33 subject mean differences than that for
making comparisons unreliable.
the 99 measurement differences. If between-subject
variance were the main cause of total variance, these
Finally, the division of subjects according to blood pressure
would not differ significantly. If, on the other hand, a
level did not lead to independent analysis. Indeed, tertile
device gave practically the same average error with each
analysis, included in the 1993 BHS protocol [4], was used
subject, the between-subject variation would be close to
only as a guide to accuracy, and the final recommendation
zero, and the F-test would show a very significant result.
was based on the overall analysis. The reason for this was
that most devices fared poorly in the upper tertile, with
Tests on data from previous experiments yield results of greater variability in this range being at least partly
probabilities of the between-subject variance and the total responsible [47].
variance being the same as lying between 0.1 and 0.01 for
SBP and between 0.2 and 0.02 for DBP. There should Given these difficulties and the fact that the comparisons
therefore be little difference between using single in these extremes are diluted in the overall analysis,
comparisons on 99 subjects and three comparisons on each specific requirements to include them are omitted in the
of 33 subjects. recommendations in this protocol. Although the range of
pressures has been reduced, all subjects must fit into a
The use of 99 subjects allows for an even distribution of specific category, whereas in the earlier protocols 10% of
blood pressures as these can easily be broken into three subjects could lie in any range (Table 1) [4].
ranges. It is also close to 100, which allows targets to be
considered as being approximate to percentages. Recommendations
The non-parametric recommendation system, shown in
Table 4(a–c) demonstrates how the choice-specific values Table 2a–c, considers both the subject/measurement and
for 5, 10 and 15 mmHg bands are more flexible and subject accuracy. White-coat hypertension and the morning
16 Blood Pressure Monitoring 2002,Vol 7 No 1
Table 4a Percentage of comparisons of devices satisfying Association for the Advancement of Medical Instrumentation criteria that fall
within a 5 mmHg error band
Within 5 mmHg Standard deviation (mmHg)
1 2 3 4 5 6 7 8
Mean (mmHg) 0 100.0% 98.6% 90.1% 78.5% 68.0% 59.3% 52.3% 46.6%
1 100.0% 97.4% 88.3% 77.1% 67.0% 58.7% 51.8% 46.3%
2 99.8% 93.1% 82.9% 73.0% 64.2% 56.7% 50.5% 45.4%
3 97.6% 84.0% 74.2% 66.6% 59.8% 53.7% 48.4% 43.8%
4 84.0% 69.1% 62.8% 58.5% 54.1% 49.7% 45.6% 41.8%
5 50.0% 50.0% 49.9% 49.3% 47.6% 45.0% 42.2% 39.3%
This table shows the expected percentage of errors of at most 5 mmHg for devices passing with mean absolute di¡erences of 0^5 mmHg and standard deviations of
1^8 mmHg.
Table 4b Percentage of comparisons of devices satisfying Association for the Advancement of Medical Instrumentation criteria that fall
within a 10 mmHg error band
This table shows the expected percentage of errors of at most 10 mmHg for devices passing with mean absolute di¡erences of 0^5 mmHg and standard deviations of
1^8 mmHg. The gray area indicates the improvement that might be obtained if the standard deviation is related to the mean, which in this instance is set so that the expected
number of di¡erences within 10 mmHg will be at least 85%.
Table 4c Percentage of comparisons of devices satisfying Association for the Advancement of Medical Instrumentation criteria that fall
within a 15 mmHg error band
Within15 mmHg Standard deviation (mmHg)
1 2 3 4 5 6 7 8
Mean (mmHg) 0 100% 100% 100% 100% 99.6% 98.6% 96.5% 93.6%
1 100% 100% 100% 100% 99.6% 98.4% 96.3% 93.4%
2 100% 100% 100% 99.9% 99.4% 98.1% 95.8% 92.8%
3 100% 100% 100% 99.8% 99.0% 97.4% 94.9% 91.8%
4 100% 100% 100% 99.6% 98.5% 96.4% 93.6% 90.4%
5 100% 100% 99.9% 99.3% 97.6% 95.0% 91.9% 88.5%
This table shows the expected percentage of errors of at most15 mmHg for devices passing with mean absolute di¡erences of 0 mmHg to 5 mmHg and standard deviations of
1^8mmHg. It shows that devices with 88.5% of measurements within this range could pass. One of the problems with the AAMI protocol is that, by setting the error indepen-
dently for mean and standard deviation, it permitted a very liberallevel of accuracy.FromTable 4a, it can be seen that where a device barely passes with a mean error of 5 mmHg
and a standard deviation of 8 mmHg, one could not expect even 40% of measurements to be accurate. Even devices passing more comfortably would have more than half of
theirexpected measurements classed asinaccurate.The acceptable standard deviation must be inversely related to the mean error.In practice, however, this tends not to be the
case as standard deviation tends to increase with error.This makes practical parametric passing criteria problematic.
alarm response are just two examples in which single When comparing a device measurement with its preceding
measurements are crucial. It is much easier for devices to and succeeding observer measurements, the nearer ob-
pass when only average subject measurements are used, server measurement is used. This poses a dilemma only if
and it would be wrong to assume that devices being the two observer measurements are equally close except for
recommended for use under such protocols are also sign, for example a device measurement of 150 mmHg and
accurate for individual measurements. observer measurements of 146 and 154 mmHg. One choice
would indicate that the device overestimates pressure
It must also be recognized that measurements near the whereas the other would indicate that it underestimates
extremes of the pressure range are more variable. Decisions pressure. The protocol recommends that whichever of the
should not therefore be based on small differences at these two observer measurements was taken first is selected.
limits, and zones are used to allow for this. The target This eliminates bias, and it is likely that the overestimating
requirements are based on existing protocols and the and underestimating selections will balance out over the 99
evidence from previous validation data. measurements.
Protocol for validating BP measuring devices in adults ESH Working Group 17
Membership of European Society Gianfranco Parati, Istituto Scientifico Ospedale San Luca,
of Hypertension Working Group on Blood IRCCS, Instituto Auxologico Italiano, 20149 Milan, Via
Pressure Monitoring Spagnoletto 3, Italy.
Roland Asmar, Société Française D’Hypertension Artérielle,
Fillale de la Société Française de Cardilogie, 15, rue de Paolo Palatini, Dipartimento di Medicina Clinica e
Cels-75014, Paris, France. Sperimentale, Universita’ di Padova, Via Giustiniani 2, I-
Lawrie Beilin, Department of Medicine, University of 35128 Padua, Italy.
Western Australia, Australia, GPO Box x2213, 35 Victoria
Square, Perth WA 600, Australia. Thomas G. Pickering, Director, Integrative and Behavioral
Cardiovascular Health Program, Mount Sinai Medical
Denis L. Clement, Afdeling Hart-en Vaatziekten, Uni-
Centre, New York, NY 10029-6574, USA.
versitair Ziekenhuis, De Pintelaan 185, B-9000 Gent,
Belgium.
Josep Redon, Hypertension Clinic, Internal Medicine,
Peter De Leeuw, Interne Geneeskunde, Academisch Hospital Clinico, University of Valencia, Avda Blasco
Ziekenhuis, P. Debyelaan 25, postbus 5800, 6202 AZ Ibañez, 17. 46010. Valencia, Spain.
Maastricht, The Netherlands.
Jan Staessen, Katholieke Universiteit Leuven, Hypertensie
Robert Fagard, Katholieke Universiteit Leuven, Hyperten- en Cardiovasculaire Revalidatie Eenheid, Inwendige Gen-
sie en Cardiovasculaire Inevalidatie Eenheid, Inwendige eeskunde-Cardiologie, U.Z. Gasthuisberg, Herestraat 49,
Geneeskunde-Cardiologie, U.Z. Gasthuisberg, Herestraat 3000 Leuven, Belgium.
49, 3000 Leuven, Belgium.
George Stergiou, Hypertension Center, Third University
Yutaka Imai, The Department of Clinical Pathology and Department of Medicine, Sotiria Hospital, Athens, Greece.
Therapeutics, Tohoku University Graduate School of
Pharmaceutical Science and Medicine, 1-1 Seiryo-Cho,
Gert van Montfrans, Academisch Medisch Centrum,
Aoba-Ku, Sendai 980-8574, Japan.
Interne Ziekten, Meibergdreef 9, AZ 1005 Amsterdam,
The Netherlands.
Jean-Michel Mallion, Médecine Interne et Cardiologie,
Chef de Service, Centre Hospitalier Universitaire de
Grenoble, B.P. 217 - 38043 Grenoble Cedex, France. Paolo Verdecchia, Departimento di discipline Cardiovasco-
lari, Ospedale R. Silvestrini, Perugia, Italy.
Giuseppe Mancia, Universita Degli Studi di Milano-
Bicocca, Cattedra di Medicina Interna, Ospedale San Bernard Waeber (Secretary), Centre Hospitalier Universi-
Gerardo Dei Tintori, Via Donizetti, 106, 20052 Monza, taire Vaudois, Division D’Hypertension, Departement de
Italy. Medecine Interne, 1011 Lausanne, Switzerland.
Thomas Mengden, University Clinic Bonn, Department of William White, Section of Hypertension and Vascular
Internal Medicine, Wilhelmstrasse 35 5311 Bonn, Germany. Diseases, University of Connecticut Health Center, 263
Farmington Avenue, Farmington, Connecticut 06030-3940,
Martin G. Myers, Division of Cardiology, Sunnybrook and USA.
Women’s College Health Sciences Centre, 2075 Bayview
Avenue, Toronto, Ontario M4N 3M5, Canada.
Statistical Assistance:
Eoin O’Brien (Chairman), Blood Pressure Unit, Beaumont Neil Alkins, Blood Pressure Unit, Beaumord Hospital,
Hospital, Dublin 9, Ireland. Dublin 9, Ireland.
Paul Padfield, Department of Medicine, Western General William Gerin, Mount Sinai School of Medicine, New York,
Hospital, Edinburgh EH4 2XU, UK. NY, USA.