Sei sulla pagina 1di 15

SPECIAL ARTICLE

Working Group on Blood Pressure Monitoring of the


European Society of Hypertension International Protocol for
validation of blood pressure measuring devices in adults
Eoin O’Briena, Thomas Pickeringb, Roland Asmarc,
Martin Myersd, Gianfranco Paratie, Jan Staessenf,
Thomas Mengdeng, Yutaka Imaih, Bernard Waeberi and
Paolo Palatinij and with the statistical assistance of Neil Atkinsa
and William Gerink, on behalf of the Working Group on Blood
Pressure Monitoring of the European Society of Hypertension

Blood Pressure Monitoring 2002, 7:3^17 Introduction


Keywords: protocol, device validation, blood pressure, blood pressure With the increasing marketing of automated and semi–
measurement, British Hypertension Society (BHS), Association for the automated devices for the measurement of blood pressure,
Advancement of Medical Instrumentation (AAMI)
potential purchasers need to be able to satisfy themselves
a
Blood Pressure Unit, Beaumont Hospital, Dublin, Ireland; bHypertension that such devices have been evaluated according to agreed
Center, New York Hospital, Cornell Medical Center, New York, USA;
c
Socie¤te¤ Franc/aise D’Hypertension Artere¤rielle, Fillale de la Socie¤te¤ Fran- criteria [1]. With this in mind, the Association for the
c/aise de Cardilogie, Paris, France; dDivision of Cardiology, Sunnybrook Advancement of Medical Instrumentation (AAMI) pub-
and Women’s College Health Sciences Centre,Toronto, Canada; eInstituto
Scienti¢co Ospedale San Luca,IRCCS,Instituto Auxologico ItalianoMilan, lished a standard for electronic or aneroid sphygmoman-
Italy; fKatholieke Universiteit Leuven, Hypertensie en Cardiovasculaire ometers in 1987 [2] that included a protocol for evaluating
Revalidatie Eenheid, Inwendige Geneeskunde-Cardiologie, Leuven, Bel-
gium; gUniversity Clinic Bonn, Department of Internal Medicine, Bonn, the accuracy of devices, this being followed in 1990 by
Germany; hSecond Department of Internal Medicine, Tohoku University the protocol of the British Hypertension Society (BHS)
School of Medicine, Sendai, Japan; iCentre Hospitalier Universitaire Vau-
dois, Division D’Hypertension, Department de Medecine Interne, Laus- [3]. Both of these were revised in 1993 [4,5]. These
sanne, Switzerland; jDipartimento di Medicina Clinica e Sperimentale, protocols, which differed in detail, had a common
Universita’di Padova, Padua, Italy; kMount Sinai School of Medicine, New
York, USA objective, namely the standardization of validation proce-
dures to establish minimum standards of accuracy and
Correspondence and requests for reprints to Professor E. O’Brien,
Blood Pressure Unit, Beaumont Hospital, Dublin 9, Ireland. performance, and to facilitate the comparison of one device
with another [6].
Received 13 November 2001 Accepted 15 November 2001
Since their introduction, a large number of blood pressure
measuring devices have been evaluated according to one or
both protocols. Experience has, however, demonstrated
that the conditions demanded by the protocols are
extremely difficult to fulfil. This is especially so because
of the large number of subjects who have to be recruited
and the ranges of blood pressure required. The time
required to complete a validation study is such that it is
difficult to recruit trained staff for the duration of an
investigation. These factors have made validation studies
difficult to perform and very costly, with the result that
fewer centers are prepared to undertake them. This is
particularly unfortunate as more devices than ever before
are in need of independent validation.

When the BHS dissolved its Working Party on Blood


Pressure Measurement, the Working Group on Blood
Pressure Monitoring of the European Society of Hyperten-
sion (ESH) undertook to produce an updated protocol,
which it has named the International Protocol. The ESH
Working Group on Blood Pressure Monitoring is composed
of experts in blood pressure measurement, many of whom

1359-5237 & 2002 Lippincott Williams & Wilkins


4 Blood Pressure Monitoring 2002,Vol 7 No 1

have considerable experience in validating blood pressure 3. Validation measurements. Observer and device measure-
measuring devices. ments are recorded on subjects in two phases. In the
first phase, 15 subjects are recruited; devices passing
In setting about its objective, the ESH Working Group this primary phase proceed to the secondary phase, for
recognized the urgent imperative to provide a simplified which a further 18 subjects are recruited.
protocol that does not sacrifice the integrity of the earlier 4. Analysis. An analysis of the recorded measurements is
protocols. When the AAMI and BHS protocols [2–5] were carried out after each phase.
published, the relevant committees did not have evidence 5. Reporting. The results are presented in tabular and
from previous studies on which to base their recommenda- graphical forms.
tions. The ESH Working Group has had the advantage of
being able to examine and analyze the data from 19
validation studies performed according to the AAMI and Observer training and assessment
BHS protocols at the Blood Pressure Unit in Dublin [7– Consideration must first be given to the technique of blood
23]. A critical assessment of this database of evidence has pressure measurement, which should be as follows
permitted a rationalization and simplification of validation throughout the validation procedure.
procedures without losing the merits of the much more
Blood pressure measurement technique
complicated earlier protocols. The basic recommendations
of the simplified International Protocol have been pre- A standard mercury sphygmomanometer, the components
sented at meetings of the ESH Working Group, and the of which have been carefully checked before the study, is
proceedings of these meetings have been published in used as a reference standard. It is appreciated that terminal
order to invite comment and discussion [24–27]. digit preference is a problem with conventional mercury
sphygmomanometry, and care should be taken to reduce
The International Protocol has been drafted in such a way this in the observer training session. The Hawksley
as to be applicable to the majority of blood pressure random-zero sphygmomanometer only disguises digit pre-
measuring devices on the market. The validation procedure ference, and its accuracy has been questioned [7,28];
is therefore confined to adults over the age of 30 years (as therefore, its use is not recommended in validation studies.
these will constitute the majority of subjects with All blood pressures should be recorded to the nearest
hypertension), and does not make recommendations for 2 mmHg.
special groups, such as children, pregnant women and the
elderly, or for special circumstances, for example exercise. Blood pressure should be measured with the arm supported
It is anticipated that the relative ease of performance of the at heart level [29]; the level of the manometer does not
International Protocol will encourage manufacturers to affect the accuracy of measurement, but it should be at eye
submit blood pressure measuring devices for validation in level and within 1 m of the observer. The quality of the
order to obtain the minimum approval necessary for a stethoscope is also crucial to performing the evaluation
device to be used in clinical medicine, and that, in time, procedure. Stethoscopes with badly fitting earpieces and
most devices on the market will be assessed according to poor-quality diaphragms preclude precise auscultation of
the protocol for basic accuracy. It does not preclude the Korotkoff sounds. A well-maintained quality stetho-
manufacturers of devices from subjecting their products to scope is recommended.
more rigorous assessment and validation.
Observer training
Validation procedure The first prerequisite for this validation test is to ensure
Summary that the observers have adequate auditory and visual acuity,
The validation team should consist of four persons and that they have achieved the required accuracy as laid
experienced in blood pressure measurement: two observers out below. It is, however, possible that observers who fulfil
and a supervisor (generally nurses), and an ‘expert’ (a these criteria at the outset of the study will not do so at the
doctor overseeing the entire procedure). If the doctor can end, and if this happens the observers must be re-assessed
be present throughout the entire validation procedure, he/ for accuracy. To avoid this, analysis should be performed as
she can take over the role of supervisor, thereby reducing the study proceeds to detect any drift in agreement
the number of personnel to three. The validation between the observers.
procedure consists of the following steps
Observers may be trained in the following ways:
1. Observer training and assessment. Two observers are
trained in accurate blood pressure measurement. 1. By fulfilling the test requirements of the CD-ROMs
2. Familiarization session. The validation team becomes produced by the BHS or the Société Française
familiar with the workings of the device and the d’Hypertension Artérielle as described in Appendix B
accompanying software. [30,31].
Protocol for validating BP measuring devices in adults ESH Working Group 5

2. By formal training and assessment is described in test device recommends different cuff sizes, the appro-
Appendix B [32, 33]. priate cuff/bladder should be used, but no other part of the
3. By using an audio-visual method for validation, such as apparatus should be changed. It is important to ensure,
the Sphygmocorder [34,35] as discussed in Appendix when assessing auscultatory devices, that the same
B. microphone(s) are used throughout the validation test.

Devices for measuring blood pressure at the wrist

Familiarization session The International Protocol may be used to validate devices


As automated devices for blood pressure measurement may that measure blood pressure at the wrist. There is little
be complex, it is important that the personnel performing a literature regarding the accuracy of devices for wrist
validation study are fully conversant with the equipment. measurement, and most studies have shown these devices
The observers, having satisfied the training criteria, should to be inaccurate [1]. Measurements of blood pressure at
next be instructed in the use of the device to be validated the wrist using oscillometric devices generally overestimate
and any accompanying computer software. For uncompli- blood pressure compared with conventional sphygmoma-
cated devices designed to provide a straightforward blood nometry on the upper arm, and the differences can be
pressure measurement, the familiarization session should substantial [36–38].
consist of performing a series of practice measurements on
volunteers. A more formal session should, however, be It must, however, be emphasized that although a device
applied to complex devices such as systems for measuring designed for measuring blood pressure at the wrist may be
24-h blood pressure. This session has two functions: first, it accurate when tested according to the International
serves as a familiarization period for the personnel Protocol, it may be inaccurate for the self-measurement
performing the validation study, and second, any technical of blood pressure if the instructions to have the wrist at
peculiarities of the device being tested, which might heart level during measurement are not strictly followed.
influence the validation procedure, may be identified. Devices for self-measurement that measure blood pressure
at the finger are not recommended because vasoconstric-
Validation measurements tion of the digital arteries can introduce substantial errors.
General considerations
Subject selection
Device validation should be performed at room tempera-
ture without disturbing influences such as telephones and In selecting 33 subjects (15 for the phase 1, and a further
bleeps in the area. 18 for phase 2) with a wide range of blood pressure it is
probable that there will be a representative range of arm
circumference, and subjects should not be selected on the
Some automated devices have more than one method of
basis of their arm circumference. Subjects may be taking
measuring blood pressure. It may be claimed for a
particular device, for example, that electrocardiogram antihypertensive medication but must not present in atrial
gating may be used when more accurate measurement is fibrillation or any sustained arrhythmia.
required. In these circumstances, validation must be Number Phase 1 Fifteen subjects
performed with and without electrocardiogram gating. Phase 2 Thirty-three subjects
Similarly, some Korotkoff sound-detecting devices provide Sex Phase 1 At least five male and five
an oscillometric back-up when sound detection fails. female
When both systems generate simultaneous readings, only Phase 2 At least 10 male and 10
one comparative validation is required, but when the female
oscillometric method is a back-up to the auscultatory Age range All subjects should be at least 30
method and provides a separate measurement, both years of age
systems of measurement must undergo individual Arm circumference Distribution by chance
validation. Blood pressure range As in Table 1
There are three ranges for systolic (SBP) and three for
Arm circumference and bladder dimensions
diastolic (DBP) blood pressure, with 11 subjects in each
The circumference of the arms should be measured to range to provide 99 pairs of measurements. To optimize
ensure that the bladder being used is adequate for the recruitment, it is recommended that subjects for the high-
subject. Measurements made with the test device should diastolic and low-systolic groups should be recruited first.
use the appropriate bladder according to the manufac- The emphasis should then be placed on filling the
turers’ instructions. Standard mercury manometer mea- remaining high-systolic and low-diastolic groups. Finally,
surements must be taken with a bladder of sufficient the remaining gaps in the middle groups should be filled.
length to encircle 80% of the arm circumference [29]. If a The blood pressure used in this categorization is the entry
6 Blood Pressure Monitoring 2002,Vol 7 No 1

Table 1 Blood pressure ranges for entry blood pressure (BPA) BPB Device detection blood pressure, observer 3. This
SBP DBP blood pressure is measured to allow the test
instrument to determine the blood pressure
Low 90^129 40^79
Medium 130^160 80^100 characteristics of the subject; more than one
High 161^180 101^130 attempt may be needed with some devices; this
For the primary phase, ¢ve of the 15 subjects should have a systolic blood pressure
measurement is not included in the analysis. If
(SBP) in each of the ranges. Similarly, ¢ve of the 15 subjects should have a diastolic the device fails to record a measurement after
blood pressure (DBP) in each of the ranges. For the secondary phase,11 of the 33 three attempts, the subject is excused.
subjects (including the ¢rst 15 subjects) should have SBP and DBP in each of the
ranges.It is recommended that recruitment should commence by targeting subjects
BP1 Observers 1 and 2 with the mercury standard.
likely to have pressures in the low-systolic and high-diastolic ranges, then progres- BP2 Supervisor with the test instrument.
sing to complete the high-systolic and low-diastolic ranges so that it will be easy to BP3 Observers 1 and 2 with the mercury standard.
complete the recruitment with the remaining medium ranges.
BP4 Supervisor with the test instrument.
BP5 Observers 1 and 2 with the mercury standard.
blood pressure at the time of the validation procedure BP6 Supervisor with the test instrument.
(BPA), rather than that at the time of recruitment for BP7 Observers 1 and 2 with the mercury standard.
validation.
3. Documentation must be provided for data omitted for
legitimate technical reasons. Once a subject has been
Observer measurement included, the data for that subject should not be
Measurements can be either assessed live using two excluded from the study if blood pressure values are
observers or recorded and later re-assessed using the obtainable; if blood pressure measurements using
Sphygmocorder [34,35]. Measurements made simulta- either the reference method or the test instrument
neously by two observers must be checked by the are unavailable, data entry for that individual may be
validation supervisor. If the systolic and diastolic measure- excluded, with an accompanying explanation. Addi-
ments are no more than 4 mmHg apart, the mean value of tional individuals must then enter into the study to
the two observer measurements for both systolic and ensure a sample size of 33.
diastolic blood pressures is used. Otherwise, the measure-
ment must be taken again.
Analysis
When the Sphygmocorder is used, two observers should For a detailed discussion on the statistical methods used in
assess the recording separately. If their opinion differs, they the protocol, see Appendix D. A software program has been
should re-assess the values together until agreement is designed specifically to analyze the data (Société Française
reached. Further references to ‘observer measurement’ d’Hypertension Artérielle, Paris).
indicate either the mean of two observer measurements or
the agreed measurement using the Sphygmocorder. At least Accuracy criteria
30 s should be allowed between each measurement to The BHS protocol introduced the concept of classifying
avoid venous congestion, but not more than 60 s or the differences between test and control measurements
variability may be increased. according to whether these lay within 5, 10 or 15 mmHg, or
were over 15 mmHg apart. The final grading was based on
Procedure the number of differences falling into these categories.
This protocol seeks to keep this concept but expand its
1. The subject is introduced to the observers, and the flexibility.
procedure is explained. Arm circumference, sex, date
of birth and current date and time are noted. The Differences are always calculated by subtracting the
subject is then asked to relax for 10–15 min (in order observer measurement from the device measurement.
to minimize anxiety and any white-coat effect, which When comparing and categorizing differences, their
will increase variability). absolute values are used. A difference is categorized into
2. Nine sequential same-arm measurements using the one of four bands according to its rounded absolute value
test instrument and a standard mercury sphygmoman- for SBP and DBP:
ometer are recorded as follows:
0–5 mmHg These represent measurements consid-
BPA Entry blood pressure, observers 1 and 2 each with ered to be very accurate (no error of
the mercury standard. The mean values are used clinical relevance).
to categorize the subject into a low, medium or 6–10 mmHg These represent measurements consid-
high range separately for SBP and DBP (Table 1). ered to be slightly inaccurate.
Protocol for validating BP measuring devices in adults ESH Working Group 7

Table 2a Requirements to pass phase 1


11–15 mmHg These represent measurements consid-
ered to be moderately inaccurate. Measurements Within 5 mmHg Within10 mmHg Within15 mmHg
415 mmHg These represent measurements consid- At least one of 25 35 40
ered to be very inaccurate. After completing the assessment on 15 subjects, the data (45 comparisons) should
be analyzed to determine the number of comparisons falling within the 5 10 and
The analysis is based on how values in these bands fall 15 mmHg error bands. At least 25 comparisons must lie within 5 mmHg, at least 35
within10 mmHg orat least 40 within15 mmHg.If none of these counts are reached the
cumulatively into three zones: device is deemed to have failed.
Within 5 mmHg This zone represents all values falling
in the 0–5 mmHg band. Table 2b Requirements to pass phase 2.1
Within 10 mmHg This zone represents all values falling Measurements Within 5 mmHg Within10 mmHg Within15 mmHg
in the 0–5 and 6–10 mmHg bands.
Two of 65 80 95
Within 15 mmHg This zone represents all values falling All of 60 75 90
in the 0–5, 6–10 and 11–15 mmHg
After completing all 33 subjects, the data (99 comparisons) should be analyzed to
bands. determine the number of comparisons falling within the 5, 10 and 15 mmHg error
bands. For the device to pass, there must be a minimum of 60, 75 and 90 compari-
sons falling within 5 10 and 15 mmHg, respectively. Furthermore, there must be a
Subject measurements minimum of either 65 comparisons within 5 mmHg and 80 comparisons within
10 mmHg, or 65 comparisons within 5 mmHg and 95 comparisons within 15 mmHg,
For assessment of accuracy, only measurements BP1 to BP7 or 80 comparisons within 10 mmHg and 95 comparisons within 15 mmHg.
are used. The mean of each pair of observer measurements
is calculated; this is denoted as observer measurement Table 2c Requirements to pass phase 2.2
BP1, BP3, BP5 or BP7. Each device measurement is Subjects 2/3 within 5 mmHg 0/3 within 5 mmHg
flanked by two of these observer measurements, and one of
At least 22
these must be selected as the comparative measurement. At most 3
The data should now be analyzed per subject to determine the number of compari-
From these, further measurements are derived as follows. sons per subject falling within 5 mmHg. At least 22 of the 33 subjects must have at
least two of their three comparisons lying within 5 mmHg. (These include those who
1. The differences BP2 – BP1, BP2 – BP3, BP4 – BP3, have all three comparisons within 5 mmHg.) At most, three of the 33 subjects can
BP4 – BP5, BP6 – BP5 and BP6 – BP7 are calculated. have all three of their comparisons over 5 mmHg apart.
2. The absolute values of the differences are calculated
(i.e. the signs are ignored).
3. These are paired according to the device reading. phase (Table 2a) being eliminated from further testing.
4. If the values in a pair are unequal, the observer Devices passing this proceed to a secondary phase in which
measurement corresponding to the smaller difference a further 18 subjects (giving a total of 33) are recruited
is used. (Table 2b).
5. If the values in a pair are equal, the first of the two
observer measurements is used. Assessment of phase 1
When this has been completed, there are three device Once there are five subjects in each of the six blood
readings for SBP and three for DBP for each subject. Each pressure ranges (Table 1), recruitment should be stopped
of these six readings has a single corresponding observer and an assessment performed. Data from only the first five
measurement, a difference between the two and a band for subjects in each range are used. (In filling these ranges,
that difference as described above. some ranges may be over-subscribed because of subjects
having different SBP and DBP ranges.) This will yield 45
Experience with existing protocols has demonstrated sets of measurements for both SBP and DBP.
that the overall outcome of a device can be apparent
1. The number of differences in each zone is calculated
from a very early stage. This is particularly so with poor
as described above.
devices and is in accordance with statistical expectancy –
2. A continue/fail grade is determined according to Table
the larger the error, the smaller the sample size required to
2a (see also Table 3 for an example).
prove it. To persist with the validation of a device that is
3. If the device fails, the validation is complete; if the
clearly going to fail is an unnecessary waste of time and
device passes, it proceeds to phase 2.
money, and an inconvenience to participating subjects. A
mechanism for eliminating poor devices at an appropriate
stage is therefore introduced by dividing the validation
Assessment of phase 2
process into two phases. In the primary phase, three pairs
of measurements are performed on 15 subjects in the This phase determines how accurate the device will be for
pressure ranges given in Table 1, any device failing this individual measurements (Phase 2.1) and for individual
8 Blood Pressure Monitoring 2002,Vol 7 No 1

Table 3 Example of device validation table in report


Phase1 within 5 mmHg within10 mmHg within15 mmHg Recommendation
Required One of 25 35 40
Achieved SBP 22 35 43 Continue
DBP 35 42 44 Continue
Phase 2.1 within 5 mmHg within10 mmHg within15 mmHg Recommendation Mean di¡erence Standard deviation
Required Two of 65 80 95
All of 60 75 90
Achieved SBP 52 79 90 Fail 3.4 mmHg 8.4 mmHg
DBP 77 90 94 Pass ^0.6 mmHg 6.9 mmHg
Phase 2.2 2/3 within 5 mmHg 0/3 within 5 mmHg Recommendation
Required X22 p3
Achieved SBP 17 4 Fail
DBP 28 2 Pass
The device passes for diastolic blood pressure (DBP, but fails for systolic blood pressure (SBP), thereby failing overall.

subjects (Phase 2.2) by determining the number of Phase 1


differences within 5, 10, and 15 mmHg, and then The number of differences falling in the Within 5 mmHg,
determining the accuracy. Within 10 mmHg and Within 15 mmHg zones (Table 2a),
together with the requirements, should be reported in
After all ranges have been filled, there will be 99 sets of text and tabular form as in Table 3. The basis on which
measurements for both SBP and DBP. the decision to continue or stop at this stage should be
stated.
1. The number of differences in each zone as described
above is calculated.
Phase 2
2. A pass/fail grade for phase 2.1 is determined according
The number of differences falling in the Within 5 mmHg,
to Table 2b (see Table 3 for example).
Within 10 mmHg and Within 15 mmHg zones (Table 2b),
3. For each of the 33 subjects, the number of measure-
together with the requirements, should be reported in text
ments falling within 5 mmHg is determined.
and tabular form as in Table 3. The number of subjects
4. A pass/fail recommendation for phase 2.2 is deter-
with all three differences, at least two differences and no
mined according to Table 2c (see Table 3 for example).
differences falling within 5 mmHg (Table 2c) should be
5. If the device passes both phase 2.1 and phase 2.2, it
reported in text and tabular form as in Table 3. The mean
passes the validation and can be recommended for
and standard deviation of the observer and device
clinical use. If it does not, it fails and is not
measurements and the differences should be stated. The
recommended for clinical use.
basis on which the decision to pass or fail the device should
be stated.
Reporting
Statistical report Graphical representation
Difference-against-mean plots should be presented for the
The report should be prefaced with subject data in order to
data at the phase at which the study ceased. Phase 1 data
describe the key characteristics of the subjects in the study.
should be plotted for devices failing at that stage, and
An example of a device validation is shown in Table 3.
phase 2 data for those passing. The x-axis of these plots
1. Sex distribution. The number of males and females. represents blood pressures in the systolic range 80–
2. Age distribution. The mean, standard deviation and 190 mmHg and the diastolic range 30–140 mmHg, and
range of the subjects’ ages. the y-axis values from 30 to þ30 mmHg. Horizontal
3. Arm circumference distribution. The mean, standard reference lines are drawn at 5 mmHg intervals from þ15 to
deviation and range of the subjects’ arm circumfer- –15 mmHg. The mean of each device pressure and its
ences and, when different cuff sizes are used, the corresponding observer pressure is plotted against their
number of subjects on which each size was used. difference using a point. Differences greater than
4. Blood pressure. The mean, standard deviation and range 30 mmHg are plotted at 30 mmHg. Differences less than
of the subjects’ entry SBP and DBP (BPA). –30 mmHg are plotted at –30 mmHg. The same scales
should be used for both SBP and DBP plots. An example is
The report should then give the results of the validation. shown in Fig. 1 [39].
Protocol for validating BP measuring devices in adults ESH Working Group 9

Problems tional instructions, it is difficult to obtain optimal


Any problems encountered during the validation proce- performance.
dure, the date of their occurrence, the date of any repairs to
the device and the effect of these on the validation List of components
procedure should be recorded. All major components of the system should be listed. The
dimensions of the bladders supplied and those of the range
Operational report of bladders available should be indicated.

The following information should be provided with blood Method(s) of blood pressure measurement
pressure measuring devices, and the final report should The basic method of pressure detection (e.g. auscultatory
acknowledge that such information is available, and or oscillometric) should be stated, and if more than one
although this need not be presented in detail, any method is used, the indications for changing methods and
deficiencies should be listed in the report. the means of denoting this on the recording should be
stated. With Korotkoff sound-detecting devices, whether
Basic information phase IV or phase V is being used for the diastolic end-
The information provided in operational manuals is often point must be disclosed. If data are derived from recorded
deficient. Without appropriate specifications and opera- measurements, such as mean pressure, the method of
calculation must be stated.
Fig. 1

Factors a¡ecting accuracy


Many factors, such as arm movement, exercise, arm
position, cuff or cloth friction may affect the accuracy of
automated recordings. All such factors should be listed by
the manufacturer.

Operator training requirements


Some automated systems require considerable expertise on
the part of the operator if accurate measurements are to be
obtained, whereas other systems require relatively little
instruction. These requirements should be stated.

Computer analysis
Some automated systems are compatible with personal
computer systems. The exact requirements for linking with
computer systems and their approximate cost should be
stated. If the automated system is dependent on its own
computer for plotting and analysis, this should be made
clear, and the cost of the computer facility, if it is an
optional extra, should be stated.

Clear instructions should be provided for setting recording


conditions (e.g. frequency of recordings during defined
periods and the on/off condition of the digital display);
retrieving recordings and saving data to disk; retrieving data
from disk; displaying numerical data and graphics; export-
ing data to statistical, graphic and spreadsheet software
Devices passing and failing phase 2.1 The x^axis represents the
mean of the device and observer measurements. Both systolic blood programs; and printing the results (partial or complete). If
pressure (upper plot) and diastolic blood pressure (lower plot) data cannot be exported, information on how they are
ranges should be plotted on the same scale. Recruitment limits are stored should be available to facilitate the external analysis
indicated by the vertical lines. The y^axis represents the di¡erence
between the device and observer measurements. The 5 mmHg of several monitoring events. The manufacturer should list
bands from þ15 to ^15 mmHg are indicated by the horizontal hatched compatible computers (PC or other) and printers together
lines. The 99 comparisons are presented in a di¡erence-against- with memory requirements, operating systems, compatible
mean scatterplot. In this example, the systolic blood pressure plot
depicts a poor device whereas the diastolic blood pressure plot graphic adaptors and additional software or hardware
depicts an accurate one. requirements (including interfaces and cables if these are
not supplied).
10 Blood Pressure Monitoring 2002,Vol 7 No 1

Acknowledgements determined by the British Hypertension Society protocol. J Hypertens


1993; 11:(suppl 2):1^7.
19 O’Brien E, Mee F, Atkins N, O’Malley K. Accuracy of the Pro¢lomat
The report should state whether the equipment was ambulatory blood pressure measuring system determined by the British
Hypertension Society protocol. J Hypertens 1993; 11:(suppl 2):S9S^15.
purchased for the evaluation or donated or loaned by the 20 Mee F, O’Brien E, Atkins N, O’Malley K. Comparative accuracy of the CH-
manufacturer. The data analysis should ideally be carried Druck, Pro¢lomat, SpaceLabs 90207, DIASYS 200, Pressurometer IV and
out by the laboratory doing the evaluation. If it has been Takeda TM-2420 ambulatory blood pressure measuring (ABPM) devices
determined by the British Hypertension Society (BHS) protocol. J Hum
done by the manufacturers, this should be stated. Any Hypertens 1993; 7:98.
consultancies or conflict of interest should be acknowl- 21 O’Brien E, Mee F, Atkins N, Thomas M. Evaluation of three devices for
self-measurement of blood pressure: according to the revised British
edged by the investigator. Hypertension Society Protocol: the Omron HEM-705CP, Phillips HP5332,
and Nissei DS-175. Blood Press Monit 1996; 1:55^61.
22 Mee F, Atkins N, O’Brien E. Evaluation of the Pro¢lomat II s ambulatory
blood pressure system according to the protocols of the British
Hypertension Society and the Association for the Advancement of
References Medical Instrumentation. Blood Press Monit 1999; 3:353^361.
23 O’Brien E, Mee F, Atkins N. Evaluation of the Schiller BR-102 ambulatory
1 O’Brien E, Waeber B, Parati G, Staessen G, Myers MG, on behalf of the blood pressure system according to the protocols of the British
European Society of Hypertension Working Group on Blood Pressure Hypertension Society and the Association for the Advancement of
Monitoring. Blood pressure measuring devices: validated instruments. Medical Instrumentation. Blood Press Monit 1999; 4:35^43.
BMJ 2001; 322:531^536. 24 O’Brien E. Criteria for validation of devices. Blood Press Monit 1999;
2 Association for the Advancement of Medical Instrumentation. The 4:279^293.
national standard of electronic or automated sphygmomanometers. 25 Asmar R, Zanchetti A. On behalf of the Organizing Committee and
Arlington, VA: AAMI; 1987. participants. Guidelines for the use of self-blood pressure monitoring:
3 O’Brien E, Petrie J, Littler W, de Swiet M, Pad¢eld PL, O’Malley K, et al. a summary report of the ¢rst international consensus conference.
The British Hypertension Society protocol for the evaluation of J Hypertens 2000; 18:493^508.
automated and semi-automated blood pressure measuring devices
26 O’Brien E. Proposals for simply¢ng the validation protocols of the British
with special reference to ambulatory systems. J Hypertens 1990;
Hypertension Society and the Association for the Advancement of
8:607^619.
Medical Instrumentation. Blood Press Monit 2000; 5:43^45.
4 O’Brien E, Petrie J, Littler WA, de Swiet M, Pad¢eld PL, Altman D,
27 O’Brien E, De Gaudemaris R, Bobrie E, Agabiti Rosei E, Vaisse B.
et al. The British Hypertension Society protocol for the evaluation of
Proceedings from a conference on self blood pressure measurement:
blood pressure measuring devices. J Hypertens 1993; 11:(suppl 2):
devices and validation. Blood Press Monit 2000; 5:93^100.
S43^S63.
28 Conroy R, O’Brien E, O’Malley K, Atkins N. Measurement error in the
5 Association for the Advancement of Medical Instrumentation. American
Hawksley random zero sphygmomanometer: What damage has been
national standard. Electronic or automated sphygmomanometers.
done and what can we learn? BMJ 1993; 306:1319^1322.
Arlington, VA: AAMI; 1993.
29 O’Brien E, Petrie J, Littler WA, de Swiet M, Pad¢eld PD, Dillon MJ, et al.
6 O’Brien E, Atkins N. A comparison of the BHS and AAMI protocols for
Blood pressure measurement: recommendations of the British
validating blood pressure measuring devices: can the two be reconciled?
Hypertension Society. 2nd edn. London: BMJ Publishing Group; 1997.
J Hypertens 1994; 12:1089^1094.
30 Blood Pressure Measurement. CD-ROM. London: British Hypertension
7 O’Brien E, Mee F, Atkins N, O’Malley K. Inaccuracy of the Hawksley
Society/BMJ Books; 1998.
random-zero sphygmomanometer. Lancet 1990; 336:1465^1468.
31 Socie¤te¤ Franc/aise d’Hypertension Arte¤rielle. La prise de la pression
8 O’Brien E, Mee F, Atkins N, O’Malley K. Inaccuracy of seven popular
arte¤rielle au cabinet me¤dical. Socie¤te¤ Franc/aise d’Hypertension
sphygmomanometers for home-measurement of blood pressure.
Arte¤rielle; 1998.
J Hypertens 1990; 8:621^634.
32 O’Brien E, Mee F, Atkins N, O’Malley K, Tan S. Training and assessment of
9 O’Brien E, More O’Ferrall J, Galvin J, Costello R, Lydon S, Sheridan J,
observers for blood pressure measurement in hypertension research.
et al. An evaluation of the Accutracker II non-invasive ambulatory blood
pressure recorder according to the AAMI standard. J Ambul Monit 1991; J Hum Hypertens 1991; 5: 7^10.
4:27^33. 33 Mee F, Atkins N, O’Brien E. Problems associated with observer training in
10 O’Brien E, Atkins N, Mee F, O’Malley K. Evaluation of the SpaceLabs blood pressure measurement. J Hum Hypertens 1994; 8: 293.
90202 according to the AAMI standard and BHS criteria. J Hum 34 O’Brien E, Atkins N, Mee F, Coyle D, Syed S. A new audio-visual
Hypertens 1991; 5:223^226. technique for recording blood pressure in research: the Sphygmocorder.
11 O’Brien E, Mee F, Atkins N, O’Malley K. Accuracy of the Del Mar Avionics J Hypertens 1995; 13:1734^1737.
Pressurometer IV determined by the British Hypertension Society 35 Atkins N, O’Brien E, Wesseling KH, Guelen I. Increasing observer
Protocol. J Hypertens 1991; 9:(suppl 5):S1S^7. objectivity with audio-visual technology: : the Sphygmocorder. Blood
12 O’Brien E, Mee F, Atkins N, O’Malley K. Accuracy of the Novacor DIASYS Press Monit 1997; 2: 269^272.
200 determined by the British Hypertension Society Protocol. 36 Forsberg SA, de Guzman M, Berlind S. Validity of blood pressure
J Hypertens 1991; 9:(suppl 5):S9S^15. measurement with cu¡ in the arm and forearm. Acta Med Scand 1979;
13 O’Brien E, Mee F, Atkins N, O’Malley K. Accuracy of the Takeda TM-2420/ 188: 389^396.
TM-2020 determined by the British Hypertension Society Protocol. 37 Blackburn H, Kihlberg J, Brozek J. Arm versus forearm blood pressure in
J Hypertens 1991; 9:(suppl 5):S17S^23. obesity. Am Heart J 1965; 69: 423^427.
14 O’Brien E, Mee F, Atkins N, O’Malley K. Accuracy of the SpaceLabs 90207 38 Rytand DA, Boyer SH. Auscultatory determination of arterial pressure at
determined by to the British Hypertension Society Protocol. J Hypertens wrist and ankle. Am J Med 1954; 17:112^115.
1991; 9:(suppl 5):S25S^31. 39 Bland JM, Altman DG. Statistical methods for assessing agreement
15 O’Brien E, Mee F, Atkins N, O’Malley K. Accuracy of the SpaceLabs between two methods of clinical measurement. Lancet 1986; i:307^310.
90207, Novacor DIASYS 200, Del Mar Avionics Presurometer IV and 40 European Commission for Standardisation. European Standard EN 1060-1
Takeda TM-2420 ambulatory systems according to the AAMI and BHS (British Standard BSSEN 1060-1:1996). Speci¢cation for Non-invasive
criteria. J Hypertens 1991; 9:(suppl 6):S332S^333. sphygmomanometers. Part I. General requirements. European
16 O’Brien E, Mee F, Atkins N, Halligan A, O’Malley K. Accuracy of the Commission for Standardisation; 1995.
SpaceLabs 90207 ambulatory blood pressure measuring system in 41 European Commission for Standardisation. European Standard EN 1060-2
normotensive pregnant women determined by the British Hypertension (British Standard BSSEN 1060-2:1996). Speci¢cation for Non-invasive
Society Protocol. J Hypertens 1993; 11:(suppl 5):869^873. sphygmomanometers. Part 2. Supplementary requirements for
17 O’Brien E, Mee F, Atkins N, O’Malley K. Short report: Accuracy of the mechanical sphygmomanometers. European Commission for
Dinamap portable monitor, Model 8100, determined by the British Standardisation; 1995.
Hypertension Society protocol. J Hypertens 1993; 11:761^763. 42 European Commission for Standardisation. European Standard EN 1060-3
18 O’Brien E, Mee F, Atkins N, O’Malley K. Accuracy of the CH-Druck/ (British Standard BS EN 1060-3: 1997). Non-invasive
Pressure ERKA ambulatory blood pressure measuring system sphygmomanometers. Part 3. Supplementary requirements for electro-
Protocol for validating BP measuring devices in adults ESH Working Group 11

mechanical blood pressure measuring systems. European Committee for of factors may make it difficult or impossible to perform
Standardization; 1997.
simultaneous comparison on the same arm.
43 O’Brien E. Modi¢cation of blood-pressure measuring devices and the
protocol of the British Hypertension Society. Blood Press Monit 1999;
4: 53^54.
Devices, for example, that deflate at a rate of more than
44 European Commission for Standardisation. European Standard EN 540.
Clinical investigation of medical devices for human subjects. European 5 mmHg per second do not permit accurate measurement
Commission for Standardisation; 1993. by an auscultating observer, leading to inaccurate compar-
45 West NW, Sheridan JJ, Littler WA. Direct measurement of blood pressure.
In: O’Brien E, O’Malley K, editors: Handbook of hypertension. Vol. 14.
ison between the test and reference devices [4]. At fast
Blood pressure measurement. Amsterdam: Elsevier; 1991. pp. 315^333. deflation rates, an auscultating observer will tend to
46 Parati G, Casadei R, Gropelli A, Di Rienzo M, Mancia G. Comparison of underestimate SBP and overestimate DBP by recording
¢nger and intra-arterial blood pressure monitoring at rest and during
laboratory testing. Hypertension 1989; 13: 647^655. the first definite pressure phase at which Korotkoff sounds
47 O’Brien E, Atkins N, Mee F, O’Malley K. Comparative accuracy of six are audible as the systolic value and the last definite phase
ambulatory devices according to blood pressure levels. J Hypertens
1993; 11: 673^675.
of audible sounds as the diastolic. The device may have a
facility for slowing the rate of deflation so that the
simultaneous comparison can be performed, but this is
not permissible as any modification of the usual operational
Appendix A. Comparison with previous mode may alter its accuracy.
protocols
Our approach to simplifying previous validation procedures
Other factors that may preclude simultaneous same-arm
has concentrated on the following areas:
testing are confusion of noise from the device with
Korotkoff sounds, failure of the inflating mechanism to
Elimination of pre-validation phases reach the required pressure, sudden deflation before DBP
The main validation procedure of the existing BHS can be confirmed and uneven deflation, making accurate
protocol has five phases: (i) before-use device calibration; auscultation impossible. The most important objection to
(ii) the in-use (field) phase; (iii) after-use device calibra- simultaneous comparisons is that true simultaneous
tion; (iv) static device validation; and (v) report of the measurement cannot be achieved with oscillometric
evaluation (4). Phases (i)–(iii) were originally introduced devices, which now constitute virtually all automated
to identify intra-device variability, but if a device has devices available for blood pressure measurement.
fulfilled the general requirements of the European Union
directives [40–42] or the AAMI standard [5], it is not Simultaneous opposite-arm comparisons are not permitted
necessary to subject these devices to phases (i), (ii) or (iii) because the blood pressure difference between the arms is
of the BHS protocol. These pre-validation phases are thus a variable rather than a constant factor, and the measure-
not included in the present protocol, thereby resulting in ments are not truly simultaneous. To overcome the
considerable reduction in time and labor. problems associated with simultaneous measurements in
either the same or opposite arms, sequential testing is
Improving observer recruitment and training advocated in this protocol.
The most fallible component of blood pressure measure-
ment is the human observer, and consideration must be
Minimizing observer error during validation
given to the role of the education and certification of The role of the supervisor has been modified from that in
observers. CD-ROMs are available to facilitate the training the BHS protocol [4] so that he or she observes the result
and assessment of observers [30,31]. of each paired measurement made by observers 1 and 2,
and if either the SBP or DBP values are more than 4 mmHg
The Sphygmocorder, a device that provides an audio apart, the supervisor will simply state that the measure-
recording of Korotkoff sounds with a video recording of a ment must be taken again, without giving a reason, so that
mercury column, has been designed to provide objective neither observer will be biased when re-taking the blood
evidence of validation blood pressures [34,35]. The pressure. In this way, errors will be minimized. Experience
Sphygmocorder removes the expensive need to employ has shown, for example, that errors of 10 mmHg can be
two observers and a supervisor throughout the validation made by observers simply misreading the mercury column.
procedure and has greatly facilitated device validation. Another change in the protocol has been to use the mean of
the two observers’ results rather than analyzing the results
for each observer separately, these mean values being
Use of simultaneous or sequential comparisons
The basis of device evaluation is the comparison between referred to simply as ‘observer measurements’.
blood pressure measured by the device being tested and
measurements made by trained observers using a mercury Reduction in the number of subjects recruited
sphygmomanometer and stethoscope to auscultate the Reducing the number of subjects required for validation
Korotkoff sounds. With most automated devices, a number would greatly simplify the procedure, and there are now
12 Blood Pressure Monitoring 2002,Vol 7 No 1

sufficient data from the many validation studies performed fail, a tolerance factor whereby one of the above targets is
to review the number of subjects required [7–23]. not achieved for five measurements is allowed.

The first AAMI protocol required a sample of 85 subjects, Algorithm integrity and design modi¢cation
the paired measurements being averaged to give a total of 85 The first BHS protocol emphasized the importance of
paired comparisons [2]. The BHS protocols [3,4] and the manufacturers indicating by a change in model number any
revised AAMI protocol [5] did not average the values, modifications made to blood pressure measuring devices
leaving 255 sets of measurements for analysis. In the current [3]. The revised BHS protocol, published in 1993, went
protocol, reducing the number of paired measurements to 99 further by stipulating not only that manufacturers must
(which allows for easy conversion to equivalent percentage indicate clearly all modifications to the technological and
values) brings the sample size back to the original AAMI software components of automated devices by changing
recommendation but reduces the number of subjects to 33. the device number, but also that modified devices must be
Reducing the number of subjects results, of course, in some subjected to renewed validation [4]. These stipulations
loss in measurement independence, but an analysis of 19 were influenced by consequences that had resulted from
validation studies has shown that reducing the number of changes made by manufacturers to the algorithms of
subjects recruited from 85 to 33 is possible without affecting devices for measuring ambulatory blood pressure [43].
the accuracy of the validation (Appendix D) [7–23]. Manufacturers have, however, from time to time expressed
the view that the BHS stipulations were unreasonable, in
The subjects should be at least 30 years of age in order to that they obliged the manufacturer to go to the unneces-
ensure those recruited are representative of the age range sary expense of re-evaluating a device that had undergone
in which most hypertensive patients lie. some design modifications without any alteration of the
algorithm. Moreover, the stipulation might inhibit bene-
Relaxing the range of blood pressures ficial modifications to device design, which need not
Experience has shown that recruiting subjects at the involve adjusting an algorithm previously shown to have
extremes of high and low pressure is impractical. Further- fulfilled the accuracy criteria of the protocol. This
more, as blood pressure variability is greater at these stipulation remains in principle in the present protocol
extremes, sequential comparisons may become unreliable. but can be waived if a manufacturer of a device that has
The relaxation of these requirements to those shown in previously fulfilled the accuracy criteria of the protocol can
Table 1 above, with an equal number of subjects being provide the following: (1) independent evidence that the
recruited to each range, facilitates the validation procedure algorithm in the modified device is identical to that in the
without unduly affecting the results. originally validated model; (2) evidence that the proposed
modifications cannot alter the performance of the algo-
rithm; a system of model numbering that (3) acknowledges
Eliminating ‘hopeless’devices a common algorithm and (4) denotes the features of the
Our data support dividing the validation process into two
modification [43].
phases. In the primary phase, three pairs of measurements
are performed in 15 subjects in the stipulated pressure
ranges, any device failing this phase being eliminated from Intra-subject variability
further testing. Devices passing this phase then proceed to The influence of intra-subject variability is substantial and
a secondary phase, a further 18 subjects (to provide a total can disadvantage devices, particularly when sequential
of 33) being recruited, in whom comparisons must fulfil the measurements differ by over 10 mmHg, as happens
criteria shown in Table 2. These alterations do not especially in the higher pressure ranges. Two simple
substantially alter the results of the validation studies measures to cope with this problem have been incorpo-
examined, but by eliminating ‘hopeless’ devices at an early rated into this protocol.
stage, the validation process is simplified and unnecessary 1. Exclusion of subjects with extremely high and extremely low
testing avoided [7–23]. pressures. Not only do measurements in these ranges
tend to vary considerably, but also large differences,
Expression of validation results which would be substantial in the mid-range pressures,
In this protocol, the BHS grading system and AAMI are in practice unlikely to affect treatment at these
assessment according to the mean and standard deviation extremes.
of the differences have been abandoned in favour of a 2. Tolerance for comparative differences over 15 mmHg. It must
straightforward pass/fail system. Moreover, a degree of be accepted that sequential measurements may vary
tolerance in deciding the pass/fail category has been quite considerably in some subjects, especially at high
incorporated into the protocol. Ideally 65, 80 and 95 of pressures, and that these are not errors. An analysis of
the 99 measurements should lie within 5, 10 and 15 mmHg previous studies has shown that sequential SBP
respectively, but because a device might only marginally measurements typically lie within 5, 10 and 15 mmHg
Protocol for validating BP measuring devices in adults ESH Working Group 13

of each other 75, 93 and 97% of the time, respectively. Observer assessment
The mean difference is typically 1 mmHg, with a As as alternative to self-assessment, observers can be tested
standard deviation of around 5 mmHg. formally as in the BHS protocol [4].

Trainee observers are seated at a bench fitted with


Suitability of the device for individuals temporary partitions so that each observer is isolated in a
There is a fundamental paradox in the design of booth in which the only objects are a mercury column, a
previous protocols, which has been identified by an analysis stethoscope, a pencil and 50 numbered cards on which to
of the Dublin database [personal communication from write down assessments (Fig. 2). The rationale for this
Gerin W and Pickering T, 2001]. Whereas the procedures in procedure is that when more than one observer is being
previous protocols were designed to determine whether a trained and assessed, it becomes difficult to prevent an
given device would, on average, provide valid measure- observer who is unsure of a reading gaining sight of a
ments for a population, there is in practice a need to know neighbouring observer’s reading. It is therefore necessary
whether the device will give accurate measurements for a to separate observers by a series of partitions.
particular subject. The protocol therefore introduces a
1. The expert observer occupies a similar adjoining
tertiary phase whereby the device is assessed according to
booth, the only difference being the presence of a
the number of subjects in whom it gives accurate
hand bulb to inflate and deflate the cuff on the arm of
measurements in addition to its overall accuracy.
the subject.
(Table 2c).
2. Five subjects with a range of blood pressure from about
110/60 to 170/100 mmHg are seated behind a partition.
The ‘supervisor’ places the cuffs in random order on
Appendix B. Observer training the arms without the expert or trainee observers being
Self-assessment aware of the order. When the stethoscope head and
The observers, usually nurses who understand blood cuff are in place, the ‘supervisor’ gives a verbal cue to
pressure measurement, are retrained in blood pressure the observers and the expert observer operates the
measurement using a CD-ROM such as that produced by cuff and deflates it at a rate of 2 mmHg/s.
the BHS or the Société Française d’Hypertension Artérielle 3. As the inflatable bladder is connected to each of the
[30,31]. These demonstrate the technique of blood columns of mercury in the observer booths, all the
pressure measurement and permit an assessment period columns of mercury fall simultaneously for each of the
during which trainees can test themselves against a blinded observers and for the expert, all of whom write
standard mercury sphygmomanometer in which the mer- down their measurements. Using a series of man-
cury column falls against a background of recorded ometers, time must be allowed for each manometer to
Korotkoff sounds. Observers should not move on to the deflate fully and the mercury meniscus to return to
next stage until they have satisfied the test requirements zero.
of the CD-ROM. It is helpful for an expert in blood 4. Ten measurements are made by each observer on each
pressure measurement to take trainee observers through of five subjects, giving a total of 50 measurements for
the different stages of blood pressure measurement [29]. each observer.
Difficult aspects of interpretation, such as the auscultatory
gap and observer bias, should be discussed and illustrated
The accuracy criteria for the test procedure are the
by example. It is recommended that observers have
following.
audiograms to detect any hearing deficit.
1. Forty–five systolic and diastolic differences between
Fig. 2 each trainee and between trainees and expert should
differ by not more than 5 mmHg, and 48 by not more
than 10 mmHg.
2. Failure to achieve this degree of accuracy necessitates
a repeat training and assessment session for the failed
observer(s).

Audio-visual techniques
Training observers as described above is a labour–intensive
procedure, and even when observers are instructed to a
Diagram of observer assessment procedure high degree of accuracy, there is the problem of maintain-
ing that accuracy throughout the study [32,33].
14 Blood Pressure Monitoring 2002,Vol 7 No 1

A need has been recognized, therefore, for an electronic device performance in tracking fast beat-by-beat blood
audio visual system to measure blood pressure in validation pressure changes (46).
studies that is not dependent on observers but will
nevertheless retain the traditional auscultatory methodol-
ogy using the mercury sphygmomanometer. An example of Appendix D. Statistical considerations
such is the Sphygmocorder which was developed for this Sample size
purpose and, since it was first described in 1995 [34], a The AAMI published its first protocol for the validation of
number of improvements have been made to the system blood pressure measuring devices in 1987 [2]. The
[35]. This system is being developed for commercial accuracy component of the protocol basically consisted of
distribution. a comparison of the mean of three test device measure-
ments with simultaneous observer measurements, measur-
ing blood pressure with a mercury sphygmomanometer, on
Appendix C. Intra^arterial comparison each of eighty-five subjects. The selection of 85 subjects
The ESH Working Group agrees with the stipulations of was made on the ability to detect a somewhat arbitrary
the previous BHS protocol that intra-arterial comparisons error of 5 7 8 mmHg at a significance level of 0.05 and a
should not be recommended for general validation, while power of 0.98. The calculation was based on independent,
acknowledging that intra-arterial comparisons may in some rather than paired, samples for comparison, thereby
instances give information that cannot be obtained non- allowing for the fact that devices and observers may not
invasively [4]. If, however, intra-arterial comparisons are to measure blood pressure on exactly the same heart beat
be performed, they should be confined to centers with even when using simultaneous readings.
proven expertise in the technique, and the requirements of
EN 540, Clinical investigation of medical devices for human A blood pressure measuring device could pass the AAMI
subjects,, which requires among other stipulations that the protocol, but still be inaccurate. The BHS protocol
World Medical Declaration of Helsinki is fulfilled, that the identified two difficulties [3,4]. The first was that only
Ethics Committee be provided with information to assess average measurements were used in the analysis whereas
whether the risks to subjects, who cannot be expected to individual measurements would be identified in practice.
derive any direct therapeutic benefit, can be justified by The second was that, in using means and standard
the collective benefit, that provisions have been made to deviations, the percentage of measurements required to
compensate subjects in the event of injury, and that full be reasonably accurate, that is lying within 5 mmHg, was
informed consent is obtained from all subjects [44]. insufficient. Paradoxically, few outlying measurements are
permitted in the normal model whereas a more relaxed
A comparison between blood pressure measuring systems approach may be necessary in practice as variability can be
that utilize indirect measurement and those using the considerable in some subjects and may make a truly
direct intra-arterial measurement of blood pressure is not accurate reading appear otherwise.
recommended in this protocol. Apart from ethical con-
siderations, there are several reasons for this. Systolic and When the first BHS protocol was published in 1990 [3], the
diastolic blood pressure values obtained by the direct requirement to take three simultaneous measurements on
technique differ from measurements obtained by indirect each of 85 subjects was therefore retained, but the
methods [4,46]. Clinical practice derives from data measurements were no longer averaged, thus giving 255
obtained by the indirect rather than the direct technique. pairs of measurements for comparison. The accuracy
There is considerable beat-to-beat variation in blood criteria were based on the percentage of measurements
pressure, which is not reflected in indirect readings. Blood lying within 5, 10 and 15 mmHg. Furthermore, the
pressures measured directly and indirectly from the same possibility of device-induced bias was highlighted with a
artery are rarely (if ever) identical. Discrepancies in SBP as recommendation that bracketing sequential measurements
great as 24 mmHg and in DBP as much as 16 mmHg have be used as an alternative to simultaneous measurement. A
been observed when blood pressure has been measured by grading system was introduced to describe accuracy [3].
both techniques on the same arm at the same time. In
addition, these differences are random, displaying no In its revised protocol in 1993, the AAMI also recom-
schematic pattern [4,45]. mended that measurements no longer be averaged; it also
permitted the sequential technique when simultaneous
It is, however, recognized that valuable information on measurements were not feasible [5]. The 5 7 8 mmHg
device performance may derive from intra-arterial compar- accuracy criteria were retained.
isons in certain circumstances, such as validating devices
that analyse beat-by-beat blood pressure non-invasively, It has proved extremely difficult to recruit 85 subjects
but the International Protocol would need to be modified within the pressure range requirement of the previous
procedurally to allow intro-arterial comparisons and to test protocols; in practice, more than 100 subjects have been
Protocol for validating BP measuring devices in adults ESH Working Group 15

needed to fulfil the pressure range stipulations. A number preferable to a mean and standard deviation method of
of factors were considered in reducing sample size. validation.
1. The original statistical criteria were based on 85
Pressure ranges
measurements [2] whereas later protocols used 255
Prior to the introduction of the 1993 BHS protocol [4],
[3–5].
there was no specific recommendation on the range of
2. For grading results, percentage values are conceptually
blood pressure required for validation. As a consequence,
the most appropriate.
these varied greatly from one validation to the next. As
3. The more practical the study is, the more easily and
most devices fared worse in the high pressure ranges, this
more often it will be performed.
reduced the reliability and comparability of results [47].
Having performed 19 studies [7–23], we were able to re-
analyze the data from these studies to check the validity of To redress this problem, specific ranges were introduced.
new proposals. Taking all factors into consideration, the In particular, at least eight subjects in both the hypotensive
most appropriate sample size was 99 measurements, which and severe hypertensive ranges had to be recruited. The
provides more than the 85 measurement pairs required in reasoning behind the inclusion of the hypotensive range
the previous BHS and AAMI protocols [4,5] but with a was not only to assess accuracy in subjects with hypoten-
sample size of only 33 rather than 85 subjects. Although sion, but also to give some indication of accuracy for
there is some loss of measurement independence, the devices measuring ambulatory blood pressure during sleep
results compare well with independent measurements when the values can fall to low levels. Subjects with severe
[7–23]. hypertension were included because such levels are quite
common in hypertension clinics.
The selection of 33 subjects is based on two factors. First,
In practice, however, these two groups have proved
each subject has three measurements, each of which is
extremely difficult to find. The prevalence of persistent
used individually. This gives 99 sets of measurements,
hypotension is very low, and whereas severely hypertensive
which is larger than the 85 sets of measurements accepted
patients were to be found in specialist clinics, the
as the minimum necessary in the AAMI and BHS protocols
validation study had to be performed before blood
[4,5].
pressure-lowering drugs were prescribed, which was often
ethically impractical. Furthermore, during the resting
In comparing the variance of all 99 differences (total
laboratory phase of the validation procedure, blood
variance) with the 33 differences obtained from the mean
pressures in such subjects tended to fall below the
differences for each subject (the between-subject var-
required level required. Next and importantly, blood
iance), the F-test consistently yields a significantly lower
pressure in these subjects tended to be highly variable,
variance for the 33 subject mean differences than that for
making comparisons unreliable.
the 99 measurement differences. If between-subject
variance were the main cause of total variance, these
Finally, the division of subjects according to blood pressure
would not differ significantly. If, on the other hand, a
level did not lead to independent analysis. Indeed, tertile
device gave practically the same average error with each
analysis, included in the 1993 BHS protocol [4], was used
subject, the between-subject variation would be close to
only as a guide to accuracy, and the final recommendation
zero, and the F-test would show a very significant result.
was based on the overall analysis. The reason for this was
that most devices fared poorly in the upper tertile, with
Tests on data from previous experiments yield results of greater variability in this range being at least partly
probabilities of the between-subject variance and the total responsible [47].
variance being the same as lying between 0.1 and 0.01 for
SBP and between 0.2 and 0.02 for DBP. There should Given these difficulties and the fact that the comparisons
therefore be little difference between using single in these extremes are diluted in the overall analysis,
comparisons on 99 subjects and three comparisons on each specific requirements to include them are omitted in the
of 33 subjects. recommendations in this protocol. Although the range of
pressures has been reduced, all subjects must fit into a
The use of 99 subjects allows for an even distribution of specific category, whereas in the earlier protocols 10% of
blood pressures as these can easily be broken into three subjects could lie in any range (Table 1) [4].
ranges. It is also close to 100, which allows targets to be
considered as being approximate to percentages. Recommendations
The non-parametric recommendation system, shown in
Table 4(a–c) demonstrates how the choice-specific values Table 2a–c, considers both the subject/measurement and
for 5, 10 and 15 mmHg bands are more flexible and subject accuracy. White-coat hypertension and the morning
16 Blood Pressure Monitoring 2002,Vol 7 No 1

Table 4a Percentage of comparisons of devices satisfying Association for the Advancement of Medical Instrumentation criteria that fall
within a 5 mmHg error band
Within 5 mmHg Standard deviation (mmHg)
1 2 3 4 5 6 7 8
Mean (mmHg) 0 100.0% 98.6% 90.1% 78.5% 68.0% 59.3% 52.3% 46.6%
1 100.0% 97.4% 88.3% 77.1% 67.0% 58.7% 51.8% 46.3%
2 99.8% 93.1% 82.9% 73.0% 64.2% 56.7% 50.5% 45.4%
3 97.6% 84.0% 74.2% 66.6% 59.8% 53.7% 48.4% 43.8%
4 84.0% 69.1% 62.8% 58.5% 54.1% 49.7% 45.6% 41.8%
5 50.0% 50.0% 49.9% 49.3% 47.6% 45.0% 42.2% 39.3%
This table shows the expected percentage of errors of at most 5 mmHg for devices passing with mean absolute di¡erences of 0^5 mmHg and standard deviations of
1^8 mmHg.

Table 4b Percentage of comparisons of devices satisfying Association for the Advancement of Medical Instrumentation criteria that fall
within a 10 mmHg error band

This table shows the expected percentage of errors of at most 10 mmHg for devices passing with mean absolute di¡erences of 0^5 mmHg and standard deviations of
1^8 mmHg. The gray area indicates the improvement that might be obtained if the standard deviation is related to the mean, which in this instance is set so that the expected
number of di¡erences within 10 mmHg will be at least 85%.

Table 4c Percentage of comparisons of devices satisfying Association for the Advancement of Medical Instrumentation criteria that fall
within a 15 mmHg error band
Within15 mmHg Standard deviation (mmHg)
1 2 3 4 5 6 7 8
Mean (mmHg) 0 100% 100% 100% 100% 99.6% 98.6% 96.5% 93.6%
1 100% 100% 100% 100% 99.6% 98.4% 96.3% 93.4%
2 100% 100% 100% 99.9% 99.4% 98.1% 95.8% 92.8%
3 100% 100% 100% 99.8% 99.0% 97.4% 94.9% 91.8%
4 100% 100% 100% 99.6% 98.5% 96.4% 93.6% 90.4%
5 100% 100% 99.9% 99.3% 97.6% 95.0% 91.9% 88.5%
This table shows the expected percentage of errors of at most15 mmHg for devices passing with mean absolute di¡erences of 0 mmHg to 5 mmHg and standard deviations of
1^8mmHg. It shows that devices with 88.5% of measurements within this range could pass. One of the problems with the AAMI protocol is that, by setting the error indepen-
dently for mean and standard deviation, it permitted a very liberallevel of accuracy.FromTable 4a, it can be seen that where a device barely passes with a mean error of 5 mmHg
and a standard deviation of 8 mmHg, one could not expect even 40% of measurements to be accurate. Even devices passing more comfortably would have more than half of
theirexpected measurements classed asinaccurate.The acceptable standard deviation must be inversely related to the mean error.In practice, however, this tends not to be the
case as standard deviation tends to increase with error.This makes practical parametric passing criteria problematic.

alarm response are just two examples in which single When comparing a device measurement with its preceding
measurements are crucial. It is much easier for devices to and succeeding observer measurements, the nearer ob-
pass when only average subject measurements are used, server measurement is used. This poses a dilemma only if
and it would be wrong to assume that devices being the two observer measurements are equally close except for
recommended for use under such protocols are also sign, for example a device measurement of 150 mmHg and
accurate for individual measurements. observer measurements of 146 and 154 mmHg. One choice
would indicate that the device overestimates pressure
It must also be recognized that measurements near the whereas the other would indicate that it underestimates
extremes of the pressure range are more variable. Decisions pressure. The protocol recommends that whichever of the
should not therefore be based on small differences at these two observer measurements was taken first is selected.
limits, and zones are used to allow for this. The target This eliminates bias, and it is likely that the overestimating
requirements are based on existing protocols and the and underestimating selections will balance out over the 99
evidence from previous validation data. measurements.
Protocol for validating BP measuring devices in adults ESH Working Group 17

Membership of European Society Gianfranco Parati, Istituto Scientifico Ospedale San Luca,
of Hypertension Working Group on Blood IRCCS, Instituto Auxologico Italiano, 20149 Milan, Via
Pressure Monitoring Spagnoletto 3, Italy.
Roland Asmar, Société Française D’Hypertension Artérielle,
Fillale de la Société Française de Cardilogie, 15, rue de Paolo Palatini, Dipartimento di Medicina Clinica e
Cels-75014, Paris, France. Sperimentale, Universita’ di Padova, Via Giustiniani 2, I-
Lawrie Beilin, Department of Medicine, University of 35128 Padua, Italy.
Western Australia, Australia, GPO Box x2213, 35 Victoria
Square, Perth WA 600, Australia. Thomas G. Pickering, Director, Integrative and Behavioral
Cardiovascular Health Program, Mount Sinai Medical
Denis L. Clement, Afdeling Hart-en Vaatziekten, Uni-
Centre, New York, NY 10029-6574, USA.
versitair Ziekenhuis, De Pintelaan 185, B-9000 Gent,
Belgium.
Josep Redon, Hypertension Clinic, Internal Medicine,
Peter De Leeuw, Interne Geneeskunde, Academisch Hospital Clinico, University of Valencia, Avda Blasco
Ziekenhuis, P. Debyelaan 25, postbus 5800, 6202 AZ Ibañez, 17. 46010. Valencia, Spain.
Maastricht, The Netherlands.
Jan Staessen, Katholieke Universiteit Leuven, Hypertensie
Robert Fagard, Katholieke Universiteit Leuven, Hyperten- en Cardiovasculaire Revalidatie Eenheid, Inwendige Gen-
sie en Cardiovasculaire Inevalidatie Eenheid, Inwendige eeskunde-Cardiologie, U.Z. Gasthuisberg, Herestraat 49,
Geneeskunde-Cardiologie, U.Z. Gasthuisberg, Herestraat 3000 Leuven, Belgium.
49, 3000 Leuven, Belgium.
George Stergiou, Hypertension Center, Third University
Yutaka Imai, The Department of Clinical Pathology and Department of Medicine, Sotiria Hospital, Athens, Greece.
Therapeutics, Tohoku University Graduate School of
Pharmaceutical Science and Medicine, 1-1 Seiryo-Cho,
Gert van Montfrans, Academisch Medisch Centrum,
Aoba-Ku, Sendai 980-8574, Japan.
Interne Ziekten, Meibergdreef 9, AZ 1005 Amsterdam,
The Netherlands.
Jean-Michel Mallion, Médecine Interne et Cardiologie,
Chef de Service, Centre Hospitalier Universitaire de
Grenoble, B.P. 217 - 38043 Grenoble Cedex, France. Paolo Verdecchia, Departimento di discipline Cardiovasco-
lari, Ospedale R. Silvestrini, Perugia, Italy.
Giuseppe Mancia, Universita Degli Studi di Milano-
Bicocca, Cattedra di Medicina Interna, Ospedale San Bernard Waeber (Secretary), Centre Hospitalier Universi-
Gerardo Dei Tintori, Via Donizetti, 106, 20052 Monza, taire Vaudois, Division D’Hypertension, Departement de
Italy. Medecine Interne, 1011 Lausanne, Switzerland.

Thomas Mengden, University Clinic Bonn, Department of William White, Section of Hypertension and Vascular
Internal Medicine, Wilhelmstrasse 35 5311 Bonn, Germany. Diseases, University of Connecticut Health Center, 263
Farmington Avenue, Farmington, Connecticut 06030-3940,
Martin G. Myers, Division of Cardiology, Sunnybrook and USA.
Women’s College Health Sciences Centre, 2075 Bayview
Avenue, Toronto, Ontario M4N 3M5, Canada.
Statistical Assistance:
Eoin O’Brien (Chairman), Blood Pressure Unit, Beaumont Neil Alkins, Blood Pressure Unit, Beaumord Hospital,
Hospital, Dublin 9, Ireland. Dublin 9, Ireland.

Paul Padfield, Department of Medicine, Western General William Gerin, Mount Sinai School of Medicine, New York,
Hospital, Edinburgh EH4 2XU, UK. NY, USA.

Potrebbero piacerti anche