European Journal of Obstetrics & Gynecology and

Reproductive Biology 137 (2008) 146151

www.elsevier.com/locate/ejogrb

Development of a nomogram to predict occurrence of preeclampsia

Stephanie Deis, Roman Rouzier *, Gilles Kayem, Christophe Masson, Bassam Haddad
Department of Obstetrics and Gynecology, Centre Hospitalier Intercommunal de Creteil and University of Paris 12, Creteil, France

Received 12 November 2006; received in revised form 11 March 2007; accepted 21 May 2007

Abstract

Objective: The objective was to create a nomogram for the individual prediction of preeclampsia (PE).
Study design: In a prospective population-based study that included 4777 patients, PE occurred in 2.4%. Age, body mass index, parity,
previous preeclampsia (PPE), chronic hypertension, diastolic blood pressure (DBP), and proteinuria at first visit, and second trimester
ultrasonography and umbilical artery Doppler resistance index (UARI) data were used to develop and calibrate a nomogram based on a
multivariate logistic regression model.
Results: Based on multivariate analysis, nulliparity (P = 0.002), PPE (P = 0.004), DBP (P < 0.0001), biparietal diameter (P = 0.011), and
UARI (P = 0.08) were introduced into a nomogram. Based on these variables, the nomogram had good discrimination (area under the ROC
curve = 0.73, P < 0.01) and calibration (unreliability index = 5.2  10 4). This nomogram was validated by bootstrapping.
Conclusion: Our nomogram predicts the probability of preeclampsia. After validation in an independent population, this tool could be used to
plan a preventive trial.
# 2007 Elsevier Ireland Ltd. All rights reserved.

Keywords: Preeclampsia; Nomogram; Individual prediction; Ultrasonography

1. Introduction ultrasonography in a nomogram for predicting the individual

Preeclampsia complicates 3% of pregnancies and is one
of the principal causes of maternal and perinatal morbidity
and mortality. Its prevention requires the availability of
markers to predict or identify those women at high risk of
this complication. Several clinical tests have been proposed
for the prediction of preeclampsia. Most of these tests,
however, have poor sensitivity and poor positive predictive
values [1,2]. At present, there is no single screening test that
is considered reliable for predicting preeclampsia. As a
result, all studies on prevention have included women with
various risk factors for preeclampsia [3].
Assessment of the risk of preeclampsia is challenging as
there is no available tool that can clearly separate high-risk
patients from low-risk patients. The aim of this study was to
combine first visit clinical markers and second trimester
* Corresponding author at: Department of Obstetrics and Gynecology,
Centre Hospitalier Intercommunal de Creteil, 40 Avenue de Verdun, 94010
Creteil Cedex, France. Tel.: +33 1 45 17 55 43; fax: +33 1 45 17 55 42.
E-mail address: roman.rouzier@tnn.aphp.fr (R. Rouzier).
risk of preeclampsia.
2. Materials and methods
Between 2001 and 2003, 5011 patients were prospec-
tively included in the study at their first prenatal visit at the
Maternity University Creteil. Women who were preeclamp-
tic at the first visit were excluded. Women who did not
deliver at our maternity were excluded from the analysis.
The final cohort comprised 4777 patients. Patient char-
acteristics are reported in Table 1. Patients were followed
from their first prenatal visit in our center until delivery with
a monthly visit. The examination included blood pressure
measurement and a urinary dipstick test to detect
proteinuria. Second trimester ultrasonography was usually
performed at 2023 weeks gestation. Measurement of
biparietal diameter, abdominal perimeter, femur length, and
umbilical artery Doppler resistance index (UARI) was
routinely performed. Raw values were converted to
percentiles to account for gestational age at ultrasonography.

0301-2115/$ see front matter # 2007 Elsevier Ireland Ltd. All rights reserved.
doi:10.1016/j.ejogrb.2007.05.022
 S. Deis et al. / European Journal of Obstetrics & Gynecology and Reproductive Biology 137 (2008) 146151
Table 1
Patient characteristics (n = 4777)
Age (mean  S.D., years)
Nulliparous (%)
Twin pregnancy (%)
Ethnicity
Caucasian (%)
Arabic (%)
Black (%)
Mixed/other (%)
Tobacco use (%)
BMI (kg/m2)
<25 (%)
2530 (%)
>30 (%)
Previous preeclampsia (%)
Chronic hypertension (%)
Auto-immune disease (%)
Gestational age at entry (mean  S.D., weeks)
BMI: body mass index.
Percentiles were obtained from the Colle`ge Francais
dEchographie Ftale at https://www.cfef.org/archives/
communication/biometrie2000/selectframe.html.
Preeclampsia was defined as the association of gesta-
tional hypertension (systolic blood pressure 140 mmHg
and/or a diastolic blood pressure 90 mmHg on at least two
occasions at least 6 h apart after 20 weeks gestation in
women known to be normotensive) plus proteinuria
(300 mg/24 h). When 24-h urine collection was not
feasible, proteinuria was defined as a concentration
30 mg/dl in at least two random urine samples collected
at least 6 h apart [4]. Patients referred to our tertiary care
center for preeclampsia were not included in the study.
Multivariate logistic regression analysis was used to test
the association between clinical variables obtained at the
first prenatal visit and second trimester ultrasonography
characteristics and occurrence of preeclampsia. Clinical
variables tested included age, ethnicity, body mass index
(<25; 2530; >30 kg/m2), twin pregnancy, parity, previous
preeclampsia (PPE), auto-immune disease, chronic hyper-
tension, tobacco use, previous intra-uterine fetal death,
history of first trimester miscarriage, arterial blood pressure
(systolic blood pressure [SBP], diastolic blood pressure
[DBP], and mean arterial blood pressure), and urinary
protein sampling (categorized as 0, 30, 100, and
300 mg/dl). Ultrasonography variables included UARI,
biparietal diameter, abdominal circumference, and femur
length. To determine informative covariates, we performed a
backward variable selection using a method based on
Lawless and Singhal [5]. This method uses the fitted
complete model and computes approximate Wald statistics
by calculating conditional (restricted) maximum likelihood
estimates assuming multivariate normality of the estimates.
Informative variables were selected according to the
Akaikes information criterion [6]. We did not include
147
interaction terms in the model. The logistic regression model
was used to construct the nomogram.
30  5 The model performance was quantified with regard to
55
discrimination and calibration [7,8]. Discrimination, i.e.,
4
whether the relative ranking of individual predictions is in
the correct order, was quantified with the concordance index,
58
18
which is similar to the area under the receiver operating
17 characteristic curve. The concordance index is the prob-
7 ability that given two randomly selected patients, the patient
25 with the worse outcome will in fact have a worse prediction.
The concordance index ranges from 0 to 1, with 1 indicating
68
perfect concordance, 0.5 indicating no association (no better
20 than flipping a coin), and 0 indicating perfect discordance.
12 We used the bootstrapping technique to obtain relatively
1 unbiased estimates (200 repetitions): it provides an estimate
2 of the average optimism of the concordance index when all
1 data are included. Calibration, i.e., agreement between
21  1 observed outcome frequencies and predicted probabilities,
was studied with graphical representations of the relation-
ship between the observed outcome frequencies and the
predicted probabilities (calibration curves) for groups of
patients defined by quartiles. This was done by grouping
patients with regard to their nomogram-predicted probabil-
ities and then comparing the mean of the group with the
observed occurrence of preeclampsia. A calibration curve
can be approximated by a regression line with intercept a
and slope b. These parameters can be estimated in a logistic
regression model with the event as outcome and the linear
predictor as the only covariate. Well-calibrated models have
a = 0 and b = 1. Therefore, a sensible measure of calibration
is a likelihood ratio statistic testing the null hypothesis that
a = 0 and b = 1, which we used. The statistic has a chi-
squared distribution with two degrees of freedom (unrelia-
bility [U] statistic) [9]. We used the bootstrapping technique
to internally validate the nomogram. Internal validation does
not demonstrate the exportability of a model, but is
necessary before external validation. All analyses were
performed using the R package with the Survival, Design,
Hmisc, Party, and Lexis libraries (http://lib.stat.cmu.edu/R/
CRAN/) [10,11]. Local ethics committee agreement was
obtained for this study.
3. Results
Preeclampsia occurred in 110 patients (2.4%). In a
multivariate model, nulliparity (P = 0.0021), previous
preeclampsia (P = 0.0039), DBP at the first visit
(P < 0.0001), UARI (P = 0.08), biparietal diameter
(P = 0.011), and twin pregnancy (P = 0.0007) were asso-
ciated with occurrence of preeclampsia (Table 2) and were
informative in the model, while chronic hypertension, body
mass index, ethnicity, history of first trimester miscarriage,
previous intra-uterine fetal death, auto-immune disease,
tobacco use, urinary proteinuria at first prenatal visit,
abdominal perimeter, and femur length were not.
148 S. Deis et al. / European Journal of Obstetrics & Gynecology and Reproductive Biology 137 (2008) 146151

Table 2
Factors associated with occurrence of preeclampsia (multivariate analysis)
OR 95% CI P
Nulliparous 2.06 1.632.60 0.0021
Previous preeclampsia 5.08 2.898.92 0.0039
DBP 1.08 1.071.09 <0.0001
UARI 1.98 1.342.93 0.08
Biparietal diameter 0.36 0.240.54 0.011
Twin pregnancy 3.60 2.475.27 0.0007
OR: odds ratio; CI: confidence interval; DBP: diastolic blood pressure;
UARI: umbilical artery resistance index.

A nomogram was built from this model (Fig. 1). A

nomogram is used by first locating the patients position on
each predictor variable scale. Each scale position has
corresponding prognostic points at the top axis. The points
for each variable are added and the probability of
preeclampsia occurrence is estimated from the bottom line.
Fig. 2 represents the discrimination of the nomogram. The
nomogram based on the variables included in the model had Fig. 2. Discrimination of the model: the concordance indices before and
good discrimination: the concordance indices before and after bootstrapping were 0.735 (P < 0.001) and 0.729 (P < 0.001).
after bootstrapping were 0.735 (P < 0.001) and 0.729
(P < 0.001), respectively. Fig. 3 shows how the comparison
between predictions from the nomogram relates to the actual

Fig. 1. Nomogram to estimate the probability of preeclampsia. The nomogram can be used by finding predictor points indicated by the uppermost point scales
that correspond to each patient variable value; the sum of the points (six variables) is projected to the probability of preeclampsia scale at the bottom in order to
determine the probability of preeclampsia. PE: preeclampsia; DBP: diastolic blood pressure; UARI: umbilical artery resistance index.
 S. Deis et al. / European Journal of Obstetrics & Gynecology and Reproductive Biology 137 (2008) 146151
Fig. 3. Calibration of the model: the x-axis is the probability of preeclamp-
sia calculated with the nomogram and the y-axis is the actual rate of
preeclampsia. The dashed line represents the performance of an ideal
nomogram. The dotted line is the apparent accuracy without correction
for over-fit. The solid line is the bootstrap-corrected nomogram perfor-
mance.
outcomes of the 4777 patients. The x-axis is the prediction
calculated by the nomogram and the y-axis is the actual rate
of preeclampsia. The dotted line represents the performance
of an ideal nomogram in which predicted outcome perfectly
corresponds to actual outcome. The performance of the
nomogram is plotted in two ways. The dashed line is the
apparent accuracy without correction for over-fit. The solid
line is the bootstrap-corrected nomogram performance, a
scatter estimate of future accuracy. The unreliability index
was 5.2  10 4. The predictions calculated with use of the
nomogram correspond accurately to the actual outcomes of
patients with a 15% or less risk of preeclampsia. Above
15%, the prediction was less accurate. Of note, 90% of
patients have an estimated probability of preeclampsia
occurrence less than 4.4 and 95% of patients less than 5.9%.
Fig. 4. Example of the web interface available at http://www.ecsvd.org/nomogram.html.
149
We developed a computer program to help patients and
physicians use the nomogram. The program called
preeclampsia! (version 1) is programmed in Java text. An
Internet browser with Java enablement is required to run the
applets. The applets will be available on line at http://
www.ecsvd.org/nomogram.html. An example of the web
interface is represented in Fig. 4.
4. Comment
In this study, we demonstrate that clinical variables and
second trimester ultrasonography data can be combined into
a nomogram for predicting the individual risk of pre-
eclampsia. This nomogram could be used to identify
candidates for intensive screening, prevention, or research
studies. Based on a cohort of 4777 patients, we found along
with others that nulliparity [12], previous history of
preeclampsia [13], twin pregnancy [14], diastolic blood
pressure at the first prenatal visit [1417], and biparietal
diameter were associated with occurrence of preeclampsia.
In contrast with previous studies that identified risk factors
for preeclampsia, we were able to build and validate a tool
that might be used to predict individual risk of preeclampsia
[18,19]. Other predictors have been reported, but they are
based on serum markers or ultrasonography data that are not
routinely performed [18,19]. Moreover, some of these
predictors are based on case control studies and their
utilization in a wider setting needs to be demonstrated [20].
In contrast, the predictor we developed is based on widely
used criteria. We provide a combination of well-known risk
factors for preeclampsia rather than new criteria to predict
preeclampsia. This tool can be used on an individual level to
decide whether to carry out close follow-up or prevention in
high-risk patients.
The most significant covariate in our nomogram was
diastolic blood pressure at the first visit. In fact, diastolic
blood pressure may account for chronic hypertension. We
initially tested chronic hypertension as a potential covariate
during the development of the nomogram. Chronic
150 S. Deis et al. / European Journal of Obstetrics & Gynecology and Reproductive Biology 137 (2008) 146151
hypertension was associated with occurrence of preeclamp-
sia in the univariate analysis (P = 0.03, data not shown);
however, it did not remain significant in the multivariate
analysis because of the interaction with diastolic blood
pressure. Finally, insulin-deficient diabetes was also
analyzed in our study. However, the number of patients
with that condition (n = 13) was too low to draw any
conclusion.
The nomograms provide probability estimates that might
be useful on an individual level. For example, a primiparous
woman (10 points), with a singleton pregnancy (0 point), a
diastolic blood pressure at first visit of 80 mmHg (42 points),
an umbilical artery resistance index >90th percentile (10
points), and a biparietal diameter at the 50th percentile (8
points) scores 70 points, which translates into a 5%
probability of having preeclampsia. A woman with a history
of preeclampsia during a previous pregnancy (23 points), a
diastolic blood pressure at first visit of 85 mmHg (50 points),
an umbilical artery resistance index >90th percentile (10
points), and a biparietal diameter <10th percentile (13
points) scores 96 points, which translates into a 38%
probability of subsequent preeclampsia, which should be
interpreted as a more than 15% probability of preeclampsia
(Fig. 3). Conversely, a woman with a previous history of
preeclampsia (23 points), but with a diastolic blood pressure
at first visit of 50 mmHg (10 points), an umbilical artery
resistance index at the 50th percentile (5 points), and a
biparietal diameter at the 50th percentile (8 points) scores 46
points, which translates into a <4% probability of
subsequent preeclampsia. These two latter examples
demonstrate that two patients with the same population-
based estimate of preeclampsia occurrence (2235% based
on their personal history of preeclampsia [21]) actually have
different individual risk estimates. This may explain the
negative results of trials that studied the prevention of
preeclampsia by low-dose aspirin in a high-risk population
[22].
We plan to further improve our nomogram by adding
biological markers such as early second trimester serum
human chorionic gonadotropin [23] and second trimester
uterine Doppler [24]. It is likely that combining clinical data
and serum markers will provide a more accurate predictor
compared with the use of only one of these elements [25].
When preventive medication for preeclampsia becomes
available, we believe that our nomogram will be a useful tool
for selecting eligible patients.
However, there are limitations to our study. First, even
though our study population included almost 5000 patients,
some characteristics that have been previously reported as
risk factors for preeclampsia (body mass index, chronic
hypertension, auto-immune disease, ethnicity) did not
emerge in our model [14]. The low prevalence of
preeclampsia in our study is likely to be the reason for
this finding and may be explained by patients character-
istics (ethnicity, low prevalence of obesity and chronic
hypertensive disorders, etc.), which are different between
our population and the published data (mainly by North
American teams). In the Medical Birth Registry of Norway,
which includes data on 547,238 women with a first and
second pregnancy, the prevalence of preeclampsia in the first
pregnancy was 3.6% (n = 19,970) and in the second
pregnancy 1.7% (n = 9535), corroborating our figures
[26]. In fact, more than 3% of patients delivering in our
tertiary center have preeclampsia. However, for the purposes
of the study, we did not include patients referred for
preeclampsia and performed a population-based study on
the cohort of patients with a first prenatal visit in our
Maternity University before 25 weeks gestation.
Second, variables of our nomograms were obtained
between 20 and 23 weeks gestation; therefore, the
nomogram should be used within this range. Interestingly,
even if percentiles were obtained from the Colle`ge
Francais dEchographie Ftale, percentiles could be
obtained within the study population with similar results.
The size of our population (more than 4500 women)
explains this finding.
Third, we validated our nomogram by bootstrapping,
which essentially allows correction for over-fitting, but the
generalizability of our nomogram has to be demonstrated by
applying it in an independent population [8]. The boot-
strapping of the calibration curve in our study showed that
individual estimates were not well calibrated over 15%,
suggesting that the nomogram was able to identify a group
at high risk of preeclampsia (>15%), but was not able to
provide individual predictions over 15%. We do not think
that it is a matter of concern in daily practice for two
reasons. First, one has to remember that a 10% risk of
preeclampsia corresponds to a relative risk >4. Second, the
identification of a group of very high-risk patients is more
important than a raw score. In contrast, for other patients
(the vast majority of patients), any preventive intervention
would carry a huge financial cost and will lead to over-
treatment because of the low prevalence of preeclampsia.
Therefore, extremely well-calibrated predictors for low-
and intermediate-risk patients are mandatory. Our nomo-
gram is very well calibrated for these patients and could be
useful for calculating the cost/benefit ratio of a preventive
trial and selecting the most efficient design. The utility of
our nomogram can be illustrated by designing a preventive
trial based on it. If we consider that a preventive action will
reduce the occurrence of preeclampsia by 20%, for
example, in order to detect an effect in the general
population (prevalence 2.5%), the trial should include 1136
patients in each arm to demonstrate the benefit with a = 5%
and b = 20%. Conversely, if only patients with more than an
8% probability of preeclampsia are included in the trial, the
number of patients to include will be 137 in each arm. In the
first design, 6 occurrences of preeclampsia out of 27 would
have been prevented versus four out 21 in the second design.
The number of patients over-treated and the cost of the first
design clearly favor the second design and indicate the
utility of our nomogram.
 S. Deis et al. / European Journal of Obstetrics & Gynecology and Reproductive Biology 137 (2008) 146151
5. Conclusion
Our nomogram predicts the probability of preeclampsia.
We also developed a web-based interface to provide the
physician with a friendly version of our nomogram. After
validation in an independent population, this tool could be
used to plan a preventive trial and could serve as the basis for
integrating future markers into a bio-clinical prediction
model.
References
[1] Friedman SA, Lindheimer MD. Prediction and differential diagnosis.
In: Lindheimer MD, Roberts JM, Cunningham FG, editors. Chesleys
hypertensive disorders in pregnancy. Appleton and Lange: Stamford,
Connecticut; 1999. p. 20127.
[2] Dekker GA. Risk factors for preeclampsia. Clin Obstet Gynecol
1999;42:42235.
[3] Sibai BM. Prevention of preeclampsia: a big disappointment. Am J
Obstet Gynecol 1998;179:12758.
[4] Report of the National High Blood Pressure Education Program.
Working group report on high blood pressure in pregnancy. Am J
Obstet Gynecol 2000;183:S122.
[5] Lawless JF, Singhal K. Efficient screening of nonnormal regression
models. Biometrics 1978;34:31827.
[6] Akaike H. Information theory and extension of the maximum like-
lihood principle. In: Petrov BN, Csaki F, editors. Proceedings of the
2nd international symposium of information theory. 1973. p. 26781.
[7] Kattan MW, Scardino PT. Prediction of progression: nomograms of
clinical utility. Clin Prostate Cancer 2002;1:906.
[8] Harrell Jr FE. Regression modeling strategies, with applications to
linear models. Logistic regression, and survival analysis. New York:
Springer; 2001.
[9] Cox DR. Two further applications of a model for binary regression.
Biometrika 1958;45:5625.
[10] Harrell Jr FE. Hmisc: a package of miscellaneous S functions. Nash-
ville, Tennessee, USA: Vanderbilt University; 2005, http://biostat.mc.-
vanderbilt.edu/s/Hmisc.
[11] R Development Core Team. R: a language and environment for
statistical computing. Vienna, Austria: R Foundation for Statistical
Computing; 2005, http://www.R-project.org/.
[12] Eskenazi B, Fenster L, Sidney S. A multivariate analysis of risk factors
for preeclampsia. JAMA 1991;266:23741.
151
[13] Mostello D, Catlin TK, Roman L, Holcomb Jr WL, Leet T. Pre-
eclampsia in the parous woman: who is at risk? Am J Obstet Gynecol
2002;187:4259.
[14] Duckitt K, Harrington D. Risk factors for pre-eclampsia at antenatal
booking: systematic review of controlled studies. BMJ 2005;330:
5657.
[15] Higgins JR, Walshe JJ, Halligan A, et al. Can 24-hour ambulatory
blood pressure measurement predict the development of hypertension
in primigravidae? Br J Obstet Gynaecol 1997;104:35662.
[16] Sibai BM, Ewel M, Levine RJ, et al. Risk factors associated with
preeclampsia in healthy nulliparous women. Am J Obstet Gynecol
1997;177:100310.
[17] Sibai BM, Gordon T, Thom E, et al. Risk factors for preeclampsia in
healthy nulliparous women: a prospective multicenter study. Am J
Obstet Gynecol 1995;172:6428.
[18] Fayyad AM, Harrington KF. Prediction and prevention of preeclamp-
sia and IUGR. Early Hum Dev 2005;81:86576.
[19] Papageorghiou AT, Yu CK, Erasmus IE, Cuckle HS, Nicolaides KH.
Assessment of risk for the development of pre-eclampsia by
maternal characteristics and uterine artery Doppler. BJOG
2005;112:7039.
[20] Spencer K, Yu CK, Savvidou M, Papageorghiou AT, Nicolaides KH.
Prediction of pre-eclampsia by uterine artery Doppler ultrasonogra-
phy and maternal serum pregnancy-associated plasma protein-A, free
beta-human chorionic gonadotropin, activin A and inhibin A at
22 + 0 to 24 + 6 weeks gestation. Ultrasound Obstet Gynecol
2006;27:65863.
[21] Sibai BM. Diagnosis, prevention, and management of eclampsia.
Obstet Gynecol 2005;105:40210.
[22] Caritis S, Sibai B, Hauth J, et al. Low-dose aspirin to prevent
preeclampsia in women at high risk. N Engl J Med 1998;338:70115.
[23] Ashour AMN, Lieberman ES, Wilkins Haug LE, Repke JT. The value
of elevated second-trimester beta-human chorionic gonadotropin in
predicting development of preeclampsia. Am J Obstet Gynecol
1997;176:43842.
[24] Bower S, Schuchter K, Campbell S. Doppler ultrasound screening as
part of routine antenatal scanning: prediction of preeclampsia and
intrauterine growth retardation. Br J Obstet Gynaecol 1993;100:
98994.
[25] Nevins JR, Huang ES, Dressman H, et al. Towards integrated clinico-
genomic models for personalized medicine: combining gene expres-
sion signatures and clinical factors in breast cancer outcomes
prediction. Hum Mol Genet 2003;12(S2):1537.
[26] Trogstad LI, Eskild A, Magnus P, Samuelsen SO, Nesheim BI.
Changing paternity and time since last pregnancy; the impact on
pre-eclampsia risk. A study of 547 238 women with and without
previous pre-eclampsia. Int J Epidemiol 2001;30:131722.