Sei sulla pagina 1di 6

See

discussions, stats, and author profiles for this publication at: http://www.researchgate.net/publication/7168759

Spanish Cross-Cultural Adaptation and Validation of the


National Institutes of Health Stroke Scale
ARTICLE in MAYO CLINIC PROCEEDINGS APRIL 2006
Impact Factor: 6.26 DOI: 10.4065/81.4.476 Source: PubMed

CITATIONS

READS

20

70

8 AUTHORS, INCLUDING:
Federico Augustovski

Vilma Irazola

Institute for Clinical Effectiveness and Health Policy

Institute for Clinical Effectiveness and Health Policy

109 PUBLICATIONS 837 CITATIONS

40 PUBLICATIONS 127 CITATIONS

SEE PROFILE

SEE PROFILE

Pablo R. Castillo

Roberto. Juan Rotta Escalante

Mayo Clinic

Argentine Foundation Against Neurological Diseases and old a

36 PUBLICATIONS 1,115 CITATIONS

29 PUBLICATIONS 125 CITATIONS

SEE PROFILE

SEE PROFILE

Available from: Roberto. Juan Rotta Escalante


Retrieved on: 11 December 2015

ORIGINAL
SPANISH-VERSIONARTICLE
NIHSS

Spanish Cross-Cultural Adaptation and Validation of the


National Institutes of Health Stroke Scale
RAL DOMNGUEZ, MD; JOS F. VILA, MD; FEDERICO AUGUSTOVSKI, MD, MSC; VILMA IRAZOLA, MD;
PABLO R. CASTILLO, MD; ROBERTO ROTTA ESCALANTE; THOMAS G. BROTT, MD; AND JAMES F. MESCHIA, MD

OBJECTIVES: To adapt and validate a Spanish-language version


(SV) of the National Institutes of Health Stroke Scale (NIHSS) to
facilitate its use in Spanish-speaking contexts.
PATIENTS AND METHODS: The methods recommended by the
International Quality of Life Assessment Project were followed.
Two forward translations and 1 back translation of the NIHSS were
developed to ensure lingual and cultural equivalence. A final
revised SV-NIHSS was administered by 8 physicians to patients
with stroke in 3 clinics in Buenos Aires, Argentina, from September 2003 to December 2003.
RESULTS: The study included 102 patients (mean SD age,
73.36.5 years; 56% women) with stroke (86% ischemic). The SVNIHSS mean baseline score was 9.787.04. Interrater reliability
was independently evaluated for 98 patients, showing a high
agreement: , 0.77 to 0.99 for the 15 items; interrater correlation
coefficient, 0.991 (95% confidence interval, 0.987-0.994). Intrarater reliability was excellent: , 0.86 to 1.00 for the 15 items;
mean intrarater correlation coefficient, 0.994 (95% confidence
interval, 0.991-0.996). Construct validity was also adequate; the
SV-NIHSS had a negative correlation with baseline Glasgow Coma
Scale (Spearman coefficient = 0.574, P<.001) and with Barthel
index at 3 months (Spearman coefficient = 0.658, P<.001). Patients with different Rankin scores at 3 months also had significantly different baseline SV-NIHSS scores, from a mean of 4.292.21
for Rankin score of 0 to a mean of 29.403.97 for Rankin score of 6
(P<.001).
CONCLUSION: This study shows that a Spanish-language version of
the NIHSS developed with internationally recommended methods
is reliable and valid when applied in a Spanish-speaking setting.

Mayo Clin Proc. 2006;81(4):476-480


CI = confidence interval; NIHSS= National Institutes of Health Stroke
Scale; SV-NIHSS= Spanish-language version of the NIHSS

he National Institutes of Health Stroke Scale (NIHSS)


is a 15-item clinical evaluation instrument widely used
in clinical trials and practice to assess neurologic outcome
and the evolution and degree of recovery in patients with
stroke. The original English scales reliability and validity
are widely documented in the literature.1-8 More recently,
shorter forms have been developed to decrease administration time and to define adequate work-up and
treatment more quickly in patients with stroke.9,10 However, it would be helpful to have a validated Spanishlanguage version of the NIHSS to be used by Spanishspeaking physicians with Spanish-speaking patients. The
development of such a Spanish version should follow the
standard methods of cross-cultural adaptation and valida476

Mayo Clin Proc.

tion. In this study, we developed and evaluated a Spanishlanguage version of the NIHSS (SV-NIHSS).
PATIENTS AND METHODS
The study was performed in 2 phases. The first phase involved the construction of the SV-NIHSS, and the second
aimed to test its validity and reliability. The study was approved by the ethics committees of the participating centers.
PHASE 1
Development of the SV-NIHSS. The first phase of the
study consisted of the cross-cultural adaptation of the original English version of the NIHSS. The methods of adaptation followed those proposed by the International Quality
of Life Assessment Project11-13 and are similar to those
recommended by other authors.14 The cross-cultural adaptation process was composed of 4 steps.
Step 1: Two forward translations of the original NIHSS
were made by 2 independent physician translators (F.A.
and P.R.C.). From these initial versions, a first intermediate
Spanish version was produced.
Step 2: A back translation of the first intermediate
Spanish version was performed by a translator in the
International Office at Mayo Clinic in Jacksonville, Fla.
The back translation was compared with the original
NIHSS to assess conceptual equivalence and detect possible misunderstandings or misinterpretations. On the basis of the comparison of the back translation and the
original, a second intermediate Spanish version of the
NIHSS was produced.
From the University of Buenos Aires Hospital Sirio Libans (R.D.), Instituto
Mdico ENERI Ciudad Autnoma de Buenos Aires (J.F.V.), Instituto de
Efectividad Clnica y Sanitaria (F.A., V.I.), Unidad de Medicina Familiar y
Preventiva (F.A.), Hospital Italiano and Policlnica Bancario (R.R.E.), Buenos
Aires, Argentina; and Sleep Medicine (P.R.C.) and Department of Neurology
(T.G.B., J.F.M.), Mayo Clinic College of Medicine, Jacksonville, Fla. Dr Castillo
is now with the University of Minnesota, Minneapolis.
This study was supported by an independent grant from Aventis Pharma,
Argentina.
Individual reprints of this article are not available. Address correspondence to
James F. Meschia, MD, Department of Neurology, Mayo Clinic College of
Medicine, 4500 San Pablo Rd, Jacksonville, FL 32224 (e-mail: meschia.james
@mayo.edu).
2006 Mayo Foundation for Medical Education and Research

April 2006;81(4):476-480

www.mayoclinicproceedings.com

For personal use. Mass reproduce only with permission from Mayo Clinic Proceedings.

SPANISH-VERSION NIHSS

Spanish words
(English in parentheses)

Spanish phrases
(English in parentheses)

Mam
(Mother)

Usted sabe cmo


(You know how)

Ta te ti
(Tic tac toe)

Con los pies sobre la tierra


(With the feet on the ground)

Mitad y mitad
(Half and half)

Llegu del trabajo a mi casa


(I arrived home from work)

Gracias
(Thank you)

Cerca de la mesa en el comedor


(Near the table in the dining room)

rbol
(Tree)

Anoche lo escucharon hablar en la radio


(They heard him talk on the radio yesterday)

Futbolista
(Soccer player)

FIGURE 3. Items to test for physical findings of alexia.

Step 3: The second intermediate version was reviewed by


a panel of 7 Mayo Clinic staff physicians whose native
language is Spanish to assess comprehension and acceptability of the translated scale. The countries of origin of the
reviewers were Ecuador, Mexico, Puerto Rico, Peru, Venezuela, Colombia, and Dominican Republic. The different
Spanish backgrounds of the reviewers helped to accommodate
any differences in dialect among Spanish-speaking countries
and to produce a neutral version. They reviewed the SVNIHSS individually and made corrections and additions. Any
discrepancies that arose were resolved by consensus to produce a third intermediate version of the SV-NIHSS.
Step 4: The third intermediate version was evaluated
by 2 independent proofreaders to correct any spelling,

grammar, or other mistakes; this version became the final preliminary SV-NIHSS (Appendix 1 is available at
www.mayoclinicproceedings.com linked to this article).
To evaluate language, visual extinction, and inattention,
2 pages with drawings are shown to patients. After conducting a pilot study among physicians and patients, some
modifications to the original drawings were made so that
they would be readily understandable by the Spanishspeaking population. A list of Spanish words was created to
test for dysarthria, and a list of Spanish phrases was created
to test for alexia (Figures 1 through 4).
Interviewer Training. A total of 8 physician interviewers participated in the study. Three were certified by the
NIH for using the original NIHSS. The other 5 were intensively trained to increase the reliability of the SV-NIHSS.
The training consisted of watching the original NIHSS
instructional videos, studying the SV-NIHSS and its in-

FIGURE 2. Items to test for physical findings of impairment in visual


confrontation naming.

FIGURE 4. Picture that the test subject is to describe to the examiner to


test for visual hemineglect and to test for impairment of verbal fluency.

FIGURE 1. Items to test for physical findings of dysarthria.

Mayo Clin Proc.

April 2006;81(4):476-480

www.mayoclinicproceedings.com

For personal use. Mass reproduce only with permission from Mayo Clinic Proceedings.

477

SPANISH-VERSION NIHSS

TABLE 1. Baseline Characteristics of the Study Group* (N=102)


Characteristic

Value

Age (y)
Glasgow Coma Scale
Women
Right handedness
Hemorrhagic stroke
Ischemic stroke
Lesion type (n=88)
TACI
PACI
LACI
POCI

73.36.5
13.62.5
57 (56)
92 (90)
14 (14)
88 (86)
16 (18)
19 (22)
43 (49)
10 (11)

*LACI = lacunar infarct; PACI = partial anterior circulation infarct; POCI =


posterior circulation infarct; TACI = total anterior circulation infarct.
Values are mean SD or number (percentage).
Lesion types for ischemic stroke were classified according to the Oxfordshire Community Stroke Project.

structions, and finally performing the SV-NIHSS under


direct supervision and feedback by a certified examiner for
20 patients during a 2-month period before the beginning
of the study.
PHASE 2
The main goal of this phase was to evaluate the psychometric properties of the SV-NIHSS. The study was prospective
with a 3-month follow-up period. Consecutive patients
hospitalized at 3 clinics in Buenos Aires, Argentina (Hospital Sirio Libans, Clnica Adventista Belgrano, and
Policlnico Bancario), were enrolled from September 2003
to December 2003. Adult patients (>18 years) of both
sexes with a diagnosis of ischemic or hemorrhagic stroke
were evaluated between their third and fifth day after the
onset of stroke. We excluded patients with surgical indications, those who were transferred to another hospital
before the third to fifth day of stroke onset, and those
patients or relatives who did not give informed verbal
consent.
During the first 24 hours of inclusion, 2 trained neurologists (R.D. and J.F.V.) with at least 5 years in practice
independently administered the SV-NIHSS to the patient.
Twenty-four hours after the first administration, the scale
was administered again by 1 of the 2 initial observers
(randomly chosen). During the first 7 days of hospitalization, other patient characteristics were recorded, including
the Glasgow Coma Scale,15 sociodemographic characteristics, and stroke type and localization.
An infarction or hemorrhagic type of stroke was
identified by computed tomography, and ischemic strokes
were classified according to the Oxfordshire Community
Stroke Project classification.16 After excluding intracerebral
hemorrhage with computed tomography, ischemic strokes
were classified as total or partial anterior circulation infarct,
lacunar infarct, or posterior circulation infarct.
478

Mayo Clin Proc.

Three months after inclusion, functional capacity of the


surviving patients was evaluated with the modified Rankin
scale17 and the Barthel index.18 Follow-up of study subjects
ended in March 2004. The protocol followed the recommendations of the Declaration of Helsinki. Verbal consent
was obtained from patients or relatives.
STATISTICAL ANALYSES
Baseline population characteristics were reported using the
median or mean SD for continuous variables according to
their distribution. Categorical variables were reported in
terms of their relative frequency. The SV-NIHSS total
score was calculated by following the recommended
guidelines of the original NIHSS. Total scores of the
Barthel index and the modified Rankin scale were also
calculated according to the methods proposed by the
authors.17,18
To evaluate interrater reliability, the intraclass correlation coefficient between the scores of the 2 observers was
calculated for each patient. To evaluate intrarater reliability, the intraclass correlation coefficient was calculated
between the scores of 2 administrations of the test 24 hours
apart by the same observer.
To evaluate the construct validity of the scale, we calculated the Spearman correlation coefficient () between the
initial SV-NIHSS score and Glasgow Coma Scale. We
evaluated the association of the initial SV-NIHSS score
and total score of the Barthel index and the modified
Rankin score at 3 months with the Spearman correlation
coefficient. We obtained 95% confidence intervals (CIs)
for all the estimations.
RESULTS
A total of 102 patients with stroke were included in the
study (Table 1). The age range was 54 to 86 years (mean
SD, 73.36.5 years), and 56% were women. In 86%
of patients the stroke was ischemic and in 14% it was
hemorrhagic. The SV-NIHSS mean baseline score was
9.787.04. None of the patients received thrombolytic
therapy. The most common lesion was lacunar infarct,
followed by partial anterior circulation infarct, total anterior circulation infarct, and posterior circulation infarct.
The median duration of the SV-NIHSS administration interview was 20 minutes (mean SD, 24.48.1 minutes).
INTERRATER RELIABILITY
A total of 98 patients were independently evaluated by 2
different examiners (R.D. and J.F.V.) during the first day of
the study with a median of 3 hours between assessments.
Interrater reliability was excellent between the 2 examiners, with ranging from 0.77 to 0.99 for the 15 items on the

April 2006;81(4):476-480

www.mayoclinicproceedings.com

For personal use. Mass reproduce only with permission from Mayo Clinic Proceedings.

SPANISH-VERSION NIHSS

TABLE 2. SV-NIHSS Reliability: Interrater Agreement* (n=98)

test (Table 2). The interrater correlation coefficient was


very high (0.991; 95% CI, 0.987-0.994).
INTRARATER RELIABILITY
A total of 98 patients were evaluated at 2 different times by
the same examiner (R.D. or J.F.V.) with a median of 20
hours between assessments. Intrarater reliability was excellent between the 2 examinations, with ranging from 0.86
to 1.00 for the 15 test items (Table 3). Intrarater correlation
coefficient was also high (mean, 0.994; 95% CI, 0.9910.996).
CONSTRUCT VALIDITY
At the 3-month follow-up, 6 (6%) of the patients had died,
7 patients (7%) had a Rankin score of 0, 62 (61%) a score of
1 or 2, and 23 (23%) a score of 3 or 4 (Table 4). The mean
Barthel index was 81.916.9.
The SV-NIHSS functioned as it was hypothesized. It
had a negative correlation with the baseline Glasgow Coma
Scale (=0.574, P<.001). It also had a negative correlation with the Barthel index at 3 months (=0.658,
P<.001). As shown in Table 4, Rankin categories at 3
months also had significantly different baseline SV-NIHSS
scores, from a mean of 4.292.21 for a Rankin score of 0 to
29.403.97 for a Rankin score of 6 (nonparametric test for
trend Z=7.35; P<.001).
DISCUSSION
The current study, which used state-of-the-art cross-cultural
adaptation methods, shows that the SV-NIHSS has excellent
reliability and validity, which raises the possibility of using
the SV-NIHSS in Spanish-speaking areas of the world.
A somewhat surprising result was that the scale shows
higher than expected reliability, even higher than the original instrument.1,2 Nevertheless, reliability was similar to that
reported for the NIHSS in a setting of highly trained examiners.6 Some of the factors that may have overestimated the
reliability of the SV-NIHSS in our study compared with
reliability in a more real-life setting include the clinical
stability of our study patients, the intensive examiner training, and the detailed explanation of the instructions available
to the examiners. Other reports have shown that intensive
training positively affects the NIHSS scale reliability in its
original English-language version.3 Another difference in
our study is the length of time for the SV-NIHSS administration, which was longer than the duration usually reported in
the literature.1,2 This may have influenced the excellent reliability shown in this patient sample.
The current study can be viewed in a broader context; we
believe that valid and reliable Spanish instruments should be
available for use in this growing population. An example of a
Mayo Clin Proc.

Item
1a. LOC
1b. LOC questions
1c. LOC commands
2. Best gaze
3. Visual
4. Facial palsy
5a. Motor arm (left)
5b. Motor arm (right)
6a. Motor leg (left)
6b. Motor leg (right)
7. Limb ataxia
8. Sensory
9. Best language
10. Dysarthria
11. Extinction and
inattention
Interrater correlation
coefficient

Crude
agreement
(%)

SE

P value

97.6
96.4
99.5
99.5
97.3
94.2
98.7
99.7
97.7
98.4
99.5
94.9
99.7
97.6

0.87
0.87
0.98
0.97
0.91
0.77
0.96
0.99
0.92
0.94
0.97
0.83
0.98
0.91

0.08
0.09
0.09
0.09
0.09
0.07
0.07
0.08
0.07
0.08
0.10
0.07
0.08
0.08

<.001
<.001
<.001
<.001
<.001
<.001
<.001
<.001
<.001
<.001
<.001
<.001
<.001
<.001

96.4

0.89

0.08

<.001

0.991

<.001

*LOC = level of consciousness; SV-NIHSS = Spanish-language version


of the National Institutes of Health Stroke Scale.
To test the null hypothesis that the rate of agreement is due to chance
alone.

similar initiative in which some of the authors were involved


(P.R.C., T.G.B., and J.F.M.) was the development of a bilingual instrument for evaluation of stroke-free status.19
Although further studies should evaluate the SV-NIHSS
in other settings, the SV-NIHSS is a valid and reliable tool
in the evaluation of stroke, similar to its parent Englishlanguage version. The results of this study indicate that
TABLE 3. SV-NIHSS Reliability:
Test-Retest/Intrarater Agreement* (n=98)

Item
1a. LOC
1b. LOC questions
1c. LOC commands
2. Best gaze
3. Visual
4. Facial palsy
5a. Motor arm (left)
5b. Motor arm (right)
6a. Motor leg (left)
6b. Motor leg (left)
7. Limb ataxia
8. Sensory
9. Best language
10. Dysarthria
11. Extinction and
inattention
Intrarater correlation
coefficient

Crude
agreement
(%)

SE

P value

98.3
98.5
99.5
99.5
98.6
96.3
99.0
99.7
98.7
99.5
100
98.0
99.7
98.3

0.91
0.95
0.98
0.97
0.95
0.86
0.97
0.99
0.96
0.98
1.00
0.93
0.98
0.93

0.08
0.09
0.09
0.09
0.09
0.07
0.07
0.08
0.07
0.08
0.09
0.07
0.08
0.08

<.001
<.001
<.001
<.001
<.001
<.001
<.001
<.001
<.001
<.001
<.001
<.001
<.001
<.001

96.4

0.89

0.08

<.001

0.994

<.001

*LOC = level of consciousness; SV-NIHSS = Spanish-language version


of the National Institutes of Health Stroke Scale.
To test the null hypothesis that the rate of agreement is due to chance
alone.

April 2006;81(4):476-480

www.mayoclinicproceedings.com

For personal use. Mass reproduce only with permission from Mayo Clinic Proceedings.

479

SPANISH-VERSION NIHSS

TABLE 4. Construct Validity: SV-NIHSS Score According to


Rankin Score at 3 Months*
Rankin
score

No. of
patients

SV-NIHSS score
(mean SD)

0
1
2
3
4
5
6

7
34
28
16
7
4
6

4.292.21
6.295.64
7.752.27
12.624.29
12.572.57
21.003.83
29.403.97

*SV-NIHSS= Spanish-language version of the National Institutes of


Health Stroke Scale.

Spanish-speaking health professionals trained in SVNIHSS administration can be more confident in using this
test with Spanish-speaking patients.
CONCLUSION
Stroke is one of the most common causes of disability and
death worldwide. The NIHSS is a commonly used clinical
scale to assess degree of neurologic impairment in clinical
trials and observational studies. The scale was originally
developed and validated in English. We developed and
culturally adapted a Spanish version of the scale (SVNIHSS). The SV-NIHSS was shown to be valid and reliable in 102 stroke-affected Argentine study subjects. Instructions for using the SV-NIHSS and testing materials
are provided in the figures and Appendix 1. It is hoped that
this instrument will be of practical use to Spanish-speaking
physicians examining Spanish-speaking patients with
stroke, as well as to researchers who study the effects of
stroke on Spanish-speaking populations.
We thank the other physicians who administered the SV-NIHSS:
Eduardo Bartolom, MD, Luis Curtolo, MD, Silvia Gonzalez,
MD, Marisa Lourido, MD, and Gabriela Vigo, MD. We also
thank the Spanish-speaking physicians at Mayo Clinic who gave
valuable feedback for the creation of the SV-NIHSS: Adriana R.
Vasquez, MD, Andy Abril, MD, Candido E. Rivera, MD, Jorge F.
Trejo-Gutierrez, MD, Javier F. Aduen, MD, Juan C. Guarderas,
MD, and Salvador Alvarez, MD.

480

Mayo Clin Proc.

REFERENCES
1. Goldstein LB, Bertels C, Davis JN. Interrater reliability of the NIH
Stroke Scale. Arch Neurol. 1989;46:660-662.
2. Brott T, Adams HP Jr, Olinger CP, et al. Measurements of acute cerebral
infarction: a clinical examination scale. Stroke. 1989;20:864-870.
3. Lyden P, Brott T, Tilley B, et al, NINDS TPA Stroke Study Group.
Improved reliability of the NIH Stroke Scale using video training. Stroke.
1994;25:2220-2226.
4. DOlhaberriague L, Litvan I, Mitsias P, Mansbach HH. A reappraisal of
reliability and validity studies in stroke. Stroke. 1996;27:2331-2336.
5. Hacke W, Donnan G, Fieschi C, et al, ATLANTIS Trials Investigators,
ECASS Trials Investigators, NINDS rt-PA Study Group Investigators. Association of outcome with early stroke treatment: pooled analysis of ATLANTIS,
ECASS, and NINDS rt-PA stroke trials. Lancet. 2004;363:768-774.
6. Albanese MA, Clarke WR, Adams HP Jr, Woolson RF. Ensuring reliability of outcome measures in multicenter clinical trials of treatments for acute
ischemic stroke: the program developed for the Trial of Org 10172 in Acute
Stroke Treatment (TOAST). Stroke. 1994;25:1746-1751.
7. Bulau P, Fuger J, Horn H. Validating rehabilitation after stroke (in
German). Nervenarzt. 1994;65:836-840.
8. Wityk RJ, Pessin MS, Kaplan RF, Caplan LR. Serial assessment of acute
stroke using the NIH stroke scale (published correction appears in Stroke.
1994;25:1300). Stroke. 1994;25:362-365.
9. Tirschwell DL, Longstreth WT Jr, Becker KJ, et al. Shortening the NIH
stroke scale for use in the prehospital setting. Stroke. 2002;33:2801-2806.
10. Lyden PD, Lu M, Levine SR, Brott TG, Broderick J, NINDS rtPA Stroke
Study Group. A modified National Institutes of Health Stroke scale for use in
stroke clinical trials: preliminary reliability and validity. Stroke. 2001;32:13101317.
11. Gandek B, Ware JE Jr, IQOLA Project Group. Methods for validating
and norming translations of health status questionnaires: the IQOLA project
approach. J Clin Epidemiol. 1998;51:953-959.
12. Bullinger M, Alonso J, Apolone G, et al, IQOLA Project Group. Translating health status questionnaires and evaluating their quality: the IQOLA
project approach. J Clin Epidemiol. 1998;51:913-923.
13. Ware JE Jr, Keller SD, Gandek B, Brazier JE, Sullivan M, IQOLA
Project Group. Evaluating translations of health status questionnaires: methods
from the IQOLA project. Int J Technol Assess Health Care. 1995;11:525-551.
14. Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of
health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. 1993;46:1417-1432.
15. Teasdale G, Jennett B. Assessment of coma and impaired consciousness:
a practical scale. Lancet. 1974;2:81-84.
16. Bamford J, Sandercock P, Dennis M, Burn J, Warlow C. Classification
and natural history of clinically identifiable subtypes of cerebral infarction.
Lancet. 1991;337:1521-1526.
17. van Swieten JC, Koudstaal PJ, Visser MC, Schouten HJ, van Gijn J.
Interobserver agreement for the assessment of handicap in stroke patients.
Stroke. 1988;19:604-607.
18. Mahoney FI, Barthel DW. Functional evaluation: the Barthel index. Md
State Med J. 1965;14:61-65.
19. Castillo PR, Brott TG, Alvarez S, Meschia JF. Creation of a bilingual
Spanish-English version of the questionnaire for verifying stroke-free status.
Neuroepidemiology. 2004;23:236-239.

April 2006;81(4):476-480

www.mayoclinicproceedings.com

For personal use. Mass reproduce only with permission from Mayo Clinic Proceedings.

Potrebbero piacerti anche