Schizophrenia Research 158 (2014) 710

Contents lists available at ScienceDirect

Schizophrenia Research
journal homepage: www.elsevier.com/locate/schres

Predicting psychosis and psychiatric hospital care among adolescent

psychiatric patients with the Prodromal Questionnaire
Sebastian Therman a,, Maija Lindgren a, Marko Manninen a, Rachel L. Loewy b, Matti O. Huttunen a,
Tyrone D. Cannon c, Jaana Suvisaari a
a
b
c

National Institute for Health and Welfare, Department of Mental Health and Substance Abuse Services, P.O. Box 30, FI-00271 Helsinki, Finland
University of California, San Francisco, Department of Psychiatry, Box 0984 PAR, 401 Parnassus Ave., LangPorter, San Francisco, CA 94143, United States
Yale University, Department of Psychology, Box 208205, 2 Hillhouse Ave., New Haven, CT 06520-8205, United States

a r t i c l e

i n f o

Article history:
Received 27 February 2014
Received in revised form 18 June 2014
Accepted 24 June 2014
Available online 22 July 2014
Keywords:
Adolescent
Psychosis prodrome
Follow-up
Self-report instrument
Screening

a b s t r a c t
The Prodromal Questionnaire (PQ) identies psychiatric help-seekers in need of clinical interviews to diagnose
psychosis risk. However, some providers use the PQ alone to identify risk. Therefore, we tested its predictive
utility among 731 adolescent psychiatric help-seekers, with a 39-year register-based follow-up. Nine latent
factors corresponded well with postulated subscales. Depersonalization predicted later hospitalization with a
psychosis diagnosis (HR 1.6 per SD increase), and Role Functioning predicted any psychiatric hospitalization
(HR 1.3). Published cut-off scores were poor predictors of psychosis; endorsement rates were very high for
most symptoms. Therefore, we do not recommend using the PQ without second-stage clinical interviews.
2014 Elsevier B.V. All rights reserved.

1. Introduction
Psychotic disorders often develop gradually, and most patients have a
health care contact due to psychiatric symptoms before the onset of psychosis (Anderson et al., 2013). Initial psychiatric care should thus be an
excellent opportunity for detecting such a disease course. Though structured interviews focused on the early detection of psychoses can be
employed, they are time consuming and require special training. Consequently, several specic questionnaires have been constructed for the
screening of psychotic symptoms (e.g. Heinimaa et al., 2003; Ord et al.,
2004; Liu et al., 2013). These questionnaires can select patients for
targeted interviews. Some of the new instruments have been validated
against gold-standard interview methods for detecting incipient psychosis (Loewy et al., 2005), but research on prospective predictive value is
still scant. Despite this, some providers use the screeners without referring for clinical interview. Therefore, we sought to test the accuracy of
the Prodromal Questionnaire (PQ) in predicting psychosis on its own.
The rst visit to a non-specic psychiatric outpatient clinic is an
optimal situation for testing the utility of such inventories, balancing
base risk and volume in a situation acceptable to the patients. Accordingly,

Corresponding author. Tel.: +358 44 5405559.

E-mail addresses: Sebastian.Therman@thl. (S. Therman), Maija.Lindgren@thl.
(M. Lindgren), Marko.Manninen@thl. (M. Manninen), RachelL@lppi.ucsf.edu
(R.L. Loewy), Matti.Huttunen@konsmoh.inet. (M.O. Huttunen),
Tyrone.Cannon@yale.edu (T.D. Cannon), Jaana.Suvisaari@thl. (J. Suvisaari).

http://dx.doi.org/10.1016/j.schres.2014.06.031
0920-9964/ 2014 Elsevier B.V. All rights reserved.

we prospectively tested the predictive validity of a putative psychosis

screening questionnaire in a general adolescent psychiatric setting, with
a long-term high-coverage hospitalization register follow-up.
2. Participants and questionnaire
The Helsinki Prodromal Study (HPS) is a prospective study of
psychosis risk among adolescent psychiatric patients in Helsinki,
Finland (Lindgren et al., 2010). The questionnaire validation cohort
included all consecutive new patients aged 1518 years who presented
to any public adolescent psychiatric clinic in Helsinki during the years
2003 to 2008 (some clinics did not maintain screening during 2005
2006). The only exclusion criterion was psychiatric treatment within
the previous 24 months. At their rst or second clinic visit, the
adolescents were asked to ll in the Finnish version of the Prodromal
Questionnaire (PQ, Loewy et al., 2005, reproduced in Appendix 1), a
validated 92-item self-report measure for screening putatively prodromal symptoms, and 819 questionnaires were returned, representing
75% of eligible patients. A random subsample was later interviewed as
part of the same study and asked for informed consent; the use of
register data was refused by 61 individuals. Of the remaining patients,
27 had a previous psychosis diagnosis or received one during the
same hospital stay as they lled in the PQ; the sample size for the
analyses presented here was thus 731. The participants' average (SD)
age was 16.4 (0.9) years, 67.9% were female, and 4.0% were inpatients
at baseline.

S. Therman et al. / Schizophrenia Research 158 (2014) 710

The PQ has 92 items in a yes/no format, and extensively covers a

large spectrum of problems commonly observed in the prodromal
period, grouped into positive, negative, disorganized, and general
symptoms, with the latter including anxiety, depression, and sleep
difculties (Loewy et al, 2005).
3. Outcome measures
Lifetime psychiatric admission and discharge diagnoses (ICD-10)
from any hospital were obtained up to the end of 2011 from the Finnish
hospital discharge register (Care Register for Health Care; a.k.a. HILMO),
resulting in a 39-year (mean 5.9) follow-up period. The register includes all public and private hospitals, and has an excellent accuracy
in detecting psychosis cases (Perl et al. 2007).
Our primary outcome was hospitalization with a psychosis diagnosis, including both non-affective psychotic disorders and affective disorders with psychotic features (F20, F22F29, F30.2, F31.2, F31.5, F32.3,
and F33.3). In addition, psychiatric hospitalization in itself as an indicator of general illness severity and functioning was considered relevant
to this study; the any psychiatric hospitalization outcome was dened
as a stay at a psychiatric hospital, or any hospital stay with a primary or
secondary psychiatric diagnosis (F00F99, X60X85, or Y87.0). The 77
participants with a baseline or previous psychiatric hospitalization
were excluded from follow-ups with the latter outcome (n = 654).
As register discharge diagnoses are assigned by hospital stay, the
starting date of each stay was used for calculating time intervals from
PQ assessment.

As models of up to nine Oblimin-rotated factors were readily interpretable, this number of factors was retained (RMSEA 0.014, CFI .99). Factor
loadings, thresholds, and factor correlations are shown in Supplementary
Table 1.
KaplanMeier curves of hospitalization with a psychosis diagnosis
and any psychiatric hospitalization are depicted in Fig. 1, by gender.
After three years, which was the shortest follow-up time, 3.6% of
females and 7.2% of the males had been hospitalized for psychosis.
Cox regression results are reported in Table 1. Four predictors of hospitalization with a psychosis diagnosis were statistically signicant
individually (in order of effect size): Depersonalization, PQ Total sum
score, Role Functioning, and Dysphoria. When the strongest predictor
Depersonalization (HR 1.6, 95% CI 1.22.2, p = 0.005) was accounted
for, none of the other variables offered signicant improvements to
the model (p = .94, .14 and .35, respectively). Five predictors of any
psychiatric hospitalization were statistically signicant individually:

a) Psychosis

4. Analyses
No PQ item had more than 0.6% missing responses, and the overall
missingness rate was 0.2%. The means (SDs) for the PQ Positive subscale
sum score and PQ Total sum score were 10.6 (7.8) and 31.3 (18.7),
respectively.
A conrmatory factor analysis of the PQ indicated that the a
priori four-dimensional model showed only a moderately good t
(RMSEA = 0.04, CFI b .90). Therefore, to determine the empiric factorial structure of the PQ, exploratory factor analysis of the 731 response
sets was conducted with the WLSMV algorithm, Oblimin rotation, and
default parameters in Mplus version 7.11 (Muthn and Muthn,
2012). In this model, a response threshold parameter is calculated for
each item, and one factor loading parameter for each item-factor combination is estimated from the tetrachoric item correlations. The model
was computed with an increasing number of dimensions, until there
was no improvement in RMSEA.
Symptom factor scores and the a priori PQ Total and Positive Symptom subscale sum scores were the hypothesized predictors in Cox proportional hazards models of a) any psychiatric hospitalization (n =
120) and b) hospitalization with a psychosis diagnosis (n = 41) during
the individual's full follow-up time. Predictors were rst used singly,
and those that were signicant at the p = .01 level individually were included in a forward-stepping Cox model with the same p = .01 criterion
for entry. For comparability of coefcient estimates, all factor scores and
PQ sum scores were normalized before survival analyses. Age was used
as a covariate in all analyses. Due to a larger baseline psychosis and hospitalization risk among boys in our sample, all survival analyses were
conducted with gender as a stratum.
In addition, for facilitating comparison with previously published
results, we assessed the one-year predictive values for psychosis of
the previously proposed (Loewy et al, 2007) cut-offs for the Total and
Positive Symptom subscale sum scores.

b) Psychiatric hospitalization

5. Results
The multidimensional WLSMV latent factor models of the PQ provided
improving t using RMSEA, CFI, WRMR indices with up to 10 dimensions.

Fig. 1. Survival curves for psychosis and psychiatric hospitalization outcomes in register
follow-up, by gender. a) Psychosis and b) psychiatric hospitalization.

S. Therman et al. / Schizophrenia Research 158 (2014) 710

Table 1
Cox proportional hazard models predicting psychosis and hospitalization with individual
PQ factors and sum scores.
Normalized PQ factor

Hazard ratio (HR)a

95% CI for HR
Lower

Upper

1.45
1.28
1.49
1.23
0.97
1.26
1.24
1.04
1.41
1.60
1.37

1.04
0.93
1.08
0.89
0.70
0.92
0.91
0.75
1.00
1.16
0.99

2.02
1.75
2.07
1.70
1.34
1.73
1.68
1.43
1.99
2.23
1.89

0.027
0.130
0.017
0.206
0.864
0.156
0.174
0.821
0.047
0.005
0.056

1.28
1.20
1.32
1.16
1.18
1.08
1.24
1.17
1.18
1.23
1.17

1.07
1.01
1.11
0.97
0.99
0.91
1.05
1.00
0.98
1.03
0.98

1.53
1.43
1.58
1.37
1.39
1.29
1.47
1.38
1.43
1.47
1.39

0.007
0.035
0.002
0.097
0.059
0.355
0.013
0.057
0.080
0.024
0.085

a) Psychosis, n = 731.
PQ Total Score
PQ Positive Symptoms
F1 Role Functioning
F2 Delusional Ideation
F3 Hallucinations
F4 Odd
F5 Social Avoidance
F6 Magical Thinking
F7 Dysphoria
F8 Depersonalization
F9 Anhedonia
b) Hospitalization, n = 654
PQ Total Score
PQ Positive Symptoms
F1 Role Functioning
F2 Delusional Ideation
F3 Hallucinations
F4 Odd
F5 Social Avoidance
F6 Magical Thinking
F7 Dysphoria
F8 Depersonalization
F9 Anhedonia

a
HR is per standard deviation increase of the factor. Predictors with p b0.05 are marked
in bold.

Role Functioning, PQ Total sum score, Positive Subscale sum score, Social Avoidance, and Depersonalization. When Role Functioning (HR
1.3, 95% CI 1.11.6, p = 0.002) was accounted for, none of the other
variables improved the model (p = .38, .30, .18, and .22, respectively).
At the one-year follow-up, the Positive Symptom subscale score
criterion of 14 or more provided a sensitivity of 48% and a specicity of
68% for predicting psychosis, with a 5% PPV and 97% NPV. The Total
Score criterion of 36 or more had a 64% sensitivity and a 57% specicity,
with a 5% PPV and 98% NPV. Adjusting cutoffs or including only the
items of the shortened PQ-B (Loewy et al., 2011) did not signicantly improve results.
6. Discussion
In the exploratory factor analysis, we found interpretable latent
factors, demonstrating structural validity of the PQ. Of the measured
constructs, Depersonalization, Role Functioning, and Dysphoria predicted later hospitalization for psychosis, though none of the others improved prediction after Depersonalization was taken into account. That
Depersonalization was the strongest predictor in questionnaire-based
psychosis screening is to our knowledge a novel nding, which has
been anticipated in theoretical work on the hallucination-predisposing
effects of loss of atmospheric cues (Graux et al., 2012). Nevertheless,
the Hazard Ratio of 1.6 per SD increase in Depersonalization is insufcient for effective screening, at least when used without a second
stage interview.
Also the predictive value of previously suggested sum score cut-offs
intended for two-stage screening was poor; the rate of false positives to
true positives was approximately twenty-fold. In sum, our results indicate that the instrument should only be used as originally intended,
i.e. for selecting respondents to secondary screening with a structured
interview.
The version of the instrument used in this study was unfortunately
limited to a Yes/No response format, where response bias and item
comprehension play a large part. The revisions coding distress and frequency (Loewy et al., 2007) may thus be preferable. Notably, a version

of the PRIME Screen (Kobayashi et al., 2008), which used a sevenpoint agreement scale and symptom duration, achieved fairly good predictive power in a similar study with a 6-month follow-up.
Our results do not support the use of the PQ for strict screening
of psychosis risk, though it provides an easy way to discuss and uncover psychotic-like experiences in a young patient; these experiences
deserve attention in their own right, especially when associated with
distress (Yung et al., 2006). Our ndings show that psychotic-like experiences also predict a general functional outcome as assessed by
psychiatric hospitalization; however, the added value was small after
general Role Functioning was accounted for.
Supplementary data to this article can be found online at http://dx.
doi.org/10.1016/j.schres.2014.06.031.
Role of funding sources
Funding for this study was provided by the Academy of Finland Research Programme
Strategy Grant 115479 and a personal grant to S.T. by the Finnish Graduate School in
Psychiatry. Neither of these institutions had any further role in the study design, in the
collection, analysis, or interpretation of data, in the writing of the report, or in the decision
to submit the paper for publication.
Contributors
T.D.C. designed the study and wrote the protocol. All authors collaborated in designing
the data collection or analyses. S.T. undertook the statistical analyses and wrote the rst
manuscript draft. All authors contributed to and have approved the nal manuscript.
Conict of interest
All authors declare that they have no conicts of interest.
Acknowledgment
We thank all of our respondents and our collaborators at the Hospital District of
Helsinki and Uusimaa for making this study possible.

Appendix 1. Prodromal Questionnaire (Loewy et al., 2005)

The English language version of the 92-item version of the
Prodromal Questionnaire is reproduced here, with permission by
the rst author. Note that the study used the Finnish-language
version. The response alternatives are Yes and No.
1. I am easily distracted by noises or other people talking.
2. The passage of time feels unnaturally faster or slower than usual.
3. I often have difculty organizing my thoughts or nding the right
words.
4. When I look at a person, or look at myself in a mirror, I have seen the
face change right before my eyes.
5. I sometimes get strange feelings on or just beneath my skin, like
bugs crawling.
6. I do not get along well with people at school or at work.
7. Familiar surroundings sometimes seem strange, confusing,
threatening or unreal.
8. I often seem to live through events exactly as they happened before
(dj vu).
9. I sometimes smell or taste things that other people can't smell or
taste.
10. I have difculty concentrating, listening or reading.
11. I have had troubles at school or work recently.
12. Sometimes I think that people can read my mind.
13. I have heard things other people can't hear like voices of people
whispering or talking.
14. I can't express my feelings as well as I used to.
15. I have interests that other people nd odd.
16. I have lost a sense of who I am.
17. I am less interested than I used to be in keeping clean or dressing
well.
18. I often hear unusual sounds like banging, clicking, hissing, clapping,
or ringing in my ears.
19. I often mistake shadows for people or noises for voices.

10

S. Therman et al. / Schizophrenia Research 158 (2014) 710

20. Things that I see appear different from the way they usually do
(brighter, duller, larger, smaller, or changed in some other way).
21. I tend to be very quiet and keep in the background on social
occasions.
22. People sometimes stare at me because of my odd appearance.
23. I wander off the topic or ramble on too much when I am speaking.
24. I believe in telepathy, psychic forces, or fortune-telling.
25. I often feel that others have it in for me.
26. My sense of smell sometimes becomes unusually strong.
27. Sometimes I have felt that I'm not in control of my own ideas or
thoughts.
28. I have been feeling unhappy or depressed lately.
29. Everyday things affect me more than they used to.
30. I believe that I am especially important or have abilities that are out
of the ordinary.
31. Other people think that I am a little strange.
32. Sometimes my thoughts seem to be broadcast out loud so that other
people know what I am thinking.
33. I often feel that I have nothing to say or very little to say.
34. I am unusually sensitive to noise.
35. I am superstitious.
36. I have heard my own thoughts as if they were outside of my head.
37. I have trouble focusing on one thought at a time.
38. I often feel that other people are watching me or talking about
me.
39. I get very nervous when I have to make polite conversation.
40. People comment on my unusual mannerisms and habits.
41. I am less interested in school or work lately.
42. I nd it hard to be emotionally close to other people.
43. I tend to avoid social activities with other people.
44. I feel very guilty.
45. I am an odd, unusual person.
46. I sometimes feel that things I see on television or read in the newspaper have a special meaning for me.
47. My moods are highly changeable and unstable.
48. I have been unable to enjoy things that I used to enjoy.
49. My thinking feels confused, muddled, or disturbed in some way.
50. Sometimes I feel suddenly distracted by distant sounds that I am not
normally aware of.
51. Recently, I have begun talking to myself.
52. I have had the sense that some person or force is around me, even
though I could not see anyone.
53. I am in danger of failing out of school, or have been red from my job.
54. I have some eccentric (odd) habits.
55. At times I worry that something may be wrong with my mind.
56. I have felt that I don't exist, the world does not exist, or that I am dead.
57. I have been confused at times whether something I experienced
was real or imaginary.
58. People nd me aloof and distant.
59. I tend to keep my feelings to myself.
60. I have experienced unusual bodily sensations (tingling, pulling,
pressure, aches, burning, cold, numbness, shooting pains, vibrations
or electricity).
61. I hold beliefs that other people would nd unusual or bizarre.
62. People say that my ideas are strange or illogical.
63. I feel worthless.
64. I feel that parts of my body have changed in some way, or that parts
of my body are working differently than before.
65. My thoughts are sometimes so strong that I can almost hear them.
66. I am not very good at returning social courtesies and gestures.
67. I sometimes see special meanings in advertisements, shop
windows, or in the way things are arranged around me.
68. I often pick up hidden threats or put-downs directed at me in what
people say or do.
69. I sometimes use words in unusual ways.
70. I am often angry, easily irritated or offended.

71. I have felt like I am looking at myself as in a movie, or that I am a

spectator in my own life.
72. I am less able to do usual activities or tasks.
73. I have not been sleeping well lately.
74. At times I have felt that some person or force interferes with my
thinking or puts thoughts into my head.
75. I have had experiences with the supernatural, astrology, seeing the
future or UFOs.
76. Some people drop hints about me or say things with a double
meaning.
77. I am often concerned that my closest friends, classmates, or coworkers are not really loyal or trustworthy.
78. I have little interest in getting to know other people.
79. I have seen unusual things like ashes, ames, blinding light, or
geometric gures.
80. I get extremely anxious when meeting people for the rst time.
81. I have felt like I am at a distance from myself, as if I am outside my
own body or that a part of my body did not belong to me.
82. I nd that when something sad happens, I am no longer able to feel
sadness, or when something joyful happens, I can no longer feel
happy.
83. I cry often.
84. I have seen things that other people apparently can't see.
85. I feel unable to carry out everyday tasks because of fatigue or lack of
motivation.
86. Everyday things are more stressful than before, like school or work,
social situations, deadlines or changes in a schedule.
87. I often avoid going to places where there will be many people
because I will get anxious.
88. I have felt more nervous or anxious lately, and nd it hard to relax.
89. I feel uninterested in the things I used to enjoy.
90. People often nd it hard to understand what I am saying.
91. I have trouble remembering things.
92. People say that I seem spacey or out of it.
References
Anderson, K.K., Fuhrer, R., Wynant, W., Abrahamowicz, M., Buckeridge, D.L., Malla, A.,
2013. Patterns of health services use prior to a rst diagnosis of psychosis: the importance of primary care. Soc. Psychiatry Psychiatr. Epidemiol. 48 (9), 13891398.
Graux, J., Lemoine, M., El Hage, W., Camus, V., 2012. From depersonalization to hallucination. Psychopathology 45 (1), 4252.
Heinimaa, M., Salokangas, R.K.R., Ristkari, T., Plathin, M., Huttunen, J., Ilonen, T., Suomela,
T., Korkeila, J., McGlashan, T.H., 2003. PROD-screena screen for prodromal symptoms of psychosis. Int. J. Methods Psychiatr. Res. 12 (2), 92104.
Kobayashi, H., Nemoto, T., Koshikawa, H., Osono, Y., Yamazawa, R., Murakami, M.,
Kashima, H., Mizuno, M., 2008. A self-reported instrument for prodromal symptoms
of psychosis: testing the clinical validity of the PRIME screen-revised (PS-R) in a
Japanese population. Schizophr. Res. 106 (23), 356362.
Lindgren, M., Manninen, M., Laajasalo, T., Mustonen, U., Kalska, H., Suvisaari, J., Moilanen,
K., Cannon, T.D., Huttunen, M., Therman, S., 2010. The relationship between
psychotic-like symptoms and neurocognitive performance in a general adolescent
psychiatric sample. Schizophr. Res. 123 (1), 7785.
Liu, C.C., Tien, Y.J., Chen, C.H., Chiu, Y.N., Chien, Y.L., Hsieh, M.H., Liu, C.M., Hwang, T.J.,
Hwu, H.G., 2013. Development of a brief self-report questionnaire for screening putative pre-psychotic states. Schizophr. Res. 143 (1), 3237.
Loewy, R.L., Bearden, C.E., Johnson, J.K., Raine, A., Cannon, T.D., 2005. The Prodromal Questionnaire (PQ): preliminary validation of a self-report screening measure for prodromal and psychotic syndromes. Schizophr. Res. 79 (1), 117125.
Loewy, R.L., Johnson, J.K., Cannon, T.D., 2007. Self-report of attenuated psychotic experiences in a college population. Schizophr. Res. 93 (13), 144151.
Loewy, R.L., Pearson, R., Vinogradov, S., Bearden, C.E., Cannon, T.D., 2011. Psychosis risk
screening with the Prodromal Questionnairebrief version (PQ-B). Schizophr. Res.
129 (1), 4246.
Muthn, L.K., Muthn, B.O., 2012. Mplus User's Guide, 7th ed. Muthn & Muthn, Los
Angeles.
Ord, L.M., Myles-Worsley, M., Blailes, F., Ngiralmau, H., 2004. Screening for prodromal adolescents in an isolated high-risk population. Schizophr. Res. 71 (23), 507508.
Perl, J., Suvisaari, J., Saarni, S.I., Kuoppasalmi, K., Isomets, E., Pirkola, S., Partonen, T.,
Tuulio-Henriksson, A., Hintikka, J., Kiesepp, T., Hrknen, T., Koskinen, S.,
Lnnqvist, J., 2007. Lifetime prevalence of psychotic and bipolar I disorders in a general population. Arch. Gen. Psychiatry. 64 (1), 1928.
Yung, A.R., Buckby, J.A., Cotton, S.M., Cosgrave, E.M., Killackey, E.J., Stanford, C., Godfrey, K.,
McGorry, P.D., 2006. Psychotic-like experiences in nonpsychotic help-seekers: associations with distress, depression, and disability. Schizophr. Bull. 32 (2), 352359.