Psychiatry Research 216 (2014) 146154

Contents lists available at ScienceDirect

Psychiatry Research
journal homepage: www.elsevier.com/locate/psychres

Bifactor structural model of symptom checklists: SCL-90-R and Brief

Symptom Inventory (BSI) in a non-clinical community sample
Rbert Urbn a, Bernadette Kun a, Judit Farkas a,b, Borbla Paksi c, Gyngyi Kknyei a,
Zsolt Unoka d, Katalin Felvinczi a, Attila Olh a, Zsolt Demetrovics a,n
a
Institute of Psychology, Etvs Lornd University, Izabella utca 46, Budapest 1064, Hungary
b
Doctoral School of Psychology, Etvs Lornd University, Izabella utca 46, Budapest 1064, Hungary
c
Centre for Behavioural Research, Corvinus University of Budapest, Fvm tr 8., Budapest 1093, Hungary
d
Department of Psychiatry and Psychotherapy, Semmelweis University, Balassa J. utca 6, Budapest 1083, Hungary

art ic l e i nf o a b s t r a c t

Article history: The Derogatis symptom checklist (SCL-90-R) and its short version, the Brief Symptom Inventory (BSI), are
Received 29 July 2013 widely used instruments, despite the fact that their factor structures were not clearly conrmed.
Received in revised form The goals of this research were to compare four measurement models of these instruments including
11 January 2014
one-factor, nine-factor, a second-ordered factor model and a bifactor model, in addition to testing the
Accepted 16 January 2014
Available online 27 January 2014
gender difference in symptom factors in a community sample. SCL-90-R was assessed in a large
community survey which included 2710 adults who represent the population of Hungary. Statistical
Keywords: analyses included a series of conrmatory factor analyses and multiple indicator multiple cause (MIMIC
Symptom checklist (SCL-90-R) modeling). The responses to items were treated as ordinal scales. The analysis revealed that the bifactor
Brief Symptom Inventory (BSI)
model yielded the closest t in both the full SCL-90-R and BSI; however the nine-factor model also had
Conrmatory factor analysis
an acceptable level of t. As for the gender differences, women scored higher on global severity,
Bifactor model
Gender differences somatization, obsession-compulsion, interpersonal sensitivity, depression and anxiety factors. Men
scored higher on hostility and psychoticism. The bifactor model of symptom checklist supports the
concept of global symptom severity and specic symptom factors. Global symptom severity explains the
large correlations between symptom factors.
& 2014 Elsevier Ireland Ltd. All rights reserved.

1. Introduction concluded that the structural stability of SCL-90-R is weak across

The SCL-90-R and its short version, the Brief Symptom Inven-
tory (BSI), are widely used instruments to measure self-reported
psychological distress and psychopathological symptoms in sam-
ples of psychiatric patients and community non-patients
(Derogatis and Savitz, 2000). These instruments encompass three
global indices and nine subscales covering clinically relevant
psychiatric and psychosomatic symptoms (Derogatis, 1983).
Extensive research effort has been devoted to investigating the
factor structure of both full and briefer versions of symptom
checklist in different countries and languages; however, there is
still no agreement regarding the factor structure of SCL-90-R and
BSI. Exploratory factor analytical studies have reported various
solutions from one to nine factors (Hoffmann and Overall, 1978;
Brophy et al., 1988; Hafkenscheid, 1993; Holi et al., 1998; Schmitz
et al., 2000; Prunas et al., 2012). In their review, Cyr et al. (1985)
n
Correspondence to: Department of Clinical Psychology and Addiction, Institute
of Psychology, Etvs Lornd University, Izabella utca 46, Budapest 1064, Hungary.
Tel.: 36 30 97 610 97; fax: 36 1 461 2697.
E-mail address: demetrovics@t-online.hu (Z. Demetrovics).
different diagnostic groups, social groups and genders. Further-
more, the fact that cross-loadings of several items on several
factors are frequently reported, and that the rst unrotated factor
usually explains disproportionally higher variance than the fol-
lowing factors, implies that the symptom dimensions might
measure a general distress or discomfort factor and question the
multidimensional nature of any version of the instrument.
In the conrmatory factor analysis framework the most fre-
quently tested measurement model specied nine correlating
rst-order factors; however, the vast majority of psychometric
studies did not support this model and reported its failure to meet
the conventional criteria for goodness-of-t (Hoffmann and
Overall, 1978; Hafkenscheid, 1993). Yet, only a few studies com-
pared several measurement models of SCL-90-R (Carpenter and
Hittner, 1995; Vassend and Skrondal, 1999; Schmitz et al., 2000;
Hafkenscheid et al., 2006) and concluded that none of the
proposed models t satisfactorily to the data.
There might be several reasons behind the failure to nd an
adequate level of t in research on SCL-90-R. Among them, we
highlight two possible statistical reasons. The rst is that the vast
majority of previous studies applied the maximum likelihood (ML)

0165-1781/$ - see front matter & 2014 Elsevier Ireland Ltd. All rights reserved.
http://dx.doi.org/10.1016/j.psychres.2014.01.027
 R. Urbn et al. / Psychiatry Research 216 (2014) 146154
estimation method which is prone to serious deviation from
multivariate normal distribution. value is inated under the
condition of moderate non-normality with values becoming more
and more inated as non-normality increases, resulting in an
enlarged level of Type I error (Finney and DiStefano, 2006). The
second possible reason is the treatment of responses as a linear
scale instead of an ordinal scale. This is a frequent practice and is
something of a tradition in psychological and psychiatric research;
however, in cases of severe oor and ceiling effect it might be
misleading when estimating the model t. However, it is well
documented that participants or clients are sometimes confused in
using the rating scales (Low, 1988). Using Rasch analysis there is
evidence that although the SCL-90-R response categories advance
monotonically from zero (not at all) to four (extremely), the
patients did not effectively discriminate between two (moder-
ately) and three (quite a bit) in their answers (Elliott et al.,
2006). In case of non-normally ordered categorical data, maximum
likelihood-based values are inated and values of comparative
t index (CFI) are underestimated (Finney and DiStefano, 2006). If
the selection of estimation method ignores the nature of rating
scales then a correctly specied model might not t the data well.
The structure and structural stability of SCL-90-R deserves
further investigation, thus alternative solutions and approaches
were proposed based on exploratory or theory-based approaches.
Recently, item response theory was also applied to investigate the
measurement properties of SCL-90-R (Olsen et al., 2004; Elliott
et al., 2006; Paap et al., 2011). Olsen et al. (2004) presented
evidence that the items belonging to six subscales formed a strong
unidimensional scale in a Danish community sample. Elliott et al.
(2006) identied one big factor measuring overall clinical distress,
with two small residual subscales measuring depressive motiva-
tional decit and social distress. In their recent analysis, Paap et al.
(2011) applied a theory-driven item response theory approach and
identied 60 items that were clustered in seven scales supporting
the multidimensionality of SCL-90-R.
There is increasing agreement that psychiatric symptoms and
disorders maintain hierarchical structure where general or com-
mon and domain-specic or unique components play important
roles (Watson, 2005; Thomas, 2012). However the size of these
general and specic components might differ markedly across
disorders (Mineka et al., 1998; Watson, 2005). Estimating the
bifactor measurement structure is proposed to be an effective
approach to modeling construct-relevant multidimensionality
(Reise, 2012). Similar to the second order factor model, the bifactor
model enables the estimation of specic and general factors
simultaneously (Gibbons et al., 2007). This measurement model
allows for the indicators of psychological symptoms to load on an
overall primary factor such as global severity and also to have a
secondary loading on a specic dimension of symptoms.
Recently, bifactor model has been applied in several research
investigations on psychopathology including, for example, the
modeling of the structure of the Psychopathy Checklist-Revised
(Flores-Mendoza et al., 2008), the estimation of the bifactor
structure of Quality of Life Interview for the Chronically Mentally
Ill (Gibbons et al., 2007), the validation of the Inventory of Callous-
Unemotional Traits among young offenders (Kimonis et al., 2008),
the estimation of the measurement model of Beck Depression
Inventory (Al-Turkait and Ohaeri, 2010), and the estimation of the
bifactor model of ADHD (Martel et al., 2011). Only a few studies
estimated the bifactor structure of the SCL-90-R (Vassend and
Skrondal, 1999) or its abbreviated version, the BSI (Thomas, 2012).
Vassend and Skrondal (1999) found that the bifactor model
and the second-order factor model only marginally increased the
model t compared with the one-dimensional model; however,
none of these models yielded an adequate level of model
t. Thomas (2012) also contrasted the bifactor model against the
147
one-dimensional model and an oblique simple structure model.
Direct comparisons suggested that the bifactor model slightly
outperformed the simple structure model in accounting for the
internal structure of BSI, which is the brief version of SCL-90-R.
1.1. Gender difference in symptomatology
Women tend to report more somatic and psychological symp-
toms than men (Barsky et al., 2001; A, 2007). The gender
difference in symptomatology is reported in adolescent samples
as well. Girls reported signicantly higher levels of emotional
distress, in particular depressed mood and anxiety, than did boys
(Ostrov et al., 1989; Casper et al., 1996). Apart from adolescents,
gender difference in psychiatric symptoms and disorders is fre-
quently reported in adults as well. In primary care patients, mood,
anxiety, and somatoform disorders and psychiatric comorbidity
were signicantly more common in women than men (Linzer
et al., 1996). Large epidemiological surveys using diagnostic inter-
view methods presented evidences that women have a higher
prevalence of affective disorders, anxiety disorders and non-
affective psychosis than men, while men have higher rates of
substance use disorders and antisocial personality disorders.
Moreover women are more likely to report psychiatric comorbid-
ities of three or more disorders than men (Kessler et al., 1994;
Vicente et al., 2006). The gender difference in symptoms should be
similar in the cases of SCL-90-R and BSI; however, it also depends
on the factorial structure of the symptom checklist. For example,
in the case of a bifactor model it becomes possible to identify
gender differences in global severity. Another hypothesis could
also state that gender difference exists only in specic symptoms.
However, in order to test gender differences in psychiatric symp-
toms, gender invariance of measurement model should be sup-
ported. Meaningful comparisons of means and regression
coefcients can only be made if the measures are comparable
across different groups such as gender (Chen et al., 2005). There-
fore the examination of gender differences should be preceded by
the testing of gender invariance in the measurement models.
1.2. The goals of the present study
The primary aim of our study was to compare four measure-
ment models of SCL-90-R and BSI in a large nationally represen-
tative Hungarian community sample. The secondary aim of the
study was to test the gender difference in primary psychiatric
symptoms and global distress.
2. Methods
2.1. Participants and procedure
SCL-90-R was assessed within the framework of the National Survey on
Addiction Problems in Hungary (NSAPH) (Paksi et al., 2009). In this survey, in
addition to the assessment of addictive behaviors, also aimed to assess other
symptoms. The research protocol was approved by the Parliamentary Commis-
sioner for Data Protection and Freedom of Information and the University's
Institutional Review Board. Data were handled anonymously and on a
voluntary basis.
The target population of the survey was the total population of Hungary
between the ages of 18 and 64 (6,703,854 persons). The sampling frame consisted
of the whole resident population with a valid address, according to the register of
the Central Ofce for Administrative and Electronic Public Services on January 1,
2006 (6,662,587 persons). Data collection was executed on a gross sample of 3183
people, stratied according to geographical location, degree of urbanization and
age (overall 186 strata) representative of the sampling frame. Participants were
surveyed using the so-called mixed-method via personal visits. Questions regard-
ing background variables and introductory questions referring to specic disorders
were asked in the course of face-to-face interviews, while symptom scales,
including SCL-90-R, were applied using self-administered paper-and-pencil
148 R. Urbn et al. / Psychiatry Research 216 (2014) 146154
questionnaires. These questionnaires were returned to the interviewer in a closed
envelope to ensure condentiality. The net sample size was 2710 (response rate:
85.1%). The ratio of samples belonging to each stratum was adjusted to the
characteristics of the sampling frame by means of a weighted matrix for each
stratum category. The weights applied have normal distribution (S.D.: 0.228;
skewness: 0.639; std. error of skewness: 0.047; kurtosis: 2.397; std. error of
kurtosis: 0.094).
2.2. Instrument
2.2.1. Symptom checklist 90-Revised (SCL-90-R)
The SCL-90-R is a 90-item self-report symptom inventory designed to reect
psychological symptom patterns of psychiatric and medical patients. Each item of
the questionnaire is rated on a 5-point scale of distress from 0 (not at all) to 4 (very
much). The SCL-90-R consists of the following nine primary symptom dimensions:
somatization (SOM, which reects distress arising from bodily perceptions),
obsessive-compulsive (O-C, which reects obsessive-compulsive symptoms), inter-
personal sensitivity (I-S, which reects feelings of personal inadequacy and
inferiority in comparison with others), depression (DEP, which reects depressive
symptoms, as well as lack of motivation), anxiety (ANX, which reects anxiety
symptoms and tension), hostility (HOS, which reects symptoms of negative affect,
aggression and irritability), phobic anxiety (PHOB, which reects symptoms of
persistent fears as responses to specic conditions), paranoid ideation (PAR, which
reects symptoms of projective thinking, hostility, suspiciousness, fear of loss of
autonomy), and psychoticism (PS, which reects a broad range of symptoms from
mild interpersonal alienation to dramatic evidence of psychosis) (Derogatis, 1983;
Derogatis and Savitz, 2000).
2.2.2. Brief Symptom Inventory (BSI-53)
The BSI is the short version of the SCL-R-90 and contains only 53 items
(Derogatis, 1975; Derogatis and Spencer, 1982). BSI-53 measures the same dimen-
sions and the items for each dimension of the BSI-53 were selected based on a
factor analysis of the SCL-R-90.
2.3. Data analysis strategy
We used conrmatory factor analysis with Mplus 6.0 (Muthn and Muthn,
19982007) to estimate the degree of t of previously proposed measurement
models to the data. We tested four measurement models of both SCL-90-R and BSI.
The starting point was a one-dimensional model which is called Model 0. The next
model (Model 1) includes nine rst-order factors. In the third model (Model 2) in
addition to the nine rst-order factors we specied one second-order factor
representing the global severity dimension and the covariances between factors.
The fourth model (Model 3) includes a bifactor model representing a global
severity dimension on which each item is loaded; nine problem specic factors
on which only the problem specic items are loaded; and the correlations between
specic factors which were xed to zero. The usual specication of a bifactor model
requires that the specic factors do not correlate with either each other or the
global factor (Reise et al., 2010), despite the fact that covariance between specic
factors is identied. Therefore we also estimated the bifactor model with correlat-
ing specic factors (Model 4), but all correlations between each specic factor and
the global factor were xed to zero.
Both SCL-90-R and BSI contain items that do not belong to any factors, but they
are included in the global scale because of the items' clinical relevance. Performing
conrmatory factor analyses with these items would be misleading because they
are not be associated theoretically with any factors. For this reason we excluded
these items when we compared different measurement models.
Due to severe oor effect in the responses, we treated the items as ordinal
indicators and used the weighted least squares mean and variance adjusted
estimation method (WLSMV; Brown, 2006; Finney and DiStefano, 2006). We used
the full information maximum likelihood estimator to deal with missing data
(Muthn and Muthn, 19982007). In conrmatory factor analysis, a satisfactory
degree of t requires the CFI and the TuckerLewis index (TLI) to be higher than or
close to 0.95, and the model should be rejected when these indices are less than
0.90 (Brown, 2006). The next t index was the root mean square error of
approximation (RMSEA). RMSEA below 0.05 indicates excellent t, a value around
0.08 indicates adequate t, and a value above 0.10 indicates poor t. Closeness of
model t using RMSEA (CFit of RMSEA) is a statistical test (Brown, 2006), which
evaluates the statistical deviation of RMSEA from the value.05. Non-signicant
probability values (p 40.05) indicate acceptable model t, though some methodol-
ogists would require larger values such as p 40.50 (Brown, 2006).
In order to compare the degree of t of the four models, we have performed the
difftest procedure, which is available in the case of WLSMV estimation in Mplus
program (Muthn and Muthn, 19982007). Difftest procedure is used to obtain a
correct difference test when WLSMV estimator is used because the difference in
values for two nested models using WLSMV values is not distributed as
(Muthn and Muthn, 19982007, p. 501).
In order to quantify the degree of unidimensionality in bifactor models, we
applied the percent of common variance attributable to the general factor through
the use of explained common variance index (ECV; Ten Berge and Soan, 2004;
Bentler, 2009). We also used omega and omega hierarchical indices to measure
how precisely a self-reported symptom scale score assesses the combination of
general and specic constructs and a certain target construct (Brunner et al., 2012).
Before testing the gender difference, we needed to examine the gender
invariance of factor structure. Thus we performed a series of analyses to examine
the gender differences in parameters of measurement models including factor
loadings and thresholds. In case of WLSMV estimation, comparing the of models
with different level of constrains, we should rely on DIFFTEST procedure provided
by MPLUS (Asparouhov and Muthn, 2006). However, this procedure is sensitive to
large sample size; therefore additional goodness-of-t indexes are often used to
assess model t. Based on simulation studies (Cheung and Rensvold, 2002; Chen,
2007) a difference of larger than 0.01 in the CFI would indicate a meaningful
change in model t for testing measurement invariance. Thus we also examined
the change in goodness-of-t index such as CFI. In order to determine the gender
differences in factor loadings on the global distress factor, we compared models
with different level of constrains. In the rst step, we estimated the measurement
model separately for men and women. In the second step, we estimated a model in
both gender together with freely estimated threshold and factor loadings; therefore
we assumed only the congural invariance. In the third step, we constrained the
factor loadings to be equal in both genders; this is the assumption for metric
invariance. In the fourth step we constrained the factor loadings and thresholds
being equal in both genders; this is the assumption for scalar invariance. In the nal
step, we constrained the correlations between specic factors to be gender
invariant.
In order to examine the gender difference in reporting psychiatric symptoms in
the bifactor model, we applied multiple indicators and multiple causes (MIMIC)
modeling in which gender is modeled as a predictor variable, and the psychiatric
symptoms and the general distress as latent variables. The MIMIC technique, a
specication of structural equation modeling, was chosen for the present study
because MIMIC models can estimate the effect of indicators on latent variables
when direct effects of grouping variables or other continuous variables on the
latent variables are also included.
3. Results
3.1. Measurement models for SCL-90-R
We have tested ve measurement models. The t indices are
reported in Table 1. The test is signicant in all four models;
however this test is oversensitive to large sample size. In the case
of WLSMV estimation the traditional difference test is not
applicable; thus we performed difftest procedure (Muthn and
Muthn, 19982007) provided by Mplus for similar cases. The
bifactor model (Model 3) yielded signicantly closer t to the data
than the nine rst-ordered factors model (Model 1; 160.5, d.
f. 47, p o0.0001). The bifactor model (Model 3) also produced
closer t than the second-order factor model (Model 2;
1697.5, d.f. 74, p o0.0001). The bifactor model had the
closer t compared to the competing models (one-factor, nine-
factor, and second-ordered factor models). We also tested the
bifactor model with correlating specic factors (Model 4) which
yielded a signicantly better t to the data than Model 3
( 2063, d.f. 36, p o0.0001). Factor loadings are presented
in Table 2. In the bifactor model, all items loaded signicantly on
global distress factors. The factor loadings ranged from 0.38 to
0.82. With the exception of seven items (11, 21, 24, 45, 65, 78, 79),
all items loaded signicantly on their specic factors. These seven
items seemed to represent only global distress rather than the
specic psychiatric symptoms. We estimated the common var-
iance index in the bifactor model and found that regarding SCL-
90-R the Global severity factor explains 84 % of common variance
which supports the presence of a strong global factor. The
explained common variances of specic factors are also reported
in Table 2. Only Somatization scale has relatively large proportion
of variance (5.5%). We also calculated omega and omega hierarch-
ical indices to denote how precisely a self-reported symptom scale
score assesses the combination of general and specic constructs,
and a certain target construct. To evaluate the measurement
 R. Urbn et al. / Psychiatry Research 216 (2014) 146154 149

Table 1
Conrmatory factor analysis of ve measurement models of SCL-90-R.

d.f. CFI TLI RMSEA CFit of

RMSEA

Symptom checklist-90-Revised
Model 0 One-factor model 21,882 3915 0.90 0.90 0.043 1.00
Model 1 Nine rst-ordered factors 15,005 3284 0.93 0.93 0.038 1.00
Model 2 Second-order factor model 16,189 3311 0.93 0.93 0.039 1.00
Model 3 Bifactor model 14,856 3237 0.93 0.93 0.038 1.00
Model 4 Bifactor model with correlating specic 11,539 3201 0.95 0.95 0.032 1.00
factors
Brief Symptom Inventory
Model 0 One-factor model 8627 1127 0.93 0.92 0.051 0.014
Model 1 Nine rst-ordered factors 5825 1091 0.95 0.95 0.041 1.00
Model 2 Second-order factor model 6416 1118 0.95 0.95 0.043 1.00
Model 3 Bifactor model 5573 1078 0.95 0.95 0.041 1.00
Model 4 Bifactor model with correlating specic 4614 1042 0.96 0.96 0.037 1.00
factors
Gender invariance testing of Brief Symptom Inventory (Model 4)
Males separately 2198 1042 0.97 0.97 0.030 1.00
Females separately 2826 1042 0.97 0.96 0.036 1.00
Congural invariance: no equality constrains 5009 2084 0.97 0.97 0.033 1.00
Metric invariance: equality constrains on factor loadings 3776 2182 0.97 0.97 0.024 1.00
Scalar invariance: equality constrains on factor loadings and 4890 2319 0.97 0.97 0.030 1.00
thresholds

Note: chi-square test statistic; d.f. degrees of freedom; CFI comparative t index; TLI TuckerLewis index; RMSEA root mean square error of approximation;
CFit closeness of t. Direct comparison of values of different models directly from t indices is not possible in case of wlsmv estimation. The only way to compare nested
models is to use difftest procedure (see text).

precision of each subscale in assessing the blend of global distress
and specic symptoms we calculated coefcient omega; and in
assessing only specic problems or only global distress we
computed coefcient omega hierarchical (for details, see Brunner
et al., 2012). We report all omega and omega hierarchical coef-
cients in Table 2. Omega coefcients are above 0.90 in all factors
with the exception of Paranoid ideation. Although there is not any
clearly dened cut-off for omega hierarchical coefcients, we
propose that coefcients around 0.10 can be regarded salient;
therefore the specic factor explained approximately at least 10%
variance of the symptoms score. Somatization, depression, anxiety,
phobic anxiety hostility and paranoid ideation yielded salient
omega hierarchical coefcients.
The range of correlations between specic factors is between
0.87 and 0.94, and details are presented in Table 3. The pattern
of correlations demonstrated that somatization, obsessive-com-
pulsive, anxiety, phobic anxiety, interpersonal sensitivity and
depression symptoms correlated largely and positively. On the
other hand, we identied moderate positive correlations among
hostility, paranoid ideation and psychoticism. Correlations
between these two groups of symptoms are usually negative and
strong.
3.2. Measurement models for BSI
We tested the four measurement models in the restricted range
of items constructing the BSI. The t indices are reported in
Table 1. The test was signicant in all four models in this case
as well; however, this test is oversensitive to large sample size.
Thus we performed the difftest procedure to compare the bifactor
model with the other models. The model containing nine rst-
ordered factors (Model 1) resulted in a signicantly better t to the
current data than the second-ordered factor model (Model 2;
test 559, d.f. 27, p o0.0001). Model 3 (bifactor model) yielded
signicantly closer t to the data ( test 1058, d.f. 40,
p o0.0001) than Model 2. However, in this case comparison of
Model 1 with Model 3 computation of was not possible due to
the singular matrix during the computation process. As for other
t indices, the bifactor model yielded an excellent degree of t,
because CFI and TLI are larger than 0.95 and RMSEA is below 0.05.
However, the nine-factor model also has an acceptable degree of
t. We also tested the bifactor model with correlating specic
factors (Model 4) which yielded a signicantly better t to the data
than Model 3 ( 887, d.f. 36, p o0.0001). Therefore we use
this model (Model 4) in the further analyses. In the bifactor model,
all items loaded signicantly on global distress factors (see
Table 3). The factor loadings ranged from 0.59 to 0.84. With the
exception of nine items (8, 11, 34, 37, 41, 45, 46, 69, 78 according to
their original numbers), all items loaded signicantly on their
specic factors. These nine items seem to represent the global
distress more than the specic psychiatric symptoms. One impor-
tant consideration is that interpersonal sensitivity factor does not
have any signicant factor loadings (items 34, 37, 41, 49) after
controlling the global distress factor; therefore this factor is not
separated from the global distress in the model.
We estimated the explained common variance index in the
bifactor model and found that regarding BSI the Global severity
factor explains 83% of common variance supporting the presence
of a strong global factor. The explained common variances of
specic factors are also reported in Table 3. Only Somatization
scale has relatively large proportion of variance. We also reported
omega and omega hierarchical indices in Table 3. All omega
coefcients are above 0.80. Salient omega hierarchical indices
are present in cases of somatization, depression, hostility, paranoid
ideation and psychoticism.
Correlations between specic factors are presented in Table 3.
The range of correlations between specic factors is between  0.65
and 0.79. The pattern of correlations demonstrated that somatiza-
tion, obsessive-compulsive, anxiety, phobic anxiety and depression
symptoms correlated largely and positively. On the other hand, we
identied moderate positive correlations among hostility, paranoid
ideation and psychoticism. The correlations between these two
groups of symptoms are usually negative and strong.
3.3. Gender invariance testing and gender differences: BSI
The measurement invariance of BSI was examined in men and
women by use of multiple group CFA. We estimated the model t
150 R. Urbn et al. / Psychiatry Research 216 (2014) 146154

Table 2
Standardized factor loadings of the bifactor model of SCL-90-R.

Itema (number) SOM O-C I-S DEP ANX HOS PHOB PAR PSY Global

Headaches (1) 0.42 0.38

Faintness (4) 0.52 0.60
Pains in heart/chest (12) 0.56 0.61
Pains in lower back (27) 0.57 0.38
Nausea (40) 0.30 0.68
Soreness of muscles (42) 0.58 0.54
Trouble getting breath (48) 0.46 0.64
Hot/cold spells (49) 0.48 0.59
Numbness (52) 0.42 0.69
Lump in throat (53) 0.31 0.71
Weakness of body (56) 0.54 0.68
Heavy arms/legs (58) 0.58 0.62

Unpleasant thoughts (3) 0.19 0.70

Trouble remembering (9) 0.18 0.62
Worried about sloppiness (10) 0.09 0.70

Feeling blocked (28) 0.41 0.66

Doing things slowly (38) 0.17 0.70
Having to double-check (45) 0.02 0.70
Difculty deciding (46) 0.06 0.69
Mind going blank (51) 0.08 0.81
Trouble concentrating (55) 0.20 0.76
Repeating same actions (65)  0.03 0.77

Feeling critical of others (6) 0.18 0.56

Feeling shy opposite sex (21)  0.02 0.72
Feeling easily hurt (34) 0.22 0.67
Others are unsympathetic (36) 0.35 0.75
People dislike you (37) 0.16 0.71
Feeling inferior to others (41) 0.13 0.77
Uneasy when people are watching you (61) 0.10 0.69
Self-conscious with others (69) 0.09 0.76
Uncomfortable eating/drinking in public (73) 0.24 0.73

Loss of sexual interest (5) 0.19 0.59

Low energy/slow (14) 0.51 0.65
Thoughts of ending life (15) 0.26 0.68
Crying easily (20) 0.33 0.58
Feeling trapped (22) 0.09 0.77

Blaming yourself (26) 0.15 0.74

Feeling lonely (29) 0.28 0.68
Feeling blue (30) 0.41 0.72
Worrying too much (31) 0.35 0.70
No interest in things (32) 0.09 0.73
Hopeless about future (54) 0.23 0.71
Everything is an effort (71) 0.15 0.78
Feeling worthless (79)  0.03 0.79

Nervousness (2) 0.32 0.65

Trembling (17) 0.35 0.73
Suddenly scared (23) 0.18 0.82
Feeling fearful (33) 0.11 0.76
Heart pounding/racing (39) 0.55 0.62
Feeling tense (57) 0.26 0.77
Spells of terror/panic (72) 0.24 0.83
Can't sit still/restless (78)  0.01 0.71
Something bad is going to happen to you (80) 0.08 0.80
Frightening thoughts (86) 0.20 0.73
Easily annoyed (11)  0.03 0.70
Temper outbursts (24) 0.02 0.78
Urges to harm someone (63) 0.56 0.65
Urges to break things (67) 0.53 0.71
Arguing frequently (74) 0.39 0.62
Shouting/throwing (81) 0.44 0.65

Afraid on the street (13) 0.50 0.75

Afraid to go out alone (25) 0.48 0.78
Afraid of public transport (47) 0.51 0.72
Having to avoid things/places/activities (50) 0.17 0.77
Uneasy in crowds (70) 0.20 0.62
Nervous when alone (75) 0.18 0.73
Afraid to faint in public (82) 0.39 0.74

Others are to blame (8) 0.12 0.63

Most people can't be trusted (18) 0.08 0.73
Feeling watched (43) 0.23 0.71
Having beliefs that others do not share (68) 0.43 0.64
 R. Urbn et al. / Psychiatry Research 216 (2014) 146154 151

Table 2 (continued )

Itema (number) SOM O-C I-S DEP ANX HOS PHOB PAR PSY Global

Not getting enough credit (76) 0.35 0.67

People will take advantage (83) 0.21 0.70

Someone can control your thoughts (7) 0.05 0.68

Hearing voices (16) 0.12 0.72
Others knowing your private thoughts (35) 0.01 0.55
Thoughts not your own (62) 0.11 0.79
Feeling lonely with others (77) 0.17 0.79
Thoughts about sex that bother you a lot (84) 0.24 0.71
You should be punished for your sins (85) 0.28 0.78
Something is wrong with your body (87)  0.28 0.70
Never feeling close to another person (88) 0.17 0.77
Something is wrong with your mind (90) 0.10 0.81

Explained common variance 6% 1% 1% 2% 1% 2% 2% 1% 1% 84%

b
Omega 0.95 0.92 0.91 0.94 0.94 0.90 0.93 0.87 0.93 0.99

Omega hierarchicalc 0.37 0.03 0.05 0.09 0.08 0.16 0.17 0.09 0.02 0.97

Correlations between specic factors

Somatization
Obsessive-compulsive 0.64
Interpersonal sensitivity  0.03 0.19
Depression 0.56 0.74 0.28
Anxiety 0.94 0.62 0.03 0.74
Hostility  0.13  0.33 0.05  0.35  0.26
Phobic anxiety 0.39  0.06  0.67  0.02 0.45  0.07
Paranoid ideation  0.22  0.19 0.71  0.21  0.40 0.35  0.58
Psychoticism  0.64  0.70  0.55  0.82  0.87 0.45  0.09  0.03

Note: SOM: somatization; O-C: obsessive-compulsive; I-S: interpersonal sensitivity; DEP: depression; ANX: anxiety; HOS: hostility; PHOB: phobic anxiety; PAR: paranoid
ideation; PSY: psychoticism.
Boldfaced factor loadings and correlations are signicant at least p o 0.05.
a
We used the abbreviated version of items instead of the full item similarly to Paap et al. (2011).
b
Omega refers to the proportion of explained variance in the scale score attributed to the global and specic factors.
c
Omega hierarchical refers to the proportion of explained variance of the scale score attributed to the specic factor, and italicized loadings are salient ( Z 0.20).

in both genders separately, which yielded an adequate degree of t
in both groups (see Table 1). Three nested models with increasing
constraints were estimated. The t indices are reported in Table 1.
First, the measurement model was estimated freely in men and
women together (congural invariance). This unconstrained solu-
tion tted the data satisfactorily supporting the congural invar-
iance of the measurement model. In the second model, the factor
loadings were set as equal between genders (metric invariance).
According to difftest procedure, the degree of t decreased
signicantly ( 164.5, d.f. 98, p o0.0001), however CFI is
less than 0.01. Based on the signicance of , the metric
invariance is not supported; however, CFI index indicated that
factor loadings are invariant between genders. In the third model,
the thresholds were set as equal (scalar invariance). The degree of
t decreased signicantly ( 859.0, d.f. 196, p o0.0001) but
CFI index, which is less than 0.01, indicated that thresholds are
invariant between genders, therefore we cannot reject the scalar
invariance hypothesis.
Explained common variances of specic factors and global
factor, omega and omega hierarchical coefcients in men and
women are reported in Table 4. The explained common variances
of global factor and omega coefcients are similar in men and
women. We observed relatively large differences in several omega
hierarchical indices; larger proportion of explained variance is
attributed to specic factors of somatization, depression, hostility,
phobic anxiety and psychoticism in women than men.
Based on the results supporting the gender invariance of
measurement models, we tested the gender difference in psychia-
tric symptoms and global distress with using a MIMIC model. Men
were coded with 1 and women were coded with 2. Women have
higher scores on global severity index (0.08, p o0.001), soma-
tization ( 0.23, p o0.001), obsession-compulsion ( 0.15,
po 0.001), depression ( 0.26, p o0.001) and anxiety (0.68,
po 0.001). Men have higher scores on hostility (  0.20,
po 0.001) and psychoticism ( 0.14, p o0.001). We did not
nd gender difference in phobic anxiety (  0.07, ns.) and paranoid
ideation ( 0.06, ns.).
4. Discussion
Overall, our analysis supported the multidimensional measure-
ment model of symptom checklist (SCL-90-R); however, we
observed a strong global distress factor and weak specic symp-
tom factors. All multifactorial model tted the present data better
than the univariate model. Furthermore, a bifactor model yielded
the closest t to data both in the case of SCL-90-R and BSI similarly
to the most recent report (Thomas, 2012); however, the original
nine-factor measurement model yielded an acceptable level of t
as well in both versions. Our result contradicts several previous
conrmatory factor analyses (Carpenter and Hittner, 1995;
Vassend and Skrondal, 1999; Schmitz et al., 2000; Hafkenscheid
et al., 2006). The reason for the major difference between the
previous and current analyses might be the fact that we applied
estimation method applicable for ordinal scale and non-normal
distribution of item responses. Besides conrming multidimen-
sionality, the present bifactor model also supports the hierarchical
structure of psychiatric symptoms which species the general and
domain-specic components (Watson, 2005; Thomas, 2012).
Although the analysis of general and domain-specic components
revealed that the general or global distress factor explained the
vast majority of variance in the present sample, some specic
symptom factors such as somatization, depression, hostility phobic
anxiety have also salient contribution in explaining the variance of
152 R. Urbn et al. / Psychiatry Research 216 (2014) 146154

Table 3
Standardized factor loadings of the bifactor model of Brief Symptom Inventory.

Itema (number) SOM O-C I-S DEP ANX HOS PHOB PAR PS Global

Faintness (4) 0.53 0.59

Pains in heart/chest (12) 0.54 0.61
Nausea (40) 0.30 0.68
Trouble getting breath (48) 0.53 0.63
Hot/cold spells (49) 0.50 0.58
Numbness (52) 0.42 0.68
Weakness of body (56) 0.50 0.68

Trouble remembering (9) 0.18 0.63

Feeling blocked (28) 0.31 0.67
Having to double-check (45) 0.00 0.70
Difculty deciding (46) 0.02 0.70
Mind going blank (51) 0.14 0.81
Trouble concentrating (55) 0.17 0.77

Feeling easily hurt (34)  0.07 0.68

People dislike you (37) 0.04 0.71
Feeling inferior to others (41)  0.01 0.78
Self-conscious with others (69) 0.10 0.74

Thoughts of ending life (15) 0.18 0.69

Feeling lonely (29) 0.48 0.67
Feeling blue (30) 0.65 0.72
No interest in things (32) 0.10 0.74
Hopeless about future (54) 0.16 0.73
Feeling worthless (79) 0.06 0.79

Nervousness (2) 0.27 0.65

Suddenly scared (23) 0.20 0.83
Feeling fearful (33) 0.15 0.76
Feeling tense (57) 0.23 0.79
Spells of terror/panic (72) 0.28 0.84
Can't sit still/restless (78)  0.04 0.72

Easily annoyed (11) 0.05 0.70

Temper outbursts (24) 0.06 0.79
Urges to harm someone (63) 0.61 0.64
Urges to break things (67) 0.54 0.70
Arguing frequently (74) 0.35 0.61

Afraid on the street (13) 0.54 0.70

Afraid of public transport (47) 0.56 0.68
Having to avoid things/places/activities (50) 0.24 0.76
Uneasy in crowds (70) 0.24 0.60
Nervous when alone (75) 0.15 0.72

Others are to blame (8) 0.03 0.62

Most people can't be trusted (18) 0.08 0.73
Feeling watched (43) 0.24 0.70
Not getting enough credit (76) 0.38 0.68
People will take advantage (83) 0.25 0.69

Someone can control your thoughts (7) 0.08 0.66

Feeling lonely with others (77) 0.33 0.76
You should be punished for your sins (85) 0.37 0.72
Never feeling close to another person (88) 0.35 0.74
Something is wrong with your mind (90) 0.25 0.79

Explained common variance 0.06 0.01 0.00 0.02 0.01 0.03 0.02 0.01 0.01 0.83
Omega 0.92 0.87 0.82 0.91 0.91 0.89 0.89 0.84 0.89 0.98
Omega hierarchical 0.33 0.03 0.00 0.11 0.05 0.16 0.18 0.06 0.11 0.96

Correlations between specic factors

Somatization
Obsessive-compulsive 0.73
Interpersonal sensitivity N/A N/A
Depression 0.25 0.36 N/A
Anxiety 0.79 0.27 N/A 0.53
Hostility  0.10  0.27 N/A  0.21  0.32
Phobic anxiety 0.53 0.22 N/A 0.06 0.58  0.04
Paranoid ideation  0.24  0.65 N/A  0.21  0.64 0.23  0.29
Psychoticism  0.08  0.22 N/A 0.09  0.54 0.34 0.20 0.31

Note: SOM: somatization; O-C: obsessive-compulsive; I-S: interpersonal sensitivity; DEP: depression; ANX: anxiety; HOS: hostility; PHOB: phobic anxiety; PAR: paranoid
ideation; PSY: psychoticism.
Boldfaced factor loadings and correlations are signicant at least p o 0.05, and italicized loadings are salient ( Z 0.20).
a
We used the abbreviated version of items instead of the full item similarly to Paap et al. (2011).
 R. Urbn et al. / Psychiatry Research 216 (2014) 146154 153

Table 4
Gender differences in model based reliability indices.

Indices SOM O-C I-S DEP ANX HOS PHOB PAR PS Global

Males
Explained common variance 0.03 0.01 0.01 0.02 0.01 0.02 0.01 0.01 0.01 0.86
Omega 0.93 0.89 0.85 0.91 0.91 0.89 0.88 0.85 0.90 0.98
Omega hierarchical 0.20 0.03 0.07 0.09 0.00 0.17 0.04 0.10 0.05 0.97
Females
Explained common variance 0.06 0.01 0.00 0.03 0.01 0.03 0.03 0.01 0.02 0.80
Omega 0.92 0.87 0.81 0.91 0.91 0.89 0.90 0.84 0.89 0.97
Omega hierarchical 0.35 0.06 0.00 0.14 0.06 0.23 0.23 0.08 0.13 0.95

Note: SOM: somatization; O-C: obsessive-compulsive; I-S: interpersonal sensitivity; DEP: depression; ANX: anxiety; HOS: hostility; PHOB: phobic anxiety; PAR: paranoid
ideation; PSY: psychoticism. Salient omega hierarchical indices ( Z0.10) are in bold.

the scores of these symptom factors. This bifactor model ts well
with the ndings of high comorbidity among mental disorders in
large scale epidemiological studies (Kessler et al., 1994). Comor-
bidity among mental disorders is rather a rule than an exception,
and symptomatically homogenous groups without symptoms of
other mental disorders are rarely found in empirical data on
psychopathological symptoms neither in representative nor in
clinical population (Krueger et al., 2001; Krueger and Piasecki,
2002; Roca et al., 2009). However, besides the global distress
factor, the bifactor model also strengthens the separation of the
nine self-reported symptom-clusters. Furthermore, the bifactor
model also highlights that a few items may represent the global
severity more than the specic symptoms in a large non-clinical
sample, although these items should be tested in clinical samples
as well.
A recent study (Reise et al., 2013) proposes that in case of high
explained common variance of the general factor might be an
indicator of unidimensionality of the measure, which raises criti-
cism regarding our conclusion about the multidimensionality of
SCL-90-R and BSI-53. Nevertheless, the t indices and examination
of omega hierarchical indices supported our proposition. More-
over, the advantage of bifactor model is that the strength of global
factor and symptom specic factors can be quantied and pre-
dicted by other variables, which is not possible when using the
unidimensional model. When the degree of model t allows, the
bifactor model can be used to compare groups with different
distress levels or with different sociocultural backgrounds. Recent
studies demonstrated that the gender and level of psychological
distress were related to dimensionality (Paap et al., 2012). Low
level of psychological distress can support unidimensionality,
while research on samples characterized with high level of
psychological distress supports multidimensionality (Paap et al.,
2011, 2012). However, these studies used a different statistical
approach to decide about the dimensionality. Our future goal is to
test the impact of distress on the model t of bifactor model and
model-based reliability indices such as explained common var-
iances, omega hierarchicals in groups characterized with severe
psychological distress.
In order to support the construct validity of the symptom check-
list, we tested the gender differences in global and specic factors.
Before examining gender differences we performed the gender
invariance analysis of the bifactor model, and we supported the
gender invariance with applying new criteria related to change in a
goodness of t index. The examination of gender difference revealed
that on the one hand women scored higher on the global severity
factor, which supports the general view that women tend to report
more somatic and psychological symptoms than men (Barsky et al.,
2001; A, 2007). On the other hand, women reported higher level
of somatization, obsession-compulsion, interpersonal sensitivity,
depression, and anxiety, and men reported higher level of hostility
and psychoticism. Our results are consistent with previous reports
regarding gender difference in somatization (Wool and Barsky, 1994),
depression (Nolen-Hoeksema et al., 1999), symptoms of anxiety
(Pigott, 1999; McLean et al., 2011), hostility (Huesmann and Eron,
1989; Fava et al., 1995), and psychoticism (Golomb et al., 1995).
Our study has several strengths and limitations. Using
community-based sampling involves non-clinical samples in
which the level of distress and the number of symptoms are lower
than in a clinical sample. Moreover, owing to this sampling, we
could rely only on self-report and had no opportunity to validate
the responses with additional clinical interviewing. Another lim-
itation of this study is that the analyses of SCL-90-R and BSI are
based on administration of SCL-90-R. External validity of the
present study is limited to community samples. On the other
hand, however, SCL-90-R and BSI have never been studied before
on a national representative sample.
5. Conclusion
The present study demonstrated the usefulness and feasibility
of the administration of SCL-90-R in large scale community studies
and supported the construct validity of both the SCL-90-R and BSI.
The best tting model reinforces the use of global severity index
and the specic symptoms. Our results explain the large correla-
tions between symptom factors, which are represented in the
global severity factor. Latent variables modeling also provides the
statistical opportunity to disentangle the global severity factor and
specic symptom factors in relation to covariates and other
determinants in further research on non-clinical and clinical
samples. For example, further research could clarify whether the
correlates of mental health problems such as drug or alcohol use,
Internet addiction, or smoking are associated with global severity
factor and/or specic symptom factors.
Acknowledgments
Present work was supported by the European Union and co-
nanced by the European Social Fund (Grant agreement no.
TAMOP 4.2.1/B-09/1/KMR-2010-0003) and the Hungarian Scienti-
c Research Fund Grant 83884 and 109375. Gyngyi Kknyei and
Zsolt Demetrovics acknowledge nancial support of the Jnos
Bolyai Research Fellowship awarded by the Hungarian Academy
of Science.
References
A, M., 2007. Gender differences in mental health. Singapore Medical Journal 48,
385391.
Al-Turkait, F.A., Ohaeri, J.U., 2010. Dimensional and hierarchical models of depres-
sion using the Beck Depression Inventory-II in an Arab college student sample.
BMC Psychiatry 10, 60.
154 R. Urbn et al. / Psychiatry Research 216 (2014) 146154
Asparouhov, T., Muthn, B., 2006. Robust Chi square difference testing with mean
and variance adjusted test statistics. Mplus Web Notes, 10
Barsky, A.J., Peekna, H.M., Borus, J.F., 2001. Somatic symptom reporting in women
and men. Journal of General Internal Medicine 16, 266275.
Bentler, P.M., 2009. Alpha, dimension-free, and model-based internal consistency
reliability. Psychometrika 74, 137143.
Brophy, C.J., Norvell, N.K., Kiluk, D.J., 1988. An examination of the factor structure
and convergent and discriminant validity of the SCL-90R in an outpatient clinic
population. Journal of Personality Assessment 52, 334340.
Brown, T.A., 2006. Conrmatory Factor Analysis for Applied Research. The Guilford
Press, New York
Brunner, M., Nagy, G., Wilhelm, O., 2012. A tutorial on hierarchically structured
constructs. Jornal of Personality 80, 796846.
Carpenter, K.M., Hittner, J.B., 1995. Dimensional characteristics of the SCL-90-R:
evaluation of gender differences in dually diagnosed inpatients. Journal of
Clinical Psychology 51, 383390.
Casper, R.C., Belanoff, J., Offer, D., 1996. Gender differences, but no racial group
differences, in self-reported psychiatric symptoms in adolescents. Journal of the
American Academy of Child and Adolescent Psychiatry 35, 500508.
Chen, F.F., 2007. Sensitivity of goodness of t indices to lack of measurement
invariance. Structural Equation Modeling 14, 464504.
Chen, F.F., Sousa, K.H., West, S.G., 2005. Testing measurement invariance of second-
order factor models. Structural Equation Modeling 12, 471492.
Cheung, G.W., Rensvold, R.B., 2002. Evaluating goodness-of-t indexes for testing
measurement invariance. Structural Equation Modeling 9, 233255.
Cyr, J.J., McKenna-Foley, J.M., Peacock, E., 1985. Factor structure of the SCL-90-R: is
there one? Journal of Personality Assessment 49, 571578.
Derogatis, L.R., 1975. Brief Symptom Inventory. Clinical Psychometric Research,
Baltimore, MD
Derogatis, L.R., 1983. SCL-90-R: Administration Scoring and Procedures Manual.
Clinical Psychometric Research, Baltimore, MD
Derogatis, L.R., Savitz, K.L., 2000. The SCL-90-R and the Brief Symptom Inventory
(BSI) in primary care. In: Maruish, M.E. (Ed.), Handbook of Psychological
Assessment in Primary Care Settings. Lawrence Erlbaum Associates, Mahwah,
pp. 217334
Derogatis, L.R., Spencer, P.M., 1982. The Brief Symptom Inventory (BSI): Adminis-
tration and Procedures Manual-I. Clinical Psychometric Research, Baltimore,
MD
Elliott, R., Fox, C.M., Beltyukova, S.A., Stone, G.E., Gunderson, J., Zhang, X., 2006.
Deconstructing therapy outcome measurement with Rasch analysis of a
measure of general clinical distress: the Symptom Checklist-90-Revised.
Psychological Assessment 18, 359372.
Fava, M., Nolan, S., Kradin, R., Rosenbaum, J., 1995. Gender differences in hostility
among depressed and medical outpatients. Journal of Nervous and Mental
Disease 183, 1014.
Finney, S.J., DiStefano, C., 2006. Nonnormal and categorical data in structural
equation modeling. In: Hancock, G.R., Mueller, R.D. (Eds.), Structural Equation
Modeling: A Second Course. Information Age, Greenwich, pp. 269314
Flores-Mendoza, C., Alvarenga, M.A.S., Herrero, O., Abad, F.J., 2008. Factor structure
and behavioural correlates of the Psychopathy Checklist-Revised [PCL-R] in a
Brazilian prisoner sample. Personality and Individual Differences 45, 584590.
Gibbons, R.D., Bock, R.D., Hedeker, D., Weiss, D.J., Segawa, E., Bhaumik, D.K., 2007.
Full-information item bifactor analysis of graded response data. Applied
Psychological Measurement 31, 419.
Golomb, M., Fava, M., Abraham, M., Rosenbaum, J.F., 1995. Gender differences in
personality disorders. American Journal of Psychiatry 152, 579582.
Hafkenscheid, A., 1993. Psychometric evaluation of the symptom checklist (SCL-90)
in psychiatric inpatients. Personality and Individual Differences 14, 751756.
Hafkenscheid, A.J., Maassen, G.H., Veeninga, A.T., 2006. The dimensions of the
Dutch SCL-90-R: more than one, but how many? Netherlands Journal of
Psychology 63, 2935.
Hoffmann, N.G., Overall, P.B., 1978. Factor structure of the SCL-90 in a psychiatric
population. Journal of Consulting and Clinical Psychology 46, 11871191.
Holi, M.M., Sammallahti, P.R., Aalberg, V.A., 1998. A Finnish validation study of the
SCL-90. Acta Psychiatrica Scandinavica 97, 4246.
Huesmann, L.R., Eron, L.D., 1989. Individual differences and the trait of aggression.
European Journal of Personality 3, 95106.
Kessler, R.C., McGonagle, K.A., Zhao, S., Nelson, C.B., Hughes, M., Eshleman, S.,
Wittchen, H.U., Kendler, K.S., 1994. Lifetime and 12-month prevalence of DSM-
III-R psychiatric disorders in the United States. Results from the National
Comorbidity Survey. Archives of General Psychiatry 51, 819.
Kimonis, E.R., Frick, P.J., Skeem, J.L., Marsee, M.A., Cruise, K., Munoz, L.C., Aucoin, K.J.,
Morris, A.S., 2008. Assessing callous-unemotional traits in adolescent offen-
ders: validation of the inventory of callous-unemotional traits. International
Journal of Law and Psychiatry 31, 241252.
Krueger, R.F., McGue, M., Iacono, W.G., 2001. The higher-order structure of common
DSM mental disorders: internalization externalization and their connections to
personality. Personality and Individual Differences 30, 12451259.
Krueger, R.F., Piasecki, T.M., 2002. Toward a dimensional and psychometrically-
informed approach to conceptualizing psychopathology. Behaviour Research
and Therapy 40, 485499.
Linzer, M., Spitzer, R., Kroenke, K., Williams, J.B., Hahn, S., Brody, D., deGruy, F.,
1996. Gender, quality of life, and mental disorders in primary care: results from
the PRIME-MD 1000 study. American Journal of Medicine 101, 526533.
Low, G.D., 1988. The semantics of questionnaire rating scales. Evaluation & Research
in Education 2, 6979.
Martel, M.M., Roberts, B., Gremillion, M., von Eye, A., Nigg, J.T., 2011. External
validation of bifactor model of ADHD: explaining heterogeneity in psychiatric
comorbidity, cognitive control, and personality trait proles within DSM-IV
ADHD. Journal of Abnormal Child Psychology 39, 11111123.
McLean, C.P., Asnaani, A., Litz, B.T., Hofmann, S.G., 2011. Gender differences in
anxiety disorders: prevalence, course of illness, comorbidity and burden of
illness. Journal of Psychiatric Research 45, 10271035.
Mineka, S., Watson, D., Clark, L.A., 1998. Comorbidity of anxiety and unipolar mood
disorders. Annual Review of Psychology 49, 377412.
Muthn, L.K., Muthn, B.O., 1998-2007. Mplus User's Guide, 5th ed. Muthn &
Muthn, Los Angeles, CA
Nolen-Hoeksema, S., Larson, J., Grayson, C., 1999. Explaining the gender difference
in depressive symptoms. Journal of Personality and Social Psychology 77,
10611072.
Olsen, L.R., Mortensen, E.L., Bech, P., 2004. The SCL-90 and SCL-90R versions
validated by item response models in a Danish community sample. Acta
Psychiatrica Scandinavica 110, 225229.
Ostrov, E., Offer, D., Howard, K.I., 1989. Gender differences in adolescent sympto-
matology: a normative study. Journal ofthe American Academy of Child and
Adolescent Psychiatry 28, 394398.
Paap, M.C., Meijer, R.R., Van Bebber, J., Pedersen, G., Karterud, S., Hellem, F.M.,
Haraldsen, I.R., 2011. A study of the dimensionality and measurement precision
of the SCL-90-R using item response theory. International Journal of Methods in
Psychiatric Research 20, e39e55.
Paap, M.C., Meijer, R.R., Cohen-Kettenis, P.T., Richter-Appelt, H., de Cuypere, G.,
Kreukels, B.P., Pedersen, G., Karterud, S., Malt, U.F., Haraldsen, I.R., 2012. Why
the factorial structure of the SCL-90-R is unstable: comparing patient groups
with different levels of psychological distress using Mokken Scale Analysis.
Psychiatry Research 200, 819826.
Paksi, B., Rzsa, S., Kun, B., Arnold, P., Demetrovics, Z., 2009. A magyar npessg
addiktolgiai problmi: az Orszgos Lakossgi Adatfelvtel az Addiktolgiai
Problmkrl (OLAAP) reprezentatv felmrs mdszertana s a minta ler
jellemzi. [Addictive behaviours in Hungary: the methodology and sample
description of the national survey on addiction problems in Hungary (NSAPH)].
Mentlhigin s Pszichoszomatika 10, 273300.
Pigott, T.A., 1999. Gender differences in the epidemiology and treatment of anxiety
disorders. Journal of Clinical Psychiatry 60 (Suppl. 18), 415.
Prunas, A., Sarno, I., Preti, E., Madeddu, F., Perugini, M., 2012. Psychometric
properties of the Italian version of the SCL-90-R: a study on a large community
sample. European Psychiatry 27, 591597.
Reise, S.P., 2012. The rediscovery of bifactor measurement models. Multivariate
Behavioral Research 47, 667696.
Reise, S.P., Moore, T.M., Haviland, M.G., 2010. Bifactor models and rotations:
exploring the extent to which multidimensional data yield univocal scale
scores. Journal of Personality Assessment 92, 544559.
Reise, S.P., Scheines, R., Widaman, K.F., Haviland, M.G., 2013. Multidimensionality
and structural coefcient bias in structural equation modeling. A bifactor
perspective. Educational and Psychological Measurement 73, 526.
Roca, M., Gili, M., Garcia-Garcia, M., Salva, J., Vives, M., Garcia Campayo, J., Comas,
A., 2009. Prevalence and comorbidity of common mental disorders in primary
care. Journal of Affective Disorders 119, 5258.
Schmitz, N., Hartkamp, N., Kiuse, J., Franke, G.H., Reister, G., Tress, W., 2000. The
Symptom Check-List-90-R (SCL-90-R): a German validation study. Quality of
Life Research 9, 185193.
Ten Berge, J.M., Soan, G., 2004. The greatest lower bound to the reliability of a test
and the hypothesis of unidimensionality. Psychometrika 69, 613625.
Thomas, M.L., 2012. Rewards of bridging the divide between measurement and
clinical theory: demonstration of a bifactor model for the Brief Symptom
Inventory. Psychological Assessment 24, 101113.
Vassend, O., Skrondal, A., 1999. The problem of structural indeterminacy in
multidimensional symptom report instruments. The case of SCL-90-R. Beha-
viour Research and Therapy 37, 685701.
Vicente, B., Kohn, R., Rioseco, P., Saldivia, S., Levav, I., Torres, S., 2006. Lifetime and
12-month prevalence of DSM-III-R disorders in the Chile psychiatric prevalence
study. American Journal of Psychiatry 163, 13621370.
Watson, D., 2005. Rethinking the mood and anxiety disorders: a quantitative
hierarchical model for DSM-V. Journal of Abnormal Psychology 114, 522536.
Wool, C.A., Barsky, A.J., 1994. Do women somatize more than men? Gender
differences in somatization. Psychosomatics 35, 445452.