Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
965
Martin E. P.
Seligman
lasting benefits; that cognitive therapy works very well in
panic disorder; that systematic desensitization relieves specific phobias; that "applied tension" virtually cures blood
and injury phobia; that transcendental meditation relieves
anxiety; that aversion therapy produces only marginal improvement with sexual offenders; that disulfram (Antabuse)
does not provide lasting relief from alcoholism; that flooding
plus medication does better in the treatment of agoraphobia
than either alone; and that cognitive therapy provides significant relief of bulimia, outperforming medications alone
(see Seligman, 1994, for a review).
The high praise "empirically validated" is now virtually
synonymous with positive results in efficacy studies, and
many investigators have come to think that an efficacy study
is the "gold standard" for measuring whether a treatment
works.
I also had come to that opinion when I wrote What You
Can Change & What You Can't (Seligman, 1994). In trying
to summarize what was known about the effects of the panoply of drugs and psychotherapies for each major disorder, I
read hundreds of efficacy studies and came to appreciate the
genre. At minimum I was convinced that an efficacy study
may be the best scientific instrument for telling us whether a
novel treatment is likely to work on a given disorder when
the treatment is exported from controlled conditions into the
field. Because treatment in efficacy studies is delivered under tightly controlled conditions to carefully screened patients, sensitivity is maximized and efficacy studies are very
useful for deciding whether one treatment is better than
another treatment for a given disorder.
But my belief has changed about what counts as a
"gold standard." And it was a study by Consumer Reports
(1995, November) that singlehandedly shook my belief. I
came to see that deciding whether one treatment, under
highly controlled conditions, works better than another treatment or a control group is a different question from deciding
966
mental health professional like a psychologist or a psychiatrist; your family doctor; or a support group." Twenty-two
thousand readers responded. Of these, approximately 7,000
subscribers responded to the mental health questions. Of
these 7,000, about 3,000 had just talked to friends, relatives,
or clergy, and 4,100 went to some combination of mental
health professionals, family doctors, and support groups. Of
these 4,100, 2,900 saw a mental health professional: Psychologists (37%) were the most frequently seen mental health
professional, followed by psychiatrists (22%), social workers (14%), and marriage counselors (9%). Other mental health
professionals made up 18%. In addition, 1,300 joined selfhelp groups, and about 1,000 saw family physicians. The
respondents as a whole were highly educated, predominantly middle class; about half were women, and the median
age was 46.
Twenty-six questions were asked about mental health
professionals, and parallel but less detailed questions were
asked about physicians, medications, and self-help groups:
What kind of therapist
What presenting problem (e.g., general anxiety, panic,
phobia, depression, low mood, alcohol or drugs, grief,
weight, eating disorders, marital or sexual problems,
children or family, work, stress)
Emotional state at outset (from very poor to very
good)
Emotional state now (from very poor to very good)
Group versus individual therapy
Duration and frequency of therapy
Modality (psychodynamic, behavioral, cognitive,
feminist)
Cost
Health care plan and limitations on coverage
Therapist competence
How much therapy helped (from made things a lot
better to made things a lot worse) and in what areas
(specific problem that led to therapy, relations to
others, productivity, coping with stress, enjoying life
more, growth and insight, self-esteem and confidence,
raising low mood)
Satisfaction with therapy
Reasons for termination (problems resolved or more
manageable, felt further treatment wouldn't help,
therapist recommended termination, a new therapist,
concerns about therapist's competence, cost, and
problems with insurance coverage)
Figure 1
Duration of Therapy
250
240
CD
230
220
210
200
190
NN
Note. N - 2 , 8 4 6 . The 300-point scale is derived from the unweighted sum of responses to three 10O-point subscales. The subscales measured specific improvement
(i.e., how much treatment helped with problems that led to therapy), satisfaction with therapist, and global improvement (i.e., how respondents felt at time of survey,
compared with when they began treatment).
968
Methodological
Table 1
Limitations on Insurance Coverage and Improvement
Coverage limited
Limitations on your
insurance coverage
20
26
24
43
Overall score
Specific improvement
Overall score
211
214
212
212
77
79
78
78
224
224
224
226
Specific improvement
83
82
83
83
Note. N - 2,900. All differences for the overall scores were statistically significant at p < . 0 1 . The same held true for the specific score, except for "How often 1 met with
my therapist," which was significant at p < .05. Statistical controls for both severity and duration were applied. Source: Consumer Reports 1994 Annual Questionnaire.
"multiple responses permitted.
969
Figure 2
Improvement for Presenting Symptoms
D < 6 months
> 6 months
60
~50
c
01
I 40
o
130
o
S"20
10
&
Figure 3
Self-correction.
Because the CR study was naturalistic, it informs us of how treatment works as it is actually
performedwithout manuals and with self-correction when
a technique falters. This also contrasts favorably to efficacy
studies, which are manualized and not self-correcting when
a given technique or modality fails.
Multiple problems. The large majority of respondents in the CR study had more than one problem. We can
also assume that a good-sized fraction were "subclinical" in
their problems and would not meet DSM-IV criteria for any
disorder. No patients were discarded because they failed
exclusion criteria or because they fell one symptom short of
a full-blown "disorder." Thus the sample more closely reflected people who actually seek treatment than the filtered
and single-disordered patients of efficacy studies.
Note. N - 2,738. Mean percentage who reported that treatment "made things a
lot better" with respect to three domains: ability to relate to others, productivity at
work, and coping with everyday stress. Those treated by psychiatrists, psychologists,
social workers, marriage counselors, and physicians are segregated by treatment
for more than six months versus treatment for less than six months.
Figure 4
Improvement Over Personal Domains
40
I < 6 months!
I > 6 months
30
E
CD
20
o
(8"
10
Note. N - 2, 738. Mean percentage who reported that treatment "made things a
lot better" with respect to four domains: enjoying life more, personal growth and
insight, self-esteem and confidence, and alleviating low moods. Those treated by
psychiatrists, psychologists, social workers, marriage counselors, and physicians
are segregated by treatment for more than six months versus treatment for less than
six months.
also pay their own postage and are not compensated. So the
return rate is rather low absolutely, although the 13% return
rate for this survey was normal for the annual questionnaire.
But it is still possible that respondents might differ systematically from the readership as a whole. For the mental
health survey (and for their annual questionnaires generally), CR conducted a "validation survey," in which postage
was paid and the respondent was compensated. This resulted in a return rate of 38%, as opposed to the 13%
uncompensated return rate, and there were no differences
between data from the two samples.
The possibility of two other kinds of sampling bias,
however, is notable, particularly with respect to the remarkably good results for AA. First, since AA encourages lifetime membership, a preponderance of successesrather
than dropoutswould be more likely in the three-year time
slice (e.g., "Have you had help in the last three years?").
Second, AA failures are often completely dysfunctional and
thus much less likely to be reading CR and filling out extensive readers' surveys than, say, psychotherapy failures
who were unsuccessfully treated for anxiety.
A similar kind of sampling bias, to a lesser degree,
cannot be overlooked for other kinds of treatment failures.
At any rate, it is quite possible that there was a large
oversampling of successful AA cases and a smaller
oversampling of successful treatment for problems other
than alcoholism.
Could the benefits of long-term treatment be an artifact
of sampling bias? Suppose that people who are doing well in
treatment selectively remain in treatment, and people who
are doing poorly drop out earlier. In other words, the early
dropouts are mostly people who fail to improve, but later
dropouts are mostly people whose problem resolves. CR
disconfirmed this possibility empirically: Respondents reported not only when they left treatment but why, including
leaving because their problem was resolved. The dropout
rates due to the resolution of the problem were uniform
across duration of treatment (less than one month = 60%; 1
2 months = 66%; 3-6 months = 67%, 7-11 months = 67%; 12 years = 67%; over two years = 68%).
A more sweeping limit on generalizability comes from
the fact that the entire sample chose their treatment. To one
degree or another, each person believed that psychotherapy
and/or drugs would help him or her. To one degree or
another, each person acknowledged that he or she had a
problem and believed that the particular mental health professional seen and the particular modality of treatment chosen would help them. One cannot argue compellingly from
this survey that treatment by a mental health professional
would prove as helpful to troubled people who deny their
problems and who do not believe in and do not choose
treatment.
N o control groups. The overall improvement rates
were strikingly high across the entire spectrum of treatments and disorders in the CR study. The vast majority of
people who were feeling very poor orfairly poor when they
entered therapy made "substantial" (now feeling/air/}' good
or very good) or "some" (now feeling so-so) gains. Perhaps
971
the best news for patients was that those with severe problems got, on average, much better. While this may be a
ceiling effect, it is a ceiling effect with teeth. It means that if
you have a patient with a severe disorder now, the chances
are quite good that he or she will be much better within three
years. But methodologically, such high rates of improvement
are a yellow flag, cautioning us that global improvement over
time alone, rather than with treatment or medication, may be
the underlying mechanism.
More generally, because there are no control groups,
the CR study cannot tell us directly whether talking to sympathetic friends or merely letting time pass would have produced just as much improvement as treatment by a mental
health professional. The CR survey, unfortunately, did not
ask those who just talked to friends and clergy to fill out
detailed questionnaires about the results.
This is a serious objection, but there are internal controls which perform many of the functions of control groups.
First, marriage counselors do significantly worse than psychologists, psychiatrists, and social workers, in spite of no
significant differences in kind of problem, severity of problem, or duration of treatment. Marriage counselors control
for many of the nonspecifics, such as therapeutic alliance,
rapport, and attention, as well as for passage of time. Second,
there is a dose-response curve, with more therapy yielding
more improvement. The first point in the dose-response
curve approximates no treatment: people who have less than
one month of treatment have on average an improvement
score of 201, whereas people who have over two years of
treatment have an average score of 241. Third, psychotherapy does just as well as psychotherapy plus drugs for all
disorders, and there is such a long history of placebo controls inferior to these drugs that one can infer that psychotherapy likely would have outperformed such controls had
they been run. Fourth, family doctors do significantly worse
than mental health professionals when treatment continues
beyond six months. An objection might be made that since
total length of time in treatmentrather than total amount of
contactis the covariate, comparing family doctors who do
not see their patients weekly with mental health professionalswho see their patients once a week or moreis not fair.
It is, of course, possible that if family doctors saw their
patients as frequently as psychologists do, the two groups
would do equally well. It was notable, however, that there
were a significant number of complaints about family doctors: 22% of respondents said their doctor had not "provided
emotional support"; 15% said their doctor "seemed uncomfortable discussing emotional issues"; and 18% said their
doctor was "too busy to spend time talking to me." At any
rate, the CR survey shows that long-term family doctoring
for emotional problemsas it is actually performed in the
fieldis inferior to long-term treatment by a mental health
professional as it is actually performed in the field.
It is also relevant that the patients attributed their improvement to treatment and not time (determined by responses to "How much do you feel that treatment helped
you in the following areas?"), and I conclude that the benefits of treatment are very unlikely to be caused by the mere
passage of time. But I also conclude that the CR study could
be improved by control groups whose members are not
972
CR's indexes
view, the patient retrospectively reports on his or her (presumably) less troubled emotional state before the diagnosis.
Therapy junkies. Perhaps the important finding
that long-term therapy does so much better than short-term
therapy is an artifact of therapy "junkies," individuals so
committed to therapy as a way of life that they bias the
results in this direction. This is possible, but it is not an
artifact. Those people who spend a long time in therapy may
well be "true believers." Indeed, the long-term patients are
distinct: They have more severe problems initially, are more
likely to have an emotional disorder, are more likely to get
medications, are more likely to see a psychiatrist, and are
more likely to have psychodynamic treatment than the rest of
the sample. Regardless, they are probably representative of
the population served by long-term therapy. This population
reports robust improvement with long-term treatment in the
specific problem that got them into therapy, as well as in
growth, insight, confidence, productivity at work, interpersonal relations, and enjoyment of life.
Perhaps people who had two or more years of therapy
are likely still to be in therapy and thus unduly loyal to their
therapist. They might then be more likely to distort in a rosy
direction. This seems unlikely, since a comparison of people
who had over two years of treatment and then ended therapy
showed the same high improvement scores as those with
over two years of treatment who were still in therapy (242 and
245, respectively).
Nonrandom assignment. The possibility of such
biases could be reduced by random assignment of patients
to treatment, but this would undermine the central virtue of
the CR studyreporting on the effectiveness of psychotherapy as it is actually done in the field with those patients
who actually seek it. In fact, the lack of random assignment
may turn out to be the crucial ingredient in the validity of the
CR method and a major flaw of the efficacy method. Many
(but assuredly not all) of the problems that bring consumers
into therapy have elements of what was called "wanhope" in
the middle ages and is now called "demoralization." Choice
and control by a patient, in and of itself, counteracts wanhope
(Seligman, 1991).
Random assignment of patients to a modality or to a
particular therapist not only undercuts the remoralizing effects of treatment but also undercuts the nonrandom decisions of therapists in choice of modality for a particular
patient. Consider, for example, the finding that drugs plus
psychotherapy did no better than psychotherapy alone for
any disorder (schizophrenia and bipolar depression were too
rare for analysis in this sample). The most obvious interpretation is that drugs are useless and do nothing over and
above psychotherapy. But the lack of random assignment
should prevent us from leaping to that conclusion. Assume,
for the moment, that therapists are canny about who needs
drugs plus psychotherapy and who can do well with psychotherapy alone. The therapists assign those patients accordingly so appropriate patients get appropriate treatment.
This is just the same logic as a self-correcting trajectory of
treatment, in which techniques and modalities are modified
with the patient's progress. This means that drugs plus
psychotherapy may actually have done pretty well after all
973
974