Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
A manual
JCJM de Haes
M Olschewski
P Fayers
MRM Visser
A Cull
P Hopwood
R Sanderman
e
2 druk
2012
www.rug.nl/share
R. Sanderman PhD
Juni 2012,
4 Acknowledgement
Introduction
. Rotterdam Symptom Checklist, the history and development
Description of the and instructions
. Content of the
Content of scales and subscales
Response categories
Experienced distress
Time frame
. Scoring and analysis
Scoring of items and (sub)scales
Standardization of scoring
Missing values
Descriptive use
Use in treatment comparison
Case detection
. Guidelines for use of the
Populations
Adaption and modules
Administration modes
‘Help’ and ‘Proxy’s’
Time needed
Use for case detection
Number of measurements and measurement moments
Structure, reliability and validity
. Description of samples
The original Dutch validation studies
The study
The Fatigue study
The study
. Structure of the symptom distress scale
Original Dutch validation studies
The study
The Fatigue study
. Reliability
. Validity
Construct validity
Clinical validity
Normative data
Bibliography
References
Hanneke de Haes 1
6 Over the past decades the importance of investigat- 1.1 Rotterdam Symptom Checklist, the
ing the quality of life of cancer patients has been history and development
stressed increasingly. Quality of life is regarded as a
subjective report of the patients’ experience of dis- One instrument to measure the quality of life of
ease and treatment. Assessing quality of life is meant cancer patients is the Rotterdam Symptom Checklist
to (). The was originally developed as a tool
) enhance the insight into the consequences of the to measure the symptoms reported by cancer patients
disease and its treatment, participating in clinical research. It soon proved
) indicate groups of patients at risk for developing applicable to monitor the levels of the patients’
high levels of distress, and anxiety and depression and the presence of psycho-
) enable the comparison of the effectiveness of logical illness (Trew & Maguire, ).
different treatment modalities or care programs, The was constructed on the basis of secondary
and thus support decision making in clinical analyses of the data from three studies done with
oncology. different checklists (Pruyn et al., ): ) the
An instrument to assess quality of life must in the Hopkins Symptom Checklist, which was used in a
first place cover the relevant content: the concept to population of psychiatric patients, patients
be measured. Quality of life is generally considered with rheumatoid arthritis, and ‘normal’ controls
to be a multidimensional concept. In their review of (Luteijn et al., ), ) a symptom checklist used in
instruments, Moinpour and colleagues (Moinpour a study on the symptoms of breast cancer patients
et al., ) have suggested that the inclusion of a (Linssen et al., ), and ) a Dutch version of the
physical and a psychological domain is to be consid- Symptom Distress Scale developed by McCorkle
ered a minimal requirement for quality of life assess- and Young (McCorkle & Young, ) applied to a
ment in cancer patients. Cella and Tulsky () group of hospitalized cancer patients (Leendertse
have advocated a three dimensional approach, in et al., ). The selection of items from these
line with the WHO definition of health. In such an checklists was based on factor loadings, relevance
approach a physical dimension, covering the func- according to a group of experts in oncology, and the
tional status and physical symptom experience, an distribution of answers. This process resulted in an
emotional dimension and a social dimension are item list comprising physical and psychological
distinguished. De Haes and others (De Haes et al., symptoms. Patients were asked to indicate the degree
) have advocated the addition of an overall to which they had been bothered by the indicated
measure to assess quality of life as well. Such global symptoms over the past three days, on four-point,
valuation has been described as encompassing the Likert-type rating scales. Items referring to the
evaluations of specific dimensions or attributes in an activities of daily living were added to cover the
overall way. patients’ functional status. One item regarding the
In the context of clinical trials repeated measure- overall evaluation of life quality was added later. On
ment requires assessment forms which are easy to the basis of the early validation studies four items
handle for clinicians, nurses and datamanagers. were deleted (De Haes et al., ).
Moreover, compliance of patients, especially when
ill, will be enhanced if an instrument is easy to com- An instrument should have adequate psychometric
plete and short. properties. This means it should be reliable as well as
valid. As part of the validity of quality of life instru-
ments the sensitivity to changes occurring as a result
of clinical events, i.e. clinical validity, needs to be
established (Hays et al., ).
The was validated originally in a Dutch study 7
(De Haes et al., ) and since then has been used
in a number of Dutch and British investigations
(Hopwood, ; De Haes & Welvaart, ;
Fallowfield et al., ; Morris & Royle, ).
These also provided evidence for its ease of applica-
tion.
Further validation studies have substantiated the
reliability and validity of the in the Nether-
lands, Italy, the United Kingdom and Portugal (De
Haes et al., ; Paci et al., ; Watson et al.,
; Dos Santos et al., ). On the basis of its
feasibility and validity the use of the was advo-
cated in clinical research (Maguire & Selby, ) as
well as cost-effectiveness studies (Uyl-de Groot et
al., ).
8 2.1 Content of the RSCL been decided that concentration included earlier is
no longer part of the psychological distress subscale.
Content of scales and subscales The ‘concentration’ item has been excluded from
The is a self-report measure to assess the qual- the psychological distress subscale as it proved less
ity of life of cancer patients. The was designed clearly related to psychological distress. It turns out
to cover, originally, domains: physical symptom to be linked to physical distress, notably fatigue (see
distress, psychological distress, activity level and Table ). With concentration omitted from the
overall global life quality. These form the main psychological distress scale, none of the psychologi-
scales. The items regarding psychological distress cal symptoms then has an explicit physical compo-
have been interspersed among the items regarding nent (Plumb & Holland, ). Moreover, in Brit-
physical distress in order to avoid response sets in ish studies the version without the item concentra-
the first place. Also, the possibility of inducing dis- tion turned out to indicate clearly the absence or
tress by putting together all psychological symptoms presence of psychological morbidity 1.
has served as an argument. On the basis of factor
analyses presented in the current manual and earlier, The activity level scale consists of items regarding
more specific subscales within the physical symptom functional status. These items form a scale in which
distress experience can be constructed in some stud- mobility as well as social and role activities are cov-
ies (De Haes et al., ; De Haes et al., in prepara- ered. The scale is not related to cancer specifically.
tion). An overview of subject headings and number In some populations, like the elderly, the last item:
of items in the main scales is given in Table . More ‘go to work’ may not be relevant for any respondent.
specific subscales are referred to later (see chapter If so this item may have to be omitted from the
.). questionnaire. Also it must be noted that in some
populations, especially in breast and gynaecological
The items constituting the main scales in the cancers the item is not applicable for many women.
are given in Table . Two solutions are available: either substitution (see
par. .) or transformation over seven items.
The physical symptom distress scale consists of
items referring to different physical symptoms. The overall valuation of life (sometimes called either
Some symptoms such as headaches or fatigue, may global quality of life or overall quality of life) is
be experienced by people in general as well as by measured by one single item included in the check-
cancer patients. Other symptoms are more specifi- list. The item is derived from research in the quality
cally to related to cancer or cancer treatment: e.g of life of the population in general. An affective
gastro-intestinal and chemotherapy related symp- operationalisation was chosen for this item as such
toms. The items constituting the physical symptom approach proved, in line with theory, more sensitive
subscale are given in Table . to clinical change (De Haes et al., ).
act1: care for myself (wash etc.) all1: all things considered
act2: walk about the house
act3: light housework/household jobs
act4: climb stairs
act5: heavy housework/household jobs
act6: walk out of doors
act7: go shopping
act8: go to work
10 ent about answering the question (please note that ‘a Time frame
little’ is circled in the instruction, see Appendix). Originally, the was chosen to cover ‘the past
The overall valuation of life is assessed on a seven three days’. This time frame has been changed to
point Likert-type scale, as is usual in the literature ‘during the past week’. The decision to opt for
regarding the population at large from which the somewhat longer period was based on the wish to
question was derived. Answers range from ‘excellent’ cover more fully the presence of side effects while
to ‘extremely poor’. remaining within the time span easily remembered
by the patient. Moreover, although longer periods of
Experienced distress time have been found to result in a stronger relation
In quality of life questionnaires the experience of between symptom reporting and ‘complaining ten-
symptoms or complaints can be formulated either in dency’ or neuroticism, this tendency to report or
a (dys)function mode (e.g. is the patient consti- experience physical distress on the basis of personal-
pated?) or from the experienced burden perspective ity characteristics rather than on the basis of external
(e.g. to what extent is the patient bothered by con- influences is still less evident when asking questions
stipation). In the the level of symptom experi- over a week (Linssen et al., ).
ence is formulated in an experienced distress mode.
The respondent is asked to what extent they are
bothered by the symptom mentioned (see note ).
In the activity level scale a functional approach is
followed. Respondents are asked to what extent they
are able to perform activities.
Table 2
* When reversing the activity level and overall valuation of life scale for reasons of consistency the scoring will be
it may be used once at the beginning of treatment to version of the : 1) the questions referred to: “to what extent did you
test patients for being at risk for developing psycho- have” instead of “to what extent were you bothered by”. 2) In these studies
logical morbidity. Evidently, it could also be used at the version referring to a 3-day period was still used. 3) The item loneliness
other times during the course of disease and treat- was still included.
0 (not at all) to 3 (very much). Such approach implies that any deviation
3
Some caution is needed here. When items are related more closely, i.e. the
scale is more homogeneous, a higher score in one item goes along more
evidently with a higher score in another item. As a result, as some scales are
4
Evidently, authors using a scoring system of 0-3 for the individual items
5
The cut off point here would be 9 for a scoring system counting from 0-3.
3 Structure, reliability and validity
3.1 Description of samples ease-free ‘patients’ (n=), and % of the normal 15
controls (n=). The patients under treatment had
For a quality of life instrument to be useful in any a mean age of , the disease free patients of , the
kind of research its psychometric properties should comparison group of . Of the under treatment
be substantiated. Several studies have been carried sample .% was male, .% was female; of the
out to establish the psychometric qualities of the disease free patients .% was male; .% was
. In the overview given below, first the structure female, of the comparison population .% was
of the symptom distress list is described as to support male, .% was female.
the construction of scales and subscales, secondly the
reliability of the (sub)scales is described, thirdly the The ZEBRA study
validity of the instrument is discussed. In this trial pre-/peri-menopausal patients, aged less
Over the last years the has been used in numer- than years, with histologically confirmed stage II
ous studies in oncology. It would be impractical to node positive breast cancer were recruited. The
report on the data from all these here. We have patients originate from different countries and
therefore made a selection of three. The data from speaking seven different languages. Of the eligi-
these studies will be supplemented with data reported ble patients, (%) completed the first quality of
in the literature 6. life questionnaire, at baseline. Currently, these
patients were potentially available for the second
The original Dutch validation studies measurement. (%) questionnaires were avail-
The was validated initially in three study sam- able for analysis. patients completed both the
ples. The first consisted of a group of female patients first and the second questionnaire.
visiting an outpatient clinic for either follow up or For the cross cultural comparison clusters of lan-
chemotherapy administration. The patients (n=, guage/culture background have been formed. The
response rate %) were asked to fill in the ques- sample was divided in six subsamples with a mini-
tionnaire whilst waiting in the hospital, and later mum number of respondents of on the basis of
give it to their oncologist. the countries involved: ) an Eastern European sam-
The second sample consisted of a group of pa- ple involving Hungary, Czechia and Slovakia
tients participating in a randomized trial comparing (n=), ) an English speaking cluster involving
two chemotherapy regimens (Hexacaf and -) Australia, Ireland and the United Kingdom (n=),
for the treatment of advanced ovarian cancer. These ) the Finnish sample (n=), ) the French cluster
patients completed the questionnaire in the clinic involving France and Belgium (n=), ) the Ger-
several times in the course of the trial (mean number man sample (n=), and ) a ‘Latin’ cluster involv-
completed: . and ., respectively). ing Argentina, Portugal and Spain (n=).
In the third study a heterogeneous group of cancer Multivariate analyses of variance with the quality of
patients who had either been operated in the past life scales as dependent variables, were performed to
three months or were receiving chemotherapy, and a establish whether indeed the cluster of countries
group of cancer patients who had been without were similar. Significant differences were found
symptoms of the disease for three years or more, within the Latin cluster only. Patients from Spain
were compared with a random sample of the Dutch differed with respect to quality of life from the ones
population. Patients and controls were sent a letter from Argentina and Portugal but only at baseline.
inviting them to participate, a copy of the question- The results regarding this cluster have, therefore, to
naire, and a return envelope. The questionnaire was be considered with caution. Because the on-treat-
completed and returned by % of the patients ment measurement is especially relevant in the con-
currently under treatment (n=), % of the dis- text of this manual, the data from the second meas-
urement point are reported here.
16 The Fatigue study Original Dutch validation studies
In the Fatigue study two patient samples were ap- In the earlier report on the structure of the
proached. The Dutch sample involved patients in three different factor analyses were reported (De
their last week of radiation treatment (n=). Haes et al., ). In the first sample, the first factor
Patients were given a letter inviting them to partici- (explaining .% of the variance) referred to the
pate, a copy of the questionnaire and a return enve- experience of psychological distress. The highest
lope. They were asked to fill in the questionnaire loading on the second factor (explaining .% of the
within the next three days. The questionnaire was variance) had sore muscles and pain in the back.
returned by (%) of these patients. These symptoms, as well as headaches, referred to
The Scottish sample involved radiotherapy pa- the experience of pain. A number of symptoms was
tients. They were invited to fill in the questionnaire correlated with this factor less directly related to the
either in the waiting room in the hospital, or at experience of pain. The third factor (explaining
home within hours. patients (%) returned .% of the variance) referred to the experience of
the questionnaire. gastro-intestinal complaints: vomiting, nausea and
The Dutch sample filled in the Multidimensional lack of appetite loaded highly on this factor. On the
Fatigue Inventory () along with the . The fourth factor (explaining .% of the variance) fatigue
Scottish patients completed the , the and and lack of energy were important items. The pattern
the Hospital Anxiety and Depression scale (). emerging from this analysis is a structure of four
factors, more or less unambiguously concerned with
The SORK study relevant elements in the experience of the disease
In the project a cancer patient sample as well and treatment. The content of the first factor, psy-
as a control group were addressed: newly diag- chological distress, is the clearest, that of the other
nosed cancer patients (gynaecological, colorectal, factors, i.e. pain, gastro-intestinal complaints, and
lung and breast cancer) and ‘controls’ from the fatigue, is less distinct.
general population matched for age, sex and region In the principal component analysis in the third
of origin. Patients were interviewed by trained inter- study two factors had an Eigenvalue of . or higher.
viewers, but the was filled out as a self-report These explained .% of the variance. The first
measure. All subjects came from the North of the factor (explaining .% of the variance) referred
Netherlands. evidently to the experience of psychological distress.
Patients were interviewed three, nine and months All items that at face value would be considered
after diagnosis. Data from the first and third wave psychological were strongly related to this factor. The
are presented here. second factor (explaining .% of the variance) con-
cerns almost all of the physical symptoms in the
3.2 Structure of the RSCL symptom distress checklist. A number of symptoms were weakly related
scale to this factor (sore muscles, low back pain, abdomi-
nal aches).
When constructing the , the symptoms selected
to be inserted in the symptom distress scale were The ZEBRA study
gathered, as outlined in the Introduction, on the In the baseline measurement of the Zebra study, the
basis of secondary analyses of results from earlier two factor structure of symptom checklist gave the
studies and of interviews regarding relevance of the best interpretable solution. In this factor analysis the
items. Subsequent studies have investigated whether symptoms that were originally part of the first factor
the symptoms selected could be organised into dif- were so again: anxiety, tension, nervousness, worry-
ferent scales reflecting more specific quality of life ing, depression, despairing about the future, irritabil-
domains. These results will now be reported. ity and concentration problems. Three other symp-
toms loaded also, although less strongly, on this first factor structure was not replicated. As a consequence 17
factor. All other physical symptoms had higher factor it was decided to do a factor analysis in which the
loadings on the second factor. The earlier two factor number of factors was set free. On the basis of the
solution was thus, largely, replicated. plot of eigenvalues a three and a five factor solution
For the second measurement, where patients com- were inspected. The five factor solution gave the best
pleted the questionnaire during chemotherapy, the interpretable results. These are given in Table .
Table 3
1 2 3 4 5
anxiety .79 - - - -
tension .74 .29 - - -
worrying .73 - - - -
nervousness .72 - - - -
despairing about the future .69 - - - -
depressed mood .65 - - - -
irritability .64 - - - -
decreased sexual interest .46 - - .29 -
lack of energy .30 .65 - - -
tiredness - .63 - - -
shortness of breath - .58 - - -
difficulty concentrating .44 .54 - - -
difficulty sleeping .30 .50 - - -
dry mouth - .44 - - .31
nausea - - .78 - .29
vomiting - - .74 - -
lack of appetite - .32 .64 - -
dizziness .33 - .44 - -
diarrhoea - .26 .33 - -
low back pain - - - .71 -
sore muscles - .25 - .62 -
constipation - - - .58 .27
abdominal aches - - .33 .52 -
shivering - - - .43 -
headaches - .35 .29 .35 -
loss of hair - - - - .65
sore mouth/pain when swallowing - .25 - - .63
heartburn/belching - .28 .30 - .46
burning/sore eyes - - .34 - .44
tingling hands or feet - - - .31 .40
Fatigue study
Scottish sample .83 .90 .863
Dutch sample .87 .88 .923
As can be seen from Table the reliability of the Reliability is consistently good for the psychological
physical symptom distress subscale is good consist- distress subscale also. In many of the samples studied
ently over the samples involved. Based on the results the reliability is good enough to warrant the use of
of the factor analysis in the second sample of the the instrument for screening purposes.
original validation study and the results from the The scale regarding the activity level has a varying
Zebra study specific physical distress subscales were internal consistency. Inspection of the data of the
defined. These pertain to fatigue, pain related symp- Zebra study reveals that this is weak when the
toms, gastro-intestinal symptoms and chemotherapy amount of variance on some items is low in a sam-
related symptoms. In the first study the pain, fatigue ple. No patients in these samples experience impair-
and gastro-intestinal symptom subscale have ad- ment in the lower levels of activity dysfunction, such
equate reliabilities (., . and . respectively). In as self care. Where variance is sufficient, the reliabil-
the Zebra study the reliabilities of these subscales is ity of the activity subscale is satisfactory to good.
less good (.; . and . respectively) 7. As in fact the activity level scale is built in a hierar-
20 chical way we finally computed the reliability using Construct validity
the Mokken model in the Fatigue study. This model Whether an instrument is measuring what it is sup-
is used to establish whether a hierarchical approach posed to measure can be established in different
would fit the data. The analysis is used to establish ways. To established construct validity hypotheses
whether indeed the items can be seen as a scale formulated on theoretical grounds are tested. The
where the level of function in one item predicts the scores obtained with the instrument have to behave
level of function in another although the latter may in a predicted manner to support its validity. In this
be systematically higher or lower. Violations of the paragraph predictions are based on the fact that the
assumption of hierarchy are given in the analysis. scale is supposed to be related with criteria )
For the Dutch as well as the Scottish data the hierar- measuring the same construct, ) measuring a re-
chical structure of the activity level scale was con- lated construct or ) measuring an attribute that is
firmed. No major violations were found. The reli- known to be related to the scale 9.
ability of the scale was good for both samples 8.
The hierarchical order of the activity scale items was Physical symptom distress was related to the Medi-
equivalent in both samples: cal Outcome Study instrument (-) that meas-
walk about the house ures the patient health state generically (Stewart et
care for myself (wash etc.) al., ) 10. The results are given in Table . Physical
walk out of doors symptom distress turned to correlate strongly to
climb stairs physical and role function and health perception (all
light housework/household jobs r’s >.). It is related less strongly to mental health.
go shopping It is unrelated to a one item pain scale.
heavy housework/household jobs
go to work As on the scale level on the individual item level
relations can be studied. In the Fatigue study scores
To conclude on the tiredness item discriminated significantly
It can be concluded that the reliability of the physi- between patients scoring high and low on the Multi-
cal distress scale is good. The reliability of the psy- dimensional Fatigue Inventory ().
chological distress scale is good and excellent in a
number of samples. The use of that scale as a screen- The psychological distress scale has been related to
ing instrument is therefore acceptable. The activity several other instruments assessing psychological
level scale has a good reliability in most studies: the distress or morbidity. Watson and others (Watson et
internal consistency as well as the hierarchical struc- al., ) found the psychological distress scale to be
ture was proven good to excellent. In some samples, highly related with anxiety (r=.) as well as depres-
as a result of a lack of variance in the samples in- sion (r=.) as measured with the . The scale
volved, reliability could not be established. was also related to the Psychosocial Adjustment to
Illness Scale (r=.). Similarly, Paci (Paci, )
3.4 Validity correlated the psychological distress scale to
Spielberger’s Stait Trait Anxiety scale () and
For an instrument to be considered valid it must be found a strong relationship (r=.). In the Scottish
established whether indeed it is measuring what it is sample of the Fatigue Study the correlation between
supposed to measure. The main data available from the psychological distress scale and the
the studies mentioned and from the literature re- was . for anxiety and . for depression. In a
ported are given in the next overview. Dutch sample from the normal population (n=)
the relation with the - depression score was
Table 5 21
RSCL
Physical Psychological
Symptom Symptom
MOS SF-20
Physical Function -.67 ** -.40 **
Role Function -.61 ** -.37 *
Social Function -.58 ** -.33 *
Mental Health -.53 ** -.78 **
Health Perception -.62 ** -.60 **
Pain .09 .16
., in a sample of cancer patients (i.e. the - The item concerning overall evaluation of quality
study) it was . (Bouma et al., ). In testicular of life (‘How was your quality of life over the past
cancer papients the relation with the - men- week?’) has proven, as expected, to be related to
tal health scale was -. (see Table ) (Kiebert, ). individual domains in the quality of life of cancer
In one of the original Dutch validation studies the patients under treatment, notably the evaluation of
psychological distress scale was related to per- physical function (r=. and r=.), the evaluation of
sonality characteristics (De Haes, ). The scale psychological function (r=. and .), the evalua-
scores turned out to be strongly correlated with tion of activities (r=.) and the evaluation of social
neuroticism (r=.) and self esteem (r=.) in disease function (r=.). Similarly, the overall valuation of
free patients as well as cancer patients after treat- life question proved to be related to individual do-
ment (r=. and -. respectively). mains in the quality of life of disease free cancer
patients: the evaluation of physical function (r=.),
The activity level scale has turned out to be related the evaluation of psychological function (r=.), the
to age as expected in a sample from the original evaluation of activities (r=.) and the evaluation of
Dutch validation study. In the Fatigue study the social function (r=.). As in the general population
activity level scale was related to fatigue subscales as it was related to personality characteristics as well. In
expected: in the Scottish data the correlation with patients under treatment neuroticism correlated
the physical fatigue subscales was ., . and .. As with overall valuation of life (r=.-. and r=-.) as
expected the correlation between activity level and well as self esteem (r=. and r=.). The same was
the motivational and cognitive subscales of the true in disease free cancer patients (r=-. and .
was weaker (r=. and r=.). respectively) (De Haes, ).
22 Clinical validity others (Richards et al., ) found cancer-related
An instrument meant to measure the quality of life symptoms, such as pain and breathlessness, decreased
of patients must not only have construct validity, over the course of treatment while treatment-related
but must also be able to distinguish between clinical symptoms, such as vomiting and diarrhoea, increased
characteristics of patients and reflect change in situa- at weeks and reverted to pretreatment levels after
tions which influence their quality of life 11. In this completion of chemotherapy.
paragraph clinical validity or responsiveness is sub-
stantiated for the different scales. The psychological distress subscale has in the first
place, as reported earlier, proved sensitive to clinical
The physical symptom distress as measured with morbidity as assessed by psychiatric interview in two
the has been found to be responsive to clinical studies (Hopwood et al., ; Ibbotson et al., ).
characteristics. In the first original Dutch validation Secondly, the scale was related to clinical events as
study chemotherapy patients receiving cisplatinum expected in a number of studies. In the original
turned out to experience more gastro-intestinal Dutch validation studies psychological distress level
symptoms as well as hair loss as expected. In the has proved to be negatively related to a worse prog-
second Dutch study reported on here, the physical nosis, receiving chemotherapy, and the expected
symptom distress level was higher in patients receiv- amount burden of the chemotherapy regimen.
ing chemotherapy and in patients being operated In the Zebra study as in the study by Jones and
upon for a malignancy compared to the general popu- Coleman (Jones & Coleman, ) the level of psy-
lation and was worse for patients with a poorer prog- chological distress was lower after the initiation of
nosis. Patients undergoing chemotherapy reported hormonal or chemo-therapy. In the Jones and
more physical symptom distress than surgically Coleman study the patients level of psychological
treated patients and among surgically treated patients distress, interestingly, deteriorated over the second
physical symptom distress varied with time since the chemotherapy cycle. Soukop and others (Soukop et
operation (De Haes, ). al., ) found patients who received anti-emetic to
In the Zebra study patients were experiencing a experience less psychological distress. Richards and
higher level of physical symptom distress when they co-authors (Richards et al., ) found the psycho-
underwent adjuvant chemotherapy in all seven cul- logical distress scale to differentiate between patients
tural groups distinguished. who received chemotherapy every weeks and those
The Fatigue study can not be used to establish clini- who were treated weekly.
cal validity as no clinical features were distinguished.
In other studies reported in the literature the clinical The daily activity scale differentiated between can-
validity of the physical symptom distress meas- cer patients. Patients attending their general practi-
urement has been substantiated. Watson and others tioner and a general population sample in the original
(Watson et al., ) found the scale to be sensitive Dutch validation study (De Haes, ). In the Zebra
to the impact of chemotherapy: patients with disease study, unexpectedly, the activity level of patients
on chemotherapy had significantly higher mean turned out to be higher after the initiation of
scores for physical distress scale compared to the therapy.
disease free group. This difference was greatest for
the scale containing gastro-intestinal symptoms. The overall valuation of life-scale proved sensitive
Soukop and colleagues (Soukop et al., ) found to a number of clinical parameters in the Dutch
vomiting to be less prevalent, as expected, in patients validation study. Both surgery and chemotherapy
receiving anti-emetic treatment. Among patients patients reported an impaired overall valuation of
receiving palliative chemotherapy Richards and life as compared to the general population. Also
patients with the worst prognosis had the lowest 6
It may be noted that in the original validation study a 27 item version of 23
quality of life. After surgery the overall valuation of the symptom checklist was used. In the Zebra and the fatigue study the
life was worse one month after surgery, when there now accepted version including 30 symptoms has been used.
where clinical problems (e.g. infections) post-sur-
gery. Chemotherapy patients reported a worse over- 7
It should be noted that reliabilities depend on the population under study
all valuation when the expected burden of chemo- (De Haes & Welvaart, 1985). Therefore, the reliability of subscales has to
therapy was intermediate. be computed anew for new studies before these scales are used again.
process.
The psychological scale differentiates between cases 11
A problem in establishing clinical validity is that it is not always sure that
and non-cases. Its relation with disease and treat- indeed the clinical characteristic or situation influences the quality of life as
ment states has to be clarified further on theoretical expected. Therefore absence of the finding may either mean that the
grounds. The activity level scale differentiates where instrument is not clinically valid or that the hypothesis has be formulated
expected. The overall valuation is less sensitive, as erroneously. As will be seen the relation between especially psychological
expected, but still differentiating. distress and clinical characteristics is not clear yet.
4 Normative data
Table 6
Zebra study
II. Early breast cancer, receiving adjuvant treatment, hormonal or chemotherapy (n = 478)
I. Lung cancer (n=127). Small cell lung cancer, prior to chemotherapy (MRC LU16)
II. Bladder cancer (n=157). Advanced disease bladder cancer patients, before treatment (MRC BA09)
III. Head and neck cancer, prior to radiotherapy (n=274) (MRC CH01)
I. Various tumour sites, newly diagnosed cancer patients (3 months after diagnosis) n=400
III. Various tumour sites, newly diagnosed cancer patients (15 months after diagnosis) n=400
28 Anderson H, Hopwood P, Prendiville J, Radford JA, De Haes JCJM, Van Knippenberg FCE & Neijt JP.
Thatcher N & Ashcroft L. (). A Randomized (). Measuring psychological and physical dis-
Study of Bolus vs Continuous Pump Infusion of tress in cancer patients: structure and application of
Ifosfamide and Doxorubicin with Oral Etoposide the Rotterdam Symptom Checklist. Br J Cancer, ,
for Small-Cell Lung-Cancer. Br J Cancer, , - -.
.
De Haes JCJM, Van Oostrom MA & Welvaart K.
Ballinger AB, McHugh M, Catnach SM, Alstead (). The effect of radical and conserving surgery
EM & Clark ML. (). Symptom Relief and on the quality of life of early breast cancer patients.
Quality-of-Life After Stenting for Malignant Bile- Eur J Surg Oncol, , -.
Duct Obstruction. , , -.
De Haes JCJM, Welvaart K. (). Quality of Life
Clavel M, Soukop M & Greenstreet YLA. (). after breast cancer surgery. J Surg Oncol, , -.
Improved Control of Emesis and Quality-of-Life
with Ondansetron in Breast-Cancer. Oncology, , De Ruiter JH, De Haes JCJM & Tempelaar R.
-. (). Cancer-Patients and Their Network - The
Meaning of the Social Network and Social Interac-
Cull A, Cowie VJ, Farquharson DI, Livingstone JR, tions for Quality-of-Life. Supp Care Cancer, , -
Smart GE & Elton RA. (). Early stage cervical .
cancer: psychosocial and sexual outcomes of treat-
ment. Br J Cancer, , -. Dobbs NA, Twelves CJ, Ramirez AJ, Towlson KE,
Gregory WM & Richards MA. (). Patient Ac-
De Haes JCJM, De Ruiter JH, Tempelaar R & ceptability and Practical Implications of
Pennink BJW. (). The distinction between Pharmacokinetic Studies in Patients with Advanced
affect and cognition in the quality of life of cancer Cancer. Eur J Cancer, A, -.
patients - sensitivity and stability. Qual Life Res, ,
-. Dos Santos MJH, Da Costa FL, Watson M, Greer
S, De Haes J, Van Knippenberg F, Pruyn J & Van
De Haes JCJM, Pennink BJW & Welvaart K. den Borne B. (). Adaptaçao psicológica e
(). The distinction between affect and cogni- qualidade de vida em doentes oncológicos - Escalas
tion. Soc Indic Res, , -. de avaliao. Psiquiatria Clinica, , -.
De Haes JCJM, Pruyn JFA & Van Knippenberg Fallowfield LJ, Baum B & Maguire GP. ().
FCE. (). Klachtenlijst voor kankerpatienten. Addressing the psychological needs of the conserva-
Eerste ervaringen. Ned Tijdschr Psychol, , - tively treated breast cancer patients: discussion pa-
. per. J Royal Soc Med, , -.
De Haes JCJM, Raatgever JW, Van der Burg MEL, Fallowfield LJ, Hall A, Maguire GP & Baum M.
Hamersma E & Neijt JP. (). Evaluation of the (). Psychological outcomes of different treat-
Quality of Life of patients with advanced ovarian ment policies in women with early breast cancer
cancer treated with combination chemotherapy. In: outside a clinical trial [see comments]. BMJ,
Aaronson NK and Beckman J (eds), The Quality of (), -.
Life of cancer patients. Raven Press, New York, -
.
Fallowfield LJ & Hall A. (). Psychosocial and Kiebert GM, De Haes JCJM, Kievit J & Van de 29
sexual impact of diagnosis and treatment of breast Velde CJH. (). The effects of perioperative
cancer. Br Med Bull, , -. chemotherapy on the quality of life of patients with
early breast cancer. Eur J Cancer, , -.
Findlay M, Cunningham D, Norman A, Mansi J,
Nicolson M, Hickish T, Nicolson V, Nash A, Kiebert GM, Welvaart K & Kievit J. (). Psycho-
Sacks N, Ford H, Carter R & Hill A. (). A logical Effects of Routine Follow-Up on Cancer-
Phase-II Study in Advanced Gastroesophageal Can- Patients After Surgery. Eur J Surg, , -.
cer Using Epirubicin and Cisplatin in Combination
with Continuous-Infusion -Fluorouracil (ECF). Morris J & Ingham R. (). Choice of surgery for
Ann Oncol, , -. early breast cancer: psychosocial considerations. Soc
Sci Med, , -.
Greer S, Moorey S, Baruch JD, Watson M,
Robertson BM, Mason A, Rowden L, Law MG & Morris J. (). Widening Perspectives - Quality-
Bliss JM. (). Adjuvant psychological therapy for of-Life as a Measure of Outcome in the Treatment
patients with cancer: a prospective randomised trial of Patients with Cancers of the Head and Neck.
[see comments]. , (), -. Oral Oncol-Eur J Cancer part B, B, -.
Hopwood P, Howell A & Maguire P. (). Psychi- Neises M, Sir MS, Strittmatter HJ, Wischnik A &
atric morbidity in patients with advanced cancer of Melchert F. (). Influence of Age and of Differ-
the breast: prevalence measured by two self-rating ent Operative Methods on the Quality-of-Life in
questionnaires. Br J Cancer, (), -. Patients with Breast-Cancer. Onkol, , -.
Hopwood P, Howell A & Maguire P. (). Paci E. (). Assessment of validity and clinical
Screening for psychiatric morbidity in patients with application of an Italian version of the Rotterdam
advanced breast cancer: validation of two self-report Symptom Checklist. Qual Life Res, , -.
questionnaires. Br J Cancer, (), -.
Purohit OP, Anthony C, Radstone CR, Owen J &
Hopwood P & Thatcher N. (). Preliminary Coleman RE. (). High-Dose Intravenous
experience with Quality of Life evaluation in pa- Pamidronate for Metastatic Bone Pain. Br J Cancer,
tients with lung cancer. Oncol, , -. , -.
Ibbotson T, Maguire P, Selby P, Priestman T & Richards MA, Hopwood P, Ramirez AJ, Twelves
Wallace L. (). Screening for Anxiety and De- CJ, Ferguson J, Gregory WM, Swindell R, Scrivener
pression in Cancer-Patients - The Effects of Disease W, Miller J, Howell A, et al. (). Doxorubicin in
and Treatment. Eur J Cancer, A, -. advanced breast cancer: influence of schedule on
response, survival and quality of life. Eur J Cancer,
Jones AT & Coleman RE. (). Influence of com- A (-), -.
bination antiemetic therapy on the quality of life of
patients receiving chemotherapy. J Cancer Care, , Soukop M, McQuade B, Hunter E, Stewart A, Kaye
-. S, Cassidy J, Kerr D, Khanna S, Smyth J, Coleman
R, et al. (). Ondansetron compared with
metoclopramide in the control of emesis and quality
of life during repeated chemotherapy for breast
cancer. Oncology, (), -.
30 Twelves CJ, Dobbs NA, Lawrence MA, Ramirez AJ,
Summerhayes M, Richards MA, Towlson KE &
Rubens RD. (). Iododoxorubicin in Advanced
Breast-Cancer - A Phase-II Evaluation of Clinical
Activity, Pharmacology and Quality-of-Life. Br J
Cancer, , -.
Bouma J, Ranchor AV, Sanderman R & Van Hays RD & Hadorn D. (). Responsiveness to 31
Sonderen E. (). Het meten van symptomen van change: an aspect of validity, not a separate dimen-
depressie met de CES-D. Noordelijk Centrum voor sion. Qual Life Res, , -.
Gezondheidsvraagstukken, Rijksuniversiteit
Groningen. Hopwood P. (). Measurement of psychological
morbidity in advanced breast cancer. In: Psychoso-
Cella DF & Tulsky DS. (). Measuring the cial Issues in Malignant Disease, Watson M, Greer
quality of life today: methodological aspects. Oncol, S. (eds). Pergamon Press, Oxford.
-.
Hopwood P, Howell A & Maguire P. ().
De Haes JCJM, Pruyn JFA & Van Knippenberg Screening for psychiatric morbidity in patients with
FCE. (). Klachtenlijst voor kankerpatiënten, advances breast cancer: validation of two self report
eerste ervaringen. Ned Tijdschr Psychol, , - questionnaires. Brit J Cancer, , -.
.
Hopwood P, Stejphens RJ & Machin D. ().
De Haes JCJM & Welvaart K. (). Quality of life Approaches to the analysis of quality of life data:
after breast cancer surgery. J Surg Oncol, , - experiences gained from a Medical Research Council
. Lung Cancer Working Party Palliative Chemo-
therapy Trial. Qual Life Res, , -.
De Haes JCJM. (). Kwaliteit van leven van
kankerpatienten. Swets & Zeitlinger, Lisse. Ibbotson T, Maguire P, Selby P, Priestman T &
Wallace L. (). Screening for anxiety and depres-
De Haes JCJM, Van Knippenberg FCE & Neijt JP. sion in cancer patients - The effects of disease and
(). Measuring psychological and physical dis- treatment. Eur J Cancer, , -.
tress in cancer patients: structure and application of
the Rotterdam Symptom Checklist, Brit J Cancer, Jones AT & Coleman RE. (). Influence of com-
, -. bination antiemetic therapy on the quality of life of
patients receiving chemotherapy. J Cancer Care, ,
De Haes JCJM, De Ruiter JH, Tempelaar R & -.
Pennink BJW. (). The distinction between
affect and cognition in the quality of life of cancer Kiebert G. (). Choices in Oncology. Thesis
patients - sensitivity and stability. Qual Life Res, , University Leiden, The Netherlands. (in press)
-.
Leendertse F, Van Rockel G, Sparreboom T &
Dos Santos MJH, Da Costa FL, Watson M, Greer Zuidema M. (). Slaapstoornissen bij
S, De Haes JCJM, Van Knippenberg F, Pruyn J & kankerpatiënten in een ziekenhuis. Instituut voor
Van den Borne B. (). Adaptaçao psicológica e Sociale en Preventieve Psychiatrie, Rotterdam.
qualidade de vida em doentes oncológicos - Escalas
de avaliaçao. Psiquiatria Clinica, , -. Linssen ACG, Hanewald GJFP, Huisman S & Van
Dam FSAM. (). The development of a well-
Fallowfield LJ, Baum M & Maguire GP. (). being (quality of life) questionnaire at the Nether-
Effects of breast conservation on psychological mor- lands Cancer Institute. Proc Third Workshop
bidity associated with diagnosis and treatment of EORTC Study group on Quality of Life, Paris :
early breast cancer. Brit Med J, , -. -.
32 Linssen ACG, Van Dam FSAM, Engelsman E, Van Soukop M, McQuade B, Hunter E, Stewart A, Kaye
Benthem J & Hanewald GJFP. (). Leven met S, Cassidy J, Kerr D, Khanna S, Smyth J, Coleman
cytostatica. Pharmaceut Weekbl, , -. R. et al. (). Ondansetron compared with
metoclopramide in the control of emesis and quality
Luteijn F, Kok AR, Hamel LF & Poiesz A. (). of life during repeated chemotherapy for breast
Enige ervaringen met een klachtenlijst. Ned cancer. Oncol, , -.
Tijdschr Psychol, , -.
Sprangers MAG & Aaronson NK. (). The role
Maguire P & Selby P. (). Assessing quality of of health care providers and significant others in
life in cancer patients. Brit J Cancer, , -. evaluating the quality of life of patients with chronic
disease: a review. J Clin Epidemiol, , -.
McCorkle R & Young K. (). Development of a
symptom distress scale. Cancer Nurs, -. Stewart AL, Hays RD & Ware JE. (). The MOS
Short-form General Health Survey. Med Care, ,
Moinpour CM, Feigl P, Metch B, Hayden KA, -.
Meyskens FL & Crowley J. (). Quality of life
endpoints in cancer clinical trials: review and recom- Trew M & Maguire P. (). Further Comparison
mendations. J Nat Cancer Inst, (), -. of two instruments for measuring quality of life in
cancer patients. In: Quality of life, Beckmann J. (ed)
Morris J & Royle G. (). Choice of surgery for Proc Third workshop of the EORTC Study group
early breast cancer: psychosocial considerations. Soc on Quality of Life, Paris : -.
Sci Med, (), -.
Uyl-de Groot CA, Rutten FFH & Bonsel GJ.
Nunnally JC. (). Psychometric Theory. New (). Measurement and valuation of Quality of
York, McGraw Hill. Life in economic appraisal of cancer treatment. Eur
J Cancer, A, -.
Paci E. (). Assessment of validity and clinical
application of an Italian version of the Rotterdam Watson M, Law M, Maguire P, Robertson B, Greer
Symptom Checklist. Qual Life Res, I, -. S, Bliss JM & Ibbotson T. (). Further develop-
ment of a quality of life measure for cancer patients:
Plumb MM & Holland J. (). Comparative the Rotterdam Symptom Checklist (revised). Psy-
studies of psychological function in patients with cho-Oncol, , -.
advanced cancer-I: self reported depressive symp-
toms. Psychosom Med, (), -. Zwinderman AH. (). Statistical analysis of lon-
gitudinal quality of life data with missing measure-
Pruyn JFA, Van den Heuvel WJA & Jonkers R. ments, Qual Life Res, , -.
(). Verantwoording van de klachtenlijst voor
kankerpatiënten. Studiecentrum Sociale Oncologie,
Rotterdam.
• Dutch
• English
• German
• Portugese ( versions)
• French
• Hungarian
• Spanish
• Tjecho-Slovene
• Finnish
• Servic
Hieronder vragen wij u in hoeverre u last heeft van een aantal klachten. Het gaat
er steeds om hoe u zich de afgelopen week voelde. Wilt u een cirkel zetten om
het antwoord dat het meest op u van toepassing is.
pijn onder in de rug helemaal niet een beetje nogal heel erg
○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○
wanhopig zijn over de toekomst helemaal niet een beetje nogal heel erg
○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○
verminderde sexuele belangstelling helemaal niet een beetje nogal heel erg
○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○
36 tintelingen in handen of voeten helemaal niet een beetje nogal heel erg
○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○
je moeilijk kunnen concentreren helemaal niet een beetje nogal heel erg
○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○
een droge mond helemaal niet een beetje nogal heel erg
○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○
Hieronder staat een aantal activiteiten. Het is niet van belang of U deze
activiteiten ook werkelijk doet, maar of U ze kunt uitvoeren in Uw huidige
lichamelijke toestand. Het is de bedoeling dat U telkens die omschrijving
aankruist, die het beste past bij Uw situatie van de afgelopen week.
lopen binnenshuis O O O O
○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○
traplopen O O O O
○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○
lopen buitenshuis O O O O
○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○
boodschappen doen O O O O
○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○
In this questionnaire you will be asked about your symptoms. Would you please, for all
symptoms mentioned, indicate to what extent you have been bothered by it, by circling
the answer most applicable to you. The questions are related to the past week.
low back pain not at all a little quite a bit very much
○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○
despairing about the future not at all a little quite a bit very much
○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○
decreased sexual interest not at all a little quite a bit very much
○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○
abdominal (stomach) aches not at all a little quite a bit very much
○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○
38 tingling hands or feet not at all a little quite a bit very much
○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○
sore mouth/pain when swallowing not at all a little quite a bit very much
○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○
climb stairs O O O O
○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○
go shopping O O O O
○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○
go to work O O O O