ELSEVIER Patient Education and Counseling 24 (1994) 341-345

Development and evaluation of an inventory for rating client

satisfaction with outcome in HIV counseling: the Albion Center
scale

Kim Begley”, Michael W. Rossb *, Patricia Austina, Kathleen Casey”,

Patricia Collins”, Grant Hennings”, Larry Agriestia, Keith Marshall’
aAlbion Street(AIDS) Center, Division of Medicine, Prince of Wales Hospital, Surry Hills. NSW 2010. Sydney, Australia
bCenter for Health Promotion Research and Development, School of Public Health, University of Texas, PO Bo.x 20186. Houston.
TX 77225, USA
‘AIDS Council of New South Wales, Surry Hills, NSW 2010, Sydney. Australia

Received 25 September 1993; accepted 23 March 1994

Abstract

We developed an inventory for rating client satisfaction with outcome in HIV counseling, based on interview
responses with clients. The resulting 19-item scale was subject to factor analysis and four factors, accounting for 56%
of the variance, emerged. The factors described dimensions of Perception of progress and improved mood; Recognition
of a specific need for counseling; Behavior change from counseling; and Counseling climate. Factor-scored scales were
significantly associated with time in counseling and for the Specific need for counseling scale, HIV-seropositive
respondents had a significantly higher score. Scale reliabilities (Cronbach’s alpha) were between 0.85 and 0.50. Concur-
rent administration with the Counseling Evaluation Inventory indicated that there were significant correlations be-
tween the two scales.

Keywords: HIV; AIDS; Counseling; Evaluation; Client satisfaction; Measurement

-

1. Introduction HIV counseling content. A number of significant

With the recognition that HIV disease and
AIDS and the issues surrounding HIV infection
are highly psychologically charged, and the
stigmatization associated with HIV issues, there
has been considerable focus in the literature on
* Corresponding author.
publications provide guides to clinical issues and
counseling in HIV/AIDS [l-5], but there is little
evaluation of the satisfaction of clients with coun-
seling. This is despite the emphasis on the impor-
tance of counseling before HIV testing as well as
post-test counseling and the very widespread pro-
vision of such counseling in the western world, and
the recognition of the extensive psychological con-

0738-3991/94/%07.00 0 1994 Elsevier Science Ireland Ltd. All rights reserved

SSDI 0738-3991(94)00645-3
342 K. Begley et al. /Patient
comitants of HIV-related concerns [6]. What data
are available on the impact of interventions sug-
gest that, using standardized distress measures, de-
pression, anxiety and psychological symptoms in
HIV-seropositive clients decrease after stress
reduction training, and do not increase following
standard counseling protocols and informational
interventions [7], while in HIV-seronegative clients
there are equal decreases in these measures follow-
ing each intervention. A similar cognitive-
behavioral stress management intervention show-
ed significantly lower depression after notification
of positive HIV serostatus compared with con-
trols, suggesting that stress reduction has a buffer-
ing effect [I?]. While such studies document the
efficacy of counselling in terms of changes in mood
states, evaluation of client satisfaction has not
been described with regard to HIV counseling. It
must be recognized that satisfaction and mood
changes are not necessarily related issues: im-
provement in mood may occur without satisfac-
tion, and vice versa. We emphasise satisfaction
with counseling is an evaluation of the counseling
service and climate as much as it is with outcome.
Satisfaction with HIV-related services is frequently
an essential part of program evaluation, and the
need for a measure of client satisfaction for such
evaluation was the basis for the present research.
There are a number of existing indices of client
satisfaction which are generic rather than directed
toward specific areas of counseling, such as the
Counseling Evaluation Inventory (CEI) [9, lo],
and the Counselor Rating Form and the
Counselor Effectiveness Rating Scale. Review of
these instruments [ 1l] emphasizes the centrality of
evaluation of counselor effectiveness to counseling
practice and research, and the importance of hav-
ing psychometrically reliable and valid measures.
However, very little has been done in the area of
effectively evaluating counseling in the HIV/AIDS
area, although in a pilot study of 20 clients, Broad-
bent [12] found that counseling did help clients to
cope better with the medical and social aspects of
HIV disease.
In evaluation of the effectiveness of HIV coun-
seling services, specific issues such as perceived
mood and behavior changes associated with
HIV/AIDS counseling of those with both positive
Educ. Couns. 24 (1994) 341-345
and negative HIV status, as well as the more gener-
al ones such as counseling climate and client satis-
faction must be addressed. However, the issues
associated with HIV-related counseling satisfac-
tion did not, based on reading conventional in-
dices, relate particularly closely to the issues which
our clients, on the basis of exit interviews, sug-
gested were related to their satisfaction with the
service. These issues included support from coun-
seling, perception of improvement in coping and
mood, behavior change, and specific aspects of the
clinic and counseling climate. Traditional mea-
sures such as the CEI are based on scales measur-
ing counseling climate, counselor comfort, and
client satisfaction. We determined to develop a
client satisfaction scale to evaluate HIV counseling
which was both reliable and valid, and which con-
tained scales which were specific to client-
expressed HIV-related satisfaction issues as well as
more general questions relating to client satisfac-
tion as demonstrated by the CEI.
2. Method
Questions were generated by examining the ob-
jectives of HIV counseling and developing ques-
tions which would tap these. All psychologists at
the Albion (AIDS) Center and AIDS Council of
New South Wales provided questions and 19 items
were selected from the comments of client’s exit in-
terviews for the questionnaire. All were rated on a
5-point Likert scale (strongly agree, agree, uncer-
tain, disagree, strongly disagree). Demographic
and other data collected included age, gender, HIV
status, highest educational level, and months in
HIV counseling. In addition, the CEI [9,10] was
administered to all clients who had been in coun-
seling for a minimum of four sessions. Responses
were placed in a plain envelope and then in a box
at the clinic reception to ensure anonymity. The
study was approved by the relevant ethics com-
mittee,
The sample consisted of 62 clients at the Albion
(AIDS) Center, 52 men and 10 women, mean age
35.0, median 33.0, S.D. 7.0. Over one-third
(33.9%) had completed tertiary education, with
only 21.0% not having completed high school.
Mean number of months of counseling (with
 K. Begley et al. /Patient Educ. Cauns. 24 (1994) 341-345 343

weekly appointments) was 12.7, median 7.0, SD. 3. Results

14.7 (range, l-78). All therapists were licensed
psychologists.
Data analysis was by factor analysis (principal
components analysis followed by varimax rota-
tion), and the factors were factor-scored by
multiplying the factor loading by the item score
and adding the products for each factor. Com-
parisons of factors were carried out using Pearson
product-moment correlation coefftcients for con-
tinuous data, and t-test for comparison of groups
on factors. Reliability was calculated using Cron-
bath’s alpha.
Factor analysis of the Albion Center Scale
(ACS) produced seven factors with eigenvalues
> 1. However, several of these factors were
monofactors or had high loadings on more than
one factor, and when the criteria of Walkey and
McCormick [ 131 were applied, a four-factor solu-
tion appeared to provide both the most inter-
pretable factors and account for the highest
amount of variance (56.10/o). The principal com-
ponents solution was thus rotated to four factors,
which encompassed the domains of Progress and

Table I
Factor structure of ACS questionnaire

Item Loading Percentage

positive”

Factor I: Progress and improved mood

I 1 have made progress with the issues that brought me in to counseling 0.87 95
5 The depression 1 experienced before entering counseling has been reduced by the sessions 0.83 94
1I I am satisfied with the progress I have made so far in counseling 0.76 94
6 I feel more satisfied with myself as a result of counseling 0.69 x9
I4 My level of anxiety has not been reduced by counseling -0.59 9
I9 I would recommend this counseling service to others 0.56 100
2 I feel satisfied with the counseling services I receive at this center 0.54 100
(26.2% of variance)

Factor 2: Specific need for counseling

9 I feel the support I receive from friends is as beneficial as the counseling
I receive from this
center -0.82 I6
16 If I did not have counseling available I would not be able to discuss my fears and concerns as
easily with anyone else 0.72 74
IO There are things I can only discuss with my counselor and no one else 0.67 71
8 I believe my medical practitioner has the ability to provide the counseling I receive from my
counselor -0.52 2
(12.0% of variance)

Factor 3: Behavior change from counseling

12 Counseling has helped me maintain safer sex practices 0.77 44
I3 Counseling has helped me seek support from other places and people 0.74 77
I5 I feel more hopeful as a result of counseling 0.52 92
(9.9% of variance)

Factor 4: Counseling climate

4 The staff keep me waiting when 1 attend for an appointment 0.70 7
17 I feel more distressed as a result of counseling 0.68 0
I8 The counselors at this center have insufficient understanding about HIV 0.58 5
7 I feel uncomfortable and distressed in needing to attend for counseling 0.47 5
3 I have found it difficult to get appointments to see a counselor when I need one at this center 0.40 I9
(8.0% of variance)

‘Percentage responding ‘Agree’ or ‘Strongly agree’ to item.

344 K. Begley et al. /Patient Educ. Couns. 24 (1994) 341-345

improved mood, Specific need for counseling, Be- higher score (4.28 vs. 5.34, t = -2.52, df = 31.28,
havior change from counseling, and Counseling P < 0.02).
climate (Table 1). Reliabilities for the four scales of the ACS were,
Intercorrelations of the ACS factors and the respectively 0.85, 0.70, 0.73 and 0.50. Reliabilities
CEI factors indicated that there were modest cor- for the three scales of the CEI were, respectively
relations between the factors (Table 2), with the -1.17, -0.35 and 0.65. Mean scores (*S.D.) and
exception of the correlation between the CEI medians on the scales as scored multiplying by fac-
Client satisfaction factor and the ACS Behavior tor score (possible total in parentheses) were Pro-
change factor which was substantial. ACS factor gress and improved mood, 7.5 f 2.3, Md 1.2
intercorrelations indicated that there was a (24.2); Specific need for counseling, 5.1 f 1.9, Md
moderate intercorrelation between factors 1 and 3, 5.0 (13.7); Behavior change from counseling,
which had nearly 25% of their variance in 4.4 f 1.5 (10.2); and Counseling climate,
common. 12.3 f 1.5, Md 12.5 (14.1). Percentage responding
There were significant correlations between ‘Agree’ or ‘Strongly agree’ to each question is in-
months of counseling and three of the four ACS dicated in the second column of Table 1.
factors, and the CEI Client satisfaction factor. No
correlations between age and the ACS and CEI 4. Discussion
scales were significant, nor those with educational
level. t-Tests comparing the ACS and CEI factors These data are based on a relatively small sam-
with those HIV-seropositive and -seronegative, ple of people attending for counseling, who are
and gender, were all insignificant with the excep- unlikely to be representative of those attending for
tion of ACS factor, Specific need for counseling, counseling or of those with HIV related concerns,
on which the HIV-positive respondents had a and thus of uncertain external validity. Never-
theless, these data do indicate that the ACS has
four distinct and empirically derived subscales
which have acceptable reliabilities. These
Table 2
Correlations between ACS questionnaire, CEI and demo- psychometric properties and its relatively brief for-
graphic variables mat recommend it as a useful instrument for
measuring client satisfaction with HIV-related
Variable ACSl ACB ACS3 ACS4 counseling. Further, there is a degree of associa-
49**
tion with the existing CEI which provides an index
ACSl (Progress and 0.24 -0.10
improved mood) of concurrent validity. On the other hand, the
ACS2 (Need 0.19 0.10 amount of variance in some of the responses was
counseling) low, which may (if this is replicated on broader
ACS3 (Behavior -0.20 samples) make it more difficult to measure pro-
change)
gress. However, measures of progress in outcome
ACS4 (Counseling
climate) will commonly use longitudinal measures of client
mood and attitude rather than changes in evalu-
CEI 1 (Counseling -0.09 -0.09 0.03 0.25’ ation of service impact, so the issue of service eval-
climate) uation is usually based on cross-sectional designs.
CEI2 (Counselor -0.42** -0.17 -0.37** 0.24
comfort)
The significance of the ACS is that it was
CE13 (Client 0.46** 0.17 0.61** -0.38** developed with the intention of measuring those
satisfaction) aspects of counseling which were most closely link-
Months counseling -0.34** -0.31; -0.31* -0.01 ed to HIV/AIDS issues and counseling practice,
Age 0.20 0.08 0.16 -0.10 rather than using a more general instrument such
Education level -0.18 -0.01 -0.07 0.13
as the CEI. The factor analysis revealed that there
P < 0.05.
?? are four discrete dimensions of client satisfaction
**p < 0.01. in HIV/AIDS counseling: improved mood and
 K. Begley et al. /Patient Educ. Couns. 24 (1994) 341-345 345

feeling of making progress; a recognition of a spe- needs to be further validated with other measures
cific need for counseling, as opposed to social sup- of progress, psychological symptomatology and
port; a recognition that behavior change has use of HIV/AIDS services to confirm its efficacy as
resulted from counseling; and an assessment of the a measure of HIV/AIDS counseling satisfaction.
counseling climate. All except the last were
significantly associated with months in counseling. References
The fact that three of the four ACS scales had
1 Miller D, Weber J, Green J. The Management of AIDS
significant correlations with months in counseling Patients. London: Macmillan, 1986.
suggests that it measures some aspect of satisfac- 2 Green J, McCreaner A. Counseling in HIV Infection and
tion, assuming that greater time in counseling is AIDS. Oxford: Blackwell, 1989.
associated with increased satisfaction. However, it 3 Ross MW, Channon-Little LD. Discussing Sexuality: A
Guide for Health Practitioners. Sydney: Maclennan &
may also be the case that time in counseling
Petty, 1991.
reflects severity of the concerns which lead the in- 4 Miller R, Bor R, Duiley JW, AIDS. A Guide to Clinical
dividual to counseling in the first place, and vali- Counseling (second edn). Philadelphia: Science Press,
dation of this scale with other less ambiguous 1991.
measures is needed. Alternatively, greater time in 5 Bor R, Miller R, Goldman E. Theory and Practice of HIV
Counseling. A Systemic Approach. London: Cassell,
counseling may lead to clients to have greater in-
1992.
vestment in the outcome and to want it to be effec- 6 Ross MW, Rosser BRS. Psychological issues in AIDS-
tive: as no data on social desirability were related syndromes. Patient Educ Couns 1988; 11: 17-28.
obtained, this cannot be discounted. Further, the 7 Perry S, Fishman B, Jacobsberg L, Young J, Frances A.
lack of significant association with age, educa- Effectiveness of psychoeducational interventions in reduc-
ing emotional distress after Human Immunodeficiency
tional level and gender suggests that the ACS is
Virus antibody testing. Arch Gen Psychiatry 1991; 48:
not unduly biased by demographic characteristics. 143-147.
The significant association between HIV status 8 Antoni MH, Baggett L, Ironson G, LaPerriere A, August
and the ACS scale measuring specific need for S, Klimas N, Schneiderman N, Fletcher MA. Cognitive-
counseling confirms that those with HIV infection behavioral stress management intervention butlers dis-
do have a greater perceived need for counseling. tress responses and immunologic changes following
notification of HIV-I seropositivity. J Consult Clin
Further, the item responses indicate that in the Psycho1 1991; 59: 906-915.
study population there was a strong recognition 9 Linden JD, Stone SC, Shertzer B. Development and eval-
both of the efficacy of HIV/AIDS counseling as ef- uation of an inventory for rating counseling. Pers Guid J
fective and as providing a need that could not be 1965; 44: 267-216.
10 Haase RF, Miller CD. Comparison of factor analytic
met elsewhere. A degree of concurrent validation
studies of the Counselling Evaluation Inventory. J Couns
was provided by the correlations between the CEI Psycho1 1968; 15: 363-367.
and the ACS subscales. II Ponterotto JG, Furlong MJ. Evaluating counselor effec-
In summary, the ACS provides an index of tiveness: a critical review of rating scale instruments. J
client satisfaction with HIV/AIDS counseling is- Couns Psycho1 1985; 32: 597-616.
12 Broadbent J. Assessing the counseling of HIV positive
sues which is generated from items more specific to
clients. Health Visitor 1987; 60: 262-264.
the area than those encountered in more generic 13 Walkey FH, McCormick IA (1985). Multiple replication
counseling satisfaction scales, and has adequate of factor structure: a logical solution for a number of fac-
psychometric properties for this. However, it tors problem. Mult Behav Res 1985; 20: 57-67.