Sei sulla pagina 1di 5

520794

research-article2014
APY0010.1177/1039856214520794Australasian PsychiatryHolmes et al.

AP
Psychiatric services

Australasian Psychiatry

Developing performance indicators 2014, Vol 22(2) 160­–164


© The Royal Australian and
New Zealand College of Psychiatrists 2014

for homeless outreach psychiatric Reprints and permissions:


sagepub.co.uk/journalsPermissions.nav
DOI: 10.1177/1039856214520794
services apy.sagepub.com

Alexander Holmes  Associate Professor and Psychiatrist, Department of Psychiatry,) University of Melbourne, Melbourne,
VIC, Australia
Gail Bradley  Senior Clinical Psychologist, Inner West Area Mental Health Service, Melbourne, VIC, Australia
Trevor Carslie  Social Worker, Homeless Psychiatric Outreach Service, Inner West Area Mental Health Service, Melbourne,
VIC, Australia
Kate Lhuede  Senior Occupational Therapist, Homeless Psychiatric Outreach Service, Inner West Area Mental Health Service,
Melbourne, VIC, Australia

Abstract
Objective: Performance indicators (PIs) aim to improve services by measuring key activities in a way that allows
comparison over time, between services and against benchmarks. This paper describes the development and imple-
mentation of Homeless Psychiatric Service PIs and explores their potential benefits and limitations.
Method: We collected descripton of quality service from key stakeholders. We identified eight key parameters, from
which PIs were developed and tested over a 12-month period.
Results: The use of the PIs led to increased awareness of the practice being measured. PIs were used to stimulate
practice changes. They played a positive role in team dynamics and were useful in clarifying team aims and identity.
The main challenge to their use was the burden of data collection and analysis.
Conclusion: Homeless service PIs can assist in determining how well the programs are performing in activities that
are relevant to clients and non clinical services for the homeless. With the movement of homeless clients away from
inner urban areas, homeless performance measures may aid teams to develop the capacity to work effectively with
homeless clients.

Keywords:  benchmarking, homeless, homeless services, performance indicators, program planning, quality

P
erformance indicators (PIs) aim to improve ser- public mental health services. These were part of the
vices by measuring key activities in a way that ‘information infrastructure’ designed to encourage good
allows comparison over time, between services practice, inform about outcomes and value for money,
and against benchmarks. Within Australian public men- and contribute to state and national data.4 In broad
tal health, the focus to date has been on PIs applicable terms, these PIs are used to determine the degree that
over a wide range of services,1 yet the need to develop services are effective, responsive, continuous, appropri-
program-specific measures has been clearly articulated.2 ate, accessible, capable, efficient, safe and sustainable.4
Program-specific indicators measure activities that are PIs need to be observable, understandable, valid, rele-
more closely related to the key aims and objectives of the vant and efficient.5 An example of the current national
program, and have the advantage of promoting quality indicator is ’28-day readmissions to acute mental health
in their more specialised activities. Homeless Outreach units’, for which the threshold set at 15%.6
Psychiatric Services is one example of where specific
measures may have advantages. This paper describes the
development and implementation of homeless psychi-
atric service PIs and explores their potential benefits and Corresponding author:
limitations. Alex Holmes, Department of Psychiatry, University of
Melbourne, Royal Melbourne Hospital, Grattan Street,
The Second National Mental Health Plan3 stipulated the Parkville, Melbourne, VIC 3050, Australia.
development of performance indicators for Australian Email: acnh@unimelb.edu.au

160
Holmes et al.

PIs were also developed for the Child and Adolescent


Mental Health Services (CAMS)7 and Consultation- Table 1.  Needs that may be addressed by a
Liaison (C-L) psychiatry services.8 Both assess ‘time to homeless mental health service, as identified by
initial assessment’, a measure of responsiveness, which different stakeholders
is important when providing mental health expertise to
general health services. The CAMS PIs also measure Stakeholders Themes
dropout rate (continuity) and activity (hours of contact),
highlighting the challenges of engagement and the Clients Autonomy
value of the therapeutic alliance in this population. In   Improved housing
C-L, PIs measure successful communication, in keeping   Improved finances
with the consultation and collaboration at the core of
Carers Effective treatment
C-L practice. Use of C-L PIs over a year can lead to
improved communication,9 at the cost of regular data   Communication
collection and analysis. Accommodation Management of mental illness
services Expertise and advice
No single method has been identified by which PIs are
  Responsiveness
developed. A common approach is to first develop a bat-
tery of measures representing all the functions of a ser-   Outreach
vice, as identified by professionals, carers, patients and   Collaboration
affiliated organisations.10,11 For example, Shield et al.,11   Communication
working in primary care, identified a potential of 334   Support
indicators. These were rated for validity by representa- Homeless Psychiatric Engagement of disengaged
tive panels, with 26% achieving consensus across all Services homeless persons
groups. This method is comprehensive and promotes a   Developing a pathway out of
balance of perspectives, but requires the capacity to
homelessness
engage all stakeholders. An alternative technique is for
expert groups to formulate indicators with reference to   Assertive outreach
their experience and the literature.2   Responsiveness and accessibility
  Continuity of care
Once a PI is defined, irrespective of the method used, a
  Shared care and collaborative
benchmark is established. This is informed by analysis of
the literature, information about ‘current reasonable casework
practice’ and the benefits of a particular activity. For
example, the threshold for ‘28-day readmissions’ was set
using length-of-stay data and the principle that
unplanned early admission reflects a breakdown of com- with reference to the literature13–15 and face-to-face com-
munity care.6 Finally, there is a limit on the number of munication with the limited number of staff working in
PIs that can be applied in any one setting, because their specialised programs state-wide.
utility is known to decrease as their number and com-
The raw data was subjected to a simple thematic induc-
plexity increase.12
tive analysis. Initially, the themes were identified and
coded. These themes were then taken back to each case,
in order to determine if other themes were identified
Developing performance measures that were not covered thematically. This process allowed
for homeless psychiatric services for confidence that, aside from individual or idiosyn-
The PIs for the homeless were developed within an inner cratic needs, no major needs were excluded. The final
urban homeless mental health service,13 using both themes are listed in Table 1.
stakeholder and expert data. Stakeholders (clients, car-
PIs were developed to address the themes identified
ers, accommodation and mental health services)
(Table 2). Not all themes were fully covered. A measure
required different methods for collecting information.
of improved finances was not included, as the factors
Client and carer needs were determined by reviewing
influencing this were seen to extend well beyond the
the documented response to the question, ‘What are
influence of the clinical service. Agency support was
your current needs?’ for the approximately 100 new
conceptually similar to ‘responsiveness and accessibil-
cases undergoing needs assessment during the previous
ity’, and subsumed under that theme.
year. Stakeholders in accommodation services were
asked to complete a written survey, which asked: ‘What Two PIs required the use of scales. For the ‘assessment of
are the most important aspects of a homeless mental improved engagement’, we used a 5-point scale (1 =
health service?’ We sent the survey to the 25 services refuses to engage, 2 = poor engagement, 3 = some
within the catchment area: with the aid of telephone engagement, 4 = moderate engagement and 5 = actively
follow-up, all were able to provide information. The engaged). If the Life Skills Profile is being collected as an
Homeless Mental Health Service perspective was derived outcome measure, then item 19 there can be used as an

161
Australasian Psychiatry 22(2)

Table 2.  Performance indicators for Homeless Psychiatric Services

Name Themes Definition Threshold


1 Contact Success Outreach N patients seen, as a percentage of the n patients 70%
referred
2 Timeliness Responsiveness N patients seen within 72 hours, as a percentage 50%
of n patients seen
3 Engaged in active Mental illness N patients engaged in active treatment, as 50%
treatment management percentage of n patients seen
4 Shared care and Collaboration, N patients with active collaborative case plan with 50%
collaborative casework communication, homeless agency, as a percentage of n patients
shared care engaged in active treatment
5 Communication with Communication N patients for who there was regular contact with 50%
family/carers family/carers, as a percentage of n patients for
whom family/carers are identified
6 Improved engagement at Engagement N patients with improved engagement at discharge, 50%
discharge as a percentage of n patients discharged
7 Successful discharge Continuity of care N patients discharged to, and engaged with, 50%
planning appropriate mental health service; as a percentage
of n patients discharged
8 Improved Improved housing N patients with improved accommodation type 50%
accommodation at at discharge, as compared with admission, as a
discharge percentage of n patients discharged

N or n: number

alternative16 (Does this person co-operate with health tion of approximately 180,000 persons. The catchment
services? Always = 4, Usually = 3, Rarely = 2 and Never included two male homeless shelters containing 120
=1). Assessment of an improved accommodation requires beds. During the year, no substantial changes occurred
‘ranking’ by accommodation type. The hierarchy used within mental health nor accommodation services.
was developed in reference to the definition of homeless During the initial implementation phase, concern was
used by the Australian Bureau of Statistics17 (1 = shelter- raised by participating clinicians regarding the burden of
less, 2 = emergency accommodation, 3 = single room the collection and recording of data, and the complexity
without amenities, 4 = transitional accommodation and of interpreting numerous PIs. In response, cognisant of
5 = stable and secure housing). the principle limiting the number of PIs, we removed the
two that were proving the most difficult to collect (com-
The benchmarks for the PIs were set with reference to
munication with family and collaborative casework).
what was known about the current performance, aspira-
tions for improvement and a judgement as to the limits Data was collected at entry and at discharge from the
of effectiveness. For example, prior to commencement service. When data was incomplete or unclear, neither
the number of referred patients whom were assessed was the numerator nor the denominator were entered.
< 50%. At the same time, the capacity to engage the dis- Numerators and denominators for each PI were calcu-
engaged was identified by three stakeholders: it is a key lated for each month, and then entered into a pre-
component of the Homeless Outreach Psychiatric formatted Excel spread sheet, which generated a
Service model.13 Setting the threshold at 70% reflected 12-month summary and graph (Figure 1). We presented
an expectation of improvement, balanced with the prac- and discussed the data at monthly team meetings, where
tical difficulty of finding homeless persons. we devised and implemented strategies related to
improving specific activities.
Overall, the activities recorded by the PIs occurred at levels
Implementing performance measures similar to established thresholds (Table 3). When changes
We collected data over a calendar year, for all of the cases occurred, they emerged over the course of the year. In the
referred to a homeless psychiatric service.13 This service first months, their primary impact was through an
covers the central business district and inner northern increased awareness of the practice being measured.
suburbs of Melbourne, Australia, comprising a popula- Subsequently, sub-optimal performance relative to the

162
Holmes et al.

100

Proportion engaged at discharge (%)


90
80
70
60
50
40
30
20
10
0
Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec
Month of the year

Figure 1.  Homeless Psychiatric Service Performance Indicator: Improved engagement at discharge.

Table 3.  Mean percentage and range of individual PIs over a 12-month period

Description Threshold Yearly mean, % Range (over 1 year)


Contact success 70% 85 70 – 100
Timeliness 50% 50 14 – 100
Engaged in active treatment 50% 58 25 – 100
Improved engagement at discharge 50% 50 17 – 100
Successful discharge planning 50% 54 33 – 100
Improved accommodation at discharge 50% 45 17 – 100

PI: Performance indicator

arbitrary threshold (timeliness, active treatment or practices, having much in common with those applied
improved discharge accommodation) drew greater atten- in generic mental health services, place specific empha-
tion, leading to both analysis and practice change. For sis on access, engagement, continuity and interagency
example, exploration of the low levels of engagement at collaboration. When incorporated into a routine team
discharge revealed the impact of a precipitous unplanned process, homeless PIs can lead to practice change and
exit from an emergency accommodation. These exits were improved care.
instigated either by the client or the service, in response to
The PIs had a range of effects on the various stakehold-
particular behaviours. The team action was to become an
ers. In the client domain, accommodation at discharge
active participant in accommodation planning (coordi-
improved. Although the complex interplay between
nated by the housing worker) and to develop manage-
housing type, stability and mobility, and mental illness
ment plans, to deal with the behaviours associated with
precludes a comprehensive analysis of all potential
eviction. In time, fewer clients left the homeless setting
mechanisms for this finding, one is worth describing.
without follow-up care being arranged. Similar improve-
When information about individuals with a mental ill-
ments occurred in accommodation at discharge and suc-
ness is communicated to housing workers it changes
cessful discharge planning.
expectations and improves engagement, resulting in
Not all PIs improved over the year. Where early levels greater tolerance and more appropriate discharge plan-
were high (contact success), there was little scope for ning. Within the team, the PIs had a positive role in
improvement. For others, for example ‘engaged in active team dynamics. The monitoring of activities defined as
treatment’, initial improvements were not sustained. One ‘most important’ was seen as more meaningful and dem-
interpretation of this finding is that initial improvements onstrating gains for clients that enhanced team agency
relating to an increased focus on a function, but not sup- and identity. This support of group process is particu-
ported by effective change in method, will not be lasting. larly useful when working with homeless clients, whom
can be difficult and time consuming to work with, and
about whom pessimism and frustration are commonly
Discussion expressed.
We designed these PIs to measure activities identified as The major drawback of using PIs is the burden of collec-
important by those involved in the care of the homeless tion, analysis and use of data within cycles of quality
mentally ill, including the clients themselves. These improvement. A quantification of this effort was not

163
Australasian Psychiatry 22(2)

possible in our project, due to the disparate elements Disclosure


involved, and the overlap of development and imple- The authors report no conflict of interest. The authors alone are responsible for the content
mentation. Data collection can be integrated into clini- and writing of the paper.
cal documentation, and review incorporated into
routine team meetings, but the analysis and formatting References
does require time and commitment. This activity is only 1. Australian Institute of Health and Welfare. A set of performance indicators across the
likely to occur if it is perceived as useful. For this reason, health and aged care system. Canberra: Australian Institute of Health and Welfare,
it may be that these PIs are used by some services over a 2008.
defined period of time. 2. National Mental Health Working Group Information Strategy Committee Performance
Indicator Drafting Group. Key performance indicators for Australian Public Mental
The methods we used to develop the PIs have limitations.
Health Services. Report, Discussion paper no. 6, 2005. Canberra: Australian Government
Face-to-face interviews have a greater capacity to explore Department of Health and Aging.
responses than written questions, but they require a level
3. Australian Health Ministers. Second national mental health plan. Mental Health Branch,
of access and collaboration that is not easily found in Commonwealth Department of Health and Family Services, July 1998.
homeless settings. Not all perspectives were equally repre-
sented. Only one need that was specifically identified by 4. National Health Performance Committee. National health performance framework.
Report to the Australian Health Ministers, 2001. Brisbane: Queensland Health.
clients (improved accommodation) was included. The
request for autonomy by clients was difficult to address, 5. Poertner J and Rapp CA. Designing social work management information systems:
The case for performance guidance systems. In: Patti RJ, Poertner J and Rapp CA
as for some it more accurately represented a desire to be
(eds) Managing for service effectiveness. New York: Haworth Press, 1998, pp.
‘left alone’; however, working with the client’s autonomy 177–190.
is implicit to having a collaborative relationship and
6. Callaly T, Hyland M, Trauer T, et al. Readmission to an acute psychiatric unit within 28
improvement in that domain may be seen to reflect this.
days of discharge: Identifying those at risk. Aust Health Rev 34: 282–285.
The removal of the PIs relating to family and collabora-
tion also limited the scope of representation. 7. Birleson P, Brann P and Smith A. Using program theory to develop key performance indi-
cators for child and adolescent mental health services. Aust Health Rev 2001; 24: 10–21.
Homeless PIs may be useful in developing homeless 8. Holmes AC, Judd FK, Lloyd JH, et al. The development of clinical indicators for a consul-
functions in less-specialised community teams. tation-liaison service. Aust NZ J Psychiatry 2000; 34: 496–503.
Originally, the homeless teams were established to
9. Holmes AC, Judd FK, Yeatman R, et al. A 12-month follow up of the implementation
enhance access to care, largely in inner urban settings.13 of clinical indicators in a consultation-liaison service. Aust NZ J Psychiatry 2001; 35:
The gentrification of the inner city, housing pressures 236–239.
and urban sprawl have all contributed to a migration of
10. O’Brien AP, O’Brien AJ, Hardy DJ, et al. The New Zealand development and trial of
those with accommodation instability to the middle and mental health nursing clinical indicators: A bicultural study. Int J Nurs Stud 2003; 40:
outer suburbs. Periods of homelessness are common in 853–861.
patients attending community clinics.18 The capacity to
11. Shield T, Campbell S, Rogers A, et al. Quality indicators for primary care mental health
access and work with homeless people may be required services. Qual Saf Health Care 2003; 12: 100–106.
for many urban mental health services, practices which
12. Addington D, Kyle T, Desai S, et al. Facilitators and barriers to implementing quality
may be measured using these PIs.
measurement in primary mental health care: Systematic review. Can Fam Physician 56:
1322–1331.

13. Holmes A, Hodge M, Newton R, et al. Development of an inner urban homeless mental
Conclusion health service. Australas Psychiatry 2005; 13: 64–67.

Homeless service PIs can determine how well programs 14. Catholic Social Services. Summary report: Access to psychiatric care. Twelve months on.
are performing in activities that are relevant to clients and Report, 1995. Victoria: Catholic Social Services.

service providers within settings where there are home- 15. Human Rights and Equal Opportunity Commission. Human rights and mental illness.
less people. Using them can improve care and facilitate Report of the National Inquiry into the Human Rights of People with Mental Illness, 1993.
team morale. These measures may also be useful to non- 16. Rosen A, Hadzi-Pavlovic D and Parker G. The Life Skills Profile: A measure assessing
specialised teams, when working with homeless clients. function and disability in schizophrenia. Schizophr Bull 1989; 15: 325–337.

17. Australian Bureau of Statistics. Information paper - A statistical definition of homless-


Acknowledgements ness. Report, 2012. Canberra: Australian Bureau of Statistics.

Thanks to Lisa Gibbs, of the University of Melbourne School of Population and Global Health, 18. Holmes AC, Hodge MA, Bradley G, et al. Accommodation history and continuity of care
for her advice and expertise in qualitative research methods. in patients with psychosis. Aust NZ J Psychiatry 2005; 39: 175–179.

164

Potrebbero piacerti anche