Sample Non-Clinical

PIP
The Effectiveness of Peer and Family Support
Specialists in County Behavioral Health Settings
Dr. Steven Tally, UCSD
TEQI Summit
October 16, 2015

Overview of PIP Process

Step 1: Select & Describe the Study Topic
Step 2: Define the Study Questions
Step 3: Use a Representative and
Generalizable Study Population
Step 4: Select the Study Indicator(s)
Step 5: Use Sound Sampling Techniques
Step 6: Reliably Collect Data
Step 7: Analyze Data and Interpret Study Results
Step 8: Implement Intervention and Improvement
Strategies
Step 9: Plan for Real Improvement
Step 10: Achieve Sustained Improvement

PIP Must be Client Centered

IDENTIFY PROBLEM

Impact on
Programs

Impact on Family
INTERVENTION

Impact on
Community

Impact on
CLIENT

Step 1: Select and Describe the Study

Topic
Define the Problem
The role of Peer and Family support specialists (P/FSSs)
has grown in the delivery of behavioral health services
More programs are integrating Support Specialist roles
into program operations and client recovery plans.
While there is supporting literature that PSSs and FSSs
play key role in behavioral health services, impact on
the clients treatment, recovery, and engagement has
not been measured.
New proposed legislation (SB 614) requires the State
Department of Health Care Services to develop a peer
and family support specialist certification program.

Step 1: Select and Describe the Study

Topic
What is the overarching goal of the PIP?
The overarching goal of this PIP is to assess
the impact on clients of interaction with
Peer and Family support specialists (P/FSSs).

Step 1: Select and Describe the Study

Topic
This will be accomplished through:
Assessment of the impact on interaction
with clients
Assessment of Function and Effectiveness
(Program Managers and P/FSSs)

Step 2: Define the Study Questions

MAIN STUDY QUESTION
Does interaction with a Peer or Family
Support Specialist Impact Client
Outcomes?
SECONDARY STUDY QUESTIONS
What is their current role?
What are the perceptions of program
managers and P/FSSs of their impact?

Simple Conceptual Model

Does interaction with a Peer or Family
Support Specialist lead to
Improvements in Client Outcomes?
Clients with P/FSS
Interaction

Clients without P/
FSS Interaction

Client Outcomes

Step 3: Use a Representative and

Generalizable Study Population
Main data collection is through the
State Satisfaction Survey
Thus, our study population will be all
clients who receive services at an
eligible program during a 1-week
period.

Step 4: Select the Study Indicator(s)

Study indicators will be collected from
multiple sources:
1. Client Data
MHSIP State Satisfaction Survey data
and supplement
Outcomes Measures Currently
collected (HOMS)
Client-specific system data (CCBH)

Step 4: Select the Study Indicator(s)

2. Peer and Family Support Specialists:
Websurvey asking about function and
interaction with clients

Step 4: Select the Study Indicator(s)

3. Program Managers
Web survey of Program Managers who
Staff P/FSS
Ask about function, effectiveness and
impact on client

Step 4: Select the Study Indicator(s)

Client Study indicators:
Illness Management (IMR)
Recovery (RMQ)
Functioning (MHSIP)
Social Connectedness (MHSIP
General Outcomes from Services (MHSIP)
Client Satisfaction (MHSIP)
Child and Adolescent Measurement System (CAMS)
Direct Peer impact on Client (State Survey
Supplement, 7-items)

Complete Model

Step 5: Use Sound Sampling Techniques

Demonstrate that the data collection
methods currently used for the State
Satisfaction survey are sound.
Identify potential biases between
intervention groups.
Include programs for all groups of
interest.
Analyze past data for non-response
bias.

Step 6: Reliably Collect Data

State Satisfaction Survey
Inclusion of all relevant programs
Multiple languages if necessary
Clear instructions

Collection of Client Outcomes data from MIS

System
Choice of time windows
Web survey of Program Managers and P/FSS
Sufficient sample size

Step 7: Analyze Data and Interpret Study

Results
Answer research questions:
Did the Intervention have an impact?
Was it differential by group?
Age
Dx
Race/Ethnicity
Gender
Program Type

Step 8: Implement Intervention and

Improvement Strategies
So what? What next?
Using what you have found, develop a plan:
Implementation:
If there was an impact, how does this affect future
plans?
If no impact or differential impact?
Is there adequate supply for scaling up?
Improvement:
Were there training gaps?
Are there barriers to hiring and retaining P/FSSs that
need to be addressed?
Training Providers?
Cultural Competency?

Steps 9 and 10: Plan for Real Improvement

and Achieve Sustained Improvement
It is essential to determine if the reported change is real
change, or the result of an environmental or unintended
consequence, or random chance.

The following questions should be answered in the

documentation:
Reliability
Validity
Strength of the statistical evidence supporting that the
improvement is true improvement.
Plan for monitoring and sustaining improvement

PIP Validation Tool