Can J Diabetes 37 (2013) 18e26

Contents lists available at SciVerse ScienceDirect

Canadian Journal of Diabetes

journal homepage:
www.canadianjournalofdiabetes.com

Original Research

Behaviour Change CounsellingdHow Do I Know If I Am Doing It Well?

The Development of the Behaviour Change Counselling Scale (BCCS)
Michael Vallis PhD *
Dalhousie University, Halifax, Nova Scotia, Canada

a r t i c l e i n f o a b s t r a c t

Article history: Objective: The purpose of this article is to operationalize behaviour change counselling skills (motivation
Received 13 October 2012 enhancement, behaviour modification, emotion management) that facilitate self-management support
Received in revised form activities and evaluate the psychometric properties of an expert rater scale, the Behaviour Change
8 January 2013
Counselling Scale (BCCS).
Accepted 8 January 2013
Methods: Twenty-one healthcare providers with varying levels of behaviour change counselling training
interviewed a simulated patient. Videotapes were independently rated by 3 experts on 2 occasions over
6 months. Data on item/subscale characteristics, interrater and testeretest reliability, preliminary data on
Keywords:
behaviour change counselling
construct reliability, were reported.
behaviour modification Results: All items of the BCCS performed well with the exception of 3 that were dropped due to infrequent
emotion management endorsement. Most subscales showed strong psychometric properties. Interrater and testeretest reliability
motivation coefficients were uniformly high. Competency scores improved significantly from pre- to posttraining.
reliability Conclusions: Behaviour change counselling skills to guide lifestyle interventions can be operationalized
scale development and assessed in a reliable and valid manner.
self-management support Practice Implications: The BCCS can be used to guide clinical training in lifestyle counselling by oper-
ationalizing the component skills and providing feedback on skill achieved. Further research is needed to
establish cut scores for competency and scale construct and criterion validity.
Ó 2013 Canadian Diabetes Association

r é s u m é

Mots clés: Objectif : Le but de cet article est mettre à profit les compétences de counselling en changement de
counselling en changement de comportement comportement (renforcement de la motivation, modification de comportement, prise en charge des
modification de comportement émotions) qui facilitent les activités de soutien à la prise en charge autonome et d’évaluer les propriétés
prise en charge des émotions psychométriques d’une échelle d’évaluation experte, la Behaviour Change Counselling Scale (BCCS).
motivation
Méthodes : Vingt-et-un (21) prestataires de soins de santé de différents niveaux de counselling en
fiabilité
changement de comportement ont interviewé un patient simulé. Des bandes vidéos ont été indé-
conception d’échelles
soutien à la prise en charge autonome pendamment cotées par 3 experts à 2 occasions durant 6 mois. Les données sur les caractéristiques d’un
item et des sous-échelles, la fiabilité interévaluateurs et la fiabilité de test-retest, et les données pré-
liminaires sur la validité conceptuelle ont été rapportées.
Résultats : Tous les items de la BCCS ont été bien réalisés à l’exception de 3 qui avaient été abandonnés en
raison de leur rare appui. La plupart des sous-échelles ont montré d’excellentes propriétés psychomé-
triques. Les coefficients de fiabilité interévaluateurs et de fiabilité de test-retest ont été uniformément
élevés. Les scores de compétence se sont significativement améliorés par rapport à la période avant la
formation et à la période après la formation.
Conclusions : Les compétences de counselling en changement de comportement pour orienter les
interventions sur le mode de vie peuvent être mises à profit, et évaluées de façon fiable et valide.
Pratique et conséquences : La BCCS peut être utilisée pour orienter la formation clinique en counselling sur
le mode de vie en mettant à profit les compétences constituantes et en fournissant une rétroaction sur
les compétences atteintes. D’autres recherches sont nécessaires pour établir des scores minimaux pour
les compétences, les conceptions d’échelle et la validité critérielle.
Ó 2013 Canadian Diabetes Association

* Address for correspondence: Michael Vallis, Diabetes Management Centre, 7071

Bayers Rd, Suite 165, Halifax, Nova Scotia, Canada B3L 2C2.
E-mail address: tvallis@dal.ca.

1499-2671/$ e see front matter Ó 2013 Canadian Diabetes Association

http://dx.doi.org/10.1016/j.jcjd.2013.01.005
 M. Vallis / Can J Diabetes 37 (2013) 18e26
Introduction
Health behaviours are pivotal with regard to disease onset and
management and general health and wellness (1,2) and especially
with regard to type 2 diabetes mellitus and its associated obesity
and cardiovascular diseases (3e5). Four modifiable health behav-
iours; smoking, poor diet, physical inactivity and alcohol misuse
may account for a much as 38% of annual US deaths (6). Equally
alarming are estimates that only 3% of the US population do not
smoke, are at a healthy weight and are physically active and eat 5 or
more fruit/vegetable servings daily (7). Those who demonstrate
health behaviours appear to be at lower risk of disease. Considering
the cluster of healthy weight, healthy diet, adequate activity and
not smoking cross-sectional studies have shown substantial
reduced risks of all cause mortality (up to 58% in men; 51% in
women) (8), as well as diabetes, myocardial infraction, stroke and
cancer (9). One longitudinal study (4 years) demonstrated that
those who went from not meeting criteria for healthy lifestyle to
meeting criteria reduced their risk of all cause mortality (40%) and
cardiovascular events (35%) (10).
Intervention studies have also shown that changing behaviour
improves medical outcomes. The Diabetes Prevention Program
(DPP) (11) compared behaviour change intervention to metformin
and placebo (RCT). Risk of diabetes was reduced by 58% in the
behaviour change group, twice that achieved with metformin.
Meta-analyses confirm the benefit of lifestyle intervention for
diabetes prevention (12), including one study demonstrating
benefit at 20 years (13). Following the DPP, a large number of life-
style intervention studies have appeared in the medical literature.
A PubMed search using the mesh terms “lifestyle” and “interven-
tion studies” returned 225 citations (conducted May 23, 2012).
Evidence justifies integrating behaviour change counselling into
clinical management for clinicians to operationalize the concept of
self-management support. However, few clinicians have been
trained in behaviour change counselling and there is a lack of
standardization regarding how to implement behaviour change
counselling. Clinicians need help going beyond recommendations
(14e16) and education (17) to helping people with the motiva-
tional, behavioural and emotional challenges associated with long-
term (lifestyle) behaviour change.
The behavioural sciences have made positive contributions to
behaviour change interventions. In particular, the work on moti-
vation (transtheoretical model, social cognitive model, theory of
planned behaviour) and cognitive-behavioural interventions
(cognitive behaviour therapy, behaviour modification, stress
management) have provided theory-driven and evidence-based
strategies that can be used in clinical practice to support behav-
iour change (18,19). However, models are required that take these
principles and translate them into skills that can be learned by
nonbehaviourally trained clinicians who dominate the clinical
world and have little training or confidence in behaviour change
counselling (20,21).
The Behaviour Change Institute staff (an interdisciplinary group
of clinician researchers at Capital Health/Dalhousie in Halifax) have
been interested in providing behaviour change counselling training
to general practice clinicians in their role of self-management
support derived from behaviour change theory and evidence-
based behaviour change interventions. The model guiding this
training involves 3 categories of behaviour change counselling
skills. First, skills associated with helping people commit to the
health behaviour in question when there is an initial lack of moti-
vation or readiness to change; i.e. readiness assessment and
motivational interviewing. Second, behaviour modification skills
that increase the likelihood of successful change when the indi-
vidual is ready to change; i.e. goal-setting, behaviour shaping,
stimulus control and reinforcement management. Third, emotion
19
management skills that facilitate maintenance of change once
changes have been made; i.e. stress management and managing
unhealthy behaviours that serve a purpose for the individual. This
study is a report on the development of a scale to assess compe-
tency in these three skill areas. If a reliable and valid scale can be
developed it can be used for training and research.
There are available scales to assess some, but not all, of the
above behaviour change skills. Of note are scales assessing moti-
vational interviewing skills. The most common scales are the
Motivational Interviewing Treatment Integrity Code (MITI) (22) and
the Motivational Interviewing Skill Code (MISC) (23). Although
these scales have been validated, they are not suitable for our
purposes for 2 reasons. First, the scales only address motivational
interviewing skills and not behavioural or emotion management
skills and as such are incomplete. Because we propose a model of
training that is more comprehensive than motivational interview-
ing we wanted a scale that reflected this model. Second, the MITI
and MISC scales involve ratings of verbatim, transcribed state-
ments. Individual statements are evaluated on specific scales and
then tallied into categories such as percent complex questions,
percent open questions, ratio of reflections to questions and
percent MI adherent statements (MITI) or the ratio of reflections to
questions, percent open questions, percent complex questions,
percent MI consistent and MI inconsistent responses and percent
change talk (MISC). This level of analysis is more specific than what
we were interested in. We wanted to assess skill at a more inte-
grated level. Rather than analyzing each comment we wanted
a scale that reflected judgements of skill for an entire intervention
(e.g. how well did the clinician adopt nonjudgemental curiosity in
this interview as opposed to what percentage of comments re-
flected nonjudgemental curiosity?). Another scale of note is the the
Behaviour Change Counselling Index (BECCI) (24). The BECCI
involves more general rating of activities such as inviting the
patient to talk about behaviour change, encouraging the patient to
talk about change, and conveying respect for the patient choice
about behaviour change. In addition to these rater-based assess-
ments, there are self-report and stimulus-response methods
(25e27). These scale were not suitable for our purposes because
there is little overlap between the items and the core skill sets of
the BCI training model, other than the names of the scales. Given
that our training model is comprehensive and highly articulated it
was clear to us that we needed to develop a new scale that
specifically measured the skills trained.
For training to be effective, counselling skills need to be defined
and shown to be assessed in a reliable and valid manner. The
purpose of this article is to operationalize behaviour change
counselling skills and to assess the psychometric properties of
a resulting competency rating scale.
Methods
A 3-step approach to scale construction of the Behaviour Change
Counselling Scale (BCCS) was followed. Step 1 was item generation,
step 2 involved generation of relevant stimulus material and step
3 involved a detailed item evaluation (28).
Item generationdthe BCCS
Based on a review of the literature and the experiences of
training primary healthcare providers in behaviour change skills,
3 core skill areas were identified to provide behaviour change
counselling training in: motivational enhancement, behaviour
modification and emotion management. Each of these skill areas
was identified using relevant theoretical models (18,19) in partic-
ular the transtheoretical model, the social cognitive model, the
theory of planned behaviour as well as cognitive-behavioural
20 M. Vallis / Can J Diabetes 37 (2013) 18e26

models (cognitive behaviour therapy, behaviour modification, emotional issues was addressed. Table 4 illustrates how these skills
stress management). were operationalized and rated.
Motivational enhancement skills involve motivational inter- Within each skill set component skills were defined and 7-point
viewing (29) and readiness assessment (30). Motivational inter- Likert type rating scales, in which the scores were behaviourally
viewing skills include asking questions by minimizing statements, anchored, were developed as illustrated in Tables 1e4.
expressing empathy, demonstrating nonjudgemental curiosity,
Generation of stimulus material/procedure
avoiding argument, rolling with resistance and supporting self-
efficacy. Readiness assessment, developed by the BCI, involves
To improve the quality of the data generated in this study we
defining the behaviour to be changed in specific terms and then
addressed 2 important issues: the nature of the patient and the
asking 4 sequential questionsdIs the behaviour (or lack of)
direction taken by the interviewer.
a problem for you? Does the behaviour (or lack of) cause you
Most competency rating scales involve trainees submitting
distress? Are you interested in changing this behaviour? Are you
recordings (video or audio) to be rated by experts. Although
ready to take action toward change at this time? Finally, feedback is
convenient, differences in presentation by different patients can
given regarding readiness to change by using a traffic light meta-
add error variance to the judgement of skill (e.g. a less skilled
phor: red light (change not likely) if the clinician considered the
clinician might appear more skilled if working with an “easy”
patient’s response as indicating not ready, yellow light (proceed
patient, where a skilled clinician might look less skilled if working
with caution), or green light (go) if the clinician judged the patient
with a very challenging patient). We controlled for the stimulus
to be ready to change. Table 1 and Table 2 show how each of these
(the patient) by engaging a simulated patient (a professional actor)
skills was operationalized and rated.
who was provided with a script of a patient persona and would stay
Behaviour modification involves the skills of goal setting,
“in role,” maintaining a consistent response to different lines of
behaviour shaping, stimulus control and reinforcement manage-
inquiry.
ment. Table 3 illustrates how these skills were operationalized and
We also controlled the focus of the clinical interview by struc-
rated.
turing the interview at its outset. Clinician participants were
Emotion management skills involve identifying the function
instructed:
served by unhealthy behaviours and supporting the individual to
replace the function as a means of change, identifying the need for “In this study we are asking you to demonstrate how to counsel
stress management and recommending stress management activ- a patient in behaviour change. You will interview a simulated
ities as a means of overcoming stress as a barrier to change. As well, patient. The interview will be separated into 3 segments; each
given that most healthcare providers are not trained as mental limited to no more than 12 minutes. Each segment is focused on
health professionals the issue of referring to a mental health a distinct issue; segment 1 is focused on motivational issues,
provider if the clinician is not comfortable with managing the segment 2 on behaviour modification, and segment 3 on

Table 1
Motivational interviewing skills

Motivational Scale score

interviewing
Low
skill
1 2
Effective The clinician almost exclusively makes
questioning statements. Questions are not used to
evoke the individual’s experience
Use of empathy to The clinician does not clearly convey to
promote change the individual that the story being told
talk makes sense given the experience of
the individual
Demonstrates The clinician does not express curiosity
nonjudgemental as to the reasons behind the
curiosity individual’s choice. If curiosity is
expressed, it’s done in a judgemental
way (goodebad; shouldeshouldn’t)
Demonstrates The clinician does not explore occasions
ability to roll when the individual struggles to
with resistance change but moves to another area
Able to avoid The clinician focuses exclusively on
argument reinforcing the Yes and doesn’t label
the But, resulting in a debating style in
which the clinician focuses on reasons
to change and the patient barriers to
change
Supports The clinician focuses on the completion
self-efficacy of the behavioural goal with no
assessment of confidence
Moderate
3 4 5 6
The clinician makes some use of questions but makes as
many statements
The clinician is able to express understanding of the
individual’s experience but doesn’t invite a discussion
of the other side
The clinician is able to express curiosity in a way that
does not imply judgement but does not do this
extensively and does not invite the individual to
suspend judgement
The clinician encourages the individual to explore how
their attempts did not work, but does not link these to
new solutions
The clinician is able to avoid getting into a debate but does
not re-interpret what the patient says to clarify that there
are more barriers than motivators
The clinician is able to assess altering expectations so as to
maintain high efficacy
High
7
The clinician almost exclusively relies on
questioning to evoke the experience of
the individual; even difficult areas are
discussed in a way that minimizes the
defensiveness of the individual
The clinician is both able to explicitly
feed back to the individual how his/her
choice reflects his/her experience and
invite the individual to discuss the
other side of the issue
The clinician is both able to explicitly
express nonjudgemental curiosity and
is able to encourage the individual to
adopt a similar perspective
The clinician is able to frame behavioural
roadblocks as opportunities for
learning, is able to guide to identify
factors associated with the roadblock
and effectively uses this to help the
person learn something new to
overcome the barrier
The clinician is able to reinterpret the
Yes/But into No/Because in such a way
that advances the person’s
understanding of their behaviour and
their willingness to try something new
The clinician is able to assess confidence
and help the patient learn that
confidence should take precedence
over behavioural expectations in
situations where confidence gets shaky
 M. Vallis / Can J Diabetes 37 (2013) 18e26 21

Table 2
Readiness assessment skills

Readiness Scale score

assessment skills
Low Moderate High

1 2 3 4 5 6 7
Assesses patient The clinician does not ask the question The clinician asks the question but does not elicit a clear The clinician asks the question and
perceives “Do you consider X a problem” response from the person clarifies the person’s response so that
a problem it is clear to both individuals
Assesses the The clinician does not ask the question The clinician asks the question but does not elicit a clear The clinician asks the question and
perceived “ Are you distressed by X?” response from the person clarifies the person’s response so that
problem causes it is clear to both individuals
distress
Assesses interest The clinician does not ask the question The clinician asks the question but does not elicit a clear The clinician asks the question and
in changing “ Are you interested in doing response from the person clarifies the person’s response so that
behaviour something to change X?” it is clear to both individuals
Determines if The clinician does not ask the question The clinician asks the question but does not elicit a clear The clinician asks the question and
patient is ready “ Are you prepared to do something response from the person clarifies the person’s response so that
to act now now to change X?” it is clear to both individuals
Gives the patient The clinician does not provide feedback The clinician makes oblique reference to readiness but The clinician clearly shares with the
feedback about to the individual does not present the traffic light assessment person the results of the assessment
readiness using the traffic light concept; the
clinician is able to use this concept
to identify relevant issues and goals

emotion issues. You should not feel compelled to address issues
outside of this segmentation (for example, in the segment on
behaviour modification assume that motivation or stress/
emotions are not a problem at this time). Also, you will be given
basic information about the patient and you are encouraged not
to take a history or to complete a background assessment.
Assume that you have already seen the patient for an assess-
ment and this is a follow-up session.”
Participants
A convenience sample of 21 healthcare providers with mixed
training backgrounds (psychology, nursing, dietetics, physiotherapy,
pharmacy and PhD psychology graduate students) were recruited to
interview the simulated patient. A total of 25 professionals were
approached, with 21 (84%) consenting to participate. Individuals
who were familiar with the work of the Behaviour Change Institute
and who had varying level of exposure to the training were
approached. Some were aware of the BCI and had attended a lecture
describing the Institute and the training model. Others had partici-
pated in either a half day or 2 day training program with no follow-
up. These participants were deemed to be minimally trained. Other
participants had received follow-up support for the use of the
behaviour change counselling skills for 6 months or more (labelled
the extensively trained group).
Participating clinicians interviewed the simulated patient to
demonstrate their use of behaviour change counselling skills. The
interviews were videotaped and divided into 3 segments, each
limited to 12 minutes and focused on motivation (segment 1),
behaviour change (segment 2) and emotion management (segment 3),
in that order. All 3 segments were conducted on a single occasion,
with instructions provided before each episode. These tapes were
considered to be the stimulus material.
Raters
The stimulus material was evaluated using the BCCS by
3 experts in behaviour change counselling. These 3 experts
(2 psychologists and 1 dietitian) were the Director and staff of the
Behaviour Change Institute. All were experts in the area of
behaviour change and all had at least 4 years of experience
working within the BCI to provide systematic training to varied

Table 3
Behaviour modification skills

Behaviour Scale score

modification skills
Low Moderate High

1 2
Goal setting uses The clinician either does not set goals
SMART goals with the individual or accepts goals
that do not meet the SMART criteria
(specific, measurable, achievable,
relevant, timely)
Behaviour shaping The clinician does not use the shaping
concept but tends to simply
encourage the individual to “just
do it”
Stimulus control The clinician focuses only on personal
factors associated with behaviour and
does not address the importance of
external factors that elicit behaviour
Reinforcement The clinician does not introduce the
management concept of using reinforcing
experiences to help maintain
motivation/behaviour but tends to
encourage willpower
3 4 5 6
The clinician introduces goal-setting and makes some
attempt to use the SMART acronym but is unable to
achieve the goal
The clinician is able to introduce some aspects of shaping
but does not make the most of it by educating the person
in the use of the skill
The clinician is able to introduce the idea that environment
controls behaviour but does not help the person identify
personal triggers
The clinician encourages the individual to feel good about
their health behaviours but does not educate in the use of
reinforcing activities
7
The clinician is able to guide the
individual in the use of the SMART
acronym and does so in a manner that
educates the person to use SMART
goals on their own
The clinician skillfully teaches the
individual the concepts of shaping in
a manner that facilitates skill
development and supports
self-efficacy
The clinician skillfully helps the
individual analyze the factors that
elicit unhealthy behaviours and how
to take control of these stimuli
The clinician is very skilled at introducing
the use of reinforcements to help
support behaviour change and is
able to help the person establish
clear reinforcement contingencies
22 M. Vallis / Can J Diabetes 37 (2013) 18e26

Table 4
Emotion management skills

Emotion Scale score

management skills
Low Moderate High

1 2 3 4 5 6 7
Identifies The clinician does not introduce the The clinician identifies that the unhealthy behaviour serves The clinician is very skilled at introducing
unhealthy concept of unhealthy behaviours a purpose; does not link to replacing the function the idea of unhealthy coping strategies
coping and serving a purpose that in some way and supports the person in replacing
focuses on works well the function as a means of giving up
replacing the the unhealthy behaviour
function
Educates The clinician does not address stress The clinician encourages the individual to manage The clinician is thorough in encouraging
about stress management or pays only lip service to stress and might suggest some specific activities and supporting the individual to use
management it; e.g. just tells the patient to manage but does not do this in a systematic way the range of stress management
stress activities
Refer to mental The clinician does not identify extreme The clinician identifies emotional issues but does The clinician is skillful in identifying
health if emotional issues that are apparent not provide a sensitive referral extreme emotion and working to
appropriate support and facilitate an appropriate
referral

health professionals. Once all of the stimulus material was (85.7%). Almost 40% of the sample were psychologists, almost 25%
generated, all the tapes were independently rated by all 3 experts were dietitians (23.81%), followed by nurses, physiotherapists, as
using the BCCS. well as a pharmacist, social worker and an interdisciplinary trained
professional. Over half of the sample (52.37%) had more than
10 years experience with the remainder having either 5 to 10 years
Data analysis
experience or being a senior PhD Psychology student. Almost 40%
of the sample had extensive training with the BCI.
The items of the BCCS were scored on 1e7 Likert scales and
items were tallied into 4 subscales, reflecting the 3 skill-sets:
motivational interviewing and readiness assessment, behaviour Scale characteristics
modification, and emotion management. Data are presented on:
Item characteristics
1. Item characteristics. The mean, standard deviation (SD) and Table 6 presents the item means, SD and skewness scores for
distribution of scores (skewness) for each item were calculated. each item and subscale of the BCCS. The purpose of presenting data
Poorly performing items would be dropped from the subscale. at the level of the item is that training in behaviour change coun-
2. Subscale characteristics. The mean score and SD was calculated selling is at the individual skill level. As such psychometric prop-
for each subscale. As well, internal consistency was calculated erties should also be evaluated at this level. Table 6 indicates that
using the alpha statistic. most of the items perform well. The exceptions were roll with
3. Interrater reliability. For each subscale intraclass correlation resistance and avoid argument within motivational interviewing,
coefficients (ICC) were calculated. The ICC takes into consider- and making appropriate referrals within emotion management.
ation multiple raters and absolute differences between raters. These behaviours occurred infrequently (once for making a referral,
Second, the interrater reliability was evaluated by calculating twice for roll with resistance and 3 times for avoid argument) and
correlation coefficients between each rater pair for each item. were dropped from their subscales. All other items showed positive
4. Testeretest reliability. After the completion of the initial
Table 5
ratings, the videotapes were set aside and, 6 months later, were Professional participants generating stimulus material
rerated by the same experts to generate testeretest reliability
Participants
estimates using correlation coefficients for each rater. Reli-
ability coefficients were calculated for items and subscales. n %
5. Construct validity. Competency scores were compared Gender
between those in the minimally and extensively trained group. Male 3 14.3
Female 18 85.7
Finally, the BCI recently completed a training program
Discipline
involving 16 primary healthcare providers (6-month training) Psychologists 8 38.10
and had available videotapes of a standardized interview pre- Dietitians 5 23.81
and posttraining. These tapes were rated with an earlier form Nurses 3 14.29
of the BCCS to provide preliminary data on whether the BCCS is Physiotherapists 2 9.52
Pharmacists 1 4.76
sensitive to training initiatives. Social workers 1 4.76
Interdisciplinary studies 1 4.76
This protocol was reviewed and approved by the Research Ethics Professional experience
Board of Capital Health, Halifax. 20 years þ 2 9.52
10e20 years 9 42.85
5e10 years 4 19.05
Results <Senior PhD Student 2 years 6 28.57
Extent of training with BCI*
High 8 38.09
Sample Low 13 61.91

* High, or extensive training, was defined as having at least 2 full days of training
Twenty-one health professionals were recruited into the study. by the BCI with at least 6 months of ongoing follow-up supervision. Low, or minimal
The characteristics of the professional participants are summarized training, was defined as having no more than 2 days of training and no follow-up
in Table 5. The majority of participants in this study were women supervision by the BCI.
 M. Vallis / Can J Diabetes 37 (2013) 18e26 23

Table 6 motivational interviewing and behaviour modification the alpha

Scale item characteristics index is above 0.8, indicating that the items of the subscale are
BCCS item Mean score SD Skew Subscale highly intercorrelated (measure the same thing). This was not true
(2 normal) alpha for readiness assessment, where the intercorrelation among items
Motivational interviewing 3.25 1.26 0.61 0.832 was moderate. As well, intercorrelations for the emotion manage-
Questioning 4.90 1.43 0.89 ment subscale was low. This suggests that within readiness
Express empathy 3.33 1.71 0.05
assessment and emotion management skills are somewhat inde-
Nonjudgemental curiosity 3.30 1.67 0.30
Roll with resistance* d d d pendent (they do not vary together).
Avoid argument y d d d
Support self-efficacy 1.50 1.40 3.24 Reliability
Readiness assessment 3.34 1.14 0.278 0.632 Table 7 presents the subscale and item interrater reliability for
Define behaviour 3.33 1.76 0.03
Perceived problem 3.51 2.06 0.02
the BCCS. The ICC is a single statistic that combines raters and
Perceived distress 3.22 2.08 0.02 considers absolute differences between raters. These results indi-
Interest in change 3.27 1.84 0.01 cate very high interrater reliability for all subscales. ICC coefficients
Ready to change now 3.70 2.29 0.22 ranged from 0.951 to 0.990 for the subscales. Similarly high inter-
Gives readiness feedback 2.59 2.08 1.11
rater subscale reliability was found for all of the rater pairs, with the
Behaviour modification 2.82 1.05 0.636 0.814
Goal setting 3.70 1.65 0.58 range of reliability indices from 0.928 to 0.999. Finally, the indi-
Shaping 3.14 1.70 0.78 vidual item reliability coefficients between the rater pairs was high.
Stimulus control 2.60 1.64 0.87 Of the 48 rater pairs, 40 coefficients were above 0.90 and the lowest
Reinforcement management 2.23 2.04 1.36 interrater reliability estimate was 0.811 (p < 0.001 for all).
Emotion management 2.36 1.20 1.87 0.405
Unhealthy coping 2.14 1.60 1.29
Table 8 presents the testeretest reliability of the BCCS over
Stress management 2.58 0.99 0.52 a 6-month interval. Overall the testeretest reliability for the
Referral z d d d subscales was high (p < 0.001). For motivational interviewing skills
SD, standard deviation. it was 0.801, with the lowest individual rater reliability being 0.769.
* This item dropped from the subscale score because it was rated on only 2 Assessing readiness yielded an overall testeretest reliability of
interviews. 0.867, with individual testeretest reliabilities also high (lowest
y
This item dropped from the subscale score because it was rated on only 3 0.783). The behaviour modification subscale yielded an overall
interviews.
z
This item dropped from the subscale score because it was rated on only 1
testeretest reliability of 0.902, with the lowest individual reliability
interview. being 0.828. The overall testeretest reliability for the managing
emotions subscale was 0.799, with the lowest individual rater
characteristics. Mean scores on a 1 to 7 scale were generally reliability being 0.717. The stability of the individual scale items was
approximately 3.5, with SDs approximately 1.5 and skewness also acceptable (p < 0.006 for all except one significant at p ¼ 0.01).
estimates within the accepted 2 criterion. The exception to this Of the 48 individual rater reliability coefficients most were above
was questioning, which was rated high (4.9/7) and support 0.7. The least stable items (below 0.7) were asking questions
self-efficacy, which was rated low (1.5/7). Further, support (motivational interviewing skill), and assessing interest in change
self-efficacy was the only item that was skewed (3.24), which was and defining behaviour (readiness assessment).
reflected in that 90% of the scores were 2 or below.
Subscale score characteristics are also reported in Table 6. All Construct validity
subscales appear to be normally distributed (within limit skewness Although this study was designed to operationalize the
indices) and mean scores are centrally distributed with SDs just component skills involved in behaviour change counselling and
over 1. Internal consistency of the items on the subscales vary. For evaluating the item/subscale characteristics and reliability of the

Table 7
Interrater reliability (ICC and r) of the Behaviour Change Counselling Scale

Raters combined Rater pairs Item Rater pairs

Subscale (ICC) Correlations (r) Correlations (r)

1e2 1e3 2e3 1e2 1e3 2e3

Motivational interviewing 0.960* 0.960* 0.999* Asking questions 0.865* 0.869* 0.990*
(0.970)*
Expresses empathy 0.884* 0.879* 0.993*
Nonjudgemental curiosity 0.948* 0.956* 0.992*
Supports self-efficacy 0.982* 0.998* 1.00*
Readiness assessment 0.987* 0.998* 0.993* Define behaviour 0.972* 0.984* 0.984*
(0.990)*
Perceives problem 0.973* 0.983* 0.999*
Distress experience 0.975* 0.984* 0.990*
Interest in change 0.948* 0.952* 0.979*
Ready to change now 0.991* 0.996* 0.996*
Traffic light feedback 0.983* 0.983* 1.00*
Behaviour modification 0.972* 0.976* 0.995* Goal setting 0.951* 0.968* 0.983*
(0.981)*
Behaviour shaping 0.815* 0.811* 0.983*
Stimulus control 0.943* 0.962* 0.944*
Reinforcement manage 0.995* 0.995* 1.00*
Manage emotions 0.938* 0.939* 0.988* Unhealthy coping 0.886* 0.886* 1.00*
(0.955)*
Stress management 0.967* 0.974* 0.967*

* p < 0.001.
24 M. Vallis / Can J Diabetes 37 (2013) 18e26

Table 8
Testeretest reliability (r) of the Behaviour Change Counselling Scale

Raters combined (r) Rater Item Rater

Subscale 1 2 3 1 2 3
Motivational interviewing (0.801)* 0.769* 0.793* 0.811* Asking questions 0.684y 0.655y 0.671y
Expresses empathy 0.636z 0.770* 0.794*
Nonjudgemental curiosity 0.780* 0.732* 0.741*
Supports self-efficacy 0.724* 0.735* 0.713*
Readiness assessment (0.867)* 0.783* 0.854* 0.883* Define behaviour 0.639z 0.660y 0.698y
Perceives problem 0.605z 0.724* 0.676y
Distress experience 0.719* 0.760* 0.827*
Interest in change 0.595z 0.543jj 0.566x
Ready to change now 0.715* 0.843* 0.844*
Traffic light feedback 0.786* 0.836* 0.817*
Behaviour modification (0.902)* 0.936* 0.828* 0.897* Goal setting 0.856* 0.724* 0.800*
Behaviour shaping 0.763* .492 0.607z
Stimulus control 0.677y 0.605z 0.752*
Reinforcement manage 0.972* 0.951* 0.956*
Manage emotions (0.799)* 0.851* 0.766* 0.717* Unhealthy coping 0.842* 0.720* 0.674z
Stress management 0.881* 0.852* 0.826*

* p < 0.001.
y
p < 0.002.
z
p < 0.003.
x
p < 0.005.
jj
p < 0.006.

scale some data were collected that can address the validity of the significant increase in skills at the end of the training compared to
scale. As the study was not designed to assess validity these anal- the beginning (p < 0.001).
ysis should be considered post-hoc and preliminary. For this reason
only subscale scores were analyzed. The participants in this study Discussion
were categorized into high and low categories based on their
involvement in the BCI. In Table 9 the average skill rating on each of The evidence suggests that healthy lifestyle behaviours play
the BCI Counselling Scale subscales is presented. Those in the high a strong role in disease and health, and that lifestyle change is
training group scored significantly higher on the motivational possible. This evidence justifies knowledge translation efforts to
interviewing subscale than those in the low trained group. empower primary and tertiary care clinicians with lifestyle coun-
Although scores on the readiness assessment and the emotion selling skills; i.e. to become an effective behaviour change agents to
management subscales were also higher for those in the high operationalize their self-management support role. This means
trained group these differences were not significant. There was no that in addition to evidence-based recommendations, advice and
difference between groups on the behaviour modification subscale. education, clinicians need to be able to help patients find the
Table 9 shows the prepost training scores on an earlier version motivation to change when it does not exist, help patients make the
of the BCCS in a group of 16 primary care health providers (nurses, change when motivated, and help patients maintain the change
dietitians and social workers). At the time that this training was once it has been made. For this to happen there needs to be training
carried out, only the motivational interviewing and readiness programs and these programs need to operationalize their training
assessment scales had been fully developed. For both motivational models and demonstrate that what is trained can actually be
interviewing and readiness assessment that was a strong learned. This article reports on a scale developed to measure
competency within a specific training model.
The creation of a reliable and operationalized behaviour change
Table 9
Subscale score comparisons across those with low and high training/prepost counselling scale will be helpful in guiding the acquisition of
training from BCI behaviour change competencies. First, the scale helps novices
Current study sample (n ¼ 21) Low training High training p value
become familiar with the component skills involved in effective
counselling. Second, the scale can be used to ensure that skills
Motivational interviewing
Mean score 2.23 3.88 p ¼ 0.001 learned meet competency criteria. Tables 1e4 define the specific
SD 0.78 1.09 p ¼ 0.001 skills and operationalize the characteristics that define low,
Readiness assessment moderate or high skill levels.
Mean score 3.12 3.47 ns Our data suggests that the BCCS is a potentially useful scale to
SD 1.36 1.03 ns
guide training and competency assessment in behaviour change
Behaviour modification
Mean score 2.82 2.82 ns counselling. Generally, each of the subscale items are psychomet-
SD 1.02 1.12 ns rically sound and contribute to their respective subscales. The
Emotion management exceptions were roll with resistance and avoid argument items of
Mean score 1.95 2.60 ns
the motivational interviewing subscale. Within the interviews with
SD 0.57 1.42 ns
the simulated patients there was almost no opportunity to rate
Independent sample prepost Pretraining Posttraining these skills. The same occurred for the referral item in the emotion
6 month training (n ¼ 16)
management subscale. This result does not mean that these items
Motivational interviewing
are not reliable. It does mean that specific stimulus material needs
Mean 2.91 5.00 p < 0.001
SD 1.30 1.10 p < 0.001 to be created on which to test these items. In this study these items
Readiness assessment were dropped from the subscales for psychometric reasons.
Mean 2.73 5.18 p < 0.001 At the level of the subscales, motivational interviewing, readi-
SD 1.19 1.54 p < 0.001 ness assessment, behaviour modification and emotion manage-
SD, standard deviation. ment all showed strong psychometric characteristics. The measure
 M. Vallis / Can J Diabetes 37 (2013) 18e26
of internal consistency, which evaluates the extent to which
subscales items interrelate showed that the items on the readiness
assessment and emotion management do not corelate well, which
would suggest that the skills within these subscales are separate
skills. If one compares the behaviour modification with the readi-
ness assessment subscales the data from this study suggest that the
behaviour modification skills are learned in an integrated fashion
(high alpha), such that a person skilled in one area (goal setting, for
instance) would also be skilled in the other areas (behaviour
shaping, for instance). This was not the case for the readiness
assessment or emotion management subscales, suggesting that
each of these subscale skills are learned separately. This raises the
issue as to whether or not the items belong in a single scale. It
might be the case that these skills should be considered as separate.
These data question, for instance, if addressing unhealthy coping
strategies and stress management should be considered as part of
a cohesive skill set (emotion management) or as separate skills.
Within the BCI research program we intend to further evaluate this
issue as we build a sample size necessary to make this evaluation.
For instance, a factor analysis of the items within the subscales will
be informative. This will allow us to blend rational and empirical
methods of scale development.
The advantage of examining the psychometric properties of the
individual items of the BCCS is that this allows the scale to be used
as a guide for training. Operationalizing and measuring the skills
will provide direct corrective feedback for learners.
The data on the subscale scores is also very strong. The subscales
demonstrate sound psychometric properties. Importantly, the data
on interrater reliability and testeretest reliability are strong. When
used by experts in behaviour change counselling training the
ratings across raters was very consistent which suggests that the
scale can be used by different trainers. This is necessary if the scale
is to be used by other groups interested in training similar behav-
iour change counselling skills. As well, the stability of the ratings
over time was high. After a period of 6 months the raters replicated
their ratings very well. This is important because if the scale is to be
used to index changes in competency over time it must be
demonstrated that the scale will yield the same results if there is no
change in competency. Because the exact same videos were rated 6
months later in this study we know that the stimulus material did
not change. High testeretest reliability coefficients tells us that the
competency ratings were also stable over time.
This study analyzed some limited data to examine competency
scores between those with less and more training, and in a separate
sample, before and after training. These results were mixed. In the
sample where there was pre- and posttraining data significant
improvements were seen on both scales evaluated; motivational
interviewing and readiness assessment. In the sample used for item
evaluation, in which clinicians were divided into high and low
training only motivational interviewing scores were significantly
higher in the highly trained group. This might reflect lack of
responsiveness to training, ineffective training or lack of validity of
the categorization. These data should be interpreted cautiously as
a result. The decision was made to include these data because they
were available and are hard to obtain. It must be understood that
the data are of low quality, at least the current study sample.
The BCCS was developed as opposed to modifying existing
scales. This path was chosen so that the scale would line up directly
with the training model. This will increase the ability of the scale to
be used not only as a means of assessing outcomes of training but
also as a means of educating learners about the component skills of
training.
The feasibility of the BCCS is worth considering. Although the
rigour associated with item operationalization and measurement is
a methodological strength this level of detail impacts on feasibility.
In it’s current form, expert trainers viewing a total of 36 minutes of
25
video complete the BCCS. This might be challenging given the lack
of expert trainers. Although currently a limitation, future research
can examine whether there is an acceptable short-form of the scale.
As well, ratings of audiotapes and determining a minimal length of
interview to rate are useful questions to address. For this reason
this study needs to be considered preliminary.
This study is also preliminary in that additional data are
required on the cut score for indexing competency in behaviour
change counselling and on the testeretest reliability of the scale. As
well, further data showing relationships between competency and
outcomes are required. It is hoped that the BCCS will be of interest
to the field as it will allow a framework to better define the skills of
behaviour change counselling. There will be value in understanding
the skills that are most associated with successful change. The
limited sample size in this study should also be noted. Replication
on broader samples would be of benefit.
Acknowledgments
I would like to acknowledge the contributions of Jennifer
Hayley, Jacklynn Humphrey and Holly Dempsey in the ratings of
the videotapes and the data management activities. This work
was funded by a grant from the Capital Health Research Fund.
References
1. Daar AS, Singer PA, Persad DL, et al. Grand challenges in chronic non-
communicable diseases. Nature 2007;450:494e6.
2. Prochaska JJ, Spring B, Nigg CR. Multiple health behavior change research: an
introduction and overview. Prevent Med 2008;46:181e8.
3. Katzmarzyk PT, Janssen I, Morrison KM, Tremblay MS. Classification of
overweight and obesity in children and adolescents. Can Med Assoc J 2007;
176(Suppl 8):27e32.
4. Katzmarzyk PT, Ardern CI. Overweight and obesity mortality trends in Canada,
1985-2000. Can J Public Health 2004;95:16e20.
5. Heart and Stroke Foundation. The growing burden of heart disease and stroke
in Canada. Ottawa; Canada 2003.
6. Woolf SH. The power of prevention and what it requires. JAMA 2008;299:2437e9.
7. Reeves MJ, Rafferty AP. Healthy lifestyle characteristics among adults in the
United States, 2000. Arch Intern Med 2005;165:854e7.
8. Tamakoshi A, Tamakoshi K, Lin Y, et al. Healthy lifestyle and preventable death:
Findings from the Japan Collaborative Cohort (JACC) Study. Prev Med 2009;48:
486e92.
9. Ford ES, Bergmann MM, Kroger J, et al. Healthy living is the best revenge:
Findings from the European Prospective Investigation Into Cancer and
Nutrition-Potsdam study. Arch Intern Med 2009;169:1355e62.
10. King DE, Mainous 3rd AG, Geesey ME. Turning back the clock: adopting
a healthy lifestyle in middle age. Am J Med 2007;120:598e603.
11. Knowler WC, Barrett-Connor E, Fowler SE, et al. Reduction in the incidence of
type 2 diabetes with lifestyle intervention or metformin. N Engl J Med 2002;
346:393e403.
12. Gillies CL, Abrams KR, Lambert PC, et al. Pharmacological and lifestyle inter-
ventions to prevent or delay type 2 diabetes in people with impaired glucose
tolerance: Systematic review and meta-analysis. BMJ 2007;334:299e308.
13. Li G, Zhang P, Wang J, et al. The long-term effect of lifestyle interventions to
prevent diabetes in the China Da Qing Diabetes Prevention Study: a 20-year
follow-up study. Lancet 2008;371:1783e9.
14. Galuska DA, Will JC, Serdula MK, Ford ES. Are health care professionals advising
obese patients to lose weight? JAMA 1999;282:1576e8.
15. Miller WR, Rose GS. Toward a theory of motivational interviewing. Am Psychol
2009;64:527e37.
16. Britt E, Hudson SM, Blampied NM. Motivational interviewing in health
settings: A review. Patient Educ Counsel 2004;53:147e55.
17. McGowan P. The efficacy of diabetes patient education and self-management
education in type 2 diabetes. Can J Diabetes 2011;35:46e53.
18. Glasgow RE, Fisher EB, Anderson BJ, et al. Behavioral science in diabetes.
Contributions and opportunities. Diabetes Care 1999;22:832e43.
19. Glasgow RE, Hiss RG, Anderson RM, et al. Report of the health care delivery
work group: Behavioral research related to the establishment of a chronic
disease model for diabetes care. Diabetes Care 2001;24:124e30.
20. Alexander SC, Ostbye T, Pollak KI, et al. Physicians’ beliefs about discussing
obesity: Results from focus groups. Am J Health Promot 2007;21:498e500.
21. Ampt AJ, Amoroso C, Harris MF, et al. Attitudes, norms and controls influencing
lifestyle risk factor management in general practice. BMC Fam Pract 2009;10:59.
22. Moyers T, Martin T, Manuel J, Miller W. The Motivational Interviewing
Treatment Integrity (MITI) Scale. Version 2.0. Albuquerque (NM): Center on
Alcoholism, Substance Abuse and Addictions, University of New Mexico;
2003.
26 M. Vallis / Can J Diabetes 37 (2013) 18e26
23. Moyers T, Martin T, Catley D, et al. Assessing the integrity of motivational
interventions: Reliability of the Motivational Interviewing Skills Code. Behav
Cognit Psychother 2003;31:177e84.
24. Lane C, Huws-Thomas M, Hood K, et al. Measuring adaptations of motivational
interviewing: the development and validation of the behavior change coun-
seling index (BECCI). Patient Educ Counsel 2005;56:166e73.
25. Leffingwell TR. Motivational Interviewing Knowledge and Attitudes Test
(MIKAT) for evaluation of training outcomes. MINUET 2006;13:10e1.
26. Rosengren DB, Baer JS, Hartzler B, et al. Video assessment of simulated
encounters (VASE-R): Administration and scoring manual. Seattle (WA):
University of Washington; 2005.
27. de Jonge JM, Schippers GM, Schaap CPDR. The Motivational Interviewing Skill
Code: Reliability and a critical appraisal. Behav Cognit Psychother 2005;33:
285e98.
28. Keszei A, Novak M, Streiner D. Introduction to health measurement scales.
J Psychosomat Res 2010;68:319e23.
29. Rubak S, Sandbaek A, Lauritzen T, Christensen B. Motivational inter-
viewing: A systematic review and meta-analysis. Br J Gen Pract 2005;55:
305e12.
30. Holt DT, Helfrich CD, Hall CG, Weiner BJ. Are you ready? How health profes-
sionals can comprehensively conceptualize readiness for change. J Gen Intern
Med 2010;25(Suppl 1):50e5.