Methodology

In order to test the hypothesis that the implementation of an eLearning

platform will have an effect on volunteer engagement, a quasi-experimental pretest-

posttest without control quantitative study will be undertaken. Volunteers within
the St. John Ambulance MFR unit will complete a survey designed to measure
engagement both prior to the initiation of the eLearning platform and six weeks
after. The pretest scores can then be compared against the posttest scores as well as
normalized data from an international database of survey responses.
Participants
The volunteer members of the SJA MFR unit at the Cambridge, Ontario
branch are men and women between 18 and 65 years of age. All members of this
unit hold current first aid certification (which is the topic being communicated in
the eLearning platform) and some hold higher related certifications ranging from
medical first responder to paramedic. Members have a wide variety of educational,
ethnic, religious, and cultural backgrounds. The unit reported 40 members at the
time of the initiation of this study, which represents the population from which
participants are recruited. Recruitment and participation in this study is performed
online through e-mail and a dedicated survey website. All 40 members receive an e-
mail invitation (Appendix A) to participate in the study anonymously and, thus, the
specific demographics of the sample cannot be communicated beyond the broad
demographics of the population. There is no incentive offered to potential
participants for this study.

Potential participants who identify that they are not members of the MFR
unit, do not have access to e-mail, do not have basic computer skills, or cannot read
and understand English are not permitted to access the survey study and are
excluded.
Measures
The shortened, nine question version of the Utrecht Workplace Engagement
Survey (UWES-9) is chosen as a measure of engagement, both prior to the
implementation of the eLearning platform and six weeks after its implementation.
The UWES-9 uses a six point Likert scale to assess the degree to which the
respondent agrees with each statement. Each numeric point is assigned a descriptor
(1=Almost never, 2=Rarely, 3=Sometimes, 4=Often, 5=Very Often, 6=Always). The
nine statements being assessed represent three questions from each of the three
dimensions of engagement: vigor, dedication and absorption. Although timeframe
dimensions were originally attached to the Likert values in the creation of this
survey (e.g. 1/Almost never/once a year or less, etc.), they were not used due to the
episodic nature of volunteering within the community being studied. The survey is
further changed for the volunteer population by modifying phrases that refer to
work in the questions to reflect the particular work being assessed (e.g. At my
work, I feel bursting with energy became At my volunteer work, I feel bursting with
energy).
Tabulating scores from each of the nine statements is completed by first
calculating the mean of each of the group of three questions assessing the
dimensions of engagement: dedication, absorption and vigor. The means of each of

the dimensions are then combined to reveal a mean score of engagement for the
respondent. Collecting data in this way reveals how two respondents could be
equally engaged but have significantly different engagement experiences.
Procedures
On December 18, 2014, a recruitment e-mail (Appendix A) and letter of
information (Appendix B) was sent to 41 volunteers of the SJA Cambridge MFR unit
through personal e-mail addresses supplied by Cambridge SJA. One reply was
received indicating that the recipient was no longer a member of the MFR unit; his
request to stop receiving any future updates about the study was honored. The
recruitment e-mail was sent again December 28 and January 11, 2015 in an effort to
recruit as many subjects as possible.
Study participants were directed to the questionnaire web site using the URL
specified in the recruitment e-mail (Appendix D). They were presented with the
letter of information and asked to self-identify exclusionary criteria. If they chose to
consent to the survey, they anonymously completed the UWES-9 modified for this
study. The online survey platform left no opportunity to collect personally
identifiable data. Although it was originally designed to allow for a two-week period
of data collection, the pretest period was increased by one week due to a delay in
implementing the eLearning platform by SJA. The pretest round of data collection
stopped automatically on Monday, January 12 at midnight eastern standard time.
The intervention for this study, the MFR units eLearning platform, became
active on Tuesday, January 13, 2015. The eLearning platform, Moodle learning
management system version 2.6.2, was developed for the MFR unit by the principal

investigator in this study. A volunteer within the unit was given administrator
privileges to fulfil the role of system administrator and maintain the daily operation
of the LMS. The LMS was designed to fulfil both administrative and educational
roles for the MFR unit and serves as a repository of reference materials,
announcements for members and operational documents required for the volunteer
members. The educational aspects of the LMS were developed by the Divisional
Training Officer and Training Assistant, both Ontario certified Primary Care
Paramedics. Each educational module was designed to complement the topic being
discussed at weekly meetings occurring between January 13, 2015 and February 24,
2015. Both training developers did not have previous experience creating Moodle
learning modules, nor do they have any formal education on adult learning but they
were the persons responsible for weekly training and had sufficient subject matter
expertise on the content required. During the study period, four training modules
were made available for the entire volunteer membership of the MFR unit. One of
the purposes for the MFR unit to develop the LMS was to provide an alternative
pathway for volunteers to access weekly training subjects. Volunteers who
completed a weekly training module online rather than in class were credited with
one hour of weekly training credit; two hours are typically granted to volunteers
who attend weekly meetings where they are physically co-located. During the study
period, there were no significant changes in the MFR unit outside the eLearning
program.
The posttest round of data collection began on February 24, 2015, exactly six
weeks after the eLearning platform went active. A recruitment e-mail (Appendix C)

and letter of information (Appendix B) was sent to the 40 volunteers of the SJA
Cambridge MFR unit who had previously been contacted for the pretest portion of
the study. A second copy of the recruitment e-mail was sent on March 6, 2015 as a
reminder to complete the final portion of the study. The survey website was an
exact copy of the pretest survey website with the data being collected in a fresh
database.
Survey results were transcoded from the descriptive responses collected
from the website to numeric responses on a scale of one to six. The term almost
never is transcoded to one, rarely is two, sometimes is three, often is four,
very often is five, and always is six. Using this method, means and standard
deviations were calculated from the pretest data and posttest data for comparative
analysis. The raw data for this study is found in Appendix E. The table below
presents the means calculated from averaging the question groups responsible for
assessing vigor, dedication, and absorption, as well as the overall engagement score
calculated from the average of those means.
In order to analyze the results, a students t-test was employed to compare
each statement assessed in each phase of the study from pre-test to post-test results
as well as the means for each of the three groups of statements representing the
dimensions of engagement (dedication, vigor and absorption) and, finally, the
overall combined means for the three dimensions of engagement which includes all
nine statements and represents overall engagement. A descriptive analysis was also
employed and further statistical analysis was completed by Dr. Bill Goodman using
his method of measuring the shifts in distribution (Goodman, 2012).

The anticipated required sample size was calculated based on normative data
presented by Schaufeli & Bakker (2004, p. 40). In order to achieve statistical
significance to reflect a moderate change in engagement (Cohens d=0.5), it was
determined a sample size of 64 was required but not possible given a total
population of 40. A large change in engagement (Cohens d=0.8), reflected by a
change in the mean score of 0.936, can be said to be statistically significant if a
sample size of 19 is achieved.