Running head: THE ONLINE QUEST FOR RECOVERY SCIENCE CREDITS

The Online Quest for Recovery Science Credits:

Comparing the Impact of Instruction Methods on Achievement in Grade 10 Repeater
Science
Danielle F. Couture
University of British Columbia

THE ONLINE QUEST FOR RECOVERY SCIENCE CREDITS

Abstract
This study contributes to the growing field of knowledge characterizing the effects of online
learning on student achievement by focusing on K-12 students obtaining recovery credits. This 4
month experimental pre-test post-test design study will compare the achievement level on the
final provincial science exit exam of 30-40 grade 10 science students enrolled in a face-to-face
and online two week repeater summer course. Demographic surveys, NASA TLX and ESPRI
measures completed by participants will help isolate the impact of instruction method on
achievement results. This study hopes to shed light on the affordances that online education has
to offer for credit recovery in regards to the achievements of grade 10 science students to better
support policy makers in their decisions.
Key words: Online learning, high school, K-12, recovery credit

THE ONLINE QUEST FOR RECOVERY SCIENCE CREDITS

The Online Quest for Recovery Science Credits:

Comparing the Impact of Instruction Methods on Achievement in Grade 10 Repeater Science
Objective and Purpose of the study
The objective of this four month study is to assess the impact of online asynchronous
learning on the achievement levels of grade 10 repeater-science students on the Qubec
provincial science exit exam. A comparison between the achievement levels of students
receiving face-to-face instruction and those of students receiving online asynchronous instruction
will be performed based on a two week observation period. This research is designed to focus on
assessing the impact of the instruction methods and ultimately to address the following
questions:
1. What are the differences in achievement levels on the provincial grade 10 science exit
exam between Qubec grade 10 repeater-science students who received face-to-face
versus online asynchronous instruction?
2. To what extent does the method of instruction account for these results?
Answering these questions will serve to further the knowledge of policy makers in the
province of Qubec.
Perspectives and Theoretical Framework
Distance education has changed a lot since the rise of the internet and advent of social
media. Currently noted for its flexibility and low operational cost (Corry & Carlson-Bancroft,
2014), online learning has grown in popularity. Yet, of the numerous studies on the topic, very
few have actually focussed on isolating the method of instruction (Emerson & MacKay, 2011).
Due to the complex nature of learning, seldom will a research be able to discriminate all
variables at play and observe absolute cause and effect relationships. Studies on online learning

THE ONLINE QUEST FOR RECOVERY SCIENCE CREDITS

often focus on the implications of perspectives and motivation on the learning achievement (e.g.,
Hughes, McLeod, Brown, Maeda, & Choi, 2007; Johnson, Aragon, Shaik, & Palma-Rivas,
2000). To isolate the impact of the instruction method, one needs to limit existing confounding
variables, such as the material covered and instructor differences (Emerson & MacKay, 2011).
Furthermore, to stay true to the method of instruction, one must consider the affordance of each
and not try to determine whether one is better than the other; both have their own merits
(Johnson et al., 2000).
When it comes to assessing the merits or limitations of one form of instruction over
another, studies are quite divided (Nora & Snyder, 2008). Certain studies show the positive
impact of online education in current (e.g., Hughes et al., 2007; Institute of Education, 2015;
Lim, Kim, Chen, & Ryder, 2008) and future (e.g. Dodd, Kirby, Seifert, & Sharpe, 2009)
achievement levels. Others have found that online learning has in fact lead to lower achievement
levels (e.g. Emerson & MacKay, 2011) or gave way to a poorer perception of learning (e.g.,
(Oliver, Osborne, Patel, & Kleiman, 2009). Still some studies state that there exists no notable
difference between the two methods of instruction (e.g., Liu & Cavanaugh, 2011). Suffice to say
that comparing the results of face to face instruction with that of distant education is not a novel
or a unique concept. Yet, in spite of the raising enrollment in K-12 institution for recovery
credits (Barbour & Siko, 2012; Corry & Carlson-Bancroft, 2014), very few studies cover this
population subgroup. In fact many studies cover the opposite (Barbour & Siko, 2012), such as
accelerated programs or post-secondary education.
This 4 month study hopes to add to the growing body of research by focussing on grade
10 students completing recovery credits and to further the knowledge of policy makers in the
province of Qubec. To limit the impact of confounding variables, this study hopes to make use

THE ONLINE QUEST FOR RECOVERY SCIENCE CREDITS

of standardized tests, which will regulate achievement levels in both groups (Hughes et al.,
2007), and of other measures such as the NASA-TLX (Emerson & MacKay, 2011), which
accounts for perceived stress, and the ESPRI (Roblyer & Marshall, 2002), which predicts success
rate in online learning test. This timeframe was selected to decrease the threat of attrition; a
constant issue in online education (Corry & Carlson-Bancroft, 2014).
Description of Methods
A pretest post-test control group experimental design will be followed during this study.
Descriptive and inferential statistical analysis will be performed to summarize collected data and
to assess statistical correlations between the method of instruction and the achievement level on
the ST 406 exam. A passing grade in the province of Qubec is set at 60%.
Participants
A group between 30 - 40 students will be recruited from the list of students identified by
the guidance counselors at the Riverside School Board as being at risk of failing the grade 10
science (ST 406) provincial exit exam. ST 406 is a high school graduation requirement.
Riverside School Board has been selected due to its socio-economic and cultural diversity, large
population and relatively limited geographical location. Inclusion criteria specifies that
participants should be between the ages of 15-17, have attended within the last year ST 406, be
able to travel to the location of the face-to-face course and the computer lab should the need arise
and be eligible for summer school (have received a final grade for ST 406 between 30% and
59%). From this group, participants will be selected and distributed randomly into the control
group and the experimental group.
Instruments

THE ONLINE QUEST FOR RECOVERY SCIENCE CREDITS

Final provincial exit exam (Standardized test). The provincial exit exam is a
standardized test administered in a 3 hour window. This test is comprised of 25 questions each
worth 4 points; 15 multiple choice questions and 10 short answer questions. During the
completion of the exam a silent 2 minute video is played in constant loop to offer guidance on 4
of the short answer questions. The multiple choice questions are graded by Scan-tron, whereas
the short answer questions are graded by the subject teachers. Each teacher receives a detailed
rubric and participates in a marking center for standardization purposes. All exams are collected
and a random selection is verified by the Ministry for grading consistency.
Measures
Subjective cognitive workload test (NASA task load index or NASA TLX). The
NASA TLX (Hart & Staveland, 1988) evaluates perceived workload and stress using a 21
gradation scale on five criteria: mental demand, physical demand, temporal demand,
Performance, effort and frustration). A question accompanies each of the criteria to help
participants reflect on their situation.
Educational success prediction instrument (ESPRI). The ESPRI (Roblyer & Marshall,
2002) is a 70 item instrument which focusses on four factors of student success in virtual high
school (VHS): Achievement and self-esteem beliefs, responsibility risk taking, technology skills
and access, organization and self-regulation. Each item is evaluated using a seven point Likert
scale. It is designed to identify students that would require extra support to succeed in a digital
learning environment.
Descriptive Demographic Survey. A descriptive survey will be completed by the
participants and provide an indication of their age, gender and mother tongue. Their online

THE ONLINE QUEST FOR RECOVERY SCIENCE CREDITS

habits, attitudes towards science and attitudes towards both instruction methods will be assessed
using a 5 point Likert scale.
Procedures
Ethics approval will be obtained through the Research Ethic Board (REB). Participants
will be recruited from the Riverside School Board in Qubec. In mid-April, the cover letter
guaranteeing confidentiality, detailing the nature of the study, the potential benefits and risk and
asking for parental assent will be sent along with the school boards letter regarding summer
science school to the parents of students at risk of failing ST 406. Participants will be asked to
return the signed assent form along with the summer school registration in mid-June. The
official list of summer students will be generated on July 11th after the final course grades are
received. Of those who showed interest in participating and matched the inclusion criteria, a
group of 30-40 students will be randomly selected to take part in the study. Those not selected
will be placed in another group or in one of the existing groups; their results will not be
considered. All participants will meet on July 13th to fill in the descriptive survey and to receive
a brief training on the online-learning system. All participants will also be able to meet the
teacher and familiarize themselves with the computer lab at everyones disposal. At the end of
their orientation students will be randomly distributed into the control and experimental group.
Both courses will consist of 40 hours of material and will take place from July 14th to July 27th.
Both NASA TLX and ESPRI will be administered on the 1st and last day of class to verify testvalidity, assess change and attempt to control confounding variables. The provincial exit exam
will be written on July 29th. Results from both standardized tests (June and July) will be gathered
from the school board on August 16th.

THE ONLINE QUEST FOR RECOVERY SCIENCE CREDITS

Data Analysis
Descriptive statistical analysis in the form of frequency distributions will be used to
summarize the data and compare the equivalency of the two groups. Measures of central
tendency and variability will also be used to view the manner in which the data is spread.
Inferential statistical analysis in the form of an independent samples t-test will be used to test the
following hypothesis:
H0: The method of instruction will have no effect on the achievement level of grade 10
repeater science students on the provincial science exit exam.
The data collected by the completion of the July ST 406 exam will be compared to the
grades of the June exam through multiple regression analysis.
Description of Data Sources
The demographic survey will combine nominal scales (e.g. male =1, female =2) with
ordinal scales in which students self-rank their attitudes towards science and past experiences
ranging from strongly agree (represented by 1) to strongly disagree (represented by 5). The
NASA TLX and ESPRI will be scored using a seven point Likert scale. The results obtained
from the criterion-referenced provincial exit exam will be scored as a percentage.
Results and Conclusions
As existing research assessing the impact of online learning on achievement are at times
contradictory, predictions cannot be made on the expected results of this study. It is expected
however, that the gathered quantitative data will inform policy makers regarding the affordances
and limitations of the application of online learning to credit recovery. Due to the limited short

THE ONLINE QUEST FOR RECOVERY SCIENCE CREDITS

time frame of this study, the validity threats of attrition, history and maturation are lessened.
Through the analysis of ESPRI and NASA TLX we hope to be able to isolate variables and
reveal the relationship between the dependant and independent variables in spite of the limited
exposure to the method of instruction.
Educational Significance
Currently, many school boards are no longer offering recovery credits unless the student
numbers warrant a course. As such, students are often forced to retake an 8 month course,
delaying their graduation an entire year. Furthermore, such a policy disadvantages rural regions
which might not have the numbers or resources to staff a summer class. Providing an online
option for recovery credits is therefore a logical option. However studies need to be conducted
to ensure, at the very least, equivalency with current in place face-to-face practices. Considering
the sparse amount of previous research focussing on the K-12 repeater population, the
assessment of online education proposed in this study represents a significant contribution by
providing support for informed policy decisions and by acting as a starting point for large-scale
studies.

References

THE ONLINE QUEST FOR RECOVERY SCIENCE CREDITS

Barbour, M. K., & Siko, J. P. (2012). Virtual schooling through the eyes of an at-risk
student: A case study. European Journal of Open, Distance and E-Learning,
15(1).
Corry, M., & Carlson-Bancroft, A. (2014). Transforming and Turning around LowPerforming Schools: The Role of Online Learning. Journal of Educators Online,
11(2).
Dodd, C., Kirby, D., Seifert, T., & Sharpe, D. (2009). The impact of high school
distance e-learning experience on rural students university achievement and
persistence. Online Journal of Distance Learning Administration, 12(1).
Emerson, L., & MacKay, B. (2011). A comparison between paperbased and online
learning in higher education. British Journal of Educational Technology, 42(5),
727-735.
Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index):
Results of empirical and theoretical research. Advances in psychology, 52,
139-183.
Hughes, J. E., McLeod, S., Brown, R., Maeda, Y., & Choi, J. (2007). Academic
achievement and perceptions of the learning environment in virtual and
traditional secondary mathematics classrooms. The American Journal of
Distance Education, 21(4), 199-214.
Institute of Education, S. (2015). Comparing Success Rates for General and Credit
Recovery Courses Online and Face to Face: Results for Florida High School
Courses : Institute of Education Sciences, Regional Educational Laboratory
Program Reports;2015 ASI 4816-7.159;REL 2015-095. Johnson, S. D., Aragon,
S. R., Shaik, N., & Palma-Rivas, N. (2000). Comparative analysis of learner

THE ONLINE QUEST FOR RECOVERY SCIENCE CREDITS

satisfaction and learning outcomes in online and face-to-face learning
environments. Journal of interactive learning research, 11(1), 29.
Lim, J., Kim, M., Chen, S. S., & Ryder, C. E. (2008). An empirical investigation of
student achievement and satisfaction in different learning environments.
Journal of Instructional Psychology, 35(2), 113.
Liu, F., & Cavanaugh, C. (2011). Success in online high school biology: Factors
influencing student academic performance. Quarterly Review of Distance
Education, 12(1), 37.
Nora, A., & Snyder, B. P. (2008). Technology and higher education: The impact of elearning approaches on student academic achievement, perceptions and
persistence. Journal of College Student Retention: Research, Theory &
Practice, 10(1), 3-19.
Oliver, K., Osborne, J., Patel, R., & Kleiman, G. (2009). Issues surrounding the
deployment of a new statewide virtual public school. Quarterly Review of
Distance Education, 10(1), 37-49.
Roblyer, M., & Marshall, J. C. (2002). Predicting success of virtual high school
students: Preliminary results from an educational success prediction
instrument. Journal of Research on Computing in Education, 35(2), 241-255.