Educational Technology Publications, Inc.

Effectiveness and Cost-Effectiveness of Online Education: A Review of the Literature

Author(s): Insung Jung and Ilju Rha
Source: Educational Technology, Vol. 40, No. 4 (July-August 2000), pp. 57-60
Published by: Educational Technology Publications, Inc.
Stable URL: https://www.jstor.org/stable/44428629
Accessed: 04-04-2020 01:40 UTC

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide
range of content in a trusted digital archive. We use information technology and tools to increase productivity and
facilitate new forms of scholarship. For more information about JSTOR, please contact support@jstor.org.

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at
https://about.jstor.org/terms

Educational Technology Publications, Inc. is collaborating with JSTOR to digitize, preserve

and extend access to Educational Technology

This content downloaded from 42.153.51.116 on Sat, 04 Apr 2020 01:40:21 UTC
All use subject to https://about.jstor.org/terms
 students' acquisition of information technology skills
Effectiveness and and increased familiarity with technology.
In her extensive evaluation studies regarding the
Cost- Effectiveness of effectiveness of online education, Hiltz (1994) found
that:

Online Education: • Mastery of course material was equal or superior

to that of conventional courses.
A Review of the • Students reported improved access to instructors
and educational experiences.
Literature • Student participation in courses increased.
• Students reported higher satisfaction with courses
in general.
• Students' ability to synthesize information and
Insung Jung deal with complex problems improved.
Ilju Rha • Level of interest in the content of courses
increased.
But these results were not true of all the courses. Hiltz
explained these different results by pointing out that
Since the mid-1990s, there have been numerous instructor and student factors affected the effectiveness
studies investigating various aspects of online of the online courses. That is, the online treatment
education. The vast majority of those studies have been outperformed the conventional treatment when it
conducted to examine the educational advantages of incorporated more appropriate instructor interventions,
online learning and to explore effective design depending on students' characteristics, than the
strategies. Only a few attempts have been made to competing treatment. All the studies that have
investigate the effectiveness of online education based examined the effectiveness of online education remind
on empirical data. This article discusses the us of Clark's argument that the way the medium is used
effectiveness and cost-effectiveness of online education determines the effectiveness, not the medium itself
based on a review of the related literature. (Clark, 1983). Then the next question is "what are the
factors affecting the effectiveness of online learning?"
Effectiveness of Online Education
A review of early studies (Capper & Fletcher, 1 996; Factors Affecting Effectiveness of
Carter, 1996; Chu & Schramm, 1975; Clark, 1985, Online Education
1994; Kozma, 1994; Kulik et a/., 1985; Moore & Recent studies on the effectiveness of online
Thompson, 1997) comparing the effectiveness of instruction seem to focus more on factors affecting
distance education with that of face-to-face instruction learning process, satisfaction, and achievement in
or the effectiveness of different technologies leads to online instruction than on comparative effectiveness of
the conclusion that: online courses over traditional courses. Instructional
• in many instances, distance education is as design, social matters, and students' personal factors
effective as conventional classroom teaching and have been identified as three major variables
shows no differences in learning outcomes; and contributing to success in online learning.
• there is no general learning superiority for one Instructional design factors such as flexible course
type of technology over another. structure, quick and frequent feedback, effective visual
However, in some studies (Daugherty & Funke, 1998; layouts, and multiple zones of content knowledge
Hiltz, 1994; Jonassen et ai, 1999), online instruction influence online interaction and learner satisfaction
has led to significantly better results on examinations, (McLoughin, 1999; Stevenson et ai, 1996; Vrasidas &
in solving complicated problems, and in perceived Mclsaac, 1999).
learning outcomes. In addition, Thompson (1996), Social factors also affect the effectiveness of online
based on a review of comparative studies, found that learning. Anderson and Harris (1997) identified factors
online distance education was better in regard to predicting the use and perceived benefits of the Internet
as an instructional tool. Interpersonal interaction and
social integration were among the most influential
Insung Jung is Associate Professor, Institute of Distance factors. This result is supported by another study,
Education, Korea National Open University, Seoul, Korea (e- conducted by McDonald and Gibson (1998). In
mail: isjung@mail.know.ac.kr). Ilju Rha is Associate Professor, addition, the study of Gunawardena and Zittle (1997)
School of Education, Seoul National University, Seoul, Korea revealed that social presence exhibited by participants
(e-mail: iljurha@snu.ac.kr). contributed more than 60% of learner satisfaction with
computer conferencing courses, and Kanuka and

EDUCATIONAL TECHNOLOGY/July-August 2000 57

This content downloaded from 42.153.51.116 on Sat, 04 Apr 2020 01:40:21 UTC
All use subject to https://about.jstor.org/terms
Anderson (1 998) found that social-cognitive processes and are influenced by a number of factors. Generally,
among participants in the online forum included cost-effectiveness of distance education increases as the
significant time engaged in social interchange. number of students increases and the number of
Students' personal factors also play an important role courses declines.
in online learning. For example, students' prior While these studies are useful in providing a
knowledge of technology or subject matter affected comparative look at identifying the cost and
learning in online courses (Hill & Hannafin, 1997; effectiveness of media-mediated courses, not much
Limbach, Weges, & Valcke, 1997; Wishart & Blease, research has been conducted to assess cost-
1999). In addition, Biner et al. (1995) found several effectiveness of online education. Even in the studies of
personality factors, such as self-sufficiency, cost-effectiveness of online education, "costs of
introversion, and relative lack of compulsiveness, were development or costs borne by students" are often
related to achievement by telecourse students. Learners excluded, and "these studies often use competing
acting as autonomous individuals constructing their methodologies, making them difficult to compare"
own knowledge (Bullen, 1998; Jonassen et al., 1999; (Bakia, 2000). With methodological limitations of cost-
Laffey et al., 1998; Naidu, 1997) and getting involved effectiveness studies in mind, the following paragraphs
actively in their learning (Hillman, 1999; Shneiderman discuss findings from the previously-mentioned studies.
et al., 1998) also tended to maximize their own Cukier (1997) summarized four of the cost-benefit
learning. methodologies examined in the previous studies and
provided an integrated methodology for the cost-benefit
Cost-Effectiveness of Online Education analysis of network-based learning. Four approaches to
Many educators or decision-makers believe that the cost-benefit analysis include: a value-based approach,
primary benefit of online education is that costs can be a mathematical modeling approach, a comparative
distributed over a large number of students, resulting in approach, and a return-on-investment approach. The
economies of scale for educational institutions (Inglis, proposed approach to cost-benefit analysis of online
1999; Kearsley, 2000; Whalen & Wright, 1999). It is learning, called an integrated approach, integrates
assumed that large student enrollment would increasemajor concepts in these four previous approaches.
revenue and lower the cost per student. Cukier emphasized that the analysis of cost and
While the possibility of reducing costs appears to be benefits should be conducted separately and the
one of the main factors that motivates decision-makers approach should be multi-leveled. But costs and
to adopt online education, two other factors also seem benefits will ultimately be evaluated subjectively.
to be important: improving the quality of students' Based on Cukier's (1 997) frameworks of cost-benefit
learning experiences through various types of online analysis, six case studies have been conducted by the
interaction, and increasing access (Inglis, 1999). From NCE-Telelearning project team in Canada, and two of
the student's perspective, online education means these are available online. Cost measures assessed in
increased opportunities for interaction with other the two case studies (Bartolic-Zlomislic & Bates, 1999;
students and instructors and wider access to a variety of Bartolic-Zlomislic & Brett, 1999) include: (1) capital
multimedia resources and experts worldwide. and recurrent costs, (2) production and delivery costs,
There is a relatively large body of literature and (3) fixed and variable costs. The cost structure of
discussing the costs and benefits of distance education each technology is analyzed and the unit cost per
across technologies and in a variety of contexts. In learning is measured. The costs assessed in Bartolic-
general, the literature has shown that "distance Zlomislic and Brett's study did not include overhead
education can be more cost-effective than face-to-facecosts as these were unknown. Benefit data include: (1)
education and that costs are predominantly dependent performance-driven benefits, (2) value-driven benefits,
upon student enrollment and the fixed costs of course and (3) societal or value-added benefits. Both
development and delivery" (Cukier, 1997, p. 138). quantitative and qualitative data were collected and
Capper and Fletcher (1996) analyzed previous included student, faculty and staff, and administrator
studies on the cost-effectiveness of distance education perspectives.
and identified factors influencing costs. Those factors The case study by Bartolic-Zlomislic and Brett (1 999)
include: number of courses offered (since the cost of analyzed costs and benefits of an entirely online
developing a course is one of the major expenses in graduate course at the Ontario Institute for Studies in
distance education, the most cost-efficient approach is Education of the University of Toronto in changing the
to offer fewer courses for larger numbers of students), software from Parti, a UNIX-based mail and
frequency of course revision, type of media used, type conferencing software system/product, to WebCSILE, a
and amount of student support, and attrition rate. They Web-based software product. The result of the study
concluded that even though cost-effectiveness of projected that their online program will make a small
distance education is supported in most of the studies, profit of 1,962 Canadian dollars per year during five
costs vary substantially from one situation to another years and that 19 students will be needed to achieve a

58 EDUCATIONAL TECHNOLOGY/July-August 2000

This content downloaded from 42.153.51.116 on Sat, 04 Apr 2020 01:40:21 UTC
All use subject to https://about.jstor.org/terms
break-even point. It concluded that it is possible to
develop highly cost-effective online courses within a Factors Affecting Cost-Effectiveness of
niche market, at relatively moderate cost to learners. It Online Education
also recognized that despite the change in software
After analyzing previous studies on cost-effectiveness
from Parti to WebCSILE, the largest cost of the online
of information and communications technologies (ICT)
course was in tutoring and grading time spent by the
in higher education, Bakia (2000) concludes that "the
instructors due to the nature of the course, which
most obvious obstacles (in implementing online
emphasized active online discussions. These costs
could be lowered if the format of the course were education in developing countries) include prohibitive
Internet connection costs and inadequate technical
changed to a less constructivistic environment. A case
infrastructures. Several factors suggest that the use of
study from the University of British Columbia (Bartolic- ICT in education, at least in the short-term, will be
Zlomislic & Bates, 1999) reported similar results. The
relatively more costly in developing countries, even if
researchers found that the annual break-even
Internet access were readily available and affordable."
enrollment based on the projected costs and revenues Besides factors associated with technical infrastructure,
over four years was 44 students. several other factors that affect cost and/or effectiveness
The paper by Inglis (1999) is an attempt to examine
of online education are identified in previous studies:
the costs of shifting from a print-based course to an • Number of students in a course.
online course and to seek the rationales for moving to • Number of courses offered.
online delivery. Inglis showed that online delivery was
• Amount of multimedia component in online
less economical, when measured on a cost per student courses.
basis, than print-based delivery for four different intake • Amount of instructor-led interaction.
levels (50/1 00/1 50/200 students). The distribution costs
• Type of online education platforms.
(such as ISP charges and individual support) for online
• Choice of synchronous vs. asynchronous o
courses represented a major component of overall interaction.
costs. The results of this study, in part, reflect the fact
• Completion rate.
that in traditional print-based distance education, most
of the economies of scale that are obtainable in the Given the ongoing development of information and
communication technologies, we can expect that
design, development, and delivery stages have already
online technology will bring changes in forms of
been obtained. Several strategies to balance costs with
teaching-learning at all levels throughout the world. □
benefits in online education are suggested.
Whalen and Wright (1999) report that Web-based
References and
training has higher fixed costs than classroom-based
training, but these higher course development costs are Suggested Readings
offset by lower variable costs in actual course delivery.
In general, Web-based training is more cost-effective Anderson, S. E., & Harris, J. B. (1997). Factors associated with
than classroom teaching, mainly due to the reduction amount of use and benefits obtained by users of a
in course delivery time and the potential to deliver statewide Educational Telecomputing Network.
Educational Technology Research and Development,
courses to a larger number of students.
45(1), 19-50.
A preliminary report of cost-effectiveness of online
Bakia, M. (2000). The costs of ICT use in higher education,
courses at the Korea National Open University (Jung & what little we know. TechKnowLogia.
Leem, 2000) shows that the development and delivery
Bartolic-Zlomislic, S., & Bates, A. W. (1999). Assessing the
costs for online education decreases over time (cost per costs and benefits of telelearning: A case study from the
online course was US$12,768 in 1998 and US$7,902 University of British Columbia, Vancouver.
in 1999). When compared with a traditional distance Bartolic-Zlomislic, S., & Brett, C. (1999). Assessing the costs
education course, which used television and textbooks, and benefits of telelearning: A case study from the Ontario
an online course had a higher completion rate (55.25% Institute for Studies in Education, University of Toronto.
in the traditional course and 93.1% in the online Biner, P. M., Bink, M. L., Huffman, M. L., & Dean, R. S.
course) and thus lower cost per completer. (1995). Personality characteristics differentiating and
While the studies reviewed above provide some predicting the achievement of televised-course students
ideas about cost-effectiveness of online education, we and traditional-course students. American Journal of
Distance Education, 9(2), 46-60.
still need more rigorous, experimental studies in
Bullen, M. (1998). Participation and critical thinking in online
various contexts to make any firm conclusions on the
university distance education. Journal of Distance
cost-effectiveness of online education.
Education, 13(2), 1-32.
Capper, J., & Fletcher, D. (1996). Effectiveness and cost-
effectiveness of print-based correspondence study. A paper
prepared for the Institute for Defense Analysis, Alexandria,
VA.

EDUCATIONAL TECHNOLOGY/July-August 2000 59

This content downloaded from 42.153.51.116 on Sat, 04 Apr 2020 01:40:21 UTC
All use subject to https://about.jstor.org/terms
Carter, V., (1996, Feb.). Do media influence learning? importance of outcome studies: A reply to Clark.
Revisiting the debate in the context of distance education. Educational Computing Systems, 1, 381-386.
Open Learning, 31-40. Laffey, J., Tupper, T., Musser, D., & Wedman, J. (1998). A
Chu, G., & Schramm, W. (1975). Learning from television . computer-mediated support system for project-based
What the research says. ERIC Document Reproduction learning. Educational Technology Research and
Service No. ED 109 985. Development, 46(1), 73-86.
Limbach, R., Weges, H. G., & Valcke, M. M. A. (1997).
Clark, R. (1983). Reconsidering research on learning from
media. Review of Educational Research , 53, 445-459. Adapting the delivery of learning materials to student
preferences: Two studies with a course model based on
Clark, R. (1985). Confounding in education computing 'cases/ Distance Education, 78(1), 24-43.
research. Journal of Educational Computing Research ,
McDonald, J., & Gibson, C. C. (1998). Interpersonal
1(2), 445-460.
dynamics and group development in computer
Clark, R. (1994). Media will never influence learning.
conferencing. American Journal of Distance Education,
Educational Technology Research and Development12(1),
; 7-25.
42(2), 21-29.
McLellan, H. (1999, Sept.-Oct.). Online education as
Cukier, J. (1997). Cost-benefit analysis of telelearning:
interactive experience: Some guiding models. Educational
Developing a methodology framework. Distance
Technology, 39(5), 36-42.
Education , 78(1), 137-152.
McLoughlin, C. (1999). Culturally responsive technology use:
Daugherty, M., & Funke, B. (1998). University faculty and
Developing an online community of learners. British
student perceptions of Web-based instruction. Journal of
Journal of Educational Technology, 30(3), 231-244.
Distance Education, 73(1), 21-39.
Moore, M. G., & Thompson, M. M. (1997). The effects of
Gunawardena, C. N., & Zittle, F. J. (1997). Social presence as
distance learning: Revised edition. ACSDE Research
a predictor of satisfaction within a computer-mediated
Monograph, 15, Penn State University.
conferencing environment. American Journal of Distance
Naidu, S. (1997). Collaborative reflective practice: An
Education, 7 7(3), 8-26.
instructional design architecture for the Internet. Distance
Hedberg, J., Brown, C., & Arrighi, M. (1997). Interactive Education, 18(2), 257-283.
multimedia and Web-based learning: Similarities and
Romiszowski, A. J. (1997). Web-based distance learning and
differences. In B. H. Khan (Ed.), Web-based instruction
teaching: Revolutionary invention or reaction to necessity?
(pp. 47-58). Englewood Cliffs: Educational Technology
Publications. In B. H. Khan (Ed.), Web-based instruction (pp. 25-37).
Englewood Cliffs: Educational Technology Publications.
Hill, J. R., & Hannafin, M. (1997). Cognitive strategies and
Sherry, L., & Wilson, B. (1997). Transformative
learning from the World Wide Web. Educational communication as a stimulus to Web innovations. In B. H.
Technology Research and Development, 45(4), 37-64.
Khan (Ed.), Web-based instruction (pp. 67-74). Englewood
Hillman, D. C. A. (1999). A new method for analyzing Cliffs: Educational Technology Publications.
patterns of interaction. American Journal of Distance Shneiderman, B., Borkowski, E. Y., Alavi, M., & Norman, K.
Education, 13(2), 37-47.
(1998). Emergent patterns of teaching/learning in
Hiltz, S. R. (1994). The virtual classroom : Learning without
electronic classrooms. Educational Technology Research
limits via computer networks. Norwood, NJ: Ablex and Development, 46(4), 23-42.
Publishing Corporation.
Shotsberger, P. (1997). Emerging roles for instructors and
Inglis, A. (1999). Is online delivery less costly than print and is learners in the Web-based instruction classroom. In B. H.
it meaningful to ask? Distance Education, 20(2), 220-239. Khan (Ed.), Web-based instruction (pp. 101-106).
Jonassen, F., Prevish, T., Christy, D., & Stavrulaki, E. (1999). Englewood Cliffs: Educational Technology Publications.
Learning to solve problems on the Web: Aggregate Stevenson, K., Sander, P., & Naylor, P. (1996). Student
planning in a business management course. Distance perceptions of the tutor's role in distance learning. Open
Education, 20(1), 49-63. Learning, 11(1), 22-30.
Jung, I. S., & Leem, J. H. (2000). Comparing cost-effectiveness Thompson, M. (1996). Distance delivery of graduate-level
of Web-based instruction and televised distance teacher education: Beyond parity claims. Journal of
education. Manuscript under review. Continuing Higher Education, 44(3), 29-34.
Kanuka, H., & Anderson, T. (1998). Online social Vrasidas, C., & Mclsaac, M. S. (1999). Factors influencing
interaction in an online course. American Journal of
interchange, discord, and knowledge construction. Journal
of Distance Education, 73(1), 57-74. Distance Education, 13(3), 22-36.
Kearsley, G. (2000). Online education: Learning and teaching
Whalen, T., & Wright, D. (1999). Methodology for cost-
in cyberspace. Belmont, CA: Wadsworth. benefit analysis of Web-based telelearning: Case study of
Khan, B. (Ed.). (1997). Web-based instruction. Englewoodthe Bell Online Institute. American Journal of Distance
Cliffs: Educational Technology Publications. Education, 13(1), 23-44.
Khan, B. H. (1997). Web-based instruction: What is it and B., & Dickinson, J. (1997). Distance education and the
Willis,
why is it? In B. H . Khan (Ed.), Web-based instruction World
(pp. Wide Web. In B. H. Khan (Ed.), Web-based
5-18). Englewood Cliffs: Educational Technology
instruction (pp. 81-84). Englewood Cliffs: Educational
Publications. Technology Publications.
Kozma, R. (1994). Will media influence learning? Reframing Wishart, J., & Blease, D. (1999). Theories underlying
the debate. Educational Technology Research and perceived changes in teaching and learning after installing
Development, 42(2), 7-19. a computer network in a secondary school. British Journal
Kulik, J., Kulik, C. L., & Bangert-Drowns, R. (1985). The of Educational Technology, 30(1), 25-42.

60 EDUCATIONAL TECHNOLOGY/July-August 2000

This content downloaded from 42.153.51.116 on Sat, 04 Apr 2020 01:40:21 UTC
All use subject to https://about.jstor.org/terms