Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
1. Topic. This project investigated the principles for designing and developing advanced e-
Learning systems that employ a learner centered pedagogical approach, in contrast to the more
common delivery centered approach found in the majority of the e-Learning systems that are
developed by Korean e-Learning companies and organizations.
2. Research Purpose and Needs. A central argument in this report is that in order to advance
the overall quality of e-Learning (and learning and training more generally) in the Republic of
Korea, it will be necessary to identify major types and characteristics of advanced e-Learning
systems and the principles for developing and implementing these systems. The main goal of this
project is to provide information about the characteristics and design principles of different
types of advanced e-Learning systems that implement learner-centered approaches and that have
been found to foster cognitively and socially robust levels of student interaction and learning.
3. Research Content. Two main tasks were undertaken in this project. First, a comprehensive
literature review was conducted of advanced e-Learning projects from around the world and of
recent work on HCI (human-computer interaction) and LCD (learner centered design). Second,
a series of case study interviews were conducted with project directors, researchers, and
developers involved with advanced e-Learning systems in Korea and in the United States to
identify explicit and tacit knowledge about techniques for creating robust and effective learner-
centered e-Learning systems.
4. Research Results. A set of nine exemplary and representative learner-centered e-Learning
systems were identified in the literature review and from the case studies that met the criteria for
inclusion in the report. These e-Learning systems were discussed in a working framework that
consisted of six categories: (a) educational hypermedia, (b) modeling and simulation genre
systems, (c) virtual worlds for learning, (d) computer supported collaborative learning, (e)
intelligent systems, and (f) handheld devices. Based on an analysis of these systems, a framework
for advanced e-Learning systems based on principles from recent research in the science of
learning was proposed. The science of learning framework was illustrated using examples from
the literature review and case study e-Learning systems. Also provided are a set of suggested
initiatives to advance the development of advanced e-Learning systems in Korea, as well as a
consideration of implementation challenges for learner centered e-Learning in Korea and
significant opportunities for affecting educational reform initiatives.
5. Contributions. The results of this project should make contributions to e-Learning
conferences and workshops and also be disseminated in papers submitted to referred journals
and in a book.
6. Conclusion. It is hoped that the information in this report will be of value to a wide range of
individuals in the Korean e-Learning development community, including commercial, university,
and government content developers. This report could also be used as part of university classes
in instructional design and in professional development workshops that professional developers
take to become more familiar with the latest trends and approaches in e-Learning. The findings
and recommendations in this report should also be of value to personnel in the Korea IT
Industry Promotion Association and to industry groups such as the Korean e-Learning Industry
Association (KELIA) to inform policy decisions concerning opportunities and issues for future
funding projects, conferences, workshops, and so on. It is also hoped that the information in this
report will be of value to other policy makers involved with e-Learning in Korea.
6 Discussion.........................................................................................................................................58
6.1 Type II E-Learning Systems and Science of Learning Based Design Principles...........59
6.1.1 Provide Contextualized Learning.....................................................................................60
6.1.2 Provide Scaffolds and Tools.............................................................................................60
6.1.3 Consider the Preconceptions and Prior Knowledge .....................................................60
6.1.4 Make Organizing Conceptual Structure Explicit ...........................................................61
6.1.5 Make Thinking Visible.......................................................................................................61
6.1.6 Support Learning-By-Doing .............................................................................................62
6.1.7 Foster Thoughtful Collaborations amongst Communities of Learners .....................62
6.1.8 Provide Formative and Summative Assessment of Learning ......................................62
6.2 Application of Science of Learning Principles for the Design of E-Learning...............63
6.3 Lessons Learned Related to the Development of Type II E-Learning Systems ...........64
6.4 Research Limitations ..............................................................................................................65
6.5 Directions for Future Research and Policy Initiatives.......................................................65
7 Conclusion ........................................................................................................................................66
8 Acknowledgements..........................................................................................................................70
9 References ........................................................................................................................................71
10 Appendix............................................................................................................................................76
E-Learning
Paradigm Examples Learning Outcomes
Types
However, describing e-Learning in terms of the enabling technologies is not very useful as
this does not distinguish between the types of design features for various e-Learning
approaches, and more important, between different paradigms for teaching and learning.
Consequently in this report, a distinction is made between two general types e-Learning
systems (see Table 1). Type I e-Learning systems consist of more traditional approaches for
designing computer-based learning programs, such as tutorials and computer-based drills
(Roblyer, Castine, & King, 1988; Taylor, 1980). Systems such as these, which basically
employ a delivery-centered paradigm of instruction (similar to a teacher-centered
approach), have been found to help students learn basic factual information about a topic,
although they do not help students construct deep understandings of the content and
important concepts (Jacobson & Spiro, 1994). Despite this, Type I e-Learning systems such
as tutorials, one of the oldest approaches for designing computer-based learning systems, are
still the most widely used for commercial and government developed e-Learning programs
in Korea and globally. It is important to note that more recent Type I e-Learning tutorials
may be very sophisticated in terms of using multimedia, animations, and so on. However,
1 Few people know that the label eLearning or e-Learning was actually first coined by Jay Cross, a
Berkeley, U.S.A. based business consultant in 1998, and has been used very generally by many individuals since
that time to describe nearly any type of technology system used for educational or training purposes.
Figure 2. A typical VOD screen of teacher speaking (i.e., delivering her lecture).
2 The review of the HCI literature was requested by the funding agency.
Table 3 provides summary data related to the three e-Learning areas being reviewed, the
databases searched, and representative search terms. Over 400 papers, books, and other
documents were reviewed by the project research staff and the report includes 80 documents
in the final reference list.
The next section provides a discussion of the main themes and principles for the design of
Type II e-Learning systems identified in the HCI and learner centered design documents
that were reviewed, followed by a non-technical description of exemplary Type II e-Learning
Systems the e-Learning literature review and from the case studies.
3 The funding agency requested that a section of this report provide a discussion of HCI and e-Learning.
4One of the case studies discussed in this report involved discussions at the University of Michigan with Dr.
Soloway and members of his group; see section 5.3.
5 It must be stressed that this commitment to learning-by-doing is based on a growing body of scientific
research in the cognitive and learning sciences (National Research Council, 2000, 2001).
Table 4. A Working Framework for Type II e-Learning Systems and Programs Reviewed in Report
BioLogica and
Case Study
Pedagogica
2. Modeling and Simulation Genre CDC Case Study
Systems Scenario
Discussion
Environmental Discussion
6. Handheld Devices Detectives
From the systems identified in the literature review, a set of nine exemplary e-Learning
systems in the six Type II e-Learning category types were selected for inclusion in this report
(see Table 4). It is interesting to observe that several examples of educational Type II e-
Learning systems were identified for each of the category types (from which one was
selected for discussion in this report), while for business and HRD, the only Type II e-
Learning systems that met the criteria for inclusion in the report were both in the category
type Modeling and Simulation Genre Systems. Of these exemplary systems, six are presented
below in this section and four are discussed in section 5 Case Studies.6
4.1.1 Type II e-Learning Systems in Education
This section provides detailed illustrative discussions of exemplary Type II e-Learning
systems that are mainly used in pre-college or college education. Each of these systems has
been developed as part of funded research projects and the learning technology research
6Note that there are five case studies, but only four listed in Table 3. The Michigan case study with Dr. Elliot
Soloway involved more general design and development considerations related to a number of different e-
Learning systems that were either desktop modeling or handheld systems, and thus this case study is not
explicitly listed in the table.
Figure 7. Case screen with pop-up list of the conceptual explanations available for the antibiotic resistance
case in a new SCORM compliant version of the Knowledge Mediator.
4.1.2.2 LCD and Learning Scaffolds
There are several learning scaffolds embedded in Knowledge Mediator e-Learning
hypermedia. One of the most important of these is conceptual criss-crossing, which is a special
type of problem solving with cognitive scaffolding. Conceptual criss-crossing provides the
learner with a problem or challenge question that requires the integration of knowledge
distributed across multiple cases and conceptual lessons in the e-Learning systems
knowledge base. A set of conceptually-based hyperlinks are provided for the problems that
are intended to support expert-like non-linear navigation through the knowledge space of
cases, conceptual lessons, abstract concepts, and context specific concept explanations. For
Figure 8. Screen showing the natural selection conceptual explanation for the antibiotic resistance case.
7 Some researchers in the field refer to the use of gaming technologies to create e-Learning systems as
educational computer games or simulation games. However, in this report the term virtual worlds for learning is used as
modern students, with their extensive experience with commercial computer games, will tend to play an
educational computer game rather than to learn with it.
8See the case study section 5.2.2 for a discussion with Dede and his group on development and
implementation issues related to an e-Learning system of this type.
Figure 11. River World screen with an avatar and different informational and communication displays.
Through data gathering, students observe the patterns that emerge and wrestle with
questions such as Why are many more poor people getting sick than rich people? Multiple
causal factors are involved, including polluted water runoff to low-lying areas, insects in
swampy areas, overcrowding, and the cost of access to medical care. Students are asked to
identify problems through observation and inference, form and test hypotheses, and deduce
evidence-based conclusions about underlying multi-causes.
4.1.3.2 LCD and Learning Scaffolds
MUVEES systems are explicitly designed to support learner-centered activities. The main
learning design principles in these systems are:
Learners do not passively observe situations, but actively investigate multivariate
problems.
Learners conduct authentic tasks in contexts similar to the real world.
Students learn important scientific knowledge and skills while they collect data on
water purity, crop yields, or pollution levels in order to solve problems posed in the
system.
Students learn in a collaborative problem-solving community in which they gain
knowledge and skills through co-interpreting data with other participants at varied
levels of skills. (Research has shown that novices may learn as much or more from
the observation of other, somewhat more advanced participants, as they would by
direct guidance from an expert.)
One of the distinctive features of WebCaMILE is Thinking Types (see Figure 13). When a
user indicates that she wants to create a new note, she is prompted to identify the type of her
note, such as a New Idea, Rebuttal, or Alternative. These thinking types are modifiable for
each instance of WebCaMILE. For example, in a class on object-oriented modeling and
design, the note types include Analysis Issue, Design Issue, and Programming Issue. This
feature supports students as they think about a posting and its function among the whole
discussion, which enhances students metacognitive skills.
Suthers, D., & Weiner, A. (1995). Groupware for developing citical discussion skills. In J. L.
Schnase & E. L. Cunnius (Eds.), CSCL'95 Proceedings (pp. 341-348). Bloomington, IN:
Lawrence Erlbaum Associates.
4.1.4.6 Online Resources
Main WebCaMILE website: http://www.cc.gatech.edu/gvu/edtech/CaMILE.html
This section provides a summary of the Cognitive Tutor Algebra I program that is
demonstrated on the Carnegie Learning website (see URL below). The Cognitive Tutor
Algebra I curriculum gives students the opportunity to work with various representations of
9According to the Anderson (1996), ACT-R theory provides insights into how students learn new skills and
concepts, and, in doing so allows teachers to see where students may need extra practice to master the new
work. ACT-R suggests that complex cognition arises from an interaction of procedural and declarative
knowledge. Declarative knowledge is encodes facts (such as Washington, D.C. is the capital of the United
States, 5 + 3 = 8); procedural knowledge encodes skills or how we do things (such how to drive or how to
perform addition). According to the ACT-R theory, the power of human cognition depends on how people
combine these two types of knowledge.
Figure 14. Various representations of functions are interconnected in the Cognitive Tutor
Progressing through the curriculum, students learn to generalize specific instances into
algebraic formulas. Students complete the Worksheet (which functions like a spreadsheet) by
recording answers to questions posed in the Problem Scenario (see Figure 15).
Figure 15. The Problem Scenario screen, left, and a Worksheet screen on the right.
The Just-in-Time Help Messages feature, shown in Figure 16, is an important component of
the Cognitive Tutor. If a student becomes unsure of a solution path, help is readily
Figure 16. Just-in-Time Help Messages gives students immediate feedback when students make errors.
Another module of the Cognitive Tutor, Solver, encourages students to solve equations
within the context of problems (see Figure 17). Students learn techniques to solve problems
and may begin to discover the value of mathematical skills beyond the classroom.
As the Tutor guides a student down an individualized learning path, the student can also
access this information on demand, which encourages accountably for his or her learning
progress. Throughout the various problem-solving activities, students behavior and
knowledge growth are monitored and traced. This knowledge tracing dynamically updates
estimates of how well the student knows each production rule (Anderson et al., 1995). These
estimates are used to select problem-solving activities and to adjust pacing to adapt to
individual student needs. Using the Cognitive Tutor, students receive the benefits of
By clicking Video button in Figure 22, students watch a 60 second digital video briefing
from the University president where they are enlisted to investigate the spill of the toxin, a
carcinogenic degreasing agent that is commonly found in machine shops, cafeterias, and
hospitals. The goal of the game is to locate the source of the spill, identify the responsible
party, design a remediation plan, and brief the president of the University on any health and
legal risks so that he will be prepared for a meeting with the EPA all within three hours. At
the end of the game, students make a five minute presentation to their peers outlining their
theory behind the spill.
Digging wells and associated activities can be another important resource. Students can
virtually walk to the location they are interested in sampling and click the Dig button.
After the three minute wait, they return to the well location they can "Take Rig" and then
"Sample." Sample analysis offers three options according to its accuracy and time (see Figure
25). Students have to decide between getting more accurate results while waiting a relatively
long time versus getting less accurate results while waiting a shorter time. To review the
water sample data, they go to the Resources menu and choose Wells, which will display all of
the locations and readings.
10This issue was discussed by the author of this report with a senior Vice-President of an Australian-based e-
Learning company. He believed that there were a number of organizations that in fact employed Type II e-
Learning approaches and technologies, but that they keep the design specifications and evaluation findings of
their systems confidential to maintain a competitive training advantage over rival organizations.
Figure 26. Opening screen for the CDC Botulism in Argentina e-Learning program.
Figure 28. A conceptual information page to supplement the case study data.
Figure 31. Concluding screen of CDC case study with summary investigation information.
4.2.2.2 LCD and Learning Scaffolds
The CDC Botulism in Argentina program employs real-world case based materials that are
intended to be engaging to adult learners as they investigate and solve problems related to
the epidemiological outbreak scenario (see Figure 26). Rather than relying on an instructor to
give advice or guide participants, the program provides access to a variety of support tools
or learning scaffolds, such as hints and reference materials, which may be consulted before
they answer questions. Another important learning and design feature is the availability of
The questions were used in the semi-structured interviews conducted for case studies. The
interviews, which were audio or video taped, were conducted in a conversational style that
typically began with the first two or three questions and then were allowed to proceed
naturally based on the responsibilities the particular interviewees had for each project. The
interviewer would ask follow-up questions to probe the responses made and would ask
questions at the end if information related to particular questions had not been covered
during the interview. The notes from the case study interviews were analyzed for common
themes and issues that are discussed below.
11Note that none of the questions specifically asked about Type I and Type II e-Learning issues, since all of the
case studies were selected because they were explicitly Type II approaches for the design of e-Learning systems.
Figure 32. Dr. Paul Horowitz, Dr. Michael Jacobson (PI), and Dr. Robert Tinker at the Concord Consortium.
12Teachers felt they could identify misconceptions, but the project team observed that the teachers tended to
just tell the students what was "right" rather than engage in pedagogical techniques that learning science
research has found to be most effective for changing misconceptions (National Research Council, 2000).
13 The credit for the introduction of the concept of scaffolding in the science of learning and in the area of
learning technologies or e-Learning may be attributed to Allan Collins in his seminal papers from the late 1980s
(Brown, Collins, & Duguid, 1989; Collins, 1989; Collins, Brown, & Newman, 1989).
14 For the remaining science of learning principles for e-Learning, not every system in the report implemented
all the principles.
15The exact legality of commercial versions of open-source systems, particularly in an international context,
should be done with appropriate legal consultation regarding intellectual property issues.
16 This issue was discussed during a mid-project review meeting at the end of August, 2004.
7 Conclusion
This report has discussed ways that advanced learning technologies may be designed and
used to help foster powerful and highly effective learner centered or Type II e-Learning
approaches to education and training. Indeed, it may well be that the power of Type II e-
Learning will be first demonstrated in Korea in industry oriented contexts that require
advanced understandings of challenging new knowledge, as in the Crdu project discussed in
this report. This would allow the e-Learning industry in Korea to develop the technical and
design skills necessary to implement innovative Type II e-Learning systems and to seed a
broader awareness for how such approaches represent viable and desirable alternatives to
conventional didactic approaches to training in industry and to teaching in pre-college and
college education.
As noted in the beginning of this report, there are beginning to be calls to transform the
educational system in Korea from a very didactic and teacher centered paradigm to a learner
centered paradigm that fosters advanced learning goals such as problem solving, creativity,
and critical thinking (Kim, 2003). However, at present, the pre-college educational system in
Korea is dominated by the college entrance examination system that is primarily a test of
memorization of declarative information and certainly not a test of cognitive problem
solving skills or critical thinking. As such, e-Learning developers who are interested in
creating Type II learner centered tools for the pre-college education market are certainly to
be commended, but caution is advised until the Korean educational system aligns its high
stakes examinations with procedures that actually assess the kinds of higher ordered
knowledge and skills cultivated by the Type II e-Learning approaches advocated in this and
other reports (National Research Council, 2001). On the positive side, however,
2. Sponsor advanced e-Learning curriculum Currently, no Korean universities offer a coordinated set
development projects for universities with graduate of courses that prepare Korean masters and Ph.D.
programs that prepare students for the e-Learning students to design, develop, and implement advanced
field. These projects should develop courses and Type II e-Learning systems. Students in the U.S. are able
technology laboratories so that graduate students to obtain this type of training in new types of courses now
could not only read about categories of Type II e- being offered at leading U.S. universities in what are often
Learning, but also use, design, and develop such called Learning Sciences programs that combine
systems. cognitive science, computer science, education, and
research perspectives. By funding an advanced e-
Learning curriculum development initiative, KIPA could
help motivate universities to move more quickly to
develop the necessary academic programs to prepare
students for careers in a world class e-Learning industry
in Korea and as researchers in this important new field.
3. Sponsor a 2 or 3 day international conference on the A conference of this type could be valuable for both
design of advanced learner centered technologies practical and policy issues related to the Korean e-
with invitations to a selected group of internationally Learning industry and its impact on education and
regarded researchers, learning scientists, computer training/professional development in Korea. The explicit
scientists, and e-Learning industry developers to focus on learner centered uses of e-Learning
discuss design, development, and implementation technologies would be aligned with Ministry of Education
issues for a Korean e-Learning industry and efforts to foster educational reform using ICT. This focus
education audience. would also provide an opportunity to discuss the lack of
alignment of Type II e-Learning systems with current
educational practices in Korea and most Asia Pacific Rim
countries.
4. Sponsor KIPA awards for innovative Type II Korean A program that would provide awards (perhaps given at a
e-Learning systems. special banquet) for innovative Korean Type II e-Learning
systems could be a relatively low cost method to both
acknowledge outstanding work in this area as well as to
publicly link advanced ways to learn (i.e., learner
centered approaches) with advanced ways to develop e-
Learning technologies.
4. How large are the technical development and content development groups?
10. What did the initial usability testing reveal and what changes were made in the system?
11. Describe the issues you encountered in the initial use of the system in the environment
for the targeted group of learners:
14. Describe any changes made in the system and the programming or content development
that was required to make these changes.
15. Describe the current status of the system in terms of its technical development.
16. Describe the current status of the system in terms of its content development.
19. Are you aware of the technical specifications of currently proposed industry standards
for e-Learning systems and LCMS?
20. Does the e-Learning system developed for this project comply with current industry
standards for LMC and LCMS?
21. If not, are there plans to make the system standards compliant?
22. If there are no plans to make the system standards compliant, what plans are there to
make the technology system developed in this project more widely available?