Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Abstract
Augmented Reality (AR) is the technology of adding virtual objects to real
scenes through enabling the addition of missing information in real life. As the
lack of resources is a problem that can be solved through AR, this paper
presents and explains the usage of AR technology we introduce Augmented
Reality Student Card (ARSC) as an application of AR in the field of education.
ARSC uses single static markers combined in one card for assigning different
objects, while leaving the choice to the computer application for minimizing
the tracking process. ARSC is designed to be a useful low cost solution for
serving the education field. ARSC can represent any lesson in a 3D format
that helps students to visualize different learning objects, interact with theories
and deal with the information in a totally new, effective, and interactive way.
ARSC can be used in offline, online and game applications with seven
markers, four of them are used as a joystick game controller. One of the
novelties in this paper is that experimental tests had been made for the ARTag
marker set for sorting them according to their efficiency. The results of those
tests were used in this research to choose the most efficient markers for
ARSC, and can be used for further research. The experimental work in this
paper also shows the constraints for marker creation for an AR application. As
we need to work in both online and offline application, merging of toolkits and
libraries has been made, as presented in this paper. ARSC was examined by
a number of students of both genders with average age between 10 and 17
years and it found great acceptance among them.
Keywords
Augmented reality;
Mixed reality;
Computer Assisted learning;
Optical tracking
Index terms
1. Introduction
Augmented Reality (AR) is the technology of adding an unreal object to real
life (The EDUCAUSE Learning Initiative). AR applications and research were
triggered by a dream of the computer graphics godfather, Ivan Sutherland,
over 45 years ago (Sutherland, 1965). A lot of problems and points in the AR
are still under research due to the need to combine different technologies and
features to create such an application. The AR working process can be
classified into a sequence of steps leading to the desired goal achieved as
follows (Azuma, 1997):
Image Segmentation.
Feature Extraction.
Marker Detection.
Camera Orientation.
Rendering.
Augmentation.
Two main decisions have to be made for each AR application. One decision
regarding the display type and the other is the tracking system technique
(Diggins, 2005, Joele et al., 2005 and Vallino, 1998). For the proposed
application the decision was made to use the Spatial Displays and Optical
tracking system.
Talking about AR will lead to discussions about some of the technology
benefits and drawbacks. The advantages of the AR can be classified into two
parts: advantages of AR application and advantages at the AR creation
phase. Using an AR application allows the simulation, visualization, addition of
information, and interaction with the virtual objects without being totally
immersed in the virtual life (Diggins, 2005). The AR creation process is less
expensive than that of the Virtual Reality (VR); this is considered one of the
most important advantages of AR. Until now, all AR drawbacks are some
weak points in the technology starting from the tracking time, registration
error, and rendering quality.
Increasing the visualization ability for students, with the minimum tools used,
was our motivation for ARSC. ARSC leads to better understanding by
increasing the visualization ability, according to the survey results (will be
mentioned in brief in Section 8). Also ARSC supports the edutainment
learning techniques by the 3D objects augmentation with real scene, and the
availability for connecting to extra outsource edutainment lesson materials.
The greater percentage surveyed agreed that using ARSC make learning
more fun and effective. Using ARSC adds to the learning tools a practical,
interactive and creative ways, which is now heavily needed (Zhou, Duh, &
Billinghurst, 2008) to increase the student creativity and practical skills.
This paper is organized as follows: Section 2 shows the previous work, fields
of augmented reality and the most common categories this technology serves.
Section 3 discusses, in brief, the hardware, software and card deigns in the
ARSC system. Section 4 shows how the ARSC solution works; starting from
system initialization through system shutdown. Section5 describes ARSC
scenarios in detail. Section 6 shows experimental results. Section 7shows the
paper conclusion. Section 8 shows the future work.
2. Related work
According to year 2009 Hype Cycle for Emerging Technologies (Fig. 1), it
shows 2009 was the year of Mixed and Virtual Reality. Augmented Reality
made good steps towards the peak stage with an adoption status 510
years getting the technology back after disappearing for a year during 2007.
Many technologies were triggered due to the Mixed and Virtual Reality used in
holographic displays, head mounted displays, and eye tracking. In 2009,
Virtual Reality appeared in the stable state, and the popularity of the public
virtual Worlds increased.
Fig. 1.
Hype cycle for emerging technologies, 2009
Figure options
Over the past ten years many applications and research have been published.
The related work of AR can be classified into two classifications (Awad &
Dziadosz):
First, by the fields it serves which can be classified mainly as follows:
Game application
Marketing and Advertising
Filming
Education
Navigation
Medical
Military
Second, by the number of research papers published in each
field. Fig. 2 shows the classification according to the papers research
categories done in the International Symposium on Mixed and Augmented
Reality [ISMAR] over the past ten years and the number of papers made in
each research for each research track.
Fig. 2.
ISMAR past ten years published papers.
Figure options
Regarding the benefits that can be achieved using the AR technology in the
education field, AR made a great progress in the education applications. The
most popular augmented applications that have inspired our work and have
been studied were: The AR-Jam books (Hornecker & Dnser, 2008) made by
BBC, which developed two augmented storybooks as a combination of
physical pages and desktop interaction (screen, mouse) for children up to age
7. Put a Spell, which was a way for children to learn spelling using AR as an
iPhone application, In-Place Augmented Reality 3D Sketching of Mechanical
Systems (Bergig, 2009), which was an ISMAR09 research paper where Hand
sketching was used as a natural way for creating AR mechanical experiments,
where the user can sketch experiments and watch them simulated in 3D.
Using Augmented Reality for Teaching EarthSun Relationships to
Undergraduate Geography Students (Shelton & Hedley, 2002), which was an
evaluation research of using the AR application in education, it used ARToolkit
application to help teach undergraduate geography students about earthsun
relationships. Finally, The MagicBook (Billinghurst et al., 2001), was a visionbased tracking method paper used to overlay virtual models in real book
pages, creating an Augmented Reality (AR) scene. Other systems were also
made for learning and phobia treatment as AR maintenance of an armored
personnel carrier turret (Henderson & Feiner, 2009), and AR system for
treating psychological disorders: application to phobia to cockroaches (Juan
et al., 2004).
3. Design
ARSC System design can be classified into hardware architecture, software
design, and card design.
3.1. Hardware architecture
Regarding our need for creating a low cost application, a machine and a
normal web camera had been used as follows [Table 1]:
Table 1.
Hardware qualification.
ARSC machine hardware qualification
Processor
Intel core 2 Duo 2.13 GHz
Memory
4.00 GB
Operating system
Windows vista home premium (Service Pack2)
Web camera qualification:
640 480 VGA sensor, compatible with computers that have either USB 2.0 or USB 1.1
Table options
The System architecture is divided into two parts, online and offline.
Regarding the offline use, the ARSC can be used with one of the ARSC three
offline school solutions which are: maximum offline solution, medium offline
solution, and minimum offline solution. At each solution the student/teacher
can use the ARSC by presenting the ARSC card input to the camera, thus
providing the appearance of the subject in 3D format. Table 2 shows the
School ARSC application packages features. The difference between the
different applications is described briefly in the following scenario part.
Table 2.
School ARSC application packages features.
Full-size table
Regarding the online part, the ARSC can be used with a computer, webcam,
and Internet access. The ARSC online part requires no specific hardware
installation or architecture, as the user can use the ARSC through a flash
ARSC application accessible through the Internet.
Fig. 3 shows the hardware architecture of the maximum solution. At the ARSC
maximum solution the classroom contains one machine per student, a teacher
machine, and server. The server gives the availability of the lesson feedback,
real-time test correction, and lesson interruption option availability by the
student. Fig. 4 shows the ARSC medium solution, which differs from the
maximum solution by the absence of the server, which will minimize the cost
at the expense of sacrificing some features as shown in Table 2. Fig. 5 shows
the ARSC minimum solution. At this solution level there is a computer,
projector, and ARS carpet (Augmented Reality Student Carpet) per school
class. (Fig. 5)
Fig. 3.
Miximum ARSC solution.
Figure options
Fig. 4.
Medium ARSC solution.
Figure options
Fig. 5.
Minimum ARSC solution.
Figure options
3.2. Software
The ARSC application had been divided into two parts: offline application and
online application. At each part of the application a group of libraries and
toolkits for camera detection, tracking, 3D objects loading and rendering have
been used.
3.2.1. Offline toolkits chosen
Due to the obstacles we had faced and due to experimental work, the ARTag
toolkit was found to be the best choice in the offline scenario. The application
was developed using ARTag toolkit (Cawood & Fiala, 2008a, p. 328; ARTAG),
OpenGL and C++ Programming languages. The selection of the ARTag had
been made to repair broken sides and for high resolution tracking at different
environment parameters.
3.2.2. Online toolkits chosen
For the online application the FLARToolKit and FLARManager, and
PaperVision used with AS3.0 (Action Script 3.0) are used. The purpose of
using these toolkits is for their availability to export as. SWF hence allowing
the availability of publishing through the Internet. The FLARManger functions
of the Marker Added and Removal was used for creating the ARSC joystick
controller.
For designing ARSC we faced a lot of challenges as:
Initial design
Final design
Design dimension
14.3 8 cm2
9.4 5.3 cm2
Reason/achievement
Due to large size card a group of experiments had been
made and code development upgrades, leading to a
decrease in the card size to 66% of the initial size
Regarding the code modifications and marker design
upgrades, we had decreased the separation space
between the ARSC markers from 1.0 cm to 0.7 cm
between each marker.
Table options
Toolkit
ARTag
FLARToolkit
FlARManger
Marker types
Preserved 2048
Marker creation
Marker creation
Table options
Thus, at the marker development stage the work was divided into two phases.
First choosing the marker, and second creating the marker.
3.3.2.1. Marker choice
It is the stage of choosing the best of the preserved markers. This phase was
used with the ARTag part to choose from 2048 preserved, the border can
either be black on or white on black. The choosing phase had the following
stages:
3.3.2.1.1. Stage 1
Regarding our research and study to the daytime education environment, we
had cancelled the markers that are white on black. This decision minimized
the choosing set to 1024 markers.
3.3.2.1.2. Stage 2
The Forbidden markers had been neglected. These markers are of 23 values
that are illegitimate (Cawood & Fiala, 2008b, pp. 147148). These tags did
not perform well enough to be included in the library, leaving us with 1001
markers to choose from.
3.3.2.1.3. Stage 3
The final 1001 markers were carefully tested. Fig. 6 shows some results of the
most effective markers, and some of the worse ones. Fig. 7 shows the
markers that were used for this experimental stage. The distance value was
tested by fixing the light parameter, camera, and camera shooting
environment, then recording the maximum distance the marker can reach
while still detectable. The Following formula is used to calculate the efficiency
ratio for each marker.
Re=(D/Dm)100
Re: Marker Efficiency Ratio
D: maximum distance made by the marker in cm
Dm: Maximum distance made the whole set of the ARTag markers in cm
Fig. 6.
ARTag marker results.
Figure options
Fig. 7.
ARTag marker results.
Figure options
Based on the previous results we had chosen the two markers; Marker Code
#103 and Marker Code#518 to be used as they give the highest efficiency
ratio.
3.3.2.2. Marker creation
Many tests have been conducted on different marker designs for the online
application to get the best tracking performance and minimize the error
factor. Fig. 8 shows the first marker test that is not recommended to use for its
big error factor gained (due to the Symmetry error, and the near
edges). Fig. 9 shows the final marker that had been used for the ARSC online
application.
Fig. 8.
Marker not to be used.
Figure options
Fig. 9.
Used Marker.
Figure options
Fig. 10.
ARSC card front design.
Figure options
Fig. 11.
ARSC card back design.
Figure options
Fig. 12.
Before starting the ARSC learning session.
Figure options
learning session, which can be done either by keyboard hitting or using the
markers on the ARSC card. If N key on the keyboard is pressed, the ARSC
application will give the availability to swap between the valid lessons. If Q
key on the keyboard is pressed, the ARSC application will open the relative
lesson quiz. ARSC also gives the availability to open a PHP online test. If O
key on the keyboard is pressed, the ARSC application will open an extra
lesson support material (videos, games, and online recourses) to achieve the
edutainment goals. Also ARSC features can be activated not only by the
keyboard, but also by the ARSC card. Fig. 13 shows that if the student flips
the ARSC cover, the application will record an interruption lesson request
(Question request), and produces a beep sound to the teacher. The
interruption request was made for lesson evaluation purpose also.
Fig. 13.
During the ARSC learning session.
Figure options
Fig. 14.
After finishing the ARSC learning session.
Figure options
Fig. 15.
ARSC state diagram.
Figure options
Fig. 16.
ARSC applications and features.
Figure options
Fig. 17.
Offline maximum solution.
Figure options
Using this solution, the teacher enters the subject code which corresponds to
a subject in a 3D format. If the code is valid the students start to put their
cards in the camera view, leading to augmenting the 3D object on the real
scene that is finally rendered on the student machine display. Using such a
solution gives the ability to the teacher to trigger a prepared test, navigate
between lesson objects, and record all the actions made by the machine to
create a lesson report. Also using the ARSC card enables the student to use
the back of the card to request a lesson interruption (as the card will produce
a beeping sound and record an interruption request on the server) leading to
added functions for education processing regarding lesson upgrading and
modification.
Fig. 18.
Offline medium solution.
Figure options
Fig. 19.
Offline minimum solution.
Figure options
Fig. 20.
Online solution.
Figure options
Fig. 21.
ARSC joystick controller.
Figure options
6. Results
During the ARSC application creation the following results were achieved:
The most two common Augment Reality toolkits, ARTag and ARToolkit,
are compared using the experimental work. This practical test led us to
choose the ARTag mainly for its better light immunity, broken edge
correction, minimum falsely reporting a marker factor.
Testing the 1001 ARTag set led us to choose the best two according to
our experimental work (Marker Code #103, and Marker Code#518)
During the ARSC creation stage, multiple sizes of the cards were
made. According to the experiments made on the card size the results
with the detected maximum distance according to light effects, are
summarized as follows [Table 5]:
Table 5.
ARSC Distance result.
ARSC size
8 14 cm2 Initial size
6 10 cm2
Distance (incandescent)
67 cm
48 cm
Distance (fluorescent)
73 cm
55 cm
Table options
Table 5 shows that the small size of the card can be used up to a distance of
about 50 cm from the camera centre, and the large size of the card can be
used up to a distance of about 70 cm from the camera centre.
As an example of using ARSC, Fig. 22 a,b shows some lessons displayed on
the ARSC; animal cell and solar system respectively. Fig. 22-c shows the
question request made by a student when visualizing the back of the card to
the camera view. Fig. 22 d shows one of the students while using the ARSC.
Fig. 22.
ARSC in operation.
Figure options
Full-size table
Table 7 shows the subjects the students had suggested to be taught using the
ARSC system. The results had given the Science and biology the highest
voted value with 34.7% followed by Art & Design with 16.9%,Chemistry
with 15.3%, and History with 8.6% leaving the lowest value to the
Languages subjects with 5.1%.
Table 7.
The subjects the students suggest to educate using ARSC.
Full-size table
Compressing all the controls in one card leads to minimizing the cost without
sacrificing the education benefits. Minimizing the cost had been achieved by
two ways; first minimizing the time needed for information gained by the
student using more effective and interactive way, with the availability for
lesson visualization. Table 8 shows that 89% of the students were satisfied by
the efficiency of the ARSC. It also shows that 70% of the students think that
they need a teacher/instructor when using ARSC application. More than 87%
agreed that the education process needs such a system as shown in Table 8.
The results mentioned in Table 6, Table 7 and Table 8 shows that ARSC
solution supports the education process with lesson visualization features
minimizing the time needed for lessons class discussions, which decreases
the education cost. According to a student questionnaire, it had been
observed that using the ARSC can minimize the extra and private lessons
which cost 551 L.E per month for each student according to the questioner
results.
Table 8.
ARSC options & features feedback.
Full-size table
Fig. 23.
ARSC to traditional system cost analysis.
Figure options
Table 9 shows the cost of each system for five years. Although the cost of the
ARSC maximum and medium solution is greater than the traditional system in
year 1, but calculating the average cost per the five years for all systems
shows that, the ARSC maximum, medium, and minimum solutions are less
than the traditional system by 32.89%, 35.99%, and 90.9% respectively as
shown in Table 9. All the calculated data are made per according to the
national education cost at schools and regarding the national student/class
average capacity.
Table 9.
Traditional system cost verses ARSC cost.
Year
Traditional
ARSC Maximum
System (cost
solution (cost L.E)
L.E)
1
386.82
1573.2
2
425.50
2.5
3
468.05
2.8
4
514.86
3.0
5
566.34
3.3
AVG 472.31 L.E
316.96 L.E
ARSC Medium
solution (cost L.E)
ARSC Minimum
solution (cost L.E)
1500
2.5
2.8
3.0
3.3
302.32 L.E
121.95
20
22
24.2
26.62
42.95 L.E
Table options
Table 9 shows the cost of the traditional system which has been achieved, by
calculating the total printing cost and dividing it by the total number of
students, then adding the average cost of the student books. The traditional
cost increases by the 10% annual raise for each year, making the cost
average 472 L.E. To calculate the ARSC solution, we have calculated the
expected cost of five succeeding years. In the first year, we have included the
cost of the hardware and the printing cost with an excepted annual raise 10%.
For the ARSC maximum solution, the cost of hardware is calculated by adding
the server cost and the machine cost of each user. For the ARSC medium
solution, the hardware cost is only the cost of the users machine. For the
ARSC minimum solution, the hardware cost is that of the class machine and
video projector divided by the average capacity of each class. The printing
cost for the maximum and medium are both the cost of printing the ARSC
cards, whereas that of the minimum is the cost of printing the ARSC carpet.
7. Conclusions
According to our research aim scope for creating an educational application
using the AR technology, creating the ARSC will lead to decreasing the
education expenses at schools and increasing the visualization ability to
students with minimum tools used.
ARSC only uses some static markers which lead to decreasing the tracking
work needed and the array of objects to be read. Using a mixture of different
augmented reality toolkits gives the availability of showing any lesson in a 3D
format with a full control of zooming and viewing. It was experimentally proved
that we can use a card of size 6 10 cm up to a distance 50 cm from the
camera lens, and a card of size 8X 14 up to 70 cm. Each student only had
one ARSC from the beginning of the education trip till the end without any
need to change, hence minimizing the printing cost.
During the ARSC research, a full dataset for the ARTag 1001 markers
regarding the efficiency value had been created, which can be used for further
use or research.
Each student can use the ARSC for offline, and online usage due to merging
different library files.
ARSC was examined by 51 students of both genders showing that 89% of the
students were satisfied by the efficiency of the ARSC. The previous survey
shows the suggestion for the subjects to be used within the ARSC solution
gave the highest value to Science and Biology with 34.7% of the total votes,
following by Art & Design, Chemistry , History, and leaving the lowest
value to the Languages subject with 5.1%.
Compressing all the controls in one card led to minimizing the cost without
sacrificing the education benefits. Minimizing the cost had been achieved by
two ways. First, by minimizing the time needed for gaining information, 89% of
the students were satisfied by the efficiency of the ARSC and more than 87%
agreed that the education process needs such a system and secondly by
minimizing the educational financial costs. Calculating the average cost per
the five years for all systems shows that, the ARSC maximum, medium and
minimum solutions cost are less than the traditional system cost by 32.89%,
35.99%, and 90.9% respectively.
Using different toolkits, programming tools and Operating on different
platforms leads to some problems that the system had to solve such as
rendering quality and tracking time.
8. Future work
References
1.
ARTAG
ARTAG, [Online]. Available: http://www.artag.net/. [Last Visit: Aug. 20,
o
o
2010].
o
[SD-008]
o
o
o
2.
o
o
3.
o
o
o
Azuma, 1997
R.T. Azuma
A survey of augmented reality; hughes research laboratories
[SD-008]
4.
o
o
Bergig, 2009
Bergig, O, Hagbi, N, El-Sana, J, & Billinghurst, M, (Oct 2009). n-place 3D
sketching for authoring and augmenting mechanical systems, Symposium on mixed and
augmented reality. In:Proceedings of the 2009 8th IEEE International Symposium on
Mixed and Augmented Reality, pp. 8794.
[SD-008]
5.
o
o
o
[SD-008]
o
o
o
[SD-008]
6.
7.
o
o
o
8.
o
o
o
o
o
9.
o
o
o
Diggins, 2005
D. Diggins
ARLib: A C++ augmented reality software development kit
[SD-008]
10.
o
o
o
FLARManager
"FLARManager," [Online].
Available: http://words.transmote.com/wp/flarmanager/flarmanager-documentation/. Last
Visit: Aug. 20, 2010.
[SD-008]
11.
o
o
o
FLARToolKit
"FLARToolKit", [Online].
Available http://www.libspark.org/wiki/saqoosha/FLARToolKit/en. Last Visit: Aug. 20,
2010.
[SD-008]
12.
o
o
o
13.
o
o
o
[SD-008]
14.
o
o
o
[SD-008]
15.
o
o
o
16.
o
o
o
17.
o
o
18.
o
o
o
19.
o
o
o
Sutherland,
Sutherland, I.E. (1965). The ultimate display, In: Proceedings of the IFIP
Congress pp. 506508.
[SD-008]
20.
o
o
o
o
o
o
Vallino, 1998
J.R. Vallino
Interactive augmented reality
21.
o
o
22.
Zhou et al., 2008
Zhou, F., Duh, H.B.L., & Billinghurst, M. (2008). Trends in augmented reality
tracking, interaction and display: a review of ten years of ISMAR. In: Proceedings of
IEEE International Symposium on Mixed and Augmented Reality (IEEE/ACM ISMAR).
o
[SD-008]
Corresponding author. Tel.: +20 2 0101355491.
Copyright 2010 Elsevier Ltd. All rights reserved.
o
o
No articles found.