Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Abstract—A wide variety of wearables are now being sold such as real-time constraints that take into account preemption
on the market. Improvements in hardware performance and and context switching times as critical variables. In the same
miniaturization allowed the creation of new advanced devices. way, input/output (I/O) efficiency, memory management and
Despite this focus on hardware optimization, the software part
may also have an essential role inside wearables continuous graphics drawing time may be addressed [7].
development. There is a need to understand the main impacts of In this paper, the authors present a comparison between a
operating systems in this ecosystem, particularly on performance set of different OS’ used by wearables solutions. In a lack
aspects. This work presents a functional comparison between of a proper benchmark to evaluate them, an assortment of
some of the most common operating systems used by wearables. tests was built covering complex and straightforward scena-
In the absence of a standardized benchmark, authors have
created a set of evaluations to figure out the system with the best rios that enclose I/O data performance, graphical operations,
performance. Results presented at the end helps to determine simple/parallel processing and general processing tasks. This
the best operating system in each scenario, forcing designers and research helps to demonstrate the impact of an OS designed for
developers to choose the software solution taking into account wearables, decoupling software from hardware, and analyzing
this environment-specific features. the first one separately. Still, the authors general objective is
not to define “which is” the best OS for wearables, but examine
I. I NTRODUCTION
the popular ones and describe the best scenario where each one
The wearables devices are increasingly gaining market can be applied. The main contributions of this paper are:
space, taking advantage of electronic components miniaturi- • A characterization of wearables according to its functio-
zation. As these elements become smaller and smaller, the nalities;
hardware performance evolves, and the range of existing wear- • An initial set of benchmark tests to evaluate wearables
ables increase. Devices like watches (smartwatches), fitness OS’;
trackers, smart bands, augmented reality (AR) / virtual reality • An OS comparison based on specific constraints, mea-
(VR) glasses and others can be easily found on stores [1][2]. suring performance and impact of each one;
Some of these products have also led to new performance • A comparative review of hardware available to be used
requirements, demanding more hardware processing power by wearables.
(e.g. devices with an interactive display that provides real-
II. M OTIVATION AND BACKGROUND
time feedback to the end user). In contrast, other solutions are
using simple approaches (e.g. fitness trackers that commonly A. Main Challenges
act as a single vital signal monitoring equipment, without One of the main ways to quickly improve wearables
having any visual interface). For the connectivity side, there performance is the addition of better hardware. However,
are wearables with local connectivity needs (e.g. Bluetooth or this is not the only way to achieve the same result, with
NFC), while others, aim to support a broad range of networks several researches being conducted in software part, including
and protocols. Briefly, the greater the device connectivity approaches to offload processing of heavy tasks to the cloud
degree and processing power the greater are the hardware (or to devices in the neighborhood) [8] and others [9].
resources necessary to sustain the whole system [3]. Analyzing the software side, an OS is in charge to control
From the software perspective, complex wearables solutions and manage system resources. Embedded Linux (raw version),
commonly consider the use of operating systems (OS’). Many Google Android Wear, Samsung Tizen, Linux Foundation
proposals vary from real-time approaches [4] to mobile OS’ Zephyr, RIOT OS, Google Android Things and Apple Watch
adaptations [5][6], providing more or less performance ac- OS are some examples that fit the wearables universe. Many
cording to its purpose. This fact may be associated with the of them are adaptations from smartphones/tablets versions,
broad range of applications covered by wearables solutions, what may force final products to consider a more powerful
making tough to decide the optimization focus an OS must hardware. These facts can be used as motivation to investigate
have. Despite that, an OS commonly has peculiarities that must the best OS’ to be employed in each wearable device appli-
be considered when applying it to a specific function or area, ance. A correct OS choice can improve system performance
and diminish energy consumption demand, besides reducing display that has its focus on data gathering and transmission
unnecessary expenses. (e.g.: health monitors and fitness trackers). This is the closest
Considering the research context, there are studies covering point to simple Internet of Things (IoT) devices; Layer 1:
wearables benchmarking, enclosing proposals for specific [10] Wearables using a simple display to show a layer of text
and general [11] solutions. In the same way, OS’ area also and 2D graphics (squares, circles, icons, and pre-processed
has a relevant number of researches covering its performance images) on the screen to represent physical places or points
evaluation through benchmarks [12]. However, an OS evalu- of interest; Layer 2: Addition of interaction to the features
ation inside wearables context is particularly difficult to find provided by Layer 1. Users can manipulate graphics on screen
in the literature, once most of the time the hardware is tightly overlaying the real world; Layer 3: Equipment with 3D-
coupled to the software. This challenge was addressed by this rendering capabilities that may (or may not) be able to process
paper isolating the OS’ inside virtual machines as a way to user interaction (used by AR/VR solutions); and Layer 4:
run performance tests and parametrize the obtained resulting Use of pattern recognition, machine learning and other AI
data. Another issue was the absence of a benchmark covering techniques to extract and recognize real-world objects. The
wearables context, what motivates the creation of an initial set GPU hardware may help graphics processing tasks to reach
of tests applicable to this embedded environment. real-time execution.
OS: Apple’s wearable OS used by Apple Watch products. by this research should be compared only between them, once
Works synchronized with iOS through a Bluetooth connection. some variation may be observed when contextualizing with
Developers can use an SDK to design and build watch OS real platforms.
applications [19].
C. Test Applications
IV. O PERATING S YSTEMS C OMPARISON M ETHODOLOGY Eight sets of tests focusing on I/O, graphics drawing,
A. Selection Criteria parallelism, and data processing were created. Except for
Three of those previously presented OS’ were chosen to Embedded Linux OS, which has used DFB library and GNU C
have a detailed evaluation: Generic Embedded Linux: Se- library (glibc), SDK functions provided by the manufacturers
lected due to its customization capabilities. Buildroot tool was were taken into account.
used to generate a customized embedded Linux. Only strictly • I/O: Test I/O operations reading/writing data to the local
necessary software modules were selected to be inside the filesystem. It was divided into two different sets: Latency:
kernel and root filesystem. For graphics side, Direct Frame Evaluate the time spent to read and write 30MB of raw
Buffer (DFB) library (version 1.7.1) was selected, once it is a data, separated in 512bytes-size packets; and Through-
lightweight solution with low overhead on drawings. As a way put: Evaluate the time spent to read and write 30MB of
to reach a good coverage, Linux OS was considered in three raw data, distributed in five different packets;
different architectures: X86, ARM and RISC-V, despite this • Image rendering time: Render time was verified through
latest one still in an “under development” phase; Samsung an application that draws an image to the display. Same
Tizen: A specific solution for wearables environment and 800x600 pixels image was considered for every scenario.
currently embedded on market products, such as Samsung When available, graphics processing unit (GPU) was
Gear S2 smartwatch. Tests were made using Tizen Wearable enabled to evaluate the OS behavior when receiving
OS version 2.3.1; and Google Android Wear: Provides a external help;
wearable environment specific solution. Android Wear version • Graphics drawing time: Check OS performance when
taken into account was based on Android 5.1 API 22. drawing vector figures. It was assessed the time needed
Operating systems other than those listed above were not to serially draw on the screen, 200 squares of the same
deeply examined due to particular reasons, such as limitations size. Like the previous test, GPU was also enabled when
on provided SDK, lack of an SDK (or its documentation), available;
low images maximum resolution, and a display with a limited • Processing time: Evaluate how OS’ deal when a high
scenario, with the two higher and two lower outliers being
removed. At every test, applications were restarted to ensure
no OS optimization/cache has remained in the background.
V. R ESULTS
Information presented by Table I makes possible to conclude
that, an OS has its focus in IoT or wearables area, the majority
of OS’ are open-source and have an SDK available. Figures
1,2,3 and 4 compare the OS’ running in virtual machines
and real hardware. Although this kind of comparison is not
technically correct, here it is used as a way to mensurate the
impact of real hardware. The data shown here considers 95%
of confidence interval.
A. I/O Latency and Throughput
Figure 1 summarizes the OS’ I/O performance tests results,
where the first graph is related to I/O latency, and the second
one describes I/O throughput data.
1) Latency: No matter the operation is being executed (read
or write) Android Wear has the lowest latency between the
three OS’, followed by the Linux Embedded (X86, ARM, and
RISC-V, respectively). Tizen OS reached the highest values
for latency, with its average value greater than four times the
value obtained during Android Wear experiments. Still, Tizen Fig. 1. I/O performance results of evaluated operating systems.
error rate is the highest one, letting us conclude that it also has
a high jitter on this type of I/O. Additionally, all OS’ spent
less time reading than writing data (even when considering 2) Graphics Rendering: In this set of tests, Embedded
real hardware). Linux OS’ got the worst average time, while Tizen and
2) Throughput: I/O throughput execution time is signifi- Android Wear presented the better results. This data may
cantly smaller than I/O latency. As happened in latency tests, also lead to conclude that Tizen and Android Wear has been
Tizen OS got the worst performance, with Android Wear and optimized to prioritize tasks that draw on the device display.
Embedded Linux (ARM) achieving best results, even when Unlike previous tests, real hardware platforms have performed
compared to real hardware platforms. Again, the time spent much better when compared to solutions running in a virtual
reading data is much smaller than the time spent writing this. machine. However, they were not good enough to surpass the
On both previous cases, Embedded Linux (RISC-V) has not behavior presented by the two best OS’ (Tizen and Android
performed well, a fact that could be due to its “under de- Wear).
velopment” status. In general, real hardware performed better
than OS’ running in virtual machines. However, Android Wear C. Processing Time
reached a performance in the same level of real hardware, Figure 3 depicts the OS’ processing time performance
suggesting that the SDK was modified with optimizations. results. The first graph is related to matrix multiplication, and
the latest one describes matrix transpose.
B. Graphics 1) Matrix Multiplication: It is possible to verify an “al-
Figure 2 summarizes the OS’ graphics performance results, most” homogeneous behavior when considering only OS’
where the first graph is related to image rendering, and the running on virtual machines. Due to the hypothesis raised
latest one describes graphics drawing. before, Embedded Linux (RISC-V) architecture had the worst
1) Image Drawing: Unlike previous results, the average performance. Embedded Linux (X86) architecture reached the
time spent to draw an image on the screen was statistically best results, a behavior the authors believe is due to internal
equivalent on Tizen and Android Wear OS’. Given the execu- configurations of caching. Performance presented by Intel
tion time difference when comparing Tizen/Android Wear with Edison and Wand boards are much better than the other two
Linux version, it is possible to affirm that some optimization platforms. This fact can be easily explained analyzing the
was made on the first two OS’. By its turn, both versions of hardware configuration of each board.
Embedded Linux with DFB library had not performed well, 2) Matrix Transpose: Execution time presented by matrix
even when GPU hardware was enabled. Another interesting transpose tests maintain the same behavior verified on matrix
point is the poor performance obtained by real hardware multiplication results, despite it takes less time: Embedded
platforms that were as bad as the Embedded Linux OS’ Linux (X86) has the best performance, while Embedded Linux
running on virtual machines. (RISC-V) the worst one. The explanation rises when analyzing
2018 15th IEEE Annual Consumer Communications & Networking Conference (CCNC)
TABLE I
O PERATING S YSTEMS C LASSIFICATION ACCORDING TO F EATURES
OS / Characteristic Graph. Support User Interface IoT Focus Wearab. Focus RTOS Open Source SDK Available
Samsung Tizen OS
Linux Found. Zephyr OS
Embedded Linux
Google Android Wear OS
Google Android Things
RIOT OS
Apple Watch OS
Fig. 2. Image rendering and graphics drawing performance of evaluated OS’. Fig. 3. Processing performance tests results.
real hardware platforms: Wand board hardware has the best 2) I/O Throughput + Graphics Drawing: It is possible
execution time. Still, Linux (X86) has a performance near to to verify the same behavior as in previously presented I/O
the real hardware platform. This behavior is impressive when throughput tests. As the execution time was reduced on
considering the context and previous results, once no particular throughput tests, Android Wear and Tizen have almost equaled
optimization was made on this Linux distribution. its behaviors. Embedded Linux (ARM) presented the worst
D. Parallelism execution time, followed by Embedded Linux (X86). The
authors believe this behavior is due to the lack of optimizations
Figure 4 depicts the tests results of OS’ parallel processing.
on Embedded Linux versions. In real hardware Wand board
The first graph is related to “I/O latency + graphics drawing”,
platform still leading as the best scenario. Indeed, when
and the second one describes “I/O throughput + graphics
comparing the top graph with below graph, it is possible to
drawing”.
verify the same behavior.
1) I/O Latency + Graphics Drawing: On most of OS’, the
addition of a new simultaneous task had no significant impact
E. General Analysis
when comparing the results presented here to those one listed
by I/O latency tests on Figure 1. When considering GPU- Results presented in previous subsections allow verify-
enabled, the execution time was reduced, however without ing the real impact of software on wearables environment
a notable variation. Additionally, results on real hardware performance. On almost every scenario (I/O, Graphics and
platforms show the hardware impact on performance, once Processing Time) there was a significant difference between
Wand board platform has a quad-core processor. the best versus the worst OS/platform. For instance, the
2018 15th IEEE Annual Consumer Communications & Networking Conference (CCNC)