Design of a Vehicle Driver Drowsiness Detection

System through Image Processing using Matlab

Melissa Yauri-Machaca Brian Meneses-Claudio Natalia Vargas-Cuentas
Image Processing Research Laboratory Image Processing Research Laboratory Professional Member, IEEE
(INTI-Lab) (INTI-Lab) Image Processing Research Laboratory
Universidad de Ciencias y Universidad de Ciencias y (INTI-Lab)
Humanidades Humanidades Universidad de Ciencias y
Lima, Perú Lima, Perú Humanidades
yaurimelissa@gmail.com bmeneses@uch.edu.pe Lima, Perú
natalia.i.vargascuentas@ieee.org
Avid Roman-Gonzalez
Senior Member, IEEE
Image Processing Research Laboratory
(INTI-Lab)
Universidad de Ciencias y
Humanidades
Lima, Perú
avid.roman-gonzalez@ieee.org

Abstract— A person when he or she does not have a proper
rest especially a driver, tends to fall asleep causing a traffic
accident. It is why the present work wants to realize a system
that can detect the drowsiness of the driver, in order to reduce
traffic accidents. For that system, it will take the processing of
images through a camera which will focus on the driver. In
that, it is going to analyze the changes that happen in the face
and then will be processed through a program in order to
detect drowsiness to send an alert to the driver.
Keywords— Drowsiness, system, processing of images, face.
I. INTRODUCTION
The human has a fundamental need called sleep because
with adequate rest, helps to maintain efficient operation of
the capacities that realize a person. However, when this
activity is interrupted by several factors, especially in a
driver, it deteriorates the psychomotor and cognitive
functions such as reaction time, the capacity of surveillance,
judgment, and attention.
When the driver does not have adequate rest, the driver
will try to sleep while driving and this is the main sign of
drowsiness. In most cases, the driver does not pay attention,
and then the yawns come, the attempt to close the eyes every
moment and the movement of the head from side to side [1].
It has as a consequence a traffic accident, which is an
unfortunate event for the driver of the vehicle, as well as for
the pedestrian.
Currently, there are studies for the creation of a
drowsiness detection system, which extracts the essential
characteristics of drowsiness of the driver to determine the
level of drowsiness. In these systems, which are controlling
the state of drowsiness of the driver through a webcam with
night vision to track the driver in real time and when
drowsiness is detected, the system will send a warning [2].
The objective of these systems is to improve the safety of
people with the use of detection and alarm to avoid
accidents caused by drowsiness of the driver and which are
harmful to the users of the route [3].
In Peru, according to the Instituto Nacional de Estadística e
Informática shows that 0.6% of traffic accidents are caused
by drowsiness because the driver believes that taking a rest
for a few seconds, closing his eyes, it can cause to losing the
control of the vehicle [4]. For this reason, in the present
investigation, the designs of a system able to detect the
drowsiness of the driver in order to be alerted are presented.
Also, drowsiness detection of the driver's approach driver is
confronted with image processing to recognize drowsiness
patterns.
In section II, the development of the objectives. Then, in
section III presents the results according to the objectives. In
section IV, a discussion about the results is shown and,
finally, in section V, which exposes the conclusions.
II. METHODOLOGY
The development of the research is following the next
objectives below:
A. The causes of driver drowsiness
• Sleep less than 8 hours: According to a study
carried out in the final bus station of Huancayo to
100 interprovincial bus drivers, it states that in the
24 hours prior to the survey, 47% of the drivers had
slept six hours or less [5]. This sample, the driver
does not obey the eight hours appropriated for the
quality of sleep in order to affect the health. For
example, the person gets sick every moment,
decrease in the mood and quick reaction to avoid
some inappropriate event.
In addition, in [6] developed a study on the number of
hours that the driver sleeps in a formal and informal
business. This study focuses on day and night and shows in
tables 1 and 2.

978-1-5386-6122-2/18/$31.00 ©2018 IEEE

 Therefore, the largest percent of the sleeping environment
is in the trunk of the car. This causes various sleeping problems
such as body ache and is not an appropriate place due to the
existence of noise from other cars. So the person recognizes
that the noise damages the quality of sleep.
• No work schedule: Drivers realize long work days
without systematic programming does not allow
adequate rest. They work in a disorganized way on day
shifts and night shifts and rest an average of four to five
hours per day [6]. On the other hand, the driver's job
does not end when they arrive at their destination,
Fig.1: hours of sleep in formal and informal drivers on day shift because they have to clean the car. And in [7] according
to Table 4 shows an alarming number of days the driver
works. 20% in the formal business and 29% in the
informal companies, where the drivers work every day.
Fig. 4: work shifts by night shift per week

Fig. 2: Hours of sleep in formal and informal drivers on night shift

In Table 2, there is not much difference between

informal and formal drivers in the night. The study presents
42% of formal drivers and 30% of informal drivers sleeping
around 5 to 7 hours. This happens when the drivers modify B. The patterns of drowsiness
the circadian rhythm because the human is destined to
perform some action in the morning and at night is sleeping The drowsiness in the driver presents several
for the hormone melatonin. And for that reason, the driver facial changes such as:
cannot sleep well. • Frequent flicker
• Moving the head from side to side
• There is no appropriate sleep environment: The driver
• Yawn
has to make a change with another driver for rest. But
the sleep environment does not have the necessary In table 5, a study is shown in Arequipa about the
implements, nor the conditions of space and calm to
company Corattsa and other companies. In this study, the
sleep. In [5] it describes a study at the bus station Fiori
common patterns of drowsiness in the driver are detailed.
and Huancayo. The first has 81% of drivers sleep in the
trunk of the car and in the second is 62%. In addition, in
the first final bus station, 50% of the drivers sleep when
the car is in motion and 42% of the final second bus
station. In table 3 shows what has been described.

Fig. 5: Sleepiness patterns

In the study of the first route is towards Cerro Verde, the

second is towards the province of Arequipa and the third route
are provinces adjacent to Arequipa and the fourth route is from
Arequipa to Lima. According to the image, the most relevant
Fig. 3: Driver's sleep environment model is to open and close the eyes (blink), which has 59% in
the third route [8]. And this pattern will be analyzed to
detect drowsiness.
Blinking is a small eyelid depression whose main
purpose is to keep the external part of the eye moist,
avoiding the evaporation of the tear film and maintaining the
integrity of the ocular surface and the optical quality of the
cornea [9]. The importance of opening and closing the eyes,
offers good vision because a driver with drowsiness has
blurred vision. In drowsiness opening and closing the eyes is
more frequent and does not fully realize. Since the blinking
is very fast and does not close the eyelids. Fig. 7: Camera Configuration
In the blink of an eye, two factors that influence it will In this stage, which consists in connecting the camera
be studied, which are: with the MATLAB software, so it emits a command called
"imaqhwinfo". This command permission is to know the
• Flicker frequency: The number of blinking that a driver
name of the camera in the software, in this case, it is called
makes during a certain time. According to equation 1.
°
1
"WinVideo". And, finally, the "imaqhwinfo ('WinVideo', 1)",
the command is executed to check the characteristics of the
The blinking frequency in a person with sleep is connected camera, as well as the resolution.
approximately 21 blinks per minute and in the normal state,
For the capture of the images, two environments are
the person has 15 blinks per minute. chosen, the first is considered in the day and the second in the
• Opening of the eyes: It is the amplitude between the night. At the time of image acquisition, it focuses on the
two eyelids at the time of opening and closing the eyes driver's face. In the software, an algorithm is developed
in the closing stage, as shown in image 6. The process generating a frame with a rectangle to the face, which indicates
begins when the pupils of the eyes are covered by the that there is a face to be processed as shown in figure
eyelids. that the upper and lower eyelids are open. 8. This process is done to more effectively produce the
detection of the patterns of drowsiness.

Fig. 8: Face Detected

Fig. 6: The closing process of a blinking
• Image Processing: In this stage, the image becomes a two-
And for the calculation of this factor PERCLOS is used, to dimensional matrix when it is processed in MATLAB as
determine the percentage of the closing of the eyelids. The shown in figure 9. Each element of the matrix corresponds
following mathematical formula presented in equation 2 is to a pixel of the image. The fundamental thing is the
used, where t1 to t4 is the time by which the opening of the eye preparation of the image so that the detection is effective.
will be limited to being completely open from the closure. The process begins with the modification of the
100% 2
characteristics of the image to achieve an improvement in
the image. The characteristics are: the contrast and the noise
that exists in the image. On the other hand, several filters
C. The stages of the system are applied to the image before detection.
In this objective, each part of the driver's drowsiness
detection system is developed, which consists of the stage of
image acquisition, processing, detection and warning. These
pieces are [10]:
• Image acquisition: In this stage, the camera is used to
acquire the image of the driver. In the camera, an
adjustment is made in the center of the camera as shown
in figure 7. The axis moves around 90 ° to capture a
good image and when it is necessary to use the lighting
built into the camera.
 Fig. 9: Matrix of the image
• Detection: At this stage, we will proceed with the
identification of drowsiness patterns. To process the
patterns will begin with the isolation of the section of
interest as the eyes. Then, the process of extracting
characteristics begins, which analyzes the moment of
closing the eyes and the distance from the opening of the
eyes.
As shown in Figure 10, the programming takes 3 frames where
the first one will capture the eyes of the driver, in the second
frame, it places the eyes and finally, in the third frame the
program analyzes that the eye is opened by a green circle.
Fig. 10: Image processing with open eyes
Figure 11 shows the same process, but in this case with
closed eyes indicating that there is a blinking.
Fig. 11: Image processing with close eyes
• Alarm: Finally, after detection, the system will emit an
audible alarm to warn the driver to have drowsiness. The
alarm varies according to the pattern that has been detected
in relation to the sensitivity level. So the driver does not get
used to a repetitive tone and ignores the warning.
The system consists of the camera, which acquires the
image. The camera used belongs to the brand Micronics
with the model W360 MIC and consists of high resolution
with great clarity and is connected via USB to the laptop.
The use of the laptop will proceed with image processing
and driver drowsiness detection using MATLAB software.
Finally, the alarm is integrated into the software. The system
is in figure 12.
Fig. 12: System elements
The MATLAB program is showing next:
clear all
clf('reset');
cam=webcam(); %camara
parpadeo=imread('hay_parpadeo.jpg');
noParapadeo=imread('no_parpadeo.jpg');
detector =
vision.CascadeObjectDetector();
detector1 =
vision.CascadeObjectDetector('EyePairBig
');
contador =0;
while true
vid=snapshot(cam); %
vid = rgb2gray(vid);
img = flip(vid, 2); %

bbox = step(detector1, img);
if ~ isempty(bbox)
biggest_box=1;
for i=1:rank(bbox) %find the
biggest face
if
bbox(i,3)>bbox(biggest_box,3)
biggest_box=i;
end
end
faceImage =
imcrop(img,bbox(biggest_box,:)); % stop
the face
bboxeyes = step(detector1,
faceImage); % eyes located
subplot(3,2,1),subimage(img);
hold on; % showing
for i=1:size(bbox,1) % focus
the face
rectangle('position',
bbox(i, :), 'lineWidth', 2, 'edgeColor',
'y');
end
subplot(3,2,3),subimage(faceImage);
if ~ isempty(bboxeyes) %check
it eyepair is available
biggest_box_eyes=1;
for i=1:rank(bboxeyes) %find
the biggest eyepair
if
bboxeyes(i,3)>bboxeyes(biggest_box_eyes,
3)
biggest_box_eyes=i;
end
end
floor(r+r/2)], 'ObjectPolarity','dark',
'Sensitivity', 0.93); % Hough transform
[M,I] = sort(radii,
'descend');
eyesPositions = centers;
subplot(3,2,2),subimage(eyesImage); hold
on;
viscircles(centers,
radii,'EdgeColor','g');
%% count
cent= numel(eyesPositions);
if cent ==0 ;
contador= contador+1;
disp('hay parpadeo')
subplot(3,2,4);
subimage(noParapadeo);
else cent==1; contador
=contador; disp('no hay
parpadeo');
subplot(3,2,4);
subimage(parpadeo);
end
if contador >= 5;
disp('Tienes
somnolencia');
end
end
end
end
set(gca,'XtickLabel',[],'YtickLabel',[]) ;

bboxeyes=[bboxeyes(biggest_box_eyes,1),b hold off;

boxeyes(biggest_box_eyes,2),bboxeyes(big
gest_box_eyes,3)/3,bboxeyes(biggest_box_ As shown in the program, it has a limit of 5 blinks for a
eyes,4)]; %resize the eyepair width in period of time, in case it is exceeded, an image is shown
half indicating that it has exceeded the number of blinking norms.

eyesImage = III. RESULTS

imcrop(faceImage,bboxeyes(1,:)); The analysis of the causes of drowsiness, the axis of the
%extract the half eyepair from the face causes lies in that it is not a quality of sleep, which is
image dominated by the hours the driver sleeps, the place where
eyesImage = the driver and there is not a correct work schedule.
imadjust(eyesImage); %adjust contrast Then in the second objective, the result is that the pattern of
drowsiness is in the eyes because drowsiness does not allow the
eyes to be kept open. Finally, it was obtained that the most
r = bboxeyes(1,4)/4; notable characteristics are the frequency of blinking and the
[centers, radii, metric] = opening of the eyes. The design of the system is based mainly
imfindcircles(eyesImage, [floor(r-r/4) on the analysis of the patterns of these when driving so, when
the driver shows these variations on his face, which is
detected.
The result of the study of the stages of the system, you
get the hierarchical knowledge of how the system will
work, this information is important because it will have to
be declared hierarchically in the Matlab software. Also for
each stage the necessary characteristics were obtained to
make adjustments when processing the image.
IV. DISCUSSION
The present study confirms that drivers do not have
adequate rest, which causes drowsiness. And drowsiness is
one of the factors that cause traffic accidents. Therefore, it
is necessary to perform a system that detects the drowsiness
of the driver.
There are several studies, where the system detects
drowsiness of the driver of the vehicle, the drowsiness
patterns are such as: blinking, level of distraction and
yawns. These patterns determine a level of drowsiness
however the level of distraction presents difficulties in the
case of this outside a range established to detect the face.
In this case only in the frequency of blinking and the
opening of the eye since they are the most dominant
symptoms that fit the camera within the rectangle that is set
to detect the face, which helps to have more efficiency in
the system.
Also in this study as in others, it is important to order
the stages of the system as this verifies the elements that
are used and the requirements that are needed to make the
system more efficient. In conclusion, the analysis of
drowsiness patterns using image processing is innovative
due to the detailed analysis of facial changes, that is, they
have a real-time analysis of what the driver experiences
when drowsiness enters.
V. CONSLUSIONS
In conclusion, drivers have a bad habit to sleeping
anywhere, which provides a bad break and as a
consequence they have drowsiness.
In summary, drowsiness patterns are a fundamental part
of the detection process, as this alerts the driver to reduce
traffic accidents
In summary, each stage of the system depends on the
other, therefore, it is important to know what stages, our
system will have. Knowing the stages, we can know the
hierarchy of the installation or operation.
REFERENCES
[1] R. Jimenez Moreno, «Sistema de detección de nivel de cansancio
en conductores mediante técnicas de visión por computador,»
Universidad Nacional de Colombia , Bogotá , 2011.
[2] G. D. Brito Moncayo y W. L. López Romero, «Sistema de control
del estado de somnolencia en conductores de vehículos,»
Universidad Técnica de Ambato, Ambato, 2016.
[3] K. Saldaña Altamirano y R. Rodríguez Urquiaga, «Detección de
la somnolencia usando maquinas de soporte vectorial a partir de,»
Universidad Nacional De Trujillo, Trujillo, 2014.
[4] Instituto Nacional de Estadistica e Informatica, Noviembre 2015.
[En línea]. Available:
https://www.inei.gob.pe/media/MenuRecursivo/publicaciones_dig
itales/Est/Lib1308/libro.pdf. [Último acceso: Mayo 10 2017].
[5] E. Rosales Mayor , I. S. Durand Vila , N. E. Montes Ccaccro , C. .
G. Alonso Cueva , L. L. Merino Baquerizo , . M. J. Rey de
Castro , M. T. Egoavil Rojas Martha Teresa, R. E. Flores Herrera
y S. L. Rivera García , «Accidentes de carretera y su relación con
cansancio y somnolencia en conductores de ómnibus,» Revista
Medica Herediana, vol. 20, nº 2, pp. 48-59, 2009.
[6] E. Rosales Mayor y J. Rey De Castro Mujica, «Somnolencia: Qué
es, qué la causa y cómo se mide,» Acta Médica Peruana, vol. 27,
nº 2, pp. 137-143, 2010.
[7] G. Liendo, C. Castro y J. Rey de Castro, «Cansancio y somnolencia en
conductores de ómnibus interprovinciales: estudio comparativo entre
formalidad e informalidad,» Revista Peruana de Medicina
Experimental y Salud Publica, vol. 27, nº 2, pp. 187-194, 2010.
[8] R. Castro y S. Soriano, «Hipersomnia durante la conducción de
vehículos ¿causa de accidentes en carreteras? A propósito de un
estudio cualitativo.,» Rev. Soc. Peru Med Interna, vol. 15, p. 142,
2002.
[9] N. Quevedo López, « Estudio del parpadeo durante la conducción
de vehículos (aspectos cognitivos y de flujo de información),»
Universidad Politécnica de Catalunya., Barcelona, 2012.
[10] Vargas-Cuentas, Natalia I., and Avid Roman-Gonzalez. "Facial
image processing for sleepiness estimation." Bio-engineering for
Smart Technologies (BioSMART), 2017 2nd International
Conference on. IEEE, 2017.
 PENGOLAHAN CITRA DAN VISION

REVIEW JURNAL INTERNASIONAL TEMA

PENGOLAHAN CITRA DIGITAL

Oleh :
Panji Budi Satria (E41160482)

PROGRAM STUDI TEKNIK INFORMATIKA

JURUSAN TEKNOLOGI INFORMASI
POLITEKNIK NEGERI JEMBER
2019
1. Judul Jurnal
Design of a Vehicle Driver Drowsiness Detection System through Image
Processing using Matlab
2. Abstrak
Seseorang ketika dia tidak memiliki istirahat yang layak terutama sopir,
cenderung tertidur menyebabkan kecelakaan lalu lintas. Itu sebabnya pekerjaan
saat ini ingin mewujudkan suatu sistem yang dapat mendeteksi kantuk
pengemudi, untuk mengurangi kecelakaan lalu lintas. Untuk sistem itu, ia akan
mengambil pemrosesan gambar melalui kamera yang akan fokus pada
pengemudi. Dalam hal itu, ia akan menganalisis perubahan yang terjadi di wajah
dan kemudian akan diproses melalui program untuk mendeteksi rasa kantuk
untuk mengirim peringatan kepada pengemudi.
3. Introduction
Manusia memiliki suatu kebutuhan yang harus dipenuhi salah satunya yaitu tidur. Disaat
manusia kurang dalam tidur maka akan memperburuk fungsi psikomotorik dan kognitif pada otak.
Terutama pada pengemudi, pengemudi diharuskan memiliki tidur yang cukup pada saat
berkendara, disaat pengemudi kurang tidur pada saat berkendara, pengemudi akan mencoba untuk
tidur ini adalah tanda utama kantuk. Kantuk karena kurangnya tidur menyebabkan kecelakaan lalu
lintas.
Saat ini sudah ada penelitian tentang system deteksi kantuk. System ini akan mengawasi
pengemudi melalui webcam dengan night vision untuk melacak pengemudi secara real time.
Ketika kantuk terdeteksi maka system akan mengirimkan peringatan dalam bentuk alarm. Tujuan
dari sistem ini adalah untuk meningkatkan keselamatan orang dengan menggunakan deteksi dan
alarm untuk menghindari kecelakaan yang disebabkan oleh kantuk pengemudi dan yang berbahaya
bagi pengguna jalan.
4. Method
Mencari data dengan parameter frekuensi kedipan dan pembukaan mata,
1. Frekuensi Kedipan

Frekuensi berkedip pada seseorang dengan tidur sekitar 21 berkedip per menit
dan dalam kondisi normal, orang tersebut memiliki 15 berkedip per menit.
2. Pembukaan Mata
 Dan untuk perhitungan faktor ini PERCLOS digunakan, untuk
menentukan persentase penutupan kelopak mata. Rumus matematika berikut
ini disajikan dalam persamaan 2 digunakan, di mana t1 ke t4 adalah waktu di
mana pembukaan mata akan terbatas untuk sepenuhnya terbuka dari
penutupan.

5. Experiment and Result

Eksperimen dilakukan dengan menggunakan kamera Micronics dengan model
W360 MIC yang terhubung dengan laptop. Kamera ini digunakan untuk mengambil
gambar pengemudi saat berkendara. Lalu kamera ini terhubung dengan laptop
melalui usb yang nantinya laptop ini akan memproses gambar yang diperoleh dari
kamera dengan program MATLAB. Disaat mata pengemudi terdeteksi tertutup maka
system akan membunyikan alarm.
Analisis penyebab kantuk, poros penyebabnya terletak pada bahwa itu bukan
kualitas tidur, yang didominasi oleh jam tidur pengemudi, tempat di mana
pengemudi dan tidak ada jadwal kerja yang benar.
Kemudian pada tujuan kedua, hasilnya adalah bahwa pola kantuk ada di mata
karena kantuk tidak memungkinkan mata tetap terbuka. Akhirnya, diperoleh bahwa
karakteristik yang paling menonjol adalah frekuensi berkedip dan pembukaan mata.
Desain sistem ini terutama didasarkan pada analisis pola-pola ini saat mengemudi
sehingga, ketika pengemudi menunjukkan variasi ini di wajahnya, yang terdeteksi.
Hasil dari studi tahap-tahap sistem, Anda mendapatkan pengetahuan hirarkis
tentang bagaimana sistem akan bekerja, informasi ini penting karena harus
dinyatakan secara hierarkis dalam perangkat lunak Matlab. Juga untuk setiap tahap
karakteristik yang diperlukan diperoleh untuk membuat penyesuaian saat memproses
gambar.
6. Conclusion
Kesimpulannya, pengemudi memiliki kebiasaan buruk untuk tidur di mana saja,
yang memberikan istirahat yang buruk dan akibatnya mereka mengantuk. pola
kantuk adalah bagian mendasar dari proses deteksi, karena ini mengingatkan
pengemudi untuk mengurangi kecelakaan lalu lintas
Singkatnya, setiap tahap sistem tergantung pada yang lain, oleh karena itu,
penting untuk mengetahui tahap apa, sistem kita akan miliki. Mengetahui tahapan-
tahapannya, kita bisa mengetahui hierarki instalasi atau operasi.