CCTV CAMERA WITH FACE RECOGNITION ATTENDANCE

CCTV CAMERA WITH FACE RECOGNITION

ATTENDANCE

Submitted in fulfilment of the requirements of

SUBJECT CODE- PRODUCT DESIGN (MMZG541)

BIRLA INSTITUTE OF TECHNOLOGY AND SCIENCE, PILANI

CAMPUS
PRESENTED BY

ROHAN KOLARKAR (2017HT30169)

ROHIT DHAGE (2017HT30103)

ROHIT SACHDEVA (2017HT79645)

ROSHNI RAVINDRAN (2017HT30204)

RUBINA PARVIN (2017HT30206)

RUCHI MORE (2017HT30049)

RUCHIKA SAWANT (2017HT30053)

RUPESH KUMAR (2017HT9066)

FIRST SEMISTER 2017

CCTV CAMERA WITH FACE RECOGNITION ATTENDANCE

INDEX

NO CONTENT PAGE
1 Abstract 1
2 Introduction 2
3 Customer survey 3
4 Mission statement 4
5 Structure design 4
I. Functional requirements 5
II. Non Functional requirements 5
6 CCTV camera with face recognition 4
attendance system 1
I. Problem statement 6
II. Proposed system 7
1. Registration 8
2. Image capturing 8
3. Identification 8
4. Verification 9
5. Marking attendance 9
6. Security 9
III. Algorithm 10
7 CCTV camera with face recognition 11
attendance system 2
I. Problem statement 11
II. Lecture attendance system 12
1. Architecture 12
2. Estimating student existence 13
3. Shooting plan 13
4. Face detection and recognition 14
5. Estimating the seat of each student 14
III. Experiment 17
1. Result of estimating the seat of 17
student
IV. Conclusion 18
CCTV CAMERA WITH FACE RECOGNITION ATTENDANCE

ABSTRACT
Uniqueness or individuality of an individual is his face. In this project face of an individual is
used for the purpose of the attendance making automatically. Attendance of the student is
very important for every collage, universities, school. Conventional methodology for taking
attendance is by calling the name or roll number of the student and the attendance is
recorded. Time consumption for this purpose is an important point of concern. Assume that
the duration for the one subject is around 60 minutes or 1 hour and to record attendance takes
5 to 10 minutes. For every tutor this is consumption of time.

To stay away from these losses, an automatic process is used in this project which is based on
the image processing. In this project attendance management system (AMS) can be made into
smarter way by using face recognition technique. Where, we use a CCTV camera to be fixed
at entry point of classroom, which automatically captures the image of the person and checks
the observed image with the face database using android enhanced smart phone, computer.

It is typically used for two purposes. Firstly, marking attendance for student by comparing the
face images produced recently and secondly, recognition of human who are strange to the
environment i.e. an unauthorized person.

This Automated attendance system developed for attendance eliminates the need of stationary
material and personnel for keeping records. This result improved performance over manual
attendance management system.

1
CCTV CAMERA WITH FACE RECOGNITION ATTENDANCE

INTRODUCTION
Attendance systems of old practises are not quite efficient today for keeping the track on
student’s attendance due to the availability of large resources over the internet today. In order
to drag the attention of students and make them interactive in observing technologies we
move on to the latest upcoming trends on developing attendance systems. This is the strong
reason for college attendance management system has to come up with an approach that
ensures strong contribution of student in classrooms.

To track attendance of the students, many attendance management systems are introduced in
the market. There are some attendance making systems used earlier in market are based on
RFID systems, punch card systems, swipe card systems, biometric system that includes
fingerprint analysis, iris analysis etc. Although it is automatic and a step ahead of traditional
method it fails to meet the time constraint. The student has to wait in queue for giving
attendance, which is time taking.

This project introduces an involuntary attendance marking system, devoid of any kind of
interference with the normal teaching procedures. The system can be also implemented
during exam sessions or in other teaching activities where attendance is highly essential. This
system eliminates classical student identification such as calling name of the student,
checking respective identification cards of the student, which can not only interface with the
on-going teaching process, but also can be stressful for students during examination sessions.

2
CCTV CAMERA WITH FACE RECOGNITION ATTENDANCE

CUSTOMER SURVEY
Survey/Customer Requirements on Attendance management system using face
recognition

1) Attendance Management system using Face Recognition: For the attendance

management, attendance marked for the student imaged matched and information
is sent to the server which controls the overall data base of the student. The
software installed in the mobile phone that would help to improve the report
features. When the server receives the message of student who are absent that
particular day will send SMS to the parent of that particular student.
2) False detection skin classification technique is used in face recognition based
attendance marking system and Two data base has to maintained , first one is face
database to store face images and extracted features at the time of enrolment
process and second attendance database contains the information about the
employees and is also used to mark attendance.
3) Face Recognition –based Lecture attendance system requirement for active
student detection method to estimate the existence of a student on a seat.
Otherwise student will enter the class just for attendance and will leave class after
completion of attendance.
4) Provide a valuable attendance service for both teachers and students.
5) Reduce the manual process error by provide automated and a reliable attendance
system uses face recognition technology.
6) Increase privacy and security which student cannot present him or his friend while
they are not present.
7) Produce monthly report for lecturers.
8) Flexibility, lectures capability of editing attendance records.
9) Calculate absenteeism percentage and send remainder messages to students.

3
CCTV CAMERA WITH FACE RECOGNITION ATTENDANCE

MISSION STATEMENT
CCTV Camera with Face Recognition Feature for
Product Description
Attendance
 Automatic data capture & data maintenance
Benefit Proposition  Wi-Fi controllable

 30% Profit margin

 Early market entry
Goals  Development of local languages
 Capturing 50% market in educational
institutes
Primary market Educational Institutions
Secondary market Government & Commercial Organizations
 Life – 8 years
 Maintenance free
Assumptions  Continuous usage
 Wi-Fi connectivity
 Software Controllable

 Purchasers and users

 Manufacturing operations
Stakeholders  Service operations
 Distributors and resellers

STRUCTURE DESIGN

Fig 1- Structure

4
CCTV CAMERA WITH FACE RECOGNITION ATTENDANCE

In figure we show the system architecture in which the camera detects and recognize
students' faces then sent the captured pictures to the system that compare the sent pictures
with retrieved images from the database which contains students' information and images.
Once the recognized face match a retrieved image, the attendance is marked for that person
and the attendance sheet is updated. If there are missed faces while detecting and recognizing
the process, the process will work repeatedly until all faces recognized. Finally, the result will
be shown at the display board desktop of computer or mobile screen attached with the
camera.

1. FUNCTIONAL REQUIREMENTS

System functional requirement describes activities and services that must provide.
1) Taking and tracking student attendance by facial recognition in specific time.
2) Sending the name of absent student directly to lecturer.
3) Permitting the lecturer to modify student absent and late.
4) Showing the name of who is absent or late on the screen to avoid error.

2. NON FUNCTIONAL REQUIREMENTS

Nonfunctional Requirements are characteristics or attributes of the system that can judge its
operation. The following points clarify them

1) Accuracy and precision- The system should perform its process in accuracy and
precision to avoid the problems.
2) Modifiability- The system should easy to modify, any wrong should be correct.
3) Security- The system should be secure and saving student privacy.
4) Usability- The system should easy to deal with and simple to understand.
5) Maintainability- The maintenance group should be able to fix any problem occurs
suddenly.
6) Speed and responsiveness- Execution of operation should be fast.

5
CCTV CAMERA WITH FACE RECOGNITION ATTENDANCE

CCTV CAMERA WITH FACE RECOGNITION ATTENDANCE

SYSTEM 1
I. PROBLEM STATEMENT

In the standalone application, face was captured by the webcam cameras and the detected
faces are stored in desktop webcam folder. For matching the captured images with the
database, Eigen faces methodology was used. Observance of Eigen faces method was
explained as,

1) Single structure of face pattern only allowed.

2) Gallery images must be of same size.
3) Requires full frontal face to be presented for each time.
4) Does not endure to the variations of brightness effect, pose and different expressions
of faces.
5) Effective only for low dimensional structure of face patterns.
6) The image which does not match with database is given by error message on desktop.
Where the instructor has to take the attendance to those unmatched students images
manually.

6
CCTV CAMERA WITH FACE RECOGNITION ATTENDANCE

II. PROPOSED SYSTEM

The smart applications for AMS are designed as shown in the architecture diagram. The
CCTV camera is fixed at the entry of the class room and used to observe the face of the
students. The observed image of the person is sent to the android mobile which is connected
to the CCTV camera.

Android mobiles are enriched with the face recognition software technique which actually
produces the possibilities of the human expression variations. The use of 3D face technology
enables the work of identifying, verifying and detecting the match images in the face
database. The diagram illustrates how the phases of the system are carried out. The smart
AMS consists of six modules which are explained detailed below.

CAPTURES
THE
ANDROID
STUDENT APPLY 3D
PHONE CROP THE STUDENT
FACE WHILE FACE
RECEIVE IMAGE DATABASE
ENTERING ALGORITHM
IMAGE
THE
CLASSROOM

Fig 2- General process

CHECK FOR CHECK FOR

MATCHING IMAGE MATCHING IMAGE

MATCHED UNMATCHED

MARK THE STRANGER

ATTENENCE DATABASE

Fig 3- Verification phase

7
CCTV CAMERA WITH FACE RECOGNITION ATTENDANCE

1. RESGISTRATION

First phase of the system, actually deals with registering the information of the student in a
particular classroom. The information includes

1) Name of the student

2) Register or roll number
3) Image of the student (taken by camera)

These details are placed in the student database from which the actual comparison will be
done.

2. IMAGE CAPTURING

The CCTV cameras are used for capturing the images of the student which will be in active
mode during the hours of college. The camera will be placed at the entry point of the
classroom when the student enters it automatically captures the image and send to the android
mobile to which it has been connected with.

The use of CCTV camera is that it is capable of capture the image of high quality and also at
different angles view.

3. INDENTIFICATION

To identify the student image, smart phone which holds the image database of the student,
checks for the match using 3D face recognition software technique.

In 3D face recognition technique 3D sensors are used to capture information about the
outline of a face surface. The obtained information is then used to identify the distinct
features on the face surface, such as the axis of the eye sockets, nose, and Chin

Fig 4- 3D sensors used for capturing information on the surface of a face

Steps followed in 3D face recognition technique are:

1) Obtained image is cropped.

2) To the cropped image a 3D face algorithm of canonical face matching is applied to
get different face reaction of the particular image.

8
CCTV CAMERA WITH FACE RECOGNITION ATTENDANCE

4. VERIFICATION

By the time of verification, dual process is done. One, the images of the students that are
captured recently is compared for the match in student database. In two of the probabilities
the images are checked. If the captured image matches with the image that has been
registered before are processed for attendance management.

Second, if it is observed to be unmatched with student database then the image of the person
will be consider as new and saved in the separate database called stranger database. The
separation of the database will provide some information about the stranger who is new to the
environment and gives the information about the person who has been entered. It not only
ensures security but also make some fear to the people who needed to be entered without any
authority.

5. MARKING ATTENDANCE

The image of the student which is obtained, matched with student database and the
attendance will be marked and the information is sent to the server which controls the overall
database of the student. The software is installed in the smart phone that would have much
additional functionality that would improve the AMS features and helps in finding the report
of each student either in the percentage or pie chart form as shown below. The figure shows
the overall percentage of attendance by considering the total number of working days and
number of days the student attend the college.

Fig 5- Pie-chart representation of student attendance month wise

6. SECURITY

In respect with the smart application doesn’t work for AMS, it is also useful for identifying
any new stranger who comes inside the classroom. Avoids a proxy attendance of the student
and ensures that the students of other class are not entering. The stranger can be the people
who are not allowed to the particular environment. By which security is enhanced. This
enhanced technology development can be used in various departments of government for
taking attendance for their working employees and ensure security over there.

9
CCTV CAMERA WITH FACE RECOGNITION ATTENDANCE

III. ALGORITHM

3D Face Algorithm is a canonical face matching, which a simple and efficient surface is
matching method based on high-order moments. The main idea is to represent the surface by
its moment’s µpqr up to some degree say P. which is smaller and equal to p+q+r. Compare
the moments as vectors in the Euclidean space. Given two face surfaces S1 and S2 with their
corresponding canonical forms X’1 and X’2. We can define the moments-based distance
between them faces as d^ (mom) (S1, S2) = ∑ (µ (pqr) (X’1) µ (pqr) (X’2)) 2

In practice, the vectors of moments are stored in the face database and compared to those at
verification.

Example-

Fig 6- Illustrates how the 3d Face Technology Applies Canonical Face matching to get
different faces expressions

The possibilities of face reactions are stored in the database called student database. Main
advantages of 3D face recognition are that it is not affected by any changes related to
brightness of the face surface unlike other techniques practised earlier. It can also identify a
face from a range of viewing away in the obtuse angled; this also includes a profile view of
faces.

Canonical analysis is a multivariate technique which is concerned with determining the

relationships between the groups of variables in a data set. The data set is divided into two
groups; let's call these groups as P and Q, based on some common characteristics defined.

The purpose of Canonical analysis is then to find the relationship between P and Q, i.e. can
some form of P represent Q. It works by finding the linear combination of P variables, i.e. P1,
P2 etc., and linear combination of Q variables, i.e. Q1, Q2 etc., which are most highly
correlated. This combination of highly correlated variables is known as the "first canonical
variants" which are usually denoted S1 and T1, with the pair of S1 and T1 being called a
"canonical function". The next canonical functions, S2 and T2 are then restricted so that they
are uncorrelated with S1 and T1. Everything is scaled so that the variance equals to one. The
relationships between the functions are constructed which are made to agree with constraint
restrictions arising from theory or to agree with common sense/experiment. These models are
called as “maximum correlation models”.

Mathematically, canonical analysis maximize S'P'QT subject to S'P'PS=1 and T'Q'QT=1,

where P and Q are the data matrices (row for instance and column for feature).

10
CCTV CAMERA WITH FACE RECOGNITION ATTENDANCE

CCTV CAMERA WITH FACE RECOGNITION ATTENDANCE

SYSTEM 2
CCTV camera with face recognition attendance system 2 able to mange to the context of the
students for the classroom lecture by using note pcs for the all students. Because this system
uses the note PC of each student, the attendance and the position of the students are obtained.
However, it is difficult to know the detailed situation of the lecture. This system takes images
of faces.

In recent decade, a number of algorithms for face recognition have been proposed, but most
of these works deal with only single image of a face at a time. By continuously observing of
face information, approach can solve the problem of the face detection, and improve the
accuracy of face recognition.

I. PROBLEM STATEMENT
1) It is difficult to estimate the attendance precisely using each result of face
recognition independently because the face detection rate is not sufficiently high.
2) It is time consuming process.
3) It does not represent students’ context in face to face classroom. It is difficult to
grasp the contexts by the data of a single moment. Students’ context such as
presence, seat position, status and comprehension.
4) Sometimes it is difficult to detect the faces according to the students when placed
in group and then system becomes ineffective.

11
CCTV CAMERA WITH FACE RECOGNITION ATTENDANCE

II. LECTURE ATTENDANCE SYSTEM

1) ARCHITECTURE

This system consists of two kinds of cameras. One is the sensing camera on the ceiling to
obtain the seats where the students are sitting. The other is the capturing camera in front
of the seats to capture images of student’s face. The procedure of system consist the
following steps

1) Seat information processing- This process determines the target seat to direct the
camera. These systems adopt the approach called Active Student Detecting
method (ASD). The idea of this approach is to estimate the existence of a student
sitting on the seat by using the background subtraction and inter-frame subtraction
of the image from the sensing camera on the ceiling.
2) Shooting plan- This system selects one seat from the estimated sitting area
obtained by ASD, directs the camera to the seat and captures images.
3) The system processes the face image- The face images are detected from the
captured image, archived and recognized. Face detection data and face recognition
data are recorded into the database.
4) Attendance information processing- This process estimates the attendance by
interpreting the face recognition data obtained by continuous observation. The
module obtains the most likely correspondence between the students and the seats
under the constrained condition. The system regards a student corresponded to
each seat as present. The position and attendance of the student are recorded into
the database.

The procedure is repeated during lecture, and estimated the attendance of the students in
real time.

Fig 7- Architecture of the system

12
CCTV CAMERA WITH FACE RECOGNITION ATTENDANCE

2) ESTIMATING STUDENTS’ EXISTANCE

Active Student Detecting Method (ASD) to estimate the existence of a student sitting on the
seat. In this approach, an observation camera with fisheye lens is installed on the ceiling of
the classroom and looks down at the student area vertically. ASD estimates students’
existence by using the background subtraction and inter-frame subtraction of the images
captured by the sensing camera. In the background subtraction method, noise factors like
bags and coats of the students are also detected, and the students are not detected if the colour
of clothes of them are similar the seats. ASD makes use of the inter-frame subtraction to
detect the moving of the students.

Fig 8- Active student detecting method

3) SHOOTING PLAN

Camera planning module selects one seat from the estimated sitting area in order to
determine where to direct the front camera. Actually, the module selects a seat by
scanning the seats sequentially. This approach is insufficient because it wastes time
directing the camera to where the student-and-seat. In other words, if we direct the
camera to each seat with the same probability, it is difficult to detect the faces according
to the student or the seat, and the system judges the students who are actually present to
be absent consequently. In order to solve this problem, it is important to the information
of each student’s position. The camera is directed to the selected seat using the
pan/tilt/zoom that has been registered in the database. The camera captures the image of
the student.

13
CCTV CAMERA WITH FACE RECOGNITION ATTENDANCE

4) FACE DETECTION AND RECOGNITION

Face detection and recognition module detects faces from the image captured by the
camera, and the image of the face is cropped and stored. The module recognizes the
images of student’s face, which have been registered manually with their names and ID
codes in the database. Face detection data and face recognition data are recorded into the
database.

5) ESTIMATING THE SEAT OF EACH STUDENT

In order to solve the problem of ineffectiveness, we integrated students’ seat information

into the camera planning. In this way, we can solve the problem such as misrecognition of
faces and seats by constraints of the correspondence relationship between them. The face
detected from the captured image may be another neighbour student’s face. Therefore, it
is necessary to consider the possibility that the face image is the one of a neighbour
student even if the camera is directed to the target seat. Considering the points we
mentioned above, we propose the following method.

Fig 9- The face of the student on back seat is detected.

We assume that every seat has a vector of values that represent the relationship between
the seat and each student. In the case that the module of face image processing recognizes
Student A’s face from the image of Seat B, our module votes for Student A’s component
of the vectors of the seats in the neighbourhood of Seat B. We assume the voting weights
in Figure. Each cell means a seat, and the grey centre cell means the focused seat. This
assumption means that, when Student A is recognized at Seat B, 0.24 is voted to Seat B,
and 0.11 is voted to the front seat of Seat B, and so on, for Student A’s components.

Fig 10- An example of voting weights

14
CCTV CAMERA WITH FACE RECOGNITION ATTENDANCE

Fig 11- Student A’s component of each seat when Student A is recognized at the grey seat

Fig 12- Student A’s component of each seat when Student A is recognized at the grey seat
after 1

Fig 13- An example of 2 students and 2 seats

15
CCTV CAMERA WITH FACE RECOGNITION ATTENDANCE

For example, Figure shows Student A’s components of each seat when Student A is
recognized at the grey seat, and Figure shows the case that Student A is recognized at the
grey seat in the next step. Considering the bipartite graph of the students and the seats,
voting can be thought of as the addition to the scores of the edges between the students
and the seats, and the cost of the edge is defined as the inverse of the score of the edge.
Before the seat information processing, we set two conditions as the premises:

1) More than two students are not sitting on the same seat.
2) The students do not move to different seats frequently.

The process of seats information does not select independently the seat that has the
highest score for each student but use the approach that find the matching in the bipartite
graph such that the sum of the costs of the edges are minimized where the premises are
satisfied. Figure shows an example of the bipartite graph in the case that two students and
two seats exist. In this case, our approach obtains the two thick arrows as the
correspondence. Our process solves linear sum assignment problem (LSAP) to estimate
the correspondence. We assume the assignment of student i to seat j incurs a cost c (ij).
The problem is formulated as follows:

The least complexity of the best sequential algorithms for the LSAP is O (n^3), where n is
the larger one of the numbers of the students or the seats. Thus, this problem is solved in
real time. In this procedure, the system regards the students corresponded to the seats as
present.

16
CCTV CAMERA WITH FACE RECOGNITION ATTENDANCE

III. EXPERIMENT
1. RESULT OF ESTIMATING THE SEAT OF THE STUDENT

19 students existed in the canter area, and we ran the process of camera control and
detection for 20 minutes. We labelled the images of the detected faces with the name of
the students manually. The system detected faces 186 times, and 15 students were
detected. Table shows the accuracy of seat estimation. We have compared the result of
estimating the seat of each student by using the estimating the seat of each student
method. Method 1 is the method that correspond each student to the seat where them the
most faces of the student are detected. Method 2 is the method that correspond each
student to the seat that has the lowest cost of the student. Method 3 is the estimating the
seat of each student. Denominator of fractions in this table is the number of the face-
detected students. This table shows that accuracy is improved by the estimating the seat
of each student method.

Method Accuracy
Method 1 60.0% (9/15)
Method 2 73.3% (11/15)
Method 3 80.0% (12/15)
Table 1: Estimating the seat of each student

Time Face detection rate

1cycle only 37.5% (3.8/10)
79 min 80.0% (8/10)
Table 2: Face detection rate

Time Precision Recall F score

1 cycle only 89.2% 33.8% 48.3%
79 min 70.0% 70.0% 70.0%
Table 3: Result of estimating the attendance

17
CCTV CAMERA WITH FACE RECOGNITION ATTENDANCE

IV. CONCLUSION
1) The system estimates the attendance and the position of each student by
continuous observation and recording. The result of our preliminary experiment
shows continuous observation improved the performance for estimation of the
attendance.
2) It is also possible to know whether students are awake or sleeping and whether
students are interested or bored in lecture if face images are annotated with the
students’ name, the time and the place.

18