Using Infrared Array Devices in Smart Home Observation and Diagnostics

IPASJ International Journal of Electronics & Communication (IIJEC)
Web Site: http://www.ipasj.org/IIJEC/IIJEC.htm

Email: editoriijec@ipasj.org
ISSN 2321-5984
A Publisher for Research Motivation........
Volume 4, Issue 1, January 2016
Using Infrared Array Devices in Smart Home

Observation and Diagnostics
Galidiya Petrova1, Grisha Spasov2, Vasil Tsvetkov 3,
1
Department of Electronics at Technical University Sofia, Plovdiv branch, Bulgaria
Department of Department of Computer systems and technologies Sofia, Plovdiv branch, Bulgaria
Department of Department of Computer systems and technologies Sofia, Plovdiv branch, Bulgaria
ABSTRACT
This paper describes a low-cost solution for smart home observation and diagnostics with the use of a new IR array sensor. The
developed system is based on PIC32-MAXI-WEB development board with integrated infrared sensor MLX90620. The system
structure and consequent steps of data processing in order to construct thermal image are described. Some details in the
process of raw data conversion to the assigned RGB values for every pixel in the field of view of IR sensor are discussed. The
developed system is considered to be with reasonable cost and affordable for the potential users in their everyday environment.
The paper demonstrates the benefits of having an infrared sensor as a part of the developed monitoring module for detection of
thermal losses and water leakage in a system for smart home diagnostics.
Keywords: smart home diagnostics, home automation, infrared sensor, thermography.
1. INTRODUCTION
There are many currently existing fields where infrared emission of the objects is measured, but with the increasing of
the capabilities of the affordable microcontrollers and IR sensors it is possible to develop low-cost solutions for
applications of infrared measurements in everyday environment. In a smart home network the IR sensor could be used
as a presence device or temperature sensor [1]. It also could alert if there is any appliances left turn on, like stoves,
heating devices, coffee makers, toasters and others that could potentially cause fire and having a way of preventing this
in the earliest stages is something that must be a feature of the smart home network. IR array sensor could serve as a
presence detector in security systems or in Heating, Ventilation and Air Conditioning (HVAC) systems, with the ability
to determine the number and positioning of the people. The people detection, counting and localization are considered
important and key control variables in smart sensing and modern HVAC on demand systems [2]. Having the precision
of a fraction of a degree Celsius and based on the size of the object it is possible to distinguish different objects like
animals that could potentially falsely alert that there is an intruder. Another possible useful application is in hospitals,
personal and ambient assisted leaving (AAL) systems where is necessary to have observation if the patient has fallen
and is not moving. In all these cases because of privacy issues it is recommended not using a normal camera for patient
monitoring [3]. However, the usage of the traditional thermal cameras that provide very high-resolution thermal images
for all these applications is very expensive. It is important these systems to have reasonable cost and to be affordable for
potential users in their everyday environment. In comparison of human-sensing methods for detecting presence, count,
location and tracking [4] using an inexpensive thermal infrared sensor gives promising alternative [5].
The goal of this paper is to present and demonstrate a low-cost solution based on application of a comparatively new IR
array sensor in the field of smart home diagnostics and observation.
2. OVERVIEW AND DESCRIPTION OF THE DEVELOPED SYSTEM

The developed solution is based on integration of an open source hardware platform and a small size IR array sensor.
The basic structure of the developed system is shown on Figure 1. It consists of 16x4 array infrared and temperature
sensor MLX90620, microcontroller development board PIC32-MAXI-WEB, router and PC.
The Melexis MLX90620 IR array sensor is easy to integrate, small in size and with low power consumption [6]. Due to
the factory calibrated temperature measurement with calibration parameters stored in the internal EEPROM it has high
precision of measurement. All these features are a good reason for selection of this reliable sensor to be used in the
developed solution for smart home diagnostics. In addition to 64 (16x4) IR pixels with dedicated low noise chopper
Page 9


ISSN 2321-5984
stabilized amplifier and integrated fast ADC, a Proportional To Absolute Temperature (PTAT) sensor is integrated to
measure the ambient temperature of the sensor chip. The output data of both IR and PTAT sensors are stored in the
internal RAM (16-bit result of IR measurement for each individual sensor (64 words) and 16-bit result of PTAT sensor)
and are accessible through I2C digital interface that supports clock speed up to 1MHz. The microcontroller connected to
the MLX90620 sensor can read the different RAM data and, based on the calibration data stored in the internal
EEPROM memory (256x8) of the sensor, compensate for difference between sensors to build up a thermal image, or
calculate the temperature at each spot of the imaged scene.
Figure 1 Basic structure of the system

Open source hardware platform PIC32-MAXI-WEB produced by Olimex is a development board equipped with a high
performance microcontroller (PIC32MX795F512L) [7]. It is easy for programing and supports the needed
communication protocols and the necessary ports. It has embedded 100Mbit Ethernet module that allows to develop
easily Ethernet connectivity applications. Along with the PIC KIT3 programmer there are some other useful features of
the microcontroller which are in help in the process of program development and debugging, like the 240x320 TFT
LCD with touchscreen, trimmer potentiometer and the buttons and leads.
The microcontroller periodically samples the sensor, gathers the information from the 64 infrared sensors (pixels) ant
the temperature sensor, then performs calculations for each pixel individually using the result from the temperature
sensor and the calibration constants obtained in the process of sensor manufacturing which are stored in its internal
EEPROM memory. The raw data from the sensor first are converted to human readable values, which in turn are
translated to values for red, green and blue for each corresponding temperature with the help of a look-up table of
values. The RGB values are stored in the microcontroller memory. Those values could be sent through a network to a
client with the help of the Microchip TCP/IP Stack. The stack has all the necessary network management
responsibilities and facilitates the process of establishing connection with the IR sensor via Internet. After the stack is
configured a small web application which is stored in the memory of the microcontroller services all external requests.
Upon request from the client side the developed web application provides the RGB data.
The microcontroller is connected to the network using RJ45 connector (Ethernet LAN). The microcontroller could be
accessed via web browser through the router using the fixed IP address. It is possible to enter a mode where
measurements are taken only at the users request. In this way the power consumption of the IR sensor is greatly
decreased and the microcontroller has resources to do some other useful work.
3. DATA PROCESSING
The data gathering and further processing is described in this section. As it was mentioned earlier inside of the TO-39
package the MLX90620 sensor contains the 16x4 IR array, thermometer, 65 words (16 bit) of RAM, 256 words of
EEPROM and two internal configuration registers. After the microcontroller establishes communication with the
sensor and initializes the configuration registers the sensor starts measuring the temperature of its field of view and the
proportional to ambient temperature of the package. The data is then being read periodically by the microcontroller
using the I2C digital interface protocol with clock speed supported from the sensor up to 1MHz. In order to extract
some human readable data from the raw 16 bit sequence of zeros and ones obtained from each IR pixel, the
microcontroller has to perform some heavyweight calculations, such as: offset compensation, thermal gradient
compensation, pixel to pixel normalization and emissivity compensation. This step could potentially become a threat
for a bottleneck if a microcontroller with lower computation power and limit of the frames per second visualization was
selected, despite the refresh rate of the sensor being of 0,5Hz to 512 Hz. At that point the raw data is an array of 64
Page 10


ISSN 2321-5984
floating point values that represent the temperature that each pixel sees. That itself is useful information, if the goal
is to target the temperature of a certain point and simply display it as a floating point value or to alert the system and
start executing some other application specific functions.
In the applications where the goal is to construct a thermal picture based on the values from the array it is necessary to
assign RGB values to each pixel. This process is very flexible as there are many ways of doing that. A very
straightforward approach is used in our system, because the colors of the table are chosen as follows: blue for the low
temperatures, red for the hot, with the transition from light-blue, blue-green, green, yellow and orange in between.
However, the range of the temperature that is measured is selected by the user, depending on the specific preferences
and the current application. The number of the entries of the RGB look-up table and each separate value for red, green
and blue is fixed. Determining which entry of the look-up table to pick is done by determining where inside the user
selected range is the value of the current pixel if it is below the low value we assume RGB values as if it is the lowest
in the table. If it is above the high boundary we assign the highest RGB values from the table. Anything value in
between means assigning RGB values for the corresponding temperature. This data processing is done by using the
following two relations:
S = (H - L) / N,
(1)
where: S is the step, temperature value that separates two entries in the table, H is the high boundary, L is the low
boundary and N is the number of entries in the table.
T = (P L) / S,
(2)
where T is rounded to integer and is the entry of the look-up table and P is the temperature of the pixel.
An option exists that allows the high and low boundaries to be selected automatically for each frame based on the
current highest and lowest measured temperature. By constantly collecting information for each pixel we can construct
a real time representation of the temperature of the surrounding area. Sometimes in the manufacturing process or due
to errors in the calibration process errors could visibly appear in some of the pixels. A practical approach of correcting
these types of errors is applied in the developed system that corrects those pixels and brings them closer to the real
value with minimal deviation. In that case a special calibration sequence is started for the damaged pixel. The
calibration procedure consists of taking measurements from object with consistent temperature up close so that each
pixel should see the same value. The value seen from the good pixels is considered the correct one and the value of the
bad pixel is considered with error. At least two measurements need to be taken for two different temperatures and all
the values are saved. Then the values are plotted on a Cartesian plane and the function y = f(x), where f(x) is a
polynomial so that when we plug in for x we get the approximated real value for y. As higher the degree of the
polynomial as closer will get approximation but also will add more complex calculation load and could slow the system
down if it gets too high. Finally, the information from the 64 pixels is stored in the microcontroller memory in the form
of RGB values. An example of thermal picture constructed from the RGB values and the pixels position in the whole
field of view is shown on Figure 2.
Figure 2 Thermal picture and the pixels position in the sensor field of view
Page 11


ISSN 2321-5984
4. APPLICATION OF THE IR SENSOR IN SMART HOME DIAGNOSTICS

Having a way of obtaining a thermal image of a certain area presents a wide variety of applications in smart home,
building observation and diagnostics. Very useful and attractive applications in building diagnostics are thermal loss
and water leakage. The high precision measurement could detect the small temperature gradient throughout the wall,
floor or ceiling and determine the source of thermal leakage. In case of water leakage IR sensor could detect the source
even before the water breaks to the surface and damages the interior. In order to demonstrate these possible applications
of our solution we have aligned spatially and matched the image from the web camera and the thermal image from the
IR array sensor. The results of this procedure for the two described cases for building diagnostics are presented on the
next figures. On Figure 3 an example for application of MLX90620 sensor in detection of thermal loss is shown, while
Figure 4 shows an example for application of the sensor in detection of water leakage.
Figure 3 Example for application of MLX90620 sensor in detection of thermal loss
Figure 4 Example for application of MLX90620 sensor in detection of water leakage

In general, with the employment of IR array sensors more complete information for the parameters of certain objects
and processes in home environment could be obtained. This will increase the understanding and simplifies the process
of picking the best reaction to the changes of important objects parameters. Additionally, the usage of infrared sensors
instead of simple everyday camera will help to avoid privacy invasion issues but still maintain the necessary
observation duties.
5. CONCLUSION
In this paper a low-cost solution for smart home diagnostics and observation with the use of a new IR array sensor is
described. Based on an open source hardware microcontroller and an inexpensive IR sensor the developed system is
considered to be with reasonable cost and affordable for the potential users in their everyday environment. The paper
demonstrates the benefits of having an infrared sensor as a part of the developed monitoring module for detection of
thermal losses and water leakage in a system for smart home diagnostics.
In the future work the IR sensor will be separated away from the main microcontroller thus having a battery powered
device that will communicate wirelessly with the access point. This will give additional flexibility for measurement of
objects in difficult to reach places without the need of wires. The further step is to build a network of those devices in
order to have a way of observing and tracking the subjects in hospital and home environment.
Acknowledgments
The work presented in this paper is supported by project 02/12 Investigation of methods and tools for
application of cloud technologies in the measurement and control in the power system and Melexis Microelectronic
Integrated Systems.
Page 12


ISSN 2321-5984
References
[1] Li, N., Calis, G., and Becerik-Gerber, B., Measuring and monitoring occupancy with an RFID based system for
demand-driven HVAC operations, Automation in construction 24 (2012), pp. 89-99.
[2] Melexis Microelectronic Integrated Systems, White paper MLX90620 for smart & green HVAC, April 2013.
[3] Hnat, T. W., Griffiths, E., Dawson, R., and Whitehouse, K. Doorjamb, Unobtrusive room-level tracking of people
in homes using doorway sensors, In Proceedings of the 10th ACM Conference on Embedded Network Sensor
Systems (2012), ACM, pp. 309-322.
[4] Teixeira, T., Dublon, G., and Savvides, A., A survey of human-sensing: Methods for detecting presence, count,
location, track, and identity, Technical report, Embedded Networks and Applications Lab (ENALAB), Yale
University, 2010.
[5] Kumar, S., Marks, T., Jones, M., Improving Person Tracking Using an Inexpensive Thermal Infrared Sensor,
CVPRW 2014 Intl. Conf. on Computer Vision and Pattern recognition Workshops, Columbus, USA, June 2014,
pp. 217-224, doi: 10.1109/CVPRW.2014.41.
[6] Melexis, Datasheet IR thermometer 16X4 sensor array MLX90620, http://www.melexis.com/InfraredThermometer-Sensors/ Infrared-Thermometer-Sensors/MLX90620-776.aspx, 2012.
[7] Olimex, Web server PIC32-MAXI-WEB with 32 bit PIC microcontroller PIC32MX795F512L datasheet,
https://www.olimex.com/Products/PIC/Development/PIC32-MAXI-WEB/resources/PIC32-MAXI-WEB.pdf, 2013.
AUTHORS
Galidiya Petrova received MSc. degree in Electronics at Technical University of Sofia in 1985 and PhD
degree in Biomedical engineering in 2001. She is presently Associate professor in the Department of
Electronics, Technical University of Sofia, Plovdiv branch. Her research interests are within: data acquisition
systems, applications of distributed systems in medicine, personalized healthcare and ambient assisted leaving
systems, wireless body sensor networks.
Grisha Spasov received MSc. degrees in Computer engineering at Technical University of Sofia in 1983 and
PhD degree in Computer networks in 2003. He is presently Professor in the Department of Computer systems
and technologies, Technical University of Sofia, Plovdiv branch. His research interests are within: Distributed
embedded systems and distributed applications, Internet of Things, Service oriented architectures for
distributed measurements, Computer networks and Wireless sensor networks.
Vasil Tsvetkov received BSc. degrees in Computer systems and technologies Technical University of Sofia,
Plovdiv branch, Bulgaria in 2015. He is presently MSc student in the Department of Computer systems and
technologies, Technical University of Sofia, Plovdiv branch. His research interests are in the area embedded
systems, firmware, ASIC design, FPGA prototyping and Internet of Things.
Page 13

Using Infrared Array Devices in Smart Home Observation and Diagnostics

Caricato da

Informazioni sul documento

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Using Infrared Array Devices in Smart Home Observation and Diagnostics

Caricato da

Copyright:

Formati disponibili

IPASJ International Journal of Electronics & Communication (IIJEC)

Web Site: http://www.ipasj.org/IIJEC/IIJEC.htm

A Publisher for Research Motivation........

Volume 4, Issue 1, January 2016