Autonomous Quadricopter for Fire Detection

By Sander Idelson Project with Paper

Presentation Outline

Introduction Why Use a Drone? Drone Limitations The Parrot AR Drone 2.0 Fire Detection Control Algorithm Future Work Conclusion

Introduction

In June 2013, a forest fire in Arizona killed 19 fire fighters The fire approached the team unexpectedly and they were caught helpless Goal Program a drone to autonomously detect and track a forest fire Keep fire fighters constantly aware of their surroundings

Why Use a Drone?

Cost is low ~$300-5,000 Equipped with many sensors, cameras Small computers on-board Wireless communication Small size No danger to humans

Drone Limitations

Laws

Drones are not allowed above 400 feet Some states do not allow drones FAA regulation

Hot topic people dont like drones

They are associated with attacks and spying

Weather and wind effect their use

Parrot AR Drone 2.0

Commercial drone - $300 Quadricopter design stable and light Can be driven with iPhone, Android apps out-of-thebox Huge development community

Specifications

Weighs 0.96 lbs Made of carbon fiber and has protective foam On-board 1 GHz Linux computer 2 color video cameras (720p and QVGA) Full Inertial Measurement Unit Broadcasts its own Wi-Fi network

Limits range Extended with antenna

Many other features

Programming

Hayes Modem Commands AT*Commands Programmable from any language NodeJS Javascript serverside platform Node-ardrone library by Felix Geisendrfer

Large open Source Community Tough to program everything with Javascript Callback functions, asynchronous Easier language to use More support Better for creating programs

Python python-ardrone library by Bastion Venthur

Calibration

Drone is inconsistent and prone to blowing around Time of turn calibrated for different angles Linear and Quadratic functions were found (Quad was better)

Fire Detection

Infrared Camera

Preferred choice Hardware connection to on-board computer too complicated Use on-board cameras Perform frame filtering with OpenCV and SimpleCV libraries Try a few different current algorithms

Color video

Color Video Fire Detection

RGB Color Space Segmentation [15] - Binary Filter

Worked for images, not as well in video Depends on light Rmean is often too low Needs to be calibrated to the amount of light Impractical for real world use More uniform color space Better results Fire flickers at roughly 10 Hz

RGB Static Thresholds [17] Binary Filter

CIE L*a*b* Color Space Segmentation [14] Binary Filter

Frame Subtraction Movement Filter

Many more

Object Detection

Once image is filtered SimpleCV has function called findBlobs

Take average position of 5 largest blobs because fire is sparse

2-D distance measurement to fire (rough estimate)

Take image and find blobs Turn drone theta degrees Take another image and find blobs Perform , c is pixel distance, r is distance to fire

Control System

Simple Control System

Provides a base for a larger control system

Slides along fire at a safe distance Just above tree line (~200 feet) Sends GPS coordinates & video back to base Can be overridden to manual mode Performs one or many of the fire detection algorithms Checks if fire is moving with frame subtraction

Future Work Fire Detection

Equipment changes

Infrared camera ideally Aligned dual color cameras for 3-D detection Statistical and fuzzy logic algorithms (referenced in paper) Motion modeling and flicker detection Smoke detection Perform processing on-board Optimized code and algorithms

Better, more complex algorithms

Improved processing speed

Conclusion

Project lays foundation for a great control system More work is definitely needed All of the code is shared on Github (Sanderi44/ARDrone-Fire-Detection) Would like to continue this project in the future If fire fighters had these drones helping them, it could save lives and prevent tragedies like Arizona

Questions ?

Sources

J. Marshall, and J. Billeaud. "19 Firefighters Killed in Arizona Forest Fire." Yahoo! News. The Associated Press, 01 July 2013. Web. 24 July 2013. <http://news.yahoo.com/19-firefighters-killed-arizona-forest-fire-165420995.html>. "Wildfires - Dry, Hot and Windy."Environment Facts, Environment Science, Global Warming, Natural Disasters, Ecosystems, Green Living - National Geographic. National Geographic Society, n.d. Web. 24 July 2013. <http://environment.nationalgeographic.com/environment/natural-disasters/wildfires/>. L. Epatko. "How Are Drones Used in the U.S.?" Web log post. PBS. PBS, 18 Apr. 2013. Web. 25 July 2013. Parrot SA. N.p.: Parrot SA., n.d. AR.Drone 2.0. Parrot New Wi-fi Quadricopter- Specifications. Parrot SA, 2012. Web. 25 July 2013.

ARDrone API. Program documentation. ARDrone API. Vers. 2.0. Parrot DEVZONE, 2013. Web. 25 July 2013.
F. Geisendrfer, Node-ar-drone. Open-Source Computer software. GitHub. Vers. 2.0. Felix Geisendrfer, n.d. Web. 27 July 2013. <https://github.com/felixge/node-ardrone>. B. Venthur, Python-ardrone. Open-Source Computer software. GitHub. Vers. 1.0. Bastian Venthur, n.d. Web. 27 July 2013. <https://github.com/venthur/python-ardrone>. OpenCV | OpenCV. Open-Source Computer software. OpenCV | OpenCV. Vers. 2.4.6. Itseez, 2013. Web. 27 July 2013. <http://opencv.org/>. SimpleCV. Open-Source Computer software. SimpleCV. Sight Machine Inc., 2013. Web. 27 July 2013. <http://simplecv.org/>. FFMPEG. Open-Source Computer software. FFmpeg. Vers. 2.0. N.p., n.d. Web. 27 July 2013. <http://www.ffmpeg.org/>. T.H. Chen; P.H. Wu; Yung-Chuen Chiou, "An early fire-detection method based on image processing," Image Processing, 2004. ICIP '04. 2004 International Conference on , vol.3, no., pp.1707,1710 Vol. 3, 24-27 Oct. 2004 Toreyin, B.U.; Cetin, A.E., "Online Detection of Fire in Video," Computer Vision and Pattern Recognition, 2007. CVPR '07. IEEE Conference on , vol., no., pp.1,5, 17-22 June 2007 doi: 10.1109/CVPR.2007.383442 T. Celik, Kai-Kuang Ma, "Computer vision based fire detection in color images," Soft Computing in Industrial Applications, 2008. SMCia '08. IEEE Conference on , vol., no., pp.258,263, 25-27 June 2008 T. Celik, "Fast and Efficient Method for Fire Detection Using Image Processing," ETRI Journal, vol. 32, no. 6, Dec. 2010, pp. 881-890. T. Celik, Demirel, H.; Ozkaramanli, H.; Uyguroglu, M., "Fire Detection in Video Sequences Using Statistical Color Model," Acoustics, Speech and Signal Processing, 2006. ICASSP 2006 Proceedings. 2006 IEEE International Conference on , vol.2, no., pp.II,II, 14-19 May 2006 B.H. Cho; J.W. Bae; S.H. Jung, "Image Processing-Based Fire Detection System Using Statistic Color Model," Advanced Language Processing and Web Information Technology, 2008. ALPIT '08. International Conference on , vol., no., pp.245,250, 23-25 July 2008 S.Y. Foo, "A fuzzy logic approach to fire detection in aircraft dry bays and engine compartments," Industrial Electronics, IEEE Transactions on , vol.47, no.5, pp.1161,1171, Oct 2000 V. Vipin, "Image Processing Based Forest Fire Detection." International Journal of Emerging Technology and Advanced Engineering 2.2 (2012): 87-95.International Journal of Emerging Technology and Advanced Engineering. International Journal of Emerging Technology and Advanced Engineering. Web. 27 July 2013. J. De la Riva, F. Prez-Cabello& E. Chuvieco, (Eds) 2005 Proceedings of the 5th International Workshop on Remote Sensing and GIS Applications to Forest Fire Management: Fire Effects Assessment: 9-17 Universidad de Zaragoza. ISBN: 84-96214-52-4 L.Merino; Caballero, F.; Dios, J.R.M.-d.; Ollero, A., "Cooperative Fire Detection using Unmanned Aerial Vehicles," Robotics and Automation, 2005. ICRA 2005. Proceedings of the 2005 IEEE International Conference on , vol., no., pp.1884,1889, 18-22 April 2005

Image Sources

http://www.slashgear.com/parrot-ar-drone-wifi-helicoptergets-augmented-reality-iphone-control-0567744/ http://forums.nasioc.com/forums/showthread.php?p=387 47159 http://www.ljmu.ac.uk/NewsUpdate/viewarticle/500/ http://news.nationalgeographic.com/news/2013/13/13072 5-drone-uav-uas-disposable-wildfire-storm-chasers/ http://www.geekosystem.com/mecam-personalquadcopter/ http://www.maniacstore.com/Parrot-AR-Drone-2.0-PriceIn-India.html T. Celik, Demirel, H.; Ozkaramanli, H.; Uyguroglu, M., "Fire Detection in Video Sequences Using Statistical Color Model," Acoustics, Speech and Signal Processing, 2006. ICASSP 2006 Proceedings. 2006 IEEE International Conference on , vol.2, no., pp.II,II, 14-19 May 2006