Aerial Surveillance System using UAV Zainab Zaheer1, Atiya Usmani1, Ekram Khan2, Mohammed. A. Qadeer1 1 Department of Computer Engineering, 2 Department of Electronics Engineering, Zakir Husain College of Engineering and Technology Aligarh Muslim University, India {atiya.usmani, zainabz1995, ekhan67, maqadeer}@gmail.com Abstract: — In today’s world, there is a growing need for surveillance in order to maintain the decorum at a place and ensure the safety and security of its people. An aerial surveillance system will be worthwhile in this regard. This paper describes how an aerial surveillance system can be built using an unmanned aerial vehicle or a drone. We start by delineating the features of our aerial surveillance system and then discuss some of the technologies that we have used in building it. After that we mention how we have incorporated those technologies into a drone and have made them work together harmoniously in order to achieve our desired aerial surveillance system. This system will be a convenient and efficient alternative to current surveillance systems. It can be used in peace keeping activities and also real time monitoring of a place at any time of the day. The aim is to provide fast and efficient surveillance at an affordable rate so that it can be used widely at private, institutional and governmental level. Keywords—surveillance; drone; quadcopter; UAV; Waypoints; live streaming;

I. INTRODUCTION Drones can be used for surveillance for both civil and military purposes. Police officers often have to patrol within a city to ensure that law and order is maintained and hence assuring the safety of the citizens. Natural disasters are increasing at an alarming rate in the world. Every now and then we hear in the news about an area affected by earthquake, flood or a hurricane etc. There is a need to examine such a disaster stricken area before undertaking necessary rescue and help measures. Military officers often have to patrol dangerous areas in order to search for any potential threat, illegal activity or intrusion within the borders of a country that can put the lives of citizens in jeopardy. Such areas involve very high risk to human life. He has to overcome fatal natural obstacles like steep mountain slopes, forceful water currents, hostile and barren desert areas and other such areas. The Aerial Surveillance System can be used easily to get this job done without any loss of human life. Also, the speed of operation will be faster with the drones as it can continue to fly for days and days on end, unhindered. Thus drones are an excellent method for doing aerial policing An Aerial Surveillance System is basically a flying machine that can be controlled remotely with capabilities to transmit real time data to a control room.

978-1-4673-8975-4/16/$31.00 ©2016 IEEE

A. Plan We have outlined a plan for manufacturing our own Aerial Surveillance System from a drone. The proposed plan states that the project will be built in different phases. Each phase is described below: A.1 Phase1: Testing In this phase, we acquire a simple drone and test its capabilities like: • Automatic take-off and landing, custom mission planning with GPS waypoint navigation. This is a necessity for sending the drone on automatic missions and also tracking its location through GPS. • We also want it to have the capability of returning to its launch point or a predefined location so that in case of loss of GPS signal or low battery, the drone would return to a predefined location and land there automatically. • The machine should also be able to hover in air at one place so that on identifying any illegal activity we can stop the drone in its tracks and make it loiter at its current location. Thus we will be able to focus on the region of interest. In this phase we will be testing this drone thoroughly to ensure that it has desired capabilities and is fit for surveillance. The drone will be tested in both manual and automatic mode in order to ensure that it will serve as a good, solid foundation for our project and we can further add new features and functions for building our desired flying machine. A.2 Phase2: Integration In this phase we will add new features to the drone. This phase is further subdivided into three sub-phases: 1. Eyes in the Sky: The main aim in this phase is to mount a camera on our machine to get real time data on the ground. This data can be audio, video or a location. 2. We have some performance constraints like the camera must render good quality images. It should provide additional options like zoom in and zoom out. It has to be night vision compatible as most of the illegal work is done in the darkness of the night. Also, the camera should be cheap in order to make our whole machine more economical and affordable.

3. Wings of Falcon: In this phase we plan to extend the outreach of our drone by coupling the drone’s camera to an LCD TV at the control room. The data acquired through the camera is analysed by an individual to spot any illegal activity. 4. Aerial Policing: Here we will mount an alarm system and enable it to play pre-recorded warning messages. These messages will be stored in a memory module and will be played if any illegal activity is spotted anywhere. II. BACKGROUND An unmanned aerial vehicle (UAV) is an airborne vehicle which is controlled either manually or automatically from a ground control unit [1]. Or it can be defined as, “a device that is used for flight in the air without an onboard pilot.” A. Past and Present of UAVs In 1916 Americans Lawrence and Elmer Sperry created the first UAV. It was the first aircraft with automatic steering. Then a UAV named “Aerial Torpodo” was designed for military purposes. Its job was to carry explosive payload. After that UAVs were used in the Vietnam War and Cold War. The U.S. had put them to practical use in the Persian Gulf War in 1991. After that a lot of progress was made in development of military UAVs [2]. Then under the 9 year- ERAST project (Environmental Research Aircraft and Sensor Technology) UAVs were used for environment observation and measurement like surveys of ozone layer, air pollution, coastlines, wildfires, vegetation growth etc. Helios, Proteus, Altus, Pathfinder are some of the aircrafts that came out from this project [2].

For building our Aerial Surveillance System we are using a Do-It-Yourself drone kit, namely IRIS+. The reason being, that it is an extremely difficult cumbersome task to build our own flying machine while a significant number of our requirements can be easily met using ready to fly drones. We can also incorporate and attach new devices in these DIY drone kits in order to satisfy our remaining functional requirements. It is important to note here that the IRIS+ is a very simple drone which does not come with a pre-installed camera; hence it cannot be used for surveillance directly. New features that are required for surveillance are added to it by using other devices and technologies. B. Technologies Used B.1 Quadcopter This system has autopilot Pixhawk v2.4.5, Firmware ArduCopter 3.2 and GPS of 3DRuBlox GPS with Compass (LEA-6H module, 5Hz update) in itself. It has V frame type with motors of 920kV and a set of 9.5x4.5 T-Motor multirotor self-tightening propellers (2 of clockwise direction and 2 of counterclockwise direction) [9]. Its payload capacity is of 400g (i.e. about 0.8lbs). Its flight time ranges from 16-22 minutes depending on payload. It has 4 arms: two blue arms in the front and two black arms in the rear. [9]

In recent years, many new UAVs were developed as mentioned in the “2005 Aerospace Source Book”. Some of them are: i) Aerosonade: It conducts weather surveys and environmental monitoring. ii) Altus: It carries atmospheric sampling instruments for scientific research [2]. The potential of UAVs is infinite. A lot of research is being done in order to extract maximum benefits from them. UAVs today are mainly being used for agricultural chemical spraying in Japan and China, environmental monitoring, surveillance, and military purposes. At present, there are many companies who manufacture drones, which are purchased by hobbyists. They provide various models of drones suitable for respective purposes .For example Titan Aerospace provides Solara50 (a high-altitude Wi-Fi Drone), AeroVironment’s model is called Hummingbird Nano (a hummingbird shaped drone for urban surveillance), Prox Dynamic provides Black Hornet Nano (a palm-sized silent drone), SenseFly’s eBee, Game of Drones manufactured by Ballistic UAV (it is the world’s toughest drone) and 3DR drones like Solo, Iris+, X8+, Aero [8]. These are DIY or Do-ItYourself kits and are easily available.

Fig.1. Front View of IRIS+ [9]

B.2 Pixhawk We are using Pixhawk Autopilot. It consists of various sensors and modules like the Air Pressure Module, GPS Module and memory Module. The sensors include gyro, accelerometer/ magnetometer and barometer. It consists of three microprocessors and can also be connected to other microprocessors like Arduino or Raspberry Pi [10]. A detailed diagram of Pixhawk is shown below:

Fig.2. Pixhawk [10]

B.3 Lipo Battery The IRIS plus uses lithium polymer batteries for better flying. The specification of a compatible battery with IRIS is 3S 5.1Ah 8C lithium polymer [9] because a 4S battery can cause permanent damage to the gimbal electronics. Therefore, its battery cell limit is 3S. The battery has to be monitored while charging and is always charged inside a guard bag to protect it from extreme heat, cold, puncturing and flammable surfaces. B.4 Radio Controller The radio controller is used to maneuver IRIS in flight. It has an antenna to establish communication link between itself and the quadcopter and has switches to control flight modes and directions. It also shows flight data on its LCD screen. This controller operates on a voltage range of 10.5V to 12.6V [9].

Fig.3. Radio Controller [9]

B.5 915 MHz Telemetery Radio Transmitter A Telemetry radio (3DR Radio Telemetry v2) of frequency 915 MHz [9] is used for establishing communication between quadcopter and ground station while planning an automatic mission. The range of this radio is up to 1km (0.6 miles) [9]. B.6 Raspberry Pi B+ Microcontroller with wireless adapter Raspberry Pi is a cheap, single board, credit card- sized computer. SD cards are used to store its operating system like Debian and Arch Linux ARM [11]. It has a Broadcom system on chip which consists of ARM compatible CPU, an on chip GPU, 1 to 4 USB slots, HDMI and composite video output and a 3.5 mm phono jack for audio. The speed of its CPU ranges from 700MHz to 1.2GHz [12]. It also has an on board memory ranging from 256MB to 1GB RAM [13]. GPIO pins are also available to provide lower level output which support common protocols like I2C [13]. It promotes python as the main programming language. B.7 Ground Station A ground station is a software application that is needed to plan an automatic mission. It is a platform to build and monitor a flight. It runs on a device that is at the ground and enables wireless communication of the UAV with this the device. We can also see the vehicle’s performance parameters and its current position. It can be used to set parameters like enabling and disabling the safety button etc. A GCS can also

control a UAV in flight [14]. Overall, it acts as a virtual cockpit. There is a wide variety of open source ground stations available for different platforms like UNIX, Windows, Android and iOS. Many of them are open source. Some ground stations are listed below [14]: • Mission Planner (Windows, Mac OS X, Linux) • APM Planner 2 (Linux) • Droid Planner 3 (Android) • MAVProxy (Linux) • Fighting Walrus iDrone Ctrl(iOS) III. DESIGN AND INTEGRATION OF AERIAL SURVEILLANCE SYSTEM A. System Overview Our Aerial Surveillance System consists of a flying machine or a drone as the primary component. This flying machine is capable of autonomous flight and supports custom mission planning through GPS waypoint navigation. The mission is built on our laptop using previously available ground stations like Mission Planner and APM Planner. After building the mission it is uploaded on the drone. To do this a 915MHz transmitter is used for establishing connection between the flying machine and our laptop. Once, our flying machine is in flight we can see all the flight data like altitude and the drone’s current location on our laptop. The laptop is acting as a ground control station to monitor the mission. A USB camera interfaced with Raspberry Pi is also attached to the flying machine. A wireless adapter is also attached to it. The Raspberry Pi is connected to a Wi-Fi Hotspot which is on the same network as our laptop. We are connecting to the Raspberry Pi by establishing an SSH connection so that we can give commands to it and also access its GUI from our laptop. The camera on the flying machine which is also attached to the Raspberry Pi is live streaming the video data to our laptop through this path. This video data is further processed. We can also make the Raspberry Pi play stored warning messages and alarms using our laptop. Thus we have a complete Aerial Surveillance System. (Fig.8.)

Fig. 4 System Overview

B. Testing the Flying Machine We assembled the drone-kit and began by testing the drone. The drone was tested both in manual and auto mode. The results are given in the subsequent sections. B.1 Manual Mode The results of testing the drone in manual mode are given in the table 1.1. There are two manual modes: loiter and standard. As beginners we first started testing in loiter mode and then moved on to the standard mode of flying. After three failed attempts for various reasons we had the first successful flight. The table 1 lists the outcome of these attempts i.e., whether success or failure, reason of failure, inference from the outcome and the amendments that we had to make for a successful flight.

STANDA RD MODE

1st

FAIL

2nd

Succes s Fig2

Obstac le inside geofence edge

Geo fence: 100m altitude and 300m radius from launch point

__

Successfully controlled the drone

Honed our operating skills

__

We also tested the drone’s ability to return to its launch point and land. This feature was necessary to test so that in case of a failure such as low battery or GPS signal we can make the drone land at its launch point or a predefined location, if desired. The second mode in which we flew was the standard mode. In this mode we controlled the drone with the Radio Controller. Maneuvering the drone was very easy, we accomplished many successful flights. In table 1 we list our attempts only till the first successful flight and the pictures of our ‘drone in flight’ in both of these modes are also shown

Fig.5. First flight in Loiter Mode (in front of AMURoboClub, Mechanical Engg. Dept., Aligarh Muslim University)

TABLE I. EXPERIMENT

LOITER MODE

1st

RESU LT

REAS ON

INFERENCE

AMENDME NT

FAIL

Low battery signal

Inbuilt mechanism to troubleshoot Battery charge= 25%, IRIS returns to its launch point and lands.

Battery charged to 100%

Fig.6. First flight in Standard Mode (in front of AMURoboClub, Mechanical Engg. Dept., Aligarh Muslim University)

B.2 Auto Mode In this mode, the quadcopter was made to fly through a series of customized waypoints. These waypoints were designed at a ground station and then sent to the drone via a connection between the two. We experimented with various ground stations as discussed below.

2nd

FAIL

Low GPS signal

GPS lock required

Tried in open ground with clear sky

3rd

FAIL

Calibra tion error

Need Calibration

Hold down F/S button

RC Calibra tion

Motors armed

__

Calibration completed

__

4th

Succes s Fig1

__

Flight Successful

__

• Direct USB connection

RTL & Landin g

Succes s

__

Successful landing at launch point

__

• TCP connection

We started with Droid Planner 3, an android based ground station that allows us to plan and build missions. We a built a simple mission successfully. Then the next step was to connect the application with the quadcopter to upload this mission on it. There were five kinds of connection possible: • USB connection through telemetry radio

• UDP connection • Bluetooth connection We tried all of them. After a few failed attempts due to incompatible devices that did not support features like Mobile High Definition Link (MHL) and USB on-The- Go, and various other reasons we finally connected to the drone using

a 915MHz transmitter. The mission was then successfully uploaded on the drone and we had our first autonomous flight. Then many other missions were built and executed. Some of them are shown (Fig. 11 and 12). We also executed some problems like abrupt behavior of the quadcopter or landing a little ahead than the actual land waypoint. But, overall it worked well.

to it and powered it on. The Raspberry Pi was powered through the drone’s battery. After that we established an SSH connection between the Raspberry Pi and our laptop using Putty so that we could access its GUI and command line from our computer [17]. Next we built a Raspberry Pi webcam server. We first downloaded and installed Motion. Motion is a software package that examines video signal from the camera and detects if any change has occurred in the picture or a motion has taken place [18]. Then we edited its configuration file [19]. We made the following modifications: • We switched on Daemon, a computer program that runs in the background rather than being controlled by the user directly. This program performs stated operations at particular intervals or in reaction to certain events. This is done so that Raspberry Pi camera continues capturing video data in the background and streaming it to an LCD monitor.

Fig.7.Autonomous Missions.

In order to build missions on our laptops wse then moved on to Windows based ground Station called the Mission Planner [15] and also experimented with APM Planner [16] which works on Linux, Windows and MAC OS. They work for different kinds of UAVs like plane [17], copter [18] and rover [19] and also support simulation software like SITL [16]. Fig.4 shows a mission built on Mission Planner.

• We switched off webcam localhost (Raspberry Pi) to access the Raspberry Pi webcam from another computer. • Next we had set the value of webcam_maxrate to 100. This enables real time streaming but requires more bandwidth. • After that we had set the framerate to 100 so that the webcam captures 100 frames per second for an unruffled video stream. • At last we had set the desired width and height of the display.

Fig.8. Mission planned in Mission Planner

After thoroughly testing the drone we concluded that it would serve as a good foundation for our aerial surveillance system. It had the desired capabilities of handling real life situations such as insufficient battery, low GPS signal, loss of communication with the ground control station etc. The automatic mission planning using waypoints was also quite convenient and precise. It supported features like return to launch or landing at a predefined location. So, we could now confidently state that we had found a solid base for our project on which new features will be added to make our desired flying machine. On completion of the testing phase, we began the integration phase which is discussed in the subsequent sections. C. Interfacing Camera with Raspberry Pi and Live Streaming Video Data to an LCD Monitor We began by setting up the Raspberry Pi. We loaded Raspbian, its operating system on it using 8GB class 4 SD card. Then we attached a USB webcam and a wireless adapter

We then switched on the daemon in the motion file for the same reasons as stated above. Finally, we gave the command to start the motion service and checked the video stream on our remote desktop by accessing port 8081 at the IP address of Raspberry Pi. This completed our second and third sub-phases as given in the ‘Plan’ at the beginning of this paper. A live streaming example is shown below. We can see the video at the IP address of the Raspberry Pi.

Fig.9. A live- streaming example

D. Enabling Alarm System The next sub-phase was to enable the Aerial Surveillance System to play pre-recorded warning messages on detection of any illegal activity. The Raspberry Pi consists of phono jack

for audio. We can easily play audio messages using it. For this, we first checked whether our Raspberry Pi has snd_bcm2835 module (hardware abstraction layer) installed for the audio I/O [20]. By default, the O/P is set to HDMI audio interface if available, otherwise analog. We forced by the value of “n” in “amixer cset numid=3 n” command to 1. Here, n can be 0 (auto), 1 (analog), 2 (HDMI). Then, we installed mpg321 (or omxplayer) package to play mp3 file and executed it through command line [20]. The audio messaged already stored in the Raspberry Pi & are then played when desired. This completes final sub-phase of our project. E. Assembling the Aerial Surveillance System Finally, we attached together all the components of our Aerial Surveillance System, namely the drone, Raspberry Pi and camera. The Raspberry Pi was firmly fixed on the backside of the drone and the camera was tightly fastened so that it remains stationary. The Raspberry Pi was powered through the drone’s battery. For this, we created our own UBEC (universal battery elimination circuit) using LM-7085 (Fig.13).This circuit converted the 12.6V that we were getting from the drones battery to Raspberry Pi’s operating voltage i.e., 5.0 V The final view of our aerial surveillance system is shown in Fig 15.

features but our Aerial Surveillance System is relatively cheap. There is also a provision to incorporate these features in the IRIS+ using gimbal & FPV kit but they are still costlier than our Aerial Surveillance System design. In table 2, we present a cost analysis and comparison of our Aerial Surveillance System with already available surveillance drones and those which were installed using gimbal, FPV & IRIS+ altogether. TABLE II. DRONE

PRICE

SOLO w/ Pre-Installed Gimbal in 3DR Backpack

$1149.95

IRIS+ assembled

Our proposed Aerial Surveillan ce System

[21] IRIS+

$599.99

[22] Tarot T-2D Gimbal

$210.00

[23]GoPro Hero 3

$364.99

[21] IRIS+

$599.99

[24] Raspberry Pi 2

$35.00

[25] USB webcam

$21.80

$1174.98

$657.00 (approx.)

V. CONCLUSION

Fig. 10. UBEC circuit created using LM-7085

Our project can be used for surveillance in civil and military areas. It can be used for monitoring of a campus, office and industrial areas by various institutions, for monitoring the borders and in peace keeping activities by the government and can also be used to monitor private properties by individuals. Using them for surveillance will ensure the safety and security of not only the citizens but the soldiers as well. The aerial surveillance system has many advantages over the existing methods of surveillance in our country. Some of them are listed below: • First of all they will be faster as the drones can continue to fly at a high speed unhindered while a human being, even in a high speed aero plane needs rest.

Fig. 11. Final View of our Aerial Surveillance System

IV. COMPARISON WITH EXISTING WORK For creating our Aerial Surveillance System, we have used IRIS+ drone kit. IRIS+ is a very simple drone that does not come with a camera. We have incorporated a camera and also enabled live streaming of video data to a control room. We have also enabled our Aerial Surveillance System to play prerecorded messages. All these things have been assembled by us. There are already available drones that have all these

• Secondly it will be cheaper as there is no need to deploy and pay for human resource for this task. Further our Aerial Surveillance System is cheaper than the existing drones too. • Thirdly this method will be much more efficient and less erroneous because human beings are more prone to committing errors than machines and in this system most of the human errors will be avoided • It is much more convenient because many of the natural obstacles that are faced while sending soldiers to hostile terrains in order to check for potential threats

will be easily overcome. Thus the life of both, the soldier and the citizen is protected. Another use of an aerial surveillance system is to monitor disaster-stricken areas before taking necessary rescue and help measures. VI. FUTURE WORK Our project can be further expanded. We can incorporate collision avoidance in our Aerial Surveillance System. This can be done by using Infrared sensors to detect obstacles. Thus our Aerial Surveillance System will be able to make its path through the hindrances and reach the destination quickly. This project can also be used for other services rather than surveillance. For example, it can be used to carry light weight objects from one place to another thus ensuring faster delivery. We can send first aid to victims with the help of drones. As a result, the first aid will arrive flying to the victim in no time, while the ambulance is on its way thus saving every minute that is crucial for the victim’s life. The doctor will be able to see the victim through our video and also give instructions through the alarm or audio system that plays messages. We believe our project has a lot of potential in benefitting humanity. ACKNOWLEDGMENT We are indebted to Mr. Sarfaraz Alam Khan for gifting the 3DR IRIS+ drone-kit to AMU Roboclub, which we have used in our project. We thank him profusely and appreciate his generosity and his passion for promoting the innovative spirit amongst the students of AMU. REFERENCES [1] [2]

Malatone Bouasym, “USCG Search And Rescue Drone”. Engineering Technology Capstone Project. Spring 2015 Kenzo NANOMI, “Prospect and Recent Research & Development for Civil Use Autonomous Unmanned Aircraft as UAV & MAV”. Journal of System Design and Dynamics, Vol.1, No. 2, 2007

[3]

[4]

[5] [6]

[7]

[8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25]

Henri Eisenbeiss, “A Mini Unmanned Aerial Vehicle (Uav): System Overview And Image Acquisition”, International Workshop on "Processing And Visualization Using High-Resolution Imagery",18-20 November 2004, Pitsanulok, Thailand Peter M. Asaro, “The labor of surveillance and bureaucratized:killing: new subjectivities of military drone operators”, Social Semiotics, 2013 Vol. 23, No. 2, 196
224, http://dx.doi.org/10.1080/10350330.2013.777591 Anuj Puri, “A Survey of Unmanned Aerial Vehicles (UAV) for Traffic Surveillance” Rachel L. Finn, David Wright, “Unmanned aircraft systems: Surveillance, ethics and privacy in civil applications” , comp u t e r law & s e c u rity rev iew 2 8 ( 2 0 1 2 ) 1 8 4 e1 9 4 Richard M. Thompson II,” Drones in Domestic Surveillance Operations: Fourth Amendment Implications and Legislative Responses,” Congressional Research Service 7-5700 www.crs.gov R42701, April 3, 2013 http://www.inc.com/murray-newlands/6-drone-companies-to-look-outfor.html https://3drobotics.com/wp-content/uploads/2015/02/IRIS-PlusOperation- Manual-vH-web.pdf https://pixhawk.org/_media/modules/px4fmu-manual-v1.6.pdf https://www.raspberrypi.org/help/what-is-a-raspberry-pi/ https://en.wikipedia.org/wiki/Raspberry_Pi https://www.adafruit.com/datasheets/pi-specs.pdf http://ardupilot.org/copter/docs/common-choosing-a-groundstation.html#comparison http://ardupilot.org/ardupilot/index.htm#l http://python.dronekit.io/develop/sitl_setup.html http://www.circuitbasics.com/access-raspberry-pi-desktop-remoteconnection/ http://www.lavrsen.dk/foswiki/bin/view/Motion/WebHome http://pimylifeup.com/raspberry-pi-webcam-server/ http://www.raspberrypi-spy.co.uk/2013/06/raspberry-pi-command-lineaudio/ https://store.3dr.com/products/iris https://store.3dr.com/products/tarot-t-2d-brushless-gimbal-kit http://www.amazon.com/GoPro-CHDHX-301-HERO3-BlackEdition/dp/B009TCD8V8 http://www.amazon.com/Raspberry-Pi-Model-ProjectBoard/dp/B00T2U7R7I http://www.amazon.com/Digital-Additions-High-SpeedWebcam/dp/B005BPM214