Joint Localization and Activity Recognition from Ambient FM Broadcast Signals

Session: CoSDEO 2013: Device-free Radio-based Recognition UbiComp’13, September 8–12, 2013, Zurich, Switzerland Joint Localization and Activity Reco...
Author: Buddy Hoover
1 downloads 0 Views 4MB Size
Session: CoSDEO 2013: Device-free Radio-based Recognition

UbiComp’13, September 8–12, 2013, Zurich, Switzerland

Joint Localization and Activity Recognition from Ambient FM Broadcast Signals Shuyu Shi National Institute of Informatics, Japan 2-1-2 Hitotsubashi Chiyoda-ku, Tokyo 101-8430 Japan [email protected]

Stephan Sigg National Institute of Informatics, Japan 2-1-2 Hitotsubashi Chiyoda-ku, Tokyo 101-8430 Japan [email protected]

Yusheng Ji National Institute of Informatics, Japan 2-1-2 Hitotsubashi Chiyoda-ku, Tokyo 101-8430 Japan [email protected]

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. UbiComp’13 Adjunct, September 8–12, 2013, Zurich, Switzerland. c 2013 ACM 978-1-4503-2215-7/13/09...$15.00. Copyright

Abstract Due to spatial diversity, RF signals derived from a FM broadcast station differ when they arrive at the receivers placed in various locations. Also, the FM signals will be altered by the change of ambient environment. Previous works focuse either the FM-based localization or activity recognition. In this study, we propose to simultaneously classify and localize activities conducted in proximity of an FM receiver. We conducted experiments and demonstrated that the location and activities of an individual can be distinguishable with a reasonable overall accuracy in a typical indoor environment from FM broadcast signals.

Author Keywords Device-free localization, Device-free activity recognition, RF-sensing, Indoor environment

ACM Classification Keywords J.9.d [Pervasive computing]: .; H.5.5.c [Signal analysis, synthesis, and processing]: .; I.5.4.m [Signal processing]: .; J.9.a [Location-dependent and sensitive]: .

http://dx.doi.org/10.1145/2494091.2497610

521

Session: CoSDEO 2013: Device-free Radio-based Recognition

Introduction Indoor localization has the potential to realise many positioning related applications inside buildings as the GPS system has done outdoors. Due to the unavailability of GPS signals in indoor environments, many works have been conducted for indoor localization leveraging fingerprinting-based techniques, among which, a broad number of implementations require prior installation of RF-beacons [9, 20, 3]. In contrast, approaches without any hardware deployment and leveraging already available wireless signals draw increasing attention. Such approaches may profile a location based on RSSI values. For instance, in intelligent environments, attention monitoring and surveillance in emergency situations, knowledge about activities as well as their location is appreciated. Nearly all contemporary electronic devices contain some kind of interface to the RF-channel. Therefore, cases in which no RF-sensor is in a person’s proximity (e.g. in the same room) are rare [14, 11, 5]. When people conduct activities, they induce a characteristic pattern on the received RF signals of ambient wireless receivers [18]. Therefore, some research groups have proposed to sense activities of individuals in proximity of a device which are not equipped by any part of the recognition system. We use the RF-channel as a sensing source to jointly detect locations and activities of individuals by monitoring peculiarities of RF-based features induced by these very activities. It is well established that the wireless channel can be utilised for motion detection [8, 24] and localization [15, 23]. Activities have been sensed from FM radio signals fluctuation in [17]. We will demonstrate in this work the

UbiComp’13, September 8–12, 2013, Zurich, Switzerland

simultaneous localization and activity recognition leveraging an ambient FM broadcasting station as the signals source. The contributions are 1. we address the feasibility and suitability of FM signals for sensing situations and localization. 2. we propose a framework for joint localization and activity recognition, mainly consisting of three modules, data acquisition, characteristic feature extraction and instance classification. 3. a case study utilizing the proposed approaches which succeed in discriminating three typical activities, lying, standing and walking in two locations.

Related Work RF-based sensing approaches are widely investigated recently. In this section, we overview state of the art on localization and recognition of situation or activities leveraging the RF information. RF-based localization First systems utilizing RF sensing technology aim at localization of individuals equipped with a receiver (device-bound). Due to multipath propagation and spatial diversity of RF waves, the received signals differ for the various locations. As the user is always carrying the receivers, the location can be distinguished by analysing the received signal information. The seminal work utilizing information of RF signals for locating user was conducted by Bahl et al [2]. They introduced the RADAR system which utilised signals from WiFi access points for the localization in indoor settings. Other researchers also conducted experiments showing the

522

Session: CoSDEO 2013: Device-free Radio-based Recognition

feasibility of using signals on other spectrum bands, for instance, derived from GSM cellular base station [10] or FM broadcast station [4].

4 0

1

2

3

time step (minute)

receiver 1 receiver 2

5

mean amplitude

Figure 1: Average signal strength evolution for 5 minutes with a granularity of one second originating from an ambient FM broadcasting station sampled by receiver1 and receiver2 respectively in an empty corridor sketched in figure 3

Localization can, however, also be achieved by device-free RF sensing systems that don’t require that a subject is equipped with a transmit or receive device. Youssef et al. consider the localization of individuals by 802.11b nodes, located in the corners of a room, continuously transmitting packets [23]. Using 3 access points and 2 monitoring points with a data collection at 5Hz they achieved a 1.82 meter median distance error. The authors validate their system in further experimental studies in different indoor settings in [7]. They reported that the standard deviation of the RSSI turned out to be more stable to changes in the environment but more sensitive to movement when using a classifier trained on data from previous experiments [6]. Wilson and Patwari utilised radio tomographic imaging (RTI) on the two-way RSSI variance [21] or RSSI mean fluctuations [22] between nodes arranged in a rectangle surrounding the monitored area for robust localization of up to two individuals simultaneously. They further introduced a statistical, empirically verified model to approximate the position of a person based on RSSI variance [12]. In this model the motion of an object in the monitored area causes a certain quantity of multipath power to be affected. RF-based activity recognition Beyond localization, other authors have also explored the recognition of situations or activities leveraging the RF signals. In this direction, first of all, Woyach et al investigated the impact of a static environmental change on the RF signals. Further, they showed that

UbiComp’13, September 8–12, 2013, Zurich, Switzerland

the velocity of an entity can be estimated by analysing the RSSI pattern of continuously transmitted packets of a moving node. Muthukrishnan and others advanced these studies by distinguishing between moving or stationary nodes via the analysis of the fluctuation of the RSSI indicator in a network of wireless nodes with an accuracy of 0.94 [8]. Anderson et al. and Sohn et al. distinguish also between up to six velocity levels with an accuracy of about 0.84 [1, 19]. Also, some authors have further studied the feasibility of activity recognition of individuals without RF sensing equipment, simply by characterising information of signals variety resulting from distinct activities [14]. The authors describe a system to detect walking, talking on a mobile phone and the state of the door in a typical office room with two software defined radio nodes (transmitter and receiver) at 900MHz placed on both sides of the door to a typical office room. Walking was detected by the number of peak-to-peak amplitudes greater than a trained threshold. The door context was triggered by a static change of amplitude in the signal. In order to detect a phone call a predefined area of the frequency spectrum was searched for a significant signal peak. The reported accuracy of the algorithm was on average above 80% for walking and above 90% for the other contexts. False positives were mostly encountered when the individual passed through the door, incorrectly triggering the door-context. Sigg et. al considered the detection of the activities ’sitting’, ’walking’ and ’standing’ in a similar setting based on the RMS, Signal-to-Noise Ratio (SNR) and AMS of the RSSI [13]. The authors installed two or three SDRs, from which one was used to transmit a continuous signal at 900MHz or 2.4GHz in a room. The reported accuracy for these experiments was above 60%.

523

lying at S2 25

receiver 1 receiver 2

mean amplitude

30

Session: CoSDEO 2013: Device-free Radio-based Recognition

walking at S2

10

walking at S1

15 20 time step (minute)

lying at S1

5

standing at S 2

0

standing at S1

Figure 2: Average signal strength evolution over 5 minutes with a granularity of one second for an individual lying, standing or walking separately at areas S1 and S2 where two receivers deployed to acquire these raw signals from an ambient FM broadcasting station sketched in figure 3

UbiComp’13, September 8–12, 2013, Zurich, Switzerland

Feasibility Study

Joint Localization and Activity Recognition

Radio waves are electromagnetic waves, defined by their amplitude, phase and frequency. Naturally, as signal propagation is roughly omnidirectional, the energy transmitted by a sender is propagated equally in all directions. Additionally, these signals from different directions may arrive at the receiver through different paths. Figure 1 illustrates the average signal strengths per second of two Universal Software Radio Peripheral (USRP) devices which are synchronised via the clocks of the connected PCs with Network Time Protocol (NTP). Due to their spatial separation, the signals experienced are uncorrelated.

Our system is designed not only to detect activities but also to recognise the area where a particular activity occurs. Our approach focuses on utilizing the FM signals from two synchronised USRP devices to detect activities in a particular position.

In the event that a radio wave encounters any concrete structure such as an object or individual, the main signal component will be damped (continue its path with reduced energy) or even completely blocked. These effects also impact the received signals of USRP devices. The impact of these effects is proportional to the distance between the blocking object and the receiver. On the other hand, when a dynamic activity is performed by an individual, the amplitude of the RF signal strength will greatly fluctuate in response. Figure 2 depicts the average amplitude of FM signals per second for an individual standing, lying or walking in areas S1 and S2 (cf. figure 3). We observe that whatever activities performed, the received signals differ due to the position deviation of receivers. Also, the characteristic pattern of signals will be altered by presence and movement of an individual. This observation suggests that with proper features and classification schemes it is possible to simultaneously distinguish the location and the activities of an individual considered from ambient signals.

Scenario Description Figure 3 illustrates the experiment conducted, where two synchronised receivers are placed along the wall of an corridor with a distance of 4 meters, ensuring the diversity of the received signals. Two 1m×2m square regions S1 , S2 are the detection areas for receiver 1 and receiver 2. In this scenario, we distinguish three activities, namely, ”standing”, ”lying” and ”walking”. All of them are conducted in both S1 and S2 . For distinguishing these situations, three modules covering data aggregation, feature extraction and instance classification are leveraged. Data aggregation We utilised two USRP N210 SDR devices integrated with the WBX1 daughter board and the VERT9002 antenna with 3dBi gain as receivers. While a certain activity is performed, two USRPs are simultaneously tuned to the channel at 82.5MHz to sample the amplitude measurements of signals with a rate of 64 Hz per second originating from an ambient FM broadcasting station as the raw data sets of this activity. Feature extraction In our scenario, three distinct activities were performed at two areas. For localization, we require signal 1 https://www.ettus.com/product/details/WBX 2 https://www.ettus.com/product/details/VERT900

524

Session: CoSDEO 2013: Device-free Radio-based Recognition

receiverUSRPyN2-p s82C5MHzI

S-

corridorysketch

-m

featureyinstances

y

Featurey extraction

rawydata

Door

tux@linuxb

PC

4m

S2

jointylocalizationyandyactivityyrecognitionyfromyambientyFMyradio

tux@linuxb

PC

receiverUSRPyN2-p s82C5MHzI

2m

Door

yInstancey classification

Samplingyofy FM#signals

localization S2 activityyrecognition

S-

walking standing lying

Figure 3: Sketch of the evaluation setting. The joint localization and activity recognition system extracted features combining the data acquired by both USRP devices.

UbiComp’13, September 8–12, 2013, Zurich, Switzerland

features that can characterise spatial diersity of subjects. Assuming the mean amplitude of signal measurements acquired without conducting activities by two receivers are both known as baseline values, B1 , B2 , the fluctuation ratio of signal values to B1 and B2 over a time window can reflect the different impact of an activity on signals received by different receivers. Mathematically, for a window size of m seconds from time t to time t+m, the fluctuation ratio can be expressed as Pm+t P64 Ri,t,m =

τ =t

j

(

di,64(τ −1)+j −Bi ) Bi

64m

, i ∈ 1, 2.

(1)

If an activity is performed at the area i, the fluctuation ratio of Ri will be larger than the other one. For activity recognition, the activities conducted in a same detected area, however, should block the received signals in different manners, therefore, features which can characterise the alteration of signals received by the same receiver induced by different activities. We have investigated that the features of average value and measurement variance over a time window can succeed in discriminating ”empty”, ”lying” and ”walking” [16], therefore, we still adopt the two features. Another parameter for extraction is the length of window size. For our scenario, there are no sufficient raw data measurements for accurately characterising the signals with a short interval. On the other hand, our system may be insusceptible to the sudden alteration of ambient environment in practice and lack of instances for training classification algorithms by choosing a large time window. To choose a suitable window size, we tested from 0.5s to 4s with an interval of 0.5s. Experimental results demonstrate an time window from

1s to 2s is reasonable for classification. Therefore, a window of m=2s was used for derivation of features. Instance classification As two USRP receivers are utilised for data aggregation and the features are extracted independently for each USRP, we merged the instances of features obtained from receiver1 and receiver2 respectively before classification. Our system utilises three classification algorithms, namely Naive Bayes, Decision Tree (DT) and a k-nearest-neighbour classifier (k-NN) in the implementation provided by the Orange data mining Toolkit3 . We utilised a 10-fold cross validation approach to generate 10 data-subsets and recursively recognised each subset via the classifiers trained by the other nine subsets for eliminating the impact of particular instances on the classification bias. For a set of k situations A = {a1 , . . . , ak } let I(ai ) be the total number of instances for situation ai and Icor (ai ) the number of correctly classified instances for this situation in which the classification matches the ground truth. We then define the accuracy by which a situation ai could be detected as Icor (ai ) ACC(ai ) = (2) I(ai )

Experimental Evaluation We conducted the experiments in a corridor with a width of 1.5m. To obtain sufficient footage, two subjects performed each of the three activities for 3 minutes at S1 and S2 respectively. Therefore, In total, there are six situations for discrimination. The two synchronised receivers sampled the FM radio signals with a rate of 64Hz. Before feature extraction, we omitted the first 3 http://orange.biolab.si/

525

Session: CoSDEO 2013: Device-free Radio-based Recognition

and last 10 percent of the raw data of every situation for the purpose of noisy data mitigation. The features derived from receiver1 and receiver2 are extracted with a time window of 2 seconds. By the processing of feature extraction, the classifiers can be utilised to distinguish these situations. We compared the recognition accuracy achieved by Naive Bayes, Decision Tree and k-Nearest-Neighbor classifiers in the confusion matrices shown in table 1(a), table 1(b) and table 1(c) respectively. The six states are detectable with an overall accuracy of 74.7%, 81.4% and 72.3% using Naive Bayes, kNN and Decision Tree classifiers.

Conclusion In this paper, we have discussed device-free joint localization and activity recognition from FM radio signals. In particular, we utilise simple amplitude-based

UbiComp’13, September 8–12, 2013, Zurich, Switzerland

features to distinguish lying, standing or walking of an individual in two locations. For suitable interpretation of the characteristic pattern of FM signals for classification, we utilised the average and variance of a signal and, as novel feature the alteration ratio of signal values between the case when a certain activity conducted in a location and the empty environment. Classification was achieved by a Naive Bayes, a Decision Tree and a k-Nearest-Neighbour classifier. Evaluation results show that an overall accuracy of more than 70% is achievable for every classifier. With still constantly increasing spread of mobile device usage and the price advantage of FM radio, we believe that RF sensing using an FM radio signal is a promising and powerful source of contextual awareness for Ubiquitous Computing applications.

526

Session: CoSDEO 2013: Device-free Radio-based Recognition

Ground truth

standing at S1 standing at S2 lying at S1 lying at S2 walking at S1 walking at S2

standing at S1 90.77 0 0 0 26.99 21.54

standing at S2 0 92.06 4.35 2.82 0 12.31

UbiComp’13, September 8–12, 2013, Zurich, Switzerland

Classification (%) lying at S1 lying at S2 1.54 0 0 0 79.71 15.94 2.82 76.06 0 0 0 3.07

walking at S1 1.54 1.59 0 11.27 63.49 16.92

walking at S2 6.15 6.35 0 7.04 9.52 46.2

(a) Classification accuracy achieved by a Naive Bayes classifier

Ground truth

standing at S1 standing at S2 lying at S1 lying at S2 walking at S1 walking at S2

standing at S1 96.92 0 0 0 22.22 15.39

standing at S2 0 96.83 8.70 2.82 0 0

Classification (%) lying at S1 lying at S2 0 0 0 0 76.81 14.49 11.27 80.28 0 0 0 0

walking at S1 walking at S2 3.08 0 3.17 5.63 0 5.63 0 71.43 6.35 18.46 66.15

(b) Classification accuracy achieved by a k-NN classifier

Ground truth

standing at S1 standing at S2 lying at S1 lying at S2 walking at S1 walking at S2

standing at S1 81.54 0 0 5.63 19.05 7.69

standing at S2 1.54 79.37 0 0 0 7.69

Classification (%) lying at S1 lying at S2 0 1.54 14.29 1.59 79.71 18.84 1.41 78.87 12.30 0 0 12.31

walking at S1 0 0 1.45 14.08 51.19 21.54

walking at S2 15.39 4.76 0 0 17.46 50.77

(c) Classification accuracy achieved by a DT classifier

Table 1: Accuracy of joint localization and activity recognition for the activities ”lying” ”standing”, ”walking”, ’walking’ and positions S1 and S2 in a corridor depicted in figure 3

527

Session: CoSDEO 2013: Device-free Radio-based Recognition

References [1] Anderson, I., and Muller, H. Context awareness via gsm signal strength fluctuation. In 4th international conference on pervasive computing, late breaking results (2006). [2] Bahl, P., and Padmanabhan, V. Radar: an in-building rf-based user location and tracking system. In Proceedings of the 19th IEEE International Conference on Computer Communications (Infocom) (2000). [3] Bruno, R., and Delmastro, F. Design and analysis of a bluetooth-based indoor localization system. 711–725. [4] Chen, L., Hoey, J., Nugent, C. D., Cook, D. J., and Yu, Z. Sensor-based activity recognition. IEEE Transaction on Systems, Man, and Cybernetics, Part C: Applications and Reviews PP, 99 (2012), 1 –19. [5] Dey, A. K., Wac, K., Ferreira, D., Tassini, K., Hong, J.-H., and Ramos, J. Getting closer: An empirical investigation of the proximity of user to their smart phones. In Proceedings of the 13th international conference on Ubiquitous computing (UbiComp 2011) (2011). [6] Kosba, A. E., Saeed, A., and Youssef, M. Rasid: A robust wlan device-free passive motion detection system. CoRR abs/1105.6084 (2011). [7] Kosba, A. E., Saeed, A., and Youssef, M. Rasid: A robust wlan device-free passive motion detection system. In Proceedings of the 10th IEEE International Conference on Pervasive Computing and Communications (PerCom2012) (2012). [8] Muthukrishnan, K., Lijding, M., Meratnia, N., and Havinga, P. Sensing motion using spectral and spatial analysis of wlan rssi. In Proceedings of

UbiComp’13, September 8–12, 2013, Zurich, Switzerland

Smart Sensing and Context (2007). [9] Ni, L. M., Liu, Y., Lau, Y. C., and Patil, A. P. Landmarc: indoor location sensing using active rfid. Wirel. Netw. 10, 6 (Nov. 2004), 701–710. [10] Otsason, V., Varshavsky, A., LaMarca, A., and de Lara, E. Accurate gsm indoor localisation. In Proceedings of the 7th ACM International Conference on Ubiquitous Computing (Ubicomp 2005) (2005). [11] Patel, S. N., Kientz, J. A., Hayes, G. R., Bhat, S., and Abowd, G. D. Farther than you may think: An empirical investigation of the proximity of users to their mobile phones. In Proceedings of the 8th international conference on Ubiquitous computing (Ubicomp 2006) (2006), 123–140. [12] Patwari, N., and Wilson, J. Spatial models for human motion-induced signal strength variance on static links. IEEE Transactions on Information Forensics and Security 6, 3 (September 2011), 791–802. [13] Reschke, M., Starosta, J., Schwarzl, S., and Sigg, S. Situation awareness based on channel measurements. In Proceedings of the fourth Conference on Context Awareness for Proactive Systems (CAPS) (2011). [14] Scholz, M., Sigg, S., Shihskova, D., von Zengen, G., Bagshik, G., Guenther, T., Beigl, M., and Ji, Y. Sensewaves: Radiowaves for context recognition. In Video Proceedings of the 9th International Conference on Pervasive Computing (Pervasive 2011) (2011). [15] Seifeldin, M., and Youssef, M. Nuzzer: A large-scale device-free passive localization system for wireless environments. CoRR abs/0908.0893 (2009).

528

Session: CoSDEO 2013: Device-free Radio-based Recognition

[16] Shi, S., Sigg, S., and Ji, Y. Activity recognition from radio frequency data: Multi-stage recognition and features. In Proceedings of the 5th IEEE Conference on Context Awareness for Proactive Systems (CAPS2012) (2012). [17] Shi, S., Sigg, S., and Ji, Y. Passive detection of situations from ambient fm-radio signals. In Proceedings of the International Workshop on Situation, Activity and Goal Awareness, in conjunction with UbiComp 2012 (2012). [18] Sigg, S., Scholz, M., Shi, S., Ji, Y., and Beigl, M. Rf-sensing of activities from non-cooperative subjects in device-free recognition systems using ambient and local signals. IEEE Transactions on Mobile Computing (TMC) (2013). accepted for publication. [19] Sohn, T., Varshavsky, A., LaMarca, A., Chen, M. Y., Choudhury, T., Smith, I., Consolvo, S., Hightower, J., Grisworld, W. G., and de Lara, E. Mobility detection using everyday gsm traces. In Proceedings of the 8th international conference on Ubiquitous computing (2006).

UbiComp’13, September 8–12, 2013, Zurich, Switzerland

˜ V., and Gibbons, J. [20] Want, R., Hopper, A., Falcao, The active badge location system. ACM Trans. Inf. Syst. 10, 1 (Jan. 1992), 91–102. [21] Wilson, J., and Patwari, N. Through-wall tracking using variance-based radio tomography networks. CoRR abs/0909.5417 (2009). [22] Wilson, J., and Patwari, N. Radio tomographic imaging with wireless networks. IEEE Transactions on Mobile Computing 9 (2010), 621–632. [23] Youssef, M., Mah, M., and Agrawala, A. Challenges: Device-free passive localizsation for wireless environments. In Proceedings of the 13th annual ACM international Conference on Mobile Computing and Networking (MobiCom 2007) (2007), 222–229. [24] Zhang, D., and Ni, L. Dynamic clustering for tracking multiple transceiver-free objects. In Proceedings of the 7th IEEE International Conference on Pervasive Computing and Communications (PerCom 2009) (2009).

529

Suggest Documents