CARMEN: A Cognitive Networking Testbed on Android OS Devices Matteo Danieletto⋆,⋄ , Giorgio Quer⋄ , Ramesh R. Rao⋄ , Michele Zorzi⋆,⋄ ⋆ DEI, University of Padova, via G. Gradenigo 6/B – 35131, Padova, Italy ⋄ University of California, San Diego – La Jolla, CA 92093, USA

Abstract—Wireless devices running the Android operating system offer a novel opportunity to study network behaviors and to observe and modify in real time key networking parameters. This opens up an unprecedented opportunity to study, test and evaluate the performance of techniques operating at different layers of the protocol stack and adopting the cognitive networking paradigm. In this article, we describe our novel IEEE 802.11 mesh network testbed that integrates Android based devices. The aim is to build a flexible testbed to observe in-stack and out-stack parameters of interest that, with some modifications, can be used to test networking techniques in both civilian and emergency scenarios. We provide the implementation details to create an ad hoc network among these inexpensive commercial devices, and specify how to observe and modify the networking parameters at different layers of the protocol stack. We also run some standard protocols on the network, and we obtain some repeatable results, which are similar to other results from the literature and which confirm our testbed as a valid tool for ad hoc network performance evaluation.

I. I NTRODUCTION Cognitive networking (CN) [2], [3] is a network paradigm that deals with how wireless systems learn the relationships between the network behavior and its performance, and plan and make decisions to achieve local, end-to-end, and networkwide goals. CN, different from cognitive radio (CR) networking, tries to jointly optimize the different layers of the protocol stack. Novel MAC, routing, transport and application layer techniques for wireless mesh networks have been proposed and analyzed by the networking community, and can be adopted in a cognitive network. The evaluation of their performance is not an easy task and is in fact an interesting research problem. Different approaches have been used to evaluate the network performance in a realistic scenario. One of them is based on the use of a discrete event network simulator e.g., ns (http://www.nsnam.org/), which is sufficiently accurate for most research works. ns is a powerful tool for a preliminary test, and the software developed to test a specific wireless channel can be easily reused to test other techniques in slightly different scenarios. Unfortunately, it is not always clear how realistic such simulations are, as it is not possible to predict and simulate all the factors that play an important role in a real network scenario. Another approach to test the network performance relies on ad hoc network deployments, appropriately designed to test a specific layer of the protocol stack. In [4], a limited number of nodes, deployed in fixed positions, are organized A preliminary version of this paper was presented at IEEE MILCOM 2013 [1].

in a wireless network to compare the performance of two routing protocols, optimized link state routing (OLSR) [5], and better approach for mobile ad hoc network (BATMAN http://www.open-mesh.org/). In [6] another routing protocol, ad hoc on-demand distance vector routing (AODV), is compared with OLSR. Also in this case, the wireless nodes are not mobile. Another interesting network deployment is Roofnet [7], an experimental IEEE 802.11b/g mesh network composed of 50 nodes, deployed on the MIT campus to test different routing protocols. The main disadvantage of these solutions is that the deployments are designed to only test some specific networking techniques, and they can not be easily reused to test different layers of the protocol stack. We have chosen to use tablets and smartphones running the Android operating system (OS), since these devices are commercially available, relatively inexpensive, mobile, and highly customizable. With these devices, we have realized the Cognitive AndRoid MEsh Network testbed (CARMEN), based on the IEEE 802.11 standard. CARMEN can be used to test cognitive networking techniques, since each node can observe and modify its in-stack parameters, i.e., those parameters that can be directly observed and acted upon at different layers of the protocol stack. Furthermore, the devices are equipped with other sensors to observe out-stack parameters (i.e., those parameters that are not directly related to the protocol stack, e.g., environmental or positioning data) that can be exploited to design and to test novel cognitive networking protocols. We can envision its use, with minor modifications, in both standard civilian and emergency scenarios. In the rest of the paper, we overview some network testbeds, which are similar in scope to CARMEN and highlight their main features. Then we detail our approach, which is based on the following principles: 1) the nodes in our testbed are commercial devices (smartphones and tablets), which are relatively inexpensive if compared with the ad hoc hardware commonly used to build a wireless mesh network testbed; 2) the testbed allows the collection and the modification of several parameters at different layers of the protocol stack, thus many cognitive networking techniques can be tested within such framework; 3) the testbed can be reproduced with a limited effort, as detailed in this paper.

TABLE I S ELECTED C OGNITIVE N ETWORKING TESTBEDS . Testbed

ORBIT EMULAB

UCR

CORES

out-stack

Mobility Real Emu

Hardware

N. nodes

Deployment

Access

Virginia Tech http://cornet.wireless.vt.edu/

USRP2

48

multiple floors

Open1

Yes

Yes

No

No

N/A

Trinity College Dublin http://www.crew-project.eu/iris/

USRP2

N/A

N/A

Open1

Yes

Yes

No

No

N/A

WinLab (Rutgers University) http://www.orbit-lab.org/

USRP1-2 PC3

28 400

20m x 20m grid

Open1

Yes2

Yes2

No

N/A

Yes

University of UTAH http://www.emulab.net/

USRP1-2 PC3

35 36+18

N/A multiple floors

Open1

Yes2

Yes2

No

No

N/A

UCR http://networks.cs.ucr.edu/

WARP USRP PC3

6 15 58

Not specified single floor single floor

Closed

Yes2

Yes2

No

No

N/A

UCLA http://cores.ee.ucla.edu/

BEE2 USRP2

N/A

N/A

Closed

Yes

Yes

No

No

Yes

VT-CORNET IRIS

in-stack Observe Change

Location and website

1 = a web registration is necessary and the source code is not available; 2 = at the MAC layer this depends on the WiFi chipset 3 = workstations equipped with standard 802.11 a/b/g/n

II. N ETWORK TESTBEDS : AN OVERVIEW There exist a few testbeds which are flexible enough to test cognitive networking techniques, more realistic than ns, and more flexible than networking deployments designed to test specific routing techniques. They are summarized in Tab. I, where they are also compared in terms of hardware and deployment characteristics. Most of the testbeds are designed to test CR techniques, but they can be adapted to investigate CN solutions. They are based on software defined radio (SDR), a radio communication system where all the layers of the protocol stack are implemented in software on a device. The device is connected to an ad hoc hardware responsible for producing the modulated signal. The universal software radio peripheral (USRP) is a common choice to build the physical layer; another option is to use the wireless open access research platform (WARP, http://warp.rice.edu/). The main disadvantage of USRP is the latency introduced by the GNU Radio and OS kernel, by the wired communication between the host computer and the USRP, and by the USRP hardware itself [8]. With the WARP board, instead, both PHY and MAC layers can be implemented on the FPGA to reduce latency. However, the FPGA on board of WARP is not powerful enough to accommodate full-duplex communication, which is often desirable in CR testbeds. A discussion on the characteristics and drawbacks of the different hardware components used for CR and CN testbeds can be found in [9]. The paper discusses also testbeds which are not based on SDR, so the extraction of relevant MAC parameters depends on the hardware (WiFi card, with MAC layer released only in binary code) and the OS adopted. Three interesting testbeds are the Virginia Tech cognitive radio network (VT-CoRNET) [10], the open-access radio testbed for next-generation wireless network (ORBIT) [11], and EMULAB [12], which are CR testbeds with the goal of optimizing spectrum sharing. They are composed of a very flexible architecture, can be controlled remotely, and are available to the research community to run network experiments. VT-CoRNET is an SDR testbed developed using 48 USRP2

nodes, which are a newer version of the USRP nodes. The nodes are spread over four floors of a building and equipped with a custom-made daughterboard spanning a frequency range from 100 MHz to 4 GHz. There exists a similar project also in Europe, at the Trinity College Dublin, Ireland, named implementing radio in software (IRIS) [13]. The main idea is the same as for VT-CoRNET: it is composed of a highly flexible architecture for real-time radio reconfiguration. Its nodes are fully reconfigurable SDR (USRP2). ORBIT is a heterogeneous testbed developed at WINLAB, Rutgers University. It is made of 28 SDR (USRP and USRP2), and 400 programmable radio nodes with standard WiFi cards, such as the Atheros chipset with MADWifi driver or the Intel chipset with IPW2200 driver. EMULAB is a testbed focused on higher-layer network protocol research, both wired and wireless. The nodes are Pentium III PCs, 18 are deployed on multiple floors of a building, and 36 are inside a laboratory. There are also 35 SDR nodes (USRP and USRP2) connected to the testbed. Both ORBIT and EMULAB have an integrated emulation mode that makes it possible to test new protocols written in ns3, at the MAC, routing or transport layers. A different type of testbed is the cognitive reconfigurable embedded systems (CORES) [14]. This testbed is based on the Berkeley emulation engine (BEE2), a generic multipurpose emulation platform. It includes a multi-FPGA emulation engine, and can connect up to 18 front-end boards via multigigabit interfaces. These interfaces can be divided into primary and secondary users, in order to test cognitive networking protocols. CORES, like ORBIT and EMULAB, uses an emulation mode, which works inside the BEE2 with a network simulator. This hybrid approach, with code written in ns3 but run in real nodes, is useful in case the protocols need to be tested with a real physical layer, but has three major drawbacks: 1) the hardware used is in general very expensive, thus requiring significant resources to reproduce the testbed; 2) mobility of the nodes is usually not considered, but can be emulated via software, as in ORBIT and CORES; 3) depending on the specific WiFi card in the wireless device adopted, some of

the functionalities of ns3 at the MAC layer are not available. Finally, another interesting testbed is at UC Riverside [15]. It is composed of 50 IEEE 802.11a/b/g nodes, 8 IEEE 802.11n nodes, 6 WARP boards and 15 USRPs deployed on a floor of a university building. Compared to these testbeds, CARMEN has the advantage that it can be ported to any Linux kernel based devices, using the software and guidelines provided in this paper. It is not a CR testbed, since it does not have the PHY layer flexibility of SDR testbeds, but is rather a CN testbed, allowing the processing and modification of key parameters to adaptively change the transport, network, and/or MAC layers. It is based on portable devices, which can be physically moved during an experiment, creating real mobile scenarios. III. A NDROID E COSYSTEM Android is the most popular operating system for portable devices (smartphones and tablets). It is very flexible thanks to its software development kit (SDK), i.e., the toolkit needed to develop an application (APP), which is released for all the main commercial OSs for personal computers, giving rise to a very large Android software community that shares key information to develop new APPs. The core of the Android system is a Linux kernel, whose source code is publicly available. It can be modified and compiled through the native development kit (NDK), a free toolset released by the Android team to cross compile C/C++ code. Thus, it is relatively simple for a programmer to write and enable new features, e.g., to enable a pure ad hoc communication mode by modifying the Linux kernel source code with the NDK. It is also possible to observe and modify some network parameters at different layers of the protocol stack, thus allowing a real implementation of cross-layer techniques following a cognitive network paradigm. The rest of the Android OS structure is divided into three macro layers. The top layer is the Application layer, where there reside APPs like contacts, browser, location manager, network manager, etc. The application programming interfaces (APIs), provided by the SDK, are designed to exploit the OS architecture so the hardware device is completely transparent to the APP developer (except for cases in which the APP needs to use very specific hardware functionalities). The middle layer provides all libraries and the Dalvik Virtual Machine necessary to interface an APP with the lower layer. The lower layer is based on the Linux kernel which can be modified directly by rooting the device. The flexibility of the Android ecosystem has been exploited to develop an Android network testbed. The software projects most closely related to our work are: 1) Commotion (https://commotionwireless.net/), which aims to create a decentralized mesh network through an open-source communication tool that uses heterogeneous devices like mobile phones, computers, and other wireless devices; and 2) Serval (http://www.servalproject.org/), which aims to create a mesh network through mobile devices. Serval started as an independent project, and is now a branch of the Commotion project. In Serval, an Android–based multi–hop ad hoc network is implemented among smartphones running the Android OS. The mobile phones can communicate

even in the absence of a fixed infrastructure via the ad hoc network mode. Unfortunately, these projects do not allow the observation and modification of in-stack parameters, since they are designed to provide connectivity in a natural disaster scenario, in the absence of any network infrastructure. Furthermore, with the current version of Android, the ad hoc mode is disabled, so it would be necessary to activate it following a procedure like the one described in our paper. IV. C OGNITIVE A NDROID M ESH N ETWORK The implementation of the CARMEN testbed is divided into the following phases. (1) Enabling ad hoc communication among the nodes. (2) Organizing nodes in a multi–hop network. (3) Observing/modifying some networking parameters as required by cognitive networking techniques. (4) Allowing nodes’ mobility. The entire source code, with patches and Android ROM binary code, can be downloaded from https://github.com/CognetTestbed, reused, and modified. A. Ad hoc communication mode In a standard Android OS, the only node to node communication mode allowed by default is a high level paradigm based on a client/server architecture named WiFi Direct (http://www.wi-fi.org/). In WiFi Direct, routing can be implemented only as an application above the transport layer, limiting the efficiency and the flexibility of a multi– hop network. Furthermore, only few MAC parameters are observable exploiting the WiFi chipset on board of the device. We adopt a different solution to enable the ad hoc mode, dealing directly with the IEEE 802.11 modes, called service sets. The standard one is the basic service set (BSS), in which a central controller (the access point) manages the connections among the devices, organized in a star topology. Another mode is named independent basic service set (IBSS), also known as ad hoc mode, in which each driver manages its own connections without a central controller. The IBSS is disabled in the most recent versions of Android, but it must be re-enabled to create a mesh network that allows multi–hop communications. Depending on the specific device, the IBSS mode is disabled either in the Application Framework or in the Linux Kernel. In the first case, in order to connect a device in IBSS mode to a network, we use the iw tool, i.e., a tool to configure the IEEE 802.11 driver. In other words, the iw can force the device to connect to an ad hoc network. This is the case of, e.g., a Samsung Galaxy Tab 7.0 device with Atheros AR6003 chipset and ath6kl driver. In the second case, when the IBSS mode is disabled in the Linux kernel, it is necessary to act directly on the Linux kernel and to modify the driver source code to activate the IBSS mode and create an ad hoc network. This is the case of, e.g., a Nexus 7 device with BCM4330 chipset and bcmdhd Linux driver. In most devices, in order to perform these operations, it is necessary to log in as a root user and to flash the ROM memory. We refer to [1] for the specific patch to be used. Another solution to enable the IBSS mode is to add a second WiFi card to the Android device, as shown in Fig. 1. We have chosen the Atheros AR9280 chipset, running the ath9k htc driver. This driver is very similar to the standard ath9k, which is used by many laptops and which implements

USERSPACE USERSPACE APP

CognetNode

CognetNode CognetManager TCP O/M MAC O/M ServerCog Socket TCP

KERNELSPACE TCP IP DLL

CognetModule

Netlink

Save Log file Routing Netlink 80211

Fig. 2. Cognet Software architecture.

Fig. 1. Two Android devices with an external WiFi card.

the whole datalink layer, with the difference that the rate control algorithm runs on firmware, so it is difficult to modify. In the case of the AR9280 chipset, however, the firmware source code is under open source license, thus the rate control algorithm can also be modified if needed. B. Multi–hop network Once the ad hoc network mode is enabled, the multi–hop routing at the network layer should be set up. We have tested OLSR, which is a proactive routing protocol for mobile ad hoc networks working over the datalink layer. OLSR sends control packets to regularly exchange topology information with the other nodes of the network. The source code of OLSR is available online (http://www.olsr.org/), and can be compiled on standard PCs or Android devices through the NDK. It is necessary to patch ifnet.c in order to run OLSR on any Android device. We have tested also the better approach to mobile ad hoc networking routing protocol (BATMAN, http://www.openmesh.org/), available inside the kernel (it is sufficient to enable it in the kernel source code). OLSR and BATMAN use two different metrics to define the routing table, the former uses expected transmission count (ETX), i.e., the expected number of transmissions for a MAC packet (frame) in order to be received at the destination; the latter uses the estimated throughput (ET), i.e., bitrate multiplied by the probability of transmission success, from the rate control algorithm at the MAC sublayer. Both these metrics can be derived from the parameters observed by CARMEN. Another interesting routing protocol is Babel, a loop-avoiding distance-vector routing protocol whose source code is available at http://www.pps.univ-parisdiderot.fr/∼jch/software/babel/. A cognitive networking technique, implemented in our tested, could select the best among these routing protocols, as a function of the current observed conditions of the network. C. Cognitive networking platform The software developed to create a mobile ad hoc network is composed of three parts: CognetModule, CognetNode, and

CognetManager, as illustrated in Fig. 2. It has been tested in a heterogeneous network composed of 4 Android devices and 50 fixed devices with different Alix versions. All the devices were running our software. CognetModule can observe and modify the values of the TCP parameters by reading and modifying the struct tcp sock C structure. This data structure contains all the information about the TCP connections, and is refreshed every time a new TCP acknowledge packet is received by the node. CognetModule uses static struct tcp congestion ops, a kernel C structure to retrieve the struct tcp sock and modify on the fly some TCP parameters. In particular, the congestion avoidance algorithm is implemented in CognetModule by creating several proc files, stored in /proc/CARMEN, to exchange information between kernel and user space. Thanks to the proc files it is possible to change on the fly the TCP parameters. Furthermore, CognetModule can also receive as inputs the IP netmask to select the TCP connections that should be monitored. CognetNode is a user space program, which runs in a node to retrieve and change some MAC/TCP parameters, and which is connected to the other nodes in the network. CognetNode is composed of three independent threads: TCP observation/modification (O/M), MAC O/M, and ServerManager. The TCP O/M thread is connected to the CognetModule via an inter-process communication channel (Netlink socket). The thread exploits the blocking behavior of the C function read, thus it reads the Netlink buffer only when CognetModule is changing such buffer. The TCP O/M thread can modify in real time some TCP parameters like the TCP congestion window (CWND), the retransmission time out (RTO), and the maximum allowed value of the CWND (CWND CLAMP), by exploiting the proc files created by CognetModule. The MAC O/M thread can observe/modify MAC parameters through the NL80211 socket functionality (in Linux kernel), which communicates directly with the driver1 . It saves the MAC parameters of all the CognetNodes that are under its WiFi coverage. CognetManager can remotely control the nodes by sending commands to the ServerManager thread like, e.g., start/stop the experiment, change transmission channel or power, and change the sampling rate of the observed MAC parameters. We developed two versions of CognetManager (with library in C), one for desktop PC, and one for Android. It creates a C socket to open a TCP connection for each node in the experiment to 1 In order to fully exploit the ath9k htc driver, a patch of the firmware is also required, and is available in https://github.com/CognetTestbed.

control the node. In our testbed, these commands are received via a wired connection (for the Alix nodes), or via a second WiFi interface (for the Android nodes). This second control connection avoids delays in the control of each node also in the case of critical events, when the main connection may be flooded with data traffic. D. Mobility Emulator Android devices can be easily moved to test real mobility in a network scenario, but we added also a mobility emulator to test mobility in case it is not possible to physically change the position of such devices. Thus, the mobility emulator (ME) can require a change in the network topology through CognetManager. The ME uses the Netfilter library (in the Linux kernel) and NL80211 to artificially modify the transmission quality. This is enough to force a change in the routing table. E. Cognitive Network techniques We mention here some examples of Cognitive Networking techniques that can be implemented and tested in our testbed using the components detailed in this paper, and are being studied as part of our current work. In the literature, several adaptive protocols consider information from different layers to optimize the transport layer. In TCP-Cognitive Radio Adhoc Network (TCP-CRAHN) nodes exchange MAC layer information to optimally set up the congestion window. In TCP Friendly Rate Control (TFRC), several observable parameters are used to identify a true network congestion. In [16], the goal is to infer the presence of mobility events in order to optimize the network behavior. The inference is based on the observation of some MAC and TCP parameters, which are organized in a Bayesian network. This technique can also be implemented in our testbed, since the needed parameters are observable by CognetNode, and it is not difficult to also implement the modification to the protocol in case of mobility. The Bayesian network may be implemented in a central server or in selected nodes of the network. V. C OGNITIVE PARAMETERS A. Network: Observation and Action The TCP O/M thread is able to observe the following parameters (some of them can also be modified, as specified in Tab. II): • congestion window (CWND): the congestion window value at the sender; • slow start threshold (SSTHR): the slow start threshold used by the congestion avoidance algorithm; • TCP CLAMP: the maximum size reachable by CWND; • packets outstanding: the number of packets transmitted but not yet acked; • packets acked: the number of packets acked by the last ack packet received; • throughput: the number of bytes acked per unit time; • packets in flight: the number of packets outstanding plus the number of packets retransmitted. This number is a function of the size of CWND (the larger CWND, the more packets in flight, and consequently the higher the probability to have a timeout or a packet loss);

round trip time (RTT) [ms]: the time interval from when a packet is sent to when the sender receives the corresponding ack; • smoothed round trip time (SRTT): a weighted average of the past RTT. SRT T [i] = (1 − β) · (SRT T [i − 1]) + β · RT T [i], with 0 < β ≤ 1; • RTO: the retransmission timeout which is equal to twice the SRTT and varies between 200 ms and 2 min. A wrong setting of the RTO can induce a high rate of packet dropping; • SND UNA: the sequence number of the first byte of data that has been sent but not yet acknowledged; • SND NXT: the next sequence number to be sent. The MAC O/M thread can observe the following parameters at the data link layer, as a function of the chipset used (see Tab. II): • the number of frames transmitted (TX) and received (RX), as well as the number of bytes TX and RX. These parameters are recorded in the file /proc/net/dev; • the transmission channel, the transmission power and the received signal strength indicator (RSSI) can be observed through the specific driver Input/Output control function, named ioctl; • the number of fragmented packets transmitted and received (# fragments TX/RX), the number of dropped frames (# frames RX dropped), the number of retransmitted frames (# frames TX retry count) and the number of frames whose retransmission failed (# frames TX retry failed); • the arbitration inter-frame spacing (AIFS) time and the contention window range, specified by the minimum value (CWmin) and by the maximum value (CWmax) allowed. The accuracy of the TCP parameters observed is guaranteed by the fact that they are extracted directly from the protocol stack of a Linux kernel implementation, following the suggestions in the corresponding RFC. We performed a few simple tests to verify the parameters fidelity. We checked, using tcpdump, that the number of TCP segments sent is equal to the number of segments at the receiver. We also verified, after receiving an ack packet, that the sequence number in the ack corresponds to the sequence number of the last acknowledged packet extracted from the CognetModule. We also performed some tests with a spectrum analyzer to verify the accuracy of RSSi, power, frequency, and channel number, parameters extracted from CognetNode. These tests provide evidence about the fidelity of our approach and implementation. •

B. Out-stack: Observation In the CARMEN testbed we can observe several out-stack parameters, depending on the Android device adopted. This information may be relevant from a networking point of view, e.g., some geographical areas may have better connectivity options, and they can reveal some aspects of the user’s behavior and preferences. They can be observed with the Sensor object Java inside the Android API (developer.android.com/reference/android/hardware/Sensor.html), while their modification depends on the user behavior.

TABLE II TCP/IP STACK PARAMETERS OBSERVED AND MODIFIED FOR DIFFERENT DEVICES INTEGRATED INTO OUR NETWORK TESTBED . Observable/Writable

Layer

Parameters

Observable Writable

TCP TCP TCP TCP MAC MAC MAC MAC

CWND SSTHR TCP CLAMP RTO CWmin CWmax AIFS TX power

Observable

TCP TCP TCP TCP TCP TCP TCP TCP TCP TCP TCP TCP MAC MAC MAC MAC MAC MAC MAC MAC MAC MAC MAC MAC MAC

IP address Port # lost packets # packets in flight # packets outstanding # packets acked # packets lost Throughput RTTvar SRTT SND UNA SND NXT Transmission Channel RSSI Bytes RX # frames RX # frames RX duplicate # frames RX fragments # frames RX dropped Bytes TX # frames TX # fragments TX # frames TX retried # frames TX retry failed Inactive station

Laptop b43 (bdcm4331)

Alix 3d2 ath5kl (ar5413)

Tab-7 P6210 ath6kl (ar6003)

Nexus 7 bdcmhd (bdcm4330)

USB Adapter ath9k htc (AR9271)

r/w r/w r/w r/w r/w r/w r/w r/w

r/w r/w r/w r/w r/w r/w r/w r/w

r/w r/w r/w r/w NO NO NO r

r/w r/w r/w r/w NO NO NO r/w

r/w r/w r/w r/w r/w r/w r/w r/w

r r r r r r r r r r r r r r r r r r r r r r r r r

r r r r r r r r r r r r r r r r r r r r r r r r r

r r r r r r r r r r r r r r r r r r NO r r r r NO NO

r r r r r r r r r r r r r r r r NO NO NO r r r NO NO NO

r r r r r r r r r r r r r r r r r r r r r r r r r

r = observable, w = writable, NO = to be implemented, TX= transmitted, RX= received, # = number.

A standard Android tablet already includes the following out-stack parameters: •

•

•

•

acceleration: its value (in m/s2 ) and its direction are measured by an accelerometer: orientation of the device (in the three axes): it is measured by a gyroscope; position (outdoor): it is measured by a global position system (GPS); position (indoor): it is measured by the integrated Fused Sensor, which combines the values of the accelerometer and the gyroscope.

The on the go (OTG) functionality is enabled by default in every CARMEN node, thus, if the node is not natively equipped with a sensor for a specific out-stack parameter, it is possible to connect it to an external sensor to obtain such parameter. VI. R ESULTS In this section we describe a set of experiments to test the functionalities of the CARMEN testbed and the fidelity of the measurements. We repeat the same experiments performed in other real implementations or simulated platforms, in order to highlight the validity and the repeatability of the results obtained. Other experiments would be possible but are not reported due to lack of space.

In all experiments, we use four Nexus 7 tablets connected to an external WiFi USB adapter with driver ath9k htc, and four Alix miniPCs model 3d2 with WiFi driver ath5kl, which are deployed in an indoor scenario. In order to assure the repeatability of the experiment, at the beginning of the experiment we start a new TCP connection with the command: sysctl -w net.ipv4.tcp_no_metrics_save=1

A. Experiment 1: set a bound to the CWND In this experiment, we reproduce in our testbed an experiment described in [6] with OLSR, where a limit (TCP CLAMP) is imposed to the CWND value. We performed twenty 60 s long experiments for each of the chosen values for TCP CLAMP. In Fig. 3-(a) we show the results for a 2-hop topology with three nodes, where we can see that for small values of TCP CLAMP the limit is rapidly reached and then the value of the CWND remains constant, while without a limit on the CWND value we have the standard TCP behavior, similar to what observed in [6] and [7]. Using the same topology, in Fig. 3-(b) we show instead the average values of the RTT and the number of ack packets per second, as a function of the limit in TCP CLAMP. This shows that, for this particular network, in order to maximize the throughput we should use a value of TCP CLAMP = 4. In Fig. 3(c) we observe the average throughput as a function of the number of nodes in the network, organized in a multi-hop

Density Ack RTT

30

150

250

100

200

50

Density Ack

CWND

40

300 CLAMP Unlimited CLAMP 32 CLAMP 16 CLAMP 8 CLAMP 4

20

RTT [ms]

50

10

0 0

2

4

6

8 time [s]

10

12

150 4

14

8

16 TCP CLAMP

(a)

(b)

3

0.4

CLAMP Unlimited CLAMP 32 CLAMP 16 CLAMP 8 CLAMP 4

2.5

1 Hop 2 Hop 3 Hop

0.35 0.3 err

2 MAC P

Average Throughput [MBps]

0 UNLIM

32

1.5 1

0.25 0.2 0.15 0.1

0.5 0 2

0.05 3

Chain Length

4

5

0

4

8

(c)

16 TCP CLAMP

32

N

(d)

Fig. 3. Testbed measurements for different values of TCP CLAMP: (a) CWND as a function of time, (b) average RTT and estimated ack density, (c) average throughput as a function of the chain length, and (d) estimated MAC probability of error.

B. Experiment 2: OLSR vs BATMAN In this second experiment, we compare three routing strategies in terms of average TCP throughput. We adopted a fixed chain topology with 5 nodes and a maximum of 4 hops, which has a unique best routing solution. The routing techniques are 1) MAN, a fixed routing strategy, where the routing table is defined at the beginning of the transmission by minimizing the distance to the next hop in the direction of the destination; 2) BATMAN; and 3) OLSR. We also add a fourth case, 4) OLSR BLKD, in which the routing protocol is the standard OLSR protocol, but when the routing table is changed from the one chosen by MAN, the packets are blocked with iptables. This last case is used to show that OLSR sometimes makes non optimal choices. We performed 10 experiments, each of length 10 minutes. In Fig. 4 we represent the average TCP throughput as a function of the number of hops. The best performance is obtained with MAN, which does not modify the routing table during the experiment. According to this experiment, in this simple fixed topology BATMAN performs better than OLSR, because it

modifies the routing table less often than OLSR, since ET, the BATMAN metric to choose the best next hop, is more stable than ETX, the metric adopted by OLSR, as we verified. We notice also that the BATMAN control packets are shorter than the OLSR control packets. In a nutshell, we have verified that we can easily implement different types of routing protocols in CARMEN. The performance in this simple case is in accordance with our intuition, i.e., the routing protocol performance is affected by the number of times the routing table changes, since the topology is fixed 6

Average Throughput [Mbps]

chain topology. As expected, the throughput decreases as a function of the number of hops. We measured also the number of retransmissions at the MAC layer, and we estimated the retransmission probability as a function of the number of hops and of the value of TCP CLAMP.

MAN BATMAN OLSR OLSR BLKD

5 4 3 2 1 0

2

3

Chain Length

4

5

Fig. 4. Throughput with different routing algorithms varying the number of hops.

4

250

3.5

200

3

150

2.5

100

2

50 0 0

Throughput ETX

10

20

30 time [s]

40

50

4

200

3

100

2 2−Hop

1.5

1 60

Throughput ETX

ETX

300

300

Throughput [KBps]

4.5

ETX

Throughput [KBps]

350

1−Hop 1−Hop Move

0 0

20

40

(a)

60

80 time [s]

100

120

140

1 160

(b)

Fig. 5. Throughput and EXT in the presence of: (a) mobility simulated with Iptables, and (b) real mobility.

with a unique best routing solution. Indeed, every time the routing table changes, the RTT increases, and this may cause some delays or TCP packet losses. This result suggests that it is worth trying in a real scenario the CN technique proposed in [16], where the mobility of the nodes was inferred as a function of some observable parameters, and different routing and TCP strategies were adopted accordingly to optimize the network performance. C. Experiment 3: mobility In this experiment, we observe the TCP throughput variation in the presence of mobility events for a network with three nodes, the source S, the destination D, and a relay R. We show two performance metrics: the TCP throughput and the ETX metric for the path. In Fig. 5-(a), we have simulated mobility using Iptables, a tool available in every Linux distribution to set up different rules inside the firewall. In particular, we can force a node to drop or to accept the incoming packets from another node, thus forcing the network to be multi–hop even if all the nodes are within transmission range [1]. The experiment is organized as follows. At the beginning the three nodes are in range, thus node S sends packets directly to node D. After 20 seconds, with Iptables we block the direct communication between S and D, thus the topology changes and the transmissions are delivered through node R. We notice that it takes approximately 20 seconds before the path from S to D through R is set up. This is due to the standard operation of the OLSR protocol, where a link quality hello (LQH) packet is sent every 2 seconds, and the quality estimation is done by averaging over the past 10 LQH packets received. The curve of ETX confirms this behavior. Finally, in Fig. 5-(b) we show the performance in the presence of real mobility. At the beginning the three nodes are in range. After 30 seconds node D starts to move in the opposite direction with respect to S, and after approximately 90 seconds the communication between S and D is interrupted. It takes 20 seconds for the OLSR protocol to establish another path through node R. At this point node D rapidly comes back to the initial position, and approximately 140 seconds after the beginning of the experiment the link between S and D is reestablished, and the final values of the TCP throughput and the ETX are similar to their initial values.

VII. C ONCLUSIONS In this paper, we described CARMEN, a valid tool for many networking research groups interested in creating their own cognitive networking experimental testbed with limited costs, or in integrating an existing testbed with mobile devices. CARMEN is composed of relatively inexpensive devices and is flexible enough to test different cognitive networking techniques at different layers of the protocol stack. We have highlighted the main steps of the software development, tested its functionalities, and made the source code available to the research community. In future work, the flexibility of this testbed will be exploited by our research group to test our newly developed cognitive networking techniques. In the future, we expect other groups to adopt this paradigm, giving rise to a network of developers that can further enrich the functionalities of this platform. ACKNOWLEDGMENTS This work was partially supported by the U.S. Army Research Office, under Grants No. W911NF-11-1-0538 and W911NF-11-1-0336, and by Fondazione Cariparo through the program “Progetti di Eccellenza 2011-2012.” R EFERENCES [1] M. Danieletto, G. Quer, R. Rao, and M. Zorzi, “On the Exploitation of the Android OS for the Design of a Wireless Mesh Network Testbed,” in IEEE MILCOM, San Diego, CA, US, Nov. 2013. [2] B. Manoj, R. Rao, and M. Zorzi, “Cognet: a cognitive complete knowledge network system,” IEEE Wireless Communications, vol. 15, no. 6, pp. 81–88, Dec. 2008. [3] R. W. Thomas, D. H. Friend, L. A. DaSilva, and A. B. MacKenzie, “Cognitive Networks: Adaptation and Learning to Achieve End-to End Performance Objectives,” IEEE Commun. Mag., vol. 44, no. 12, Dec. 2006. [4] L. Barolli, M. Ikeda, G. De Marco, A. Durresi, and F. Xhafa, “Performance analysis of OLSR and BATMAN protocols considering link quality parameter,” in Advanced Information Networking and Applications (AINA). IEEE 23rd International Conference on, Bradford, UK, May 2009. [5] P. Jacquet, P. Muhlethaler, T. Clausen, A. Laouiti, A. Qayyum, and L. Viennot, “Optimized Link State Routing protocol for ad hoc networks,” in IEEE International Multi Topic Conference (INMIC), Lahore, Pakistan, Dec. 2001. [6] G. Anastasi, E. Ancillotti, M. Conti, and A. Passarella, “Experimental analysis of TCP performance in static multi-hop ad hoc networks,” in Multi-hop Ad hoc Networks from Theory to Reality. Nova Science Publisher, Sep. 2007.

[7] D. Aguayo, J. Bicket, S. Biswas, G. Judd, and R. Morris, “Link-level measurements from an 802.11b mesh network,” in ACM SIGCOMM Comput. Commun. Rev., Portland, OR, US, Aug. 2004. [8] N. B. Truong, Y.-J. Suh, and C. Yu, “Latency Analysis in GNU Radio/USRP-based Software Radio Platforms,” in IEEE MILCOM, San Diego, CA, US, Nov. 2013. [9] O. Gustafsson, K. Amiri, D. Andersson, A. Blad, C. Bonnet, J. Cavallaro, J. Declerck, A. Dejonghe, P. Eliardsson, M. Glasse, A. Hayar, L. Hollevoet, C. Hunter, M. Joshi, F. Kaltenberger, R. Knopp, K. Le, Z. Miljanic, P. Murphy, F. Naessens, N. Nikaein, D. Nussbaum, R. Pacalet, P. Raghavan, A. Sabharwal, O. Sarode, P. Spasojevic, Y. Sun, H. Tullberg, T. Vander Aa, L. Van der Perre, M. Wetterwald, and M. Wu, “Architectures for cognitive radio testbeds and demonstrators - an overview,” in CROWNCOM, Cannes, France, Jun. 2010. [10] T. Newman, A. He, J. Gaeddert, B. Hilburn, T. Bose, and J. Reed, “Virginia Tech cognitive radio network testbed and open source cognitive radio framework,” in 5rd International Conference on Testbeds and Research Infrastructure for the Development of Networks and Communities, TridentCom, Washington, DC, US, Apr. 2009. [11] D. Raychaudhuri, I. Seskar, M. Ott, S. Ganu, K. Ramachandran, H. Kremo, R. Siracusa, H. Liu, and M. Singh, “Overview of the ORBIT radio grid testbed for evaluation of next-generation wireless network protocols,” in IEEE Wireless Communications and Networking Conference (WCNC), New Orleans, LA, US, Mar. 2005. [12] B. White, J. Lepreau, L. Stoller, R. Ricci, S. Guruprasad, M. Newbold, M. Hibler, C. Barb, and A. Joglekar, “An integrated experimental environment for distributed systems and networks,” ACM SIGOPS Operating Systems Review, vol. 36, pp. 255–270, Dec. 2002. [13] P. Sutton, J. Lotze, H. Lahlou, S. Fahmy, K. Nolan, B. Ozgul, T. Rondeau, J. Noguera, and L. Doyle, “IRIS: an architecture for cognitive radio networking testbeds,” IEEE Communications Magazine, vol. 48, no. 9, pp. 114–122, Sep. 2010. [14] P. Urriza, E. Rebeiz, and D. Cabric, “Hardware implementation of Kuiper-based modulation level classification,” in Conference on Signals, Systems and Computers (ASILOMAR), Pacific Grove, CA, US, Nov. 2011. [15] I. Broustis, J. Eriksson, S. Krishnamurthy, and M. Faloutsos, “A blueprint for a manageable and affordable wireless testbed: Design, pitfalls and lessons learned,” in 3rd International Conference on Testbeds and Research Infrastructure for the Development of Networks and Communities, TridentCom, Orlando, FL, US, May 2007. [16] M. Mezzavilla, G. Quer, and M. Zorzi, “On the Effects of Cognitive Mobility Prediction in Wireless Multihop Ad Hoc Networks,” in IEEE ICC, Sydney, Australia, Jun. 2014.

Matteo Danieletto (S’07, M’14) was born in Padova, Italy, on May 2nd, 1984. He received the B.Sc. degree, the M.Sc. degree in Computer Science Engineering and the Ph.D. degree in Information Engineering from University of Padua, Italy, in 2007, 2009, 2014, respectively. In 2013 he was a visiting researcher at the Qualcomm Institute, University of California San Diego, designing and building a Cognitive Network testbed with Android off-the-shelf devices. His PhD studies were focused on Wireless Sensor Network integrated in the Internet of Things Architecture (IoT-A), and as well as robotics. He is currently a post-doctor at the Information Engineering Department, University of Padova, Italy. His research interests lie in the area of cognitive network and wireless communication.

Giorgio Quer [S’06, M’12] ([email protected]) received the B.Sc., M.Sc., and Ph.D. degree in Information Engineering from University of Padova, Italy, in 2005, 2007 and 2011, respectively. In 2007 he was a visiting researcher at the Centre for Wireless Communication at University of Oulu, Finland, and in 2010 he was a visiting scholar at Calit2, University of California San Diego (UCSD). Since 2011, he is a postdoc researcher at UCSD, where he leads a team of programmer analysts, graduate and undergraduate students, and visiting scholars, focusing on wireless network optimization

and physiological signal analysis. His research interests also include wireless sensor networks, cognitive networking, signal processing, machine learning, medical imaging, and wireless health.

Ramesh Rao [F’10] ([email protected]) earned his Ph.D. in Electrical Engineering from the University of Maryland, College Park in 1984, after receiving his M.S. from the same institution in 1982. He has been a faculty member at UC San Diego since 1984, and Director of the Qualcomm Institute, UCSD division of the California Institute for Telecommunications and Information Technology (Calit2), since 2001. He also holds the Qualcomm Endowed Chair in Telecommunications and Information Technologies in the Jacobs School of Engineering at UCSD, and a member of the school’s Electrical and Computer Engineering department. Prior to QI (Calit2), Professor Rao was also the Director of UCSD’s Center for Wireless Communications (CWC). Dr. Rao is involved on a day-to-day basis with a wide variety of research initiatives at QI. He leads several major interdisciplinary and collaborative projects and has been a PI on dozens of federal-, state-, foundation- and industry-funded grants. He also was named a Member of the Rady Children’s Hospital and Health Center Board IT Task Force; Member of the Board of Advisors at CommNexus San Diego (a network of communications companies); Member of the UC San Diego Health System Advisory Board; Member of the San Diego Foundation Regional Vision Council’s Education Task Force. In 2010, Dr. Rao received a Professional Gordon Engineering Leadership Award from UCSD’s Gordon Engineering Leadership Center.

Michele Zorzi [F’07] ([email protected]) received his Laurea and Ph.D. degrees in electrical engineering from the University of Padova, Italy, in 1990 and 1994, respectively. During academic year 1992-1993, he was on leave at the University of California, San Diego (UCSD). After being affiliated with the Dipartimento di Elettronica e Informazione, Politecnico di Milano, Italy, the Center for Wireless Communications at UCSD, and the University of Ferrara, in November 2003 he joined the faculty of the Information Engineering Department of the University of Padova, where he is a professor. His present research interests include performance evaluation in mobile communications systems, random access in mobile radio networks, ad hoc and sensor networks, energy constrained communications protocols, and underwater communications and networking. He was Editor-in-Chief of IEEE Wireless Communications from 2003 to 2005, Editorin-Chief of IEEE Transactions on Communications from 2008 to 2011, and Guest Editor for several Special Issues in IEEE Personal Communications, IEEE Wireless Communications, IEEE Network, and IEEE JSAC. He served as a Member-atLarge of the Board of Governors of the IEEE Communications Society from 2009 to 2011, and is currently its Director of Education.