Emulation of Ad Hoc Networks
David A. Maltz, Qifa Ke, David B. Johnson CMU Monarch Project Computer Science Department Carnegie Mellon University http://www.monarch.cs.cmu.edu/ Carnegie Mellon
CMU Monarch Project
[email protected]
Early Emulation Successes Stress testing of networking code via macfilter
Used to debug our 8 node ad hoc network testbed Description of interesting/problematic movement scenarios created Changing network topology extracted into trace file Physical machines running actual implementation synchronously read trace-file Machines prevented from receiving packets from non-neighbors Described at previous IETF and in CMU CS Tech Report 99-116 Found to be critical for achieving code stability, however cannot be used for performance evaluation
Propagation model grossly simplified MAC-layer effects unaccounted for CMU Monarch Project
[email protected]
Goal of Current Emulation Work Evaluate real systems under ad hoc network conditions:
Performance study of network protocols or applications Without implementing them in ns-2 Without deploying and operating physical machines in the field Using real implementation and real user pattern Only the network environment is simulated Evaluating real systems with just simulation is not practical:
Difficult to model real applications inside the simulator Real systems are significantly performance tuned Example: Evaluate performance of CMU Coda file system in an ad hoc network:
Coda is 100+ KLOC implemented and tuned Coda researchers have expertise to evaluate their system but can’t create network environments CMU Monarch Project
[email protected]
Method Leverage emulation work by Kevin Fall and VINT project ns-server is a single central machine running ns-2 Real application code runs on real machines directing real packets to “default router” (ns-server) User creates scenarios to control the environment inside ns-2: – Movement pattern of all mobile nodes inside ns-2 – Background traffic MN1
MN2 ns-server
MN4
MN3
Logical view of emulation setup
CMU Monarch Project
[email protected]
Informal Validation of Emulation Method Example: Compare FTP behavior between simulation and emulation:
ns-server: 450 MHz Pentium II CPU, running FreeBSD 3.1 10 simulated nodes Compare 30 MB FTP transfer between simulation and emulation 2 additional simulated FTP connections for background traffic 7
3.5
But how well does this scale?
3 bytes transfered
TCP time–sequence # plots have basically the same shape:
x 10
2.5 2 1.5 1 0.5 0 0
CMU Monarch Project
simulation emulation 50
100 150 200 time (seconds)
250
300
[email protected]
A More Challenging Example 16 simulated nodes moving in a complicated pattern with CBR and FTP background traffic 30 MB FTP transfer simulated in ns-2 30 MB FTP transfer between two physical nodes with emulation 7
3
x 10
bytes transfered
2.5 2 1.5 1 0.5 0 0
CMU Monarch Project
simulation emulation 100
200 300 time (seconds)
400
500
[email protected]
Evaluating Emulation Scalability Limitations on emulation scalability:
System effects: scheduling delays, potential buffer overruns Additional latency of transferring packets to/from ns-server Only so much one CPU can do
SINK
Experimental scenario:
Conduct real FTP transfer between 2 physical nodes Vary the number of other simulated nodes in ns-2 Vary the amount of background traffic
CMU Monarch Project
SRC
30 M Bytes
000 111 11 00 000 111 00 0011 11 000 111 00 11 1111 0000 1111 00 11 10100000 1010
CBR
Scalability Test Scenario
[email protected]
Scalability Results Summary Ideally: emulation throughput = simulation throughput
Pure simulation achieves 1.49 Mb/s on a 2 Mb/s link Emulation achieves throughput that depends on overall simulator workload: # of CBR packets/second
# of nodes
0
10
50 100 300
4
1.44 Mb/s
8
1.30 Mb/s
16
960 Kb/s
FTP Throughput Achieved
CMU Monarch Project
[email protected]
Does the Simulator Keep up with Real Time? For event scheduled at t1 but processed at t2: time-lag = t2 – t1 Record the time-lag for each delayed event Histograms show almost all time-lags are less than 4 ms 100 CBR packets/sec
16 nodes
6
# of time-lag events
10
5
10
4
10
3
10
2
10
1
10
4
54
Max lag: 355 ms
CMU Monarch Project
104
154
204 ms
Total # of events: 1.64e+07
[email protected]
16 simulated nodes
8 simulated nodes
4 simulated nodes
All of the Histograms 0 packets/sec
10 packets/sec
50 packets/sec
100 packets/sec
300 packets/sec
1.03e+06
1.30e+06
2.38e+06
3.73e+06
9.13e+06
1.65e+06
2.28e+06
4.80e+06
7.95e+06
2.06e+07
4 54 104 154 204
4 54 104 154 204
4 54 104 154 204
4 54 104 154 204
4 54 104 154 204
2.89e+06
4.24e+06
9.65e+06
1.64e+07
4.35e+07
6
10
5
10
4
10
3
10
2
10
1
10
6
10
5
10
4
10
3
10
2
10
1
10
6
10
5
10
4
10
3
10
2
10
1
10
CMU Monarch Project
[email protected]
Conclusion Emulation works for networks of sufficient size and complexity to enable study of interesting application scenarios
Using direct emulation based on ns leverages future improvements to propagation models The bottleneck is the real-time requirement in ns-server Currently using to evaluate the Coda distributed file system Open questions:
How far can this technique be generalized? How to ensure performance measurements are meaningful? – Open loop: evaluate emulation system on many scenarios using reference application with good simulation models – Closed loop: after each emulation run, examine emulation trace to determine if emulation system introduced artifacts – Validate against real system in ad hoc network testbed CMU Monarch Project
[email protected]
