1

On the Effectiveness of Peer-Assisted Internet TV Broadcasting Rossano Gaeta,

Michele Garetto,

Abstract—Despite the increasing popularity of peer-assisted video streaming applications, there have been few attempts to quantify the long-term impact of such bandwidth demanding applications on the core Internet infrastructure, as the audience grows to very large numbers (e.g. millions) of users. In this paper we analyze asymptotic scale effects of massively-deployed IPTV applications, addressing the concerns raised by many Internet Service Providers and backbone operators about the feasibility of large-scale video broadcasting over the Internet. In particular, focusing on the total network bandwidth consumption, we derive useful bounds valid for general topologies and accurate scaling laws for special cases of regular and random graphs, comparing the effectiveness of different architectural solutions. Using a snapshot of the Internet topology, we also investigate traffic concentration issues by evaluating the stress distribution on individual network elements. Finally, we analyze the transient behavior of distributed, application-layer algorithms to optimize the construction of multicast delivery trees. Our results show that the peer-to-peer approach is viable as long as a careful design of the overlay topology is performed by the application.

I. I NTRODUCTION Today a new class of systems providing high-quality realtime video streaming is fast emerging and gaining more and more popularity. Thus, the Internet has to face a new challenge: delivering these applications to large set of users. Often termed as IPTV (Internet Protocol TeleVision), this scenario is very similar to video broadcasting, in which a very large number of users (up to millions) watches “TV” over the Internet. IPTV has initially been supplied by broadband operators exploiting IP multicast functionalities in small-scale environments through proprietary solutions, de-facto limiting the distribution of contents to each Internet Service Provider (ISP) network only. These IPTV systems follow the traditional client-server model of Internet protocols and services. However, the limited deployment of IP multicast in the Internet, due to several concerns about its scalability, lack of authentication and security mechanisms, and difficult integration with hierarchical routing, makes this solution unlikely to provide the infrastructure for future massively deployed IPTV. On the contrary, peer-assisted architectural solutions, in which users send data to each other using application-level multicast on overlay topologies, has emerged as a popular and promising approach to large-scale multimedia delivery, as demonstrated by an increasing number of successful applications [1]. In this paper we will refer to peer-assisted streaming systems with the general term P2P-TV. Such applications are expected to change the way in which we watch TV, providing ubiquitous access to a vast number of channels, personalizing TV experience, and enabling roaming TV services. Furthermore, the adoption of the P2P paradigm widely simplifies the operation and maintenance of the Internet infrastructure, ∗ R. Gaeta, M. Garetto and M. Sereno are with Dipartimento di Informatica, Universit`a di Torino, Italy; E. Leonardi is with Dipartimento di Elettronica, Politecnico di Torino, Italy.

Emilio Leonardi,

Matteo Sereno

pushing complexity to the end users, while at the same time relieving the bandwidth cost burden at the server. Although, from the users’ as well as from the content publishers’ point of view, this class of P2P applications possess very attractive and interesting properties, from the network operators’ point of view serious concerns exist about the ability of the Internet to support large scale P2P-TV systems, mainly because of the high bandwidth requirements and the lack of network-aware topology maintenance procedures in current applications. These concerns have recently emerged in news reports. For example, as reported in [2], some of the largest broadband providers in the UK are threatening to ”pull the plug” from the new iPlayer, which allows users to watch BBC programmes, because this service will place an intolerable strain on their networks. To improve the performance of P2P-TV systems, while at the same time reducing the impact of P2P traffic on the underlying physical infrastructure, several techniques have been proposed in the literature [3]–[6] that exploit peers proximity while building the overlay topology. However, application developers are only marginally interested to implement such techniques, especially due to the extra complexity required to integrate them in their systems. Moreover, it is still controversial whether network-aware techniques are indeed necessary to make P2P-TV scale to large numbers of users. Indeed, the majority of performance studies appeared so far in the literature rely on the assumption that the system bottleneck is due to the limited bandwidth on the users’ access lines (especially the upload bandwidth); hence, the main concern in the analysis and design of streaming systems has been to maximize the exploitation of peers access bandwidth. This is confirmed by recent measurement studies, which have revealed that popular applications such as PPLive and Soapcast, whose internal details are not public, seem essentially to ignore peer proximity [7]. This paper shifts the attention on the impact that largescale P2P-TV applications may have on the underlying Internet infrastructure. Such impact is evaluated asymptotically, in the large users limit, showing how the network resources required to sustain P2P-TV traffic scale with the number of peers joining the streaming application, while considering at the same time the growth of the physical network infrastructure. Our findings show that a careful design of the applicationlayer distribution topology, taking into account network-layer metrics, can drastically reduce the impact of P2P-TV traffic on the Internet infrastructure. Actually, the gain of network-aware solutions increases almost linearly as the population of TV watchers increases. However, na¨ıve approaches to optimize the formation of the overlay topology can incur excessive delays in very large systems, making it crucial to devise intelligent, cooperative schemes which have not yet been incorporated in current systems. II. N ETWORK

SCENARIO

We represent the Internet topology at the router level adopting the simplified model illustrated in Figure 1. We isolate last-tier ISPs (also referred to as access ISPs) from the core

2

Fig. 1.

Model of the Internet topology at router level.

network, which is modelled as an undirected graph G(V, E). Last-tier ISPs, which include residential ISPs, corporate LANs, etc., are connected to the Internet core by leased lines contracted by higher-tier ISP’s, linking a gateway node belonging to the last-tier ISP to an edge router belonging to the core Internet infrastructure, see Figure 1. The costly bandwidth allocated over these peripheral links, which represents the main concern of last-tier ISPs, can easily constitute a bottleneck for the performance of poorly engineered P2P-TV systems. To this aim we separately compute the costs in charge of lasttier ISPs, and the overall cost incurred by the core Internet infrastructure. On the other hand, we assume that the cost of the internal, private network of an access ISP is negligible as compared to the cost of the link connecting it to the rest of Internet. This reflects the fact that inexpensive, high-speed technologies such as Gigabit Ethernet allow to over-provision the internal networks of last-tier ISPs, which are thus unlikely to become the bottleneck of P2P-TV systems. The performance of P2P applications is, instead, likely to be affected by the limited bandwidth on the access lines of end-users subscribed to residential ISPs, however we do not consider the role of such access lines here because this has already been done in several papers, while our novel focus here is on the bandwidth consumption incurred by the core Internet infrastructure. Notice that the increasing availability of broadband Internet access is expected to shift the system bottleneck from the last mile to other network segments in the near future, justifying our analysis which ignores the limitations of user access lines. We next introduce the notation that we use to describe the above network scenario, and the fundamental metrics which are the objective of our analysis. A. Notation of asymptotic analysis The number of routers in the core network is |V | = n. We assume that there are m users concurrently tuned to the same TV channel broadcasted over the Internet. We observe that any chunk or sub-stream belonging to a TV channel is delivered to the end-users over a distribution tree rooted at the source node. Thus, although we restrict our analysis to an individual chunk/sub-stream distributed over a tree-like topology, our analysis also applies to meshed P2P architectures where individual chunks/sub-stream are propagated over distinct distribution trees. The source of the stream is assumed to be located inside the network of an access ISP, just like any other user.

We denote by e the number of core routers to which the chuck/sub-stream has to be delivered. This corresponds to the number of edge routers whose attached access ISP contains at least one user (or the source). In the following we will refer to these routers with the term of relevant edge routers. Let p = e/n be the fraction of relevant edge routers within the network core. In our asymptotic analysis1 , we let n go to infinity (the Internet is growing!). Also the number m of users interested in watching a TV channel goes to infinity, and indeed it is reasonable to assume that this number grows much faster than n, although it is initially small when a new P2P-TV system is made available. The number of edge routers that have to be reached grows accordingly, and this effect can be modeled by playing on p. In our asymptotic analysis we are mainly interested in the case where e = Θ(n), hence we assume that p is a constant. This means that we assume that relevant edge routers represent a finite fractions of all Internet routers. The fact that m is expected to grow much faster than n is modeled assuming that m grows as nα , with α > 1. At last, we emphasize that the way in which the m peers are distributed among the networks attached to the relevant edge routers is not important in our analysis, as long as the number of peers belonging to the same access network is negligible with respect to the total number of peers. This means that we assume that two randomly selected peers belong, with high probability, to different access networks. B. Metrics We normalize to one the bandwidth required to transmit a chunk/sub-stream over one link, and suppose that this cost is the same for all links belonging to the core Internet infrastructure (i.e., links between two nodes in V ) and for all peripheral links connecting gateways nodes of access ISP to their corresponding edge router in V . The cost within the network of an access ISP is instead supposed to be zero. Our goal is to separately compute the aggregate bandwidth needed in the core network and the aggregate bandwidth needed over all peripheral links, comparing the effectiveness of different architectural solutions. More specifically, we consider the following three schemes: IP-multicast: this represents the optimal solution in which multicast occurs at the IP level, so that information branch can take place at any Internet router. No active contribution to the information dissemination is required by users which are leaves of the distribution tree. Notice that links within the core network are traversed at most once. Similarly, peripheral links are traversed only once in the downlink direction2, except for the last-tier ISP where the source is located, whose access link is traversed uplink (again, only once). We denote by I(n, e, m) the aggregate cost of the optimal IP multicast solution within ˆ e, m) the corresponding cost the core network, and by I(n, over the peripheral links. Optimized-P2P: according to this solution the multicast tree is created at the application layer, hence information branch can take place only at peers, not at routers. Users participating to the distribution tree are forced to act as relay 1 Given two functions f (n) ≥ 0 and g(n) ≥ 0: f (n) = o(g(n)) means limn→∞ f (n)/g(n) = 0; f (n) = O(g(n)) means lim supn→∞ f (n)/g(n) = c < ∞; f (n) = ω(g(n)) is equivalent to g(n) = o(f (n)); f (n) = Ω(g(n)) is equivalent to g(n) = O(f (n)); f (n) = Θ(g(n)) means f (n) = O(g(n)) and g(n) = O(f (n)); at last f (n) ∼ g(n) means limn→∞ f (n)/g(n) = 1. 2 We call downlink the direction from the Internet core to the users, and uplink the reverse direction.

3

for other users. The minimum-cost tree is built assuming perfect knowledge of the underlying physical topology. Notice that, differently from the IP multicast solution, underlying physical links can be traversed more than once. We denote by O(n, e, m) the aggregate cost of the optimized P2P solution ˆ e, m) the corresponding within the core network, and by O(n, cost over the peripheral links. Unaware-P2P: according to this solution, the multicast tree forms again at the application layer, but without any knowledge of the underlying physical topology. The distribution tree is created incrementally, attaching each new peer to a randomly chosen peer which is already in the tree (or directly to the source). A constraint can be put to the number of peers attached to the same peer (fan out) to avoid saturating its uplink bandwidth. Such constraint is not important in our asymptotic analysis. We denote by U(n, e, m) the aggregate cost of the unaware P2P solution within the core network, and ˆ e, m) the corresponding cost over the peripheral links. by U(n, The goal of our asymptotic analysis is to understand how ˆ O, O, ˆ U, Uˆ scale with n (and m) under different I, I, assumptions about the structure of the Internet core graph G(V, E). Such analysis is done in Section III. Moreover, since practical P2P solutions lies in between the two extreme cases Optimized-P2P and Unaware-P2P, we are also interested to understand how difficult it is to optimize the construction of the P2P distribution tree by distributed algorithms where peers discover each other in an incremental fashion. In Section IV we propose a novel fluid model to analyze the transient behavior of proximity-aware algorithms to construct the overlay topology. C. Related Work We limit ourselves to mentioning the papers more closely related to our technical contributions. The benefits of the IP-multicast solution when compared to the simple unicast solution have been extensively studied in the past since the empirical Chuang-Sirbu power-law [8] was proposed, which states that for several types of graphs I(n, e, m) scales approximately as E[HG ]e0.8 , being E[HG ] the average distance on the graph (i.e., the cost of a unicast path from the source to a random receiver). The work in [9] was the first to analytically explain the Chuang-Sirbu power-law, focusing on the case of k-ary trees (with the source placed at the root), for which the authors have derived an approximate asymptotic formula later on confirmed rigorously in [10]. The authors of [11] have also critically discussed the range of validity of the Chuang-Sirbu law, deriving useful bounds valid for general graphs as well as asymptotic results for Erd˝os-R´enyi random graphs and k-ary trees. The asymptotic performance of the Optimized-P2P solution has been studied, to the best of our knowledge, only by [12], in the case of a k-ary tree. In this paper we propose a different approach to compute O(n, e, m) for a k-ary tree, obtaining different results with respect to those reported in [12]. Moreover, we consider other types of graphs. We emphasize that previous works [9]–[12] have focused on the scenario in which the number of multicast users m coincides with the number of nodes e (i.e., routers) to be reached in the underlying graph of the physical network topology. Hence, they have limited their analysis to the case in which m = O(n) (actually, the Chuang-Sirbu law can be applied only when m is negligible as compared to n). Instead, due to the expected high penetration of IPTV applications, we argue that the most likely scenario will be that in which

m = ω(n), i.e. the case in which the number of TV watchers scale at least as fast as the number of IP routers, with an increasing number of users attached to the same edge router. This makes a fundamental difference when the asymptotic performance of peer-assisted solutions is considered, as done in the present work. In [13] the authors have proposed a first attempt to analyze the impact of proximity-aware solutions on the scalability of P2P-TV applications. They model the network topology as a simple multi-dimensional sphere in which the source is located in the center. This approximation, which maps the Internet graph onto a continuous domain, on the one hand allows the authors to obtain rather elegant closed form expressions for the main performance indexes, on the other hand it significantly impact the obtained results. Furthermore, no asymptotic results are derived when the number of peers (and the number of routers) tends to infinity. III. A NALYSIS A. General Graphs We start considering general graphs, deriving weak yet significant bounds to I, O and U. Moreover, we derive exact ˆ O, ˆ U. ˆ expressions for I, We first observe that the presence of several users in the same access network connected to one relevant edge router can be neglected in the analysis of I and O. Hence both I(n, e, m) and O(n, e, m) do not actually depend on m. Indeed, as already noticed, according to the IPmulticast solution the peripheral link connecting the border gateway of an access network to the corresponding relevant edge router has to be traversed by data only once, at the some cost as if just a single user were connected to the border gateway. Then, information branching within the access network can be employed to deliver data to all users belonging to the same access ISP, without additional cost for the network infrastructure (we remind that we do not count resources employed within access networks). Similar considerations can be done for the OptimizedP2P solution because, also in this case, it is possible to distribute the stream from one peer to all other peers within the same access ISP exploiting only local links. This means that, in the evaluation of I and O, we can ignore the fact that several peers are connected to the same edge router. The problem can be formulated as follows. Given the graph G(V, E) representing the Internet core graph: • I(n, e, m) represents the cost of the minimum Steiner Tree joining the source router (i.e., the router connected to the source) to all other relevant edge routers. • O(n, e, m) represents the cost of the minimum Steiner Tree joining the source router to all other relevant edge routers, with the extra constrain that information branching is allowed only at the relevant edge routers. As immediate consequence of previous arguments we have that I(n, e, m) ≤ O(n, e, m), being both independent of m. In our analysis we assume that relevant edge routers (whose number e includes the one connected to the source) are selected randomly among the nodes of G(V, E). We emphasize that all metrics of interest to us are intended to be averaged among all possible selections of relevant edge routers. Then, as shown in [11], I(n, e, m) satisfies the following inequality: e − 1 ≤ I(n, e, m) < n

e−1 e

(1)

Instead, we can easily prove: Lemma 1: O(n, e, m) satisfies the following inequality e − 1 ≤ O(n, e, m) ≤ (e − 1)E[HG ]

(2)

4

being E[HG ] the average distance between two nodes on graph G(V, E). Proof: e − 1 ≤ O(n, e, m) immediately descends from (1) and the fact that I(n, e, m) ≤ O(n, e, m). Moreover, O(n, e, m) ≤ (e−1)E[HG ] because one possible way to build the desired Steiner Tree is to directly connect the source router to the other e − 1 edge routers. Paths from the source to the edge routers have an average length equal to E[HG ]. As immediate consequence of the above two results we have:   e−1 , (e − 1)E[HG ] I(n, e, m) ≤ min n e and: O(n, e, m) ≤ E[HG ] (3) I(n, e, m) On the other hand the average cost of the Unaware-P2P solution depends on m, because a network-unaware solution is not able to efficiently exploit the proximity of peers connected to the same edge router. Since each peer receives the stream from another peer selected independently of its distance on the graph, it simply follows that: U(n, e, m) = mE[HG ]

We observe that, in realistic graph models of the Internet topology, E[HG ] is either c log n or c log log n [15]. Thus, by looking at (3), the penalty paid by the Optimized-P2P approach with respect to the IP-multicast approach, even if potentially unbounded for n → ∞, increases very slowly with n. On the contrary, comparing U(n, e, m) and O(n, e, m), it results: U(n, e, m) m ≥ O(n, e, m)) e−1

and thus their ratio can potentially increase very fast as the population of TV watchers increases. For example, in the case m = Θ(nα ) (α > 1), and recalling that we assume (optimistically) that e = Θ(n), it follows that U/O = Θ(nα−1 ). Now turning our attention to the costs incurred by access ISP’s due to bandwidth consumption on the peripheral links connecting their networks to the Internet core, from the above arguments it descends immediately that: ˆ e, m) = O(n, ˆ e, m) = e I(n,

whereas

ˆ e, m) = m(1 − pL ) U(n, where pL is the probability that two randomly selected users belong to the same access ISP. As already mentioned in Section II-A, we consider this probability to be negligible. For example, if users were uniformly distributed among the relevant edge routers, we would have pL = 1/e, which tends ˆ is to zero as 1/n. In this case the ratio between Uˆ and O of the same order as the ratio between U and O, meaning that access ISP’s incur the same penalty as the Internet core due to the adoption of Unaware-P2P solutions with respect to Optimized-P2P solutions. In the following subsections we specialize our analysis to three classes of graphs: k-ary trees, Erd˝os-R´enyi random graphs and planar graphs, for which we are able to obtain stronger results. B. Trees In this subsection we assume G(V, E) to be a regular k-ary tree of depth D. The video source is connected to the root of the tree. The number of nodes in the tree, excluding the root, is K D+1 − k N = k + k2 + . . . + kD = k−1

For simplicity, we consider this time a probabilistic model in which each node is assumed to be a relevant edge router with probability p = e/m. This does not change asymptotic results with respect to the case in which there are exactly e relevant edge routers. The average cost of a unicast path from the root to a peer is D 1 X l D 1 u= + lk = D − N k − 1 kD − 1 l=1

Hence the average cost to reach all the involved users m directly from the source through unicast connections, denoted by U , is U (n, e, m) = m u. The cost of the Unaware-P2P solution slightly differs from U , in light of the fact that the average distance E[HG ] between two randomly selected nodes of the tree differs from u. To evaluate E[HG ], we consider a generic link at level l, and compute the probability that this link is used to connect two distinct nodes chosen uniformly in the tree. We observe that D−l+1 there are n(l) = 1 + k + . . . + k D−l = k k−1 −1 nodes below the link, and that the link is used only if the two nodes happen to be on
different sides with respect to the link. Since there are N2 ways to select two distinct nodes in the tree, among which nl (N −nl )/2 correspond to configurations in which the two nodes are on different sides of the link, we obtain   D X nl (N − nl ) l E[HG ] = k N (N − 1) l=1

It can be easily proved that u ≤ E[HG ] ≤ 2u, thus the cost of the Unaware-P2P solution U(n, e, m) = mE[HG ] is asymptotically equivalent to the cost U . The cost of the IP-multicast solution for a k-ary tree has been computed in [9]. We briefly recall the computation here, adapting it to our notation and assumptions. A generic link at level l is used if and only if at least one of the nodes in the subtree below the link corresponds to a relevant edge router, which occurs with probability 1 − (1 − p)n(l) . Considering that there are k l links at level l, the cost of the IP-multicast solution is I(n, e, m) =

D X l=1

i h k l 1 − (1 − p)n(l)

(4)

The authors of [9] have also proposed the following approximation of (4), which later on has been confirmed in a log p mathematically rigorously way in [10]: I ∼ N p (c − log k ), where c is some constant. We now evaluate the cost of the Optimized-P2P solution in the k-ary tree. This computation has been already attempted in [12], however the methodology adopted there is rather complex and does not lead to accurate results, as shown later. Our approach is based on the computation of the average number of links o(l) required to bring the signal to a relevant edge router at level l. The cost of the Optimized-P2P solution can then be expressed as O(n, e, m) =

D X

p k l o(l)

l=1

Before describing our analysis of o(l), we need to premise a simple observation on the structure of the delivery tree resulting from the Optimized-P2P solution. Figure 2 depicts a portion of a k-ary tree in the case of k = 2 (i.e., a binary tree).

5

root ... A B D F

C E

GH

I

...

...

L

s¯(l). Indeed, if we allow any node to receive the signal also from peers located at the same level as the node itself, we can create loops among the nodes located at a given level of the tree, which turn out to be all disconnected from the source. For example, in Figure 2, it can happen that node F relies on node G (at distance 2) to get the data, and at the same time node G selects node F as the peer providing the data to it. However this configuration does not work: at least one of the two nodes must be connected to a different node of the tree (in the example, G receives data from node E located one level above). From the above discussion we can conclude that, by allowing each node to connect to peers located at the same level as the node, we underestimate the global cost of the distribution tree for the Optimized-P2P solution. Therefore we obtain the lower bound:

Fig. 2. Illustration of Optimized P2P multicast in the case of a binary tree. Black nodes represent the relevant edge routers.

Black sites stands for relevant edge routers, and the dotted path represents the optimal data delivery tree. We notice that it is always the case that a relevant edge router at level l takes the signal from a node located at a level smaller than or equal to l (either the source or another peer). For example, in Figure 2 node G takes the signal from node E, while the opposite (E receives from G) cannot happen. To see this, suppose by contradiction that node n1 at level l1 receives the signal from node n2 at level l2 > l1 . Let n3 be the first common parent of n1 and n2 . If the signal arrives at n1 from n2 , it necessarily traverses twice the links on the path n3 − n1 (first downwards and then upwards) and once the links on the path n3 − n2 . This situation cannot be optimal, since we can alternatively bring the signal first from n3 to n1 and then from n1 to n2 , traversing fewer links with the same effect of bringing the signal to both n1 and n2 (and to all nodes in between). Moreover, to minimize the global cost, peer n2 receives the signal from the closest peer n1 that has already received the signal. To evaluate the average number of links o(l) required to bring the signal to a node at level l, we proceed as follows. First, we compute the number of nodes α(d) which are at a level smaller than or equal to l and are separated from the node by exactly d links. By inspecting the graph, we can see that: α(1) = 1, α(2) = α(3) = k, α(4) = α(5) = k 2 , and so on. In general, we have α(d) = k ⌊d/2⌋ . From α(d) we can compute also the distribution of the number A(d) of nodes separated by Pd a number of links less than or equal to d: A(d) = j=1 α(j). ⌈d/2⌉

⌊d/2⌋+1

+k −k−1 We have A(d) = k . The probability P (d) k−1 that the node is separated by more than d links from the closest peer at level smaller than or equal to l is equivalent to the probability P (d) that none among A(d) nodes correspond to a relevant edge router: A(d)

P (d) = (1 − p)

(5)

We are now able to compute the average number of links s¯(l) separating a node from the closest relevant edge router at level less than or equal to l. Considering that there is for sure a relevant edge router at distance l (i.e., the source), the seeked quantity can be expressed as s¯(l) =

l−1 X

P (d)

(6)

d=0

Unfortunately the average number of links o(l) required to bring the signal to a node at level l does not coincide with

Olower (n, e, m) = =

D X l=1

kl p

l−1 X

(1 − p)

D X

k l p s¯(l) =

l=1

k⌈d/2⌉ +k⌊d/2⌋+1 −k−1 k−1

(7)

d=0

To get an upper bound, we enforce all nodes to receive data only from nodes located at a higher level in tree. For example, in Figure 2 node L can be connected only to the nodes explicitly represented in the figure. Using this restrictive peer selection policy, we guarantee the connectivity of the distribution overlay but we overestimate the total number of links to use: Oupper (n, e, m) = k p +

D X l=2

k l p [1 + (1 − p)¯ s(l − 1)] (8)

In the above equation, the term k p accounts for the special case of nodes at level 1. The other term accounts for the nodes located at level l ≥ 2. The average cost required to deliver data to a node at level l ≥ 2 can be computed exploiting the function s¯ computed above. Indeed, it is equal to 1 (the cost of the link connecting the node to its parent node) plus the cost required to deliver the data to the parent node, if this does not correspond to a relevant edge router (with probability 1 − p). For small values of p, we can approximate P (d) ∼ 1 − p A(d), and ⌈j/2⌉ ∼ ⌊j/2⌋ ∼ j/2. Doing so, it is possible to carry out a closed form approximation of the lower bound (7): Olower (n, e, m) ∼ N p u −

(N p)2 (k 2 − 1) √ √ D+1 (9) (k k − 1)( k − 1) k √

which is valid for small values of p, and is quadratical in p. A similar approximation could be developed for (8). As expected, for small p the cost of the Optimized-P2P solution approaches the unicast cost N p u. For large values of p, we can instead develop an approximation of (7) assuming that each relevant edge router employs at most L links, where L is a small given number. This approximation makes sense when there so many relevant edge routers (p → 1) that it is unlikely to use more than L links to reach each node. Moreover, when L is small compared to D, we can assume than the average number of links used by a node is the same for all nodes, irrespective of the level, o(l) ∼ o¯, ∀l. Notice that this is equivalent to neglecting nodes receiving directly from the source. Then we can write

6

The cost O(n, e, m) is related to DG(V,E) (e, n) by the following result: Theorem 1: The cost O(n, e, m) of a graph is bounded by:

1.9 1.8 1.7

O(n, e, m) ≤ (e − 1)DG(V,E) (e, n)

1.6

O/I

1.5 1.4

Oupper Olower Olower: approx large p Olower: approx small p FK07

1.3 1.2 1.1 1 0.9 1e-06

Fig. 3.

1e-05

0.0001

0.001 p

0.01

0.1

1

The ratio O/I in the case of a binary tree of level D = 20.

Olower (n, e, m) ∼ n p o¯. Quantity o¯ can be expanded into a polynomial of q = 1 − p: o¯ = 1 +

L−1 X

q A(i)

i=1

For example, choosing L = 5, we get Olower (n, e, m) ∼ n (1 − q) ×

×(1 + q + q k+1 + q 2k+1 + q k

Proof: Set V ′ comprises the relevant edge routers. Initially, we put in V ′′ only the source router, denoted by s, i.e., V ′′ = {s}. Then by definition of critical distance there is w.h.p. at least one relevant edge router in V ′ − V ′′ , denoted by e1 , whose distance from s is at most DG(V,E) (k, n). We add e1 to V ′′ , and assign s as the parent node of e1 in the distribution tree. Then we can find at least one node e2 in V ′ − V ′′ whose distance from a node a2 in V ′′ is non greater than DG(V,E) (k, n). We let a2 be the parent node of e2 in the distribution tree, and add e2 to V ′′ . We iterate the same procedure until V ′′ = V ′ , obtaining a distribution tree whose cost is not greater than (e − 1)DG(V,E) (k, n). The problem, now, consists in asymptotically evaluating DG(V,E) (e, n) for Erd˝os-R´enyi graphs. To do so, we construct a class of auxiliary Erd˝os-R´enyi graphs Gd (e, p(d)) (notice that the number of nodes in such auxiliary graphs is e, not n), in which p(d) is the probability that the distance between two randomly chosen nodes in the original G(n, p0 ) graph is less than or equal to d, i.e., p(d) =

2

+2k+1

)

(10)

The approximation becomes better keeping more terms, i.e., for increasing values of L (L < D). Figure 3 shows the ratio between the costs of the Optimized-P2P and IP-multicast solutions in the case of k = 2, D = 20, for different values of p. We compare the upper bound (8) with the lower bound (7), together with the approximations of the lower bound for small values (9) and large values (10) of p. The plot also shows the curve labelled FK07 obtained according to the computation proposed in [12], which falls outside of the region in between our bounds3. At last, by looking at (7) and (8), it is easy to see that for any finite p, being s¯(l) bounded, we have O(n, e, m) = Θ(n). C. Erd˝os-R´enyi Random Graphs In this subsection we assume G(V, E) to be a classical Erd˝os-R´enyi random graph G(n, p0 )4 [16]. When an Erd˝os-R´enyi graph is almost surely connected, which occurs for p0 large enough, the average distance between two nodes, that we could plug in (2) to get an asymptotic bound to O(n, e, m), is E[HG ] = Θ(log n/ log p0 n). In this case an asymptotic refinement of (2) can be obtained, when both n and e jointly increase to infinite, introducing the concept of critical distance of the graph, denoted by DG(V,E)(n,e) . Definition 1: Given a connected graph G(V, E), let V ′ be a set of e arbitrarily selected nodes of V . We denote by DG(V,E) (e, n) the minimum distance value such that the nodes of V ′ can be connected together, w.h.p., using a set of paths of length non greater than DG(V,E) (e, n). From the above definition it follows that, for any set V ′′ ⊂ V ′ , there exists at least one node in V ′ − V ′′ whose distance from a node of V ′′ is not greater than DG(V,E) (e, n). 3 The code used to derive the curve FK07 was kindly provided by the authors of [12]. 4 In an Erd˝ os-R´enyi G(n, p0 ) graph, every pair of vertices is connected by an edge with probability p0 , independently of other edges.

N (d) n−1

being N (d) the average number of nodes whose distance is at most d form a randomly selected node in G(n, p0 ). For Pd i d i=1 (np0 ) (np0 ) ≪ n we can write p(d) ≃ . n−1 Let d∗ be the minimum value of d such that Gd (e, p(d)) is w.h.p. connected for e → ∞. Then we can claim that, asymptotically, DG(V,E) (e, n) = d∗ w.h.p., in light of the fact that, by construction, the generic auxiliary graph Gd (e, p(d)) can be regarded as a graph whose edges represent paths of length non greater than d interconnecting e randomly picked nodes in the original graph. Since for d = d∗ the graph Gd (e, p(d)) is, w.h.p., connected, by construction the e randomly selected nodes in the original graph can be connected by paths of length non greater than d∗ . To find an expression for d∗ , we resort on the property that any Erd˝os-R´enyi graph G(n, p0 ) is w.h.p. connected iff p0 = logn n + c(n), for any c(n) → ∞ when n → ∞ [16]. Thus, in our case, imposing: Pd (np0 )i (p0 n)d log e ≃ = + c(e) p(d) ≃ i=1 n−1 n e and solving with respect to d, we obtain: d∗ =

log( ne ) log(log e + c(e)) + log p0 n log p0 n

This implies that for Erd˝os-R´enyi graphs: log( ne ) log(log e + c(e)) O(n, e, m) ≤ + I(n, e, m) log p0 n log p0 n D. Planar Graphs Even if the Internet topology differs significantly from a planar graph, we consider this class of graphs here for two reasons. Firstly, planar graphs could well be used to evaluate the performance of IPTV applications running over large-scale wireless mesh networks (instead of the global Internet) as the

7

General graphs k-ary trees Erd˝os-R´enyi graphs Planar graphs

I(n, e, m) Θ(n) Θ(n) Θ(n) Θ(n)

O(n, e, m) O(nE[HG ]) Θ(n) O(n√ log log n) O(n log n)

U (n, e, m) Θ(mE[HG ]) Θ(m log n) Θ(m√ log n) Θ(m n)

TABLE I SUMMARY OF ASYMPTOTIC RESULTS FOR

e = Θ(n)

number of nodes increases; ii) Secondly, differently from the two types of graphs considered previously (trees and Erd˝osR´enyi), in planar graphs the average √ distance E[HG ] between two nodes can scale with n as fast as n, determining a significant difference between the upper bounds (1) and (2) derived in Section III-A for I(n, e, m) and O(n, e, m), respectively; thus the penalty of the Optimized-P2P solution with respect to the IP-multicast solution could be, in principle, important in the case of planar graphs. Nevertheless we are going to show that also in this√case the ratio between O(n, e, m) and I(n, e, m) is just O( log n). To simplify the analysis we consider graph G(V, E) √ a lattice √ that is a perfect square grid of size n× n, in which routers are placed at points of integer coordinates, with router at (i, j) being connected to routers at (i+1, j),(i−1, j),(i, j+1),(i, j− 1). However our approach can be extended to the case in which routers are randomly placed on the plane according to a Poisson process. Theorem 2: The cost of O(n, e, m) in the lattice graph is q n log n O(e ) w.h.p. when both n → ∞ and e → ∞. e Proof: We partition the network into boxes of side p n log n/e. Standard concentration results based on b = Chernoff bounds [17], allow to say that, uniformly over all boxes, the number of relevant edge routers in each box is w.h.p. Θ(log e). We randomly select one relevant edge router per box as pivotal, and connect all pivotal routers in the overlay topology according to a grid configuration. Finally we connect the non-pivotal relevant edge routers in each box to the pivotal router chosen in the same box. By so doing, we obtain an Optimized-P2P solution whose cost is O(e b), since by construction each pivotal router is connected to four other pivotal routers through paths of length less than or equal to 4 b, while non pivotal relevant edge routers are connected to a pivotal router through paths of length at most 2 b. On the other hand, we can select a subset of the above boxes, regularly spaced (one box out of four), in such a way that the minimum distance between two nodes belonging to different boxes is at least equal to b. Then we mark one relevant edge router in each box as pivotal, and observe than the distance between two pivotal routers is not shorter than b. Since all pivotal routers must be interconnected, the cost of the IP-multicast solution I(n, e, m) can not be smaller than the cost of the Euclidean Steiner Tree reaching the pivotal routers, which is Θ(e b/ log n). Therefore, Theorem q 3: The cost of I(n, e, m) in the lattice graph is n e Ω( log n n log ) w.h.p. when both n → ∞ and e → ∞. e E. Final remarks Table I summarises the asymptotic results obtained in previous sections when e = Θ(n). Note that for the three considered types of graphs the penalty factor incurred by the core Internet infrastructure to support the Optimized-P2P solution, as p compared to the optimal IP-multicast solution, scales as O( log n) (even in planar graphs where the average distance

√ is n). Furthermore in graphs where E[HG ] = O(log n) (which occurs in k-ary trees and Erd˝os-R´enyi graphs) the penalty factor either increases as slow as O(log log n) (for Erd˝os-R´enyi graphs) or is limited by a constant factor (for k-ary trees). On the contrary the penalty factor of the Unaware-P2P solution which ignores peers’ locations can increase as fast as G] Θ( mE[H ) with respect to the IP-multicast solution. n These trends clearly show that, on the one hand, the convenience of adopting a location-aware P2P solution with respect to a purely layered solution (completely unaware of the underlying physical infrastructure) can be huge, especially when m is significantly larger than n; on the other hand, our results confirm the effectiveness of the P2P approach for the distribution of TV contents, showing that the extra cost paid by a well engineered peer-assisted solution with respect to the native IP-multicast approach is rather limited, also when the population of TV watchers grows very large. IV. A NALYSIS

OF TOPOLOGY DYNAMICS IN

P2P-TV

In the previous section we have shown that peer-assisted solutions can be very effective to deliver TV channels to large numbers of users, as long as the delivery tree is built taking into account the underlying physical infrastructure. However optimizing the overlay topology may have a high cost in terms of complexity if done in a centralized fashion, since, in this case, it is necessary to localize all peers, and then run some algorithm to compute the minimum spanning tree among them. The formation of the overlay topology can be alternatively performed in a fully distributed fashion, employing some gossiping algorithm, provided that peers are able to estimate their distance from other peers they get in contact with. The simplest distributed scheme works as follows: peers receive from the system the addresses of other random peers concurrently tuned to the same TV channel. Then each peer, independently of others, monitors its own distance from newly encountered peers, continuously updating the best (closest) set of encountered peers from which to download data. This solution appears to be simple enough and in line with the mechanisms already proposed and implemented in P2P file-sharing systems such as Bit-Torrent [18]. One drawback of this approach is represented by the delay required to discover close-by peers participating to the same distribution tree. Such delay can be excessively large, especially if peers are selected at random from a very large set. To evaluate the transient behavior of this scheme, we have developed a simple fluid model that can account for the characteristics of the underlying physical topology. We focus on a given peer P , and describe the evolution over the time of its distance y(t) from the closest peer discovered up to time t. Let λ denote the rate at which new peers are discovered by P , through the gossiping algorithm. Let f (x) denote the probability density function of the distance between P and other peers belonging to the same peer-to-peer system, and F (y) be the associated cumulative distribution function. This distribution essentially depends on the type of graph used to model the underlying physical topology. The evolution of y(t) can be described by the following differential equation: dy(t) = −λF (y(t))(y(t) − E[x|x ≤ y(t)]) dt R

y(t)

xf (x) dx

with E[x|x ≤ y(t)] = 0 F (y(t)) . This because, given y(t), in the interval [t, t + ∆t) peer P discovers a closer

8

λ dy(t) = (1 − ay(t) ) dt e whose analytical solution can be found exploiting the transformation u(t) = a−y(t) , which allows to reduce the above differential equation to a standard linear equation. It turns out: h i λ y(t) = loga 1 + (y(0) − 1)a− e t We assume that at the beginning (t = 0), the peer is connected log n to a randomly selected node, hence y(0) = E[HG ] ≃ log keff . ∗ Defining with t (Y ) the time taken by y(t) to go below an assigned finite treeshold Y , it turns out that for any Y strictly larger than 0, t∗ (Y ) = Θ( λe log log n). We conclude that the rate λ at which new peers are discovered must be carefully designed to avoid excessive delays before a bandwidth-effective distribution tree is constructed. In particular, the condition that t∗ (Y ) is independent from system parameters implies that λ increases as e log log n. As a consequence, the bandwidth required for signalling may become huge, especially for very large systems. Faster convergence times can be obtained by adding some intelligence to the gossiping algorithm, however the analysis of these improved schemes is beyond the scope of this paper. V. S IMULATION RESULTS We conclude presenting a set of simulation results obtained considering a real snapshot of the router-level Internet graph. In particular, we have used the measurements data collected at CAIDA during the period April 21–May 8, 2003 by the Macroscopic Internet Topology Data Kit (ITDK) project [19]. The available dataset contains the adjacency matrix of 190,914 routers, that we have used as the core graph G(V, E). The purpose of our simulations was not just to adopt a realistic representation of the Internet topology, but also to derive performance figures that cannot be easily evaluated analytically. More specifically, we have evaluated the performance of the Optimized-P2P and Unaware-P2P solutions in the case in which relevant edge routers are not uniformly selected among all nodes of the graphs, but are chosen considering the connectivity degree of the nodes. We will call

0.012 0.01 0.008 O/U

peer with probability λF (y(t))∆t; if this happens, on average the new closest distance is E[x|x ≤ y(t)], hence the rate at which y(t) diminishes during ∆t is equal to λF (y(t))(y(t) − E[x|x ≤ y(t)]). The previous equation can be analitically solved for several distance distributions. The most interesting case is that of graphs G(V, E) which exhibit locally a tree structure, because this property has been found to hold in practice in the Internet graph [14]. In this case the average number of nodes n(d) at distance d from a given node increases geometrically with d. In particular, we will consider generalized random graphs in which the degree of each node independently follows a generic distribution having first and second moments equal to < k > d−1 and < k 2 >, respectively. In this case n(d) = keff < k >,
− being keff = [15]. Since relevant edge routers are randomly placed within graph G(V, E), also the average number e(d) of relevant edge routers at distance d increases geometrically with d according to the law: e(d) = p n(d). x When n and e are sufficiently large, we have f (x) ≃ aC Rd for x ∈ [0, dmax ], where a = keff , C = 0 max ax dx is n the normalization constant and dmax = log log a . Plugging this distribution in the previous differential equation we obtain:

0.006 0.004 power-law, low-degree uniform, low-degree power-law, high-degree uniform, high-degree

0.002 0 0.0001

0.001

0.01

0.1

p

Fig. 4. Relative cost of the optimal solution with respect to the unaware solution, O(n, e, m)/U (n, e, m).

low-degree (high-degree) scenario the case in which relevant edge routers are selected only among the 20% of the nodes with the lowest (highest) degree. Actually, in the considered graph the 20% less-connected nodes have degree 1, whereas the 20% most-connected nodes have degree larger than or equal to 8. Moreover, we have considered two distributions for the number of peers connected to the same edge router: the uniform case corresponds to a uniform distribution between 50 and 150; the power-law case corresponds to a Pareto −α distribution with pdf: pk = P ∞ k k−α , (k ≥ kmin ), where k=kmin we have chosen α = 3 and kmin = 50 so as to get the same average value as in the uniform case, equal to 100. Besides the aggregate costs O(n, e, m) and U(n, e, m), we have also derived the stress distribution on individual networks element, so as to evaluate the unbalance of the traffic in the network. We define by link (router) stress the number of times that a link (router) is traversed by the same data stream. Both aggregate costs and stress distributions have been computed averaging the results of 30 simulation runs, in which we have changed the selection of relevant edge routers (including the source), as well as the number of peers attached to them. Figure 4 reports the ratio O(n, e, m)/U(n, e, m), as function of p = e/n, for the 4 combinations of cases described above (we also report 95%-level convergence intervals). As expected, the ratio is about m/e = 100, and decreases for increasing values of p in all of the scenarios. This means that the relative benefit of adopting an Optimized-P2P solution improves as the popularity of the application increases (similarly to the IP-multicast solution). Moreover, the distribution of peers in the access ISPs has marginal impact on results (notice the confidence intervals, whose width is higher for small values of p). The criterion for the selection of relevant edge routers, instead, has some effect on the relative cost between the two solutions. We argue that this is due to the fact that, in the considered topology, nodes with lower degree are topologically more peripheral, reducing the benefit of the Optimized-P2P solution with respect to the UnawareP2P solution. Figures 5 and 6 report the Complementary Cumulative Distribution Function (CCDF) of the link stress for p = 0.01, considering both low-degree and high-degree scenarios. The uniform case is reported in Figure 5, whereas the power-law case is reported in Figure 6. As expected the gain achieved by the Optimized-P2P solution over the IP-multicast solution reflects itself into a much narrow link stress distribution. Actually we observe several orders of magnitude difference when the most heavily loaded links are compared. Note that the maximum link stress under the IP-multicast solution

9

1

as for the link stress can be applied here. By optimizing the construction of the P2P overlay, it is possible to dramatically reduce the amount of data to be handled by individual routers.

0.1

CCDF

0.01

0.001

0.0001 Optimized-P2P, high-degree Optimized-P2P, low-degree Unaware-P2P, high-degree Unaware-P2P, low-degree

1e-05

1e-06 1

10

100

1000

Link stress

Fig. 5.

Link stress distribution for the uniform case; p=0.01 1

0.1

CCDF

0.01

0.001

0.0001 Optimized-P2P, high-degree Optimized-P2P, low-degree Unaware-P2P, high-degree Unaware-P2P, low-degree

1e-05

1e-06 1

Fig. 6.

10

100 Link stress

1000

10000

Link stress distribution for the power-law case; p=0.01

would be equal to 1. The Optimized-P2P solution is not as good at reducing the link stress, yet the majority of links are traversed only a few times (say less than 10). Comparing the low-degree and high-degree scenarios, we notice again the impact of the fact that low-degree nodes are more peripheral, resulting into larger peer-to-peer distances and consequently higher link stress. Interestingly, the Unaware-P2P solution in the uniform case results into a link stress distribution which drops sharply after link stress equal to 150, especially in the low-degree case. This is due to the dominant contribution of links connecting the relevant edge routers with their closest neighboors, which are traversed (in both directions) a number of times closely related to the uniform distribution of peers (between 50-150) attached to the same edge router. This sharp drop at 150 disappears in the power-law case, for the same reason. At last in Figure 7 we report the CCDF of the router stress for p = 0.01, in the uniform case. Similar considerations

1

0.1

CCDF

0.01

0.001

0.0001 Optimized-P2P, high-degree Optimized-P2P, low-degree Unaware-P2P, high-degree Unaware-P2P, low-degree

1e-05

1e-06 1

Fig. 7.

10

100 Router stress

1000

10000

Router stress distribution for the uniform case; p=0.01

VI. C ONCLUSION In this paper we have evaluated asymptotic scale effects of massively-deployed IPTV applications, focusing our attention on the costs incurred by the core network infrastructure. In thin manner we have addressed the concerns raised by many Internet Service Providers and backbone operators about the feasibility of large-scale P2P video broadcasting over the Internet. We have compared different architectural solutions, ranging from the IP multicast approach to the two extreme cases of peer-assisted solutions, in which the overlay topology is either optimized with perfect knowledge of the underlying physical topology, or built according to a network-unaware approach. Our results clearly indicate the huge convenience of adopting a location-aware, peer-assisted approach, as this strategy essentially achieves the same performance as the (optimal) IP-multicast solution, also when the population of TV watchers grows very large. On the contrary, poorly engineered peer-assisted solution do not scale well with the number of users, and can potentially place an intolerable strain on the core network infrastructure. R EFERENCES [1] PPStream, http://www.PPStream.com; PPLive, http://www.pplive.com; SOPCast, http://www.sopcast.com; TVAnts, http://www.tvants.com; Joost, http://www.joost.com; Babelgum, http://www.babelgum.com; Zattoo, http://www.zattoo.com; Tvunetworks, http://www.tvunetworks.com; [2] A. Murray-Watson, “Internet groups warn BBC over iPlayer plans,” The Independent, http://www.independent.co.uk/news/business/news/ internet-groups-warn-bbc-over-iplayer-plans-461167.html, posted on 12 August 2007. [3] M. Hefeeda, A. Habib, B. Botev, D. Xu, B. Bhargava, “PROMISE: peerto-peer media streaming using CollectCast”, ACM Multimedia, Berkeley, CA, November 2003. [4] X. Liao, H. Jin, Y. Liu, L.M. Ni, D. Deng, “AnySee: Peer-to-Peer Live Streaming”, IEEE INFOCOM, Barelona, Spain. April 2006. [5] M. Castro, P. Druschel, Y. C. Hu, and A. Rowstron. “Exploiting network proximity in peer-to-peer overlay networks,” Tech. Rep. MSR-TR-200282, Microsoft Research, 2002. [6] J.W. Byers, J. Considine, M. Mitzenmacher, S. Rost, “Informed content delivery across adaptive overlay networks,” IEEE/ACM Transactions on Networking Vol. 12, n 5, pp 767- 780, October 2004 [7] A. Horvarth, M. Telek, D. Rossi, P. Veglia, D. Ciullo, M. A. Garcia, E. Leonardi, M. Mellia, “Dissecting PPLive, SopCast, TVAnts”, submitted for publication to ACM CoNext 2008. [8] J. Chuang and M. Sirbu, “Pricing multicast communications: A costbased approach,” In Proceedings of the INET ’98, 1998. [9] G. Phillips, S. Shenker, H. Tangmunarunkit, “Scaling of Multicast Trees: Comments on the Chuang-Sirbu scaling law”, SIGCOMM ’99. [10] C. Adjih, L. Georgiadis, P. Jacquet, W. Szpankowski, “Multicast tree structure and the power law,” IEEE Trans. on Infor. Theory, Vol. 52, No. 4, 2006. [11] P. Van Mieghem, G. Hooghiemstra, R. van der Hofstad, “On the Efficiency of Multicast” IEEE/ACM Trans. on Networking, Vol 9 n. 6, 2001. [12] S. Fahmy and M. Kwon, “Characterizing overlay multicast networks and their costs,” IEEE/ACM Trans. on Networking, Vol. 15, no. 2, 2007. [13] L. Dai, Y. Cui, Y. Xue, “On Scalability of Proximity-Aware Peer-to-Peer Streaming”, INFOCOM 2007, Anchorage, Alaska, May 2007. [14] M. Faloutsos, P. Faloutsos, C, Faloutsos, “On power-law relationships of the Internet topology” ACM SIGCOMM, Cambridge, MA, 1999. [15] R. Pastor-Satorras, A. Vespignani, Evolution and Structure of the Internet: A Statistical Physics Approach, Cambridge University Press, 2004. [16] B. Bollob´as, Random Graphs, Cambridge University Press, 2001. [17] R. Motwani, P. Raghavan, Randomized Algorithms, Cambridge University Press, 1995. [18] B. Cohen. BitTorrent protocol specification. In First Workshop on Economics of Peer-to-Peer Systems (P2P ’03). [19] CAIDA Macroscopic IP Topology Data Kit. Kit #0304: Apr 21 to May 8, 2003. http://www.caida.org/tools/measurement/skitter/idkdata.xml.