Continuous Random Walks: Optimal Round and Message Complexity? Atish Das Sarma

Gopal Pandurangan

Anisur Rahaman

Google Research Google Inc., Mountain View, USA Email: [email protected]

Nanyang Technological University Singapore Email: [email protected]

Nanyang Technological University Email: [email protected]

Abstract—TO CHANGE BELOW This paper concentrate on the problem of multiple random walks in a distributed network. We extend the work [6] from computing single random walk to multiple random walks. In the work [6], they did not consider message complexity - the total number of messages exchanged and their algorithm is expensive in this aspect. Here we show that the average messages used for single random walk is much less than them. In their paper they compute η short walks for each nodes of the network and then stitch them together to compute a random walk of length `. Now the basic question is how many walks of length ` can be compute using the same set of short length walks. In other words, how many short length walks we can use for multiple random walks. And we show that a good amount of short walks are going used, infact we show a constant fraction of short walks are going used. We focus on the problem of performing random walks efficiently in a distributed network. Given bandwidth constraints, the goal is to minimize the number of rounds required to obtain a random walk sample. Despite the widespread use of random walks in distributed computing theory and practice for long, most algorithms that compute a random walk sample of length ` always do so naively, i.e., in O(`) rounds. Recently, a significantly faster sublinear time distributed algorithm was presented that ˜ 2/3 D1/3 ) rounds 1 where D is the diameter of the ran in O(` network [5]. This was the first result to improve beyond linear time (in `) despite the sequential nature of random walks. √ This ˜ `D) is work further conjectured that a running time of O( possible and that this is essentially optimal. In this paper, we resolve these conjectures and show almost tight bounds on the time complexity of distributed random walks. We present a fast distributed algorithm for performing random √ walks. Our algorithm performs a random walk of length ˜ `D) rounds on an undirected network, where D is ` in O( the diameter of the network. We then show that there is a fundamental difficulty in improving theqdependence on ` any further by proving a lower bound of Ω( log` ` + D) rounds for performing a walk of length `. This shows that our algorithm is optimal (up to polylogarithmic factors and the dependence on D). We further extend our algorithms to perform k independent √ ˜ k`D + k) rounds. Our techniques random walks in roughly O( can be useful in speeding up distributed algorithms for a variety of applications that use random walks as a subroutine. We illustrate one such application involving the decentralized computation of mixing time, a key global parameter of the underlying network. Our algorithms are fully decentralized and 1 Throughout this paper, O ˜ hides polylogarithmic factors in the number of nodes in the network

can serve as building blocks in the design of topologically-aware networks.

I. I NTRODUCTION TO CHANGE BELOW Random walks play a central role in computer science, spanning a wide range of areas in both theory and practice. The focus of this paper is random walks in networks, in particular, decentralized algorithms for performing random walks in arbitrary networks. Random walks are used as an integral subroutine in a wide variety of network applications ranging from token management and load balancing to search, routing, information propagation and gathering, network topology construction and building random spanning trees (e.g., see [5] and the references therein). Random walks are also very useful in providing uniform and efficient solutions to distributed control of dynamic networks [2], [10]. Random walks are local and lightweight and require little index or state maintenance which make them especially attractive to selforganizing dynamic networks such as Internet overlay and ad hoc wireless networks. A key purpose of random walks in many of these network applications is to perform node sampling. While the sampling requirements in different applications vary, whenever a true sample is required from a random walk of certain steps, all applications perform the walks naively — by simply passing a token from one node to its neighbor: thus to perform a random walk of length ` takes time linear in `. In this paper, we present fast, almost optimal, sublinear time distributed random walk sampling algorithms that are significantly faster than the existing ones. We then present matching lower bounds on the running time of such algorithms and demonstrate that there are fundamental limitations on the speedup that one can achieve. Finally, we apply our random walk algorithms to devise efficient decentralized algorithms for computing key global metrics of the underlying network such as mixing time, spectral gap, and conductance. Such algorithms can be useful building blocks in the design of topologically (self-)aware networks, i.e., networks that can monitor and regulate themselves in a decentralized fashion. For example, efficiently computing the mixing time or the spectral gap, allows the network to monitor connectivity and expansion properties of the network.

A. Distributed Computing Model Consider an undirected, unweighted, connected n-node graph G = (V, E). Suppose that every node (vertex) hosts a processor with unbounded computational power, but with limited initial knowledge. Specifically, assume that each node is associated with a distinct identity number from the set {1, 2, ..., n}. At the beginning of the computation, each node v accepts as input its own identity number and the identity numbers of its neighbors in G. The node may also accept some additional inputs as specified by the problem at hand. The nodes are allowed to communicate through the edges of the graph G. The communication is synchronous, and occurs in discrete pulses, called rounds. In particular, all the nodes wake up simultaneously at the beginning of round 1, and from this point on the nodes always know the number of the current round. In each round each node v is allowed to send an arbitrary message of size O(log n) through each edge e = (v, u) that is adjacent to v, and the message will arrive to u at the end of the current round. This is a standard model of distributed computation known as the CONGEST model [8] and has been attracting a lot of research attention during last two decades (e.g., see [8] and the references therein). A critical component MESSAGE COMPLEXITY - WRITE MORE B. Problem Statement and Related Work TO CHANGE BELOW The basic problem we address is the following. We are given an arbitrary undirected, unweighted, and connected n–node network G = (V, E) and a node s ∈ V . The goal is to devise a distributed algorithm such that, in the end, s outputs the ID of a node v which is randomly picked according to the probability that it is the destination of a random walk of length ` starting at s. Throughout this paper, we assume the standard random walk: in each step, an edge is taken from the current node x with probability proportional to 1/d(x) where d(x) is the degree of x. Our goal is to output a true random sample from the `-walk distribution starting from s. For clarity, observe that the following naive algorithm solves the above problem in O(`) rounds: The walk of length ` is performed by sending a token for ` steps, picking a random neighbor with each step. Then, the destination node v of this walk sends its ID back (along the same path) to the source for output. Our goal is to perform such sampling with significantly less number of rounds. This problem was proposed in [5] under the name Computing One Random Walk where Source Outputs Destination (1-RW-SoD) (for short, this problem will be simply called Single Random Walk in this paper), wherein the first sublinear time distributed ˜ 2/3 D1/3 ) rounds (O ˜ algorithm was provided, requiring O(` hides polylog(n) factors); this improves over the naive O(`) algorithm when the walk is long compared to the diameter (i.e., ` = Ω(D polylog n) where D is the diameter of the network). This was the first result to break past the inherent sequential nature of random walks and beat the naive ` round

approach, despite the fact that random walks have been used in distributed networks for long and in a wide variety of applications. It was further conjectured √ in [5] that the true ˜ `D). number of rounds for this problem is O( ˜ 2/3 D1/3 )-round algoThe high-level idea used in the O(` rithm in [5] is to “prepare” a few short walks in the beginning (executed in parallel) and then carefully stitch these walks together later as necessary. The same general approach was introduced in [4] to find random walks in data streams with the main motivation of finding PageRank. However, the two models have very different constraints and motivations and hence the subsequent techniques used in [5] and [4] are very different. Our algorithms in this paper use the same general approach as [5] but exploit certain key properties of random walks to design even faster sublinear time algorithms. Recently, Sami and Twigg [9] consider lower bounds on the communication complexity of computing stationary distribution of random walks in a network. Although, their problem is related to our problem, the lower bounds obtained do not imply anything in our setting. Other recent works involving multiple random walks in different settings include Alon et. al. [1], and Cooper et al. [3]. WRITE ABOUT PODC 2010 paper [6] and PODC 2011 [7] papers WRITE ABOUT THE IMPORTANCE OF CONTINUOUS PROCESSING/SERVING AND CITE ANY RELEVANT REFERENCES. II. T HEORETICAL A NALYSIS OF A LGORITHMS A. Algorithm descriptions We first describe the algorithm for single random walk in [6] and then describe how to extend this idea for continuous random walks. The current algorithm is also randomized and we focus more on the message complexity. The high-level idea for single random walk is to perform many short random walks in parallel and later stitch them together in clever way [6]. Then for multiple random walks we choose the source node randomly each time and perform single random walk using the same set of short length walks. The main algorithm for performing continuous random walk each of length ` is described in C ONTINUOUS -R ANDOM WALK(cf. Algorithm 4). This algorithm uses other algorithm S INGLE -R ANDOM -WALK(cf. 2). The algorithm S INGLE R ANDOM -WALK uses P RE -P ROCESSING(cf. 1)and S AMPLE D ESTINATION(cf. 3). The P RE -P ROCESSING function is called only one time throughout the process and it creates a sample table of short length walks. B. Previous Results - Rounds and Messages Theorem II.1. For any `, Algorithm S INGLE -R ANDOM WALK (cf. Theorem 2.5 in [6]) solves 1-RW-DoS (the Single Random Walk Problem) and,
with probability at least 1 − n2 , finishes in O λη log n + `D rounds. λ Theorem II.2. The message
complexity of S INGLE -R ANDOM WALK is O ηλm + `D where m is number of edges and D λ is the diameter of the network.

Proof: For message complexity we are not bothering about number of rounds it works. So for one step walk from any vertex u to its neighbor v, exactly one message needed.Therefore 1. For computing ηdeg(v) short walk of length λ it uses O(ληdeg(v)) messages. Since for a single short walk of lengthP λ it sends λ msgs and hence for n nodes it requires O(λη v deg(v)) = O(ληm) messages. 2. For constructing BFS tree and for S AMPLE -D ESTINATION it uses O(D) messages. So it uses O( `D λ ) msgs for a single walk of length `. Therefore total message
complexity for S INGLE -R ANDOM WALK is O ηλm + `D λ . C. Round Complexity Theorem II.3. For any `, Algorithm C ONTINUOUS R ANDOM -WALK (cf. Algorithm [?]) solves Continuous Random Walk Problem and, with probability at least 1− n2 , finishes in O (λη log n + κmηD) rounds where κ is fraction of used short length walk form the preprocessing table. Proof: The proof is same as Theorem 2.5 in [6] for Single Random Walk; the only difference is we are doing continuous walks of same length `. Therefore for Continuous Walks, if κ is fraction of used short length walk form the preprocessing table then total O(κmη) short walk used. Hence S AMPLE D ESTINATION 3 is invoked O(κmη) times (only when we stitch the walks) and therefore by Lemma 2.3 in [6], contributes O(κmηD) rounds. Hence total O (λη log n + κmηD) rounds. Theorem II.4. The average number of rounds per random walk of length `of C ONTINUOUS  -R ANDOM -WALK (cf. Algon κD rithm [?]) is O κ` ( log + ) with high probability. m λ Proof:  The total number of random walks of length ` is O κmηλ , as total O(κmη) short walks each of ` length λ. Hence the   average number of rounds per walk is n κD O κ` ( log + ) , (from above Theorem II.3). m λ D. Message Complexity Theorem II.5. The message complexity of C ONTINUOUS R ANDOM -WALK is O (ηλm + κmηD) where κ is fraction of used short length walk form the preprocessing table, ηv = ηdeg(v) is the number of short length walk for each node v, m is number of edges and D is the diameter of the network. Proof: For message complexity we are not bothering about number of rounds it works. So for one step walk from any vertex u to its neighbor v, exactly one message needed.Therefore 1. For computing ηdeg(v) short walk of length λ it uses O(ληdeg(v)) messages. Since for a single short walk of lengthP λ it sends λ msgs and hence for n nodes it requires O(λη v deg(v)) = O(ληm) messages. 2. For constructing BFS tree and for S AMPLE -D ESTINATION it uses O(D) messages. So it uses O(D`/λ) msgs for a

single walk of length `. Now the total number of short walks in the preprocessing table is 2mη, m is number of edges. If a constant factor(say κ) of the preprocessing table is used for multiple random walk each of length ` then total O(κmη) short walks used from the table and hence O(κmηD) messages used. Therefore total message complexity for C ONTINUOUS R ANDOM -WALK is O (ηλm + κmηD). Theorem II.6. The average number of messages per random walk of length ` of C ONTINUOUS -R ANDOM -WALK is
O κ` (1 + κD λ ) . Proof: From the above Theorem ?? we know that the total number of messages used for computing all walks of C ONTINUOUS -R ANDOM -WALK is O (ηλm+ κmηD). Now  κmηλ the total number of walks of length ` is O , as total ` O(κmη) short walks each of length λ. Hence
the average number of messages per walk is O κ` (1 + κD λ ) . Theorem II.7. The average number of rounds and messages per random walk of length ` of C ONTINUOUSn κD R ANDOM -WALK (cf. Algorithm [?]) is O κ` ( log m + λ )
and O κ` (1 + κD λ ) respectively, where λ is the length of short walks, κ is fraction of used short length walks and D is diameter. √ Corollary II.8. We choose λ = 24 `D(log n)3 then the aver-
˜ ` +`+D age rounds and
messages per walk becomes O κm ` ˜ and O κ + D respecttively. √ Proof: If we put λ = 24 `D(log n)3 inTheorem II.7 √  ˜ ` + `D then the average round and message becomes O κm  √  √ ˜ ` + `D respecttively. Now `D ≤ ` + D. So the and O κ corollary follows. E. What is κ? Conjecture or Theorem? III. E XPERIMENTS We have used the following three important models for experiments: • Regular Expander: We worked on the most commonly studied random graph denoted by G(n,p) in which every possible edge occurs independently with probability p. We choose p as logn/n which is very small. This is closed to a regular graphs with good expansion properties. • Two-tier topologies with clustering: First we construct four isolated regular expander that correspond to the clusters. Then from each cluster we pick a small number of nodes and connect them using another regular expander. • Power-law graphs: In distributed settings, many important networks are known to have power-laws. We have used the standard model of growth with preferential connectivity to generate power-law random graphs. • Random Geometric Graph: A random geometric graph is a random undirected graph drawn on a bounded region [0, 1) × [0, 1). It is generated by placing n vertices at

0.6 0.4 0.0 0

10000

20000

30000

40000

50000

0.6 0.2

Fraction of Short Walk Used

Regular Expander Power-law Graph 2-Tier Expander Random Geometric Grid Graph

0

10000

20000

30000

40000

50000

60000

ℓ

Fig. 2.

varying length of the walk `. n=10K, η = logn , λ =

C. Varying η

√ The default value of λ√is `. Here we varying λ from 0.25, 0.5, 1, 2, 4 multiple of `. And we see that more than 60% of rows uses here from the figure 5. For all the cases we found that for the power-law graph uses very small fraction of rows when sources chosen uniformly at

`. Source

0.0

The number √ of nodes n is varying form 1K, 2K, 5K and 10K; ` is n and other parameters are same as (a). Here we find that it uses more than 60% [figure 3] except power-law graph with source chosen uniformly. Also for G(n, p) and Two-tier, it uses much more than power-law.

D. Varying λ

√

0.8

Fig. 1. varying length of the walk `. n=10K, η = logn , λ = chosen randomly proportional to the degree

B. Varying n

From the figure 4 we see that the used fraction of rows is increasing with the number of short length walk η. Here we varying η from 0.25, 0.5, 1, 2, 4 multiple of logn. Actually for each node v, the number of short length walk is ηd(v) where d(v) is the degree of v.

60000

ℓ

A. Varying ` Here n is fixed √ at 10K and ` is varying from n0.5 , n0.6 , ..., n1.2 ; λ is ` and η is logn. In this case we see [figure 2] that at least 50% of the rows is used for all except the power-law graph when source chosen uniformly. Even better for G(n, p) and Two-tier topologies, it uses around 80%.

Regular Exapnder Power-law Graph 2-Tier Expander Random Geometric Grid Graph

0.2

Fraction of Short Walk Used

0.8

random. This is because of graph structure itself as the powerlaw distributions possess heavy tails. So choosing sources randomly proportional to their degrees is better than choosing uniformly at random for power-law graphs.

0.4

random uniformly and independently on the region and two vertices u, v are connected by an edge if and only if the distance q between them is at most a threshold r. We choose r as logn n so that the degree of each vertices is O(log n) w.h.p. √ • Grid Graph: Here we consider a square grid graph ( n× √ √n) which is a Cartesian product of two path graphs with n vertices. Since a path graph is a median graph, the square grid graph is also a median graph. All grid graphs are bipartite. We maintained a preprocessing table containing η = η 0 deg(v) short walks of length λ for each nodes. Then checking how many walk of length ` can be done using the table before getting stuck. We did experiments by varying many parameters like length of the walk(`), number of nodes(n), length of the short walk(λ), number of short walk (η). We worked on networks with 10K nodes when n is not varying. Since we are interested in how many random walk of length ` can be done using the preprocessing table, we plotted graph for each parameters `, n, λ, η corresponding to the fraction of rows is used in the preprocessing table. We draw graph by taking fraction used as y-axis and parameters as x-axis. We consider two ways to choose source node of the random walk of length `. One is choosing uniformly at random and another is choosing randomly proportional to their degrees. So each figure contains two line graph for each three above network models.

IV. C ONCLUDING R EMARKS

√

`

Regular Expander Power-law Graph 2-Tier Expander Random Geometric Grid Graph

0.0

0.0

0.2

0.4

Fraction of Short Walk Used

0.6

0.8

0.8 0.6 0.4 0.2

Fraction of Short Walk Used

Regular Expander Power-law Graph 2-Tier Expander Random Geometric Grid Graph

2000

4000

6000

8000

10000

100

200

300

400

n λ

varying number of nodes n. ` =n, η = logn , λ =

√

`

Fig. 5.

varying λ. n = 10K, ` =n, η = logn

avg. # message

6e+04

8e+04

1e+05

Regular Expander Power-law Graph 2-Tier Expander Random Geometric Grid Graph

1

2

3

0e+00

0.0

2e+04

4e+04

0.6 0.4

Regular Expander Power-law Graph 2-Tier Expander Random Geometric Grid Graph

0.2

Fraction of Short Walk Used

0.8

Fig. 3.

4

0

η

10000

20000

30000

40000

50000

60000

ℓ

Fig. 4.

varying η. n = 10K, ` =n, λ =

√

` Fig. 6.

varying length of the walk `. n=10K, η = logn , λ =

√

R EFERENCES

15000 10000

avg. # message

20000

Regular Expander Power-law Graph 2-Tier Expander Random Geometric Grid Graph

5000

[1] N. Alon, C. Avin, M. Kouck´y, G. Kozma, Z. Lotker, and M. R. Tuttle. Many random walks are faster than one. In SPAA, pages 119–128, 2008. [2] M. Bui, T. Bernard, D. Sohier, and A. Bui. Random walks in distributed computing: A survey. In IICS, pages 1–14, 2004. [3] C. Cooper, A. Frieze, and T. Radzik. Multiple random walks in random regular graphs. In Preprint, 2009. [4] A. Das Sarma, S. Gollapudi, and R. Panigrahy. Estimating pagerank on graph streams. In PODS, pages 69–78, 2008. [5] A. Das Sarma, D. Nanongkai, and G. Pandurangan. Fast distributed random walks. In PODC, 2009. [6] A. Das Sarma, D. Nanongkai, G. Pandurangan, and P. Tetali. Efficient distributed random walks with applications. In PODC, pages 201–210, 2010. [7] D. Nanongkai, A. Das Sarma, and G. Pandurangan. A tight unconditional lower bound on distributed randomwalk computation. In PODC, pages 257–266, 2011. [8] D. Peleg. Distributed computing: a locality-sensitive approach. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, 2000. [9] R. Sami and A. Twigg. Lower bounds for distributed markov chain problems. CoRR, abs/0810.5263, 2008. [10] M. Zhong and K. Shen. Random walk based node sampling in selforganizing networks. Operating Systems Review, 40(3):49–55, 2006.

2000

4000

6000

8000

10000

n

Fig. 7.

varying number of nodes n. ` =n, η = logn , λ =

√

`

`

30000 28000 26000 24000 22000 18000

20000

avg. # message

Regular Expander Power-law Graph 2-Tier Expander Random Geometric Grid Graph

1

2

3

4

Algorithm 1 P RE -P ROCESSING (η, `) Input: number of short walks of each node v is ηdeg(v), and desired walk length `. Output: set of short random walks of each nodes

η

varying η. n = 10K, ` =n, λ =

`

30000 20000

25000

Regular Expander Power-law Graph 2-Tier Expander Random Geometric Grid Graph

15000

avg. # message

35000

40000

Fig. 8.

√

100

200

300

400

λ

Fig. 9.

varying λ. n = 10K, ` =n, η = logn

Each node v performs ηv = η deg(v)) random walks of length λ + ri where ri (for each 1 ≤ i ≤ η) is chosen independently at random in the range [0, λ − 1]. 1: Let rmax = max1≤i≤η ri , the random numbers chosen independently for each of the ηx walks. 2: Each node x constructs ηx messages containing its ID and in addition, the i-th message contains the desired walk length of λ + ri . 3: for i = 1 to λ + rmax do 4: This is the i-th iteration. Each node v does the following: Consider each message M held by v and received in the (i−1)-th iteration (having current counter i−1). If the message M ’s desired walk length is at most i, then v stored the ID of the source (v is the desired destination). Else, v picks a neighbor u uniformly at random and forward M to u after incrementing its counter. {Note that any iteration could require more than 1 round.} 5: end for

Algorithm 3 S AMPLE -D ESTINATION (v) Input: Starting node v. Output: A node sampled from among the stored walks (of length in [λ, 2λ − 1]) from v. Algorithm 2 S INGLE -R ANDOM -WALK (s, `) Input: Starting node s, and desired walk length `. Output: Destination node of the walk outputs the ID of s. Stitch Θ(`/λ) walks, each of length in [λ, 2λ − 1] 1: The source node s creates a message called “token” which contains the ID of s 2: The algorithm generates a set of connectors, denoted by C, as follows. 3: Initialize C = {s} 4: while Length of walk completed is at most ` − 2λ do 5: Let v be the node that is currently holding the token. 6: v calls S AMPLE -D ESTINATION (v) and let v 0 be the returned value (which is a destination of an unused random walk starting at v of length between λ and 2λ − 1.) 7: if v 0 = NULL (all walks from v have already been used up) then 8: v calls G ET-M ORE -WALKS (v, λ) (Perform Θ(l/λ) walks of length λ starting at v) 9: v calls S AMPLE -D ESTINATION (v) and let v 0 be the returned value 10: end if 11: v sends the token to v 0 12: C = C ∪ {v} 13: end while 14: Walk naively until ` steps are completed (this is at most another 2λ steps) 15: A node holding the token outputs the ID of s

Sweep 1: (Perform BFS tree) 1: Construct a Breadth-First-Search (BFS) tree rooted at v. While constructing, every node stores its parent’s ID. Denote such tree by T . Sweep 2: (Tokens travel up the tree, sample as you go) 1: We divide T naturally into levels 0 through D (where nodes in level D are leaf nodes and the root node s is in level 0). 2: Tokens are held by nodes as a result of doing walks of length between λ and 2λ − 1 from v (which is done in either Phase 1 or G ET-M ORE -WALKS (cf. Algorithm ??)) A node could have more than one token. 3: Every node u that holds token(s) picks one token, denoted by d0 , uniformly at random and lets c0 denote the number of tokens it has. 4: for i = D down to 0 do 5: Every node u in level i that either receives token(s) from children or possesses token(s) itself do the following. 6: Let u have tokens d0 , d1 , d2 , . . . , dq , with counts c0 , c1 , c2 , . . . , cq (including its own tokens). The node v samples one of d0 through dq , with probabilities proportional to the respective counts. That is, for any cj 1 ≤ j ≤ q, dj is sampled with probability c0 +c1 +...+c . q 7: The sampled token is sent to the parent node (unless already at root), along with a count of c0 + c1 + . . . + cq (the count represents the number of tokens from which this token has been sampled). 8: end for 9: The root output the ID of the owner of the final sampled token. Denote such node by ud . Sweep 3: (Go and delete the sampled destination) 1: v sends a message to ud (e.g., via broadcasting). ud deletes one token of v it is holding (so that this random walk of length λ is not reused/re-stitched).

Algorithm 4 C ONTINUOUS -R ANDOM -WALK (`,) Input: The preprocessing table. Output: Number of walks of length ` before getting stuck. The source node of each walk of length ` is chosen two ways: Uniformly at random and randomly but proportional to their degrees 1: while Not Getting Stuck do 2: Select a source node s uniformly at random ( or proportional to their degrees) 3: call S INGLE -R ANDOM -WALK (s, `) (cf. Algorithm 2) 4: 5: 6:

end while