Measuring the Effects of Happy Eyeballs Vaibhav Bajpai Jacobs University, Bremen

Introduction Motivation Browser Implementations Research Contributions

Methodology

RIPE 73 Madrid, Spain

Metrics and Implementation Selection of Websites Measurement Setup Measurement Trial

Published at: Applied Networking Research Workshop July 2016: https://goo.gl/ZiIcQV

Data Analysis Trends Who connects faster? Preference Slowness

Joint work with

Lowering HE Timer

Limitations

Jürgen Schönwälder Jacobs University, Bremen

Takeway

October 2016 Supported by: Flamingo Project: flamingo-project.eu Leone Project: leone-project.eu 1 / 18

Introduction | Motivation Introduction

▶

Motivation

HE timer (300 ms) was chosen (2012) when broken IPv6 connectivity was prevalent.

Browser Implementations Research Contributions

▶ ▶

Largely attributed to failures caused by Teredo [1] and 6to4 relays [2]. Even in situations where relays work, Teredo / 6to4 add noticeable latency [3, 4].

Methodology Metrics and Implementation Selection of Websites Measurement Setup Measurement Trial

▶

These transition mechanisms have declined over the years due to efforts such as −

Data Analysis Trends

2013 Microsoft stopped Teredo on Windows and deactivated public Teredo servers [5]. 2015 The 6to4 anycast prefix has been obsoleted [6].

Who connects faster? Preference Slowness Lowering HE Timer

▶

Consequentely, failure rates over IPv6 [7] have dropped significantly −

2011 2015

Overall

Native

40% 3.5%

5.3% 2%

Limitations Takeway

2 / 18

Introduction | Browser Implementations Introduction Motivation Browser Implementations Research Contributions

Fragmentation of HE is visible in browser implementations today −

Methodology Metrics and Implementation

2011 Chrome uses 300 ms [8].

[since v11]

Selection of Websites Measurement Setup

2011 Safari uses history of witnessed latencies [9].

[since OS X 10.7]

Measurement Trial

Data Analysis

2012 Opera uses parallel TCP connections [10].

[since v12.10]

2012 Firefox uses parallel TCP connections [11].

[since v15]

Trends Who connects faster? Preference Slowness Lowering HE Timer

Firefox

[network.http.fast-fallback-to-IPv4=false]

uses 250 ms.

2015 Safari uses 25 ms + history of witnessed latencies [12].

Limitations

[since OS X 10.11 / iOS 9]

Takeway

These HE timer values are arbitrarily chosen. What is the right timer value?

3 / 18

Introduction | Research Contributions Introduction Motivation Browser Implementations Research Contributions

We measure against ALEXA top 10K websites for 3 years (2013 - 2016)

Methodology Metrics and Implementation Selection of Websites Measurement Setup Measurement Trial

Data Analysis

1. TCP connect times to websites over IPv6 have considerably improved over time.

Trends Who connects faster? Preference

2. 18% of websites are faster over IPv6 with 91% being at most 1 ms slower (May ’16).

Slowness Lowering HE Timer

3. HE (300 ms) makes 99% of websites prefer IPv6 more than 98% of the time.

Limitations

4. Slower IPv6 connections are preferred in ∼90% of the cases.

Takeway

5. Lowering HE (150 ms) gives a margin benefit of 10% and retains same preference levels.

4 / 18

Introduction Motivation Browser Implementations Research Contributions

Methodology Metrics and Implementation

Methodology1

Selection of Websites Measurement Setup Measurement Trial

Data Analysis Trends Who connects faster? Preference Slowness Lowering HE Timer

Limitations Takeway

1

Please see previous work [13] for a more detailed description of our methodology 5 / 18

Methodology | Metrics and Implementation ▶ Uses

Introduction getaddrinfo(…)

to resolve service names.

▶ Uses non-blocking TCP

Motivation Browser Implementations

connect(…)

calls.

Research Contributions

▶ DNS resolution time is not accounted.

connection establishment times (µs)

▶ Can read multiple service names as arguments. ▶ Can read service names list from a file.

1) service name 2) port

happy

▶ File locking capability. ▶ Sets a delay between

1) endpoint 2) endpoint 3) endpoint ... n) endpoint

Methodology Metrics and Implementation Selection of Websites Measurement Setup Measurement Trial

Data Analysis Trends Who connects faster?

connect(…) ;

avoids SYN floods.

happy.vaibhavbajpai.com.

▶ Can produce both human-readable & CSV output. ▶ Cross-compiled for OpenWrt; Running on SamKnows.

Preference Slowness Lowering HE Timer

Limitations Takeway

% happy -q 1 -m www.google.com www.facebook.com HAPPY.0;1360681039;OK;www.google.com;80;173.194.69.105;8626 HAPPY.0;1360681039;OK;www.google.com;80;2a00:1450:4008:c01::69;8884

6 / 18

Methodology | Selection of Websites Introduction Motivation Browser Implementations

1. www.google.com 2. www.facebook.com

Research Contributions

Methodology Metrics and Implementation Selection of Websites Measurement Setup

3. www.youtube.com ▶

We use the ALEXA top 10K websites as measurement targets [13].

4. www.yahoo.com

Measurement Trial

Data Analysis Trends Who connects faster?

5. www.wikipedia.org 6. www.qq.com 7. www.blogspot.com

Preference Slowness Lowering HE Timer

Limitations Takeway

8. …

7 / 18

Methodology | Measurement Setup Introduction Motivation Browser Implementations Research Contributions

Methodology ALEXA Top 10K Websites

Metrics and Implementation Selection of Websites Measurement Setup Measurement Trial

happy

test repeats every hour.

happy

IPv4

SamKnows Tests

Trends

results

The

Data Analysis

TCP connect(...)

IPv6

Who connects faster?

DSL/Cable Modem

Preference Slowness Lowering HE Timer

Probe

Limitations HTTPS POST Data Collector

Takeway

8 / 18

Methodology | Measurement Trial Introduction Motivation Browser Implementations

NETWORK TYPE

#

Research Contributions

Methodology RESIDENTIAL NREN / RESEARCH BUSINESS / DATACENTER OPERATOR LAB IXP

55 11 09 04 01

Metrics and Implementation Selection of Websites Measurement Setup Measurement Trial

Data Analysis Trends Who connects faster?

RIR

# Preference

RIPE ARIN APNIC AFRINIC LACNIC

42 29 07 01 01

Slowness Lowering HE Timer

Limitations Takeway

We measure from 80 dual-stacked SamKnows [14] probes. 9 / 18

Introduction Motivation Browser Implementations Research Contributions

Methodology Metrics and Implementation

Data Analysis

Selection of Websites Measurement Setup Measurement Trial

Data Analysis

[2013 - 2016]

Trends Who connects faster? Preference Slowness Lowering HE Timer

Limitations Takeway

10 / 18

Data Analysis | Trends (2013 - 2016) Introduction Motivation

∆sa (u) = t4 (u) − t6 (u)

Browser Implementations Research Contributions

Methodology

where t(u) is the time taken to establish TCP connection to website u.

Metrics and Implementation Selection of Websites Measurement Setup

50

Measurement Trial

0

Data Analysis

∆sa (ms)

−50 −100

www.bing.com

www.wikipedia.org

www.facebook.com

www.youtube.com

−150

Who connects faster? Preference Slowness

20 0 −20 −40 −60

Lowering HE Timer

Limitations www.blogspot.*

www.netflix.com www.yahoo.com

www.google.* 02 2013

▶

Trends

05

08

11

02 2014

05

08

11

02 2015

05

08

11

02 2016

Takeway

05

TCP connect times to popular websites over IPv6 have considerably improved over time. 11 / 18

Data Analysis | Who connects faster? Introduction Motivation Browser Implementations Research Contributions

ALEXA top 10K websites (as of May 2016):

Methodology Metrics and Implementation Selection of Websites

1.0

▶ ▶

18% are faster over IPv6. 91% of the rest are at most 1 ms slower. 3% are at least 10 ms slower. 1% are at least 100 ms slower.

0.6 0.4

microsoft youtube google

netflix

Measurement Trial

wikipedia linkedin

0.2 yahoo

0.0 −1.0

[05/2016]

▶

0.8 CDF

▶

Measurement Setup

facebook

−0.5 0.0 ∆sa (ms)

ALEXA (10K)

Data Analysis Trends Who connects faster? Preference Slowness Lowering HE Timer

Limitations 0.5

1.0

Takeway

∆sa (u) = t4 (u) − t6 (u)

12 / 18

Data Analysis | Preference Introduction

IPv6 (189K) IPv4 (189K)

10-1

['13 - '16]

timer value > 300 ms

Browser Implementations

300 ms

▶ Only ∼1% of samples above HE

CDF

Motivation

1.0 0.8 0.6 0.4 0.2 0.0

101 100 102 103 TCP Connect Times (ms)

104

Research Contributions

Methodology Metrics and Implementation Selection of Websites Measurement Setup Measurement Trial

Data Analysis Trends Who connects faster? Preference Slowness

Preference (300 ms)

▶ 99% of top 10K ALEXA prefer

IPv6 98% of time.

Lowering HE Timer

['13 - '16]

2% chance for IPv4.

CCDF

▶ A 300 ms HE timer value leaves

1.0 0.8 0.6 0.4 0.2 0.0

PROBES (80) ALEXA (10K) 96%

97%

98%

99%

Limitations Takeway

100%

13 / 18

1.0 0.8 0.6 0.4 0.2 0.0

1%

−30

▶ ▶

30% − at least 1 ms slower. 7% − at least 10 ms slower.

CDF

apart from IPv4

1.0 0.8 0.6 0.4 0.2 0.0

Research Contributions

Methodology

−20 −10 ∆sa (ms)

0

10

90%

189K

97%

57%

2%

Browser Implementations

7%

2%

▶ HE prefers slower IPv6 ▶ Absolute difference is not that far

Motivation

99%

30%

−40

connections 90% of the time.

Introduction

93%

189K

8%

['13 - '16]

Samples where HE prefers IPv6 −

CDF

Data Analysis | Slowness

Metrics and Implementation Selection of Websites Measurement Setup Measurement Trial

Data Analysis Trends Who connects faster? Preference Slowness

−0.6

−0.4

−0.2 ∆sr

0.0

0.2

Lowering HE Timer

Limitations Takeway

∆sa (u) = t4 (u) − t6 (u) t (u)−t (u) ∆sr (u) = 4 t (u)6 4

Can a lower HE timer provide same preference over IPv6 but not penalise IPv4 when it’s faster? 14 / 18

Data Analysis | Lowering HE Timer Introduction Motivation Browser Implementations

Are we ready to disable HE entirely?

Research Contributions

Methodology IPv6 Preference 100% 80% 60% 40% 20% 0%

Metrics and Implementation

0 ms 150 ms 300 ms

Selection of Websites Measurement Setup Measurement Trial

Data Analysis Trends

02 2013

05

08

11

02 2014

05

08

11

02 2015

05

08

11

02 2016

05

Who connects faster? Preference Slowness Lowering HE Timer

▶ 18% of ALEXA top 10K websites are faster (see slide 17) over IPv6 today.

Limitations

▶ Parallel TCP connections2 (HE with 0 ms timer) will hamper IPv6 preference.

Takeway

▶ HE timer today still should give IPv6 a fair chance to succeed.

2

such as used by Firefox and Opera today 15 / 18

Data Analysis | Lowering HE Timer Introduction

▶ Each data point is the 1th

80%

150 ms

lower the HE timer value.

Preference

▶ We control two3 parameters and

60% 40% 20%

ALEXA (10K)

['13 - '16]

Motivation

100%

0

50

100 150 200 HE timer (ms)

250

Research Contributions

Methodology Metrics and Implementation Selection of Websites Measurement Setup

0%

percentile preference towards ALEXA 10K websites.

Browser Implementations

300

Measurement Trial

Data Analysis Trends Who connects faster? Preference Slowness

(18.9K) because timer cuts early.

CDF

preference levels over IPv6.

1.0 0.8 0.6 0.4 0.2 0.0

80%

189K −0.6

3

['13 - '16]

▶ Lowering to 150 ms retains ▶ We get margin benefit of 10%

Lowering HE Timer

Slowness (150ms)

−0.4

−0.2 ∆sr

0.0

Limitations Takeway

0.2

99% ALEXA top 10K websites prefer IPv6 connections 98.6% of the time 16 / 18

Limitations Introduction Motivation Browser Implementations Research Contributions

Methodology Metrics and Implementation Selection of Websites

1. The comparison reflects the performance as seen over TCP port 80 only. 2. The measurements cover ALEXA top 10K websites only.

Measurement Setup Measurement Trial

Data Analysis Trends Who connects faster?

3. The results are biased by our vantage points (centered largely around EU, US and JP).

Preference Slowness Lowering HE Timer

Limitations Takeway

17 / 18

Takeway Introduction Motivation

1. TCP connect times to websites over IPv6 have considerably improved over time. 2. 18% of websites are faster over IPv6 with 91% being at most 1 ms slower (May ’16). 3. HE (300 ms) makes 99% of websites prefer IPv6 more than 98% of the time. 4. Slower IPv6 connections are preferred in ∼90% of the cases. 5. Lowering HE (150 ms) gives a margin benefit of 10% and retains same preference levels.

Browser Implementations Research Contributions

Methodology Metrics and Implementation Selection of Websites Measurement Setup Measurement Trial

Data Analysis Trends Who connects faster? Preference Slowness Lowering HE Timer

Limitations Takeway

www.vaibhavbajpai.com [email protected] | @bajpaivaibhav

18 / 18

Background Research Question

Related Work

Appendix

1/6

References [1] C. Huitema, “Teredo: Tunneling IPv6 over UDP through Network NATs,” RFC 4380, Internet Engineering Task Force, Feb. 2006, https://tools.ietf.org/html/rfc4380. [2] B. Carpenter and K. Moore, “Connection of IPv6 Domains via IPv4 Clouds,” RFC 3056, Internet Engineering Task Force, Feb. 2001, https://tools.ietf.org/html/rfc3056. [3] S. Zander, L. L. H. Andrew, G. J. Armitage, G. Huston, and G. Michaelson, “Investigating the IPv6 Teredo Tunnelling Capability and Performance of Internet Clients,” ser. Computer Communication Review (CCR) ’12, 2012, pp. 13–20. [Online]. Available: http://doi.acm.org/10.1145/2378956.2378959 [4] L. Colitti, S. H. Gunderson, E. Kline, and T. Refice, “Evaluating IPv6 Adoption in the Internet,” ser. Passive and Active Measurement Conference (PAM) ’10, 2010, pp. 141–150. [Online]. Available: http://dx.doi.org/10.1007/978-3-642-12334-4_15 [5] “Christopher Palmer - Teredo at Microsoft: Present and Future,” http://goo.gl/9I65Wy, [Online; accessed 10-February-2016]. [6] O. Troan and B. Carpenter, “Deprecating the Anycast Prefix for 6to4 Relay Routers,” RFC 7526, Internet Engineering Task Force, May 2015, https://tools.ietf.org/html/rfc7526. [7] “Geoff Huston - Measuring IPv6 Performance,” https://goo.gl/n78W1t, [Online; accessed 10-February-2016].

[8] “Google Chrome - Revision 85934: Add a fallback socket connect() for IPv6.” https://goo.gl/nPhilZ, [Online; accessed 25-January-2016]. [9] J. Graessley, “Apple - Lion and IPv6,” http://goo.gl/uAPlV8, [Online; accessed 25-January-2016].

Background Research Question

Related Work

[10] “Opera 12.10 - Changelog,” http://goo.gl/MGsn4K, [Online; accessed 25-Jan-2016]. [11] “Mozilla Firefox 15 - Release Notes,” http://goo.gl/hA15eu, [Online; accessed 25-January-2016]. [12] D. Schinazi, “Apple and IPv6 - Happy Eyeballs,” https://goo.gl/1nzMs6, [Online; accessed 25-January-2016]. [13] V. Bajpai and J. Schönwälder, “IPv4 versus IPv6 - who connects faster?” ser. IFIP NETWORKING ’15, 2015, pp. 1–9. [Online]. Available: http://dx.doi.org/10.1109/IFIPNetworking.2015.7145323 [14] ——, “A Survey on Internet Performance Measurement Platforms and Related Standardization Efforts,” ser. IEEE Communications Surveys and Tutorials (COMST) ’15, 2015, pp. 1313–1341. [Online]. Available: http://dx.doi.org/10.1109/COMST.2015.2418435 [15] D. Thaler, R. Draves, A. Matsumoto, and T. Chown, “Default Address Selection for Internet Protocol Version 6 (IPv6),” RFC 6724 (Proposed Standard), Internet Engineering Task Force, Sep. 2012. [Online]. Available: http://www.ietf.org/rfc/rfc6724.txt 1/6

[16] “Teemu Savolainen - Experiences of host behavior in broken IPv6 networks,” http://goo.gl/4NnRiH, [Online; accessed 25-January-2016].

[23] “Geoff Huston - Bemused Eyeballs: Tailoring Dual Stack Applications for a CGN Environment,” http://goo.gl/LMPc4h, [Online; accessed 10-February-2016].

[17] P. Richter, M. Allman, R. Bush, and V. Paxson, “A Primer on IPv4 Scarcity,” ser. Computer Communication Review (CCR), vol. 45, no. 2. New York, NY, USA: ACM, Apr. 2015, pp. 21–31. [Online]. Available: http://doi.acm.org/10.1145/2766330.2766335

[24] F. Baker, “Testing Eyeball Happiness,” RFC 6556, Internet Engineering Task Force, 2012, https://tools.ietf.org/html/rfc6556.

[18] “Internet Society - World IPv6 Launch,” http://www.worldipv6launch.org, [Online; accessed 11-January-2016]. [19] “Google - IPv6 Adoption Statistics,” http://www.google.com/intl/en/ipv6/statistics.html, [Online; accessed 11-January-2016]. [20] “Lorenzo Colitti - Google no longer returning AAAA records?” https://goo.gl/6Z7gZM, [Online; accessed 11-January-2016]. [21] “Emile Aben - Hampering Eyeballs: Observations on Two Happy Eyeballs Implementations,” https://goo.gl/3xVUIO, [Online; accessed 10-February-2016]. [22] “Geoff Huston - Dual Stack Esotropia,” http://goo.gl/N1qUib, [Online; accessed 10-February-2016].

Background Research Question

Related Work [25] S. Zander, L. L. H. Andrew, G. J. Armitage, G. Huston, and G. Michaelson, “Mitigating Sampling Error when Measuring Internet Client IPv6 Capabilities,” ser. Internet Measurement Conference (IMC) ’12, 2012, pp. 87–100. [Online]. Available: http://doi.acm.org/10.1145/2398776.2398787 [26] V. Bajpai and J. Schönwälder, “Measuring the Effects of Happy Eyeballs,” Internet Engineering Task Force, Internet-Draft draft-bajpai-happy-01, Jul. 2013, work in Progress. [Online]. Available: http://tools.ietf.org/html/draft-bajpai-happy-01 [27] S. Ahsan, V. Bajpai, J. Ott, and J. Schönwälder, “Measuring YouTube from Dual-Stacked Hosts,” ser. Passive and Active Measurement Conference (PAM) ’15, 2015, pp. 249–261. [Online]. Available: http://dx.doi.org/10.1007/978-3-319-15509-8_19

2/6

Introduction | getaddrinfo(…) behavior Background Research Question

getaddrinfo(...) preference:

TCP connection request

1) native IPv6 routes ... 2) native IPv4 routes ... 3) IPv4-IPv6 Transitioning routes

▶

returns a list of endpoints in an order that prioritizes an IPv6-upgrade path.

▶

The order is prescribed by RFC 6724 [15] and

▶

Iterating sequentially over the list of IP endpoints has repercussions − ▶ ▶

Related Work

/etc/gai.conf

Broken IPv6 connectivity makes apps stall for several seconds before trying IPv4. Studies have reported [16] browser connection timeouts in the order of 20 seconds. 2/6

Introduction | Happy Eyeballs [RFC 6555] Background

HE helps prevent bad QoE in situations where IPv6 connectivity is broken.

Research Question

Related Work

Happy Eyeballs [RFC 6555] IPv4 IPv6

t0

t0 + 300ms

time

Design Goals − ▶

Honor the destination address selection policy [RFC 6724] [15].

▶

Quickly fallback to IPv4 when IPv6 connectivity is broken.

▶

Give a fair chance for IPv6 to succeed. 3/6

Introduction | Motivation IPv6 landscape has changed today − ▶

4/5 RIRs have exhausted available pool of IPv4 address space [17]. APNIC RIPE LACNIC ARIN

Research Question

Related Work

Apr′ 11 Sep′ 12 Jun′ 14 Sep′ 15

▶

Large IPv6 broadband rollouts4 since World IPv6 Launch Day in 2012 [18].

▶

IPv6 global adoption at ∼12.2% (native) with Teredo / 6to4 at ∼0.01% [19] (July 2016)

▶

Google over IPv6 (whitelist) program replaced by a Google IPv6 blacklist [13].

▶

Google will not return AAAA to resolvers where latency over IPv6 > 100 ms worse [20].

4

Background

Comcast, Deutsche Telekom AG, AT&T, Verizon Wireless, T-Mobile USA 4/6

Introduction | Research Questions Background Research Question

The effects of HE (300 ms) on the QoE of a dual-stacked user remains largely unclear.

Related Work

We want to know − ▶

In what percentage of cases HE makes a bad decision of choosing IPv6 when it’s slower?

▶

In such situations what is the amount of imposition (in terms of latency impact) a dual-stacked user has to pay as a result of the high HE timer (300 ms) value?

Applications apply HE not only where IPv6 is broken, but also when IPv6 is comparable.

5/6

Related Work Background Research Question

2011 - 2012 Studies [21, 22, 23] have analyzed HE implementations. ▶ ▶ ▶

Related Work

Chrome reduces degraded user experience when IPv6 is broken. Firefox [network.http.fast-fallback-to-IPv4=false] behaves similar to Chrome. Safari prefers IPv4 even when IPv6 connectivity is similar (hampering eyeballs).

These studies are dated. HE implementations have changed with time (see slide 7). 2012 Baker [24] describes HE metrics and testbed configurations. 2012 Zander [25] showed that 75% of the connection attempts preferred5 IPv6. 2013 We [26] showed that HE never prefers IPv6 using Teredo. 2015 We [27] showed that HE prefers YouTube over IPv6 even when IPv4 performs better.

5

In this work, we show that this preference has increased to 98% today 6/6