Slow iCloud photo syncing with IPv6 on 2degrees?

Forums › 2degrees (including Slingshot, Orcon, Flip, Stuff Fibre, MyRepublic, 2talk and Vocus) › Slow iCloud photo syncing with IPv6 on 2degrees?

dfnt

1512 posts

Uber Geek

Lifetime subscriber

#240840 28-Sep-2018 15:28

Anyone else notice slow iCloud photo syncing on 2degrees when IPv6 is enabled?

I did a fresh install of Mojave on my MBP so I'm currently downloading my photos and it's been going for several days and barely made a dent, only 456 photos/videos, out of 2495 so its not a huge library.

Seems to be going at ~3-5Mbit.

Wondering what was going on I had a look at nettop and see the following:

nsurlsessiond.250 143 MiB 51 KiB 0 B 5672 B 1580 B

   tcp6 x:cd06:164a:ee5c:5d0e.51830<->th-in-x80.1e100.net.443 en0 Established 6747 KiB 1270 B 0 B 0 B 0 B 148.06 ms 83 KiB 60 KiB BK_SYS -

   tcp6 x:cd06:164a:ee5c:5d0e.51831<->2403:300:a41:400::7.443 en0 Established 1557 KiB 2321 B 0 B 0 B 0 B 325.09 ms 64 KiB 34 KiB BK_SYS -

   tcp6 x:cd06:164a:ee5c:5d0e.51832<->th-in-x80.1e100.net.443 en0 Established 5628 KiB 1230 B 0 B 0 B 0 B 145.34 ms 162 KiB 60 KiB BK_SYS -

   tcp6 x:cd06:164a:ee5c:5d0e.51833<->th-in-x80.1e100.net.443 en0 Established 4826 KiB 1230 B 0 B 0 B 0 B 129.31 ms 51 KiB 60 KiB BK_SYS -

   tcp6 x:cd06:164a:ee5c:5d0e.51834<->th-in-x80.1e100.net.443 en0 Established 6995 KiB 1230 B 0 B 0 B 0 B 130.44 ms 194 KiB 60 KiB BK_SYS -

   tcp6 x:cd06:164a:ee5c:5d0e.51844<->2403:300:a41:400::7.443 en0 Established 141 B 1969 B 0 B 0 B 0 B 280.75 ms 128 KiB 28 KiB BK_SYS -

Doing a traceroute to2403:300:a41:400::7, it seems to take a scenic route to San Jose (i think) then Japan and I'm not sure what the final destination is but the latency is high.

traceroute6 to 2403:300:a50:105::8 (2403:300:a50:105::8) from x:cd06:164a:ee5c:5d0e, 64 hops max, 12 byte packets
1 ::1 1.012 ms 2.253 ms 1.447 ms
2 2406:e000:2801:c::1 3.210 ms 4.061 ms 2.235 ms
3 as23655.auckland.megaport.com 15.115 ms 16.333 ms 15.505 ms
4 2001:7fa:11:6::1b1b:0:1 15.881 ms 14.635 ms 14.317 ms
5 10ge2-3.core1.syd1.he.net 39.391 ms 39.118 ms 48.607 ms
6 10ge3-5.core1.sjc1.he.net 181.170 ms 189.033 ms 180.653 ms
7 10ge7-2.core1.sjc2.he.net 180.222 ms
100ge1-1.core1.sjc2.he.net 180.409 ms
10ge7-2.core1.sjc2.he.net 181.625 ms
8 kddi-as2516.10gigabitethernet1-1-1.switch2.sjc2.he.net 181.098 ms 181.841 ms 182.752 ms
9 6sjegcs002.int-gw.kddi.ne.jp 162.361 ms
6sjegcs001.int-gw.kddi.ne.jp 162.325 ms
6sjegcs002.int-gw.kddi.ne.jp 163.800 ms
10 oteacs001.int-gw.kddi.ne.jp 264.717 ms
2001:268:fb81:72::1 268.788 ms
2001:268:fb81:6a::1 266.005 ms
11 2001:268:fa00:2fc::2 265.173 ms
2001:268:fa00:2f9::2 269.729 ms 262.656 ms
12 6kotjin204.int-gw.kddi.ne.jp 273.841 ms 268.750 ms
6kotjin204.int-gw.kddi.ne.jp 265.031 ms
13 2001:268:f702:100::2 282.773 ms 288.522 ms 288.598 ms
14 2403:300:0:80f::21 221.936 ms 223.108 ms 217.024 ms
15 2403:300:0:80f::1 220.787 ms 220.023 ms 217.712 ms
16 2403:300:0:810::52 221.658 ms 216.997 ms 222.670 ms
17 2403:300:a09:2::302 217.379 ms 223.525 ms 218.987 ms
18 2403:300:a50:105::8 220.699 ms 220.420 ms 217.889 ms

And the other host, not sure where that ends up but judging by latency US West somewhere?

traceroute to th-in-x80.1e100.net (2404:6800:4008:c03::80), 30 hops max, 80 byte packets
1 x (x) 0.292 ms 0.256 ms 0.278 ms
2 2406:e000:2801:c::1 (2406:e000:2801:c::1) 1.380 ms 1.364 ms 1.344 ms
3 * * *
4 2001:4860:1:1:0:5c67:0:1 (2001:4860:1:1:0:5c67:0:1) 34.778 ms 34.755 ms 34.746 ms
5 2001:4860:1:1:0:5c67:: (2001:4860:1:1:0:5c67::) 34.697 ms 34.715 ms 34.669 ms
6 2001:4860:0:1107::b (2001:4860:0:1107::b) 35.080 ms 39.788 ms 39.736 ms
7 2001:4860::c:4001:11b6 (2001:4860::c:4001:11b6) 142.799 ms 2001:4860::c:4000:f918 (2001:4860::c:4000:f918) 139.317 ms 139.288 ms
8 2001:4860::c:4000:e43d (2001:4860::c:4000:e43d) 162.983 ms 2001:4860::c:4000:e43e (2001:4860::c:4000:e43e) 162.901 ms 2001:4860::8:0:c70b (2001:4860::8:0:c70b) 163.194 ms
9 2001:4860::cc:4001:1e43 (2001:4860::cc:4001:1e43) 162.057 ms 162.065 ms 2001:4860::cc:4001:1c1a (2001:4860::cc:4001:1c1a) 162.785 ms
10 * * *
11 * * *
12 * * *
13 * * *
14 * * *
15 * * *
16 * * *
17 * * *
18 * * *
19 * th-in-x80.1e100.net (2404:6800:4008:c03::80) 162.168 ms 162.891 ms

Disabling IPv6 on the Mac, it's sped up but not by a huge amount ~10-15Mbit but that could be apple throttling.

Nettop:

nsurlsessiond.250 300 MiB 117 KiB 0 B 33 KiB 2106 B

   tcp4 192.168.1.120:51892<->tg-in-f128.1e100.net:443 en0 Established 5681 KiB 1228 B 0 B 2836 B 0 B 169.09 ms 124 KiB 61 KiB BK_SYS -

   tcp4 192.168.1.120:51893<->tg-in-f128.1e100.net:443 en0 Established 4122 KiB 1226 B 0 B 2836 B 0 B 159.03 ms 126 KiB 61 KiB BK_SYS -

   tcp4 192.168.1.120:51894<->tg-in-f128.1e100.net:443 en0 Established 4997 KiB 1242 B 0 B 2836 B 0 B 170.31 ms 105 KiB 61 KiB BK_SYS -

   tcp4 192.168.1.120:51895<->tg-in-f128.1e100.net:443 en0 Established 5055 KiB 1244 B 0 B 2836 B 0 B 168.66 ms 110 KiB 61 KiB BK_SYS -

   tcp4 192.168.1.120:51896<->tg-in-f128.1e100.net:443 en0 Established 3687 KiB 1242 B 0 B 0 B 0 B 158.97 ms 77 KiB 61 KiB BK_SYS -

   tcp4 192.168.1.120:51901<->tg-in-f128.1e100.net:443 en0 Established 1369 KiB 1240 B 0 B 0 B 0 B 161.22 ms 83 KiB 61 KiB BK_SYS -

Traceroute lands in Australia judging by the latency?:

traceroute tg-in-f128.1e100.net
traceroute: Warning: tg-in-f128.1e100.net has multiple addresses; using 216.239.38.128
traceroute to tg-in-f128.1e100.net (216.239.38.128), 64 hops max, 52 byte packets
1 router (192.168.1.1) 3.169 ms 1.248 ms 2.617 ms
2 69.7.69.111.static.snap.net.nz (111.69.7.69) 2.797 ms 2.163 ms 2.221 ms
3 * * *
4 72.14.215.151 (72.14.215.151) 37.438 ms 36.263 ms 35.343 ms
5 72.14.215.150 (72.14.215.150) 36.476 ms 36.423 ms 35.640 ms
6 108.170.247.49 (108.170.247.49) 37.022 ms 37.648 ms 38.965 ms
7 209.85.248.253 (209.85.248.253) 36.216 ms
209.85.254.119 (209.85.254.119) 36.254 ms
209.85.255.165 (209.85.255.165) 36.416 ms
8 any-in-2680.1e100.net (216.239.38.128) 36.358 ms 36.062 ms 36.303 ms

I believe Apple use Google Cloud Compute hence the hosts/IP's belonging to Google.

Any idea why when using IPv6 Apple/Google/someone isn't preferring Australian hosts?

edit: Although having said that I just noticed the latency figures for ipv4 are all over 100 which suggests it's not hitting Australia either :/. Guessing this is just all down to Apple and not much can be done?

1 | 2 | 3

962 posts

Ultimate Geek

Trusted

Lifetime subscriber

#2098339 28-Sep-2018 18:23

Hiya,

On the 17th of September Hurricane Electric started peering out of NZ at the AKL-IX peering exchange https://www.mail-archive.com/nznog@list.waikato.ac.nz/msg01032.html. Hurricane Electric are advertising this route who peer with us domestically, versus Apple and Google who’s PNI peerings are in Sydney and the States. You can see from the traceroute6 that this leaves the 2degrees network over Megaport Auckland, then takes a long path through Sydney to somewhere in Asia. This is the best route as we prefer those peering domestically vs. international. They appear to be advertising ipV6 and ipV4 routes differently. We will reach out next week to see if they will work with us to see what can be done.

Nick.

https://nick.mackechnie.co.nz | NZ ISP latency monitoring - https://smokeping.thenet.gen.nz

dfnt

1512 posts

Uber Geek

Lifetime subscriber

#2098341 28-Sep-2018 18:33

Thanks @NickMack

So basically HE are doing some weird scenic routing after you handoff to them at megaport?

All good, once my photos/videos finish syncing I'll enable ipv6 again

TheoM

228 posts

Master Geek

ID Verified

Trusted

#2098404 28-Sep-2018 22:50

dfnt:

Thanks @NickMack

So basically HE are doing some weird scenic routing after you handoff to them at megaport?

All good, once my photos/videos finish syncing I'll enable ipv6 again

That sounds exactly like HE to me. Their NOC team is usually pretty good about fixing stuff up quickly. I can't say the same for Cogent though...

Hi! I'm TheoM, but you know that already. I run Linux mirrors in NZ together with 2degrees. Like a mirror added? PM me!

https://theom.co.nz | https://theom.nz | https://mirrorlist.mirrors.theom.nz | Providing Free Mirrors Since Ages Ago™

dfnt

1512 posts

Uber Geek

Lifetime subscriber

#2098440 29-Sep-2018 09:20

Now ipv4 is favouring Singapore so it's back down to ~3Mbit download -_-

nsurlsessiond.280 478 MiB 457 KiB 19 KiB 1966 KiB 3824 B

   tcp4 192.168.1.120:49552<->17.248.154.113:443 en0 Established 9826 KiB 2311 B 0 B 0 B 0 B 302.84 ms 64 KiB 32 KiB BK_SYS - ledbat -

   tcp4 192.168.1.120:49556<->17.248.154.113:443 en0 Established 9694 KiB 1967 B 0 B 0 B 0 B 299.97 ms 64 KiB 32 KiB BK_SYS - ledbat -

   tcp4 192.168.1.120:49557<->17.248.154.113:443 en0 Established 9706 KiB 1955 B 0 B 0 B 0 B 298.72 ms 64 KiB 32 KiB BK_SYS - ledbat -

   tcp4 192.168.1.120:49560<->17.248.154.172:443 en0 Established 4635 KiB 2044 B 0 B 0 B 0 B 297.47 ms 64 KiB 32 KiB BK_SYS - ledbat -

   tcp4 192.168.1.120:49561<->17.248.154.172:443 en0 Established 4634 KiB 2042 B 0 B 0 B 0 B 296.41 ms 64 KiB 32 KiB BK_SYS - ledbat -

   tcp4 192.168.1.120:49566<->17.248.154.172:443 en0 Established 5652 KiB 8667 B 0 B 0 B 0 B 295.19 ms 306 KiB 45 KiB BK_SYS - ledbat -

traceroute to 17.248.154.113 (17.248.154.113), 64 hops max, 52 byte packets
1 router (192.168.1.1) 1.490 ms 1.735 ms 0.996 ms
2 69.7.69.111.static.snap.net.nz (111.69.7.69) 2.096 ms 2.748 ms 2.188 ms
3 111-69-3-201.core.snap.net.nz (111.69.3.201) 12.624 ms 13.345 ms 14.814 ms
4 ip4.gtt.net (173.205.51.174) 137.527 ms 137.251 ms 152.960 ms
5 ae7-200.cr0-sjc2.ip4.gtt.net (173.205.51.173) 138.087 ms 138.783 ms 137.231 ms
6 xe-10-2-0.cr0-tyo1.ip4.gtt.net (89.149.135.218) 230.837 ms 231.438 ms 231.669 ms
7 63-218-251-113.static.pccwglobal.net (63.218.251.113) 255.625 ms 255.455 ms 254.533 ms
8 63-218-251-178.static.pccwglobal.net (63.218.251.178) 248.415 ms 248.141 ms 248.220 ms
9 17.0.179.93 (17.0.179.93) 326.424 ms 326.674 ms 327.671 ms
10 *^C

Think I'll have to use a VPN instead.

edit: Better over an aus VPN

nsurlsessiond.280 2071 MiB 833 KiB 921 KiB 23 MiB 8707 B
tcp4 10.0.1.163:49797<->tg-in-f128.1e100.net:443 ppp0 Established 60 MiB 1888 B 0 B 92 KiB 0 B 209.16 ms 83 KiB 62 KiB BK_SYS - ledbat -
tcp4 10.0.1.163:49798<->tg-in-f128.1e100.net:443 ppp0 Established 62 MiB 2554 B 0 B 142 KiB 0 B 195.00 ms 81 KiB 63 KiB BK_SYS - ledbat -
tcp4 10.0.1.163:49799<->tg-in-f128.1e100.net:443 ppp0 Established 63 MiB 3222 B 0 B 62 KiB 0 B 174.97 ms 45 KiB 65 KiB BK_SYS - ledbat -
tcp4 10.0.1.163:49800<->tg-in-f128.1e100.net:443 ppp0 Established 62 MiB 2554 B 0 B 94 KiB 0 B 196.72 ms 52 KiB 63 KiB BK_SYS - ledbat -
tcp4 10.0.1.163:49801<->tg-in-f128.1e100.net:443 ppp0 Established 62 MiB 2550 B 0 B 129 KiB 0 B 188.56 ms 100 KiB 63 KiB BK_SYS - ledbat -
tcp4 10.0.1.163:49809<->th-in-f128.1e100.net:443 ppp0 Established 22 MiB 1228 B 0 B 46 KiB 0 B 230.22 ms 81 KiB 61 KiB BK_SYS - ledbat -

Not sure where it goes after Vocus, probably US West somewhere

traceroute to tg-in-f128.1e100.net (74.125.238.128), 64 hops max, 52 byte packets
1 10.0.1.1 (10.0.1.1) 46.312 ms 39.331 ms 37.428 ms
2 gi1-11.sw-13-3.as45671.net.au (221.121.135.97) 39.328 ms 42.686 ms 41.470 ms
3 vl2902.sw-13-1.as45671.net.au (221.121.135.50) 39.465 ms 40.265 ms 40.429 ms
4 221.121.135.0 (221.121.135.0) 42.360 ms 37.932 ms 38.570 ms
5 ten-0-7-0-5-2800.bdr04.alb01.akl.vocus.net.nz (175.45.93.225) 41.037 ms 44.030 ms 39.347 ms
6 bundle-12.cor01.akl05.akl.vocus.net.nz (114.31.202.50) 68.533 ms
bundle-11.cor01.alb01.akl.vocus.net.nz (114.31.202.48) 63.978 ms 66.819 ms
7 be-201.cor01.syd11.nsw.vocus.net.au (114.31.202.53) 62.648 ms
be-202.cor01.syd11.nsw.vocus.net.au (114.31.202.55) 68.644 ms 65.938 ms
8 be-100.bdr02.syd03.nsw.vocus.net.au (114.31.192.39) 64.938 ms 64.756 ms 63.788 ms
9 * * *

yitz

2079 posts

Uber Geek

#2098476 29-Sep-2018 11:14

Does 2degrees not having peering with Apple AS714 in Sydney?

dfnt

1512 posts

Uber Geek

Lifetime subscriber

#2098478 29-Sep-2018 11:23

I'm not sure, it's definitely much slower than it used to be on Spark. Even using an Aus VPN endpoint the photo backups seem to hit US West servers

Maybe Apple don't have any iCloud photo infrastructure in Sydney

hio77

12999 posts

Uber Geek

ID Verified

Trusted

Lizard Networks

#2098491 29-Sep-2018 12:28

yitz: Does 2degrees not having peering with Apple AS714 in Sydney?

as Nick mentioned above though, they favor peering within nz. so it's going over the HE peering.

#include <std_disclaimer>

Any comments made are personal opinion and do not reflect directly on the position my current or past employers may have.

Sharesies

Trade NZ and US shares and funds with Sharesies (affiliate link).

dfnt

1512 posts

Uber Geek

Lifetime subscriber

#2098503 29-Sep-2018 13:05

hio77:

yitz: Does 2degrees not having peering with Apple AS714 in Sydney?

as Nick mentioned above though, they favor peering within nz. so it's going over the HE peering.

That's for ipv6, for ipv4 it's going over gtt but I guess the same principle applies

yitz

2079 posts

Uber Geek

#2098505 29-Sep-2018 13:13

That doesn't really explain the traceroute to 17.248.154.113 going via ip4.gtt.net (173.205.51.174) (Transit?)

Have got to wonder how much HE traffic is actually coming back via the Auckland IXP's? Otherwise whats the point..

NickMack

962 posts

Ultimate Geek

Trusted

Lifetime subscriber

#2100902 3-Oct-2018 17:36

Hiya - Just an update on this one, the team are still investigating what we can do in this space, once a decision is made, I'll provide an update.

https://nick.mackechnie.co.nz | NZ ISP latency monitoring - https://smokeping.thenet.gen.nz

dfnt

1512 posts

Uber Geek

Lifetime subscriber

#2100914 3-Oct-2018 18:15

NickMack:

Hiya - Just an update on this one, the team are still investigating what we can do in this space, once a decision is made, I'll provide an update.

Sweet, all good thanks!

dfnt

1512 posts

Uber Geek

Lifetime subscriber

#2104343 10-Oct-2018 10:19

Any update on this?

I'm actually considering moving back to Spark as the icloud speeds are so slow on 2degrees.

I get the whole peer locally thing but in this case it's really at the detriment of the user.

NickMack

962 posts

Ultimate Geek

Trusted

Lifetime subscriber

#2104348 10-Oct-2018 10:27

dfnt:

Any update on this?

I'm actually considering moving back to Spark as the icloud speeds are so slow on 2degrees.

I get the whole peer locally thing but in this case it's really at the detriment of the user.

Hiya,

Beat me to it, I was just discussing it - We've done some jiggery pokery for Apple traffic, the change was applied this morning. Routing is now out of Sydney for IPv6 (affected traffic) due HE coming into NZ and advertising routes via NZ peering exchanges. Be great if you can test and give us some feedback in terms of the difference.

Nick.

https://nick.mackechnie.co.nz | NZ ISP latency monitoring - https://smokeping.thenet.gen.nz

michaelmurfy

meow
13249 posts

Uber Geek

Moderator

ID Verified

Trusted

Lifetime subscriber

#2104350 10-Oct-2018 10:27

Ping @Joelfreak - hopefully he should be able to help out here :)

Michael Murphy | https://murfy.nz
Referral Links: Quic Broadband (use R122101E7CV7Q for free setup)

Are you happy with what you get from Geekzone? Please consider supporting us by subscribing.
Opinions are my own and not the views of my employer.

yitz

2079 posts

Uber Geek

#2105334 10-Oct-2018 10:42

dfnt:
Any update on this?

I'm actually considering moving back to Spark as the icloud speeds are so slow on 2degrees.

Turning off IPv6 might have the same effect lol

1 | 2 | 3