Geekzone: technology news, blogs, forums
Guest
Welcome Guest.
You haven't logged in yet. If you don't have an account you can register now.


View this topic in a long page with up to 500 replies per page Create new topic
1 | 2 | 3 | 4 | 5 | 6
prat33k
171 posts

Master Geek


  #3186594 25-Jan-2024 23:10
Send private message quote this post

Does anyone see more packet loss? Tried to connect back to work computer (via AWS) and was lagging. Ran a MTR and see packet loss at core1-akl1 node. 

Network status point to 40-50% current strain on the chorus - Quic handover link but this is after the BNG into Vetta's network?

 

Edit: 
Dang rose up to 45% now.. is this normal?


 
 
 

Backblaze Unlimited Backup. World’s easiest cloud backup. Get peace of mind knowing your files are backed up securely in the cloud (affiliate link).
michaelmurfy
meow
12654 posts

Uber Geek

Moderator
ID Verified
Trusted
Lifetime subscriber

  #3186601 25-Jan-2024 23:58
Send private message quote this post

@prat33k I see some, but far far less than what there was. There is still more work to do as per the OP though. But from your MTR's it appears that pinging a router in the path is seeing some dropped packets which is fine but the end destination doesn't respond to ICMP. Here's one for me to apple.com:

 

 

We can see loss on hops 4 and 6 but no loss from the end destination. MTR pings everything in the path but many routers deprioritise ICMP which may show as loss on a MTR. Even right now as I type this I am up to over 500 sent pings and so far only loss on hops 4 and 6 with none to Akamai.

 

This aligns with what I see elsewhere on monitoring - barely any loss to Apple despite it hopping over to Sydney. I can repeat the test for most destinations and get hardly any loss.

 

My Smokeping is looking pretty funky currently as I'm testing a few things out and troubleshooting a potential beta bug but a standard Quic connection outside of the Beta looks fine to me. I think there was a slight IPv6 hiccup earlier on but think they're sharing some IPv6 between both the old and new networks currently too so we should see more improvement once their old network is gone.

 

Also I am pretty sure that network weather chart is showing the old network still on their site which is operating in parallel currently as they move stuff across to the new network.





Michael Murphy | https://murfy.nz
Referral Links: Octopus Energy ($50 Credit) | Tesla | Quic Broadband (use R122101E7CV7Q for free setup)

Are you happy with what you get from Geekzone? Please consider supporting us by subscribing.
Opinions are my own and not the views of my employer.


Gwilson
119 posts

Master Geek


  #3186612 26-Jan-2024 06:45
Send private message quote this post

Yes, it certainly looks like things are a bit 'funky' with IPV6 at the moment....

 

 

 




hio77
'That VDSL Cat'
12999 posts

Uber Geek

ID Verified
Trusted
Voyager
Subscriber

  #3186613 26-Jan-2024 07:02
Send private message quote this post

The pings being dropped or not responded to in their core is fairly common.

Based on the traces I'd suspect they have gone for a fairly standard MPLS P/PE design, likely with a underlay that's being ran as close to a bgp free core which is becoming the normal these days.


As to a single router along the path showing loss, you can safely ignore this long as the loss does not carry over. In this case what your seeing is simply ICMP deprioritization or rate limiting.



The traces actually look pretty well fine, I would say ipv6 peering does not 1:1 match ipv4 (witch is very common too) cloudflare in particular looks like NZ peering may be down. That could be a maintenance somewhere though too.




#include <std_disclaimer>

 

Any comments made are personal opinion and do not reflect directly on the position my current or past employers may have.

 

 


pc

pc
104 posts

Master Geek

Trusted
Subscriber

  #3187074 26-Jan-2024 22:32
Send private message quote this post

bartong: I’m noticing some issue with my Google Home devices (smart speakers) since the upgrade. They seem to be struggling intermittently? Often timing out on responding to commands, music streaming randomly stops, saying the lights or TVs are offline when they clearly are not… just 15 minutes ago I was looking in the Google home app on my phone and everything was just a spinning status.

 

 

 

the 3 ipv6 hosts i was smokepinging are now responding.... and so it seems are my google home devices!


bartong
33 posts

Geek


  #3187092 27-Jan-2024 00:39
Send private message quote this post


the 3 ipv6 hosts i was smokepinging are now responding.... and so it seems are my google home devices!



Well well… I’ll keep an eye on mine and see how we go! Mind sharing which servers/IPs you are pinging?

Really could be two things at play then?
1. Quic having some IPv6 routing issues (obviously being actively worked on)
2. Google devices just being totally shit at falling back to IPv4 gracefully if they aren’t getting a response via IPv6…?

michaelmurfy
meow
12654 posts

Uber Geek

Moderator
ID Verified
Trusted
Lifetime subscriber

  #3187094 27-Jan-2024 01:53
Send private message quote this post

@pc @bartong That's quite interesting. From my own testing (between beta and a friend on Prod) there is only 1 destination I can't reach and they know about it. Google works fine over IPv6 here so must be related to your connections or the BNG you're on. I've done some hunting around on Smokeping and IPv6 is looking OK currently. Mind letting me know what destinations are failing as I'll ensure I'm monitoring it.

 

I believe currently there is a bit of a hack going on currently as they're still running 2 networks (their old one, and their new one). I think IPv6 is still a bit of a hack as they can't announce the prefixes from both networks (I remember them talking about this) but it is expected that IPv6 will be way more stable once they've said goodbye to their old network and fully sorted out all their peering.

 

If you look at the peering list at https://bgp.he.net/country/NZ Vetta were first in NZ for the most BGP peers up until now (for some reason Mercury is showing first tonight which is strange). This would add more things to tidy up I'd say to ensure traffic is going to the right places.





Michael Murphy | https://murfy.nz
Referral Links: Octopus Energy ($50 Credit) | Tesla | Quic Broadband (use R122101E7CV7Q for free setup)

Are you happy with what you get from Geekzone? Please consider supporting us by subscribing.
Opinions are my own and not the views of my employer.




bartong
33 posts

Geek


  #3187247 27-Jan-2024 12:53
Send private message quote this post

@michaelmurfy I have not had the opportunity to set up a smokeping yet, but it does appear my Google speakers are responding with near 100% reliability now so perhaps recent IPv6 routing changes/improvements are making the difference. I’m on bng2, on the Enable network here in Chch.

lucasnz
43 posts

Geek


  #3187279 27-Jan-2024 13:51
Send private message quote this post

IPv6 to Google and Facebook returned for me at 2am on Fri morning for me (according to my smokeping). 


pc

pc
104 posts

Master Geek

Trusted
Subscriber

  #3187310 27-Jan-2024 14:42
Send private message quote this post

 

 

ipv6 sites that were down were:

 

google dns: 2001:4860:4860::8888

 

youtube: 2404:6800:4006:809::200e

 

london host from dotcom-monitor : 2a01:4020:0:2e::10

 

 

 

sites that were ok:

 

cloudflare dns: 2606:4700:4700::1001

 

other dotcom-monitor sites from california and new york: 2001:470:1:1ab::2 and 2604:eb80:1:6::10

 

azure: portal.azure.com

 

 

 

my outage was from approx Mon 22nd 18:30 - Fri 26th 01:40

 

 

 

 

google nest was able to time my cup of tea this morning so all good here!


RunningMan
8213 posts

Uber Geek


  #3187806 28-Jan-2024 21:31
Send private message quote this post

michaelmurfy:

 

[snip] Something else I thought I'd mention is I am using Quic's DNS servers (Primary: 103.250.90.8, Secondary: 202.179.141.8) on my Smokeping instance that can be rather important for some CDN stuff especially Akamai. 

 

 

Not sure if Apple still use Akamai, but using something other than Quic's DNS has a significant detrimental impact on their software updates. Cloudflare DNS had updates coming through at about 15 Mb/s, far quicker with Quic DNS.


michaelmurfy
meow
12654 posts

Uber Geek

Moderator
ID Verified
Trusted
Lifetime subscriber

  #3187827 28-Jan-2024 22:54
Send private message quote this post

@pc Cheers! All those work fine on all my connections and I didn't record any outages on 3 different connections (1 beta, 2 production). I think there was a routing issue but as I said above their IPv6 is a little bit of a hack currently given they're advertising prefixes on 2 different networks. This will improve when they're back to a single network and there is a change for next week to install more route reflectors which should make things a bit better.

 

Suggestion though - do double-check your IPv6 MTU. Quic should be using the full 1500 byte MTU. I'm going to ping @fe31nz who knows a heck of a load more than me on this subject (and I always look forward to his late night posts).

 

 





Michael Murphy | https://murfy.nz
Referral Links: Octopus Energy ($50 Credit) | Tesla | Quic Broadband (use R122101E7CV7Q for free setup)

Are you happy with what you get from Geekzone? Please consider supporting us by subscribing.
Opinions are my own and not the views of my employer.


michaelmurfy
meow
12654 posts

Uber Geek

Moderator
ID Verified
Trusted
Lifetime subscriber

  #3187833 28-Jan-2024 23:04
Send private message quote this post

RunningMan: Not sure if Apple still use Akamai, but using something other than Quic's DNS has a significant detrimental impact on their software updates. Cloudflare DNS had updates coming through at about 15 Mb/s, far quicker with Quic DNS.

 

Apple use a combo of Akamai and ISP caching. Amusingly, their caching hardware is just standard HPe 2U servers filled with SSD's. I'm pretty sure there are caches available over NZIX but yeah it's a bit of a trade-off by not using your ISP's DNS servers most of the times with where you get your content from.

 

I did find NextDNS to be pretty good at getting somewhat close to your ISP's caches while protecting your privacy though if you want to give that a shot. I did get ~40 Mb/s over that just before doing some Apple TV updates.





Michael Murphy | https://murfy.nz
Referral Links: Octopus Energy ($50 Credit) | Tesla | Quic Broadband (use R122101E7CV7Q for free setup)

Are you happy with what you get from Geekzone? Please consider supporting us by subscribing.
Opinions are my own and not the views of my employer.


RunningMan
8213 posts

Uber Geek


  #3187986 29-Jan-2024 09:25
Send private message quote this post

As an example, using Cloudflare DNS you get Apple content from Sydney via Vocus which was hopelessly slow.


bartong
33 posts

Geek


  #3187993 29-Jan-2024 09:59
Send private message quote this post

Wow I had not realised that using non-ISP DNS servers could have such an impact on content delivery.

 

My kids were waiting for an update for Mariokart to download on the Nintendo Switch yesterday and it took nearly an hour, which I thought was ridiculous, so I guess this could explain?

 

I'm using Unbound recursive DNS resolver (as an upstream for AdGuard Home) so I assume that would suffer the same as using Cloudflare DNS, Quad9, etc since it won't play nice with ISP caching and/or CDNs?


1 | 2 | 3 | 4 | 5 | 6
View this topic in a long page with up to 500 replies per page Create new topic





News and reviews »

One New Zealand Extends 3G Switch-off Date
Posted 11-Apr-2024 08:56


Amazon Echo Hub Review
Posted 10-Apr-2024 18:57


Epson Launches New Versatile A4 Desktop Scanners
Posted 10-Apr-2024 15:31


Motorola Mobility Launches New Android Phones in New Zealand
Posted 10-Apr-2024 14:59


Logitech G Unveils the PRO X 60 Gaming Keyboard
Posted 9-Apr-2024 19:01


Logitech Unveils Signature Slim Keyboard and Combo
Posted 9-Apr-2024 13:33


ExpressVPN Launches Aircove Go Portable Router With Built-in VPN
Posted 26-Mar-2024 21:25


Shure MoveMic Review
Posted 25-Mar-2024 12:47


reMarkable 2 Launches at JB Hi-Fi New Zealand
Posted 20-Mar-2024 08:36


Samsung Galaxy S24 Ultra review
Posted 19-Mar-2024 11:37


Google Nest Wifi Pro Review
Posted 16-Mar-2024 11:28


Samsung Galaxy A55 5G and Galaxy A35 5G
Posted 12-Mar-2024 12:41


Cricut EasyPress Mini Zen Blue launches at Spotlight New Zealand
Posted 12-Mar-2024 12:32


Logitech Introduces MX Brio Webcam
Posted 12-Mar-2024 12:24


HP Unveils Broadest Consumer Portfolio of AI-Enhanced Laptops
Posted 3-Mar-2024 18:09









Geekzone Live »

Try automatic live updates from Geekzone directly in your browser, without refreshing the page, with Geekzone Live now.



Are you subscribed to our RSS feed? You can download the latest headlines and summaries from our stories directly to your computer or smartphone by using a feed reader.







Backblaze unlimited backup