Recently MTU related issues?

Forums › 2degrees (including Slingshot, Orcon, Flip, Stuff Fibre, MyRepublic, 2talk and Vocus) › Recently MTU related issues?

1 | 2 | 3 | 4 | 5 | 6 | 7 | 8

6 posts

Wannabe Geek

#3329415 9-Jan-2025 02:43

Yep, I'm also running an older Unifi USG, and setting MSS on the USG via the GUI didn't do anything.

As for seeing larger packets in Wireshark, that doesn't look great, but it likely is being split after Wireshark captures it since I'm capturing on the device itself before they are sent out.

I would also expect that if MTU/MSS was actually the issue, I'd be running into a lot of other weird issues but the only thing affected seems to be github.io

Why is this an issue now? And given how many people seem to be hitting the same issue, it makes me think it's an upstream change at github/fastly, or our ISP.

openmedia

3332 posts

Uber Geek

Trusted

#3330972 13-Jan-2025 13:54

Looks like a user on snapshot having similar issues

https://www.geekzone.co.nz/forums.asp?forumid=81&topicid=318387&page_no=1#3330967

Generally known online as OpenMedia, now working for Red Hat APAC as a Technology Evangelist and Portfolio Architect. Still playing with MythTV and digital media on the side.

yitz

2080 posts

Uber Geek

#3330973 13-Jan-2025 14:08

openmedia:

Looks like a user on snapshot having similar issues

Same traceroute hops 101.98.5.93 so I would say related to DIA filtering (i.e. the "upstream issue") either the filter itself or somewhere on the path to (load balancing, tunneling) or the path back (Fastly CDN bad request or asymmetric routing issue).

boosacnoodle

963 posts

Ultimate Geek

#3330977 13-Jan-2025 14:35

yitz:
openmedia:

Looks like a user on snapshot having similar issues

Same traceroute hops 101.98.5.93 so I would say related to DIA filtering (i.e. the "upstream issue") either the filter itself or somewhere on the path to (load balancing, tunneling) or the path back (Fastly CDN bad request or asymmetric routing issue).

How does the filter even work nowadays? With everything encrypted, DPI couldn’t happen (or so I thought).

insane

3240 posts

Uber Geek

ID Verified

Trusted

#3330979 13-Jan-2025 14:58

boosacnoodle:
yitz:
openmedia:

Looks like a user on snapshot having similar issues

Same traceroute hops 101.98.5.93 so I would say related to DIA filtering (i.e. the "upstream issue") either the filter itself or somewhere on the path to (load balancing, tunneling) or the path back (Fastly CDN bad request or asymmetric routing issue).

How does the filter even work nowadays? With everything encrypted, DPI couldn’t happen (or so I thought).

It used to be an IP address list, and advertised a /32 via BGP to draw traffic initiated by a DNS request towards it.

No idea what it does or doesn't do now.

openmedia

3332 posts

Uber Geek

Trusted

#3331206 14-Jan-2025 09:46

yitz:

openmedia:

Looks like a user on snapshot having similar issues

Same traceroute hops 101.98.5.93 so I would say related to DIA filtering (i.e. the "upstream issue") either the filter itself or somewhere on the path to (load balancing, tunneling) or the path back (Fastly CDN bad request or asymmetric routing issue).

So why is it impacting 2Degrees but not Spark?

Generally known online as OpenMedia, now working for Red Hat APAC as a Technology Evangelist and Portfolio Architect. Still playing with MythTV and digital media on the side.

yitz

2080 posts

Uber Geek

#3331602 14-Jan-2025 21:31

openmedia:

So why is it impacting 2Degrees but not Spark?

Your guess is as good as mine but as I suggested above it could possibly be a load balancing problem on the way to the filter infrastructure seeing as traffic is tunneled per ISP.

Only publicly accessible graphs I could find are the NSW-IX traffic graphs for packets that make it out the other end at the below bold hops

From Rudster's provided traceroute:
traceroute to 185.199.108.153 (185.199.108.153), 30 hops max, 60 byte packets
1 _gateway (192.168.10.1) 0.259 ms 0.230 ms 0.259 ms
2 * * *
3 ext.cpcak4-r1.tranzpeer.net (101.98.0.66) 1.713 ms 1.773 ms 1.766 ms
4 default-rdns.vocus.co.nz (101.98.5.93) 14.306 ms 16.089 ms 16.081 ms
5 * * *
6 124.150.165.62 (124.150.165.62) 38.647 ms 37.605 ms 37.573 ms
7 124.150.165.2 (124.150.165.2) 39.124 ms 38.717 ms 40.312 ms
8 as18407.nsw.ix.asn.au (218.100.52.44) 40.045 ms 40.572 ms 40.023 ms
9 45.64.201.2 (45.64.201.2) 38.671 ms 39.781 ms 40.536 ms

Last 7 days

Last 60 days

Seems to cap out for most of the day at just under 100 Mbps like there is a Fast Ethernet interface somewhere along the path. I'm not suggesting this is where the main bottleneck lies though.

Quic Broadband

Move to New Zealand's best fibre broadband service (affiliate link). Free setup code: R587125ERQ6VE. Note that to use Quic Broadband you must be comfortable with configuring your own router.

yitz

2080 posts

Uber Geek

#3331634 15-Jan-2025 00:09

boosacnoodle: How does the filter even work nowadays? With everything encrypted, DPI couldn’t happen (or so I thought).

I did notice a request taking so long my browser retried with plain HTTP so I do wonder if inducing HTTPS downgrade is part of the arsenal.

michaelmurfy

meow
13254 posts

Uber Geek

Moderator

ID Verified

Trusted

Lifetime subscriber

#3331635 15-Jan-2025 02:20

yitz:

boosacnoodle: How does the filter even work nowadays? With everything encrypted, DPI couldn’t happen (or so I thought).

I did notice a request taking so long my browser retried with plain HTTP so I do wonder if inducing HTTPS downgrade is part of the arsenal.

This is nothing to do with the filter (which is route based) nor is it with any proxy.

Not having this issue on other ISP's, I'm not able to replicate this on my Quic connection but I can replicate it on a 2degrees connection. It's something funky with 2degrees.

Michael Murphy | https://murfy.nz
Referral Links: Quic Broadband (use R122101E7CV7Q for free setup)

Are you happy with what you get from Geekzone? Please consider supporting us by subscribing.
Opinions are my own and not the views of my employer.

openmedia

3332 posts

Uber Geek

Trusted

#3331649 15-Jan-2025 08:02

yitz:

openmedia:

So why is it impacting 2Degrees but not Spark?

Your guess is as good as mine but as I suggested above it could possibly be a load balancing problem on the way to the filter infrastructure seeing as traffic is tunneled per ISP.

Only publicly accessible graphs I could find are the NSW-IX traffic graphs for packets that make it out the other end at the below bold hops

From Rudster's provided traceroute:
traceroute to 185.199.108.153 (185.199.108.153), 30 hops max, 60 byte packets
1 _gateway (192.168.10.1) 0.259 ms 0.230 ms 0.259 ms
2 * * *
3 ext.cpcak4-r1.tranzpeer.net (101.98.0.66) 1.713 ms 1.773 ms 1.766 ms
4 default-rdns.vocus.co.nz (101.98.5.93) 14.306 ms 16.089 ms 16.081 ms
5 * * *
6 124.150.165.62 (124.150.165.62) 38.647 ms 37.605 ms 37.573 ms
7 124.150.165.2 (124.150.165.2) 39.124 ms 38.717 ms 40.312 ms
8 as18407.nsw.ix.asn.au (218.100.52.44) 40.045 ms 40.572 ms 40.023 ms
9 45.64.201.2 (45.64.201.2) 38.671 ms 39.781 ms 40.536 ms

Same trace on my Orcon connection

traceroute 185.199.108.153
traceroute to 185.199.108.153 (185.199.108.153), 30 hops max, 60 byte packets
1 _gateway (192.168.0.254) 0.630 ms 0.543 ms 0.574 ms
2 * * *
3 ext.cpcak4-r1.tranzpeer.net (101.98.0.66) 9.919 ms 9.886 ms 9.926 ms
4 default-rdns.vocus.co.nz (101.98.5.93) 13.048 ms 13.021 ms *
5 * * *
6 * 124.150.165.62 (124.150.165.62) 39.809 ms *
7 * * *
8 * * *
9 * 45.64.201.2 (45.64.201.2) 38.210 ms 38.154 ms
10 * * *
11 * * *
12 * * *
13 * * *
14 * * *
15 * * *
16 * * *
17 * * *
18 * * *
19 * * *
20 * * *
21 * * *
22 * * *
23 * * *
24 * * *
25 * * *
26 * * *
27 * * *
28 * * *
29 * * *
30 * * *

Generally known online as OpenMedia, now working for Red Hat APAC as a Technology Evangelist and Portfolio Architect. Still playing with MythTV and digital media on the side.

saf

160 posts

Master Geek

ID Verified

Trusted

Vetta Group

Subscriber

#3331659 15-Jan-2025 08:49

If it's the DIA DCEFS filter actually filtering, requests shouldn't time out if the filter is implemented correctly.

Instead, you should be confronted with a page similar to the below:

As other have said, if it was an IP subject to the DIA DCEFS filter, it's the same filter for all ISPs - meaning the same behaviour would be exhibited with all ISPs using the filter.

If however, hypothetically, an IP or subnet was a stuck route, or a route incorrectly directed at the filter somehow, that would impact just that ISP, and could expect timeouts on requests as the filter isn't expecting the traffic.

Not saying this is what's happening, but if traffic is ending up at the filter for a single ISP, and those requests are timing out, it's a possibility.

My views are as unique as a unicorn riding a unicycle. They do not reflect the opinions of my employer, my cat, or the sentient coffee machine in the break room.

jnimmo

1097 posts

Uber Geek

#3331663 15-Jan-2025 09:09

Out of interest, does disabling the `TLS 1.3 post-quantum key agreement` flag in Chrome fix the issue?

chrome://flags/#enable-tls13-kyber

yitz

2080 posts

Uber Geek

#3331727 15-Jan-2025 10:08

openmedia:

Same trace on my Orcon connection

Keep in mind that could be showing one of multiple load balanced paths that is working. For me, sometimes no further hops appear after 101.98.5.94/93 and potentially this could point to an alternate non-working path.

Asymmetric routing could potentially be an issue here when the request comes in from one ISP (in the traceroutes we can see the filter upstream appears to be Devoli in Sydney) then the response is to be sent to another (in this case directly back to 2degrees) then this could be interpreted as unsolicited traffic. A large CDN like Fastly would need to be careful handling such requests, so I do wonder whether a second filter upstream has been added but associated policy rules or filtering has not been updated to include 2degrees.

Seeing as other working ISPs like Spark are also steering the same /32 IPv4s toward the filter (you can check adjacent IP's via traceroute take a completely different path) I doubt this is a case of invalid/stale route on the 2degrees end.

This is not the first time this has happened on 2degrees either: e.g. https://www.geekzone.co.nz/forums.asp?forumid=85&topicid=280249

openmedia

3332 posts

Uber Geek

Trusted

#3331777 15-Jan-2025 13:44

via my Skinny (Spark) phone on tether

$ ping 185.199.108.153
PING 185.199.108.153 (185.199.108.153) 56(84) bytes of data.
64 bytes from 185.199.108.153: icmp_seq=1 ttl=56 time=83.1 ms
64 bytes from 185.199.108.153: icmp_seq=2 ttl=56 time=57.5 ms
64 bytes from 185.199.108.153: icmp_seq=3 ttl=56 time=51.6 ms
64 bytes from 185.199.108.153: icmp_seq=4 ttl=56 time=73.5 ms
64 bytes from 185.199.108.153: icmp_seq=5 ttl=56 time=57.6 ms
64 bytes from 185.199.108.153: icmp_seq=6 ttl=56 time=71.2 ms

$ traceroute 185.199.108.153
traceroute to 185.199.108.153 (185.199.108.153), 30 hops max, 60 byte packets
1 _gateway (192.168.115.96) 2.979 ms 3.066 ms 3.152 ms
2 10.207.86.169 (10.207.86.169) 37.369 ms 43.114 ms 43.067 ms
3 * * *
4 * * *
5 cdn-185-199-108-153.github.com (185.199.108.153) 47.704 ms 47.550 ms 47.763 ms
6 124.150.165.62 (124.150.165.62) 75.317 ms 195.583 ms 77.476 ms
7 124.150.165.2 (124.150.165.2) 77.428 ms 61.308 ms 57.582 ms
8 218.100.52.44 (218.100.52.44) 181.557 ms 52.658 ms 229.513 ms
9 45.64.201.2 (45.64.201.2) 77.347 ms 59.821 ms 192.034 ms
10 * * *
11 * * *
12 * * *
13 * * *
14 * * *
15 * * *
16 * * *
17 * * *
18 * * *
19 * * *
20 * * *
21 * * *
22 * * *
23 * * *
24 * * *
25 * * *
26 * * *
27 * * *
28 * * *
29 * * *
30 * * *

Generally known online as OpenMedia, now working for Red Hat APAC as a Technology Evangelist and Portfolio Architect. Still playing with MythTV and digital media on the side.

openmedia

3332 posts

Uber Geek

Trusted

#3331778 15-Jan-2025 13:46

Back on Orcon / 2Degrees some serious packet loss

ping 185.199.108.153
PING 185.199.108.153 (185.199.108.153) 56(84) bytes of data.
64 bytes from 185.199.108.153: icmp_seq=8 ttl=60 time=147 ms
64 bytes from 185.199.108.153: icmp_seq=9 ttl=60 time=245 ms
64 bytes from 185.199.108.153: icmp_seq=10 ttl=60 time=183 ms
64 bytes from 185.199.108.153: icmp_seq=11 ttl=60 time=104 ms
64 bytes from 185.199.108.153: icmp_seq=13 ttl=60 time=87.1 ms
64 bytes from 185.199.108.153: icmp_seq=21 ttl=60 time=73.5 ms
64 bytes from 185.199.108.153: icmp_seq=22 ttl=60 time=76.3 ms
64 bytes from 185.199.108.153: icmp_seq=24 ttl=60 time=68.3 ms
64 bytes from 185.199.108.153: icmp_seq=60 ttl=60 time=346 ms
64 bytes from 185.199.108.153: icmp_seq=61 ttl=60 time=336 ms
64 bytes from 185.199.108.153: icmp_seq=69 ttl=60 time=524 ms

--- 185.199.108.153 ping statistics ---
83 packets transmitted, 11 received, 86.747% packet loss, time 83810ms
rtt min/avg/max/mdev = 68.343/199.187/524.099/141.600 ms

Generally known online as OpenMedia, now working for Red Hat APAC as a Technology Evangelist and Portfolio Architect. Still playing with MythTV and digital media on the side.

1 | 2 | 3 | 4 | 5 | 6 | 7 | 8