2DE upload speed dropped from 500Mbit to 50Mbit

Forums › 2degrees (including Slingshot, Orcon, Flip, Stuff Fibre, MyRepublic, 2talk and Vocus) › 2DE upload speed dropped from 500Mbit to 50Mbit

1 | 2 | 3

497 posts

Ultimate Geek

ID Verified

Trusted

Lifetime subscriber

#3335947 26-Jan-2025 15:24

Traces:

(I'll put the top one in a script)

traceroute to 104.21.16.1 (104.21.16.1), 30 hops max, 60 byte packets
1 * * *
2 partner-network-device.nflxvideo.net (45.57.64.23) 11.556 ms 11.682 ms 11.712 ms
3 198.41.236.12 (198.41.236.12) 15.898 ms 15.912 ms 15.832 ms
4 104.21.16.1 (104.21.16.1) 11.222 ms 11.227 ms 12.782 ms

traceroute to sp.k99.net.nz (103.27.217.95), 30 hops max, 60 byte packets
1 * * *
2 * * *
3 198.41.236.22 (198.41.236.22) 11.722 ms 11.851 ms 11.803 ms
4 198.41.236.90 (198.41.236.90) 11.277 ms 172.69.0.24 (172.69.0.24) 11.489 ms 198.41.236.83 (198.41.236.83) 11.119 ms
5 172.69.0.30 (172.69.0.30) 11.347 ms 172.69.0.27 (172.69.0.27) 11.013 ms 172.69.0.22 (172.69.0.22) 11.318 ms
6 198.41.236.95 (198.41.236.95) 11.026 ms 172.69.0.6 (172.69.0.6) 11.628 ms 198.41.236.84 (198.41.236.84) 11.029 ms
7 103.27.217.95 (103.27.217.95) 12.377 ms !X 12.430 ms !X 12.514 ms !X

traceroute to akl1.linetest.nz (101.98.9.197), 30 hops max, 60 byte packets
1 * * *
2 default-rdns.vocus.co.nz (101.98.5.117) 10.905 ms 10.846 ms 10.788 ms
3 default-rdns.vocus.co.nz (101.98.9.197) 10.742 ms 10.696 ms 10.761 ms

traceroute to aklbroadbandcheck.sky.co.nz (202.127.8.5), 30 hops max, 60 byte packets
1 * * *
2 default-rdns.vocus.co.nz (101.98.9.208) 10.286 ms 10.292 ms 10.281 ms
3 networks.skybb.net.nz (202.127.8.5) 9.454 ms 9.728 ms 9.771 ms

yitz

2074 posts

Uber Geek

#3335948 26-Jan-2025 15:30

olivernz:

Traces:

(I'll put the top one in a script)

You should run the traceroute to the relevant speed test server host alongside the speed test itself, so if you run a speed test to Chorus Fibre Labs then run the traceroute to sp.k99.net.nz etc. (you'll notice hops like 198.41.236.22 are for Cloudflare)

olivernz

497 posts

Ultimate Geek

ID Verified

Trusted

Lifetime subscriber

#3336122 26-Jan-2025 23:34

Ok, so if I test to the Chorus linetest boxes I get (Upload) a different value for UDP and TCP. TCP also shows several retransmits.

UDP

iperf3 -c akl4.linetest.nz -V -u -p 5001 -b 990000000
iperf 3.16
Linux UDMPRO 4.19.152-ui-alpine #4.19.152 SMP Tue Dec 24 22:57:14 CST 2024 aarch64
Control connection MSS 1448
Setting UDP block size to 1448
Time: Sun, 26 Jan 2025 10:30:20 GMT
Connecting to host akl4.linetest.nz, port 5001
Cookie: q26spycw6wcmz7zh5i2qiadfyfzcjcuk5apr
Target Bitrate: 990000000
[ 5] local 163.xxx.xxx.xxx port 58780 connected to 60.234.3.9 port 5001
Starting Test: protocol: UDP, 1 streams, 1448 byte blocks, omitting 0 seconds, 10 second test, tos 0
[ ID] Interval Transfer Bitrate Total Datagrams
[ 5] 0.00-1.00 sec 112 MBytes 941 Mbits/sec 81243
[ 5] 1.00-2.00 sec 112 MBytes 939 Mbits/sec 81027
[ 5] 2.00-3.00 sec 112 MBytes 938 Mbits/sec 81007
[ 5] 3.00-4.00 sec 112 MBytes 943 Mbits/sec 81440
[ 5] 4.00-5.00 sec 113 MBytes 951 Mbits/sec 82128
[ 5] 5.00-6.00 sec 113 MBytes 950 Mbits/sec 82020
[ 5] 6.00-7.00 sec 112 MBytes 937 Mbits/sec 80908
[ 5] 7.00-8.00 sec 111 MBytes 934 Mbits/sec 80656
[ 5] 8.00-9.00 sec 111 MBytes 929 Mbits/sec 80218
[ 5] 9.00-10.00 sec 111 MBytes 927 Mbits/sec 80036
- - - - - - - - - - - - - - - - - - - - - - - - -
Test Complete. Summary Results:
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-10.00 sec 1.09 GBytes 939 Mbits/sec 0.000 ms 0/810683 (0%) sender
[ 5] 0.00-10.00 sec 565 MBytes 474 Mbits/sec 0.021 ms 0/810683 (0%) receiver
CPU Utilization: local/sender 93.9% (7.0%u/86.8%s), remote/receiver 2.6% (0.6%u/2.0%s)

TCP

iperf3 -c akl4.linetest.nz -V -p 5001 -b 990000000
iperf 3.16
Linux UDMPRO 4.19.152-ui-alpine #4.19.152 SMP Tue Dec 24 22:57:14 CST 2024 aarch64
Control connection MSS 1448
Time: Sun, 26 Jan 2025 10:25:14 GMT
Connecting to host akl4.linetest.nz, port 5001
Cookie: xr2ug3li5z7md5i3ybhxmqqvmmppdjwi3epl
TCP MSS: 1448 (default)
Target Bitrate: 990000000
[ 5] local 163.xxx.xxx.xxx port 42660 connected to 60.234.3.9 port 5001
Starting Test: protocol: TCP, 1 streams, 131072 byte blocks, omitting 0 seconds, 10 second test, tos 0
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 384 KBytes 3.15 Mbits/sec 35 2.83 KBytes
[ 5] 1.00-2.00 sec 256 KBytes 2.10 Mbits/sec 21 5.66 KBytes
[ 5] 2.00-3.00 sec 256 KBytes 2.10 Mbits/sec 22 4.24 KBytes
[ 5] 3.00-4.00 sec 256 KBytes 2.10 Mbits/sec 20 2.83 KBytes
[ 5] 4.00-5.00 sec 0.00 Bytes 0.00 bits/sec 13 2.83 KBytes
[ 5] 5.00-6.00 sec 256 KBytes 2.10 Mbits/sec 20 2.83 KBytes
[ 5] 6.00-7.00 sec 128 KBytes 1.05 Mbits/sec 17 4.24 KBytes
[ 5] 7.00-8.00 sec 256 KBytes 2.10 Mbits/sec 26 2.83 KBytes
[ 5] 8.00-9.00 sec 128 KBytes 1.05 Mbits/sec 13 1.41 KBytes
[ 5] 9.00-10.00 sec 0.00 Bytes 0.00 bits/sec 1 1.41 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
Test Complete. Summary Results:
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 1.88 MBytes 1.57 Mbits/sec 188 sender
[ 5] 0.00-10.00 sec 1.80 MBytes 1.51 Mbits/sec receiver
CPU Utilization: local/sender 1.0% (1.0%u/0.0%s), remote/receiver 0.0% (0.0%u/0.0%s)
snd_tcp_congestion cubic

iperf Done.

BMarquis

445 posts

Ultimate Geek

Trusted

Chorus

Lifetime subscriber

#3336133 27-Jan-2025 07:59

Hey @olivernz - This wont help with your problem, but might help with your troubleshooting path...
linetest.nz is nothing to do with Chorus. I dont know where that is located or who runs it.

I see you and @deckstarr are both in Wellington. Can you both please PM me your ONT serial number (starts with ALCL)
I'll investigate the Chorus side and check there is nothing common in our network between both of your services.

Cheers!

CYaBro

4582 posts

Uber Geek

ID Verified

Trusted

#3336142 27-Jan-2025 09:02

Not as bad as others in this thread but I too am now seeing much lower upload speeds, and higher latency, since Thursday afternoon.
Download speeds haven't really changed.

Speedtest to 2degrees server in Christchurch, from rural Oamaru.

Opinions are my own and not the views of my employer.

olivernz

497 posts

Ultimate Geek

ID Verified

Trusted

Lifetime subscriber

#3336143 27-Jan-2025 09:25

CYaBro:

Not as bad as others in this thread but I too am now seeing much lower upload speeds, and higher latency, since Thursday afternoon.
Download speeds haven't really changed.

Speedtest to 2degrees server in Christchurch, from rural Oamaru.

What provider are you on? Guessing 2degrees?

CYaBro

4582 posts

Uber Geek

ID Verified

Trusted

#3336144 27-Jan-2025 09:27

olivernz:

CYaBro:

Not as bad as others in this thread but I too am now seeing much lower upload speeds, and higher latency, since Thursday afternoon.
Download speeds haven't really changed.

Speedtest to 2degrees server in Christchurch, from rural Oamaru.

What provider are you on? Guessing 2degrees?

Yup 2D.
Looks like our connection is going via Auckland now for some reason.
Speedtests to Auckland has much lower download speeds but uploads are over 500Mbps.

Opinions are my own and not the views of my employer.

Equinox IT

Cloud spending continues to surge globally, but most organisations haven’t made the changes necessary to maximise the value and cost-efficiency benefits of their cloud investments. Download the whitepaper From Overspend to Advantage now.

BMarquis

445 posts

Ultimate Geek

Trusted

Chorus

Lifetime subscriber

#3336145 27-Jan-2025 09:35

I think yours is different @cyabro - that looks like a clear path/routing change in 2D. Oliver seem to have the expected latency based on the earlier posts.

olivernz

497 posts

Ultimate Geek

ID Verified

Trusted

Lifetime subscriber

#3336172 27-Jan-2025 11:55

Here is the workup on my results:
https://docs.google.com/spreadsheets/d/1tYkfD0_utthiduK9KuXXgoBzct3sNMUSnYfrhyTq9f0/edit?usp=sharing

Certainly not perfect and there are some gaps in the data but should show what is going on.

olivernz

497 posts

Ultimate Geek

ID Verified

Trusted

Lifetime subscriber

#3336187 27-Jan-2025 12:34

And I have just retested (manually) with Speedtest to a dozen or so servers and it seems to be getting proper upload now (and I am touching wood & whatever else to appease the gods). So let's hope this saga is solved. No idea if something was changed.

The only slow upload I have is to "Chorus Fibre Lab" but that could be something else.

yitz

2074 posts

Uber Geek

#3336196 27-Jan-2025 12:53

olivernz:

The only slow upload I have is to "Chorus Fibre Lab" but that could be something else.

The traceroute makes it look like traffic to that Chorus Fibre Lab server is passing through a Cloudflare DDoS scrubbing centre or clean pipe service. I have not seen that happen to a speed test server before.

olivernz

497 posts

Ultimate Geek

ID Verified

Trusted

Lifetime subscriber

#3336198 27-Jan-2025 12:58

yitz:

The traceroute makes it look like traffic to that Chorus Fibre Lab server is passing through a Cloudflare DDoS scrubbing centre or clean pipe service. I have not seen that happen to a speed test server before.

Oh I am sure security guys said "you can't have that! You need protection". LOL. Famous last words.

deckstarr

16 posts

Geek

#3336404 27-Jan-2025 17:07

Still at around 6.6Mbps upload now, this is quite annoying

olivernz

497 posts

Ultimate Geek

ID Verified

Trusted

Lifetime subscriber

#3336430 27-Jan-2025 18:56

I should stop proclaiming success. This is frigging weird as. Look at my google sheet. Look at the graphs now! This just doesn't look normal at all!

MaxineN

Max
1772 posts

Uber Geek

ID Verified

Trusted

Subscriber

#3336439 27-Jan-2025 19:10

BMarquis:

Hey @olivernz - This wont help with your problem, but might help with your troubleshooting path...
linetest.nz is nothing to do with Chorus. I dont know where that is located or who runs it.

I see you and @deckstarr are both in Wellington. Can you both please PM me your ONT serial number (starts with ALCL)
I'll investigate the Chorus side and check there is nothing common in our network between both of your services.

Cheers!

linetest.nz is Vocus.

It was the old way of bandwidth testing for Orcon and Slingshot customers (and well they own the 2degree base and they're on the same network).

Ramblings from a mysterious lady who's into tech. Warning I may often create zingers.

1 | 2 | 3