Looking for long distance hyperfibre benchmarks

Forums › New Zealand Broadband › Looking for long distance hyperfibre benchmarks

dzh

50 posts

Geek

Trusted

#306460 24-Jul-2023 19:35

I'm on a quest to understand more about international contention ratio and how that affects fibre and hyperfibre connections.

Hope members can run a couple of benchmarks for me.

Could you run speedtest.net (using Connections: Single setting) to any of the following,
* Oslo - Telenor Norge AS

* Rochester - Rochester Institute of Technology

Could you please list
* Observed speed

* ISP

* Plan

* Equipment/connection type (n, ac, ax, ethernet)

1 | 2

50 posts

Geek

Trusted

#3107687 24-Jul-2023 19:43

Alternatively you can try librespeed (it's the only one that allows long distance, single connection mode - others either don't support long distance tests or don't support single connection testing)

Atlanta
> librespeed-cli --concurrent 1 --duration 120 --server 53

Amsterdam

> librespeed-cli --concurrent 1 --duration 120 --server 51

eonsim

398 posts

Ultimate Geek

Trusted

#3107707 24-Jul-2023 20:56

Rochester - Rochester Institute of Technology

Ethernet, Orcon/2degrees, D106, U18 20:52

Oslo - Telenor Norge AS

Single: Ethernet, Orcon/2degrees, D293, U13 20:52

Multi gives: D867, U56

950/500 - Fibre Max?

danfaulknor

939 posts

Ultimate Geek

Trusted

Prodigi

#3107711 24-Jul-2023 21:01

It was my understanding that latency was the thing that impacts performance here, not contention ratios? With multi-connection we get a number of gigabits to international locations.

These are my results from a server connected to our core at 20G(2x10G), and no congestion that I know of on our international links.

./librespeed-cli --concurrent 1 --duration 120 --server 53
Retrieving server list from https://librespeed.org/backend-servers/servers.php
Selected server: Atlanta, United States (Clouvider) [atl.speedtest.clouvider.net]
Sponsored by: Clouvider @ https://www.clouvider.co.uk/
You're testing from: 103.121.34.14 - Unknown ISP
Ping: 249 ms Jitter: 0 ms
Download rate: 49.32 Mbps
Upload rate: 26.11 Mbps

./librespeed-cli --concurrent 1 --duration 120 --server 51
Retrieving server list from https://librespeed.org/backend-servers/servers.php
Selected server: Amsterdam, Netherlands (Clouvider) [ams.speedtest.clouvider.net]
Sponsored by: Clouvider @ https://www.clouvider.co.uk/
You're testing from: 103.121.34.14 - Unknown ISP
Ping: 285 ms Jitter: 0 ms
Download rate: 41.26 Mbps
Upload rate: 15.10 Mbps

they/them

Prodigi - Optimised IT Solutions
WebOps/DevOps, Managed IT, Hosting and Internet/WAN.

dzh

50 posts

Geek

Trusted

#3107721 24-Jul-2023 22:03

danfaulknor:

It was my understanding that latency was the thing that impacts performance here, not contention ratios? With multi-connection we get a number of gigabits to international locations.

Latency is what affects how quickly speed ramps up, so yeah. On small downloads probably doesn't even matter.

Also not sure that librespeed is any good as it's consistently much slower than speedtest.net - sorry for suggesting this one :)

Talkiet

4793 posts

Uber Geek

Trusted

#3107722 24-Jul-2023 22:04

Yep, contention will likely have no play here. It's all about latency, choice of TCP stack parameters on each end and some other magic sauce that isn't user adjustable.

Cheers - N

Please note all comments are from my own brain and don't necessarily represent the position or opinions of my employer, previous employers, colleagues, friends or pets.

danfaulknor

939 posts

Ultimate Geek

Trusted

Prodigi

#3107730 24-Jul-2023 22:39

Auckland server (CLI only so I can't select single):

https://www.speedtest.net/result/c/a11aef9b-c57d-4472-bb93-0cb724a13941

https://www.speedtest.net/result/c/64689a1f-48d7-4ae9-91d6-452c60990686

Home (4G hyperfibre but there's a gremlin in my home network I haven't tracked down as yet limiting my speed so this is not a good example of our network performance at all!):

https://www.speedtest.net/result/15030147340

https://www.speedtest.net/result/15030155171

they/them

Prodigi - Optimised IT Solutions
WebOps/DevOps, Managed IT, Hosting and Internet/WAN.

concordnz

474 posts

Ultimate Geek

Trusted

EMT (R)

#3107825 25-Jul-2023 11:29

@talkiet
Does these results suggest that international traffic does not benefit from Hyperfibre?
(and in many cases can't even hit 300Mbp/s (less than 1gb fibre capacity)

I have an interest in the technicals of this, as I'm regularly sending & recieving large files internationally, and I'm trying to figure out how I can improve my throughput.

Quic Broadband

Move to New Zealand's best fibre broadband service (affiliate link). Free setup code: R587125ERQ6VE. Note that to use Quic Broadband you must be comfortable with configuring your own router.

Talkiet

4793 posts

Uber Geek

Trusted

#3107827 25-Jul-2023 11:34

concordnz: @talkiet
Does these results suggest that international traffic does not benefit from Hyperfibre?
(and in many cases can't even hit 300Mbp/s (less than 1gb fibre capacity)

I have an interest in the technicals of this, as I'm regularly sending & recieving large files internationally, and I'm trying to figure out how I can improve my throughput.

I haven't done the math, but I'm pretty confident saying unless you are massively multithreading, then no, there's no real benefit from hyperfibre for international connections. (possible exception for Aussie, but still unlikely to see any benefit above gig)... For North America, unless you stop using TCP or multithread significantly, you won't see performance even approaching a gig.

Some reading here.

https://netcraftsmen.com/tcp-performance-and-the-mathis-equation/https://netcraftsmen.com/tcp-performance-and-the-mathis-equation/

Cheers - N

Please note all comments are from my own brain and don't necessarily represent the position or opinions of my employer, previous employers, colleagues, friends or pets.

concordnz

474 posts

Ultimate Geek

Trusted

EMT (R)

#3107834 25-Jul-2023 11:49

Great, thanks for the link, I'll read through that 👌

And I'll stop hoping Hyperfibre might be my solution :)
(I understood it used larger 'packets' and I thought that might help....)

Vindy500

53 posts

Master Geek

Trusted

Chorus

#3107838 25-Jul-2023 11:58

danfaulknor:

Home (4G hyperfibre but there's a gremlin in my home network I haven't tracked down as yet limiting my speed so this is not a good example of our network performance at all!):

https://www.speedtest.net/result/15030147340

https://www.speedtest.net/result/15030155171

I got similar results for NY, but quite a bit worse to Oslo. This is on a 300 Mb connection, so I guess Hyperfibre maybe does something for international?

https://www.speedtest.net/result/15032883113

https://www.speedtest.net/result/15032879865

danfaulknor

939 posts

Ultimate Geek

Trusted

Prodigi

#3107843 25-Jul-2023 12:12

concordnz:
(I understood it used larger 'packets' and I thought that might help....)

"Standard" (non-business) Hyperfibre is the same MTU as normal UFB - 2000 but usually these run at 1500, as that is 'standard' and what 'normal' internet will run at.

You can get UFB connections with 9000 MTU which may help (for $$$$), and depending on where you are pushing files to, it's possible to access international transit with higher MTU but it requires end to end support. Our Cogent transit for example is 9000 MTU

I'm assuming there's a reason you can't do multi-connection uploads?

they/them

Prodigi - Optimised IT Solutions
WebOps/DevOps, Managed IT, Hosting and Internet/WAN.

darylblake

1162 posts

Uber Geek

Trusted

#3107845 25-Jul-2023 12:24

What are you trying to achieve here? What problem do you have and you are trying to solve?

Yes it will basically be the latency that will be causing the layer4 protocols to back off.

Me and my business partner are selling a product that will get you very high speed across the globe.

Have got about 3Gbit+ from Auckland to London copying data on a 4G Hyperfibre. We can do some demos if you want. It also works very well across high-latency satellite. Single copy stream. No software required to be installed. Everything is done at the network level. Much easier solution that multiple copy streams.

Product: www.tillered.com

fe31nz

1232 posts

Uber Geek

#3108112 26-Jul-2023 01:05

Is it a Windows system you are using? TCP performance on Windows is not good - it does not properly handle increasing the TCP window size to allow for increased latency over long distance connections. So on Windows you normally just use multithreaded downloads or uploads, so the performance may not be very good on any one thread which will spend most of its time waiting for ACKs, but overall with many TCP connections running most of the available bandwidth can be used. But it requires that the far end support multiple TCP connections. It is better though to just use Linux - there you should see the speed ramp up to full speed eventually, as the TCP window size gets increased, but it can take a minute or two. And it only takes one or two missing or damaged packets to slow things down again.

Unfortunately for us here in New Zealand, we always get big latencies to anywhere further than Australia simply due to the speed of light over the distances involved, let alone having to pass through more routers to get there. In most parts of the world, the places they are connecting to are much closer and they are not badly affected by the TCP window problem. So no-one seems to have ever done much thinking about a better system than TCP already uses, because there is no point for them.

If you were to write your own programs for doing the file transfers, it should be possible to program them to do an initial ping (or to do timing between sending the initial SYN and getting the SYN-ACK packet) to get an idea of the latency of the connection, and then from that calculate the best initial TCP window size and program the connection to start with that window size. That would avoid the long ramp up time while TCP works out the best window size.

And there are also tuning parameters for TCP stacks that can help with this.

concordnz

474 posts

Ultimate Geek

Trusted

EMT (R)

#3108138 26-Jul-2023 08:47

darylblake:
What are you trying to achieve here? What problem do you have and you are trying to solve?

Yes it will basically be the latency that will be causing the layer4 protocols to back off.

Me and my business partner are selling a product that will get you very high speed across the globe.

Have got about 3Gbit+ from Auckland to London copying data on a 4G Hyperfibre. We can do some demos if you want. It also works very well across high-latency satellite. Single copy stream. No software required to be installed. Everything is done at the network level. Much easier solution that multiple copy streams.

Product: www.tillered.com

Website talks about transparency, but is light on details.

Is this system over existing fibre? Or something else?
Do you have to pay for a new Ont & router or will it use existing?

cddt

1561 posts

Uber Geek

#3108173 26-Jul-2023 10:07

concordnz: @talkiet

I have an interest in the technicals of this, as I'm regularly sending & recieving large files internationally, and I'm trying to figure out how I can improve my throughput.

What application and protocol are you using to send/receive large files?

1 | 2