International Speed Comparison (ToD)

Forums › New Zealand Broadband › International Speed Comparison (ToD)

1 | 2

JohnButt

374 posts

Ultimate Geek

Trusted

#1026314 16-Apr-2014 16:35

mercutio: i don't think comparing peak time to off peak performance is necessarily a good comparison.

in the past there has been slowness with downloading over http off-peak with some ISP's with transparent proxies that reduce maximum tcp/ip performance for file transfers. (it probably doesn't make much difference for web browsing)

if one isp gives 400kb/sec peak, and 600kb/sec off-peak, and another one gives 1200kb/sec peak, and 3200kb/sec off peak, it may be considered then the second one is obviously better, even if speeds vary. and speeds do vary normally on the internet. sometimes just cancelling and restarting a download can give a better download speed, as some systems alternate routes. (as well as this, tcp/ip can take a long time to recover from packet loss)

the other thing to look out for is US residential peak slow downs, this used to be a common problem with battle.net for instance.

Not sure that your comments make a lot of sense, or am I missing the point?

This comparison uses a simple time comparison because the result is pretty close to a Minimum/Maximum of medians but easier to calculate en-mass.

ISPs use of proxies is the ISP issue, we simply measure the net result on users connections. Unless you represent an ISP?

I very much doubt that ISPs control maximum speeds during off-peak periods, if they do then that would not only show up in median ISP speed / km and it doesn't appear to do that, but also it would cost, for what purpose?.

Cancelling and restarting? Please take a look at our publications, we make around 720 test-runs per probe on over 400 probes each month. Many of these test runs also measure more than one file and we use multiple tests for each hourly median. Restarts may be an important point for a manual test where you run two or three tests, but not with large numbers of tests. eg. Last month we completed 153,000 test downloads of a file in Dallas.

MyHeritage

Shop MyHeritage and uncover your origins and find new relatives with a simple DNA test. (affiliate link).

mercutio

1387 posts

Uber Geek

#1026318 16-Apr-2014 16:59

JohnButt:
mercutio: i don't think comparing peak time to off peak performance is necessarily a good comparison.

in the past there has been slowness with downloading over http off-peak with some ISP's with transparent proxies that reduce maximum tcp/ip performance for file transfers. (it probably doesn't make much difference for web browsing)

if one isp gives 400kb/sec peak, and 600kb/sec off-peak, and another one gives 1200kb/sec peak, and 3200kb/sec off peak, it may be considered then the second one is obviously better, even if speeds vary. and speeds do vary normally on the internet. sometimes just cancelling and restarting a download can give a better download speed, as some systems alternate routes. (as well as this, tcp/ip can take a long time to recover from packet loss)

the other thing to look out for is US residential peak slow downs, this used to be a common problem with battle.net for instance.

Not sure that your comments make a lot of sense, or am I missing the point?

This comparison uses a simple time comparison because the result is pretty close to a Minimum/Maximum of medians but easier to calculate en-mass.

ISPs use of proxies is the ISP issue, we simply measure the net result on users connections. Unless you represent an ISP?

I very much doubt that ISPs control maximum speeds during off-peak periods, if they do then that would not only show up in median ISP speed / km and it doesn't appear to do that, but also it would cost, for what purpose?.

Cancelling and restarting? Please take a look at our publications, we make around 720 test-runs per probe on over 400 probes each month. Many of these test runs also measure more than one file and we use multiple tests for each hourly median. Restarts may be an important point for a manual test where you run two or three tests, but not with large numbers of tests. eg. Last month we completed 153,000 test downloads of a file in Dallas.

Well if you compare peak to off-peak, and an ISP is limiting off peak performance due to clamping maximum TCP Window Size, then that will control maximum off-peak performance. In order to acheive 25 megabit speeds to California, approximately 512k window size is necessary, so if a 256k window size is utilised a maximum speed around half that will happen.

If minimal packet loss happens all the time, then maximum speeds will also drop, but if that same level of packet loss happens off peak and on peak, then it'll show as better on the graph, even though it's worse in practice.

TCP/IP doesn't really handle packet loss that well over distance, especially if it starts at the start of a connection, which there was a huge thread about Telstraclear having TCP/IP performance issues a while back. (before the Vodafone buy out)

It can also matter what peering is like for the combination of providers. There's been quite a few peering disputes recently. The top transit providers used from NZ seem to be NTT, Level3, Cogent, HE.Net, Verizon. Where Cogent has had quite a few peering disputes, NTT has had some minor packet loss incidents around California area, I think Level3 and HE.Net have mostly been ok, but for instance when the flooding happened in New York there was severe degredation to Europe. (which you're not testing yet I think?)

It's complicated, but testing to limit destinations could easily lead to bias. And if there's packet loss in the US, then it may effect some destinations more than others.

If it's true that most people use the internet at night that want to stream video etc and not have pauses, then absolute performance is more significant than how fast it may go off peak. And if a provider had slower speeds at 5 am because they for instance shared some US capacity with some provider doing business traffic then there may be slower 5 am speeds, but it may not matter.

JohnButt

374 posts

Ultimate Geek

Trusted

#1026321 16-Apr-2014 17:06

mercutio:
JohnButt:
mercutio: i don't think comparing peak time to off peak performance is necessarily a good comparison.

in the past there has been slowness with downloading over http off-peak with some ISP's with transparent proxies that reduce maximum tcp/ip performance for file transfers. (it probably doesn't make much difference for web browsing)

if one isp gives 400kb/sec peak, and 600kb/sec off-peak, and another one gives 1200kb/sec peak, and 3200kb/sec off peak, it may be considered then the second one is obviously better, even if speeds vary. and speeds do vary normally on the internet. sometimes just cancelling and restarting a download can give a better download speed, as some systems alternate routes. (as well as this, tcp/ip can take a long time to recover from packet loss)

the other thing to look out for is US residential peak slow downs, this used to be a common problem with battle.net for instance.

Not sure that your comments make a lot of sense, or am I missing the point?

This comparison uses a simple time comparison because the result is pretty close to a Minimum/Maximum of medians but easier to calculate en-mass.

ISPs use of proxies is the ISP issue, we simply measure the net result on users connections. Unless you represent an ISP?

I very much doubt that ISPs control maximum speeds during off-peak periods, if they do then that would not only show up in median ISP speed / km and it doesn't appear to do that, but also it would cost, for what purpose?.

Cancelling and restarting? Please take a look at our publications, we make around 720 test-runs per probe on over 400 probes each month. Many of these test runs also measure more than one file and we use multiple tests for each hourly median. Restarts may be an important point for a manual test where you run two or three tests, but not with large numbers of tests. eg. Last month we completed 153,000 test downloads of a file in Dallas.

Well if you compare peak to off-peak, and an ISP is limiting off peak performance due to clamping maximum TCP Window Size, then that will control maximum off-peak performance. In order to acheive 25 megabit speeds to California, approximately 512k window size is necessary, so if a 256k window size is utilised a maximum speed around half that will happen.

If minimal packet loss happens all the time, then maximum speeds will also drop, but if that same level of packet loss happens off peak and on peak, then it'll show as better on the graph, even though it's worse in practice.

TCP/IP doesn't really handle packet loss that well over distance, especially if it starts at the start of a connection, which there was a huge thread about Telstraclear having TCP/IP performance issues a while back. (before the Vodafone buy out)

It can also matter what peering is like for the combination of providers. There's been quite a few peering disputes recently. The top transit providers used from NZ seem to be NTT, Level3, Cogent, HE.Net, Verizon. Where Cogent has had quite a few peering disputes, NTT has had some minor packet loss incidents around California area, I think Level3 and HE.Net have mostly been ok, but for instance when the flooding happened in New York there was severe degredation to Europe. (which you're not testing yet I think?)

It's complicated, but testing to limit destinations could easily lead to bias. And if there's packet loss in the US, then it may effect some destinations more than others.

If it's true that most people use the internet at night that want to stream video etc and not have pauses, then absolute performance is more significant than how fast it may go off peak. And if a provider had slower speeds at 5 am because they for instance shared some US capacity with some provider doing business traffic then there may be slower 5 am speeds, but it may not matter.

Wow, that's a lot of reports in one doc.

Yes we measure to other places, but have focussed on the NZ scene to chase that down first. March regular report will be a major change from the past, we will be focussing on a different group of NZ websites, and comparing what we can between live websites and our test websites, that should be interesting. I hope to publish that on Tuesday after Easter.

At a later date we will do some longitudinal studies of International tests - I am actually taking a look at the data today and it is quite surprising, but we need to review it for accuracy, fairness and clarity first though.

Reports don't come cheap :-(

mercutio

1387 posts

Uber Geek

#1026327 16-Apr-2014 17:21

JohnButt:
Reports don't come cheap :-(

Is it possible to instead just publish raw anonymised data?

JohnButt

374 posts

Ultimate Geek

Trusted

#1026331 16-Apr-2014 17:24

mercutio:
JohnButt:
Reports don't come cheap :-(

Is it possible to instead just publish raw anonymised data?

Data doesn't come cheap, actually a lot more expensive, would you like to purchase?

1 | 2