OOKLA / Speedtest beta

Forums › New Zealand Broadband › OOKLA / Speedtest beta

1 | 2 | 3

374 posts

Ultimate Geek

Trusted

#1436116 27-Nov-2015 15:32

Sideface:
JohnButt:
Sideface: I use the OOKLA speedtest every day.

I have often wondered how and why Speedtest is used more by some than others, can you comment on why you use it every day? ...

@JohnButt

I am a TrueNet volunteer, but TrueNet stats, although very useful, show trends over the past week(s) rather than real-time performance.

I am on VF cable in Island Bay, and have had ongoing problems with peak congestion, even after the major upgrade of 01 October.

Therefore I test my connection with OOKLA and nPerf several times every day, and compare results with my TrueNet stats.

In summary, I am a compulsive tester

mmmm. that is one use, but I expect there are many others to drive the test numbers they see. e.g. if you are working on faulty line/PCs/home networks, you may have a need to do it more often.

Live access to the data for your personal line could be made available, but I have never thought it worth developing the code because I don't expect anyone would pay enough to recover the costs for unique access to one or two lines.

JohnButt

374 posts

Ultimate Geek

Trusted

#1436122 27-Nov-2015 15:35

sorceror:
JohnButt:
Sideface: I use the OOKLA speedtest every day.

I have often wondered how and why Speedtest is used more by some than others, can you comment on why you use it every day?

BTW it worked well for me on a Mac with OS X 10.10.5

no other speedtest site can get close to my max throughput (1 Gbps)

Neither can we, it would require a 10Gig interface at all our test sites to ensure we were getting the correct answers for everyone else. A GigE connection could swamp our interface, stopping all other tests performing without interference. For that reason we are or have returned probes from all panelists who migrate above 200Mb/s

mercutio

1392 posts

Uber Geek

#1436126 27-Nov-2015 15:38

JohnButt:
sorceror:
JohnButt:
Sideface: I use the OOKLA speedtest every day.

I have often wondered how and why Speedtest is used more by some than others, can you comment on why you use it every day?

BTW it worked well for me on a Mac with OS X 10.10.5

no other speedtest site can get close to my max throughput (1 Gbps)

Neither can we, it would require a 10Gig interface at all our test sites to ensure we were getting the correct answers for everyone else. A GigE connection could swamp our interface, stopping all other tests performing without interference. For that reason we are or have returned probes from all panelists who migrate above 200Mb/s

can you get isp's to host truenet servers on their network to get 10 gig interfaces like speedtest.net?

another alternative is to connect to multiple speedtest servers at once like dslreports etc do. dslreports also shows bufferbloat which is very useful to know about which isp's are good!

Sideface

9350 posts

Uber Geek

Trusted

Lifetime subscriber

#1436177 27-Nov-2015 16:46

sorceror: .. no other speedtest site can get close to my max throughput (1 Gbps)

nPerf claim to test up to 1 Gbps on all their servers (including Auckland and Sydney), with 10 Gbps on some of their French servers.

I presume that you've tried them out.

Sideface

JohnButt

374 posts

Ultimate Geek

Trusted

#1436182 27-Nov-2015 17:02

mercutio:
JohnButt:
sorceror:
JohnButt:
Sideface: I use the OOKLA speedtest every day.

I have often wondered how and why Speedtest is used more by some than others, can you comment on why you use it every day?

BTW it worked well for me on a Mac with OS X 10.10.5

no other speedtest site can get close to my max throughput (1 Gbps)

Neither can we, it would require a 10Gig interface at all our test sites to ensure we were getting the correct answers for everyone else. A GigE connection could swamp our interface, stopping all other tests performing without interference. For that reason we are or have returned probes from all panelists who migrate above 200Mb/s

can you get isp's to host truenet servers on their network to get 10 gig interfaces like speedtest.net?

another alternative is to connect to multiple speedtest servers at once like dslreports etc do. dslreports also shows bufferbloat which is very useful to know about which isp's are good!

You ask difficult questions :-(

We could ask ISPs, but that may compromise our service with a conflict of interest, although it is done in Singapore. There is also the cost of doing it, we don't even have a customer for 1Gig fibre testing, or indeed any fibre testing. We test fibre because we can, the Comcom only contract us for ADSL testing.

Testing to a nice router in the ISP would also compromise our tests in a more important way - we test from our probes through the ISP, unlike Speedtest which is from your computer to the ISP, so does nothing to show if the ISP is performing any service let alone a good one. We also test downloading a speedtest file and find that is often much faster than our files, i.e. when the ISP or their backhaul is congested shows on our tests but not the actual connection which is the Speedtest result.

Speedtest methodology only tests the last mile, which rarely has anything to do with the ISP

We can identify bufferbloat and have in the past, but mostly it has vanished once we got all the ISPs to understand they had it - that took a while. I have recently seen some results suggesting it is coming back on some routes, such as International. Love to do more on this, but the effort to research it would require funding, I do not have the resource.

Talkiet

4792 posts

Uber Geek

Trusted

#1436187 27-Nov-2015 17:25

JohnButt: [snip]
Testing to a nice router in the ISP would also compromise our tests in a more important way - we test from our probes through the ISP, unlike Speedtest which is from your computer to the ISP, so does nothing to show if the ISP is performing any service let alone a good one. We also test downloading a speedtest file and find that is often much faster than our files, i.e. when the ISP or their backhaul is congested shows on our tests but not the actual connection which is the Speedtest result.

Speedtest methodology only tests the last mile, which rarely has anything to do with the ISP
[snip]

Hmm John, you're not quite right here - I see what you are thinking, but you're not really being fair or accurate here.

In Spark we have three speedtest servers, (Akl, Wlg and Chc). Those servers are connected to the Internet in exactly the same way (same network elements, same basic config, different size access links) as the CDN platforms we host. I won't tell you how much of our Customer BB traffic comes from the CDNs but it's _A LOT_... So it IS a valid place to put the Speedtest servers and it does measure the service experience for a vast amount of the traffic consumed by our customers.

Certainly testing to servers outside our network will ALSO test some other parts of the E2E route, but as soon as you exit the ISP network you are no longer testing the network that the ISP has direct control over.

Customers can easily test to speedtest.net hosts outside our network as well.

To say "unlike Speedtest which is from your computer to the ISP, so does nothing to show if the ISP is performing any service let alone a good one" is a very bad way to try and get your point across.

Cheers - N

Please note all comments are from my own brain and don't necessarily represent the position or opinions of my employer, previous employers, colleagues, friends or pets.

sorceror

163 posts

Master Geek

#1436852 28-Nov-2015 14:02

I find spark's speedtest servers give me better speeds than my ISPs ones, so go figure

Equinox IT

Cloud spending continues to surge globally, but most organisations haven’t made the changes necessary to maximise the value and cost-efficiency benefits of their cloud investments. Download the whitepaper From Overspend to Advantage now.

Behodar

10501 posts

Uber Geek

Trusted

Lifetime subscriber

#1436878 28-Nov-2015 14:12

My ISP doesn't have any, but Spark's tend to give me better speeds than the autodetected ones (which seem to vary).

mercutio

1392 posts

Uber Geek

#1437368 29-Nov-2015 19:43

Behodar: My ISP doesn't have any, but Spark's tend to give me better speeds than the autodetected ones (which seem to vary).

I don't know about you, but I've found that speedtest.net's automatic server selection has gotten worse and worse over time. I just choose manual server.

JohnButt

374 posts

Ultimate Geek

Trusted

#1437648 30-Nov-2015 10:01

Talkiet:
JohnButt: [snip]
Testing to a nice router in the ISP would also compromise our tests in a more important way - we test from our probes through the ISP, unlike Speedtest which is from your computer to the ISP, so does nothing to show if the ISP is performing any service let alone a good one. We also test downloading a speedtest file and find that is often much faster than our files, i.e. when the ISP or their backhaul is congested shows on our tests but not the actual connection which is the Speedtest result.

Speedtest methodology only tests the last mile, which rarely has anything to do with the ISP
[snip]

Hmm John, you're not quite right here - I see what you are thinking, but you're not really being fair or accurate here.

In Spark we have three speedtest servers, (Akl, Wlg and Chc). Those servers are connected to the Internet in exactly the same way (same network elements, same basic config, different size access links) as the CDN platforms we host. I won't tell you how much of our Customer BB traffic comes from the CDNs but it's _A LOT_... So it IS a valid place to put the Speedtest servers and it does measure the service experience for a vast amount of the traffic consumed by our customers.

Certainly testing to servers outside our network will ALSO test some other parts of the E2E route, but as soon as you exit the ISP network you are no longer testing the network that the ISP has direct control over.

Customers can easily test to speedtest.net hosts outside our network as well.

To say "unlike Speedtest which is from your computer to the ISP, so does nothing to show if the ISP is performing any service let alone a good one" is a very bad way to try and get your point across.

Cheers - N

I stand behind my comments Neil.

Users should test to servers outside your network but I find they are often asked to test to the closest server and indeed Speedtest defaults to the closest server.

Testing to servers beyond the ISP network tests a lot within the control of the ISP, e.g. the impact of DNS, the links you have to the rest of the world and your international agreements with suppliers. Recently TrueNet discovered a couple of ISPs with excellent links to Sydney, but it would appear they did not have agreement from their supplier to link to Melbourne causing tromboning via LA. (I noticed that was corrected within hours of our publishing latency results.)

yitz

2074 posts

Uber Geek

#1437700 30-Nov-2015 10:37

JohnButt: Recently TrueNet discovered a couple of ISPs with excellent links to Sydney, but it would appear they did not have agreement from their supplier to link to Melbourne causing tromboning via LA. (I noticed that was corrected within hours of our publishing latency results.)

What level of operational notification do you get from your Melbourne hosting provider though? Perhaps they had an unplanned network event e.g. outage on some of their links or DDoS which required routing external traffic through a clean feed pipe via LA?

eXDee

4032 posts

Uber Geek

Trusted

#1437861 30-Nov-2015 14:05

I've always had concern with how small the files truenet tests with. They tend to be 1 to 5MB. I see the test results hardly get close to showing the download potential of a 200mbps connection, even nationally. I can download from a university at the other end of the country and be close to saturating the line but I believe thats because TCP has time to ramp up. With truenet i can download off a nearby city and see only just over half of that 200mbps.

Talkiet

4792 posts

Uber Geek

Trusted

#1437868 30-Nov-2015 14:14

JohnButt:
Talkiet:[snip]
To say "unlike Speedtest which is from your computer to the ISP, so does nothing to show if the ISP is performing any service let alone a good one" is a very bad way to try and get your point across.

I stand behind my comments Neil.

[snip]

To stand behind the comment I have left quoted here (especially the bits quoted in bold) would make me question your understanding of how ISPs and the Internet works, if I didn't actually know you DO know how they operate.

I let a lot of stuff slide from a lot of people a lot of the time, but someone running a business based on doing broadband performance testing? I'll hold you to a MUCH higher standard than almost anyone else.

Cheers -N

Please note all comments are from my own brain and don't necessarily represent the position or opinions of my employer, previous employers, colleagues, friends or pets.

mercutio

1392 posts

Uber Geek

#1437878 30-Nov-2015 14:31

eXDee: I've always had concern with how small the files truenet tests with. They tend to be 1 to 5MB. I see the test results hardly get close to showing the download potential of a 200mbps connection, even nationally. I can download from a university at the other end of the country and be close to saturating the line but I believe thats because TCP has time to ramp up. With truenet i can download off a nearby city and see only just over half of that 200mbps.

there's a lot of problems with truenet, but testing 1 to 5MB files is much more representative of web browsing and other interactive performance than larger files. it's especially important with smaller files that the ISP has good AQM (https://en.wikipedia.org/wiki/Active_queue_management) on connections.

mercutio

1392 posts

Uber Geek

#1437880 30-Nov-2015 14:34

yitz:
JohnButt: Recently TrueNet discovered a couple of ISPs with excellent links to Sydney, but it would appear they did not have agreement from their supplier to link to Melbourne causing tromboning via LA. (I noticed that was corrected within hours of our publishing latency results.)
What level of operational notification do you get from your Melbourne hosting provider though? Perhaps they had an unplanned network event e.g. outage on some of their links or DDoS which required routing external traffic through a clean feed pipe via LA?

I don't know about other people, but any time I do transit to Melbourne it's to consumer ISP's and not hosted sites. All the heavy user-facing sites are in Sydney with good peering.

Melbourne peering on cheap providers is likely to be sub-optimal. I'd go so far as to say that Australia's general domestic national connectivity is likely to be significantly worse than New Zealand's. (from what I understand, I haven't fully researched it, there are multiple contributing reasons, with higher cost of transit, more distance, more "mixed" international/services being sold etc.

1 | 2 | 3