Is Vodafone filtering traffic to speedtest.net to ensure better speedtest results in its favour?

Forums › New Zealand Broadband › Is Vodafone filtering traffic to speedtest.net to ensure better speedtest results in its favour?

RuralJohnny

33 posts

Geek

#139540 12-Feb-2014 11:41

As a Vodafone RBI customer, I have had an issue with the speed of my connection for some time.

Over a period of many months, my download speed has declined from under 5Mbps to under 2Mbps late last year. This is as measured by Truenet and I have run tests on a number of speedtest sites to cross check the Truenet results.

The results I get from different speedtest sites vary markedly but consistently. For example, speedtest.net (to both NZ and US servers) usually gives results around 2 times greater than using other sites such as nzspeedtest.com, testmy.net, consumer.performancetests.hughesnet.com.

The tests to the different sites were done sequentially, within the same time frame, and done multiple times to try to avoid biasing results.

I see consistently faster results on speedtest.net (ookla) compared to all the other sites.

At the moment, speedtest.net gives consistent results within a few bps of 5Mbps, a situation which started around 5 weeks ago following a message to Vodafone.

I had another RBI customer run tests on speedtest.net and nzspeedtest.com and their results were comparable to mine both in absolute speeds and relative to each other. So I have no reason to believe that this is just my connection.

The results I get are consistent across two different computers and are not affected by whether I use a wireless or wired connection to the gateway (a Huawei B970b). So I have no reason to believe that this has anything to do with my computers or home network.

My question is: Is Vodafone filtering traffic to speedtest.net to ensure better speedtest results in its favour?

1 | 2 | 3

709 posts

Ultimate Geek

Trusted

One NZ

#984979 12-Feb-2014 12:15

Speedtests are never a terribly reliable way to gauge what kind of broadband experience you can expect, even though it is a commonly used one. I believe you've been in touch with our team in this regard previously and they have supplied you with the information you seek. But on the offchance that you are not aware, let me recap the information, also for the benefit of the casual observer.

There are differences between Truenet & Speedtest.Net - TrueNet measure throughput beyond Vodafone network (to various locations on the internet). We can guarantee 5Mbps up to the edge of our network, beyond that, speed is influenced by factors beyond our control. We would be well within expectations to measure our network by testing to a server at the edge of our network, ie using speedtest.net. Note speed also depends on things like how many devices are connected at your home.

TrueNet

measures a combination of bandwidth and latency
measure the time taken to download a file using a single thread
from various locations on the internet, both national and international.
test procedure and test sites have changed over the last few months – may explain the step change in performance

Speetest.net

measures and reports separately the bandwidth and latency
measure the time taken to download a file using multiple threads – typical of modern operating systems and browsers
typically measured to the nearest national site for easy comparison

I understand that the RBI site you are linked to has been health-checked and is not suffering any performance issues. There growth rate in your area is enormous, doubling in the past year. The overall accessibility on your site is well within the targets.

Just to be clear and to categorically answer your question about whether we perform traffic filtering for speedtest.net - the answer is 'No'.

Regards,
Lon

Coil

6614 posts

Uber Geek
Inactive user

#984980 12-Feb-2014 12:17

Short answer as Lon stated: No.

We have our own Vodafone server with OOKLA on our own network, So it will perform better than others in certain situations. Off network depends on the route it takes and the upstream of the provider for the server. So we have no control over quality. A speedtest to the Vodafone server will reflect correct speeds.

Talkiet

4793 posts

Uber Geek

Trusted

#984986 12-Feb-2014 12:24

Amen to everything Demeter said.

Cheers - N

Please note all comments are from my own brain and don't necessarily represent the position or opinions of my employer, previous employers, colleagues, friends or pets.

johnr

19282 posts

Uber Geek
Inactive user

#984991 12-Feb-2014 12:28

If you want a true speed test of throughput then this is best done via FTP file size about over 10MB from a local server,

3G broadband / HSPA ramps up over time during a download as well

Talkiet

4793 posts

Uber Geek

Trusted

#984996 12-Feb-2014 12:34

johnr: If you want a true speed test of throughput then this is best done via FTP file size about over 10MB from a local server,

3G broadband / HSPA ramps up over time during a download as well

That'll certainly give you a true test of speed for a single TCP thread, dependent on the applications, TCP stacks at both ends, capacity of the FTP server, characteristics of the E2E link in terms of latency and packet loss, and possibly even impacted by things such as the local PC :-)

Cheers - N

Please note all comments are from my own brain and don't necessarily represent the position or opinions of my employer, previous employers, colleagues, friends or pets.

Talkiet

4793 posts

Uber Geek

Trusted

#985005 12-Feb-2014 12:40

But seriously folks, a test to a speedtest.net server hosted by your own ISP, and preferably in your own city - will generally give a good indication of the performance of your access link (ADSL/VDSL/Fibre)...

Even if your ISP doesn't host a speedtest.net server, most servers in NZ from large ISPs should give a pretty good reading, although as soon as you go across an ISP boundary (VF user testing to a Telecom speedtest server for example) you are now talking about testing AT LEAST TWO DIFFERENT NETWORKS.

Trying to hold VF accountable for performance to a Telecom server, or a server in Germany, is plainly stupid - so wherever possible, test to a speedtest.net server from your own ISP, where they have visibility of ALL the links between you and the server.

BTW, The three Telecom NZ speedtest servers are down at the moment... They should be back up tomorrow after a small reconfiguration.

Regards
Neil G

Please note all comments are from my own brain and don't necessarily represent the position or opinions of my employer, previous employers, colleagues, friends or pets.

surfisup1000

5288 posts

Uber Geek

#985008 12-Feb-2014 12:46

Demeter: Speedtests are never a terribly reliable way to gauge what kind of broadband experience you can expect, even though it is a commonly used one.
Lon

I've found a high correlation between speedtest.net results and internet speed. Sure, the test may not be exact due to multiple congestion point and varying servers but as a general guide I've found it to be 100% spot on.

Why is it not reliable when I've found it so reliable?

Sharesies

Trade NZ and US shares and funds with Sharesies (affiliate link).

Talkiet

4793 posts

Uber Geek

Trusted

#985017 12-Feb-2014 12:50

surfisup1000:
Demeter: Speedtests are never a terribly reliable way to gauge what kind of broadband experience you can expect, even though it is a commonly used one.
Lon

I've found a high correlation between speedtest.net results and internet speed. Sure, the test may not be exact due to multiple congestion point and varying servers but as a general guide I've found it to be 100% spot on.

Why is it not reliable when I've found it so reliable?

It is reliable within the constraints which MUST be understood if the results are to be useful.

When the results of speedtest.net and your general impression of "Internet speed" don't match, you cannot with any certainty say that the issue is on your ISP network. That's the issue.

Cheers - N

Please note all comments are from my own brain and don't necessarily represent the position or opinions of my employer, previous employers, colleagues, friends or pets.

JohnButt

374 posts

Ultimate Geek

Trusted

#985029 12-Feb-2014 13:12

Demeter: Speedtests are never a terribly reliable way to gauge what kind of broadband experience you can expect, even though it is a commonly used one. I believe you've been in touch with our team in this regard previously and they have supplied you with the information you seek. But on the offchance that you are not aware, let me recap the information, also for the benefit of the casual observer.

There are differences between Truenet & Speedtest.Net - TrueNet measure throughput beyond Vodafone network (to various locations on the internet). We can guarantee 5Mbps up to the edge of our network, beyond that, speed is influenced by factors beyond our control. We would be well within expectations to measure our network by testing to a server at the edge of our network, ie using speedtest.net. Note speed also depends on things like how many devices are connected at your home.

TrueNet

measures a combination of bandwidth and latency
measure the time taken to download a file using a single thread
from various locations on the internet, both national and international.
test procedure and test sites have changed over the last few months – may explain the step change in performance
Speetest.net

measures and reports separately the bandwidth and latency
measure the time taken to download a file using multiple threads – typical of modern operating systems and browsers
typically measured to the nearest national site for easy comparison

I understand that the RBI site you are linked to has been health-checked and is not suffering any performance issues. There growth rate in your area is enormous, doubling in the past year. The overall accessibility on your site is well within the targets.

Just to be clear and to categorically answer your question about whether we perform traffic filtering for speedtest.net - the answer is 'No'.

Regards,
Lon

This is a good answer. Although I am not sure what expectations in "We would be well within expectations to measure our network by testing to a server at the edge of our network" refers to.

We had a minor change to our Wellington servers in August 2013, which is very hard to see in the results, although it caused a lot of headaches for 4 days.

Speedtest is best used as a check for a local fault rather than proof your speed is exactly as measured. Finding the best speedtest server that is often directly connected or within the ISP is the best method of checking access-only performance. TrueNet's goal is to measure and report on performance right through the ISP to independent, but common, servers.

Lias

5590 posts

Uber Geek

ID Verified

Trusted

Lifetime subscriber

#985030 12-Feb-2014 13:15

While we have this vast pool of combined knowledge posting in here.

Given everything you've mentioned, what would you suggest is the best way for an end user to get a clear picture of how well their connection is performing for international data(which is how must users perceive "internet performance")?

I'm a geek, a gamer, a dad, a Quic user, and an IT Professional. I have a full rack home lab, size 15 feet, an epic beard and Asperger's. I'm a bit of a Cypherpunk, who believes information wants to be free and the Net interprets censorship as damage and routes around it. If you use my Quic signup you can also use the code R570394EKGIZ8 for free setup.

RuralJohnny

33 posts

Geek

#985032 12-Feb-2014 13:15

OK, thanks all who replied. I hear the emphatic NO.

However.

My previous experience with Vodafone over a CGNAT issue tells me to not take the first no as the final word. In that case, it was only my persistence to get to a solution (and the assistance of forum members) that meant the Vodafone-caused issue was resolved. Otherwise I would likely still be suffering connectivity issues.

So please excuse me as I seek an answer to the underlying problem I face rather than just accepting the NO.

Demeter: It is not the absolute reliability of speedtests that concerns me. The ookla results from US based servers (and which therefore are well outside the Vodafone network) are similarly inconsistent as results from ookla's NZ servers. That raises questions for me.

I can see a possible reason for my experience in that the ookla services will use two threads (or four for speeds >4Mbps) and Truenet uses one thread. Would that account for an approximate 2 times difference in throughput?

Another point that gives reason for suspicion: My recent speedtest.net results are now consistently close to 5Mbps. Why would that be? If the cell site I point at is lightly loaded, should I not expect speeds greater than 5Mbps? Like closer to the 7.2 Mbps DL max for HSPA? Or is 5Mbps the max speed possible for RBI connections?

That the upturn in speedtest.net results (to 5Mbps) occurred about the same time as the upturn in the Truenet results (to 2M5bps) is curious. I'll check with John Butt to see what may have changed at his end.

coffeebaron

6236 posts

Uber Geek

Trusted

Lifetime subscriber

#985045 12-Feb-2014 13:36

Typically that older RBI router will max out at 5-5.5Mbps on a good capacity site. This is due to overheads, latency and a few other variables. Similar an ADSL connection that syncs at say 10Mbps will give a maximin speedtest result at 8-9Mbps

Rural IT and Broadband support.

Broadband troubleshooting and master filter installs.
Starlink installer - one month free: https://www.starlink.com/?referral=RC-32845-88860-71
Wi-Fi and networking
Cel-Fi supply and installer - boost your mobile phone coverage legally

Need help in Auckland, Waikato or BoP? Click my email button, or email me direct: [my user name] at geekzonemail dot com

RuralJohnny

33 posts

Geek

#985046 12-Feb-2014 13:37

Talkiet:
surfisup1000:
Demeter: Speedtests are never a terribly reliable way to gauge what kind of broadband experience you can expect, even though it is a commonly used one.
Lon

I've found a high correlation between speedtest.net results and internet speed. Sure, the test may not be exact due to multiple congestion point and varying servers but as a general guide I've found it to be 100% spot on.

Why is it not reliable when I've found it so reliable?

It is reliable within the constraints which MUST be understood if the results are to be useful.

When the results of speedtest.net and your general impression of "Internet speed" don't match, you cannot with any certainty say that the issue is on your ISP network. That's the issue.

Cheers - N

The issue is not about 'general impressions of "Internet speed".' It is about the comparative results between different speedtest sites. I note that ookla's international sites are producing results similarly different to other sites that are similar to ookla's NZ servers. I accept the 'NO Vodafone do not filter by site' responses even though ISPs either do or have the capability to filter site specific traffic.

But then I do seek a cause for the different speedtest behaviours I see.

Talkiet

4793 posts

Uber Geek

Trusted

#985048 12-Feb-2014 13:40

RuralJohnny:
Talkiet:
surfisup1000:
Demeter: Speedtests are never a terribly reliable way to gauge what kind of broadband experience you can expect, even though it is a commonly used one.
Lon

I've found a high correlation between speedtest.net results and internet speed. Sure, the test may not be exact due to multiple congestion point and varying servers but as a general guide I've found it to be 100% spot on.

Why is it not reliable when I've found it so reliable?

It is reliable within the constraints which MUST be understood if the results are to be useful.

When the results of speedtest.net and your general impression of "Internet speed" don't match, you cannot with any certainty say that the issue is on your ISP network. That's the issue.

Cheers - N

The issue is not about 'general impressions of "Internet speed".' It is about the comparative results between different speedtest sites. I note that ookla's international sites are producing results similarly different to other sites that are similar to ookla's NZ servers. I accept the 'NO Vodafone do not filter by site' responses even though ISPs either do or have the capability to filter site specific traffic.

But then I do seek a cause for the different speedtest behaviours I see.

Different speedtest sites have different methodologies, use different numbers of threads, use different statistical analysis of their component results and discard a different number of best and worst results before applying differing multiplication factors to account for 'overheads'. They use different size test files as well.

Cheers - N

edit - read this... It's not a simple throughput and time calculation...

https://support.speedtest.net/entries/20862782-How-does-the-test-itself-work-How-is-the-result-calculated-

Please note all comments are from my own brain and don't necessarily represent the position or opinions of my employer, previous employers, colleagues, friends or pets.

johnr

19282 posts

Uber Geek
Inactive user

#985049 12-Feb-2014 13:41

Effective as peeing into the wind

1 | 2 | 3