Is Vodafone filtering traffic to speedtest.net to ensure better speedtest results in its favour?

Forums › New Zealand Broadband › Is Vodafone filtering traffic to speedtest.net to ensure better speedtest results in its favour?

1 | 2 | 3

220 posts

Master Geek

#985053 12-Feb-2014 13:44

The issue is not about 'general impressions of "Internet speed".' It is about the comparative results between different speedtest sites. I note that ookla's international sites are producing results similarly different to other sites that are similar to ookla's NZ servers. I accept the 'NO Vodafone do not filter by site' responses even though ISPs either do or have the capability to filter site specific traffic.

But then I do seek a cause for the different speedtest behaviours I see.

Could be that lost packets are handled differently and affect the results in varying ways with different test sites? I don't see any mention of Truenet testing for packet loss. Do they?

Demeter

709 posts

Ultimate Geek

Trusted

One NZ

#985112 12-Feb-2014 14:24

RuralJohnny:

Demeter: It is not the absolute reliability of speedtests that concerns me. The ookla results from US based servers (and which therefore are well outside the Vodafone network) are similarly inconsistent as results from ookla's NZ servers. That raises questions for me.

I can see a possible reason for my experience in that the ookla services will use two threads (or four for speeds >4Mbps) and Truenet uses one thread. Would that account for an approximate 2 times difference in throughput?

Another point that gives reason for suspicion: My recent speedtest.net results are now consistently close to 5Mbps. Why would that be? If the cell site I point at is lightly loaded, should I not expect speeds greater than 5Mbps? Like closer to the 7.2 Mbps DL max for HSPA? Or is 5Mbps the max speed possible for RBI connections?

That the upturn in speedtest.net results (to 5Mbps) occurred about the same time as the upturn in the Truenet results (to 2M5bps) is curious. I'll check with John Butt to see what may have changed at his end.

Hi RuralJohnny, I hope that coffeebaron's earlier answer makes sense as it is correct.

freitasm

BDFL - Memuneh
79288 posts

Uber Geek

Administrator

ID Verified

Trusted

Geekzone

Lifetime subscriber

#985157 12-Feb-2014 14:51

RuralJohnny: I see consistently faster results on speedtest.net (ookla) compared to all the other sites.

Also note that I didn't know of any of the other sites you mentioned, so it could be that sites using Ookla speedtest have to conform to a minimum standard of connection speeds and availability, one reason why they give better results than others.

As above, any speed test is flawed because once the packets leave your ISP's network... who knows?

RuralJohnny

33 posts

Geek

#986409 12-Feb-2014 21:24

Demeter: I hope that coffeebaron's earlier answer makes sense as it is correct.

I accept Coffeebaron's point about the Huawei B970b's speed maxing out at around 5Mbps. It addresses one of the issues I raised. But that is all.

I did a suite of speedtests this evening.

On speedtest.net, testing to a range of ookla servers (Auckland, Wellington, Melbourne, San Francisco) produced results between 4M10 and 5M22 which suggests to me that speeds when testing to a server outside of Vodafone's network is not really an issue.

Doing the same thing on testmy.net produced a similar closely spaced set of results but at 2M6 - 4Mbps - up to 50% of those of ookla.

One thing I noticed on testmy.net is that download speeds vary by file size. Performance comes with larger files! That has to be a function of network design.

This speed-file size relationship was confirmed at nzspeedtest.com but again, with tests to servers in NZ and Aus, the reported speeds were significantly less than ookla's.

The reason I have been exploring this issue is that Truenet report download speeds all in excess of 91% of the ISP's advertised speed for DSL and fibre products. My, and at least one other RBI user, get speeds at around 50% of the average speed.

I recall that when the RBI was being negotiated, the promise was for speeds of "at least 5 Mbps". That was later changed to "minimum peak speeds of 5Mbps". Now it appears that 5Mbps is actually a maximum speed.

The answer may be that average speeds have declined because the user base has increased (doubled was your term). That contention relationship is a function of Vodafone's network design.

When I read the literature and forums on ookla's speedtest, the consensus seems to be that speedtest.net overstates actual speeds. Which is to Vodafone's advantage considering that it is not delivering on its RBI promise.

johnr

19282 posts

Uber Geek
Inactive user

#986422 12-Feb-2014 21:31

Cell site capacity is not static, As traffic grows then capacity will be increased as required

JohnButt

374 posts

Ultimate Geek

Trusted

#986431 12-Feb-2014 21:47

RuralJohnny:
The reason I have been exploring this issue is that Truenet report download speeds all in excess of 91% of the ISP's advertised speed for DSL and fibre products. My, and at least one other RBI user, get speeds at around 50% of the average speed.

TrueNet publish speed for ADSL and VDSL as a percentage of maximum speed, simply because the ISP has little influence over the distance between a DSLAM and a customer, the major driver of maximum speed. However the ISP does influence the ability to use that speed, especially during peak traffic hours. The ability of ISPs to influence is demonstrated in the results reported by TrueNet over the last 2 years, with some ISPs peak period speed sitting down around 60% of maximum to the last few months where all ISPs achieved better than 90%.

Unfortunately we do not have quite enough RBI volunteers to manage a separate report, although we are getting close.

RunningMan

8955 posts

Uber Geek

#986448 12-Feb-2014 21:58

RuralJohnny:
One thing I noticed on testmy.net is that download speeds vary by file size. Performance comes with larger files! That has to be a function of network design.

That's a function of how 3G broadband works - the throughput ramps up over a period of time as demand for bandwidth continues. A small file (short in duration to download) will download at a lower rate than a large file, because the large file continues for longer, and gets allocated more bandwidth (i.e. speed) as it progresses.

If different sites use different size and numbers of files to test, then the results will vary quite significantly - it's like comparing apples with oranges and expecting them to be the same.

Sharesies

Trade NZ and US shares and funds with Sharesies (affiliate link).

bakewells5856

72 posts

Master Geek

#986528 13-Feb-2014 01:04

What I noticed with speed tests is that the result seems to be given closer to the peak speed measured over the test rather than average, so if your bandwidth is like a roller coaster then the test results will be somewhat overoptimistic.

the best way I find to test is to just leave something downloading for a few minutes and graph the usage.

RuralJohnny

33 posts

Geek

#986574 13-Feb-2014 08:46

RunningMan:
RuralJohnny:
One thing I noticed on testmy.net is that download speeds vary by file size. Performance comes with larger files! That has to be a function of network design.

That's a function of how 3G broadband works - the throughput ramps up over a period of time as demand for bandwidth continues. A small file (short in duration to download) will download at a lower rate than a large file, because the large file continues for longer, and gets allocated more bandwidth (i.e. speed) as it progresses.

If different sites use different size and numbers of files to test, then the results will vary quite significantly - it's like comparing apples with oranges and expecting them to be the same.

Yes I am aware of that RunningMan - for my tests, Truenet uses 300kB files and discards the first 75K because of this. Ookla discard the lowest 30% (and top 10%) of their results to arrive at a download speed. Yet the speeds seen by each are vastly different.

So perhaps it is more like comparing oranges with mandarins?

Even so, much of the public perception of broadband performance hinges on speedtest results. Rightly or wrongly. Perhaps it is time for Comcom to develop a standard test?

With Truenet contracted to Comcom, they would appear to be the best option for standardising on speed tests? Would Vodafone quote their speed tests? I think not.

RuralJohnny

33 posts

Geek

#986586 13-Feb-2014 09:04

bakewells5856: What I noticed with speed tests is that the result seems to be given closer to the peak speed measured over the test rather than average, so if your bandwidth is like a roller coaster then the test results will be somewhat overoptimistic.

the best way I find to test is to just leave something downloading for a few minutes and graph the usage.

Here is the Truenet data for my RBI connection in January. The daily speed is the average of the hourly data points. Before 11th Jan, speeds ranged between 0.9 and 2.3 Mbps (average 1.9). From 12th Jan range was 1.3 to 3.2, average 2.3. For all the hourly data points in the sample, only 13% are at or above 50% of the expected speed of 5Mbps. If that were the measurement on urban DSL connections, there would be repercussions I am sure.

RBI connection speed

RuralJohnny

33 posts

Geek

#986587 13-Feb-2014 09:06

mmm... not sure why that dropbox link is not working. Try https://www.dropbox.com/s/a84ehq1wiakdftj/Truenet%20data%20on%20RBI%20access%20speeds%20Jan%202014.jpg

RuralJohnny

33 posts

Geek

#986590 13-Feb-2014 09:10

johnr: Cell site capacity is not static, As traffic grows then capacity will be increased as required

So at what demand level is capacity increased? In other words, how poor does the service standard have to get before Vodafone decide to invest more capital to improve the standard?

johnr

19282 posts

Uber Geek
Inactive user

#986596 13-Feb-2014 09:28

RuralJohnny:
johnr: Cell site capacity is not static, As traffic grows then capacity will be increased as required

So at what demand level is capacity increased? In other words, how poor does the service standard have to get before Vodafone decide to invest more capital to improve the standard?

Many factors come into capacity and planning not just speed and what speed you are getting from XYZ speedtest server

RuralJohnny

33 posts

Geek

#986605 13-Feb-2014 09:48

johnr:
RuralJohnny:
johnr: Cell site capacity is not static, As traffic grows then capacity will be increased as required

So at what demand level is capacity increased? In other words, how poor does the service standard have to get before Vodafone decide to invest more capital to improve the standard?

Many factors come into capacity and planning not just speed and what speed you are getting from XYZ speedtest server

True I am sure. But that is not what your earlier post indicated. So how does Vodafone decide when to add broadband capacity to a 3G cell site? How does Vodafone monitor its service to know when it needs to upgrade? Does it monitor for service quality?

raytaylor

4014 posts

Uber Geek

Trusted

#986854 13-Feb-2014 16:38

Lias: While we have this vast pool of combined knowledge posting in here.

Given everything you've mentioned, what would you suggest is the best way for an end user to get a clear picture of how well their connection is performing for international data(which is how must users perceive "internet performance")?

That is difficult.
All popular youtube videos are served up from caches within new zealand. Akamai runs local CDN nodes that distribute content for their customers (windows, apple updates etc)
So you never know if a website you are visiting is going to serve the content from NZ or overseas.

Anyhow, the best way I can suggest direct international performance is to pick a bunch of speedtest servers on the western seaboard of the united states and average the results.

Ray Taylor

There is no place like localhost

Spreadsheet for Comparing Electricity Plans Here

1 | 2 | 3