Geekzone: technology news, blogs, forums
Welcome Guest.
You haven't logged in yet. If you don't have an account you can register now.

88 posts

Master Geek
+1 received by user: 8

Topic # 241054 8-Oct-2018 18:56
Send private message

So, as a long time IT worker and good netizen, I decided I would take part in the Commerce Commission's "Measuring New Zealand’s broadband performance" initiative.  The Comcom have partnered with British company, SamKnows, who send you a white box, direct from the UK via Royal Mail.  One side of this box (basically an off-the-shelf consumer-grade router with modified firmware) plugs into your internet router and you plug your Ethernet devices into the other end.


I've had this unit plugged into my home network for about a week.  You can log into a portal that shows your various stats.  So what's the problem?  Well, the problem is that SamKnows' stats just don't align with reality.  On their own FAQ, SamKnows states:


It's really important that the test servers are deployed close to your home (and your Whitebox!) because the further the test server is from you, the higher the latency and the more we have to cross third-party networks, which can distort the results. This is why we operate so many test servers all around the world - locality to you is super important!


Here's my results (via wifi):


The SamKnows portal shows my average latency as 228ms (worst 241 ms, best 225 ms)
Click to see full size


The next problem is the reported download speed.  If I plug my PC directly into my router, shows that I can get in excess of 800 Mbps.  Over many tests, I might get a result of 400 - 500 Mbps, but this is very rare, most results are much faster.  SamKnows is reporting my average download speed as 153 Mbps.  What's worse, is that the box has only found one day where it has decided my traffic is low enough to make an accurate test.
Click to see full size


I have a few IoT devices in my home, and it seems the tiny trickle of data that they use to phone home is enough to prevent SamKnows from testing download speed most of the time.  On their site, they state that the test only runs when there is no internet traffic.  Are you kidding?  If you have any smart devices, when is there ever no internet traffic?  They need to set a reasonable threshold, so that if there is only a few Mbps (or even Kbps) traffic, the test will run.


So in summary, it seems obvious to me that:


  • The testing servers are a long, long way away, making latency tests useless/pointless.
  • Speed tests don't work if you have anything live on your network that talks even a little to the internet
  • Speed tests don't reflect national speeds - God only know where the test servers are
  • SamKnows portal gives no indication of the location of their test servers




Create new topic
8017 posts

Uber Geek
+1 received by user: 4006

Lifetime subscriber

  Reply # 2103521 8-Oct-2018 19:12
Send private message

Not at all. The Whitebox tests to multiple services - you can see this by diving in to analytics and ticking "Split data" on what you want to look at.


For example, Latency on my connection shows a few things. The homepage has it all averaged out (taken from a 2degrees connection):



Speed is the same:



So they're getting far more value out of Whitebox compared to Truenet as it tests to multiple locations including locations here in NZ. Afraid some ISP's (cough, Spark) don't peer locally with the Speedtest probes here in NZ forcing traffic to go over to Australia and back again but this will only reflect back on an ISP that doesn't peer in APE / WIX and still gives a good insight on that ISP in terms of browsing performance here in NZ.


Furthermore, I've got an average of 10Mbit going through my network. Some days it doesn't test as much and others it'll still test heaps. I'll actually lock this as it doesn't seem you have a full understanding with how it works and instead direct you to the active thread here: - feel free to ask questions there.

Create new topic

Twitter »

Follow us to receive Twitter updates when new discussions are posted in our forums:

Follow us to receive Twitter updates when news items and blogs are posted in our frontpage:

Follow us to receive Twitter updates when tech item prices are listed in our price comparison site:

Geekzone Live »

Try automatic live updates from Geekzone directly in your browser, without refreshing the page, with Geekzone Live now.

Are you subscribed to our RSS feed? You can download the latest headlines and summaries from our stories directly to your computer or smartphone by using a feed reader.

Alternatively, you can receive a daily email with Geekzone updates.