NZ Broadband testing from Samknows

Forums › New Zealand Broadband › NZ Broadband testing from Samknows

1 | ... | 15 | 16 | 17 | 18 | 19 | 20 | 21

altered-ego
2436 posts

Uber Geek
+1 received by user: 842

Trusted

Lifetime subscriber

#2733995 24-Jun-2021 20:08

Even if Truenet's test methodology was bad in some areas, their reports certainly felt a lot more useful than the one pager email you get from SamKnows. There was some insight explaining the change of data points on the graph & what was going on in the various markets.

As a "world leader", I expect better reports from SamKnows. They operate in multiple markets & surely supply many reports to many stakeholders. What I got from them felt like a automated summary of the most basic facts. My gut feel of how my internet connection behaved over the last months was more useful than the +-3 data points in their email.

Please keep this GZ community vibrant by contributing in a constructive & respectful manner.

shim99

108 posts

Master Geek
+1 received by user: 25

ID Verified

#2734016 24-Jun-2021 21:07

ANglEAUT:
Even if Truenet's test methodology was bad in some areas, their reports certainly felt a lot more useful than the one pager email you get from SamKnows. There was some insight explaining the change of data points on the graph & what was going on in the various markets.

As a "world leader", I expect better reports from SamKnows. They operate in multiple markets & surely supply many reports to many stakeholders. What I got from them felt like a automated summary of the most basic facts. My gut feel of how my internet connection behaved over the last months was more useful than the +-3 data points in their email.

If you login to samknows.one you get much more detailed information than in their monthly summary

michaelmurfy

meow
13581 posts

Uber Geek
+1 received by user: 10914

Moderator

ID Verified

Trusted

Lifetime subscriber

#2734029 24-Jun-2021 22:33

One thing that Truenet did that doesn't happen now is improve ISP's in general and prevent crap like we're seeing with a particular ISP calling themselves the fastest based of Speedtest and Steam average download speed status alone. I read every one of the Truenet reports as flawed as they were in many ways but can safely say I've read maybe 2 Samknows reports then unsubscribed from their emails.

I was a Truenet volunteer for most of the programs life and signed up to Samknows only for the box to be kicked off my network after a mere few months after it caused issues - I emailed SamKnows to return the box, they didn't want it, it got reflashed with OpenWRT.

We really do need the reports showing ISP's performance as this is the thing that improves the internet experience of everyone - test common peering locations, CDN's, international services, gaming servers, pr0n sites etc. Knock that ISP who is claiming to be the best back down to the bottom and get providers to improve themselves like TrueNet used to do (outside of only peering with the particular provider that hosts the Speedtest servers like a particular ISP that I won't name did). This is what improves the internet experience for New Zealanders.

Michael Murphy | https://murfy.nz
Referral Links: Quic Broadband (use R122101E7CV7Q for free setup)

Are you happy with what you get from Geekzone? Please consider supporting us by subscribing.
Opinions are my own and not the views of my employer.

Lias

5655 posts

Uber Geek
+1 received by user: 3978

ID Verified

Trusted

Lifetime subscriber

#2734073 25-Jun-2021 08:36

shim99:

If you login to samknows.one you get much more detailed information than in their monthly summary

But that detailed data is only relevant for your connection, what people want is detailed, easily accessible public comparisons between ISPs.

That aside, my biggest gripe with SamKnows is that it won't test the network when there is network traffic. Leading to very unreliable statistics for me, as it's pretty much unheard of for my connection to not have traffic 24x7.. In the last month it's managed testing on one single day.

I'm a geek, a gamer, a dad, a Quic user, and an IT Professional. I have a full rack home lab, size 15 feet, an epic beard and Asperger's. I'm a bit of a Cypherpunk, who believes information wants to be free and the Net interprets censorship as damage and routes around it. If you use my Quic signup you can also use the code R570394EKGIZ8 for free setup. Opinions are my own and not the views of my employer.

cbrpilot

964 posts

Ultimate Geek
+1 received by user: 555

Trusted

Spark NZ

#2734098 25-Jun-2021 09:37

Lias:

That aside, my biggest gripe with SamKnows is that it won't test the network when there is network traffic. Leading to very unreliable statistics for me, as it's pretty much unheard of for my connection to not have traffic 24x7.. In the last month it's managed testing on one single day.

So you want your whitebox to run tests (speed tests) whilst the connection is being used for other purposes?

While the whitebox does provide participants access to the information that it collects, that's not its primary purpose. I'd hate to be the RSP that gets poorer results because all of the speedtests collide with the customer's usage. Does not sound like in your particular case being a participant is leading to good outcomes for either party.

My views are my own, and may not necessarily represent those of my employer.

cbrpilot

964 posts

Ultimate Geek
+1 received by user: 555

Trusted

Spark NZ

#2734116 25-Jun-2021 09:58

michaelmurfy:

One thing that Truenet did that doesn't happen now is improve ISP's in general and prevent crap like we're seeing with a particular ISP calling themselves the fastest based of Speedtest and Steam average download speed status alone. I read every one of the Truenet reports as flawed as they were in many ways but can safely say I've read maybe 2 Samknows reports then unsubscribed from their emails.

You mean like the improvement in Fibre Max performance made across nearly all RSPs in the last year?

It sounds to me that you're upset that the performance of RSPs is so similar these days that the reports are boring.

michaelmurfy:

We really do need the reports showing ISP's performance as this is the thing that improves the internet experience of everyone - test common peering locations, CDN's, international services, gaming servers, pr0n sites etc. Knock that ISP who is claiming to be the best back down to the bottom and get providers to improve themselves like TrueNet used to do (outside of only peering with the particular provider that hosts the Speedtest servers like a particular ISP that I won't name did). This is what improves the internet experience for New Zealanders.

You mean like the following tests?

-Netflix (CDN) download speed

-Netflix (CDN) bitrate reliably streamed?

-Youtube (CDN) bitrate reliably streamed?

-FIFA 21 Latency and packet loss

-LOL Latency and packet loss

-Dota 2 Latency and packet loss

-Diablo 3 Latency and packet loss

-HOTS Latency and packet loss

-PUBG Latency and packet loss

-SC2 Latency and packet loss

-Hearthstone Latency and packet loss

-Overwatch Latency and packet loss

-WOW Latency and packet loss

-Among Us Latency and packet loss

-Apex Legends Latency and packet loss

-Latency to a heap of social media sites/services

-Latency to the popular video conferencing apps - zoom, teams, skype, webex, google meet, gotomeeting

Guess who runs all of those tests?

Fair comment that they don't report on them all every month - only when there is something interesting.

Something that I haven't seen in the reporting that would be good is a comparison (for example) of gaming on different RSPs - pretty sure they haven't gone down to RSP level for that type of thing.

If you want to see more of certain testing then I suggest you contact the Commerce Commission.

My views are my own, and may not necessarily represent those of my employer.

Samsung

Shop now on Samsung phones, tablets, TVs and more (affiliate link).

michaelmurfy

meow
13581 posts

Uber Geek
+1 received by user: 10914

Moderator

ID Verified

Trusted

Lifetime subscriber

#2734141 25-Jun-2021 10:40

cbrpilot:

You mean like the following tests?

-Netflix (CDN) download speed

-Netflix (CDN) bitrate reliably streamed?

-Youtube (CDN) bitrate reliably streamed?

<snip>

Guess who runs all of those tests?

Yes, I get you... But as they're not making this public showing a breakdown between providers there is little information that holds ISP's accountable. CDN tests are not enough because as we know from a particular ISP they can quite easily call themselves the fastest (oh look - we've got the most gigabit customers hitting our caches at the fastest speeds. We're going to call ourselves the fastest ISP in New Zealand).

And now a pain point of yours... Cloudflare which quite happily goes over the NZ POP for every ISP except one. Only reason I am using this as an example is because it is one I know about. There is a whole lot more traffic that goes via Australia for that particular ISP too harming the user experience of customers and that particular ISP is stuck in the past with not giving their customers the best experience possible, not providing newer technologies such as IPv6 which the majority of other providers do etc. There are plenty of websites that use this particular CDN that are not "dodgy" like Geekzone, many shopping sites in NZ etc. Why not give customers the best experience possible?

Case in point and nothing has changed: https://www.geekzone.co.nz/forums.asp?forumid=39&topicid=240883&page_no=1#2099501

I really don't care about the whole "cache performance" argument because most ISP's have these, or have access to these via peering arrangements.

That is all.

Lias

5655 posts

Uber Geek
+1 received by user: 3978

ID Verified

Trusted

Lifetime subscriber

#2734536 25-Jun-2021 16:04

cbrpilot:

So you want your whitebox to run tests (speed tests) whilst the connection is being used for other purposes?

While the whitebox does provide participants access to the information that it collects, that's not its primary purpose. I'd hate to be the RSP that gets poorer results because all of the speedtests collide with the customer's usage. Does not sound like in your particular case being a participant is leading to good outcomes for either party.

The box should be able to measure the traffic going through it and take that into consideration with its tests, rather than just only test when there is no traffic. That seems like an absolute no brainer to me.

Talkiet

4819 posts

Uber Geek
+1 received by user: 3934

Trusted

#2734538 25-Jun-2021 16:07

Lias:

cbrpilot:

So you want your whitebox to run tests (speed tests) whilst the connection is being used for other purposes?

While the whitebox does provide participants access to the information that it collects, that's not its primary purpose. I'd hate to be the RSP that gets poorer results because all of the speedtests collide with the customer's usage. Does not sound like in your particular case being a participant is leading to good outcomes for either party.

The box should be able to measure the traffic going through it and take that into consideration with its tests, rather than just only test when there is no traffic. That seems like an absolute no brainer to me.

Not feasible. Bordering on not possible.

Cheers - N

Please note all comments are from my own brain and don't necessarily represent the position or opinions of my employer, previous employers, colleagues, friends or pets.

Lias

5655 posts

Uber Geek
+1 received by user: 3978

ID Verified

Trusted

Lifetime subscriber

#2734539 25-Jun-2021 16:10

michaelmurfy:

And now a pain point of yours... Cloudflare which quite happily goes over the NZ POP for every ISP except one. Only reason I am using this as an example is because it is one I know about. There is a whole lot more traffic that goes via Australia for that particular ISP too harming the user experience of customers and that particular ISP is stuck in the past with not giving their customers the best experience possible, not providing newer technologies such as IPv6 which the majority of other providers do etc. There are plenty of websites that use this particular CDN that are not "dodgy" like Geekzone, many shopping sites in NZ etc. Why not give customers the best experience possible?

Not wrong about that ISP living in the past in regards to peering. It's one of the lingering things that ISP does that reminds us all that not all of the 'customers are marks to be exploited' type thinking departed with the split and rename.

Lias

5655 posts

Uber Geek
+1 received by user: 3978

ID Verified

Trusted

Lifetime subscriber

#2734542 25-Jun-2021 16:17

Talkiet:

Not feasible. Bordering on not possible.

Cheers - N

Care to elaborate? Monitoring the level of traffic on the LAN interface (A) shouldn't be an issue, plenty of devices are capable of doing that. The expected capacity of the WAN connection is fixed at 100/20 or 950/500 or whatever (B). Run a speed test of a known quantity (e.g. download 1gb), get resulting speed C. If B = (A + C) then the connection is working at it's maximum potential and can be recorded as such, if not, record the difference?

Dyson

Shop now for Dyson appliances (affiliate link).

freitasm

BDFL - Memuneh
80661 posts

Uber Geek
+1 received by user: 41083

Administrator

ID Verified

Trusted

Geekzone

Lifetime subscriber

#2734545 25-Jun-2021 16:19

Talkiet:

Lias:

The box should be able to measure the traffic going through it and take that into consideration with its tests, rather than just only test when there is no traffic. That seems like an absolute no brainer to me.

Not feasible. Bordering on not possible.

Cheers - N

And that's probably why it wouldn't work in some places... At home we seem to have a constant stream of traffic. Cameras, NAS backup to cloud, smart speakers, streaming devices, smart TVs, smartphones, WiFi-connected BPM, etc.

Sure, at some point the usage will be very low but there's not a moment that I'd say no usage at all.

Referral links: Quic Broadband (free setup code: R587125ERQ6VE) | Samsung | AliExpress | Wise | Sharesies

Support Geekzone by subscribing (browse ads-free), or making a one-off or recurring donation through PressPatron.

Talkiet

4819 posts

Uber Geek
+1 received by user: 3934

Trusted

#2734556 25-Jun-2021 16:40

Lias:

Talkiet:

Not feasible. Bordering on not possible.

Cheers - N

Care to elaborate? Monitoring the level of traffic on the LAN interface (A) shouldn't be an issue, plenty of devices are capable of doing that. The expected capacity of the WAN connection is fixed at 100/20 or 950/500 or whatever (B). Run a speed test of a known quantity (e.g. download 1gb), get resulting speed C. If B = (A + C) then the connection is working at it's maximum potential and can be recorded as such, if not, record the difference?

Essentially multiple TCP streams don't sum in a linear fashion, and it's going to be different depending on what remote endpoints are involved - depending on their latency, CA algorithm in use, packet loss, and a few other things.

You could run the test, the results would just be hopelessly compromised by a number of compounding factors I don't think you could tune out.

Cheers - N

Please note all comments are from my own brain and don't necessarily represent the position or opinions of my employer, previous employers, colleagues, friends or pets.

cbrpilot

964 posts

Ultimate Geek
+1 received by user: 555

Trusted

Spark NZ

#2734622 25-Jun-2021 17:14

michaelmurfy:

Yes, I get you... But as they're not making this public showing a breakdown between providers there is little information that holds ISP's accountable. CDN tests are not enough because as we know from a particular ISP they can quite easily call themselves the fastest (oh look - we've got the most gigabit customers hitting our caches at the fastest speeds. We're going to call ourselves the fastest ISP in New Zealand).

Which is why Samknows test to their own private servers and that is where their speed graphs all come from. So they cover off both of your points in regards testing.

Yes it's frustrating that providers get away with misleading performance claims.

michaelmurfy:

And now a pain point of yours... Cloudflare which quite happily goes over the NZ POP for every ISP except one.

I know there are a few people out there that have very strong feelings on this topic. I'm not going to get into this discussion here.

My views are my own, and may not necessarily represent those of my employer.

Lias

5655 posts

Uber Geek
+1 received by user: 3978

ID Verified

Trusted

Lifetime subscriber

#2734636 25-Jun-2021 17:58

Talkiet:

You could run the test, the results would just be hopelessly compromised by a number of compounding factors I don't think you could tune out.

Cheers for explaining in a bit more detail.

1 | ... | 15 | 16 | 17 | 18 | 19 | 20 | 21