SamKnows Monthly Scorecard

Forums › New Zealand Broadband › SamKnows Monthly Scorecard

1 | 2 | 3

meow
13242 posts

Uber Geek

Moderator

ID Verified

Trusted

Lifetime subscriber

#2229348 2-May-2019 12:24

Mikek:

Believe it or not that result for Spark is totally normal, The SamKnows report card we get sent is showing local traffic results only.

Due to Sparks peering / non-peering to REANNZ that is the speed result to the NZ test server which most likely is going overseas then back again. Its not a true indication of the speeds in general you are going to get to most sites/area's in NZ,

I'm not going to beat a dead horse here and this is the only message you'll see from me on the subject but it is more an indication of user experience to some local NZ resources. You shouldn't have to pick servers to test to, the customer doesn't know where the server is located. How I see it is if a service was hosted at REANNZ then customers on Spark would experience traffic going via Australia, depending on the service this may have an impact on that service (via means of speed and latency). It is measuring true user experience for the customers on that ISP accessing that service.

If SamKnows, lets say, starts testing to a host that is behind a popular CDN one ISP may have that traffic go via Japan or Hong Kong where other ISP's will have traffic go via Auckland or Sydney. Not going to name anyone here, but that is what user experience testing is about.

ISP's should be focusing on providing the best user experience for its customers as possible. I get it, some ISP's do things different than others but the great thing about SamKnows is it is now pointing out those ISP's and encouraging them to change their policies around this.

Michael Murphy | https://murfy.nz
Referral Links: Quic Broadband (use R122101E7CV7Q for free setup)

Are you happy with what you get from Geekzone? Please consider supporting us by subscribing.
Opinions are my own and not the views of my employer.

Mikek

108 posts

Master Geek

#2229453 2-May-2019 13:48

That's why just below the part you clipped, I said "It is a true indication of what you get to REANNZ". But saying that it's a true indication of what spark is like for the majority of users would be a tad misleading. That's why I also put up the CDN results.
I mean I'm all for public peering and I'm not a spark fan boy, it's just I tested the spark network not to long ago and it was decent. Not the best but better then most.

Talkiet

4792 posts

Uber Geek

Trusted

#2229464 2-May-2019 13:56

Mikek: That's why just below the part you clipped, I said "It is a true indication of what you get to REANNZ". But saying that it's a true indication of what spark is like for the majority of users would be a tad misleading. That's why I also put up the CDN results.
I mean I'm all for public peering and I'm not a spark fan boy, it's just I tested the spark network not to long ago and it was decent. Not the best but better then most.

In all seriousness, if you're prepared to share the results of your testing I'd love to see it.

Cheers - N

Please note all comments are from my own brain and don't necessarily represent the position or opinions of my employer, previous employers, colleagues, friends or pets.

BarTender

3606 posts

Uber Geek

ID Verified

Trusted

Lifetime subscriber

#2229469 2-May-2019 14:04

michaelmurfy:

Mikek:

Believe it or not that result for Spark is totally normal, The SamKnows report card we get sent is showing local traffic results only.

Due to Sparks peering / non-peering to REANNZ that is the speed result to the NZ test server which most likely is going overseas then back again. Its not a true indication of the speeds in general you are going to get to most sites/area's in NZ,

I'm not going to beat a dead horse here and this is the only message you'll see from me on the subject but it is more an indication of user experience to some local NZ resources. You shouldn't have to pick servers to test to, the customer doesn't know where the server is located. How I see it is if a service was hosted at REANNZ then customers on Spark would experience traffic going via Australia, depending on the service this may have an impact on that service (via means of speed and latency). It is measuring true user experience for the customers on that ISP accessing that service.

If SamKnows, lets say, starts testing to a host that is behind a popular CDN one ISP may have that traffic go via Japan or Hong Kong where other ISP's will have traffic go via Auckland or Sydney. Not going to name anyone here, but that is what user experience testing is about.

ISP's should be focusing on providing the best user experience for its customers as possible. I get it, some ISP's do things different than others but the great thing about SamKnows is it is now pointing out those ISP's and encouraging them to change their policies around this.

As we just discussed in person. The results are clearly only a representative test to REANNZ and not at all real world. The number of customers connecting from Spark to REANNZ prior to the SamKnows probes going in would be such a minor number that it is laughable.

We also know that REANNZ have intentionally throttled inbound international connections to 100mbit that directly impacts Spark because they don't peer. So it was an intentional act on REANNZs part that helps to force Sparks hand to peer.

It doesn't have any real impact on 99.99% of Spark customers as they never attempt to connect to REANNZ. Doing a speedtest against Speedtest, Fast will give you actual real world results vs the results from SamKnows today.

dylan

1 post

Wannabe Geek

Trusted

#2229473 2-May-2019 14:07

Hi all, wearing my REANNZ hat today.

We host the M-Lab servers that SamKnows are using for some of their tests. We've hosted servers for M-Lab for something like four years now. I believe these are the only M-Lab servers in NZ.

We peer at the major exchanges around NZ (APE, AKL-IX, WIX, and CHIX).

We peer with all the RSP's being tested (except Spark) either directly or via route-servers on an exchange.

We are working to establish domestic peering with Spark and believe it's not far away now, but the process has taken longer than anyone would have liked.

In the meantime the M-Lab traffic to Spark is exchanged via Sydney where Spark appear happy to exchange routes with anyone on the Equinix exchange.

We rate-limit our customer's international internet traffic and although M-Lab isn't a customer we treat them as one in this regard. We have to ensure our expensive international bandwidth is fairly distributed among our customers so the allocation to M-Lab is designed to support remote "command and control" of the servers rather than support all of Spark's testing traffic. Domestically there aren't any limits applied to the M-Lab traffic.

I've seen a number of comments in the past along the lines of "traffic to REANNZ doesn't matter, it's only small and not representative". I think those comments are unfair for a number of reasons:

- We have thousands of staff and students from education and research institutes who access resources at their institutions remotely which is important to them (for example a student remotely attending a lecture or distance learning).

- The connectivity from the RSP's to REANNZ is likely to be the same or similar as their connectivity to other providers throughout NZ. Good connectivity to REANNZ is likely indicative of good connectivity throughout NZ.

If you just consider the volume of traffic when deciding what's important it's easy to reach the opinion that the only important parts of the internet are Google, Facebook, Netflix, and a handful of CDN's. SamKnows have separate tests for CDN's but I think domestic connectivity is also a key metric for evaluating a service.

If anyone is interested in the traffic from the M-Lab servers we have publicly available graphs (I'm not sure these links will come through, I'm new to Geekzome so may not be able to include them):

Auckland: https://weathermap.reannz.co.nz/node.php?src=and01&int=xe_0_0_9

Wellington: https://weathermap.reannz.co.nz/node.php?src=and02&int=xe_0_2_7

Thanks,

Dylan

BarTender

3606 posts

Uber Geek

ID Verified

Trusted

Lifetime subscriber

#2229507 2-May-2019 15:04

dylan:

.snip.

I've seen a number of comments in the past along the lines of "traffic to REANNZ doesn't matter, it's only small and not representative". I think those comments are unfair for a number of reasons:

- We have thousands of staff and students from education and research institutes who access resources at their institutions remotely which is important to them (for example a student remotely attending a lecture or distance learning).

- The connectivity from the RSP's to REANNZ is likely to be the same or similar as their connectivity to other providers throughout NZ. Good connectivity to REANNZ is likely indicative of good connectivity throughout NZ.

If you just consider the volume of traffic when deciding what's important it's easy to reach the opinion that the only important parts of the internet are Google, Facebook, Netflix, and a handful of CDN's. SamKnows have separate tests for CDN's but I think domestic connectivity is also a key metric for evaluating a service.

Point absolutely taken here. Would the same arguments be said for many government agencies such as IRD or DIA who do have a public facing site as well as the Banks etc and their interconnection to Spark and all other ISPs.

My only question is in regards to the real world impact to students working remotely then why is it only being sorted out now as part of the speedtest server rather than years ago when REANNZ was first established.

If it were a real issue why is no one screaming about it today pressuring Spark to peer with everyone or customers leaving them in droves because of lack of peering?

The reason why this is changing is only because a graph showing speed is being intentionally throttled because of an international rate limit.

I would be very interested once the peering links go in if anyone notices any real world difference especially remote working of students who are Spark customers, as I doubt that anyone would notice.

ANglEAUT

2320 posts

Uber Geek

Trusted

Lifetime subscriber

#2229749 2-May-2019 20:29

Getting from 2Deg what I'm paying for.

Please keep this GZ community vibrant by contributing in a constructive & respectful manner.

Wise

Send money globally for less with Wise - one free transfer up to NZ$900 (affiliate link).

sbiddle

30853 posts

Uber Geek

Retired Mod

Trusted

Biddle Corp

Lifetime subscriber

#2229883 3-May-2019 07:30

The whole SamKnows situation really has to start posing the question right now of what meaningful data it's delivered with the money that's been spent.

insane

3239 posts

Uber Geek

ID Verified

Trusted

#2239579 16-May-2019 23:02

Because someone mentioned the affects of peering and also N4L, I'll say that back when we compelled Spark to peer at APE and setup several other bi-lateral peerings for us, schools IT techs noticed a huge difference in the time taken to download ISOs and other large files from local sources.

From a user experience/responsiveness point of view, teachers and admin staff were relieved to have an order of magnitude improvement when accessing payroll and several other Ministry hosted services which presumably had limited Spark domestic connectivity. Put it this way - the complaints stopped and the phones stopped ringing hot.

Fast forward some 4 years+, the difference is far far less noticeable. We've several times dropped the peering for various maintenance and other reasons with almost no noticeable difference, or complaints.

As an aside - it till be interesting to know whether peering will suddenly become available for the smooth streaming of the Rugby later this year, or if Spark will use it as leverage to sell some more domestic transit. Who knows, maybe Sam knows?

sbiddle

30853 posts

Uber Geek

Retired Mod

Trusted

Biddle Corp

Lifetime subscriber

#2239651 17-May-2019 07:20

insane:

As an aside - it till be interesting to know whether peering will suddenly become available for the smooth streaming of the Rugby later this year, or if Spark will use it as leverage to sell some more domestic transit. Who knows, maybe Sam knows?

Spark Sport is delivered using Akamai so Spark's peering policy is irrelevant.

ABPH011

25 posts

Geek

#2246065 26-May-2019 19:24

I well it looks like something changed somewhere 4-12 May?

sbiddle

30853 posts

Uber Geek

Retired Mod

Trusted

Biddle Corp

Lifetime subscriber

#2246107 26-May-2019 19:40

ABPH011:

I well it looks like something changed somewhere 4-12 May?

Either direct peering or REANZ removing their throttling.

Dratsab

3946 posts

Uber Geek

Trusted

Lifetime subscriber

#2246111 26-May-2019 19:54

This thread made me go and have a look at my SamKnows stats which I've not previously bothered looking at despite having had a whitebox pretty much since they started operating here. Apparently it's not working despite me having followed all their set up instructions. I'll double check connections and do a power cycle of the whitebox when I get home at about 2am.

BarTender

3606 posts

Uber Geek

ID Verified

Trusted

Lifetime subscriber

#2246117 26-May-2019 20:16

sbiddle:
ABPH011: I well it looks like something changed somewhere 4-12 May?

Either direct peering or REANZ removing their throttling.

It's the former.

And I doubt anyone has noticed any improvement in overall end-user experience. It only may positively impact Staff / Students at Universities who use on-prem university services and the speedtest for the SamKnows boxes.

That's all that has changed, peering just for the SamKnows speedtest servers.

michaelmurfy

meow
13242 posts

Uber Geek

Moderator

ID Verified

Trusted

Lifetime subscriber

#2246119 26-May-2019 20:25

BarTender:

And has anyone noticed any improvement in overall experience. Only those who interconnect to a university or have a SamKnows box.

It is a small step in the right direction. Good job to those at Spark who made it happen.

1 | 2 | 3