Weird UFB Uplink issues

Forums › Spark New Zealand (including Skinny and BigPipe) › Weird UFB Uplink issues

droopanu

168 posts

Master Geek

#222947 5-Sep-2017 10:34

I recently got Fiber installed and I've been having a very weird issue with uplink from the first day. Initially I was on a standard 100/20 plan, and I was getting about 1Mbps uplink speed. I blamed it on miss configurations and things. I did not pay too much attention to it, especially as I was planning to upgrade to "gigabit"

After upgrading to gigabit, I noticed that certain times the uplink is still misbehaving. Getting about 20-30mbps uplink on ethernet. Other times I would get over 400mbps. I tried using windows laptops, using the windows app and the web version of speedtest, connected via ethernet. I even tried connecting directly to the ONT. I got pretty much the same results.

I must say that during these times, the downlink speeds are really good.

So I got an automated speedtest-cli running every 20 minutes from a linux box and I can see good speeds most of the time, but certain periods the uplink maxes out at about 120mbps. This is the first weird part. A speed test from browser on any platform or from windows app, maxes at 20-30mbps, while the speedtest-cli maxes out at 120mbps. That makes me think there is either a packet loss or a QoS being handled differently by different platforms.

A PRTG monitorig, even with large packets (smokeping for windows) does not show any packet loss during those times. So it looks like only the high throughput flows are affected.

So all this was happening while I was subscribed to Stuff-Fibre. I opened a ticket but got dismissed as I was considered a heavy user and everything looked ok to them. And then never heard anything else.

I heard that BigPipe is awesome so I switched to BigPipe, hoping that the issue will go away in the process.

But of course I have the same issue on BigPipe. It's just my luck.

I have opened a ticket with BigPipe and will see where it goes. So far they asked for times when this is happening so they go to Chorus, so I'll collect some details over the next week or so.

I am just curious if anyone else has seen any issues like this, and if anyone has any other ideas of things that could be wrong.

See the uplink graph from the speedtest-cli since I switched to BP:

Also, A good and a bad speedtest. I know that the links say wellington server, but they were actually done with the auckland server. It's a weird behavior when using the windows app :)

Good one: http://www.speedtest.net/my-result/d/d14d34bb-b6c6-41ec-a363-ef9860d60bf0

Bad one: http://www.speedtest.net/my-result/d/24b791e5-a508-4eeb-a69e-0db3ede38e4f

1 | 2

meow
13256 posts

Uber Geek

Moderator

ID Verified

Trusted

Lifetime subscriber

#1858773 5-Sep-2017 10:48

Sounds like to me your router simply isn't coping - what kind of router have you got running? Also Speedtest-CLI is rubbish on fast connections.

Michael Murphy | https://murfy.nz
Referral Links: Quic Broadband (use R122101E7CV7Q for free setup)

Are you happy with what you get from Geekzone? Please consider supporting us by subscribing.
Opinions are my own and not the views of my employer.

droopanu

168 posts

Master Geek

#1858775 5-Sep-2017 10:52

That's the thing.

I tried with 2 routers (google onhub and am Asus RT-AC85U).

I have also connected a few computers directly to the ONT. My connection is currently IPoE. So no PPPoE overhead either.

While speedtest-cli can be a bit rubbish, at least it gives me a good way of monitoring the connection. I can clearly see when the issue pops up or goes away.

hio77

12999 posts

Uber Geek

ID Verified

Trusted

Lizard Networks

#1858801 5-Sep-2017 11:21

speedtest cli fails to preform well on 100Mbit+ connections with ease. when you get faster than this it just gets worse again... (i would share 10gbit results but that gets totally off topic)

Comparing this to running a real browser test can be light and day.

For scripting, something like curl or mget would be best.

For upload though, this is a tough one to measure effectively...

For the most part, graphs look like peak time congestion or as others have mentioned simply router not preforming.

While realistically you should be able to get full speed 24/7 do remember gpon by nature is shared bandwidth.

#include <std_disclaimer>

Any comments made are personal opinion and do not reflect directly on the position my current or past employers may have.

droopanu

168 posts

Master Geek

#1858823 5-Sep-2017 11:33

I doubt that there would be peak time between 4pm and 8am :)

Plus, during these times a speedtest on a wifi connected device drops to 5-8Mbps. On the same network setup I used to get 24Mbps of constant upload speed on my old VDSL connection.

Put it this way. On VDSL, with the same router, same wifi, same internal network, I used to get better uplink speeds for real word usage than I get now on gigabit fibre, at least during certain times.

And the router is out of the question, as I can see exactly the same issue across two routers, and even without a router :)

insane

3240 posts

Uber Geek

ID Verified

Trusted

#1858869 5-Sep-2017 12:31

I suspect this is only going to happen more and more on GPON services. Where exactly are you located? If someone or multiple other people (including say a School) on the same optical splitter are running say backup replication or similar then I could see how this could affect UFB. VDSL is a different network.

The BS2a service template hasn't been designed for sustained throughput, so you'll probably get little sympathy from your residential ISP

johny99

495 posts

Ultimate Geek
Inactive user

#1858893 5-Sep-2017 12:59

insane:

I suspect this is only going to happen more and more on GPON services. Where exactly are you located? If someone or multiple other people (including say a School) on the same optical splitter are running say backup replication or similar then I could see how this could affect UFB. VDSL is a different network.

The BS2a service template hasn't been designed for sustained throughput, so you'll probably get little sympathy from your residential ISP

This bit is always fun explaining to Joe Average who has no idea as to how GPON is designed.

hio77

12999 posts

Uber Geek

ID Verified

Trusted

Lizard Networks

#1858924 5-Sep-2017 13:12

insane:

I suspect this is only going to happen more and more on GPON services. Where exactly are you located? If someone or multiple other people (including say a School) on the same optical splitter are running say backup replication or similar then I could see how this could affect UFB. VDSL is a different network.

The BS2a service template hasn't been designed for sustained throughput, so you'll probably get little sympathy from your residential ISP

This is why i love some of the LFC's

in querying the ONT, you will get handover link utilization figures and all.

this makes it really easy to go, okay lets have a look rather than the battle that is some of the more closed doors to pickup on a fault.

Hint to LFCs: Presenting real data is far better than "Expected Provisioned data"

#include <std_disclaimer>

Any comments made are personal opinion and do not reflect directly on the position my current or past employers may have.

Quic Broadband

Move to New Zealand's best fibre broadband service (affiliate link). Free setup code: R587125ERQ6VE. Note that to use Quic Broadband you must be comfortable with configuring your own router.

sbiddle

30853 posts

Uber Geek

Retired Mod

Trusted

Biddle Corp

Lifetime subscriber

#1858949 5-Sep-2017 13:35

johny99:

insane:

I suspect this is only going to happen more and more on GPON services. Where exactly are you located? If someone or multiple other people (including say a School) on the same optical splitter are running say backup replication or similar then I could see how this could affect UFB. VDSL is a different network.

The BS2a service template hasn't been designed for sustained throughput, so you'll probably get little sympathy from your residential ISP

This bit is always fun explaining to Joe Average who has no idea as to how GPON is designed.

And then explaining the only bandwidth they're guaranteed is their 2.5Mbps up and down which is their high priority CIR, and that you can only access this with the correct 802.1p tagging.

johny99

495 posts

Ultimate Geek
Inactive user

#1858963 5-Sep-2017 13:46

sbiddle:

johny99:

insane:

I suspect this is only going to happen more and more on GPON services. Where exactly are you located? If someone or multiple other people (including say a School) on the same optical splitter are running say backup replication or similar then I could see how this could affect UFB. VDSL is a different network.

The BS2a service template hasn't been designed for sustained throughput, so you'll probably get little sympathy from your residential ISP

This bit is always fun explaining to Joe Average who has no idea as to how GPON is designed.

And then explaining the only bandwidth they're guaranteed is their 2.5Mbps up and down which is their high priority CIR, and that you can only access this with the correct 802.1p tagging.

Its also annoying that most who work for the ISP's do not even know this stuff

droopanu

168 posts

Master Geek

#1858975 5-Sep-2017 13:55

johny99:

insane:

I suspect this is only going to happen more and more on GPON services. Where exactly are you located? If someone or multiple other people (including say a School) on the same optical splitter are running say backup replication or similar then I could see how this could affect UFB. VDSL is a different network.

The BS2a service template hasn't been designed for sustained throughput, so you'll probably get little sympathy from your residential ISP

This bit is always fun explaining to Joe Average who has no idea as to how GPON is designed.

I am in Wellington, in a newly "cabled" area. I'm pretty sure I was the first on my street to be installed, and one of the first few on the roadside cabinet.

My understanding is that GPON has 1Gbps uplink, divided by max 16 subscribers.

If everyone pushes it to the max, I would expect around 60Mbps uplink, not 20-30, like I get from speedtest.

Knowing my area, I would bet almost everyone is on the 100/20 plan.

It's really hard to imagine this 1Gbps was full from the installation date...If that's the case, I would really like to see a graph :D

Also, in case of congestion, I would expect the service to degrade gradually, not the way it's happening now. Right now it's really good or really bad :) No inbetween.

And it happens at very strange hours :) - between 12am and 4pm on a Saturday or between 4pm monday at 8am on a tuesday ... :D

droopanu

168 posts

Master Geek

#1858978 5-Sep-2017 13:57

And then explaining the only bandwidth they're guaranteed is their 2.5Mbps up and down which is their high priority CIR, and that you can only access this with the correct 802.1p tagging.

I'm currently using IPoE without vlan, and on Stuff I used IPoE with vlan.

What would be the correct 802.1p tag? I can always insert a switch with a mirror port in between and see if I can check the tag :D

droopanu

168 posts

Master Geek

#1859448 6-Sep-2017 09:48

I managed to get access to a linux box connected to Spark via symmetric 1GE and run some iperf tests, using only one thread:

This is how it looks when everything is good:

[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 5] 0.00-1.00 sec 38.3 MBytes 320 Mbits/sec 16 1.07 MBytes
[ 5] 1.00-2.00 sec 39.5 MBytes 332 Mbits/sec 18 831 KBytes
[ 5] 2.00-3.00 sec 42.0 MBytes 352 Mbits/sec 0 880 KBytes
[ 5] 3.00-4.00 sec 33.3 MBytes 279 Mbits/sec 1 667 KBytes
[ 5] 4.00-5.00 sec 31.2 MBytes 262 Mbits/sec 1 503 KBytes
[ 5] 5.00-6.00 sec 25.6 MBytes 214 Mbits/sec 0 544 KBytes
[ 5] 6.00-7.00 sec 26.0 MBytes 218 Mbits/sec 1 404 KBytes
[ 5] 7.00-8.00 sec 21.1 MBytes 177 Mbits/sec 0 441 KBytes
[ 5] 8.00-9.00 sec 22.4 MBytes 188 Mbits/sec 0 477 KBytes
[ 5] 9.00-10.00 sec 24.4 MBytes 205 Mbits/sec 0 515 KBytes
[ 5] 10.00-10.03 sec 491 KBytes 145 Mbits/sec 0 516 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 5] 0.00-10.03 sec 304 MBytes 254 Mbits/sec 37 sender

And this is how it looks when things are bad:

Starting Test: protocol: TCP, 1 streams, 131072 byte blocks, omitting 0 seconds, 10 second test
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 5] 0.00-1.00 sec 356 KBytes 2.92 Mbits/sec 10 7.07 KBytes
[ 5] 1.00-2.00 sec 229 KBytes 1.88 Mbits/sec 11 7.07 KBytes
[ 5] 2.00-3.00 sec 260 KBytes 2.13 Mbits/sec 8 7.07 KBytes
[ 5] 3.00-4.00 sec 320 KBytes 2.62 Mbits/sec 7 5.66 KBytes
[ 5] 4.00-5.00 sec 263 KBytes 2.15 Mbits/sec 7 9.90 KBytes
[ 5] 5.00-6.00 sec 427 KBytes 3.50 Mbits/sec 7 7.07 KBytes
[ 5] 6.00-7.00 sec 354 KBytes 2.90 Mbits/sec 7 7.07 KBytes
[ 5] 7.00-8.00 sec 338 KBytes 2.77 Mbits/sec 7 12.7 KBytes
[ 5] 8.00-9.00 sec 440 KBytes 3.60 Mbits/sec 8 5.66 KBytes
[ 5] 9.00-10.00 sec 436 KBytes 3.57 Mbits/sec 7 4.24 KBytes
[ 5] 10.00-10.02 sec 11.3 KBytes 4.30 Mbits/sec 0 5.66 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
Test Complete. Summary Results:
[ ID] Interval Transfer Bandwidth Retr
[ 5] 0.00-10.02 sec 3.35 MBytes 2.81 Mbits/sec 79 sender

This is with a linux box connected directly to the ONT.

So a single TCP session can be throttled down to about 2-3Mbps.

That explains why plex is buffering when playing videos from remote locations.

insane

3240 posts

Uber Geek

ID Verified

Trusted

#1859535 6-Sep-2017 10:57

droopanu:

...What would be the correct 802.1p tag? I can always insert a switch with a mirror port in between and see if I can check the tag :D

They'll just drop the tags and re-rewite it as best-efforts, little to no point unless it's honoured end-to-end and you're doing both cos and dscp tagging. I suspect your remote destination is well past your ISPs boarders anyway.

Only thing you can do is prioritise locally your outbound traffic, which when there is congestion semi-works a little bit - I used to do it back when I was flatting and needed to fight for my bandwidth.

IPv6pipe

98 posts

Master Geek

Trusted

BigPipe

#1860672 8-Sep-2017 08:45

@droopanu Have you logged a fault via https://www.bigpipe.co.nz/contact-us about this? If not, this would be your best bet, as our support team can chase up the appropriate parties as needed.

As others in this thread have mentioned the minimum throughput guaranteed between the active ONT port at your place and our handover point is 2.5Mbps up and down. Everything above that is treated as best effort by the LFC.

From a Bigpipe perspective, looking at the utilisation graph on our Wellington UFB handover there's ample bandwidth available at all times of the day for you to get full speeds up and down, so this is an unusual one!

droopanu

168 posts

Master Geek

#1860679 8-Sep-2017 09:12

@IPv6pipe of course I opened a ticket first :)

The reason I started the thread, is that I wanted to see if my experience is normal or not :)

1 | 2