Horrible YouTube performance

Forums › Spark New Zealand (including Skinny and BigPipe) › Horrible YouTube performance

1 | 2 | 3 | 4 | 5 | 6

blazer

15 posts

Geek

#1519583 25-Mar-2016 12:25

+1 disabling QUIC protocol on chrome fixed youtube buffering problem for me too.

I joined bigpipe 4/3/16 and had problems from then until a couple of nights ago when I disabled QUIC in chrome on Win7.

I have a static ip address.

Tradepub

Free professional, reference and technical white papers (affiliate link).

frysbil

1 post

Wannabe Geek

#1523481 31-Mar-2016 17:56

Fellow Bigpipe user here. Was having similar YouTube buffering issues till I changed the QUIC protocol on Google chrome, after which YouTube is working properly again.

YouTube videos worked on Microsoft Edge when I tried.

As mentioned by Josh here I have gone ahead and lodged an issue with Bigpipe for them to have enough examples to diagnose and resolve the issue.

MediaLight

50 posts

Geek

#1523488 31-Mar-2016 18:04

@joshbigpipe

Josh this has gone on for a while can you please advise what's the latest and when this issue will be resolve rather than just a work around

allio

883 posts

Ultimate Geek

#1530878 12-Apr-2016 15:33

I coincidentally came across this post today. Summary: the default QoS settings in some router firmware has a catch-all rule that throttles all unidentified UDP traffic to 5% of bandwidth (labelled “Crawl”). Google QUIC is caught by this rule and throttled within an inch of its life.

Seeing the variety of people experiencing the issue, and the fact that Bigpipe is BYO-device, it seems unlikely to be this exact issue occurring at the consumer end, but maybe it's something further upstream?

JoshBigpipe

122 posts

Master Geek

Trusted

BigPipe

#1531373 13-Apr-2016 09:50

allio:

I coincidentally came across this post today. Summary: the default QoS settings in some router firmware has a catch-all rule that throttles all unidentified UDP traffic to 5% of bandwidth (labelled “Crawl”). Google QUIC is caught by this rule and throttled within an inch of its life.

This is very interesting. Great find. UDP throttling thanks to dicky QoS settings seems like it could answer for a lot of the problems we're seeing here - bad performance and packet loss on specific applications, as well as why the QUIC workaround actually works for YouTube. It'd also explain why this is such a scattered issue, and one that our network ops people haven't been able to identify as being generated by our network.

allio:

Seeing the variety of people experiencing the issue, and the fact that Bigpipe is BYO-device, it seems unlikely to be this exact issue occurring at the consumer end, but maybe it's something further upstream?

To my mind the fact that we're BYO device makes it much more likely that the issue is occurring at consumer end. There's nothing tell-tale about this issue that we'd expect from a network difficulty. Our customers have a huge variety of modems and - if QoS UDP throttling is the cause of the problem - it's likely that anyone with a certain sort of firmware is getting stung. If it was a service like Spark or Voda where a huge proportion of customers have ISP-provided modems and firmware, we'd expect to see a widespread issue affecting many customers at once, and here we're seeing the opposite. Fortunately, there's a way to tell: all we need to do is look at what modems customers experiencing the issue are using, and see if there are any commonalities. If you have experienced this issue and *haven't* already lodged a support ticket, please feel free to tell me what modem (and firmware if possible) you are using either by PM or replying to the thread. If you've already lodged a ticket, please don't do this, I'd like to avoid double-ups.

I'll update once we have more on this.

www.bigpipe.co.nz

https://www.facebook.com/BigPipeNZ

https://twitter.com/BigPipeNZ

allio

883 posts

Ultimate Geek

#1532416 13-Apr-2016 10:59

Glad to hear I wasn't on the wrong track entirely.

I'm not on Bigpipe yet (should be connected in a few weeks) but I do use a router running Tomato, which has very configurable QoS and is also identified as the culprit in that blog post. I'll do some experimentation with the QoS settings to see if I can force the problem to occur for me.

The other reason I didn't suspect consumer routers as the culprit was that several people in this thread have reported that they only began to experience problems after switching ISPs to Bigpipe. However I suppose that could also be explained by a change in browser at around the same time, or even a change at Google/Youtube's end that caused the problem to begin occurring.

This could be cleared up fairly quickly if someone affected by the problem could report whether or not they have QoS enabled on their router, and if so, whether disabling it/changing the settings fixes the problem.

PS. the two guys reporting getting disconnections from their online game earlier in the thread (who were hushed for getting off topic) may indicate that this is a general issue with UDP traffic rather than QUIC/Youtube specifically.

JoshBigpipe

122 posts

Master Geek

Trusted

BigPipe

#1532464 13-Apr-2016 11:36

allio:

Glad to hear I wasn't on the wrong track entirely.

I'm not on Bigpipe yet (should be connected in a few weeks) but I do use a router running Tomato, which has very configurable QoS and is also identified as the culprit in that blog post. I'll do some experimentation with the QoS settings to see if I can force the problem to occur for me.

That'd be really interesting. Feel very free to let us know what you come up with. Experimenting with other kinds of UDP traffic besides QUIC will be telling.

The other reason I didn't suspect consumer routers as the culprit was that several people in this thread have reported that they only began to experience problems after switching ISPs to Bigpipe. However I suppose that could also be explained by a change in browser at around the same time, or even a change at Google/Youtube's end that caused the problem to begin occurring.

I think we can possibly put this down to coincidence, yeah. I actually had the problem pop up on my Bigpipe connection at home, and I've been a customer and connected at that address for ages. I started experiencing it around the same time as everyone else. I'm on a Spark-model HG659, btw.

EDIT: Actually, this could also be because a sizeable proportion of our customers change modems when they sign up with us - if they're changing up to a VDSL or fibre plan, for example. If they're shifting to a modem that has this QoS it would look like they're only experiencing the problem when switching to Bigpipe.

www.bigpipe.co.nz

https://www.facebook.com/BigPipeNZ

https://twitter.com/BigPipeNZ

allio

883 posts

Ultimate Geek

#1532480 13-Apr-2016 11:56

JoshBigpipe:

I think we can possibly put this down to coincidence, yeah. I actually had the problem pop up on my Bigpipe connection at home, and I've been a customer and connected at that address for ages. I started experiencing it around the same time as everyone else.

EDIT: Actually, this could also be because a sizeable proportion of our customers change modems when they sign up with us - if they're changing up to a VDSL or fibre plan, for example. If they're shifting to a modem that has this QoS it would look like they're only experiencing the problem when switching to Bigpipe.

That last point is a good one - I just bought a new router in preparation for Bigpipe UFB :)

Apparently Google is gradually ramping up the usage of QUIC across their services, with a goal of eventually having 100% of their traffic using the protocol. I suspect there was a bump in about February which increased the proportion of Google traffic being sent to NZ users via QUIC, which meant that the problem began occurring for those users with susceptible setups.

The confusing part is why only users of Spark and Bigpipe have been hit. Bigpipe users would certainly have a greater variety of networking equipment than other ISPs, but not to that level, surely.

Talkiet

4787 posts

Uber Geek

Trusted

#1532487 13-Apr-2016 12:03

allio:[snip]

Apparently Google is gradually ramping up the usage of QUIC across their services, with a goal of eventually having 100% of their traffic using the protocol. I suspect there was a bump in about February which increased the proportion of Google traffic being sent to NZ users via QUIC, which meant that the problem began occurring for those users with susceptible setups.

The confusing part is why only users of Spark and Bigpipe have been hit. Bigpipe users would certainly have a greater variety of networking equipment than other ISPs, but not to that level, surely.

Spark haven't been hit. I can see what percentage of our google traffic is QUIC and it's a large enough percentage that if it was causing problems then I would _definitely_ hear about it.

The short Google performance issue Spark had a few months ago was isolated to a network issue (overseas) and fixed.

Cheers - N

Please note all comments are from my own brain and don't necessarily represent the position or opinions of my employer, previous employers, colleagues, friends or pets.

MediaLight

50 posts

Geek

#1532490 13-Apr-2016 12:14

JoshBigpipe:

allio:

I coincidentally came across this post today. Summary: the default QoS settings in some router firmware has a catch-all rule that throttles all unidentified UDP traffic to 5% of bandwidth (labelled “Crawl”). Google QUIC is caught by this rule and throttled within an inch of its life.

This is very interesting. Great find. UDP throttling thanks to dicky QoS settings seems like it could answer for a lot of the problems we're seeing here - bad performance and packet loss on specific applications, as well as why the QUIC workaround actually works for YouTube. It'd also explain why this is such a scattered issue, and one that our network ops people haven't been able to identify as being generated by our network.

allio:

Seeing the variety of people experiencing the issue, and the fact that Bigpipe is BYO-device, it seems unlikely to be this exact issue occurring at the consumer end, but maybe it's something further upstream?

To my mind the fact that we're BYO device makes it much more likely that the issue is occurring at consumer end. There's nothing tell-tale about this issue that we'd expect from a network difficulty. Our customers have a huge variety of modems and - if QoS UDP throttling is the cause of the problem - it's likely that anyone with a certain sort of firmware is getting stung. If it was a service like Spark or Voda where a huge proportion of customers have ISP-provided modems and firmware, we'd expect to see a widespread issue affecting many customers at once, and here we're seeing the opposite. Fortunately, there's a way to tell: all we need to do is look at what modems customers experiencing the issue are using, and see if there are any commonalities. If you have experienced this issue and *haven't* already lodged a support ticket, please feel free to tell me what modem (and firmware if possible) you are using either by PM or replying to the thread. If you've already lodged a ticket, please don't do this, I'd like to avoid double-ups.

I'll update once we have more on this.

This has all gone quiet as people a putting a work around. I have not heard anything out of BigPipe for this issue even though a ticket was logged.

I have a Fritz VDSL modem works with most other providers fine.

JoshBigpipe

122 posts

Master Geek

Trusted

BigPipe

#1532520 13-Apr-2016 13:04

MediaLight:

This has all gone quiet as people a putting a work around. I have not heard anything out of BigPipe for this issue even though a ticket was logged.

I have a Fritz VDSL modem works with most other providers fine.

In terms of a solution for your ticket - we can't fix a problem that hasn't yet been identified. I'm not assuming this latest development is a definitive fix; it just seems to fit the pattern. I'd definitely like it to be the solution, but it very well may not be. The possibility definitely still exists that it's a network issue, something wrong between Google, our Google cache, and our customers, but bizarrely it's affecting a very small number of customers. Ironically, if it was affecting more it would be easier to pin down and fix.

Our network ops team are still investigating. As soon as we have a definitive fix - whether it's a setting customers can change, or a fix we make on our network, or something Google does - we will let our customers know. We'll keep speculation and workarounds in this thread. All the ideas and workarounds are genuinely interesting, often useful, and they're good to have here, but they're not anything we can send out to our wider customer base.

www.bigpipe.co.nz

https://www.facebook.com/BigPipeNZ

https://twitter.com/BigPipeNZ

Xeon

302 posts

Ultimate Geek

#1532539 13-Apr-2016 13:22

Prior to our ADSL being disconnected for no reason was having issues with youtube buffering which disappeared after disabling QUIC.

Modem/router is Huawei HG659 (Vodafone firmware).

allio

883 posts

Ultimate Geek

#1532549 13-Apr-2016 13:39

Talkiet:

Spark haven't been hit. I can see what percentage of our google traffic is QUIC and it's a large enough percentage that if it was causing problems then I would _definitely_ hear about it.

The short Google performance issue Spark had a few months ago was isolated to a network issue (overseas) and fixed.

Cheers - N

Thanks for the clarification - that's useful.

It's still hard to believe that Bigpipe users have such different equipment setups to other ISPs as to be hit so disproportionately by this issue. I did find one other report from a user on 2degrees with the same issue, but that's it (for Geekzone at least). Maybe there are a scattering of customers of other ISPs who had the same problem, worked out that it only happened in Chrome, switched to Firefox and forgot about it.

I'll report back on my QoS experimentation after work.

1 | 2 | 3 | 4 | 5 | 6