BigPipe high latency fibre connection

Forums › Spark New Zealand (including Skinny and BigPipe) › BigPipe high latency fibre connection

JonoNZ

274 posts

Ultimate Geek

#272174 12-Jun-2020 17:50

Looking for ideas on what might be going on here.

I have two properties, one in Auckland and one in Napier. Both have Fibre, the Auckland one on Spark, the Napier one on BigPipe.

I am experiencing high latency on the Napier property and I can't understand why.

All tests are via ethernet, and some have been done directly from a MacBook to the ONT with a properly configured PPPoE connection. All connection types present the same results.

Ping tests to popular servers (eg wikihow.com) is approx avg 1.5ms on the Auckland connection, but avg 20ms on the Napier connection. That seems unusual to me.

I have logged with BigPipe, run numerous tests (pings traceroute), had results forwarded to Chorus but they say everything is normal.

This kind of latency difference isn't normal for provinces like Hawkes Bay/Napier is it?

Any ideas advice appreciated!

1 | 2 | 3 | 4

13457 posts

Uber Geek

ID Verified

Lifetime subscriber

#2503835 12-Jun-2020 19:33

where is the traffic going from Napier?

post the trace routes for both

you are unlikely to notice that difference in everyday use

cyril7

9058 posts

Uber Geek

ID Verified

Trusted

Subscriber

#2503843 12-Jun-2020 20:06

Latency to Auckland will be 7-10ms, and that should be it.

However from Auckland each isp will potentially route quite different.

Cyril

chevrolux

4962 posts

Uber Geek
Inactive user

#2503849 12-Jun-2020 20:39

What's the issue with 20ms latency to a website?!?!

JonoNZ

274 posts

Ultimate Geek

#2503851 12-Jun-2020 20:46

chevrolux: What's the issue with 20ms latency to a website?!?!

I know it’s ok, but I’d like to understand why it is 1.5ms on another fibre connection.

cyril7

9058 posts

Uber Geek

ID Verified

Trusted

Subscriber

#2503854 12-Jun-2020 21:01

Ya cannot break the laws of physics, it takes time to travel from the bay to Auckland, and extra 10-15mS is what I would expect

You are saying the spark connection is in Auckland the BP which is the exact same network is from the Bay, so the extra transit time .

Cyril

JonoNZ

274 posts

Ultimate Geek

#2503855 12-Jun-2020 21:04

cyril7: Ya cannot break the laws of physics, it takes time to travel from the bay to Auckland, and extra 10-15mS is what I would expect

You are saying the spark connection is in Auckland the BP which is the exact same network is from the Bay, so the extra transit time .

Cyril

Thanks Cyril, I guess that is what I’m trying to understand — is this normal from Hawkes Bay.

Linux

11399 posts

Uber Geek

Trusted

Lifetime subscriber

#2503856 12-Jun-2020 21:06

Looks like you are looking for a problem when you don't actually have one

Equinox IT

Cloud spending continues to surge globally, but most organisations haven’t made the changes necessary to maximise the value and cost-efficiency benefits of their cloud investments. Download the whitepaper From Overspend to Advantage now.

EnragedStoat

14 posts

Geek

#2519503 8-Jul-2020 20:03

Did you ever find the cause of the latency? Just got hooked up with Bigpipe fibre and have a similar latency problem - 20ms to the first hop :( Am in Wellington though.

Linux

11399 posts

Uber Geek

Trusted

Lifetime subscriber

#2519505 8-Jul-2020 20:05

EnragedStoat:

Did you ever find the cause of the latency? Just got hooked up with Bigpipe fibre and have a similar latency problem - 20ms to the first hop :( Am in Wellington though.

@EnragedStoat It's not a problem or fault

EnragedStoat

14 posts

Geek

#2519834 9-Jul-2020 09:51

It's not a fault in the same way buying a flight from Wellington to Auckland and being flown in a Cessna 172 is not a fault. Sure, destination reached but it was a tortuously slow journey.

It may not be a problem for you but for me (and most gamers) it's a problem of life and death. Here's a picture, new Bigpipe fibre connection on the left, existing Vodafone fakefibre on the right.

Traceroutes taken from a desktop i7 PC plugged directly into a Ubiquiti USG LAN port linking at gigabit - so no other traffic on the LAN. (Have also performed a PPoE test plugged direct into the ONT with similar results).

The results raised a few questions.

1. A minimum 18ms to the first hop? I live in Wellington, not exactly devoid of infrastructure. Even if it's 50km away fibre lightspeed could manage that in <1ms.

2. What's with the 33ms first hop third result?

3. Blocking ICMP on the 3rd hop? Unusual for infrastructure but OK.

4. I went looking for stats and found a comcom page full of them. Gaming latency stats to server fifa18.content.easports.com, 814 test locations around NZ, >100k tests, results between 1ms and 278ms. My ping to fifa18 a consistant 55ms placing me 801st, out of 814 results.

How would you explain 18ms? What is your first hop like? Would you be satisfied with 18-30ms? Is there a way to get stats out of an ONT?

Pity, I'm more than happy with overall throughput - primary reason for getting glass was increased uplink speeds - didn't expect to have to think about latency :(

RunningMan

8953 posts

Uber Geek

#2519852 9-Jul-2020 10:06

1) your first hop is probably Auckland. More than 50km away.

2) ICMP is frequently deprioritised on routers as it is less important. Latency end to end is what you should be concerned with.

3)As (2)

4) You have 13ms total round trip difference between two ISPs that route traffic differently.

If the most important measure for a connection for you is latency to a specific server, it would have been a good idea to ask people here for some quick tests from different ISPs to see what suited you best.

Ultimately, 13ms is unlikely to be life and death.

quickymart

13927 posts

Uber Geek

ID Verified

#2519866 9-Jul-2020 10:21

Life and death? In a game, maybe.

cyril7

9058 posts

Uber Geek

ID Verified

Trusted

Subscriber

#2519881 9-Jul-2020 10:37

quickymart:

Life and death? In a game, maybe.

As a non gamer I dont understand

Cyril

EnragedStoat

14 posts

Geek

#2519883 9-Jul-2020 10:42

RunningMan:

1) your first hop is probably Auckland. More than 50km away.

2) ICMP is frequently deprioritised on routers as it is less important. Latency end to end is what you should be concerned with.

3)As (2)

4) You have 13ms total round trip difference between two ISPs that route traffic differently.

If the most important measure for a connection for you is latency to a specific server, it would have been a good idea to ask people here for some quick tests from different ISPs to see what suited you best.

Ultimately, 13ms is unlikely to be life and death.

1. Ok. 18ms is still weirdly slow.

2. Ok. Was a different route without a block the first day or so after going live. Latency end to end is mostly what I'm concerned with and that mostly seems fine (see 4 below). If priorities cause such a consistent delay over such a small footprint of the route perhaps there's something more fundamental in play.

4. Different routes/ISPs that both take <10ms from hop 4 to reach the destination, so no problem there. Before hop 4 is the problem.

I was just using one server as an example. How does your first hop look?

hio77

12999 posts

Uber Geek

ID Verified

Trusted

Lizard Networks

#2519885 9-Jul-2020 10:44

cyril7:

quickymart:

Life and death? In a game, maybe.

As a non gamer I dont understand

Cyril

as a gamer, i get it. but It's also just physics and networks.

nomatter your provider, there is always going to be somewhere that has worse latency than another provider due to a peering desision.

Providers have to balance so many thing, you could go and engineer a network for latency, but then have you accounted for being a lossless path? a fast path?

to 99% of people a few ms of latency is hardly going to cause issues, but your video loading buffering because it travels a congested path is much more important...

As a gamer, 13ms makes next to no differance, i'll take 13ms over having even 0.1% packetloss.

So many game engines expect no packetloss, particularly if you say tune the lerp out of source engine, then you really do feel those missing packets. not the extra 1 frame of latency.

#include <std_disclaimer>

Any comments made are personal opinion and do not reflect directly on the position my current or past employers may have.

1 | 2 | 3 | 4