Recent increase in latency to AWS (PUBG game servers)

Forums › 2degrees (including Slingshot, Orcon, Flip, Stuff Fibre, MyRepublic, 2talk and Vocus) › Recent increase in latency to AWS (PUBG game servers)

pooky

43 posts

Geek

#250934 1-Jun-2019 14:49

In the past couple of weeks I've noticed increased latency to north american servers for pubg (game) (+30-60ms), the servers seem to be hosted on Amazon AWS, so probably a fairly common destination. Previously I've got around 180-200ms to the game server (with 2degrees). Though in the last couple of weeks it's gone up to ~240ms consistently. Other friends using different ISPs haven't noticed any change, and still get about 180-200ms. I'm wondering if 2degrees peering has changed recently, possibly there have been changes in upstream bandwidth providers. It's possible that the change was entirely external, e.g. changes on the AWS side rather than anything changes in 2degress network, though given I haven't heard anyone else notice a recent increase in latency, it does seem to be isolated to 2degrees or their providers. Given that Amazon AWS servers should theoretically be a very common destination, I would appreciate if someone can investigate whether any improvements can be made.

So it's not technically a fault as such, though it is a decrease in performance relative to what it used to be, and seems to be generally worse than what other players are seeing via other ISPs. Fortunately there are oceanic servers for PUBG which have good connectivity, but due to low player numbers these generally only run in late evening. Many games are still played on NA servers by NZ / Aus players. Potentially I could look at using a gaming VPN to mitigate the problem, though previously I was able to get similar latency via normal 2degrees, as compared to using gaming VPN.

Unfortunately I don't have any traceroutes from before the latency increase for comparison, but I can show the results of the new higher latency route.

Tracing route to ec2-35-171-21-64.compute-1.amazonaws.com [35.171.21.64]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms fritz.box [192.168.178.1]
2 3 ms 3 ms 3 ms 104.7.69.111.static.snap.net.nz [111.69.7.104]
3 * * * Request timed out.
4 138 ms 138 ms 138 ms TWODEGREES-svc063403-ic344202.c.telia.net [62.115.175.227]
5 139 ms 141 ms 138 ms las-b24-link.telia.net [62.115.175.226]
6 168 ms 169 ms 170 ms dls-b22-link.telia.net [62.115.118.246]
7 170 ms 171 ms 177 ms dls-b21-link.telia.net [62.115.137.106]
8 170 ms 170 ms 171 ms a100-ic-300087-kbn-b3.c.telia.net [62.115.9.126]
9 * * * Request timed out.
10 * * * Request timed out.
11 * * * Request timed out.
12 * * * Request timed out.
13 236 ms 234 ms 235 ms 72.21.222.249
14 * * * Request timed out.
15 * * * Request timed out.

Trace complete.

Tracing route to ec2-35-175-212-46.compute-1.amazonaws.com [35.175.212.46]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms fritz.box [192.168.178.1]
2 2 ms 3 ms 3 ms 104.7.69.111.static.snap.net.nz [111.69.7.104]
3 * * * Request timed out.
4 137 ms 143 ms 141 ms xe-0-0-21-0.a00.lsanca07.us.ce.gin.ntt.net [128.241.4.146]
5 139 ms 139 ms 140 ms xe-0-0-21-0.sayonara-chwhite.a00.lsanca07.us.bb.gin.ntt.net [128.241.4.145]
6 171 ms 171 ms 172 ms ae-11.sayonara-chwhite.r00.lsanca07.us.bb.gin.ntt.net [129.250.2.105]
7 138 ms 139 ms 138 ms ae-2.sayonara-chwhite.r22.lsanca07.us.bb.gin.ntt.net [129.250.3.16]
8 176 ms 175 ms 176 ms ae-7.sayonara-chwhite.r22.dllstx09.us.bb.gin.ntt.net [129.250.7.68]
9 176 ms 178 ms 175 ms ae-2.sayonara-chwhite.r10.dllstx09.us.bb.gin.ntt.net [129.250.4.82]
10 177 ms 175 ms 175 ms ae-2.amazon.dllstx09.us.bb.gin.ntt.net [129.250.201.22]
11 232 ms 235 ms 235 ms 54.239.105.121
12 * * * Request timed out.
13 * * * Request timed out.
14 * * * Request timed out.
15 240 ms 238 ms 239 ms 178.236.3.31
16 * * * Request timed out.
17 * * * Request timed out.

Tracing route to ec2-35-175-212-46.compute-1.amazonaws.com [35.175.212.46]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms fritz.box [192.168.178.1]
2 2 ms 3 ms 3 ms 104.7.69.111.static.snap.net.nz [111.69.7.104]
3 * * * Request timed out.
4 141 ms 139 ms 138 ms xe-0-0-21-0.a00.lsanca07.us.ce.gin.ntt.net [128.241.4.146]
5 138 ms 140 ms 139 ms xe-0-0-21-0.sayonara-chwhite.a00.lsanca07.us.bb.gin.ntt.net [128.241.4.145]
6 171 ms 175 ms 170 ms ae-11.sayonara-chwhite.r00.lsanca07.us.bb.gin.ntt.net [129.250.2.105]
7 140 ms 139 ms 139 ms ae-2.sayonara-chwhite.r22.lsanca07.us.bb.gin.ntt.net [129.250.3.16]
8 181 ms 177 ms 179 ms ae-7.sayonara-chwhite.r22.dllstx09.us.bb.gin.ntt.net [129.250.7.68]
9 177 ms 179 ms 179 ms ae-2.sayonara-chwhite.r10.dllstx09.us.bb.gin.ntt.net [129.250.4.82]
10 178 ms 175 ms 174 ms ae-2.amazon.dllstx09.us.bb.gin.ntt.net [129.250.201.22]
11 235 ms 235 ms 238 ms 54.239.105.121
12 * * * Request timed out.
13 * * * Request timed out.
14 * * * Request timed out.
15 245 ms 240 ms 242 ms 178.236.3.31
16 * * * Request timed out.
17 * * * Request timed out.

Trace complete.

1 | 2

meow
13243 posts

Uber Geek

Moderator

ID Verified

Trusted

Lifetime subscriber

#2249655 1-Jun-2019 17:55

I edited your post as it had quite bad formatting errors. As these servers have ping disabled it is hard to gauge on the actual latency - ICMP (Ping) traffic is not prioritised.

As many threads on here state - latency to any given location can change at any given point. You've got to understand that using game servers located overseas won't give you the best experience. There are many of these threads on here in all ISP's subforums and almost always they fix themselves after a while. This is just how the internet works. You can have either fast speeds or low latency but it is a hard juggling act to get both.

Michael Murphy | https://murfy.nz
Referral Links: Quic Broadband (use R122101E7CV7Q for free setup)

Are you happy with what you get from Geekzone? Please consider supporting us by subscribing.
Opinions are my own and not the views of my employer.

pooky

43 posts

Geek

#2249778 1-Jun-2019 22:55

Thanks for tidying that up. Yep, completely understand the nature of the internet means that pings ultimately vary between providers etc.

By posting here I'm hoping that a 2degrees rep will see it and look into it further. It's also a chance for anyone else who's noticed similar to comment. I have no expectation that they will fix it (or even can), but there is a possibility that they might be able to. I have previously seen instances where providers have been able to make tweaks to improve routing based on customer feedback. It is worth trying because as above, the servers are Amazon AWS it's not really a niche gaming issue, AWS is a highly relevant destination generally for any ISP customer.

The final hops don't reply to ping, but you can see from the hops near the end of the trace the travel time is around 240ms. Under good conditions it's achievable in 180-200ms. Without knowledge of 2degrees network and their upstream providers I can't speculate on why this has changed recently, my suspicion would be either a change on the upstream providers end. Or some AWS change that has impacted the upstream provider. If it was a general AWS change I would expect it to have affected other gamers using other ISPs, which does not appear to be the case.

michaelmurfy

meow
13243 posts

Uber Geek

Moderator

ID Verified

Trusted

Lifetime subscriber

#2249785 2-Jun-2019 01:52

What I am saying is routers de-prioritize ping and the in-game "ping" is not at all accurate. Routers pass traffic, not respond to probes sent to them thus the high ping on your traceroute. There is a very good likelihood that the problem you're describing actually doesn't exist. If you've got a stable connection then you should be able to play the game fine but you really can't have any expectations going to the other side of the US, it is best effort and providers at this distance focus on speed instead. Many times I have stated that the in-game ping can have a placebo effect and this does hold true to many people as ~20ms isn't going to make you a better player.

As per my statement above also regarding speed vs latency I'd rather have faster speed to AWS in the US with +20ms latency instead of a slower speed with less latency. I have backups going there, there are many sites hosted from there and customers expect the best speed possible. You could try a gaming VPN but they're all essentially snakeoil.

2degrees have added much more capacity to the US as of late however I believe they have some high performance links (bandwidth++) and some more expensive, slower lower latency links. From your traceroutes you're actually getting decent response times over to the US then problems arise outside of their network.

According to https://pubgserverping.com/ I am getting 190ms (average) latency to North America on my connection. I don't play pubg, my online gaming consists of Payday 2 mostly and latency on that can vary quite a bit but not once have I had issues playing due to up-to a 50ms variance as my connection is stable.

@NickMack can likely investigate further to see if anything can be changed here to improve things.

Obraik

2123 posts

Uber Geek

#2250138 2-Jun-2019 23:16

As a fellow PUBG player, I too have noticed the same. I used to get pings of around 200-215 on a NA server (which are all located on the east coast) but in the last few weeks it's been around 240ish. When the ping time first increased there was a period where packet loss was quite rampant too but that seems to have calmed down. If I were to use a VPN and VPN to a server located on the east coast my ping would return to normal.

The bigger issue is that our best ping is considered to be the NA servers rather than SEA (Singapore). Ideally we should be playing on the SEA servers where all the Aussies play. Without a VPN, the latency to AWS in Singapore is 300ms+. If I use a VPN and connect the VPN to a Sydney locaton I get a ping of around 120-130ms. A Trace route to a known AWS Singapore IP of 13.228.0.251 (found here http://ec2-reachability.amazonaws.com/) shows that traffic is going to the US before going to Singapore. Why isn't it going via Aus which is a much shorter route?

Tracing route to ec2-13-228-0-251.ap-southeast-1.compute.amazonaws.com [13.228.0.251]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms [192.168.0.202]
2 16 ms 16 ms 16 ms 37.67.69.111.dynamic.snap.net.nz [111.69.67.37]
3 * * * Request timed out.
4 171 ms 181 ms 183 ms TWODEGREES-svc063403-ic344202.c.telia.net [62.115.175.227]
5 172 ms 176 ms 171 ms las-b24-link.telia.net [62.115.175.226]
6 171 ms 171 ms 171 ms tata-ic-333371-las-b24.c.telia.net [80.239.128.215]
7 342 ms 344 ms 342 ms if-ae-2-2.tcore2.lvw-los-angeles.as6453.net [66.110.59.2]
8 342 ms 342 ms 342 ms if-ae-7-2.tcore2.svw-singapore.as6453.net [180.87.15.25]
9 360 ms 332 ms 333 ms 180.87.15.206
10 * * * Request timed out.
11 * * * Request timed out.
12 * * * Request timed out.
13 318 ms 320 ms 318 ms 52.93.11.93
14 * * * Request timed out.
15 * * * Request timed out.
16 327 ms 325 ms 325 ms 203.83.223.15
17 * * * Request timed out.
18 * * * Request timed out.
19 * * * Request timed out.
20 * * * Request timed out.
21 * * * Request timed out.
22 326 ms 326 ms 326 ms ec2-13-228-0-251.ap-southeast-1.compute.amazonaws.com [13.228.0.251]

Trace complete.

Looking to buy a Tesla? Use my referral link and we both get credits

surfisup1000

5288 posts

Uber Geek

#2250206 3-Jun-2019 09:58

michaelmurfy:

What I am saying is routers de-prioritize ping and the in-game "ping" is not at all accurate. Routers pass traffic, not respond to probes sent to them thus the high ping on your traceroute. There is a very good likelihood that the problem you're describing actually doesn't exist.

But, if you have been consistently getting 180ms pings , then there is a sudden increase to 240ms that can be traced to a certain day and has not recovered for at least a week or two....

You are saying that possibly there has been no actual change? If what you say was true (just random inaccuracies), wouldn't there be more variation in pings taken at differnent times and different days. ie, sometimes it would be 180, sometimes higher.

Talkiet

4792 posts

Uber Geek

Trusted

#2250261 3-Jun-2019 11:46

The internet is built so that if one path is unavailable, other paths are available for traffic. Each direction (You to destination and destination to you) can have different paths. Your traceroute only shows the path from you to destination so you may not be able to observe a change in path.

These path changes can occur because of failures, temporary deliberate outages, economic reasons, etc. They are normal and are one of the reasons that no ISP can guarantee performance to off-net destinations.

The change may have happened between 2 entities both in the US. Your ISP may not have a relationship between themselves and either of the affected parties over there.

Sometimes an ISP can find a better path, but more often than not, "That's just the way it is". Don't bother changing ISPs because of it because it will affect all ISPs.

Cheers - N

Please note all comments are from my own brain and don't necessarily represent the position or opinions of my employer, previous employers, colleagues, friends or pets.

Obraik

2123 posts

Uber Geek

#2250291 3-Jun-2019 13:03

Talkiet:

The internet is built so that if one path is unavailable, other paths are available for traffic. Each direction (You to destination and destination to you) can have different paths. Your traceroute only shows the path from you to destination so you may not be able to observe a change in path.

These path changes can occur because of failures, temporary deliberate outages, economic reasons, etc. They are normal and are one of the reasons that no ISP can guarantee performance to off-net destinations.

The change may have happened between 2 entities both in the US. Your ISP may not have a relationship between themselves and either of the affected parties over there.

Sometimes an ISP can find a better path, but more often than not, "That's just the way it is". Don't bother changing ISPs because of it because it will affect all ISPs.

Cheers - N

Yeah, you're not wrong...but the question is more "did something change at 2degrees end?"

Looking to buy a Tesla? Use my referral link and we both get credits

Equinox IT

Cloud spending continues to surge globally, but most organisations haven’t made the changes necessary to maximise the value and cost-efficiency benefits of their cloud investments. Download the whitepaper From Overspend to Advantage now.

yitz

2075 posts

Uber Geek

#2250304 3-Jun-2019 13:43

Vodafone's transit has good peering with Amazon at many of their locations. For example to the above SEA (Singapore) IP I get 135ms (from Chch).

I recall 2degrees have a transit arrangement with Vocus in Sydney for those destinations so unsure why those paths have not been chosen.

NickMack

962 posts

Ultimate Geek

Trusted

Lifetime subscriber

#2251124 4-Jun-2019 13:54

Hi All,

We have reached out to Amazon to request they investigate this. Once I have an update I'll share.

Nick.

https://nick.mackechnie.co.nz | NZ ISP latency monitoring - https://smokeping.thenet.gen.nz

Obraik

2123 posts

Uber Geek

#2251138 4-Jun-2019 14:22

NickMack:
Hi All,

We have reached out to Amazon to request they investigate this. Once I have an update I'll share.

Nick.

Thanks!

Does that include the issue of Singapore data being routed through the US?

Looking to buy a Tesla? Use my referral link and we both get credits

NickMack

962 posts

Ultimate Geek

Trusted

Lifetime subscriber

#2251140 4-Jun-2019 14:25

Obraik:
NickMack:

Hi All,

We have reached out to Amazon to request they investigate this. Once I have an update I'll share.

Nick.

Thanks!

Does that include the issue of Singapore data being routed through the US?

Hiya - We have two paths out of NZ, one via Australia the other via the US. So depending on where things are advertised, the best path is chosen. We don't have a direct path to Singapore.

Nick.

https://nick.mackechnie.co.nz | NZ ISP latency monitoring - https://smokeping.thenet.gen.nz

Obraik

2123 posts

Uber Geek

#2251257 4-Jun-2019 14:54

NickMack:
Obraik:
NickMack:

Hi All,

We have reached out to Amazon to request they investigate this. Once I have an update I'll share.

Nick.

Thanks!

Does that include the issue of Singapore data being routed through the US?

Hiya - We have two paths out of NZ, one via Australia the other via the US. So depending on where things are advertised, the best path is chosen. We don't have a direct path to Singapor to e.

Nick.

Understood, however it seems to always favour going via the US when going to Singapore. As I mentioned in a post further up, if I use a VPN to bypass 2degrees routing I can halve my latency to Singapore by forcing the routing via Australia.

Looking to buy a Tesla? Use my referral link and we both get credits

NickMack

962 posts

Ultimate Geek

Trusted

Lifetime subscriber

#2252899 6-Jun-2019 12:03

Hi All,

Just an update, conversations are still continuing with our upstream providers - I'll share more when I know more.

Nick.

https://nick.mackechnie.co.nz | NZ ISP latency monitoring - https://smokeping.thenet.gen.nz

NickMack

962 posts

Ultimate Geek

Trusted

Lifetime subscriber

#2253706 7-Jun-2019 12:13

Hi All,

We have had a bunch of conversations and checks with both AWS and Telia (Upstream provider) to investigate your query, Telia have confirmed that they did not find any abnormalities and we agree. ICMP traffic is de-prioritised and while it's a measure, it's not entirely useful. See - A Practical Guide to (Correctly) - Troubleshooting with Traceroute & ICMP (ping)

A trace from Telia LAX to 2degrees CHC = +-140ms

A trace from Telia LAX to 35.171.21.64 AWS east – last hop reply = +-65ms

Total of +- 205ms

A trace from 2degrees CHC to Telia LAX = +- 141ms

A trace from 2degrees LAX to 35.171.21.64 AWS east – last hop reply = +-58ms

Total of +- 199ms

Unfortunately 35.171.21.64 does not respond to ICMP.

The last reply in a trace from 2degrees PMR (Palmy North) to 35.171.21.64 is consistently around 238ms even when the hops in between changes.

We found a couple of Australian websites that allow traceroute and they produce similar results.

Below are some of the traces:

Telia LAX to 2degrees CHC

upy1300@las-b24> traceroute monitor 111.69.0.1 source 62.115.175.226

My traceroute [v0.69]

las-b24 (0.0.0.0)(tos=0x0 psize=64 bitpattern=0x00) Thu Jun 6 03:07:52 2019

Keys: Help Display mode Restart statistics Order of fields quit

Packets Pings

Host Loss% Snt Last Avg Best Wrst StDev

TWODEGREES-svc063403-ic344202.c.telia.net 0.0% 4 0.9 0.9 0.8 0.9 0.0
208.55.69.111.static.snap.net.nz 0.0% 4 140.7 141.2 140.7 142.1 0.6

Telia LAX to 35.171.21.64

upy1300@las-b24> traceroute monitor 35.171.21.64 source 62.115.175.226

My traceroute [v0.69]

las-b24 (0.0.0.0)(tos=0x0 psize=64 bitpattern=0x00) Thu Jun 6 03:07:32 2019

Keys: Help Display mode Restart statistics Order of fields quit

Packets Pings

Host Loss% Snt Last Avg Best Wrst StDev

dls-b22-link.telia.net 0.0% 25 31.7 32.0 31.5 35.1 0.9
dls-b21-link.telia.net 0.0% 25 31.4 33.1 31.1 53.8 4.8
a100-ic-300087-kbn-b3.c.telia.net 0.0% 25 31.2 32.2 31.2 36.8 1.4
176.32.125.196 0.0% 25 59.8 60.7 58.9 70.1 2.7
176.32.125.207 0.0% 25 59.5 60.3 58.4 83.2 4.9
???
54.239.46.8 0.0% 25 59.6 64.9 59.6 78.0 6.0
178.236.3.31 0.0% 25 65.3 64.1 60.2 85.0 4.9
???

2degrees CHC to Telia LAX

traceroute to 62.115.175.226 (62.115.175.226) from 111.69.16.185, 30 hops max, 52 byte packets

1 62.115.175.227 142.096 ms 140.780 ms 140.680 ms

2 62.115.175.226 142.614 ms 141.159 ms 141.463 ms

2degrees LAX to 35.171.21.64

traceroute to 35.171.21.64 (35.171.21.64) from 62.115.175.227, 30 hops max, 52 byte packets

1 62.115.175.226 1.682 ms 1.757 ms 2.259 ms

2 62.115.136.46 2.255 ms 62.115.118.246 32.113 ms 62.115.136.46 1.080 ms

MPLS Label=24727 CoS=0 TTL=1 S=1

3 62.115.123.136 34.324 ms 62.115.137.106 30.809 ms 31.046 ms

4 213.248.85.222 31.136 ms 62.115.36.38 30.566 ms 213.248.85.222 31.198 ms

5 176.32.125.198 58.451 ms 176.32.125.176 70.149 ms 176.32.125.162 58.194 ms

MPLS Label=416384 CoS=0 TTL=1 S=0

MPLS Label=3720 CoS=0 TTL=1 S=1

6 176.32.125.229 57.097 ms 176.32.125.233 60.158 ms 176.32.125.219 57.709 ms

MPLS Label=5254 CoS=7 TTL=1 S=1

7 * * *

8 54.239.46.0 59.630 ms 52.93.129.239 57.365 ms 54.239.43.206 65.297 ms

MPLS Label=658317 CoS=0 TTL=1 S=1

9 54.240.229.217 58.945 ms 58.990 ms 178.236.3.31 60.626 ms

MPLS Label=333245 CoS=0 TTL=1 S=1

10 * * *

11 * * *

12 * * *

13 * * *

14 * * *

15 * * *

16 * * *

17 * * *

18 * * *

19 * * *

20 * * *

21 52.93.28.214 57.409 ms * 52.93.28.210 70.025 ms

Aus trace from http://www.hafey.org/cgi-bin/bgplg?cmd=traceroute&req=35.171.21.64

1 203.98.68.97 (203.98.68.97) [AS10145] 2.682 ms (255) 0.347 ms (255) 0.312 ms (255) (0% loss)

2 203.98.68.253 (203.98.68.253) [AS10145] 0.482 ms (63) 0.404 ms (63) 0.407 ms (63) (0% loss)

3 73.194.233.220.static.exetel.com.au (220.233.194.73) [AS10143] 0.984 ms (253) 0.988 ms (253) 0.964 ms (253) (0% loss)

4 * * * (100% loss)

5 116.255.21.108 (116.255.21.108) [AS38195, AS38195] 0.958 ms (251) 0.916 ms (251) 0.992 ms (251) (0% loss)

6 FortyGigE0-0-1-2.120.bdr01-ipt-624sgran-lax.us.superloop.com (103.200.13.168) [AS38195, AS38195] 151.109 ms (250) 150.834 ms (250) 151.165 ms (250) (0% loss)

7 ae7.er3.lax112.us.zip.zayo.com (64.124.204.201) [AS6461] 157.786 ms (245) 157.563 ms (245) 157.564 ms (245) (0% loss)

8 ae11.cr1.lax112.us.zip.zayo.com (64.125.30.72) [AS6461] 197.256 ms (241) 225.357 ms (241) 190.611 ms (241) (0% loss)

9 ae12.cs2.dfw2.us.zip.zayo.com (64.125.26.182) [AS6461] 191.012 ms (242) 190.712 ms (242) 190.666 ms (242) (0% loss)

10 ae24.er2.dfw2.us.zip.zayo.com (64.125.27.105) [AS6461] 190.552 ms (244) 189.964 ms (244) 189.285 ms (244) (0% loss)

11 * * * (100% loss)

12 176.32.125.160 (176.32.125.160) 230.027 ms (221) 176.32.125.184 (176.32.125.184) 230.2 ms (220) 176.32.125.136 (176.32.125.136) 230.2 ms (220) (0% loss)

13 176.32.125.245 (176.32.125.245) 228.235 ms (221) 176.32.125.169 (176.32.125.169) 227.22 ms (221) 176.32.125.157 (176.32.125.157) 227.088 ms (221) (0% loss)

14 * *

To summarise - You are taking the best possible path we have to the destination.

Nick.

https://nick.mackechnie.co.nz | NZ ISP latency monitoring - https://smokeping.thenet.gen.nz

yitz

2075 posts

Uber Geek

#2253731 7-Jun-2019 13:03

Got a traceroute coming back from Amazon in Dallas? Given the jump in round trip times after it reaches Amazon in Dallas that is where the indirect path is.

Ping time on Vodafone seems to have gone up to just under 220 ms from Chch since I last posted from what looks like a change in transit providers :/ No jump after it reaches Amazon though.

1 | 2