Geekzone: technology news, blogs, forums
Guest
Welcome Guest.
You haven't logged in yet. If you don't have an account you can register now.
View this topic in a long page with up to 500 replies per page Create new topic
1 | 2 | 3 
351 posts

Ultimate Geek
+1 received by user: 71


  Reply # 1980218 20-Mar-2018 01:35
Send private message quote this post

vulcannz:

 

PMTU needed to work on IPv4 as fragmentation breaks many protocols (especially proprietary encrypted stuff). Packets only get fragmented if they get a fragmentation required notification back, PMTU relied on that notification. So if PMTU failed then it's likely packets would not get fragmented and just flat out fail (aka a black hole router).

 

 

That is not my understanding at all about how fragmentation works.  When an overlong packet hits a router it is too big for, with IPv4 the router just fragments the packet and sends both new packets on.  It may or may not send notification back to the source - there is no requirement to do that, but I would hope good routers would do it.  A router receiving already fragmented packets does nothing extra - it will not try to join them.  When the fragmented packets get to their destination, they are buffered until both (or all) the fragments are available and then they are put back together again.

 

If the source of overlong packets gets a notification about it, it can choose to reduce the MTU it is using, but again, in IPv4, that is not required, as the packets will continue to be fragmented at any router where that needs to happen.

 

Fragmentation is a big burden on a router.  If a packet can just be passed through intact, often that is done with acceleration hardware and the router CPU never sees the packet.  But when it needs fragmentation, the hardware normally has to hand off the packet to the CPU, which will have to build two packets from that one packet and send them on.  That costs a fair bit of CPU effort, and also needs twice as much buffer space plus a bit more for the second set of headers.

 

A black hole router is either so heavily overloaded that it must drop lots of packets, or is broken and needs to be fixed.  When a packet needing fragmentation hits a router that is heavily overloaded, it may well be a good candidate for that router to drop, as the CPU burden of doing the fragmentation will likely increase the overloading, and the extra buffer space needed almost certainly will increase the overloading.  And the router may well be so overloaded that it will also drop any attempt to send back notification that it dropped the packet.  This sort of overloading can happen because of a reduced MTU causing most larger IPv4 packets to need fragmentation, and is one of the major reasons IPv6 does not allow fragmentation and requires that PMTU works.  PMTU was added to IPv4 to help prevent overloading of routers, but as it is not required to work on all IPv4 devices, it often does not succeed due to one or more devices along the path not supporting it.  When it does work, it is excellent.

 

So, fragmentation does not break any IPv4 protocols.  Enough packet loss (caused by a black hole router or any other reason) can break all IPv4 protocols, but some protocols take much more packet loss before they break.  Encrypted protocols are particularly sensitive to packet loss unless they have a mechanism that allows them to resend just an identical copy of lost packets.  If losing a packet requires a complete resend of the entire message in order for the encryption to work, protocols like that will break easily with minimal packet loss.


54 posts

Master Geek
+1 received by user: 9

Lifetime subscriber

  Reply # 2004208 28-Apr-2018 18:20
Send private message quote this post

michaelmurfy:

 

I, Too am running IPv6 at a MTU of 1500 and 1508 on the physical port.

 

 

Following the suggestions in this thread I've changed to using the IPv6 support in the USG (5.7.23) and noticed that if the MTUs are changed to these values rather than the defaults then the IPv6 tests here (http://test-ipv6.com/index.html.en_US) run significantly faster.

 

 

 

 


268 posts

Ultimate Geek
+1 received by user: 54


  Reply # 2004856 30-Apr-2018 09:22
Send private message quote this post

fe31nz:

 

vulcannz:

 

PMTU needed to work on IPv4 as fragmentation breaks many protocols (especially proprietary encrypted stuff). Packets only get fragmented if they get a fragmentation required notification back, PMTU relied on that notification. So if PMTU failed then it's likely packets would not get fragmented and just flat out fail (aka a black hole router).

 

 

That is not my understanding at all about how fragmentation works.  When an overlong packet hits a router it is too big for, with IPv4 the router just fragments the packet and sends both new packets on.  It may or may not send notification back to the source - there is no requirement to do that, but I would hope good routers would do it.  A router receiving already fragmented packets does nothing extra - it will not try to join them.  When the fragmented packets get to their destination, they are buffered until both (or all) the fragments are available and then they are put back together again.

 

 

It depends, on edge equipment the common configuration is for the edge router to fragment the packet. As step away from the edge the telco routers will drop packets and should send back an ICMP message fragmentation required.

 

Fragmenting can break a lot of applications so telcos tend to stay away from it. Your statement about fragmentation breaking no IPv4 protocols is quite incorrect, off the bat it will break IPSEC (and of course TLS, and SSL). Anything with encryption does not like the packet being fiddled with.

 

I see a lot of performance and applications woes on edge equipment where fragmentation is required. Avoiding fragmentation is a good thing.


351 posts

Ultimate Geek
+1 received by user: 71


  Reply # 2005564 1-May-2018 00:07
Send private message quote this post

I have no practial experience using IPSEC.  But everything I have read about it says that it should be able to be fragmented without problems.  Fragmentation means that the original packet gets put back together at the destination - so the encryption is unaffected as the full encrypted packet is available to be decrypted.  But if the connection is affected by problems with dropped packets triggered by MTU problems, then IPSEC will have performance issues with fragmentation.  And some IPSEC implementations seem to want to set the DF = Do not Fragment bit, which means they will not work if there are MTU issues.  That is likely done because of the performance issues with encryption when there is fragmentation happening - fragmentation is often associated with overloaded routers and packet loss, because the router doing fragmentation causes it to become overloaded.

 

TLS and SSL certainly are unaffected by fragmentation - as long as all the fragments of the packet arrive at the other end, there is no problem.  If they were affected by fragmentation, there would be huge problems with TLS and SSL connections around the Internet that just do not happen.  So I wonder where you are getting the idea that fragmentation kills encryption.  Fragmentation is done on fully formed packets with all the headers and checksums complete.  The fragments get put back together again at the destination in such a way that the original packet with all its headers and checksums is identical to the packet that got fragmented.  Hence any encryption inside the packet is completely unaffected - as long as all the fragments arrive.


268 posts

Ultimate Geek
+1 received by user: 54


  Reply # 2006277 1-May-2018 21:11
Send private message quote this post

I have lots of experience with IPSEC, TLS, SSL and other proprietary encrypted protocols and can assure you fragmentation is a huge issue.

 

You have two major issues, for IPSEC to handle fragmentation properly PMTU Discovery needs to work - and everyone knows PMTUD is fundamentally broken. 2nd IPSEC is intentionally very sensitive to packet changes, it just doesn't like fragmentation - it's also not "pure" - it is normally encapsulated within a UDP packet for NAT traversal and other network oddities (for many years home/SMB routers did not support AH/ESP protocol routing). It is very common to see VPNs breaking simply because an MTU is set too high. Signs of MTU issues are poor performance, VPN dropouts, and session disconnects within VPNs (very common to show up when RDP is used).

 

For TLS/SSL you have the same PMTUD challenge. Signs of MTU issues here show up as half loaded secure web sites (especially image heavy), slow internet performance, and poor SSL VPN experiences.

 

I've also seen lots of issues with proprietary encrypted protocols like Skype and Aspera. In my last job I was doing beta testing for a few products, one in particular was a cloud service using uFTP. The devs had set an MTU of 1500 that was causing havoc. Initially this was an Asia Pac issue - but it was also seen in Europe and some US states. They'd never come across it before. I guess in NZ we experience more because we typically have more hops to travel and are more likely to hit crappy or overwhelmed routers.

 

As I've said before, my field is firewalls and remote access (IPSEC/SSL VPN etc), and it's been my focus since 2002. If I had a dollar for every time I had sort out a VPN/MTU isssue I'd have a few grand :)


351 posts

Ultimate Geek
+1 received by user: 71


  Reply # 2006334 2-May-2018 00:29
Send private message quote this post

Yes, there are real problems getting encrypted connections from one side of the Internet to the other.  But the problem is not fragmentation as such, it is lost packets.  If all the fragments of an encrypted packet reach the destination, nothing gets broken.  Fragmentation does NOT change the contents of the reassembled packet from what was in the packet that was originally sent.  Encrypted connections are inherently much more sensitive to packet loss, that is the problem you are seeing.  When the MTU changes to a smaller size, it often cause packet loss by overloading the router as it tries to handle all the fragmenation.  So the loss of one fragment of one packet then causes the encrypted connection to have to resend that packet, or if it is a bad encrypted protocol, it might have to back up and resend lots of packets to re-establish the encryption.  If the connection is SSL, and the underlying TCP connection has negotiated the option of just resending one lost packet, then it is relatively unaffected by packet loss, when the loss is not too frequent.  But that is the problem with overloaded routers - loss is not infrequent if you keep sending long packets that need fragmentation, as those are the packets most likely to be dropped by an overloaded router.  If you send unencrypted connections through an MTU change and the router is overloaded, they too will suffer badly if they keep on sending long packets.  But fairly large packet loss can be coped with as an unecrypted connection does not reach a point of needing to re-establish the encryption.  You can get a working unecrypted connection with over 50% packet loss.  It will be very slow, but the connection will not necessarily drop.


268 posts

Ultimate Geek
+1 received by user: 54


  Reply # 2006367 2-May-2018 07:53
Send private message quote this post

Well all I can say is MTU can have a work/no work impact on encrypted protocols. In theory you are correct, but in practicality some things just don't work as they should (like PMTUD).


67 posts

Master Geek
+1 received by user: 5


  Reply # 2008941 6-May-2018 22:47
Send private message quote this post

Well, here I am a bit later. It works (although I'm now on Vodafone HFC, so I don't need the additional config). It's good to be in the IPv6 lane!


141 posts

Master Geek
+1 received by user: 40


  Reply # 2009269 7-May-2018 14:49
Send private message quote this post

fe31nz:

 

Yes, there are real problems getting encrypted connections from one side of the Internet to the other.  But the problem is not fragmentation as such, it is lost packets.  If all the fragments of an encrypted packet reach the destination, nothing gets broken.  Fragmentation does NOT change the contents of the reassembled packet from what was in the packet that was originally sent.  Encrypted connections are inherently much more sensitive to packet loss, that is the problem you are seeing.  When the MTU changes to a smaller size, it often cause packet loss by overloading the router as it tries to handle all the fragmenation.  So the loss of one fragment of one packet then causes the encrypted connection to have to resend that packet, or if it is a bad encrypted protocol, it might have to back up and resend lots of packets to re-establish the encryption.  If the connection is SSL, and the underlying TCP connection has negotiated the option of just resending one lost packet, then it is relatively unaffected by packet loss, when the loss is not too frequent.  But that is the problem with overloaded routers - loss is not infrequent if you keep sending long packets that need fragmentation, as those are the packets most likely to be dropped by an overloaded router.  If you send unencrypted connections through an MTU change and the router is overloaded, they too will suffer badly if they keep on sending long packets.  But fairly large packet loss can be coped with as an unecrypted connection does not reach a point of needing to re-establish the encryption.  You can get a working unecrypted connection with over 50% packet loss.  It will be very slow, but the connection will not necessarily drop.

 

 

 

 

you are right that the problem is lost packets, but the root cause IS due to fragmentation (rather the lack of).

 

instead of just sending oversized packets and expecting them to be fragmented, some endpoints try and go through the PMTUD process by sending large packets with the DF bit set. when these packets die the death a few hops upstream and the sender doesn't get an ICMP reply for whatever reason (firewall or w/e), the sender *should* back off and either send smaller packets or stop setting DF, but this doesn't always happen. so you get these weird half working 'black hole' connections (although to be fair this was more of a problem in the past).

 

there is nothing in this process that's specific to VPNs etc, it's just more likely to happen with them as the packets are larger due to overheads.


1 | 2 | 3 
View this topic in a long page with up to 500 replies per page Create new topic

Twitter »

Follow us to receive Twitter updates when new discussions are posted in our forums:



Follow us to receive Twitter updates when news items and blogs are posted in our frontpage:



Follow us to receive Twitter updates when tech item prices are listed in our price comparison site:





News »

Microsoft Dynamics 365 Business Central launches
Posted 10-Jul-2018 10:40


Spark completes first milestone in voice platform upgrade
Posted 10-Jul-2018 09:36


Microsoft ices heated developers
Posted 6-Jul-2018 20:16


PB Technologies charged for its extended warranties and warned for bait advertising
Posted 3-Jul-2018 15:45


Almost 20,000 people claim credits from Spark
Posted 29-Jun-2018 10:40


Cove sells NZ's first insurance policy via chatbot
Posted 25-Jun-2018 10:04


N4L helping TAKA Trust bridge the digital divide for Lower Hutt students
Posted 18-Jun-2018 13:08


Winners Announced for 2018 CIO Awards
Posted 18-Jun-2018 13:03


Logitech Rally sets new standard for USB-connected video conference cameras
Posted 18-Jun-2018 09:27


Russell Stanners steps down as Vodafone NZ CEO
Posted 12-Jun-2018 09:13


Intergen recognised as 2018 Microsoft Country Partner of the Year for New Zealand
Posted 12-Jun-2018 08:00


Finalists Announced For Microsoft NZ Partner Awards
Posted 6-Jun-2018 15:12


Vocus Group and Vodafone announce joint venture to accelerate fibre innovation
Posted 5-Jun-2018 10:52


Kogan.com to launch Kogan Mobile in New Zealand
Posted 4-Jun-2018 14:34


Enable doubles fibre broadband speeds for its most popular wholesale service in Christchurch
Posted 2-Jun-2018 20:07



Geekzone Live »

Try automatic live updates from Geekzone directly in your browser, without refreshing the page, with Geekzone Live now.



Are you subscribed to our RSS feed? You can download the latest headlines and summaries from our stories directly to your computer or smartphone by using a feed reader.

Alternatively, you can receive a daily email with Geekzone updates.