ASUS RT-N66U router and Jumbo frames

Forums › LAN (ethernet/Wifi/routers/Bluetooth) › ASUS RT-N66U router and Jumbo frames

Sideface

9650 posts

Uber Geek
+1 received by user: 15602

Trusted

Lifetime subscriber

#151987 12-Sep-2014 10:56

I have a gigabit LAN in my home office, storing a large amount of data with triple redundancy.
I have been experimenting with Jumbo frames, with disappointing results.

The setup:

Vodafone high-speed cable with Cisco modem
Asus RT-N66U router
D-Link DGS-1008D 8-port switch
Synology DiskStation DS1813 - 8-bay with 32 TB RAID array (Synology Hybrid RAID)
Windows Server 2008 data server with a 24 TB RAID 5 array
Lian Li EX503 with a 24 TB RAID 5 array - connected by USB3 to the Windows data server
Other PCs running Win7 x64

All patch cables are new Cat 6
The whole system is ethernet - no WiFi involved.

I use ViceVersa Pro software to synchronize back-ups.

All of these devices are less than 2 years old, all except the USB3 device support Gigabit ethernet and Jumbo frame, and all firmware is up-to-date.
I tried setting all devices to an MTU of 9000 - but the RT-N66U router only has an "Enable Jumbo Frames" option (MTU not specified)

Results with Jumbo frame are worse than without it - typical ethernet speeds are 30 MBytes/sec (with Jumbo) vs. 40 MBytes/sec (without Jumbo). I get 180 MBytes/sec via USB3.

Is the problem the RT-N66U router? - I can't find out what jumbo frame sizes it supports.

Sideface

Coil

6614 posts

Uber Geek
+1 received by user: 2153
Inactive user

#1127094 12-Sep-2014 11:02

I tried Jumbo frames with my Fritzbox and didnt have a good time. (VDSL)

Inphinity

2780 posts

Uber Geek
+1 received by user: 1184

#1127132 12-Sep-2014 12:01

If the devices are connected directly to the switch, the traffic should not traverse the router.
The RT-AC66U supports 9000 byte jumbo frames, so I imagine the RT-N66U does too.
Try ping -f -l 9000 <dest> to see if 9k jumbo frames can traverse from your device to the destination.

Sideface

9650 posts

Uber Geek
+1 received by user: 15602

Trusted

Lifetime subscriber

#1127213 12-Sep-2014 13:47

Inphinity: If the devices are connected directly to the switch, the traffic should not traverse the router.

Try ping -f -l 9000 <dest> to see if 9k jumbo frames can traverse from your device to the destination.

The good news: I just noticed that the Synology NAS was connected to the router - plugging it in into the switch improved LAN speeds by 5%. Thank you

The bad news: 100% loss on ping -f -l 9000 <dest>. The LAN still runs faster without Jumbo. (at aprx. 40 MBytes/sec)

Sideface

Inphinity

2780 posts

Uber Geek
+1 received by user: 1184

#1127233 12-Sep-2014 14:07

Do you have Jumbo Frames enabled on the Synology, with the JumboMTU set to 9000? I just checked and both our Synology devices have Jumbo Frames disabled by default, and even when you enable it, it defaults to an MTU of 2000. Remember, ALL devices involved in the connection need to support Jumbo Frames for it to work, otherwise the packets will get fragmented and performance degraded, which sounds like is happening to you. This includes the device you're connecting from, any switches and routers in between, and the destination device. The smallest supported MTU in the path will be the one in effect, essentially.

ratsun81

516 posts

Ultimate Geek
+1 received by user: 183

#1127268 12-Sep-2014 14:58

Inphinity: If the devices are connected directly to the switch, the traffic should not traverse the router.
The RT-AC66U supports 9000 byte jumbo frames, so I imagine the RT-N66U does too.
Try ping -f -l 9000 <dest> to see if 9k jumbo frames can traverse from your device to the destination.

Are you running DHCP off the router? If so all traffic will be relying on the router's routing.

If you are running the DHCP from the router a couple of options to try would be static IP's between 2 devices and see if that makes any impact. Or the 2nd option would be to turn the windows server into your DHCP server.

You shouldnt need jumboframes to get speeds over 30MB/s, any gigabit network with standard setup should easily hit 100MB/s unless you have a device that cannot process the requirements fast enough (typically home grade routers)

Quic Broadband

Use R212389ELFLL2 promo code for free setup at checkout.

Sideface

9650 posts

Uber Geek
+1 received by user: 15602

Trusted

Lifetime subscriber

#1127312 12-Sep-2014 16:05

Thanks for your suggestions.
All devices already have fixed IP addresses and are wired to the switch.
All devices have Jumbo frames enabled with an MTU of 9000 (except for the router which has Jumbo Enabled but MTU not specified)
Thus my query about the router in my original post. There is no option to set the MTU on the router. (latest firmware)
40 MBytes/sec seems ridiculously slow when you have 15TB of data to transfer

Sideface

New World

Shop on-line at New World now for your groceries (affiliate link).

Inphinity

2780 posts

Uber Geek
+1 received by user: 1184

#1127321 12-Sep-2014 16:21

ratsun81:
Are you running DHCP off the router? If so all traffic will be relying on the router's routing.

If you are running the DHCP from the router a couple of options to try would be static IP's between 2 devices and see if that makes any impact. Or the 2nd option would be to turn the windows server into your DHCP server.

If it's a single subnet, there is no routing. If it's multiple subnets, static IPs won't help, because traffic will still need to be routed.
Given the router in use, it's a single subnet. You could turn the router off and, provided both devices already have their IPs assigned, traffic will be unaffected. The fact that the router is running the DHCP service is irrelevant.

Sideface: you could change the 9000 value in that ping command to a lower value to try and determine what the largest size packet that can traverse is, e.g. ping -f -l 2000 <dest>

I'd also be curious, if one of the devices involved is a Windows box, what the output of
netsh interface ipv4 show subinterfaces
gives for the MTU for the interface in question

I do agree with ratsun, though, I'd expect far higher throughput than 40MB/sec over a gigabit lan, regardless of packet sizes (I just pulled 103MB/sec with an MTU of 1492 in a quick test now, for example).

Sideface

9650 posts

Uber Geek
+1 received by user: 15602

Trusted

Lifetime subscriber

#1127348 12-Sep-2014 17:18

Inphinity: I'd also be curious, if one of the devices involved is a Windows box, what the output of
netsh interface ipv4 show subinterfaces
gives for the MTU for the interface in question

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.

C:\Users\Administrator>netsh interface ipv4 show subinterfaces

MTU           MediaSenseState       Bytes In Bytes Out      Interface

4294967295      1                      0              2848            Loopback Pseudo-Interface 1
9198                 1                     33374621 4170479327 Local Area Connection

Sideface

9650 posts

Uber Geek
+1 received by user: 15602

Trusted

Lifetime subscriber

#1128236 14-Sep-2014 11:39

UPDATE

The reason why I have been experimenting with Jumbo Frames is because my gigabit LAN appeared to be running slowly when using file synchronization software.

I had typically been getting transfer speeds of 40MBytes/sec with ViceVersa Pro, which I have been using for years.
I have only just realised how slow this (otherwise excellent) file synchronization software is.
On substituting FreeFileSync 6.9 open source software, transfer speeds have risen to 110MBytes/sec without Jumbo frames.
The freeware is almost 3 times faster than the commercial software!
Happy Camper

The next step will be to try again with a new Cisco switch with known Jumbo frame capability - watch this space ...

Sideface

markg1275

24 posts

Geek
+1 received by user: 4

#1128297 14-Sep-2014 13:49

Hi,

I'm not sure its worth chasing the rabbit down the hole with jumbo frames unless you want to use iSCSI mounts for your NAS, I work as a datacentre engineer so am fortunate in that I get to play with lots of top end hardware and its common for us to achieve a throughput of 450Mbps on something like a Dell R510 with a raid card with 512MB of cache to SATA disks. Its all about how the software works and if it can stream data sequentially maximising the performance of SATA disks. If you have software that will do block level file comparison and compress the data stream before sending it to a target you will get a lot more throughput but its going to be harder on the client in regards CPU and RAM use. If your software is just picking up small files and copying them from A to B using SMB the throughput is going to be a lot lower.

I have no experience of the Synology products so maybe someone should comment as to if they use SMB or NFS shares and if you are able to setup iSCSI targets on them as that might be worth a test if you are able to setup a separate share as a test. I know its not always easy to test once you have everything setup on the box. Generally with iSCSI you want to use a separate switch or a direct connection to your storage but in a small network it might not be such a big concern but you will defiantly want to get Jumbo frames working between your server and the NAS if you go down this route.

Also I know you said you have enabled Jumbo frames on all your devices but can you confirm that device manager shows your network adapters as having a 9000byte MTU in their properties?

Regarding SMB throughput it might be worth having a read of this article as it seems to indicate that SMB does not benefit from a 9000 byte packets and they have had more joy with sizes around the 5000 mark - http://forum.synology.com/enu/viewtopic.php?f=14&t=32026

Thanks

Mark

Sideface

9650 posts

Uber Geek
+1 received by user: 15602

Trusted

Lifetime subscriber

#1129913 16-Sep-2014 13:37

UPDATE 2

The updated setup:

Vodafone high-speed cable with Cisco modem
Asus RT-N66U router
Cisco SG 200-08 8-port Gigabit Smart Switch <<< NEW - supports Jumbo
Synology DiskStation DS1813 - 8-bay with 32TB RAID array
Windows Server 2008 data server with a 24TB RAID 5 array
Lian Li EX503 with a 24TB RAID array - connected by USB3 to the Windows data server
Other PCs running Win7 x64
The whole system is gigabit ethernet - all cables are Cat 6.
I now use FreeFileSync software to synchronize back-ups - using SMB shares - much faster than ViceVersa Pro

There are only 3 MTU options common to all the devices on the LAN - 1500 (no Jumbo), 4000, and 9000

With MTU 9000 - performance is worse than with MTU 1500
With MTU 4000 - performance is the same as with MTU 1500

I am getting 110MBytes/sec LAN speeds with or without Jumbo (MTU 400), which is close enough to the theoretical maximum of 125MBytes/sec.

Executive Summary: I have given up on Jumbo.

Sideface