[Snap] Fibre 200/200 speeds with Linux firewall

Forums › 2degrees (including Slingshot, Orcon, Flip, Stuff Fibre, MyRepublic, 2talk and Vocus) › [Snap] Fibre 200/200 speeds with Linux firewall

solorvox

51 posts

Master Geek

#165806 21-Feb-2015 10:55

Hi all,

I need some help figuring out why my speeds are greatly off when I try to use another router/firewall than the fritzbox. I have snap 200/200 fibre installed and been trying to relocate the fritzbox as I only want to use it as a VOIP ATA. To prevent having to double NAT and better secure my LAN I want to put the fritz behind a proper firewall/router.

However, my testing is producing some very strange results.

ONT->Fritz->Desktop + Servers
+Speed 200+/200+Mbits
+1 Layer of NAT
-Fritz source NAT issues (VPN/etc)
-Poor firewall
-Fritz exposed with past security issues

ONT->Fritz->Server (NAT only)->Desktop
-Speed 175/200Mbits
-2 Layers of NAT yuck!
-Fritz source NAT issues (VPN/etc)
-Fritz still exposed
+Better firewall for LAN/servers

ONT->Server (VLAN + PPPoE + NAT)->Desktop/Fritz
-Speed 120/200Mbits
+Fritz more secure on own VLAN
+1 Layer of NAT
+Better firewall for LAN/servers

Server hardware specs for linux firewall:
i5 dual core @ 3.3Ghz
4GB RAM
3xGigabit adapters
SSD SATA2 storage
Debian Linux 3.16 kernel
MTU/MRU 1492 (1500 – 8 for PPPoE)

First off, the very odd thing is that I am getting 200Mbits almost always for uploads but download is all over the place. It seems rule out a bad/poor network adapter as I can get full speeds in some configs. Box CPU usage is less than 5% during testing.

I also can't see how a dual core i5 does worse when talking directly to the ONT using single NAT vs behind the double NAT and the Fritz. Surely VLAN tagging + PPPoE shouldn't have 80Mbits worth of overhead!?!?

I looked at doing bridging on the Fritz but couldn't find much detail on it. Snap also told me they don't support and recommend I not use it but wouldn't/couldn't tell me *why not*. Since I need to use the Fritz as an ATA I assume it needs to to reachable so bridging would be out? :)

Is anyone else running a similar setup and can provide some advice? I have considered using a EdgeRouter but not sure how much good that would be over the more direct setup. I really would like to keep my public IP in bridging mode for my server but keep Fritz for ATA. Can that be done for the edge router?

Cheers

EDIT: formatting

freitasm

BDFL - Memuneh
79287 posts

Uber Geek

Administrator

ID Verified

Trusted

Geekzone

Lifetime subscriber

#1243557 21-Feb-2015 11:08

What makes you think a PC-based firewall will be a "Better firewall for LAN/servers" than the Fritz!Box itself?

solorvox

51 posts

Master Geek

#1243580 21-Feb-2015 11:31

freitasm: What makes you think a PC-based firewall will be a "Better firewall for LAN/servers" than the Fritz!Box itself?

Simply put you have very little control over it with the fritz web interface. I'm sure it's fine for most home users, but when you need to do VLANs, VPNs, detailed port forwarding it just doesn't cut it. I also run DPI on my linux firewalls and dynamic rules (fail2ban/etc). Avoiding it also cuts out another layer of NAT that again causes problems with VPN and some apps that don't play well.

Finally, there is the addition of the security issues they (snap+fritz) had last year. I assume those are enough reasons?

I'm a poweruser, so most off-the-shelf end-user devices won't make me happy. lol

hio77

12999 posts

Uber Geek

ID Verified

Trusted

Lizard Networks

#1243602 21-Feb-2015 12:10

solorvox:
freitasm: What makes you think a PC-based firewall will be a "Better firewall for LAN/servers" than the Fritz!Box itself?

Simply put you have very little control over it with the fritz web interface. I'm sure it's fine for most home users, but when you need to do VLANs, VPNs, detailed port forwarding it just doesn't cut it. I also run DPI on my linux firewalls and dynamic rules (fail2ban/etc). Avoiding it also cuts out another layer of NAT that again causes problems with VPN and some apps that don't play well.

Finally, there is the addition of the security issues they (snap+fritz) had last year. I assume those are enough reasons? I'm a poweruser, so most off-the-shelf end-user devices won't make me happy. lol

pick and choose the right firmware and the issues snap had are patched, the fritz is actually a pretty solid device.

being concerned about the fritz being exposed is silly, you seem to know your way around enough to disable outside facing services on the firtz.

Ild suspect your issue could be a window scaling issue or something in the software configuration.

I have seen snap connections getting 200+ without the fritz given the correct configuration.

#include <std_disclaimer>

Any comments made are personal opinion and do not reflect directly on the position my current or past employers may have.

solorvox

51 posts

Master Geek

#1243611 21-Feb-2015 12:32

hio77:

pick and choose the right firmware and the issues snap had are patched, the fritz is actually a pretty solid device.

being concerned about the fritz being exposed is silly, you seem to know your way around enough to disable outside facing services on the firtz.

Ild suspect your issue could be a window scaling issue or something in the software configuration.

I have seen snap connections getting 200+ without the fritz given the correct configuration.

You realize that I listed several reasons and having it exposed was only the last one? And why is it silly to want to reduce/eliminate a possible vulnerability?

I've played with kernel tcp numbers and tweaked the rwin/wwin/mem values and it didn't make too much of a difference. (5-10Mbits out of 80 missing)

Since the CPU usage is so low, I'm inclined to think it might be a PPPoE config problem. Was hoping someone else had already been down that road. Just for testing, I also reduced to the firewall rules to about 5 rules and it made no difference there either.

hio77

12999 posts

Uber Geek

ID Verified

Trusted

Lizard Networks

#1243612 21-Feb-2015 12:36

solorvox:
You realize that I listed several reasons and having it exposed was only the last one? And why is it silly to want to reduce/eliminate a possible vulnerability?

I've played with kernel tcp numbers and tweaked the rwin/wwin/mem values and it didn't make too much of a difference. (5-10Mbits out of 80 missing)

Since the CPU usage is so low, I'm inclined to think it might be a PPPoE config problem. Was hoping someone else had already been down that road. Just for testing, I also reduced to the firewall rules to about 5 rules and it made no difference there either.

yes, i do realize that. i also picked at that particular reason on purpose.

on windows PPPoE sessions tend to top out at about 300mbit, so it would not surprise me.

The fact that your loosing 25mbit over the double nat indicates to me you could be looking at something in your server configuration itself however.

#include <std_disclaimer>

Any comments made are personal opinion and do not reflect directly on the position my current or past employers may have.

solorvox

51 posts

Master Geek

#1243616 21-Feb-2015 12:41

hio77:
yes, i do realize that. i also picked at that particular reason on purpose.

on windows PPPoE sessions tend to top out at about 300mbit, so it would not surprise me.

The fact that your loosing 25mbit over the double nat indicates to me you could be looking at something in your server configuration itself however.

But only on downloads when uploads are always 200+ Mbits. I would have expected them to both be degraded by the same amount. Very strange.

Ragnor

8221 posts

Uber Geek

Trusted

#1244128 22-Feb-2015 14:22

Interesting, instead of debian linux it might be interesting to test something like pfsense and see if it has the same throughput issues.
https://www.pfsense.org/

Sharesies

Trade NZ and US shares and funds with Sharesies (affiliate link).

hio77

12999 posts

Uber Geek

ID Verified

Trusted

Lizard Networks

#1244143 22-Feb-2015 14:52

Ragnor: Interesting, instead of debian linux it might be interesting to test something like pfsense and see if it has the same throughput issues.
https://www.pfsense.org/

ild second this.

apart from an odd kernel crash i get maybe once a month, mine runs rock solid.

#include <std_disclaimer>

Any comments made are personal opinion and do not reflect directly on the position my current or past employers may have.

SATTV

1648 posts

Uber Geek

ID Verified

#1244146 22-Feb-2015 14:59

I had a similar issue with a linux firewall, you had to change the NIC from auto negotiate to 100Mbps full duplex ( in our case it was a 50/50 connection )

force the NIC into 1000 Mbps full duplex and see how you go.

Cheers
John

I know enough to be dangerous

Aredwood

3885 posts

Uber Geek

#1244503 23-Feb-2015 01:03

Did you get my reply to your last PM that you sent me?

As for the edge router, It could be just what you need. I was getting confused about what exactly you were trying to achieve in your earlier PM's.

So an example setup could be: ONT >> Edgerouter (configured to remove VLAN tags, basic firewall to stealth all outward facing ports, and terminate PPP tunnel) >> your linux box for Firewall and NAT >> your internal network.
Then connect the 3rd port on the Edgerouter to the fritzbox. Configure static routes on Edgerouter to forward traffic from the Snap VOIP and TR069 servers. To the fritzbox.

If you wanted to be really crafty, The Edgerouter can act as a PPPoE server. So you could setup Vlan tagging and a PPPoE server on the 3rd port of the Edgerouter. So you could leave the fritzbox on the default settings. And with the right static routes. Snap should be able to see the fritzbox as normal from their end.

Note that this is all just theoretical. As Although I have both an Edgerouter and a Fritzbox. Im not using Snap VOIP. And I told them during signup to not assign me a number. So Im unable to test this myself. (Unless I ask Snap to set me up on their VOIP system). Also the Edgerouter can easily do 200/200 mbit with both PPP and VLAN. As that is exactly what it does on my connection.

D1NZ

194 posts

Master Geek

#1246023 25-Feb-2015 00:37

Ragnor: Interesting, instead of debian linux it might be interesting to test something like pfsense and see if it has the same throughput issues.
https://www.pfsense.org/

I am running latest pfSense 2.2 don't have any throughput issues.
Here goes:-

My build prolly will handle 1Gbps connection without breaking a sweat.

Publius

276 posts

Ultimate Geek

#1246601 25-Feb-2015 18:06

Try forcing the auto negotiation, although at gigabit it should auto negotiate correctly

You say you've checked MTU

Check your NIC driver is correct and you are running a kernel that's recent.
Check the latest kernel docs for your NIC driver and see if there are any bugs that have been fixed in a older kernel.

Try a different NIC type if possible?

deadlyllama

1263 posts

Uber Geek

Trusted

#1246630 25-Feb-2015 18:55

What are you using as a PPPoE client? Can you show a screenshot of "top" while you are running a speed test?

(edit) the reason I ask is that the old way of doing PPPoE on Linux involved passing all the packets though userspace. It's not fast.

michaelmurfy

meow
13246 posts

Uber Geek

Moderator

ID Verified

Trusted

Lifetime subscriber

#1246640 25-Feb-2015 19:05

I just bought a Linksys WRT1900AC and installed OpenWRT on it - runs without breaking a sweat and has really good WiFi.

Got really really sick of the Fritz!Box.

Michael Murphy | https://murfy.nz
Referral Links: Quic Broadband (use R122101E7CV7Q for free setup)

Are you happy with what you get from Geekzone? Please consider supporting us by subscribing.
Opinions are my own and not the views of my employer.