pfsense 2.7.0 only getting half gigabit speeds

Forums › LAN (ethernet/Wifi/routers/Bluetooth) › pfsense 2.7.0 only getting half gigabit speeds

miztic

21 posts

Geek

#306258 8-Jul-2023 16:57

so I've installed Proxmox and setup a pfsense VM - I've got 1x motherboard gigabit NIC going to LAN, and 1x PCI-e 1x gigabit NIC going to ONT/WAN (no modem/router), and the speeds i'm pulling are 400-440Mbit down, 500Mbit up.

Here's an output of the NIC's:

02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c)
03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06)

I have also ticked and unticked the Disable Hardware checksum offload, Disable hardware TCP segmentation offload, Disable hardware large receive offload & Enable the ALTQ support for hn NICs, no difference in speed.

I had Ubuntu Server installed before (not a VM, installed from USB) and was able to get full speed no problem, so not a CPU issue.

I'm just seeing if anyone else has had the same issue and able to help?

cychronz

66 posts

Master Geek

ID Verified

#3101411 8-Jul-2023 17:25

hi,

Might be worth trying one of the other types of network card options in proxmox:

It might rename the nic in pfsense, but worth a shot. VirtIO is the one I normally use, but I don't run pfsense in proxmox.

miztic

21 posts

Geek

#3101422 8-Jul-2023 17:56

I've tried the E1000 for both, that slows the speed down even more to 100mbit, and the ethtool still shows both cards speed at 1000Mbit with autoneg on.

The virtIO is the best one to use as it shows my LAN as 10Gbase-T.

SirHumphreyAppleby

2849 posts

Uber Geek

#3101428 8-Jul-2023 18:09

Were you previously running pfSense as a VM within Ubuntu server, or are you comparing a VM with a bare metal Ubuntu install?

If you're using PPPoE, note that this is limited to a single core under FreeBSD, so throughput is clock speed dependent. I ran pfSense for years in ESXi, and never had a performance issue with E1000 emulation, so I can't explain why you're seeing such a performance drop (although my ONT-connected NIC was PCI pass-through, same em driver).

miztic

21 posts

Geek

#3101429 8-Jul-2023 18:13

This is the first time I'm running pfsense, before I had ubuntu server pppoeconf setup and just added ppp0 as a default route - that gave us full speed (940Mbps down, 520Mbps up)

I thought i'd try the Promox way of things, (VM's and CT's) seeing as most people are doing that nowadays

I also setup OPNsense in a VM before pftsense, and had the same issue.

cychronz

66 posts

Master Geek

ID Verified

#3101433 8-Jul-2023 18:21

Sorry I am just guessing at this stage, but try set the VMs processsor type to "host" and watch htop while doing a speedtest to see if it is the single core performance causing the issue.

another extreme option is using PCI passthrough for the nics to pfsense

michaelmurfy

meow
13261 posts

Uber Geek

Moderator

ID Verified

Trusted

Lifetime subscriber

#3101443 8-Jul-2023 18:54

Something to note is PPPoE I’m pretty sure on PF-Sense / OPNSense is single threaded still and performance just simply isn’t on par compared to a proper router with full packet acceleration.

Not sure if this is related but have a look at when you do a Speedtest does a single core spike to 100%?

Michael Murphy | https://murfy.nz
Referral Links: Quic Broadband (use R122101E7CV7Q for free setup)

Are you happy with what you get from Geekzone? Please consider supporting us by subscribing.
Opinions are my own and not the views of my employer.

miztic

21 posts

Geek

#3101451 8-Jul-2023 19:19

Just done a speedtest while watching htop via System Activity, i noticed [intr{irq32: virtio_pci3}] went up to 58% both downloading and uploading - which i'm assuming is the WAN NIC virtIO

And yes, using PPPOE for Bigpipe - MTU set to 1492 on WAN

Sharesies

Trade NZ and US shares and funds with Sharesies (affiliate link).

michaelmurfy

meow
13261 posts

Uber Geek

Moderator

ID Verified

Trusted

Lifetime subscriber

#3101563 8-Jul-2023 21:33

BigPipe (Spark) support a full 1500 byte MTU. Set your WAN interface to 1508 and your PPPoE to 1500.

SirHumphreyAppleby

2849 posts

Uber Geek

#3101601 9-Jul-2023 08:43

It doesn't appear that PPPoE is the issue here, but you could also try setting power for each of the conditions to Maximum under System / Advanced / Miscellaneous. This is necessary on my N4200-based router as boost speeds are required to achieve gigabit throughput with PPPoE. Also, ensure you haven't limited the core speed in your VM (assuming KVM supports this).

ALTQ relates to traffic shaping, hn refers to NICs using the hn driver, so this won't improve things. Certainly, on low-end NICS, disabling checksum offloading may be required. Realtek probably (still) qualifies as low-end, and historically haven't been recommended under FreeBSD. For the best experience, using Intel NICs if you can.

I only use Intel (or Mellanox for 10GbE) NICs after having a motherboard close to 20 years ago with a cheap NIC (not Realtek) that randomly corrupted data if checksum offloading remained turned on.