Geekzone: technology news, blogs, forums
Guest
Welcome Guest.
You haven't logged in yet. If you don't have an account you can register now.


SavageNZL

52 posts

Master Geek
+1 received by user: 7


#311759 12-Feb-2024 13:32
Send private message

Hi All,

Recently set up a home server using unraid on a intel 6500i cpu, with a couple of refurbed 4tb nvr drives.
I was using my asus router's 4 port 1gbit switch to connect my home pc to the server and transferring files I achieved a stable 112mByes/s and drops off to ~50mB/s.

I have two crucial 500gb nvme drive in cache, how do I utilise these to minimise the drop off I assume due to HDD write speeds.

the second item, when transferring TB's of data, and it taking many hours, I thought I would upgrade my NIC in both systems to have an Asus C100C card (Brand new on trade me $99ea currently), and directly connect each system.
I only reach 700mBytes/s and quickly drops off to 50mB/s, playing around last night I installed open speed test, and was only achieving 3gbit down and 1.5gbit up.
I installed iperf3 and tested locally on the server and on the desktop, the server to localhost reaches 40bit no issue, and the desktop reaches 60gbit no issue.
But when talking to each other, I set the ip directly 192.168.xx.2x1 and 192.168.xx.2x2 they will cap at 3.29gbit/s regardless if single thread or 20 threads to send data.
I've tried enabling jumbo packets, turning off real time security and firewalls. I thought it was CPU bound but that has been ruled out from the above localhost tests.
I thought it was a cat 5e issue but replaced with 6a and no change in speed. Changing to chrome instead of edge/firefox resulted in OST 3gbit up and down.

Any ideas will be appreciated, google is limited, have tried a half dozen things. (I have not tried reinstalling windows as 2 posts suggested)





Quic Broadband Referal Code: https://account.quic.nz/refer/8901 
Use the link above if you want to help me out.
Use this unique promo code at checkout for free setup ($29 value): R8901EZZNPI
Thank you!


Filter this topic showing only the reply marked as answer Create new topic
Ruphus
469 posts

Ultimate Geek
+1 received by user: 181


  #3194349 12-Feb-2024 14:44
Send private message

You'll need to set the primary storage for the share to the Cache and then set the Array as the secondary storage. The mover action can then be set to transfer files from Cache to the Array. The mover action usually runs on a schedule, so you'll need to be mindful of that if you're trying to transfer terabytes of data into your array.




wellygary
8810 posts

Uber Geek
+1 received by user: 5287


  #3194355 12-Feb-2024 14:50
Send private message

If you are gonna dump TBs on to the array, then the HDD speed will become your choke point and everything will be limited to the write speed....

 

Have you run any metrics on the HDD Drives to see what they top out at under heavy load...


toejam316
1516 posts

Uber Geek
+1 received by user: 888

Trusted
Lifetime subscriber

  #3194358 12-Feb-2024 14:57
Send private message

Make sure your shares are set to write to cache, then move things into the array afterwards, that'll use it as your buffer. When you have two cache drives, they're essentially run in RAID 1, so the max speed you'll be getting from them is the speed of one drive, which 3.29Gbps sounds like it could be meeting expectations.

 

As for the second thing, your cache is a write cache. Anything you write to it has to be moved into the array. If you're moving TBs of data, you may as well disable the cache and write directly into the array, unless you have TBs of cache. If you wanna speed stuff up, you gotta buy more and faster equipment. Each bit written into your array requires your CPU to calculate parity, and update the parity drive, and each file moved into the array is written as a single file onto one drive, so there's no speed gains from striping, etc.





Join Quic Broadband with my referral - no sign up fee and gives me account credit

 

Anything I say is the ramblings of an ill informed, opinionated so-and-so, and not representative of any of my past, present or future employers, and is also probably best disregarded.




michaelmurfy
meow
13579 posts

Uber Geek
+1 received by user: 10910

Moderator
ID Verified
Trusted
Lifetime subscriber

  #3194366 12-Feb-2024 15:28
Send private message

Many SSD's will also only have a smaller percentage of SLC Cache which operates at full speed. I think the NVMe drive you've got has no DRAM so all operations are done on the NAND itself.

 

If you do a sustained write on these drives you'll note performance will tank once the SLC Cache is filled. For most people a smaller SLC cache doesn't matter (many drives have around a 30gb cache) but for larger operations it can be rather annoying.

 

I'm using 2x Samsung 990 Pro's in my UnRaid server which don't seem to have this issue but have been through about 3 SSD's now with this exact issue and it was simply horrible on UnRaid.





Michael Murphy | https://murfy.nz
Referral Links: Quic Broadband (use R122101E7CV7Q for free setup)

Are you happy with what you get from Geekzone? Please consider supporting us by subscribing.
Opinions are my own and not the views of my employer.


Jase2985
13730 posts

Uber Geek
+1 received by user: 6202

ID Verified
Lifetime subscriber

  #3194371 12-Feb-2024 15:40
Send private message

also going to put a tonne of writes on the NAND if you are shifting that much data. 


SavageNZL

52 posts

Master Geek
+1 received by user: 7


  #3194840 13-Feb-2024 19:49
Send private message

forgot to hit send on my previous reply, but running a command on unraid said that speed was 8GT/s downgraded, width x2. Changed pcie slots just as random troubleshooting thing as you do. 
Same command on another slot, 8GT/s width x2 same as before, run the iperf 3. some reason getting 9.5gb/s now.
Its a pcie 4x card, only using 2x slots, however the motherboard is x16 x8 x8 x8 and somehow no idea why, different performance from different slots.





Quic Broadband Referal Code: https://account.quic.nz/refer/8901 
Use the link above if you want to help me out.
Use this unique promo code at checkout for free setup ($29 value): R8901EZZNPI
Thank you!


 
 
 

Shop now at Mighty Ape (affiliate link).
toejam316
1516 posts

Uber Geek
+1 received by user: 888

Trusted
Lifetime subscriber

  #3194849 13-Feb-2024 20:15
Send private message

Possibly direct attached PCIe lanes versus chipset PCIe lanes, or electrical lane count vs slot lane count




Join Quic Broadband with my referral - no sign up fee and gives me account credit

 

Anything I say is the ramblings of an ill informed, opinionated so-and-so, and not representative of any of my past, present or future employers, and is also probably best disregarded.


Filter this topic showing only the reply marked as answer Create new topic








Geekzone Live »

Try automatic live updates from Geekzone directly in your browser, without refreshing the page, with Geekzone Live now.



Are you subscribed to our RSS feed? You can download the latest headlines and summaries from our stories directly to your computer or smartphone by using a feed reader.