Geekzone: technology news, blogs, forums
Guest
Welcome Guest.
You haven't logged in yet. If you don't have an account you can register now.


2317 posts

Uber Geek
+1 received by user: 655


Topic # 240112 21-Aug-2018 13:46
Send private message quote this post

I have 4 identical HDDs (4x 4TB WD Black SATA) that I will be installing in my PC, they will be attached to an HP P420 RAID controller with FBWC.

 

Much of what this PC will be doing is remuxing large video files, so there will be quite a bit of simultaneous reading and writing.

 

Which will perform better:

 

A. Two RAID1 arrays, where one array is read from while the other is written to.

 

B. A single RAID10 where the array is read from and written to at the same time.

 

Thanks.





 Home:                                                           Work:
Home Work


View this topic in a long page with up to 500 replies per page Create new topic
 1 | 2
686 posts

Ultimate Geek
+1 received by user: 281

Subscriber

  Reply # 2076776 21-Aug-2018 13:52
Send private message quote this post

A

 

 








2437 posts

Uber Geek
+1 received by user: 718

Trusted
Lifetime subscriber

  Reply # 2076842 21-Aug-2018 14:00
2 people support this post
Send private message quote this post

I think the difference would be small, and the 'one lump of disk' of the RAID10 would be less hassle.....  it would depend a bit on the speed of the processing (number and scale of the effects being applied, resolution, etc) whether the processor is the bottleneck or the RAID/cache/drives.

 

I'd encourage you to give it a try for yourself on a modest sized file and let us know your results.  Blowing away the one RAID and creating another is quick and easy if you have not already filled the drives!





"4 wheels move the body.  2 wheels move the soul."

“Don't believe anything you read on the net. Except this. Well, including this, I suppose.” Douglas Adams



2317 posts

Uber Geek
+1 received by user: 655


  Reply # 2076844 21-Aug-2018 14:10
Send private message quote this post

Dynamic:

 

I think the difference would be small, and the 'one lump of disk' of the RAID10 would be less hassle.....  it would depend a bit on the speed of the processing (number and scale of the effects being applied, resolution, etc) whether the processor is the bottleneck or the RAID/cache/drives.

 

I'd encourage you to give it a try for yourself on a modest sized file and let us know your results.  Blowing away the one RAID and creating another is quick and easy if you have not already filled the drives!

 

 

It's mainly remuxing rather than encoding or re-encoding, so very little processing power required. The (comparatively) slow 7200rpm spindles will almost always be the bottleneck for my usage.

 

Yeah, might have to just test both options and see if there is enough of a difference worth worrying about.


317 posts

Ultimate Geek
+1 received by user: 75


  Reply # 2076850 21-Aug-2018 14:34
Send private message quote this post

Paul1977:

 

Dynamic:

 

I think the difference would be small, and the 'one lump of disk' of the RAID10 would be less hassle.....  it would depend a bit on the speed of the processing (number and scale of the effects being applied, resolution, etc) whether the processor is the bottleneck or the RAID/cache/drives.

 

I'd encourage you to give it a try for yourself on a modest sized file and let us know your results.  Blowing away the one RAID and creating another is quick and easy if you have not already filled the drives!

 

 

It's mainly remuxing rather than encoding or re-encoding, so very little processing power required. The (comparatively) slow 7200rpm spindles will almost always be the bottleneck for my usage.

 

Yeah, might have to just test both options and see if there is enough of a difference worth worrying about.

 

 

I would say 4 drive in raid 10 would be better - write is where you get the bottlenecks, and more disks in a raid help a lot on the iops on writes.


467 posts

Ultimate Geek
+1 received by user: 141


  Reply # 2076851 21-Aug-2018 14:34
Send private message quote this post

If you're just remuxing, two independent disks without any level of RAID would likely offer the best performance.




2317 posts

Uber Geek
+1 received by user: 655


  Reply # 2076861 21-Aug-2018 14:54
Send private message quote this post

SirHumphreyAppleby:

 

If you're just remuxing, two independent disks without any level of RAID would likely offer the best performance.

 

 

I want the redundancy though as the disk will also be used for storage.




2317 posts

Uber Geek
+1 received by user: 655


  Reply # 2076862 21-Aug-2018 14:58
Send private message quote this post

bagheera:

 

I would say 4 drive in raid 10 would be better - write is where you get the bottlenecks, and more disks in a raid help a lot on the iops on writes.

 

 

I'm leaning the other way. Certainly the max write speed is better with RAID10, but this will surely be counteracted by it performing an equal number of reads at the same time.

 

I think testing it will definitely be required!


317 posts

Ultimate Geek
+1 received by user: 75


  Reply # 2076868 21-Aug-2018 15:03
Send private message quote this post

Paul1977:

 

bagheera:

 

I would say 4 drive in raid 10 would be better - write is where you get the bottlenecks, and more disks in a raid help a lot on the iops on writes.

 

 

I'm leaning the other way. Certainly the max write speed is better with RAID10, but this will surely be counteracted by it performing an equal number of reads at the same time.

 

I think testing it will definitely be required!

 

 

 

 

for raid 1 or raid 10 the read iops is nx, write is 0.5nx where n = number of drives, x is ipos of drive so for  2 x raid 1 read iops is 2x your write iops is x, for a raid 10 your read iops is 4x, your write iops is 2x - over all with reading and writing raid 10 will be better for raid 10. 50/50 blend read / write = (RIOPS * .5) + (WIOPS * .5) so (4*.5)+(2*.5) = 3x vs write raid 1 of x


467 posts

Ultimate Geek
+1 received by user: 141


  Reply # 2076873 21-Aug-2018 15:12
Send private message quote this post

Paul1977:

 

SirHumphreyAppleby:

 

If you're just remuxing, two independent disks without any level of RAID would likely offer the best performance.

 

 

I want the redundancy though as the disk will also be used for storage.

 

 

I think more details of your workflow is required to work out the best solution. E.g. do you remux in one or two steps, and do you intend to keep the original and remuxed files, or any of the intermediary data, and how much data?

 

I recently remuxed approximately 15TB of video. I did it all as batch jobs, with the source and output stored on a NAS (RAID6), and intermediary files destroyed. The actual remuxing was done on a PCI-E SSD, where read/write wasn't an issue (warranty aside - that's 20% of the drives guaranteed write life gone in about a week), but if I had to use mechanical drives, I would split the source and destination where possible. Obviously, this was a lot slower than doing it with local storage (or even installing software on the NAS), but in my case the time taken didn't matter that much. It may be more important in your case.


686 posts

Ultimate Geek
+1 received by user: 281

Subscriber

  Reply # 2076964 21-Aug-2018 17:06
Send private message quote this post

Paul1977:

 

I think testing it will definitely be required!

 

 

I wouldn't bother. As re-muxing is basically a simultaneous multi GB sequential read and write it's a no brainer. Two raid 1 volumes makes for 1 seek for the read volume and 1 for the write volume, and it's like a fire hose from there on in.

 

 

 

 










2317 posts

Uber Geek
+1 received by user: 655


  Reply # 2077077 21-Aug-2018 20:25
Send private message quote this post

Well, Tested and found the following:

 

RAID10 writing 320 MB/s

 

RAID10 reading 320 MB/s

 

RAID10 simultaneous reading and writing 120MB/s

 

RAID1 reading 160 MB/s

 

RAID1 writing 160 MB/s

 

I tested the max read and write speeds by copying to and from my existing RAID5 array.

 

Interestingly even when copying a large file off the RAID1 onto the faster existing RAID5 the read speed still topped out around 160 MB/s - I had been expecting the read speed of RAID1 one to be higher than the write speed, but this was not the case.

 

Now I think I need to do as @SirHumphreyAppleby suggests and think about my workflow. 2x RAID1 certainly makes the work on the new array(s) faster than 1x RAID10, but 1xRAID10 makes moving data on and off the new array(s) twice as fast as 2x RAID1.




2317 posts

Uber Geek
+1 received by user: 655


  Reply # 2077078 21-Aug-2018 20:30
Send private message quote this post

gbwelly:

 

Paul1977:

 

I think testing it will definitely be required!

 

 

I wouldn't bother. As re-muxing is basically a simultaneous multi GB sequential read and write it's a no brainer. Two raid 1 volumes makes for 1 seek for the read volume and 1 for the write volume, and it's like a fire hose from there on in.

 

 

Turns out you are correct. But the more I think about how this will fit in with the rest of my setup I suspect RAID10 might actually prove better overall.


530 posts

Ultimate Geek
+1 received by user: 125

Subscriber

  Reply # 2077312 22-Aug-2018 11:41
Send private message quote this post

If it's only short term processing have you considered RAID-0 ?



2317 posts

Uber Geek
+1 received by user: 655


  Reply # 2077522 22-Aug-2018 15:36
Send private message quote this post

solutionz: If it's only short term processing have you considered RAID-0 ?

 

I still want to have it redundancy.




2317 posts

Uber Geek
+1 received by user: 655


  Reply # 2077536 22-Aug-2018 15:51
Send private message quote this post

I just had a very interesting and unexpected result.

 

I put the 4 drives in single RAID5 and am getting comparable max read and write performance as with RAID10, but the really surprising part is I am actually getting better simultaneous read/write performance than either 1x RAID10 or 2x RAID1.

 

When doing a simultaneous read/write (as in remuxing from and to the same array) the speed fluctuates up and down a lot with RAID5, but is averaging about 195 MB/s (compared to steady 160MB/s between 2x RAID1 and a steady 120MB/s on 1x RAID10)

 

And RAID5 gives me 50% more usable space.

 

My only conclusion is that the parity generation on the P420 is VERY FAST, combined with the data being striped across more disks.

 

 


 1 | 2
View this topic in a long page with up to 500 replies per page Create new topic

Twitter »

Follow us to receive Twitter updates when new discussions are posted in our forums:



Follow us to receive Twitter updates when news items and blogs are posted in our frontpage:



Follow us to receive Twitter updates when tech item prices are listed in our price comparison site:



Geekzone Live »

Try automatic live updates from Geekzone directly in your browser, without refreshing the page, with Geekzone Live now.



Are you subscribed to our RSS feed? You can download the latest headlines and summaries from our stories directly to your computer or smartphone by using a feed reader.

Alternatively, you can receive a daily email with Geekzone updates.