In theory, RAID 5 is slower than RAID 1+0 for general use. In practice, our RAID controller stores writes in 1GB of RAM and reports immediate completion, giving similar write performance regardless of RAID level. It's got a backup capacitor; if you pull the plug out, it stores in-flight data in flash until power is restored.
RAID 5 has become less popular as disk capacity has increased faster than write speeds. Having to read each disk during a restore stresses them; if any more fail, you face having to restore at least part of your data from backups - and parity calculations limit rebuild performance (to around 33MB/sec in our case). But we only have four disks, and they're not too big. They've lasted several years and average a 25-35% utilization rate at 33°C, which research suggests is close to ideal.
As a bonus, RAID 5 can be faster for reads. Due to their construction, the start of a hard disk is normally faster than the end - this is why they get slower as they fill up. With data striped across three disks rather than two, the "distance" from the start is reduced by 33%. We get many requests for little files, and we can't cache all of them, so read access time is important.
That's why we moved to RAID 5. But how? On the RAID side, it took just two steps:
* Transforming from RAID 1+0 to RAID 5 (the longest part; the controller had to rewrite every bit) * Increasing the size of the logical disk (almost instant, with some background work)
The transformation was timed to fall outside our peak period - a good thing, since it took 15 hours. The controller was now presenting a 3TB "logical disk" - and once we'd poked Linux, it knew that as well.
Unfortunately, our system was set up with a master boot record for its partitions. MBR only works up to the 2TB mark. That's fine when your disk is 2TB, but poses problems after that.
What we had to do was convert to GPT, allocate a BIOS boot partition to store our bootloader's second stage, reinstall said bootloader, replace the old partition record with one indicating that the partition was now larger, reboot to get Linux to accept the new partition values, and then expand the filesystem.
As one staff member noted, it felt "like open heart and brain surgery all at once". Inkbunny's server is in a datacenter hundreds of miles away; if we'd got it wrong, we'd be left to fix it in rescue mode (the equivalent of a Live CD) over a remote connection. But it all worked out in the end.
From what we can see, there's essentially no difference in performance - we just have more space. If anything, disk utilization has decreased. As a bonus, if we have to increase storage beyond 3TB, it'll take one more disk, rather than two. Hopefully we won't have to do that for about five years, though!