The Bad News from Synology#

In previus posts I have talked about the necessity of a cheap NAS appliiance, but then I also mentioned a Synology as a pricey but decent device for your homelab. I need to be clear that I no longer recommend Synology. I use mine because I already have one, but I would not buy a new Synology system today.

New Synologies require you to buy your disks from Synology, (a form of _enshittification _) so unless you can buy a “pre-cataclysm” relic from eBay, go with something else, because “screw Synology and screw enshittification.

If you are reading this, please don’t buy a Synology because I have one. I am fortunate to have once been an IT manager and at that time I could afford such expensive kit. I know that I am privileged enough to buy not one but two dedicated NAS appliances and others may not be. If you have more room in your lab than budget, you can buy a 10 year old refurbished PC with a full sized case and put drives in there. My lab is a research project. When I am satisfied that I have found a device that will work as a “datacenter in a box” I will endorse it.

I spend money on disks, not NAS hardware#

For me, disks are easily the biggest cost of a NAS build, and being tied into Synology’s ecosystem for my drives is no bueno. Instead, I figure out my disk budget first, and pick the NAS hardware later. I went with a large number of small disks because I could. If you are deciding between two big disks and 8 small ones, all other costs being equal, I would go with the 8 small ones. I know there is a temptation to “buy once, cry once” but hard disks are not an investment. They are a consumable.

It doesn’t matter how much I spend on a hard drive, it will eventually fail. After 30 years in IT, my philosophy is that hard drives have only 3 states: new (to you) in the plastic, almost full, and failed. RAID is not a backup. Snapshots are not a backup. The only backup that I can trust is one that is on a completely different system, be it an external drive or a second server. My backup rule is 3, 2, 1: 3 copies of the data, 2 different media types, 1 either off site or offline, but preferably both. I find that safety comes from numbers of copies of data, not from expensive hardware.

RAID arrays#

Speaking of failed disks, RAID arrays have different configurations for different uses. The rule is basically that you can have an array that is fast, redundant, and efficiently allocates space. Pick two. Data safety isn’t just about redundancy, it’s also about recovery. When a disk in a RAID array fails, the rest of the array is vulnerable to failure until the failed disk is replaced AND the array has rebuilt itself:

  • In the case of RAID0 (striped), the array is very fast (2 disks=2x the speed of a single disk, 3 disks= 3x) and there are no redundant disks, so if you have 3 disks that are 3Tb in size, you get a 9Tb array. However, losing a single disk trashes the array. Like Xmas lights: when one goes out, they all go out.
  • In the case of RAID1 (mirrored) You gain no speed increase, but you can lose one of the disks and keep your data. If that other disk fails before the failed disk can be replaced, you’re still cooked. However you can add additional disks to the mirror, so if you have 3 disks that are 3Tb in size, you will get a 3Tb array that is 3 identical copies. Very safe, very wasteful for more than 3 disks. When I am building servers at work, I use 2 mirrored drives for the operating system and the remaining drives for data.
  • In the case of RAID5 (striped with 1 redundant disk) You a read speed increase, but no write speed increase. You can lose one drive. Your RAID5 array is effectively a slow moving RAID0 until the failed disk is replaced and the array has been rebuilt. This means that the two or more non-redundant drives have to rebuild the entire array. That’s probably not a big deal for a small number (3) of small (>1tb) drives. But it’s an enormous deal if you have several multi-terabyte drives, where rebuilding the array could take a week or more. Also, during the rebuild, all of the drives are running at full speed, so the array slows down significantly. Adding a hot spare into the mix will save the array from failure if a second disk fails, but it will not speed up the rebuild.
  • In the case of RAID6 (striped with 2 redundant disks) You gain a read speed improvement, but still no write speed improvement. If you lose one disk, your array only degrades to RAID5. When you rebuild, you are rebuilding even more parity data, so the rebuild actually happens significantly slower than RAID5. A hot spare improves data safety, but it doesn’t speed up the rebuild process.
  • In the case of RAID10 (striped+mirrored) you get a read speed increase equal to your total number of disks (4 disks is 4x as fast as a single disk) and a write speed increase equal to half the number of your disks(4 disks is 2x faster than a single disk). In addition you can lose at least one disk and the array will continue to function. You could also lose up to half of your disks, but only under specific conditions. A RAID10 array is a stripe (RAID1) of mirrors (RAID1), or vice versa. Each disk is mirrored to another disk, and the mirrors are striped together, meaning that if you lose both of the disks in a mirror, you could lose the entire array. When rebuilding the array, only the data from the mirrored disk needs to be restored, so recovery time is significantly faster than RAID5 or RAID6.

So which one do I choose?#

The right decision ultimately comes down to

  1. how many disks I have
  2. how important the data is (redundancy)
  3. how much speed I need

That means that:

  • if I had only two disks, I would probably use RAID1
  • unless I just need a ton of fast storage, then I’d use RAID0, but I’d definitely need a backup.
  • if I had at least 4 or more disks, RAID10 would give me a good balance of speed, safety, and recoverability
  • if I had 5 or more disks, this is where things get interesting. Assuming I had 5 drives that are 2Tb in size:
    • I could do RAID10 with a hot spare
      • It gives me speed+safety+recoverability+ peace of mind with the hot spare
      • Assuming I don’t lose the magic 2 disks at exactly the same time, I can in theory lose at least two disks before you are in trouble
      • 5x 2Tb disks will give me 4Tb of usable storage
    • You could do RAID6
      • It will give me more space at the cost of recoverability
      • 5x 2Tb disks will give me 6Tb of usable storage, but rebuilding the array will take a long time.
  • If I had 6 disks or more, I’d just use RAID10, and if I had an odd number, I’d go for the hot spare, if not, go I’d with more space.
  • If I was shopping for disks, I prefer to use smaller disks for my array, and one large disk that is roughly the size of the whole array for backups. So if I wanted 8Tb total storage, I’d buy 4x4Tb disks for 8Tb of fast-ish/resilient-ish space, and a single 8Tb disk to house a backup copy of my array. The 8Tb disk could be an external drive that I only plug in/turn on when I made backups. The rest of the time it would stay powered off, and possibly located at a friend’s or relative’s house. If I was in 16+Tb territory, it might be time to look at a second server. This is what scratchy, my temporary NAS is.

If you are shopping for a NAS appliance, I recommend an x86/AMD64 platform that you can run Proxmox on. No, Proxmox is not a NAS, but it will allow you to build a VM that you can pass your disks to directly, and you can run a NAS system from there. I know that NAS experts will hate that idea, but if you can only run one box, that is the system that I would go with. Another way is to build a ZFS volume in Proxmox and then put a Linux container on that volume. There are good reasons for this, that I can get into later, but for me, NFS seems to only work well on VMS, so that’s what I went with.