> we should stick to filesystems with data checksums such as ZFS, as well as ECC memory.
While I don't disagree with this statement, consider the reality:
- APFS has metadata checksums, but no data checksums. WTF Apple?
- Very few Linux distributions ship zfs.ko (&spl.ko); those that do, theoretically face a legal risk (any kernel contributor could sue them for breaching the GPL); rebuilding the driver from source is awkward (even with e.g. DKMS), pulls more power, takes time, and may randomly leave your system unbootable (YES it happened to me once).
- Linux itself explicitly treats ZFS as unsupported; loading the module taints the kernel.
- FreeBSD is great, and is actually making great progress catching up with Linux on the desktop. Still, it is a catch-up game. I also don't want to install a system that needs to install another guest system to actually run the programs I need.
- There are no practical alternatives to ZFS that even come close; sibling comment complains about btrfs data loss. I never had the guts to try btrfs in production after all the horror stories I've heard over the decade+.
- ECC memory on laptops is practically unheard of, save for a couple niche Thinkpad models; and comes with large premiums on desktops.
What are the practical choices for people who do not want to cosplay as sysadmins?
As a rule of thumb (IMHO) steer clear of commodity NAS Cloud add ons, such things attract ransomware hackers like flies to a tip whether it's QNAP, Synology, or InsertVendorHere.
I feel that on HN people tend to be a bit pedantic about topics like data integrity, and in business settings I actually agree with them.
But for residential use, risks are just different and as you point out, you have no options except to only use a desktop workstation with ECC. People like/need laptops so that’s not realistic for most people. Just run Linux/freebsd with ZFS isn’t reasonable advice to me.
What I feel most strongly about is that it’s all about circumstances, context and risk evaluation. And I see so much blanket absolutist statements that don’t think about the reality of life and people’s circumstances.
“Tainting” the kernel doesn’t affect operations, though. You’re not allowed to redistribute it with changes — but you, as an entity, can freely use ZFS and the kernel together without restriction.
Linux plus zfs works fine.
> - Very few Linux distributions ship zfs.ko (&spl.ko); those that do, theoretically face a legal risk (any kernel contributor could sue them for breaching the GPL)
> - Linux itself explicitly treats ZFS as unsupported; loading the module taints the kernel.
So modify the ZFS source so it appears as a external GPL module... just don't tell anyone or distribute it...
I can't say much about dracut or having to build the module from source... as a Gentoo user, I do it about once a month without any issues...
> What are the practical choices for people who do not want to cosplay as sysadmins?
The specific complaint is not at all about the kernel identifying itself as tainted, the specific complaint is about the kernel developers' unyielding unwillingness to support any scenario where ZFS is concerned, thus leaving one with even more "sysadmin duties". I want to use my computer, not serve it.
> I never had the guts to try btrfs in production after all the horror stories I've heard over the decade+.
I've been running btrfs as the primary filesystem for all of my desktop machines since shortly after the on-disk format stabilized and the extX->btrfs in-place converter appeared [0], and for my home servers for the past ~five years. In the first few years after I started using it on my desktop machines, I had four or five "btrfs shit the bed and trashed some of my data" incidents. I've had zero issues in the past ~ten years.
At $DAYJOB we use btrfs as the filesystem for our CI workers and have been doing so for years. Its snapshot functionality makes creating the containers for CI jobs instantaneous, and we've had zero problems with it.
I can think of a few things that might separate me from the folks who report issues that they've had within the past five-or-ten years:
* I don't use ANY of the built-in btrfs RAID stuff.
* I deploy btrfs ON TOP of LVM2 LVs, rather than using its built-in volume management stuff. [1]
* I WAS going to say "I use ECC RAM", but one of my desktop machines does not and can never have ECC RAM, so this isn't likely a factor.
The BTRFS features I use at home are snapshotting (for coherent point-in-time backups), transparent compression, the built-in CoW features, and the built-in checksumming features.
At work, we use all of those except for compression, and don't use snapshots for backup but for container volume cloning.
[0] If memory serves, this was around the time when the OCZ Vertex LE was hot, hot shit.
[1] This has actually turned out to be a really cool decision, as it has permitted me to do low- or no- downtime disk replacement or repartitioning by moving live data off of local PVs and on to PVs attached via USB or via NBD.
>At $DAYJOB we use btrfs as the filesystem for our CI workers and have been doing so for years. Its snapshot functionality makes creating the containers for CI jobs instantaneous, and we've had zero problems with it.
While I don't disagree with this statement, consider the reality:
- APFS has metadata checksums, but no data checksums. WTF Apple?
- Very few Linux distributions ship zfs.ko (&spl.ko); those that do, theoretically face a legal risk (any kernel contributor could sue them for breaching the GPL); rebuilding the driver from source is awkward (even with e.g. DKMS), pulls more power, takes time, and may randomly leave your system unbootable (YES it happened to me once).
- Linux itself explicitly treats ZFS as unsupported; loading the module taints the kernel.
- FreeBSD is great, and is actually making great progress catching up with Linux on the desktop. Still, it is a catch-up game. I also don't want to install a system that needs to install another guest system to actually run the programs I need.
- There are no practical alternatives to ZFS that even come close; sibling comment complains about btrfs data loss. I never had the guts to try btrfs in production after all the horror stories I've heard over the decade+.
- ECC memory on laptops is practically unheard of, save for a couple niche Thinkpad models; and comes with large premiums on desktops.
What are the practical choices for people who do not want to cosplay as sysadmins?