The original post: /r/datahoarder by /u/stenyak on 2024-08-18 23:33:11.
Hey all, I'd appreciate advice on how to organize my data and backups.
The idea is to be able to recover from bitrot, from powerloss corruptions, from accidental file deletions, etc.
Note: I don't care about offsite backups atm: I'll rethink my current remote backups at a later stage.
Currently I have a 12tb and a 16tb EXT4 drives, both already nearly filled, and both containing data as well as some daily borg backups with cron. Borg is okay for accidental file deletions, but not for bitrot/powerloss/badsectors/failure/etc.
Since I've ran out of free space, I just purchased two 22tb drives, but I'm not sure how to expand storage while adding reliability. Some options I was considering:
- snapraid spanning all 4 drives (since they are different sizes and can work with those ext4 drives). I guess I need to use a cron to remount all 4 drives as read-only during the daily snapraid sync.
- snapraid-btrfs with the 2 new drives (so i can btrfs-snapshot, instead of doing the read-only dance). But this means the original 12tb and 16tb drives are unsafe and cannot recover from bitrot or similar issues. So maybe dedicate those exclusively for borg backups with monthly borg checksums verifications?
- A ZFS mirror with the 2 new 22tb drives. This also leaves the 2 old drives unsafe to bitrot.
- Get one (or a bunch of) more 22tb drives, and do some sort of ZFS raid maybe. Again the old drives are unsafe.
Main headache is that some of these solutions require reformatting old drives (which would be a pain), or require same-sized disks everywhere (I wouldn't have any other use for those 2 drives, and I don't want to sell them 2nd hand), or require a read-only period for a daily sync (will surely lead to issues on programs trying to write), or can lose the data generated before the daily sync (so not 100% resistant to bitrot), or.....
So yeah, I'm a bit stuck with analysis paralysis atm, any help would be great :D