The original post: /r/datahoarder by /u/HoardingBitByBit on 2025-02-02 19:21:01.
I have roughly 50 TB of Movies and TV Shows on a NAS with raid6 that runs Plex. Up until now, I haven't put much thought into backing that data up, mainly because it's .. well .. just movies and I can probably get it back in case of a disaster, but it would certainly be annoying and time-consuming. Looking at other posts here, some share that sentiment (e.g. https://www.reddit.com/r/DataHoarder/comments/1es4ry2/do_you_guys_backup_your_movies/ ).
While I have looked into multiple backup solutions like Borg, and others, they all lack that feature of what I wanted to achieve and provide much more of things that I don't need for that kind of data. I considering writing a tool for this usecase, but before I start I wanted to ask here if that already exists in some form.
My thinking was: Why not just buy a few 20TB Exos HDDs and store the data on there and keep the disk mostly offline. From time to time, run a script / tool that makes compressed archives of individual movie-directories and store them on a disk. The backup disk will not be online all the time, only while backing up stuff. In case my main NAS goes to hell beyond my raid 6 fault tolerance, at least I have a single backup of them. I could build a database stored on the nas that keeps track of file hashes and what is backed up to which disk. In case the directory changes because of updates (TV Shows, or improved encoding) the backup can be replaced or updated.
To be clear: This should not become a full-fledged backup solution for business critical data with a 3-2-1 backup. I don't need data deduplication and complex versioning. I just want to make sure that I have a backup for each Movie / TV Show somewhere. I just want to attach a HDD to the NAS, run the backup that compresses new files/folders and keeps track of where it's stored. If the disk is full, I will just buy another disk and continue from there. The metadata of what is stored where, when and what fingerprint it had can stay on the nas or I can back it up to a cloud host, since it wouldn't be much data.
Is there already something around that does that or something similar? Otherwise I'll probably write something in the near future.