It's A Digital Disease!

23 readers
1 users here now

This is a sub that aims at bringing data hoarders together to share their passion with like minded people.

founded 2 years ago
MODERATORS
351
 
 
The original post: /r/datahoarder by /u/Nomad_76 on 2025-07-20 19:35:12.

Have a 16 bay dell r720 with 2.5bay drives I’m looking to fill so I can run a second NAS. My current one I bought with the drives already lovely 16 15K 300gb SAS HDD. Asking the people who’ve been doing this longer on a good SSD brand that won’t absolutely kill my wallet but are still reliable. Thank you for your time!

352
 
 
The original post: /r/datahoarder by /u/drowned_phoenix on 2025-07-20 18:59:05.

Hey fellow datahoarders, I've been data hoarding for more than ten years, but only now I am starting to implement more serious backup and error prevention measures, as only recently I've discovered all the risks and nuances of this practice.

Today I have 3 copies of my data, the production copy in the HDs/SSDs in my machine and two copies stored in external hard drives, which are kept offline. These two HDDs use exFAT format. I am also researching for a good cloud solution to keep a fourth, off-site copy.

So, my main concern right now is making sure the data in these external hard drives doesn't get corrupted and that I'm able to identify and recover from errors.

What kind of strategies can I implement to better keep my data safe?

In terms of limitations, I'm not able to buy new hardware for the next few months and this data needs to be readable in Windows OS.

Thanks in advance!

353
 
 
The original post: /r/datahoarder by /u/c0de854-T on 2025-07-20 18:52:11.

I am looking for a 3.5" hard drive with either 8TB or 6TB capacity. I want to store this hard drive for two years without power. Is this possible?

What kind of hard drive would you recommend?

I plan to use this case to store it - what do you think? https://www.orico.cc/index/product/detail/942.html

354
 
 
The original post: /r/datahoarder by /u/ThePirer on 2025-07-18 16:33:21.

Hi guys,

In the past, I just used VLC as a player for watching movies and series. However, since last year, I've been running an emby server in my laptop, since it is always on, and it's been amazing. Because of that, I want to buy a NAS in like 2-3 years, since right now it is not possible for different reasons.

When looking at NAS, I found them to be very limiting. What if I needed more disks, more ram, a more powerful CPU or whatever in the future? If I do something, I optimize the shit out of it. In the end, I thought that a custom NAS would be the best option. But the cases are very expensive, or too big, or too small or too loud, or too ugly... So, I have an old pc tower with a ton of 5.2 and 3.5 slots. I removed those racks and 3D printed a 12 bay rack in TPU with an attachment for 4 fans on the side, as well as an hexagon front mesh in PETG for airflow. A bit of walnut vinyl and now it looks like something made by Fractal Design, has as lot of storage, and can fit any MB and PSU while being smaller than a standard ATX.

With that out of the way, my 7-8 year old 5TB external HDD with movies and series is finally full, so I need to buy a new disk in the following months. But I thought that, instead of buying just another 5TB disk, the most cost-effective option would be to just go ahead and buy the disk that I would use in the NAS.

  1. Which capacity should I go for? 14 TB? 16? 20? It took me like 7 years to fill 5TB, maybe 14 would be enough to last me for years and taking into account the amount of bays at my disposal. Maybe 20TB is better because if the increased file size nowadays. Maybe the 18TB disk is of a higher quality because of the specific model. Also, in Server Part Deals there are mainly Seagate Exos and Ultrastars. Which model do you recommend? I would like to buy 2 disks to have a Raid 1, since the more data I have, the more I worry about losing it, and then going for a Raid 5, 6 or 10 or whatever when I eventually have to add more disks.

Now, once I have the disks, I have to connect them to the laptop to keep the emby server running. I've seen that there are docking stations for around 30€. I liked one from Orico. Now, the problem lies in the formats, since TrueNAS doesn't recognize NTFS and Windows doesn't recognize ZFS. 2 solutions come to mind:

  1. Since I'd have two mirrored disks, when I have the NAS set up, I can connect the mirror, create a pool, transfer the files and then set up the Raid 1. There's a risk of losing the data here, but I don't think the probability is high.
  2. I can use OpenZFS, but it doesn't seem easy nor reliable.

Which one would you choose? Is it possible? Are there more options? I'd like to hear your thoughts.

355
 
 
The original post: /r/datahoarder by /u/elsbeth-salander on 2025-07-18 20:58:25.

People may differ in their viewpoints on the quality or perspective of PBS programming in recent years, but there’s no denying that it has produced a lot of memorable series that many viewers enjoyed and which did have an intent to inform and/or educate the populace, including children.

Some of these shows ran for decades and therefore might not be on DVD box sets. For instance NOVA has aired since 1974. I’ve already noticed that some of the children’s series like The Puzzle Place are considered partially lost media due to being “copyright abandonware” (the original IP holder temporarily licensed it to public broadcasting but then went bankrupt, leaving the rights essentially in limbo).

With Paramount having obliterated all of its Daily Show archive from the website, it’s probably only a matter of time before something similar happens to those PBS series that are viewable in streaming format. Is there an effort under way to 1) download whatever can be saved to disk from their streaming video site, and/or 2) dispatch whatever else (reels, tapes, etc) is collecting dust in the vaults distributed among the various public broadcasters, to some kind of preservation service / museum (maybe outside the US?) before it gets sold off or thrown away?

356
 
 
The original post: /r/datahoarder by /u/PusheenHater on 2025-07-18 20:15:46.

I've got a bunch of external/internal hard drives, SSDs, flash drives, etc.

I'm using a cardboard box but I have so many hard drives that it's sagging. Not very sturdy.

I know plastic is static-y which is really bad for the hard drives.

So I ask if there's a container:

  • Big, that can hold many hard drives
  • Anti-static
  • Not plastic or cardboard
  • Sturdy
  • Preferably allows you to lock it up with a lock
357
 
 
The original post: /r/datahoarder by /u/Alphabethur on 2025-07-18 19:54:22.
358
 
 
The original post: /r/datahoarder by /u/Repulsive_Market_728 on 2025-07-18 18:21:05.

Just in case there's anyone who may be interested and who might have the space/resources to use something like this, I saw this up for auction. It closes at around 9pm eastern today (Friday the 18th).

https://www.allsurplus.com/en/asset/1021/13971

I also found this article which provides a pretty good overview of the system.

https://www.itpro.com/155268/quantum-scalar-i2000-tape-library

359
 
 
The original post: /r/datahoarder by /u/AshleyAshes1984 on 2025-07-18 17:35:28.
360
 
 
The original post: /r/datahoarder by /u/palepatriot76 on 2025-07-18 17:23:00.

So I have used DVDFab for well over 40 DVD boxed sets, no issues but I have an issue with my Benny Hill Megaset

I am crating ISO files fine, but when I try to watch I can hear but not see, and when I can see very messed up, pixelated and green screen

When I use those ISO files and Make MKV, same thing, just a mess

Is this a DVD protection thing? If so what is my next step?

361
 
 
The original post: /r/datahoarder by /u/aJakalope on 2025-07-18 17:18:15.

I'm mostly making this post because I googled the differences between these a lot before purchasing and wish I had seen a post like this before I had.

I currently use a Beelink Mini S12 as a Plex server and although I had been using external drives, I was running out of USB ports on the Beelink. So I was looking into a DAS to use and found very similar reviews for both products named in the title. The Terramaster was a little cheaper so I went with it, especially since I was not looking for proper RAID functionality since I use the drives for easily replaceable media files.

I used WD Red Pro 18TB drives for this.

The first drive I put in it seemed to function alright, but when I attached a second drive, there seemed to be issues. Drives randomly disconnecting, errors while transferring large files, qBitTorrent error messages I had never seen before, etc. I read that it was likely a cord issue, so I bought a nicer data cable. The issues persisted. I continued to check the drives using CrystalDiskInfo and it showed no problems on any of the drives.

I finally decided to order a QNAP to see if it was a drive issue and once I put the drives in the QNAP, they immediately were recognized, transfer speeds were faster, and I have not had any issues whatsoever.

I'd say I'm no expert at all in these fields, so it's possible that there was a small issue I was overlooking with the Terramaster. I've also only had the QNAP a few days, so it's possible I'll encounter issues down the road. But if anyone in the future is reading this and considering saving a few bucks and buying a Terramaster, go with the QNAP.

362
 
 
The original post: /r/datahoarder by /u/Gunfighter1776 on 2025-07-18 17:15:33.

I have never had a NAS. I know what it is, and I have used them in work environments - never from home network pov.

Question and Comment:

I have a PC with several hdd's -- I have data duplicated across the drives for redundancies in case one of the drives fail -- I have a total of 30tb - ish this includes all drives and duplicated data - so my conundrum is do I use this number to calculate how much actual drive space I need in my NAS setup?

Or do I just take ONE COPY of everything - and dump it onto my NAS... I ask because I don't know how the NAS -- in what will be most likely a RAID5 configuration -- will treat the data if I have several copies of the data also on my NAS... or will it just be that the duplicated data will be all spanned across all drives -- just like any other deployment of data in a NAS...

I guess I am asking -- what is best practice -and which is a best stragegy? ONE COPY of everything on my NAS... or several copies on the NAS in different folders??

I have a ugreen 4800plus -- and I am trying to buy drives big enough to grow into - but don't want to spend more than i have to -- I initially was going to go for a RAID5 3 DISK ARRAY and have an extra drive to drop in - in the event I need to save the data - or grow my data needs.

Advice?

363
 
 
The original post: /r/datahoarder by /u/itsbentheboy on 2025-07-18 15:51:11.

I have created a set of bashRC aliases for use with YT-DLP.

These make some longer commands more easily accessible without the need of calling specific scripts.

These should also be translatable to Windows as well since the commands are all in the yt-dlp binary - but I have not tested that.

Usage is simple, just use the alias that correlates with what you want to do - and paste the URL of the video, for example:

yt-dlp-archive https://my-video.url.com/video to use the basic archive alias.

You may use these in your shell by placing them in a file located at ~/.bashrc.d/yt-dlp_alias.bashrc or similar bashrc directories. Simply copy and paste the code block below into an alias file and reload your shell to use them.

These preferences are opinionated for my own use cases, but should be broadly acceptable. however if you wish to change them I have attempted to order the command flags for easy searching and readability. note: some of these aliases make use of cookies - please read the notes and commands - don't blindly run things you see on the internet.

##############
# Aliases to use common advanced YT-DLP commands
##############
# Unless specified, usage is as follows:
# Example: yt-dlp-get-metadata <URL_OF_VIDEO>
#
# All download options embed chapters, thumbnails, and metadata when available.
# Metadata files such as Thumbnail, a URL link, and Subtitles (Including Automated subtitles) are written next to the media file in the same folder for Media Server compatibility.
#
# All options also trim filenames to a maximum of 248 characters
# The character limit is set slightly below most filesystem maximum filenames
# to allow for FilePath data on systems that count paths in their length.
##############

# Basic Archive command.
# Writes files: description, thumbnail, URL link, and subtitles into a named folder:
# Output Example: ./Title - Creator (Year)/Title-Year.ext
alias yt-dlp-archive='yt-dlp \
--embed-thumbnail \
--embed-metadata \
--embed-chapters \
--write-thumbnail \
--write-description \
--write-url-link \
--write-subs \
--write-auto-subs \
--sub-format srt \
--trim-filenames 248 \
--sponsorblock-mark all \
--output "%(title)s - %(channel,uploader)s (%(release_year,upload_date>%Y)s)/%(title)s - %(release_year,upload_date>%Y)s - [%(id)s].%(ext)s"'

# Archiver in Playlist mode.
# Writes files: description, thumbnail, URL link, subtitles, auto-subtitles
#
# NOTE: The output will be a folder: Playlist_Name/Title-Creator-Year.ext
# This is different from the above, to avoid large amount of folders.
# The assumption is you want only the playlist as it appears online.
# Output Example: ./Playlist-name/Title - Creator (Year)/Title-Year.ext    
alias yt-dlp-archive-playlist='yt-dlp \
--embed-thumbnail \
--embed-metadata \
--embed-chapters \
--write-thumbnail \
--write-description \
--write-url-link \
--write-subs \
--write-auto-subs \
--sub-format srt \
--trim-filenames 248 \
--sponsorblock-mark all \
--output "%(playlist)s/%(title)s - %(creators,creator,channel,uploader)s - %(release_year,upload_date>%Y)s - [%(id)s].%(ext)s"'

# Audio Extractor
# Writes: <ARTIST> / <ALBUM> / <TRACK> with fallback values
# Embeds available metadata
alias yt-dlp-audio-only='yt-dlp \
--embed-thumbnail \
--embed-metadata \
--embed-chapters \
--extract-audio \
--audio-quality 320K \
--trim-filenames 248 \
--output "%(artist,channel,album_artist,uploader)s/%(album)s/%(track,title,track_id)s - [%(id)s].%(ext)s"'

# Batch mode for downloading multiple videos from a list of URLs in a file.
# Must provide a file containing URL's as your argument.
# Writes files: description, thumbnail, URL link, subtitles, auto-subtitles
#
# Example usage: yt-dlp-batch ~/urls.txt
alias yt-dlp-batch='yt-dlp \
--embed-thumbnail \
--embed-metadata \
--embed-chapters \
--write-thumbnail \
--write-description \
--write-url-link \
--write-subs \
--write-auto-subs \
--sub-format srt \
--trim-filenames 248 \
--sponsorblock-mark all \
--output "%(title)s - %(channel,uploader)s (%(release_year,upload_date>%Y)s)/%(title)s - %(release_year,upload_date>%Y)s - [%(id)s].%(ext)s" \
--batch-file'

# Livestream recording.
# Writes files: thumbnail, url link, subs and auto-subs (if available).
# Also writes files: Info.json and Live Chat if available.
alias yt-dlp-livestream='yt-dlp \
--live-from-start \
--write-thumbnail \
--write-url-link \
--write-subs \
--write-auto-subs \
--write-info-json \
--sub-format srt \
--trim-filenames 248 \
--output "%(title)s - %(channel,uploader)s (%(upload_date)s)/%(title)s - (%(upload_date)s) - [%(id)s].%(ext)s"'

##############
# UTILITIES:
# Yt-dlp based tools that provide uncommon outputs.
##############

# Only download metadata, no downloading of video or audio files
# Writes files: Description, Info.json, Thumbnail, URL Link, Subtitles
# The usecase for this tool is grabbing extras for videos you already have downloaded, or to only grab metadata about a video.
alias yt-dlp-get-metadata='yt-dlp \
--skip-download \
--write-description \
--write-info-json \
--write-thumbnail \
--write-url-link \
--write-subs \
--write-auto-subs \
--sub-format srt \
--trim-filenames 248'

# Takes in a playlist URL, and generates a CSV of the data.
# Writes a CSV using a pipe { | } as a delimiter, allowing common delimiters in titles.
# Titles that contain invalid file characters are replaced.
#
# !!! IMPORTANT NOTE - THIS OPTION USES COOKIES !!!
# !!! MAKE SURE TO SPECIFY THE CORRECT BROWSER !!!
# This is required if you want to grab information from your private or unlisted playlists
# 
#
# Documents columns:
# Webpage URL, Playlist Index Number, Title, Channel/Uploader, Creators,
# Channel/Uploader URL, Release Year, Duration, Video Availability, Description, Tags
alias yt-dlp-export-playlist-info='yt-dlp \
--skip-download \
--cookies-from-browser firefox \
--ignore-errors \
--ignore-no-formats-error \
--flat-playlist \
--trim-filenames 248 \
--print-to-file "%(webpage_url)s#|%(playlist_index)05d|%(title)s|%(channel,uploader,creator)s|%(creators)s|%(channel_url,uploader_url)s|%(release_year,upload_date)s|%(duration>%H:%M:%S)s|%(availability)s|%(description)s|%(tags)s" "%(playlist_title,playlist_id)s.csv" \
--replace-in-metadata title "[\|]+" "-"'

##############
# SHORTCUTS 
# shorter forms of the above commands
# (Uncomment to activate)
##############
#alias yt-dlpgm=yt-dlp-get-metadata
#alias yt-dlpa=yt-dlp-archive
#alias yt-dlpgm=yt-dlp-get-metadata
#alias yt-dlpls=yt-dlp-livestream

##############
# Additional Usage Notes
##############
# You may pass additional arguments when using the Shortcuts or Aliases above.
# Example: You need to use Cookies for a restricted video:
#
# (Alias) + (Additional Arguments) + (Video-URL)
# yt-dlp-archive --cookies-from-browser firefox <URL>

364
 
 
The original post: /r/datahoarder by /u/AlternateWitness on 2025-07-18 14:53:32.

Xfinity launched their new tier structure, and if you signed a contract you can still switch within 45 days of signing on. I have one day left to decide.

I am currently paying $30 a month for 400Mbps and a 1.2TB data cap. I only have June’s usage to compare how much data I use in my house, which is ~900GB.

The option I am mainly considering to switch to is $40 a month, 300Mbps, but unlimited data.

I just wanted to ask how important unlimited data is to you, and if it’s worth a slowdown in speed and higher price? I may be more frivolous with my network usage, and download some more stuff if I don’t have a cap shadowing over my head, but I don’t know if that would go over my previous cap or not, so it may just be wasted money, and I only have a day left to decide.

Another note - I may have to pay for an extra month if I sign the $40 contract since it would be a month after what I planned, and I may be moving at that time. However, I am assuming it would still be a better deal than just spending an additional $25 a month to add unlimited data to my current plan.

365
 
 
The original post: /r/datahoarder by /u/redditunderground1 on 2025-07-18 13:08:30.

Book disassembly of 3144 page book for scanning - Off Topic - Cinematography.com

Scanning a 3144 page book...here is how to do it!

366
 
 
The original post: /r/datahoarder by /u/Illustrious_Heart951 on 2025-07-18 08:42:50.

Hello everyone,

I asked this question in the Terramaster community but unfortunately never received an answer.

I think this issue may also apply to other DAS manufacturers.

So, here’s my setup:

  • Terramaster D4-320
  • WD Purple HDD (WD43PURZ)
  • Beelink Mini S12 Pro running Proxmox 8.4.x

When I shut down the computer, the DAS continues running. To turn it off, I have to press and hold the power button for about 3 seconds, as stated in the official documentation.

Now, about SMART:

After every DAS shutdown, the Power-Off_Retract_Count parameter increases.

Some sources say this indicates an emergency disk shutdown, while others consider it normal for the counter to increment.

Can someone finally clarify—is this bad? If so, why would the manufacturer knowingly release a DAS that behaves this way?

Before buying, I read many reviews about the Terramaster D4-320, and no one mentioned this behavior (Power-Off_Retract_Count).

I’d really appreciate an answer.

(English is not my native language.)

367
 
 
The original post: /r/datahoarder by /u/Difficult-Scheme4536 on 2025-07-18 08:35:41.

Hi everyone,

I wanted to share something unexpected that came out of a filesystem project I've been working on, ZeroFS: https://github.com/Barre/zerofs

I built ZeroFS, an NBD + NFS server that makes S3 storage behave like a real filesystem using an LSM-tree backend. While testing it, I got curious and tried creating a ZFS pool on top of it... and it actually worked!

So now we have ZFS running on S3 object storage, complete with snapshots, compression, and all the ZFS features we know and love. The demo is here: https://asciinema.org/a/kiI01buq9wA2HbUKW8klqYTVs

This gets interesting when you consider the economics of "garbage tier" S3-compatible storage. You could theoretically run a ZFS pool on the cheapest object storage you can find - those $5-6/TB/month services, or even archive tiers if your use case can handle the latency. With ZFS compression, the effective cost drops even further.

Even better: OpenDAL support is being merged soon, which means you'll be able to create ZFS pools on top of... well, anything. OneDrive, Google Drive, Dropbox, you name it. Yes, you could pool multiple consumer accounts together into a single ZFS filesystem.

ZeroFS handles the heavy lifting of making S3 look like block storage to ZFS (through NBD), with caching and batching to deal with S3's latency.

This enables pretty fun use-cases such as Geo-Distributed ZFS :)

https://github.com/Barre/zerofs?tab=readme-ov-file#geo-distributed-storage-with-zfs

Bonus: ZFS ends up being a pretty compelling end-to-end test in the CI! https://github.com/Barre/ZeroFS/actions/runs/16341082754/job/46163622940#step:12:49

368
 
 
The original post: /r/datahoarder by /u/Worried_Claim_3063 on 2025-07-18 08:20:11.

So like, to make it short.. my friend (not me lol) is trying to download a bunch of videos off Pornhub. They just got into data hoarding stuff and have a drive setup for it.

I don't usually mess with this kind of thing cause it just seems sketchy af, but they asked me to help find an app or something that works, cause most of the sites they found just seem full of popups or malware traps. I'm honestly kinda stuck now cause there's like a million tools out there and no clue which are actually safe.

They use a Mac btw, and I tried showing them yt-dlp but it just confused them, so unless theres an easier way, Id have to set it up for them. Anyone got recs for something safer and not a virus pit?

369
 
 
The original post: /r/datahoarder by /u/yesiwonagain on 2025-07-18 05:43:41.

im looking to archive some smaller streaming platforms (eternal family) and wondering if theres any way to automate this. my usual way to download from these is to use ytmp3 on the m3u8 files for each episode/movie. wondering if there would be any way to make it faster since i need to start playing each episode before i can get a link to download. would there be any way to script this or any apps i could use to automate it?

370
 
 
The original post: /r/datahoarder by /u/Kennyw88 on 2025-07-18 04:51:00.
371
 
 
The original post: /r/datahoarder by /u/PrivacyPolicyRead on 2025-07-17 23:09:40.

Hey DataHoarders,

I built a small linux CLI tool in Python called remap-badblocks. It scans a block device for bad sectors and creates a device-mapper that skips them. It also reserves extra space to remap future badblocks dynamically.

Useful if you want to keep using slightly-damaged drives without dealing with manual remapping.

Check it out:

Would love feedback, bug reports, contributions, help shaping the roadmap or even rethinking everything all over again!

372
 
 
The original post: /r/datahoarder by /u/PubicPlant on 2025-07-17 21:12:21.

I really like the idea of having a fast thumb drive sized external SSD (I use my laptop on the couch a lot and have accidentally unplugged my cable multiple times) but all of the 2230 NVME small enclosures I’ve found max out around 1 Gb/s.

Why are they so much slower compared to larger enclosures? And are there any small enclosures (preferably with built-in male port) that are faster than 1 Gb/s?

373
 
 
The original post: /r/datahoarder by /u/Sushi-And-The-Beast on 2025-07-17 19:56:56.

I need to convert 2GBs of EML/MSG emails to PDF.

I have Stirling PDF Setup and Paperless but they dont support EML and MSG files.

I have the PST files as well.

Any ideas?

374
 
 
The original post: /r/datahoarder by /u/TheSpacePope42 on 2025-07-17 19:30:56.

Ok heres the situation.

I presently have a 70tb data load and growing.

Presently I am running a PCIE RAID controller and 8 10tb platter drives.

As you can imagine building and growing this array from machine to machine is getting tricky.

I am looking for an external DAS solution that can be more portable, especially as I am staring down the barrel of needing to make some upgrades as Win11 refuses to allow for the use of the 1st gen threadripper chips.

My limitations are as follows:

Needs hardware RAID ( I simply dont like software raid and this complicates portability)

Must register to the OS as an internal or external hard drive and not as a Network Drive or USB Flash drive. (limitation of my backup software/service)

Must run on desktop Windows OS, not server (another limitation of the backup product)

I have been looking at the Qnap TL-D800C-US which appears to be USB connected and the TL-D1600S-US which looks to utilize a proprietary controller card and should register as an internal drive by way of the PCIE backplane.

What Qnap is not clear on is the hardware raid of these units.

Can someone clarify or perhaps point me in the direction of something that can fill these needs under the $1200 price point, as I will still need to spend some serious change on disks

375
 
 
The original post: /r/datahoarder by /u/Fractal-Infinity on 2025-07-17 18:24:24.

I consider a WD Red Plus 4TB to be pretty quiet, so that's the baseline. I bought the 8TB version and it was significantly louder and annoying, so I relegated it as a backup drive. I don't really care about the performance, since it will be mostly idle and used to store media files. I'd prefer either WD or Seagate. I'm using it for a desktop PC (inside its case), so that's why it needs to be quiet.

Edit: thoughts on this WD Blue 8TB CMR (WD80EAAZ)?

view more: ‹ prev next ›