It's A Digital Disease!

The original post: /r/datahoarder by /u/falco_iii on 2024-06-21 15:36:55.

The original post: /r/datahoarder by /u/RXrenesis8 on 2024-06-21 14:55:19.

Howdy r/DataHoarder!

I am still a little baby hoarder with only about 15-20TB of data floating around my noisy 10 year old desktop-turned-HTPC. I was looking to expand to ~30TB (with room to grow of course) for a couple of initiatives and initially had been set on a rack and rackmount server or NAS chassis with a bunch of 3.5" bays for price-efficient spinning drives.

The cost though! Racks = $$$ Rackmount Chassis or pre-builts = $$$$!

Every penny I would save in drive capacity I would pay back in getting a nice (home-office) rack and NAS.

So I've come to you questioning a pivot point: What do you think about smaller solutions? Specifically I am thinking about a Mini-PC running nnRAID connected to a JBOD enclosure for disks or large/cheap SSDs.

What's the performance difference between a setup like the above and a low-end rackmount setup (which is at the limit of what I could realistically afford)?

Any pitfalls I should be aware of?

Thanks in advance!

The original post: /r/datahoarder by /u/LeichenExpress on 2024-06-21 14:11:25.

The original post: /r/datahoarder by /u/Vietname on 2024-06-21 14:01:51.

I have a couple of small synology NAS's, each has a dedicated drive for regular backups, but i wanted to get a small external drive (ideally SSD) to put my really important data on for emergencies, e.g. the house is on fire and i can only grab a few things.

What would you recommend?

The original post: /r/datahoarder by /u/thenovum on 2024-06-21 11:43:33.

Hey. Is it possible to extract the data from via raid disk members without using a VIA RAID controler? Disks are from 2004.

The original post: /r/datahoarder by /u/Warcraft_Fan on 2024-06-21 10:20:14.

The largest hard drive is 30TB but if they would use 5.25" drive then it'd be possible to get some hundred TB in a standard 5.25" half height (same as CD and DVD drive)

Computer cases are still produced with 5.25" support as some people still need optical drive or possibly tape drive. So what's keeping them from putting out massive 5.25" hard drives?

The original post: /r/datahoarder by /u/LaundryMan2008 on 2024-06-21 08:42:20.

The original post: /r/datahoarder by /u/CreativeDog2024 on 2024-06-21 07:59:32.

I know about the 3-2-1 rule, but I've not been implementing it. I'm a college student with 3 years and 4 TB worth of movies/TV shows that have taken DAYS to download at times.

I have 2 local backups, one on a segate 5tb and the other on a WD 5tb.

I don't want to lose these files, I've used a plex add on to export posters and a list of my media but if I were to ever try and get them back from scratch, it would take a long time and I'm not sure I'd ever dedicated that much time.

I know about aws deep glacier as a last resort and it being pretty high cost to recover, but I don't really plan to ever recover unless I both of my local backups.

• What should I know about?
• What would you do?

The original post: /r/datahoarder by /u/Aggressive_Green_764 on 2024-06-21 07:22:04.

Hello guys i don't gonna have internet for a while and i thought about downloading the math library and i don't wanna do it manually and i wanna do it just like it's on their website not the playlists on youtube

Is there any way?

The original post: /r/datahoarder by /u/SirLouen on 2024-06-21 06:46:34.

I was reading this post commenting on the differences between SMR and CRM disks.

https://www.reddit.com/r/HomeServer/comments/hhfoqg/should_i_get_seagate_barracuda_or_seagate/

https://www.youtube.com/watch?v=aztTf2gI55k

Basically, suggesting that SMR are seriously under classic CRM. But this happened to be like 4 years ago, so I wondered if Seagate has been able to figure out something to improve this for their disk, which is unlikely, but just wondering

I saw today in my brick and mortar store, a Barracuda Compute 8 TB at $130 and a Barracuda Ironwolf at $190 w/ 8 TB also, so $60 difference for the same storage, not sure if its worth but feels that knowing all the issues attached with SMR (specially for a NAS where it's going to be used these disks), doesn't seem to be a good decision.

So I wanted to know your opinion.

The original post: /r/datahoarder by /u/Industrious_Badger on 2024-06-21 05:48:09.

The original post: /r/datahoarder by /u/The_Bipolar_Guy on 2024-06-21 05:46:15.

Building my first DIY NAS. Need 2+2 tb to start off with. Trying to be as cheap as possible. Will be storing all memories and irreplaceable things. Will get an external HDD for offsite backup. Also, I do not care about read write speeds. I just want my data to be safe and backed up at the end of the day for as cheap as possible, I will add new data maybe once a month if not even more rarely. I will be using my old motherboard (6 SATA ports) and processor. Will get a new 500 GB ssd for OS and new RAM.

Now here is the dilemma. I am from India and WD Purples are selling for 5250 INR each, cheapest. Next up, WD Blue is also selling for 6000+ INR. I can technically afford (but want to avoid at all costs) to get a 10TB UltraStar or EXOS for 22000 INR but I need two physical drives at least and my total storage need for backup right now or even 3 years later will not exceed 4-5TB. so a 10TB HDD will be useless, not to mention spending 45K INR on drives only (which is VERY VERY EXPENSIVE FOR ME).

I know WD Purple glosses over write errors and can be bad. Chucking is equally expensive, if not more. Should I just get 2x 2tb WD Purple for 10.5K INR and set up my backup server for now. Will add more HDDs later as needed (1-4 years later). Can I do anything to prevent the write errors?

Also, if there is a cheaper way, especially in the long run, I would appreciate it. Currently I have ~2TB data which grows by like 200-400GB a year. I have thought of BluRay drives but I am unable to procure new BluRay drives cheap enough.

The original post: /r/datahoarder by /u/leviathan2701 on 2024-06-21 05:34:42.

Hello,

I just set up a NAS using Ubuntu 24 LTS, mergerFS & snapRAID. I am new to using Ubuntu, mergerFS & snapRAID but I did follow some guides via YouTube videos on setting them up. The drive pool has been created and snapRAID is working as expected however, I have what may be an easy question that I can't quite figure out.

Following the guide, the mount point for mergerFS pool is at /mnt/storage which I have no issues accessing and adding/removing files via terminal using sudo or as root however, since the owner of the /mnt/storage location is root, my user account on the machine cannot write files to /mnt/storage and I cannot share it via samba. I know changing the overall permissions to 777 to allow it is risky and not worth it and I'm sure there is another way that I should be writing files to the location without logging in as root but again, I'm new with this and still learning. Any advice/help will be appreciated.

mergerFS v2.33.5

The original post: /r/datahoarder by /u/Flat-One-7577 on 2024-06-19 22:41:21.

Hi,

I am going to build a 16 disk data storage. This will mainly be a "backup" to store data for some time and not need to recreate it again. Recreation would be expensive.

I have few very big files and a lot of small one.

Main focus is having a lot of storage. I/O is not so important.

Having 16x 22TB disks attached to a MegaRAID 9500 Controller I am asking myself what would be the prefered setup?

Variant A:

• HW Raid60
• on top ZFS

Variant B:

• JBOD Mode on Controller for songle Disks
• ZFS

-- create 2 RaidZ2 vdevs (6+2) and stripe them

Variant C:

• JBOD Mode on Controller for songle Disks
• ZFS

-- create one RaidZ3 vdev

What would you do?

Regards,

Joachim

The original post: /r/datahoarder by /u/sylinen on 2024-06-21 02:49:45.

The original post: /r/datahoarder by /u/Big-Durian-5011 on 2024-06-21 00:44:41.

Hey guys, I have a collection of about 8 TB of anime on my OS od choice, Arch Linux, and was wondering if there was a way I could automatically (manually is fine too) compress all content in a singular folder and store the compresses/ zipper files in another folder.

Edit: Just to be clear, the reason I want to zip/compress my anime is to back it up, not to download more anime. My plan is to:

1. Make a copy of my folder of anime
2. Zip/compress the copied folder
3. Store the zipped/compressed folder on a seperate storage device (currently being stored in a 14TB hdd).
4. If possible, set up an automatic process to continously do this process for any new anime I download.

The original post: /r/datahoarder by /u/immranderson on 2024-06-21 00:01:49.

Original Title: Looking for help as a vlogging nomad. I film all content in 4k60p and visiting every country in the world. Guesstimating I'll need an additional ~24TB of storage for my remaining 100 countries. Budget: hopefully <$10k?

Hey! I've been living exclusively out of my backpack + carryon luggage for the last decade and have been filming my travels as I go along. I've been traveling around the world and have amassed a large collection of 2/4TB Samsung T-Series SSDs which is beginning to bump up against the physical carrying capacity in my bags.

Lucky enough I've got a 128GB Pixel 1 that I use still to this day for daily remote storage + backup, so that has served me well.

However, I also like to have local backups, and have been trying to shop around for SSDs. I'm trying to keep things as compact as possible, and I'm traveling so have avoided going the HDD route since my stuff can get pretty rough and tumbled quite often. I try not to edit directly off my external drives, and I've got an 8TB M-class macbook pro that I've been lugging around with me around the world.

I'm trying to figure out how to handle my long term storage needs for the next few years of travel. On average, I'm capturing 100GB - 400GB per week of travel. I'm figuring I've got another 100 countries left to visit around the world, so 100 weeks * ~200GB per country maybe lands me in the ~20TB range of storage I'm anticipating I'll need for the remainder of my travels?

Does anyone have any suggestions here? Could I get away with getting 2 x 16TB SSDs with external enclosures and that 32TB worth of capacity would generously cover my needs with wiggle room? If so, what SSD + enclosure combo would you recommend?

The original post: /r/datahoarder by /u/Sliced_Apples on 2024-06-20 23:37:08.

Hello, I am working for a startup in the sports industry and we have recently come into the possession of about five hundred thousand cds with 20 year old sports footage.

We are trying to train an AI model off of them so as such, they need to be digitized.

I know a little about burning cds but not much. As I have been made aware, this would be “ripping” and not burning.

What would be the best way to go about doing this? What storage solution would be the best? Any advice is greatly appreciated. I’m happy to answer any questions as well.

The original post: /r/datahoarder by /u/TripleXero on 2024-06-20 22:17:31.

I've spent a stupid amount of time backing up all my DVDs and Blu-rays but I've already capped off an 11TB hard drive and split the extras between drives meant for other stuff and I'm still not done. I don't plan on getting rid of the original physical copies but it's much easier to access them digitally.

I should also mention what I have backed up has been shrunk down already to decent looking MP4s to shove on iTunes for sleeker organization but I just don't know if it's worth keeping the original 1:1 files after. I shrunk the original files down once and realized after the fact that it wasn't in a flexible or presentable way for some so I'm hesitant to try again or wipe them completely after all the work. Is there maybe a good way to compress them? I've already went through and removed all non-English audio and subtitles

The original post: /r/datahoarder by /u/Chippomannen on 2024-06-20 21:16:08.

The original post: /r/datahoarder by /u/Dismal_Award735 on 2024-06-20 20:57:37.

The original post: /r/datahoarder by /u/_izix on 2024-06-20 20:12:52.

Has anyone been blocked or banned from bulk downloading from pixiv?

I've seen rumors that it may happen, but as far as I've been able to determine, there is no evidence of it actually happening. Is there maybe a hidden limit that's very high?

I plan to download all images from users I follow with a script to update my archive with any new images every couple days. So far I have downloaded a few thousand images with no issues. Just figured I'd see if anyone has evidence of that happening before I continue.

EDIT: I am using gallery-dl

The original post: /r/datahoarder by /u/wtf94ftw on 2024-06-20 19:23:47.

Hey!

So long story short my dad just retired and he wants to digitize his lifelong collection of hi8 tapes.

I’ve read a lot of things last few days but I’m still not sure what would be the best option (budget friendly…) for the setup.

Currently he’s been watching those tapes on TV with the sony CCD TRV78E camera, connected with both s-video and RCAs with an HDMI converter (https://amzn.eu/d/05vz9SBs) and everything’s working ok…

I’m not great at these matters and I’m a bit lost tbh, but my initial idea was to plug the HDMI to a computer and record the screen as the movies are playing with a video capture sofware… which I now doubt to be an option because HDMI port on pcs don’t receive signal, is this right?

Idk if the best option would be getting a camera that has the FireWire port and plays hi8 tapes or if it would be better to do this any other way, but I’ve read about those usb things not getting so much quality and I’m worried about it.

Right now movies are playing in a 55” TV with good image quality and audio is perfectly synchronized, if this is relevant info in any way.

He’s really motivated to do this by himself as he now have a lot of free time, but ofc this means that I’ll have to learn how to do it and teach him so he can continue by himself.

Sorry, this was a long story after all, and sorry for some eventual english mistake as well!

The original post: /r/datahoarder by /u/PrivateAd990 on 2024-06-20 18:01:15.

What software is available (paid or free) to help analyze my data on an external HD? it's only about a 1GB but 20+ backups (manually copied files over the years to this HD). MacOS or Linux. Wants:

• find data by extension (file type)
• find largest files
• identifying duplicates and handling it manually

Accepting other tips of how to sift through data. I plan to organize all data to one folder rather than 20 backup folders.

The original post: /r/datahoarder by /u/Septarchy on 2024-06-20 17:50:56.

I need to extend my current server capacity because I'm running out of space.

I used to buy the WD my books, which where kinda cheap (180 euro/16tb) comparing to hdds and shucked them (with the 3.3v pin taped).

Is this still the way to get cheap storage or are there better alternatives available in europe?