It's A Digital Disease!

The original post: /r/datahoarder by /u/ThrowRA5566787 on 2025-01-27 15:16:28.

I'm currently in the process of moving my entire photos library of like 100k photos and video from my MacBook and iPad to an external drive. It's way overdue and not backed up 😨 so I have dual use identical drives that will be a clone of one another storing my photos app

My question is, this process is long and cumbersome since it just won't do it in large sweeps I have to manually import about 1k photos at a time.

Can I make it so that the drives are essentially cloning each other in real time. As I add photos to one it also will be updating it on the second Drive as well?

The original post: /r/datahoarder by /u/Creepy_Finish1497 on 2025-01-27 15:08:31.

Does anyone know what type of WD drives are in the Easystore? I have an 8-bay NAS populated with 4 20TB Red Pro drives. These suckers are not cheap so I'm contemplating buying the Easystore and shucking.

The original post: /r/datahoarder by /u/TopdeckTom on 2025-01-27 14:46:10.

As the title says I just purchased a 10 TB Exos HDD. This is my first recertified drive, the format will be Ext 4 for my Ubuntu server. What is the first thing I should do with it? I once found a post that said what to do when you receive a used hard drive but can no longer find it. I know there is some kind of test you can run to test for potential failure.

The original post: /r/datahoarder by /u/ClaasChopper on 2025-01-27 13:53:30.

I have at least 6 8TB drives that have less than 20 bad sectors that were pulled out of a work environment. Does it make sense to use these for anything? Could put 4 of them in a NAS for a backup of a backup in a 3rd physical location or are these likely to fail?

The original post: /r/datahoarder by /u/Garry-Love on 2025-01-27 13:18:49.

So the company I work for is currently using a glorified SharePoint page to access their knowledgebase. The data consists of training videos, pdfs, manuals and other stuff developed in house for information purposes. Searching it is a nightmare and with a recent acquisition of a new site there's budget there to revamp this. The data is currently sored exclusively on the cloud with no centralized local backup. This is something I'd like to change, I think we should be putting this data on one of the servers we have here already.

• I'd like a web-interface for it that allows a user to search by title.
• Filter by document type (pdf, docx, xml, json).
• Filter by date
• A form submission that will allow a logged in user to upload files
• A tagging system so you can filter by which department the document is for (software, sales, electrical, mechanical, etc)
• A revision or version control system that will default to the latest version when looked up but still give the user access to previous versions via a dropdown or something

Do any of you use something like this? Do you have suggestions on where else I can look for more products or information?

The original post: /r/datahoarder by /u/CeFurkan on 2025-01-27 12:15:49.

The original post: /r/datahoarder by /u/MoistCarpenter on 2025-01-27 04:20:08.

Anyone care to speculate on any risks of these types of cloud services over the long-term(10+ years)?

edit: I'm referring to things outside of conventional storage risks, like a cloud provider storing something on tape drives, then messing things up somehow. I assume these services are run on tape(?) so I guess my question is not if there are any risks with storage practices, but if these tapes can hold up? For example, do they do tape backups etc...

The original post: /r/datahoarder by /u/acephalebokeh on 2025-01-27 04:19:44.

Looking for recommendations as per the title. I've realized that a show I've been downloading is basically taking up a tenth of my laptop's hard drive. Which external drive would be best for storing films? I know very little about this...

The original post: /r/datahoarder by /u/zzswol on 2025-01-27 00:52:30.

The open-source AI community is releasing powerful models. Things are moving fast. You might not have the hardware, expertise, or attention to take proper advantage of them in the moment. Many people are in this position. The future is uncertain. I believe it is important to preserve the moment. Maybe we get AGI and It becomes ashamed of its infantile forms, user AI becomes illegal, etc (humor me).

What appears to be lacking: distributions mechanisms privileging archival.

I don't know what's going on, but I want to download stuff. What training data should I download? Validation data? Which models do I download? Which quantizations? In the future, to understand the present moment, we will want all of it. How do we support this?

I am imagining a place people of all sorts can go to find various distributions prepared:

prepper package: (high storage, low compute) - save all "small" models, distillations, etc

tech enthusiast package: (medium storage, medium compute) - save all major base models with scripts to reproduce published quantizations, fine-tunes, etc? [An archeologist will want closest access to what was commonly deployed at any given time]

rich guy package: (high storage, high compute) - no work needed here? just download ~everything~

alien archeologist package: ("minimal" storage, high compute) - a complete, non-redundant set of training data and source code for all pipelines? something a particularly dedicated and resourceful person might choose to laser etch into a giant artificial diamond and launch into space

Does this exist already?

The original post: /r/datahoarder by /u/Big_Nefariousness647 on 2025-01-27 00:18:06.

i found a user shareing vcd and tv rips on okru how can i bulk download them

The original post: /r/datahoarder by /u/EnsilZah on 2025-01-27 08:52:09.

Hey, I'd like some advice on how to best move my data while relocating to a new country (Portugal in case that matters). I have around 17TB of data currently striped over 4 HDDs (3x10TB + 8TB) (+ SSD for the OS) on a Windows Storage Space pool. I figured the whole computer would be too bulky to take with me.

The options I'm considering:

• Just take the drives in protective cases in my carry-on and build a new computer to put them in at the destination (Will Windows / Storage Spaces be ok with the change of hardware?)
• I was thinking of taking the opportunity to ditch HDDs altogether and going full SSD, which would make things a bit less bulky and sensitive to damage, but looks like SSDs are a bit expensive at the moment (Looking at SAMSUNG 870 QVO 8TB at $560 not including import duties, and I'd need 4 of them)
• I currently have an 5TB IDrive cloud backup account where I back up only the critical stuff, I could upgrade to 20TB for an extra $150, store everything there and then download or use their restore drive shipping when I'm at the destination.
• Maybe just leave the file server here with someone I trust and then download from it to a new one when I arrive (What software would I use for that?)

So, would appreciate comments and suggestions.

The original post: /r/datahoarder by /u/Novaa_49 on 2025-01-27 07:29:50.

I saw that iPad Pro models do recognise NVME SSD so I decided to get lexar e300 as enclosure for Kingston kc3000. But when I tried connecting to it it doesn’t work. Although I thought would tried it on MacBook and doesn’t work again so it got to be with the setup issue I think?

Or is there something im missing? Could it be compatibility issue? The enclosure product is stated to be compatible with m.2 NVME SSDs but it just doesn’t work..

The original post: /r/datahoarder by /u/fogrampercot on 2025-01-27 06:44:40.

We can see the activities of an user in a Facebook public group with the scheme - https://www.facebook.com/groups/<group\_id>/user/<user\_id>

I want to archive the contents of this entire page. To view the content, all someone needs to do is log in to Facebook and access the URL. Since it's a public group, the content is public. I tried using ghostarchive, but that doesn't work since it requires being logged in to Facebook to view it.

I am looking for a reliable way to archive the page. Screenshots or screen scrapers won't work since my goal is to archive the content of that page as evidence for some malicious activity. What would be the way to do this?

Thanks!

The original post: /r/datahoarder by /u/cactul on 2025-01-27 05:18:21.

Hi, I know this is a common question but I have minimal skills and want to download a bunch of pages from real estate listings as a data base for home design ideas.

Can any one please assist with the most straight forward way to do this as so far I cant work out how to get it to work properly.

Thank you.

The original post: /r/datahoarder by /u/ukyorulz on 2025-01-27 04:46:44.

I was telling a friend that I was thinking to improve my data storage solution and he gave me his used DS218j for free. This is an older model and because it only has two bays, I would prefer to put a bigger hdd in it.

Looking at the synology site's compatibility list I can see that the biggest listed drives are 16TB and the selection is quite limited.

I am not too familiar with NAS systems but often laptop compatibility specs represent only what was tested with the model and a higher spec part could be installed and work just fine. Is the situation similar for NAS or should I limit myself to only those exact models listed as compatible?

I'm waiting for hdds to go on sale and it'd expand my options if I could put a 16TB WD or even Toshiba in there. Actually is there any reason a 20TB drive would be incompatible?

The original post: /r/datahoarder by /u/KENZOKHAOS on 2025-01-27 03:28:06.

The original post: /r/datahoarder by /u/InsaneProxy on 2025-01-27 03:00:33.

I collect artwork from various artists, biggest collection I have is from Pixiv. Over 2k subfolders with thousands of images in total. I'd like to find a program or something that would allow me to add tags to these images so that I can search for certain ones at some point.

I'm familiar with the program Allusion, but that is meant to categorize art references and was never meant to catalog the tens of thousands of images that I have. I've tried and it bugs out every single time.

So I was wondering if anyone had suggestions? Thanks.

The original post: /r/datahoarder by /u/Senkyou on 2025-01-27 01:54:36.

I found a handful of old CDs including stuff like Civilization I, Rainbow Six Siege: Vegas, and Warcraft 3 for MacOS among some others in an old box. I was curious what the best way to copy these into a digital format would be. I have access to Windows 10, MacOS (second to latest version, can't remember the naming/number), and Linux.

I'd love to archive and maybe even occasionally use these as they're games I played with my dad when I was a kid and I'm sure these disks were originally his.

Appreciate any advice.

The original post: /r/datahoarder by /u/evanthedrago on 2025-01-27 01:30:33.

I have a hard case and want to store my harddrives (about 12 of them for backup) for easy storage and carry. Where can I get thick anti-static foam?

The original post: /r/datahoarder by /u/Annoyingly-Petulant on 2025-01-27 01:13:23.

I’m wanting to download an entire website that uses user name and password with wget

Will this work? wget -nc —wait=300 —random-wait —http-user=user —http-password=password http://www.website.com/

The original post: /r/datahoarder by /u/zzbackguy on 2025-01-27 00:49:07.

I’m building a new computer for personal use and gaming / data storage, and most cases don’t have dedicated drive racks or even decent space for them. I was eyeing the Montech king 95 case because of its dual chamber setup, but despite the huge amount of space it has, it only has 2 dedicated hdd bays. To add more you have to remove fans and buy brackets.

Are there any dual chamber cases that have at least 6 bays? This isn’t a dealbreaker, I just wanted a nice looking case. My backup is to get a Darkrock Classico max since it includes 8 drive bags out of the box. (Does anyone know the difference between the Classico model and the max model?) Any advice for cases would be greatly appreciated.

The original post: /r/datahoarder by /u/hai10 on 2025-01-27 00:45:06.

If there are LEGO fans on this subreddit, some of you probably know Brickshelf, a classic website that since 1998 has hosted various LEGO-related images (and some other formats): people's creations, LEGOLAND trip photos, instructions, forum banners and avatars, and what not. Obviously an important piece of early 2000s web and real digital artifact.

Sadly, as Brickshelf's creator Kevin M Loch has passed away (in fact, happened in 2024), the Brickshelf homepage now says that the site will be shut down on March 1. A month is left, so I summon all the hoarders and archivists able to save the day. I could help but I've got only 500GB of free space left on my hard drive.

The structure: Brickshelf is an old school website consisting of just ~5 million files (mostly photos) + approx. the same amount of photo previews, and a total of ~5.5 million html pages (folders, subfolders and individual file pages) which host these files, so it's all pretty manageable I guess.

Since Kevin Loch was an avid webmaster and had other projects, it would be great to back up not only Brickshelf but all other Kevin's sites too. Here's the links I was able to find:

https://kevinloch.com/

https://www.n3kl.org/

https://bsrender.io/

https://nensus.com/

The legacy should live on!

The original post: /r/datahoarder by /u/Jazzlike_Hat9693 on 2025-01-27 00:33:34.

Hi friends and pros

I'm researching storage solutions and it led me to believe a DAS or NAS is what I need. I want to have around 12 TB with 1:1 redundancy.

What's the difference between something like this: https://a.co/d/4GvADDU

And something like this: https://a.co/d/0oUs1aV

Why is the price so different? Apologize for the stupid questions. I'm pretty new to this and thanks in advance

The original post: /r/datahoarder by /u/srizvi1 on 2025-01-26 23:43:27.

Just wanted to share my experience with the SABRENT 2.5 Inch SATA to USB 3.0 Free External Hard Drive Enclosure with the Samsung 8TB QVO 870 SSD. I'm not sure what's going on but I can't get the SSD to work properly with this enclosure. The good news is I am able to get it working with my other enclosure, a UGREEN USB C 3.1 Gen 2. That one I've been using with my Ugreen enclosure. But I still thought I'd share my experience - here's the play by play:

First - a pic of everything - my Windows Samsung PC, the new 8TB SSD, the Sabrent enclosure, the new Ugreen USB-C to USB-A cable, and my Ugeen Enclosure w/ my older EVO 870 4TB SSD inside.

Samsung QVO 870 SSD 8T, Sabrent USB 3.0 Enclosure, UGreen cable, and old Samsung EVO 4TB SSD inside Ugreen enclosure with Samsung Magician Running

Not pictured is me trying to get the Sabrent initialized using the Ugreen USB-C to USB-A cable, but my MacBook Pro M2 wouldn't read it. It was when I connected a USB-A adapter to the machine and used the OEM Sabrent cable that the computer read it:

Sabrent USB 3.0 Enclosure with Samsung SSD Q70 8TB turns on and mounts when connected via OEM USB-A Sabrent cable (and MacBook USB-A port adapter).jpg

However, here's the Disk Utility not being able to initialize it:

Samsung 870 QVO SSD wouldn't initialize in Sabrent USB 3.0 Enclosure.jpg

And here's First Aid failing too:

Samsung 870 QVO SSD first aid wouldn't run in Sabrent USB 3.0 Enclosure

And then when I pull the 870 QVO and put it in my older Ugreen enclosure (which normally houses my EVO 870 4TB SSD), then I can initialize fine:

Samsung SSD 870 QVO 8TB intializing fine in Ugreen enclosure

So my plan now is to return the Sabrent and go with the UGreen. I'd like to try a different, newer enclosure just so I don't have the same exact enclosures and can easily tell the difference. But if this doesn't work, then I'll go with the older Ugreen enclosure.

Hope this helps someone!

Last question, before I use the new 870 QVO 8TB SSD, should I do any sort of firmware update? I thought Samsung Magician would do that but I can't get this application working properly with neither my M2 MacBook Pro nor my Samsung Windows PC. I guess I could maybe try to download the ISO files from the site and try to install with the Windows PC but I was hoping Samsung Magician could handle it (if needed)

Samsung SSD available Firmware for EVO and QVO 20250125

The original post: /r/datahoarder by /u/SteviesBasement on 2025-01-26 23:03:37.

Over time i ended up with a number of files which have old containers/codecs and wonder if i should encode them to H264/H265 or AV1 in a .mp4 container? e.g. avi, mov, wmv, ...

Did some testing with Handbreak the results seem promising but for old formats the output are often larger than the input, especially unsure i'm about the bitrate (constant/variable) and what CF to use, since the files vary widely in what they are.

Looking for suggestions on what settings to use or if the entire thing is pointless anyways.

Note most old files are not 4k ,but 380, 720, ... and file-size isn't super important.

Playing around with encoders to see how long it takes, i assume time difference will be the same for a given length even if quality is different.

Used settings: Preset "H.265 NVENC 1080p (modified)", web optimized, AAC, Constant Framerate, CQ/RF 22, encoder as below

Source: 4k 13GB 48.8MB/s H264 mp4, quality seems similar(?)

| Codec                 | Compressed Size | Time   | Compression (%) | Bitrate        |
|-----------------------|-----------------|--------|-----------------|----------------|
| AV1 (NVEnc)(GPU)           | 5.11GB          | ~8min  | 60.69%          | 19.1 Mb/s      |
| H265 (NVEnc)(GPU)          | 4.05GB          | ~8min  | 68.85%          | 15 Mb/s        |
| AV1 (SVT)(CPU)             | 2.75GB          | ~24min | 78.85%          | 10.2 Mb/s      |
| H265 (H265)(CPU)           | 1.27GB          | ~28min | 90.38%          | 4618 kb/s      |