It's A Digital Disease!

The original post: /r/datahoarder by /u/No-Kick7885 on 2024-08-14 18:39:20.

Hey guys,

Im using an M3 Max 14 Inch and a 2021 M1 16 Inch. I have around 1.6 TB of files for soundpacks and plugins that i use for music and i have another 1.5 TB of 4K Videos that i want to keep.

Help me out here! Also, bonus points if you can recommend the best SSD with the fastest reading speed and the best reliability. Im currently looking at the Crucial X10 Pro.

P.s. I am mainly using the NAS and/or the SSD to work on music production via Ableton Live (A Digital Audio Workstation) and I just need to make sure longevity and speed wouldn't be a problem!

The original post: /r/datahoarder by /u/adjoro on 2024-08-14 18:32:10.

I'm trying to help people who are new to data storage gain perspective about common questions that don't have easy answers (i.e., "what's the shelf life of my thumb drive?"). Many have't pickup up the technical vocabulary yet, so I'm looking for ways to make it relatable while still being grounded in reliable data. When I search for studies, there's been research at LOC and by the Canadian government about optical storage media like CDs. I've seen folks here point to Backblaze's data for HDDs. I'd love to find more examples like that which are academic or from industry leaders. The internet is full of infographics and general advice/rules of thumb, from LP to LTO to SSD, but I'm unclear on a lot of the sourcing. Reliable research needs to account for use/conditions, which I'm thinking accounts for the lack of one "authoritative" source given all the variables. Any suggestions/ideas/leads appreciated!

The original post: /r/datahoarder by /u/Falcons-Fury on 2024-08-14 17:53:18.

I am looking to make a small RAID using ZFS (first time) and a friend was commenting about CMR vs SMR drives and how it could be problematic with ZFS. What recommendations do you all have for this and general RAID configurations for data?

I am thinking for now of a Raid 5 with 3 8TB hard drives to start and maybe expanding in the future.

EDIT: Note the data will be almost extensively written once and read often.

The original post: /r/datahoarder by /u/massive_toe55 on 2024-08-14 17:40:18.

Haven't looked it up in detail but the arr suite is basically about torrenting media, right?

What are you doing to prevent getting caught? VPN?

The original post: /r/datahoarder by /u/Racsor1998 on 2024-08-14 17:20:52.

Could someone help me trying to find out if this works?

I have been trying to do some research, but I am very new to both NAS building and SAS.

I was thinking of buying Jonsbo N3 and LSI 9300-8i HBA. I recently got 8x8tb SAS drives for free.

Where I get unsure about is it possible to use SAS(sff-8643) to sata cable to connect to the backplane in the case.

Or do I have to get the backplane out and and use SAS(sff-8643) to SAS(sff-8642) cable?

The original post: /r/datahoarder by /u/Lazy_Fortune_9409 on 2024-08-14 15:36:29.

Do you guys backup movies in your media servers? As they already take a bunch of space on your disks, is a complete backup an overkill?

The original post: /r/datahoarder by /u/KalybB on 2024-08-14 14:29:38.

With the recent death of Game Informer to the hands of GameStop are there any projects to preserve the last few decades worth of issues? If so I’d love to contribute

The original post: /r/datahoarder by /u/ericlindellnyc on 2024-08-14 14:16:59.

I'd like to use rmlint to deduplicate a massive, nested folder. I'd like duplicates chosen from originals so as to leave intact as many folders as possible.

Let’s say I have folder A and folder B. In A I have file1 and file2. In B in have file1 and file2.

If rmlint selects file1 as a duplicate from folder A, then I'd like it to select file2 as a duplicate from folder A also -- so folder B remains intact. Otherwise, I’ll be left with folder A with one file and folder B with one file.

Regarding rmlint duplicate selection criteria, folder depth is relevant. If folder A and B are at two different levels, then rmlint would take duplicates from the same folder. Otherwise, which selection criteria to use?

The original post: /r/datahoarder by /u/asdfghqwertz1 on 2024-08-14 12:02:38.

I have 2 sata SSDs and 2 HDDs right now, and I had to get a bit creative to fit all of them because it has 3 drive bays. I'm planning to buy more hard drives in the future, so I'm going to need something good, preferably under 120 euros, but it can be exceeded if it's very good quality.

The original post: /r/datahoarder by /u/roogie15 on 2024-08-14 11:32:21.

Hello guys,

I was wondering if you guys could help me out with a problem I've been having. Recently I ordered 2x 12TB refurbished Seagate HDD's (inb4 Seagate). Now whenever I plug these in my homeserver wont boot past the manufacturer boot screen. However when I remove them it boots fine. It also boots fine when I reattach the 4TB WD disk I tried to replace.

I know some disks require you to tape over the 3.3v pin which I have also done it still doesnt boot properly. Can anyone tell me what I'm missing here.

Specs:

Asrock Intel 5005J

8GB RAM

3x 12TB WD white label

2x 10TB HGST

2x 4TB HGST/WD (which I'm trying to replace with 2x 12TB)

550W Seasonic PSU

The new drives are Seagate Enterprise 12TB ST 12000NM0127 btw.

The original post: /r/datahoarder by /u/One_Ostrich7868 on 2024-08-14 11:15:09.

Is there a way to do this? When i try to do it i only get a text file named after the post, but the video has random numbers on it.

Here is the conf file i'm using.{

"extractor": {

"twitter": {

"postprocessors": ["content"]

},

"deviantart": {

"postprocessors": ["content"]

}

},

"postprocessor": {

"content": {

"name": "metadata",

"event": "post",

"filename": "{user[description]}_{date}.txt",

"mode": "custom",

"format": "{content}"

}

}

}

and the command i'm using with cmd.

gallery-dl -C cookies.txt --config twitter.conf "LINK"

The original post: /r/datahoarder by /u/Nagol68 on 2024-08-14 01:06:19.

Hey y'all,

Looking for a simple script or software that runs on a folder of photos to optimize and compress them without lowering resolution.

I have a several thousand photos which I don't care all that much about, but would like to hang on to. A lot of them are 5MB+ each, which can easily be lowed to ~1MB by converting to something like JPG and upping the compression factor.

Sounds like a simple script, just want to see if it's already create before I make it myself. TIA.

The original post: /r/datahoarder by /u/Paro-Clomas on 2024-08-14 10:43:08.

I've been reading a bit about the posts about Discord in this chat and i think i understand why people are against it. (Basically you get used to having newly generated data stored where someone can and often does restrict access to it, it's also not very easy to index or search).

That being said, i think live chats can be useful, i've met very interesting people and even networked for jobs in discord, if only because it's the most popular tool now.

Do you recommend any other software or website for live chatting? i'd love it if for example there were a live chat for datahoarding, could be on reddit, tough this site is not particularly aimed towards it.

I'd also be open to hearing if you still frequent forums or something else to get new data to hoard.

Thanks in advance for any answer

The original post: /r/datahoarder by /u/Opaquer on 2024-08-14 09:50:55.

Hi everyone

Not sure if this is the right place but I'm starting to get into data hoarding a bit, though I'm still far off what most people here have. I'm at the point where I'm looking at getting some backup strategies going, and I think it's time for me to get a NAS.

I'm in Australia so I don't think I necessarily have as much access to things as elsewhere, and I'm also not good at DIY stuff so would prefer to buy something pre built.

I also already have a machine for things like plex and home assistant and stuff, so this NAS will be purely for backup, so I don't think I need anything too powerful?

I was looking at qnap or synology - I'm not hugely tech savvy so don't want to spend time tinkering with things. I've heard synology is easy to use so was thinking probably the synology DS423 since I want a 4 bay, but I'm open to other ideas depending on what people recommend. Then to top it off I'd probably get a couple of hdds to start things off

So my main question is around the NAS - is the 423 enough? I could get a 423+ but they're about $400 extra here for it, and for purely backup, I don't think I need the extra power it gives? Alternatively I could go for a qnap or another brand? Or if anyone has any advice, suggestions or recommendations, I'd really appreciate it all! Thanks in advanced!

The original post: /r/datahoarder by /u/Street-Complaint-944 on 2024-08-14 08:30:46.

I've acquired a (cough)few(cough) WARC files that I'd like to mass unzip. I'd like to do this while maintaining the directory structure. Searches on the Internet have turned up some tools, but the commands and utility of those tools is a bit confusing. Does anyone know of the proper tool? Thanks in advance.

The original post: /r/datahoarder by /u/Oregano-ma on 2024-08-14 08:13:15.

Hello

I've got several folders, let's call them A, B, etc. Each has about a dozen jpegs all named numerically 1-X. So Folder A has 1-10 and B has 1-8. I'd like a way to merge them automatically into new folder C where they're named 1-18.

There are actually several hundred folders like this so I do want some script of some kind but the jist is the same. All folders have about a dozen photos and in each folder they're named from 1 to 12 or so.

Any ideas?

The original post: /r/datahoarder by /u/Minute-Angel on 2024-08-14 08:11:19.

Looking to store some large media (media, training videos, learning material, etc.) along with smaller documentation - I'll be writing once and that's it, then mostly read-only from there onwards

I had bought a 4 TB SSD (TLC) and quickly consumed all of it, wondering what is the next best option as 8TB SSDs are quite pricy and I am not too bothered about read speed - if I can get 10GBPS that would be a bonus

For home/consumer use only, I don't mind RAID 1 or 0

The original post: /r/datahoarder by /u/SaltKick2 on 2024-08-14 06:10:55.

Hello, I'd like to make a build with something similar to the Jonsbo N3, but in white. Does anyone know something similar? I would be OK with 6 bays as well. I've seen some people spray paint cases, but white tends to be the hardest.

Closest thing I've found is the Fractal Design Node 304, but it doesn't have the hot swappable bays. Also considering th N2 and modding the case to support a better fan, but it seems that it might still have heat issues.

FWIW, I'm planning primarily to use it as a PLEX/Cloud media server

The original post: /r/datahoarder by /u/External_Map5048 on 2024-08-14 02:57:45.

Title. I tried browsing old reddit posts archived under Internet Archive, but I can only see the top or 'hot' posts from when the snapshot was taken. For example I can go to the search bar and search reddit in general on the page and I can look at posts from specific subreddits from when they were archived even if it contains deleted posts, but it only lets me view a few at a time and for example I can't do 'Search in r/DataHoarder '

I know on regular reddit I can set the date posted to a week, month, year etc while still searching within a subreddit but that doesnt help because deleted posts dont show up, they only show up on internet archive

Is there any alternative tool that lets you view saved snapshots from a certain date and lets you do a 'search within a search' on a page so you can browse ANYTHING that was posted that day regardless of if it was banned deleted etc?

I would really appreciate it as I'm trying to get the full scope of a perspective and being able to browse just like normal reddit but back when it was good would really help, thanks!

The original post: /r/datahoarder by /u/grathontolarsdatarod on 2024-08-14 01:25:07.

I've made a few ad-hoc comments about Sabrent's 5-bay enclosures and recommending them.

That hasn't guite changed but I wanted to relate some observations to you guys.

I made comment about my troubles of making a 3-disk raid and a 2-disk raid within the 5-bay enclosure and it not working. Using mdadm.

I ended up making a 5-disk raid and that did work.

However, I never got performance above USB 2-ish speeds on it.

Today, messing around with it... I noticed that my device info (Debian on a beelink) puts 4 of those disks in one tree and one disk in another - in order of activitation, even) if I use all 5 disks.

I made a 4-disk array and the speed is at about 200mib/s rather than the 80mib/s for the 5-disk array.

So... This is where my knowledge breaks down...

Either Sabrent doe something funny with USB 3.1 when 5 disks are connected: doubtful.

BeeLink does something funny with ubs 3.1 passed 4 devices on the same connection: maybe

USB 3.1 is limited to 4 devices on the same connection: don't know.

Anyways. If you're having read/write rates lower than expected.... Might have something to do with this thingy I've observed.

Maybe someone else can come fill the gaps of knowledge I have.

Cheers everyone!

The original post: /r/datahoarder by /u/No_End9329 on 2024-08-14 00:07:54.

I hoard media I'll watch later. It's useful to me because I actually do watch it. I tend to rewatch things more often than I try new things so my goal was to basically have 14 TB of media to watch anything I could ever want then stop there. But.. There always seems to be something else to do. I downloaded a ton, realized it was bad quality, downloaded better versions, did that process like 3x, then was like "screw it" and downloaded all remux content and encoded x265 with max bitrates to ensure best quality lol. But all this is VERY time consuming and computer intensive. Love the hobby but it's too time consuming and KILLS my hands/eye health (as do most screen hobbies). :/

Edit: currently have around 3 TB remaining on my 12 TB HDD to play with (& it is being backed up). So just downloading until I can't. Stopping before that would prob be wise tbh but cannot get myself to actually stop. It's a hard cycle to break. My PC is always doing something for me (downloading or encoding).

The original post: /r/datahoarder by /u/look_at_my_cucumber on 2024-08-13 23:56:32.

Hi everyone. I have an old Mac Mini that i've been using for plex and attached to it is 4 external hard drives and its starting to max out. I was wondering what type of Synology NAS you'd recommend. I want something that is 8 bay? Do I need a powerful NAS when my mac mini is doing all the transcoding? and the NAS is just a network storage? or does the NAS spec needs to be decent so it wouldn't be slow finding the file and sending it across the network? Looking at getting 6x - 22TB Seagate Exos for now and another 2 later down the line. unless you have another suggestion that is better. Thank you!

The original post: /r/datahoarder by /u/thrwaway070879 on 2024-08-13 23:50:11.

The original post: /r/datahoarder by /u/frankensteinkeyboard on 2024-08-13 21:41:55.

The original post: /r/datahoarder by /u/Exotic-Fail-1388 on 2024-08-13 19:22:13.

How do I listen to audios that are saved from Soundgasm.net on the wayback machine on Internet Archive? If I go to view-source: webpage or inspect the page I can see a .m4a link. If I copy the link into the browser it shows me a black box that doesn't have any way to play it.

If I copy the

https://media.soundgasm.net/sounds/random-letters-and-numbers.m4a and not the internet archive stuff it shows a black box.

Instead of something like this:

https://web.archive.org/web/random-numbers/https://media.soundgasm.net/sounds/random-letters-and-numbers.m4a

If I copy just the second part of it which is the https://media.soundgasm.net/sounds/random-letters-and-numbers.m4a

and put it into the browser, then it gives me a 404 error. Does it just not work some of the time? It seems to happen with a lot of different audios that I have found.