It's A Digital Disease!

The original post: /r/datahoarder by /u/noideawhatimdoing444 on 2024-06-14 01:26:59.

I'm not at the place where I can donate space on my server but eventually I will be. I'd like to give support to pages like Wikipedia or any other project I feel aligns with my values. Is that something thats even feasible?

The original post: /r/datahoarder by /u/09Klr650 on 2024-06-14 01:12:41.

Perhaps my search-fu is lacking but all the small 4-bay hardware raid enclosures I find specifically state they are not Power Disable compatible. Before I go the whole "kapton tape in pin 3" route is there a manufacturer/model out there that is capable? Without costing an arm, leg and earlobe preferably. This will be my first such setup and I want to KISS for the moment. Figure that (4) 12TB Reman HGST would be a good start. At Raid 5 that's approx 36TB. Or approximately 4x all my current motley collection of externals combined. Thanks.

The original post: /r/datahoarder by /u/HardLearner01 on 2024-06-13 20:48:16.

How to download the video content of this site?

DP-100 Design a machine learning solution (1 of 6) | Microsoft Learn

The original post: /r/datahoarder by /u/Reasonable_Jelly9435 on 2024-06-13 20:39:44.

TL;DR - Nearing storage cap on my current external HDD. Want to upgrade storage and reasonably future-proof as best as possible. Considering changing my setup from PC (Plex Media Server) + external HDD (media storage), to a mini PC (Plex) + DAS. Could use advice.

Hi, I've done some research on upgrading my current Plex / general media storage setup and I've come up with a plan that looks something like the below. Please share any constructive criticisms, tips, things I might have overlooked, etc. I want to do this right the first time around.

Use case: 1-2 simultaneous 1080p streams, most clients will be able to directly play my mostly HEVC (h.265) content without needing to transcode. Not all, though, so transcoding option is nice. I have Plex Pass so HW transcoding is also an option.

My current setup:

• PC [Plex Media Server]
  • External Hard drive (12TB) [~10TB personal media]

What I'm thinking of moving to:

• Mini PC (Beelink Mini S12) [Plex Media Server]
• DAS (4-bay QNAP)
  • 3.5 12TB HDD (shuck my current drive) [~10TB personal media]
  • 3.5 12TB HDD (buy a new one) [RAID backup?]
  • empty (add a 12TB in future for more storage?)
  • empty (add a 12TB in future for more storage?)

If this looks good, I have a bonus question: How do I safely move my content on my current 12TB external hard drive to the DAS? Is it as simple as shucking the drive and installing in the DAS? I've also read that setting up RAID will wipe the drives; will it be necessary to buy a third 12TB drive before moving my media?

Thanks in advance!

The original post: /r/datahoarder by /u/aes100 on 2024-06-13 20:15:45.

If you were to build a storage computer based on NVMe SSDs, how would you do it? Nor datarate, nor storage space is priority. Just sheer number of NVMe SSDs... and you hate cables.

Build#1: AMD Threadripper with PCIe to M.2 adapter cards. Because I think that is the only motherboard that has more than one or two PCIe slots. But Threadripper is overkill expensive for just storage and not worth it. I didn't do research on Intel side.

Build#2: Get a server motherboard with lots of SATA ports and lots of SATA to M.2 adapters. Put the NVMe's in, and connect them to SATA ports. But you don't like cables.

Build#3: Get a LattePanda Mu based on Intel N100 which they advertise a NAS carrier but I don't know if it exists yet or not. Or get a FriendlyELEC CM3588 based on Rockchip. You are free of cables, but are you gonna regret accessing the data on there because it is slow? Or is it actually fast enough to, I don't know, watch a video from it?

What do you think about the above builds? What downs and ups do you think they have? How would you do it? Why?

The original post: /r/datahoarder by /u/feedmememes on 2024-06-13 18:39:22.

I just purchased this HDD of Amazon: https://www.amazon.com/gp/product/B08T3PBV57/ref=ppx_yo_dt_b_asin_title_o00_s00?ie=UTF8&th=1

However it did not come with the power extension cable mentioned in the product description. I have purchased this drive before and it would not work without that cable, so it is necessary for me to get one for the HDD to work. My question is what would I search for the get the correct part? Would any sata-to-sata power extension work or would it need to have some special spec? I've read a 4-pin Molex to SATA connector might be the solution, but I would like to confirm before purchasing because I am dumb!! Something like this for example:https://www.amazon.com/Cable-Matters-3-Pack-Molex-Power/dp/B00STNUB04/ref=sr_1_2?crid=B3ZGL1OM3IR1&dib=eyJ2IjoiMSJ9.f9jru-Gy4n_1s6sxvx6tecrDs0kKyQc_KXFNnctPSyGpgk3wiTyXQRVjv8latROCZFMth35JZj6AzNa09K_WespKmumHEQjA1Qn9tuzw8MBtUCX9qyoxPfAfS1R9c3JLDgIR1NHveuutkNGIKTrnouyP5DBd2e2yp3U9bkzTz5b_jPWn4NWf9pusLWEVS3OvqcRsb74HYfa7bE5vtR5EWmod_f3B-fJJnotQO7qTLbE.4mkQvoblx472TPQkw3pPm3UzP8ULyibyZTNH-zpyBg4&dib_tag=se&keywords=Sata-to-sata+power+adapter+cable&qid=1718302912&sprefix=sata-to-sata+power+adapter+cable+%2Caps%2C84&sr=8-2

Or if there are any other solutions that would be helpful too.

Sorry for noob question and thank you in advanced for any help!

The original post: /r/datahoarder by /u/Clive1792 on 2024-06-13 18:26:14.

Personally I feel snowed under with what I've got. There's countless files on my PC scattered everywhere. Documents, pictures (probably in the 100,000s), audio files, video files. It's most certainly not organised at all, it's scattered all over the hard drives on this PC in various folders & folders of folders.

But to make it better I had no real backup plan. I'd just drag & drop and then not really remember what I'd backed up and what I hadn't so then I'd buy a new hard drive & make another copy because the last drive was fairly full.

I now have a number of drives that surely have many multiples of various single files.

My end goal is I would like some organisation rather than random files & folders scattered here there & everywhere. I'd like to get rid of the multiple duplicate files & also delete files I no longer want/need.

Have any of you attacked this kind of task? Any tips or do you really just do it file-by-file? Which to be honest with what I've got & the fact I only have limited time outside of work, this is going to take me many months if not 12months plus .... and that's if I get stuck in & stay focused doing it day in day out.

The original post: /r/datahoarder by /u/cuterthanyourcat on 2024-06-13 17:46:57.

Hi I'm not a technical person so please use layman's terms. I'm considering cloud storage for my documents/photos/music. I'm a DJ so music storage is important and my iCloud is full. I'm hesitant to get an external hard drive for storage because I was living in a humid environment and my last one got damaged, meaning I lost 9 years worth of stuff.

What's the cheapest, most secure cloud storage I should use? I don't really understand "egress" but if you have info on how much it'd cost to redownload those photos or music if needed, please let me know. Sometimes I like to look through old photos, idk if I'll have to download those in order to see them?

Thanks in advance for explaining to a noob like me!

The original post: /r/datahoarder by /u/saltyspicehead on 2024-06-13 16:53:05.

6 x 12TB Drives, Raidz2, NVMe L2ARC + SLOG etc etc etc roughly 45 TB to work with.

I'd like to start by creating a weekly Wikipedia backup job to track change history. I'm familiar enough about storage tech to know that this should be possible without consuming a new ~120GB of storage every week, since only the delta will actually consume any disk space. However, I'm not familiar enough with non-enterprise solutions for this task. I see a few tools in the wiki but I'm not sure which one is best suited for this situation.

Any solutions or guides you can recommend? I'd prefer not to enable dedup on my primary pool, but if that's the best option then I'll likely just create a secondary pool for this specific task.

The original post: /r/datahoarder by /u/Expensive_Elk_3618 on 2024-06-13 15:47:26.

I am trying to figure out how to download all of the message board posts in a certain category along with all of the responses. I can get the first level done but not every level with responses. I've tried HTTrack without success. Years ago I used to do this sort of thing and it wasn't all that hard to figure out but I am older and everything is a little stricter. :-) So, I would need the easiest tool. Thanks for any advice you can offer. Here is one of the sites. https://www.genealogy.com/forum/surnames/topics/reed/

The original post: /r/datahoarder by /u/studioviper on 2024-06-13 15:32:08.

As the title says, the fantastic and amazing the.tnk.loft Instagram account is about to be deleted. A lot of really great and inspiring content will be lost forever. Is there an easy way to back everything up? Thanks!

The original post: /r/datahoarder by /u/digitalanalog0524 on 2024-06-13 15:26:05.

Synology and Terramaster both allow for fluid RAID setups (SHR and TRAID). I understand there is no black magic here and both are just using clever partitioning and a mix of RAID types. How come QNAP and Asustor do not offer these out of the box? I struggle to understand how Terramaster could out-innovate those two companies.

The original post: /r/datahoarder by /u/miko-zee on 2024-06-13 15:12:53.

So I recommended a seller to my brother who is highly regarded as providing mostly legitimate drives. The problem is they seem to be of varying quality almost like shucking a drive. Sometimes they're totally new, sometimes unused old stock and sometimes manufacturer refurbished.

My brother got a years old stock that had zero on power hours.

However, a review of the seller suggested the following methodology. They were reviewing a 12TB Exos

"According to my tests, this drive is legit. I was able to:

• Verify the serial number at the Seagate website- - - - - Format the drive into 11 partitions.
• Put files into the first and last 2 partitions and was able to read back the files.

Note that the drive has 11,175.98 GB of actual free space. The missing 825 GB may have been allocated to the file mapping table."

Why do you need to partition it n-1 TB times then write data on the first and last partition? Is this even sound? I think they suggested it because it's quicker and more painless than stress testing the whole drive for a legitimacy test.

EDIT. I want to clarify I know the scam of declaring high capacities using a smaller capacity medium. Also most of these drives usually have valid warranties just in a different region when checked via the their respective manufacturers website even the refurbished ones.

The original post: /r/datahoarder by /u/No-Balance-8038 on 2024-06-13 15:07:43.

I want basically a similar small case that allows for a 3.5" with tray which is 17cm long, but as small and portable! I will get a bigger antistatic bag still. The issue is that cardboard will wear down.

I looked into Pelican cases and they are just too big to fit in my backpack!

I already have 8 of https://www.amazon.de/gp/product/B01JOCHO80/ but they do not have enough space for a 3.5" with tray on ( ~ 170mm)

Dimensions are:

• 7cm height
• 20cm length
• 13cm width

https://preview.redd.it/pfl9kd0ytc6d1.png?width=541&format=png&auto=webp&s=8e3b17eba8907cd0ddabcf5fa052f2cfe6823df5

https://preview.redd.it/uifnnwg2uc6d1.png?width=709&format=png&auto=webp&s=22d8ec60ab6db16ee838a59b60a5822811c5c198

The original post: /r/datahoarder by /u/OC2k16 on 2024-06-13 14:57:18.

Hello,

I currently run a PC with win 10, older hardware. I have 6 3TB hdds, set up using storage spaces, two-way mirror, so 9TB total. They are all connected via sata.

I want to upgrade to a different PC, and transfer the data to newer hard drives.

Will it be as simple as: disconnecting some drives, connecting new ones, transfering the files to new hdds, and connecting new hdds to new PC? Or will windows get angry with that?

I am going from win 10 to win 11 if that matters.

Perhaps I should just transfer the files to the single drives, put those in the new PC, then add their backups and do the storage spaces then?

Any advice would be appreciated!

The original post: /r/datahoarder by /u/Positive_Minds on 2024-06-13 12:32:48.

This 4TB drive and also my 10 year old 2TB HGST drive. have popped up bad and weak sectors , they haven’t been switched on in 10 years ..

So was this basically my fault for not powering them up every and using disk fresh etc or is it just a case of age related demagnetisation or mechanical failure ? I just don’t get as most of my 10 year old western digital drives or seagate have been fine when tested after 10 years

I’m seriously thinking of doing yearly refreshes of my drives from now on .. My other question is can the weak sectors be reallocated or removed ?

I read this on super user , is this true ?

In order to keep the data signal from fading, you need to re-write the data. This is often known as “hard disk maintenance”, and should be done 3 or 4 times a year. While it does not prevent data from being corrupted or deleted, it can go a long way towards ensuring that the magnetic signal does not fade away completely. The way it works is to read every sector of the drive, and then re-write the data found there, provided the drive reported no errors. If this is done on a regular basis, the magnetic signal of every part of the drive will be refreshed long before the signal fades or becomes ambiguous. This technique also gives the drive controller the opportunity to decide whether to retire any sectors that are becoming too unreliable, before any important data is lost.

Magnetic Field Breakdown

Most sources state that permanent magnets lose their magnetic field strength at a rate of 1% per year. Assuming this is valid, after ~69 years, we can assume that half of the sectors in a hard drive would be corrupted (since they all lost half of their strength by this time). Obviously, this is quite a long time, but this risk is easily mitigated - simply re-write the data to the drive. How frequently you need to do this depends on the following two issues (I also go over this in my conclusion).

To periodically refresh the data on the drive, simply transfer it to another location, and re-writing it back to the drive. That way, the magnetic domains in the physical disk surface will be renewed with their original strength (because you just re-wrote the files back to the disk). If you're concerned about filesystem corruption, you can also format the disk before transferring the data back.

You can also help to avoid this issue by archiving your data with recovery data and error correction when you put the data onto the drive. Many archive formats support the inclusion of data recovery algorithms, so even if you have a few corrupted sectors, you can still re-build the lost data.

Depending on the priority of the data you've stored, you may want to refresh the hard disk more often. If it is essential data, I would recommend no less then 2 years at maximum. If you can withstand some chance of minor data loss (e.g. a few corrupted sectors here and there), go with 5 years. It doesn't take long to copy the data off the drive, and copy it back.

One thing not considered is the servo tracks and markings. These are written one time at the factory and never again (on modern disks). No amount of re-writes by the user or so-called low-level formatting freshens these. Once they fade, they fade!

It's different with the first stepper motor disks of the 80's. They don't have servo tracks and a low-level format writes ALL of the bits - fresh.

The original post: /r/datahoarder by /u/jammsession on 2024-06-13 12:01:35.

A friend of mine is a photographer. He currently uses the good old "put all photos onto one external HDD and labeling them" method for storing his files.

He now got to a point where he is scared of loosing data and asked about Flickr.

So I was wondering if anyone else here has around 10TB of photos on Flickr Pro. Is it really unlimited? Price seems almost too good to be true.

The original post: /r/datahoarder by /u/ilovecokeslurpees on 2024-06-13 03:04:38.

I have an old computer (turned it off 2.5 years ago but it was a mid-spec PC built in 2016 I bought second hand in 2017). I can barely boot it these days. I want to wipe the 1 TB SSD before throwing the whole PC into the trash or recycle or whatever. Most parts are unsalvageable. I used to use dban way back when but it appears to not have been updated in almost a decade. The PC is running Windows 10, but I would like a simple solution bootable from a USB to ensure complete data erasure. I tried ABAN but it couldn't detect anything. Any suggestions?

The original post: /r/datahoarder by /u/klnadler on 2024-06-13 11:00:30.

Hi all, I was transfering some data from my Unraid to an external drive mounted as an unassigned drive. I had to tweak the command a little to work with the permissions due to the drive being ExFat. After the transfer I realized the drive was set to read only through the Unassigned Disk settings, but the files transferred? How and why?

Command used:

rsync -avPh --no-owner --no-group --no-perms

The original post: /r/datahoarder by /u/Endeavour1988 on 2024-06-13 10:31:38.

I currently have 3x 6TB Western Digital Golds for my media server. 2x Host content and one for spare (backups).

* I do back up externally/offsite

I've always had a bad history with Seagate but some of my external drives have been solid which are Seagate so I'm open minded. Can anyone recommend from experience what brands to go for? The Golds are 7 year old now so I would like to look at replacing 2x with 12TB and possibly the third with something larger like 16TB. Which should hopefully see me through another 6 years.

The original post: /r/datahoarder by /u/thenazgul80 on 2024-06-13 06:33:19.

Do I have this right? I want to build a nas to store my digital library and stream it to my and my family's devices. Currently relying on 5 external 2.5" drives and windows media share. However I can't outright purchase something like a Synology nas due to cost. I have my old PC collecting dust. Amd fx 4100 Radeon 7700hd and 8 gb of ddr3 ram. Here is the plan. Undevolt the fx4100 to the lowest it can go. Diy mod my Asus Vento a8 PC case for cooling and dust prevention. Buy a 16-20 TB drive and start the nas. Once I can afford a second drive i plan on using them in raid 1. Once a 3rd drive is added I want to use them in raid 5. Over all I want to end up with 40-60tb of useable storage with some redundancy and keep the external 2.5" drives for backups. From what I can gather, truenas won't let me just add drives later on down the line. Is unRAID my only option if I want to start with the bare minimum and expand later on as I can afford more drives?

The original post: /r/datahoarder by /u/martian_doggo on 2024-06-13 05:21:47.

So i have a bunch of old vob files, and i want to convert them to mkv or mp4 or something. I admit i don't have much experience in video manipulation but this is turning out to be exceptionally hard.

I have around 5 movies which have 5 vob files each. The movies are around 4 hours long but around 12gb in size for barely 704x570p so i would love to concatinate and compress them.

Now here's the problem, none of the video editor software is picking up the subtitles, not handbrake, not even ffmpeg but mpv somehow plays with the subtitles, though it takes mpv 10ish seconds to load the subtitles (weird ik).

Now if i join 5 of the files to make 1 complete movie using ffmpeg, the subtitles stop loading.

I have no idea what to do :(

thanks

The original post: /r/datahoarder by /u/cloudfortynine on 2024-06-13 05:08:54.

Hi everyone,

I could use some guidance as I'm unsure what the simplest and low maintenance solution is for me. I'm looking to purchase an external HDD to backup the following:

1. Macbook
2. 1TB SSD with my travel photos to be edited
3. 1TB SSD with my partners travel photos to be edited
4. 2TB SSD with music libraries for music production (Kontakt, Omnisphere etc)

At first I was thinking of buying a 8TB external HDD and just backing everything up via Time Machine by having it all plugged in at once. However people online had also mentioned CCC and I'm unsure how that works? I was also hoping to back this all up to a portable HDD but it seems that 8TB HDDs can only be plugged in is that true? I'm not interested in NAS etc. as this is too complicated and expensive for me.

As a sidetone I'm also planning on backing up to BackBlaze as well.

Thanks in advance for any pointers!

The original post: /r/datahoarder by /u/Mode7GFX on 2024-06-13 03:30:49.

I posted this on stack overflow and they called me stupid (I am) and locked the question when someone was in the middle of actually helping me so I'm asking here because it's probably a more welcoming community for this sort of thing.

https://prcm.jp/list/akb48%20%E3%83%97%E3%83%AA

So this (formerly) huge Japanese image sharing site is shutting down in 2 weeks and my sister's begging me to archive at least some of it. I was trying to find a python script to automate it, cuz it will take me forever going one by one on tens of thousands of images, but I can't seem to find one that can archive this sort of website. It only goes by pages of 9 and the thumbnail images are shrunk down a ton so you have to click on the image twice to open the full size. Luckily it seems like every thumbnail image shares a name with the source image, but with an added suffix of the preview size, so I imagine it would be possible to have the script delete the underscore and re-add the .jpeg extension. As for going through pages, I'm not so sure on, but if I can input a list of URLs and just batch copy the original URL adding every page number (I know how to do this without a script), I could just use that.

I was only going to download AKB48 related images, because most of these images aren't available anywhere else online, and soon won't be anywhere period. They're all old and thus fairly small, so I'm not too worried about it taking a long time to download.

If anyone can direct me to a script that can do this or has some other mass image downloading method you're aware of, please let me know. Thanks!

The original post: /r/datahoarder by /u/Visual-East8300 on 2024-06-13 03:20:27.

A couple of years ago, I bricked a 10TB WD easystore by issuing "hdparm --security-set-pass" on the USB drive, un-bricked it by plugging the HDD to another USB adapter then "hdparm --security-disable". Today, I bricked a 4TB WD easystore (2.5 inch), out of luck this time because inside the enclosure it's an USB on-board WD Blue drive.

Both are using ASMedia ASM1151W.

Does anyone have ideas how to recover from this? Or it's not worth the time trying to fix?