It's A Digital Disease!

The original post: /r/datahoarder by /u/alb1234 on 2024-08-19 06:18:17.

So, I'm moving the contents of an External WD 10TB HDD to a brand new External WD 20TB HDD. It's all movies & tv shows and associated images/files for Jellyfin. A lot of small files, but hundreds of large files too.

I think Buffer Size(MB) is 256 MB by default. I can't remember why I changed it to 512 MB, but I must have. I doubt these settings would be adjustable if default settings were perfect in all situations. I don't understand if bigger is better due to hundreds of very large files. I read the manual page and it doesn't help me a bit.

https://i.imgur.com/omlCVgA.png

What does it mean to use a small value in Buffer Size versus a large value? Say 32MB versus 512MB? I have no clue what the MaxIO settings do and my Google-Fu has been terrible.

https://i.imgur.com/aCWu3iq.png

https://i.imgur.com/zV4c7DL.png

Example of a typical directory & its contents:

https://i.imgur.com/ulIRegM.png

Do you recommend using a single Buffer Size setting regardless of the copy job? Do you recommend I get a different hobby? HELP!

The original post: /r/datahoarder by /u/BoostedFrames on 2024-08-19 04:35:45.

Hi, I was messing around and somehow deleted some important stuff while trying to learn some of the things I could do on Unraid.

My question is how can I go about recovering these files? My drives are xfs encrypted for the file format. All one array. Thanks!

The original post: /r/datahoarder by /u/No_Trust_8596 on 2024-08-19 04:22:09.

I'm surprised and confused that this sub and r/Backup, r/privacy, and r/netsec don't seem to have much information about file encryption other than old references to veracrypt and cryptomator. I saw a few references to diskcryptor as well, plus cloudally and gmail backup (for email-specific backups on a regular basis) and backblaze/synology for cloud backups.

I need to regularly backup work and personal files (M365 and many audio and video recordings), plus work emails (I'm using protonmail) and personal emails (gmail). I have several external hard drives (1-4TB each), OneDrive, and Google Drive (want to switch the cloud ones to synology I guess, when I set up NAS). Everything needs to be encrypted, though. My research says that Bitlocker isn't reliable if I need to protect files from Microsoft (work files are confidential attorney-client privileged).

What do people do? Am I stuck with veracrypt (hard drives, laptop) and cryptomator (cloud backups)? Is diskcryptor a viable and private option?

The original post: /r/datahoarder by /u/Redbird7474 on 2024-08-19 04:20:59.

I am a baby hoarder who is hitting the limits of my external 16TB and 8TB drives that hold content for my Plex server. I run it on an old iMac that is almost only used for this purpose. I know that my next move is to move to an expandable system and not just tack on a third external drive. I would like to do this as economically as possible.

Here are my thoughts that I'm hoping someone can confirm or tell me why I'm wrong before I purchase anything. I am tech-savvy enough to have set up my Plex and *arr setup, but am not experienced with JBOD or RAID setups.

I am thinking of buying a 4-bay enclosure like the TerraMaster D4-300. I would purchase a new 16TB drive and add it to the enclosure. Then I would copy my 16TB drive contents to the new drive, shuck the 16TB drive, add it to the DAS, copy the 8TB drive contents to it, and finally shuck the 8TB drive and add it to the enclosure, all in JBOD (losing a drive isn't an issue, all the data is replaceable). This would add 16TB of storage to my setup and only need to purchase a single drive and enclosure. It would also give me some expandability in the future and clean things up to a single connection/device.

Thanks in advance for clarification, advice, and patience!

The original post: /r/datahoarder by /u/nlj1978 on 2024-08-19 03:01:51.

Moving on to audio cassette tapes. I've read lots of mixed opinions here and elsewhere. I know I could likely buy digital files of most of the tapes I have, but I already own these.

Has anyone had any luck with the various Chinese tape decks with USB on Amazon?

I'm also seeing 3.5mm/rca to USB capture cards on Amazon. With these would you just take a players headphone out and use that?

The original post: /r/datahoarder by /u/ge4020 on 2024-08-19 01:14:31.

My 12T HDD was mounted through a USB dock, everything had been fine until I install it to a new PC. Then all file disappeared, but HDD is accessible. I mounted it back to the USB dock and managed to recover file using DiskGenius, copied all the file to another HDD, just in case.

Wonder why that happens, shall I return the HDD or is it normal for them to be unstable when mounted through USB?

The original post: /r/datahoarder by /u/Least_Original_5754 on 2024-08-19 00:31:09.

https://www.seagate.com/products/external-hard-drives/ultra-touch-external-drives/

I know the micro-b ones are shuckable, worried the usb-c ones are built different.

Couldn't find a confirmation online. Anyone know?

Not planning to dissemble anytime soon. Just want to know if I have options in case the connector breaks, or if I want to install elsewhere later. Picking between the Ultra Touch (USB-C) and One Touch (Micro-B).

The original post: /r/datahoarder by /u/TheRealRubiksMaster on 2024-08-18 23:43:10.

I am wanting to download every steam game i own on a portable system. So something preferrably lightish weight and smaller form factor. I am looking for a range of maybe like 20Tb at the low end, 1/2Pb at the high end. It doesn't neccisarily have to be modular. But im just looking for a way to easily transport that amount of data.

I constantly take my laptop everywhere, and also am thinking about maybe getting a steamdeck as well. So I want to have to majority of my library avalible offline (i just like data "colleting") I just want convenience over erverything. Should I be using stuff like hdds or a system with ssds? Normally i just run m.2s in my laptop/main pc so idk what is best for a system that will be actively used.

I was thinking, maybe something portable that could hold like 10 ssds in them, and load them with 4Tb's. But i dont know how good of an idea something like that is, hence why im asking here.

The original post: /r/datahoarder by /u/stenyak on 2024-08-18 23:33:11.

Hey all, I'd appreciate advice on how to organize my data and backups.

The idea is to be able to recover from bitrot, from powerloss corruptions, from accidental file deletions, etc.

Note: I don't care about offsite backups atm: I'll rethink my current remote backups at a later stage.

Currently I have a 12tb and a 16tb EXT4 drives, both already nearly filled, and both containing data as well as some daily borg backups with cron. Borg is okay for accidental file deletions, but not for bitrot/powerloss/badsectors/failure/etc.

Since I've ran out of free space, I just purchased two 22tb drives, but I'm not sure how to expand storage while adding reliability. Some options I was considering:

• snapraid spanning all 4 drives (since they are different sizes and can work with those ext4 drives). I guess I need to use a cron to remount all 4 drives as read-only during the daily snapraid sync.
• snapraid-btrfs with the 2 new drives (so i can btrfs-snapshot, instead of doing the read-only dance). But this means the original 12tb and 16tb drives are unsafe and cannot recover from bitrot or similar issues. So maybe dedicate those exclusively for borg backups with monthly borg checksums verifications?
• A ZFS mirror with the 2 new 22tb drives. This also leaves the 2 old drives unsafe to bitrot.
• Get one (or a bunch of) more 22tb drives, and do some sort of ZFS raid maybe. Again the old drives are unsafe.

Main headache is that some of these solutions require reformatting old drives (which would be a pain), or require same-sized disks everywhere (I wouldn't have any other use for those 2 drives, and I don't want to sell them 2nd hand), or require a read-only period for a daily sync (will surely lead to issues on programs trying to write), or can lose the data generated before the daily sync (so not 100% resistant to bitrot), or.....

So yeah, I'm a bit stuck with analysis paralysis atm, any help would be great :D

The original post: /r/datahoarder by /u/V_alias on 2024-08-18 22:49:17.

I'm relatively new to ZFS/RAID and am looking for some advice on how to configure some old hard drives that I've had lying around for years that I haven't done anything with. They'd be primarily used for seeding torrents. I'd prefer some redundancy so I don't randomly lose data, but also because some of the drives are old and are more likely to fail. I'm not necessarily dead set on ZFS and am open to other options as well if they'd be more applicable.

The HDDs I have are:

• 1x 1 TB
• 2x 2 TB
• 1x 750 GB
• 1x 500 GB

The original post: /r/datahoarder by /u/gleep52 on 2024-08-18 21:37:30.

I have tried to do my research and realize DWPD trumps TBW - and I'm specifically aiming at consumer based variants for cost. I'm looking to replace a 2TB model. My current NVME drive has a rating of 1300 TBW and its near its end of life after only 504 days.

I am based in the US. I was curious if anyone had a site that kept track of different storage models and statistics for filtering out or filtering by variables like DWPD or TBW.

I purchased an addlink datacenter NVME NAS SSD unit here https://www.amazon.com/gp/product/B0D9NFPCZQ but I cannot get one PC to boot with the drive installed, and in a 2nd PC in windows, disk management shows it at 0GB not initialized and the init fails. I'm hoping it's a defective unit as the price for the 3800 TBW rating is appealing - but thought I'd ask here if you beautiful people had suggestions or a site I can research?

The original post: /r/datahoarder by /u/nfswsafe on 2024-08-18 21:15:50.

Basically, I have 4 hdd's in my homeserver, I download onto one (mainly) and then by hand move the files around. Sometimes things get deleted and then i can move more into those hdd's by hand, but there must be some automated way to do this right? I'm using linux

The original post: /r/datahoarder by /u/kjetil_f on 2024-08-18 20:54:30.

This seems to be a very good solution for backing up VHS tapes. It's not cheap, but it might be worth it.

The original post: /r/datahoarder by /u/Disgamer on 2024-08-18 20:17:38.

I'm looking to start my first plex media server using an old gaming PC. I want to use ICY Dock 6 with several 8TB SSDs (https://www.amazon.com/gp/product/B01M0BIPYC/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1) I know that the define 7 has 2.5 inch bays as well, but I want to use the ICY Dock to maximize the storage space.

I was going to transfer it to a Fractal Define 7 XL, however it only has 2 5.25 in bays for the docks. I was originally going to use Seagate Iron Wolf HDDs, but my concern was noise as I am going to set it up in my room(I don’t have anywhere else to put it)

I have few questions:

Are there any cases like the Fractal Define 7 XL that have room for an ATX motherboard and 3060TI(If not, my plan was use a PCI ribbon cable and mount it flat.) and several 5.25 inch bays?

If I were to use the ICY Dock, how would I daisy chain several of them together?

For the future, if I ever need to expand beyond what is in that case, what are my options? Are there any cases that consists of just 5.25 in bays? If so, could I just connect it the drives in the original case and daisy chain it? How would I do that?

I've looked into traditional server rack housings, however from what I've seen, the Fractal Define 7XL is a lot better deal in terms of bays, even if I cant use the dock. Ive seen server housings that cost more but only have a fraction of the drive bays.

Im leaning toward the Define 7 XL(even if I cant use the dock) because Im planning to use an old monitor to interface with it. From what I've seen, lot of the server racks only have USB and not the traditional IO(Though I could be wrong, this my first server.),

Is there a way to connect it to my main PC and use a program like a virtual machine to interface with it within my main PC?

I have a ton of movies that Im going to add. I should be set for a while, I just I want to make sure that in future I am able to expand if necessary. Im willing to spend a bit more now for future expandability.

Any advice is appreciated!

The original post: /r/datahoarder by /u/Towhgdk on 2024-08-18 20:05:57.

First of all, let me tell you that I am using a translator because my English is not good.

I put movies on my SSD and watch them on my TV.

But NVME SSDs often freeze when I use them, So I'm using a SATA SSD that matches the USB voltage of 5V.

I am currently using Transcend's SSD370S 128GB, an SSD equipped with 14nm Samsung MLC.

I'm using this MLC SSD well, but I'm severely feeling the lack of capacity.

And here's the question. If I were to upgrade this, would I go with the older 2D MLC or the modern 3D TLC?

I don't really need endurance since I don't add or delete data that much, but high data retention would be nice.

And I saw that the prices of used MLC like M600 and bulk SSDs like 1300 are similar.

I don't know what would be a good choice in this situation. If I buy it without thinking is no Matter? I would like to decide after hearing your opinions.

The original post: /r/datahoarder by /u/TXAndre on 2024-08-18 19:27:58.

Hi folks,

I don't post here often, my last NAS build is already 5 years old. At that time, I purchased WD40EFRX (the original WD Red 4TB 64MB cache with no "plus" or "pro" in the name as there was only that one variant, and it was CMR).

Those have been going rock solid so far. In fact, I don't even want to replace them, I just want to build a new system, capacity not yet decided.

I am considering buying the exact same drives "new" from Ebay because they are super silent and that is what matters the most to me due to where the rack will be placed.

Anyone has direct comparison with this drive vs another (newer, potentially bigger) drive? How do the WD40EFPX stack up? I read they use fewer platters so shouldn't that make them even more silent? Or are they built on another "drive platform" and thus are louder?

Number of drives (up to 12) and power consumption are of secondary concern to me. System will be HW RAID 6 so smaller drives are actually somewhat of an advantage.

Cheers

The original post: /r/datahoarder by /u/scphantm on 2024-08-18 18:11:41.

I have a pair of netapp shelves connected to a server that will have all the drives setup in zfs.

If I add a second server, say a single node k8s machine to the rack (or multi-node k8s for that matter), I can easily connect the second server to the SAN daisy chain. With that in mind, is there a way to share that zfs servers drive access thru the much faster san interface rather than nfs thru the nic?

The original post: /r/datahoarder by /u/jesse32213 on 2024-08-18 17:35:12.

I ordered a 50 pack of dvd r DLs that are in the "life" series and I've heard mixed things about it, so I was wondering if I could hear what you guys thinks of it.

P.S it's the printable hub one if that matters

The original post: /r/datahoarder by /u/petrichorax on 2024-08-18 16:49:46.

tl;dr: I need to efficiently and cheaply split up some drives in a way that's going to be easiest and quickest to mass clone in a short period of time. Need your experience

So I'm a part of the hacking community. (Still feels weird to call myself a hacker, but w/e, I'm saying it like this for brevity)

I attended DEFCON this year and went to the data dupe village with two 8tb hard drives, got myself the Infocon 2024 drive and the vxunderground malware sample drive.

I intend to attend another con on the other side of the planet within a month. Will also be visiting other hackerspaces in the region.

Cloning 8tb drives by asking people to bring their 8tb drives is a bit much, for both time and expense reasons. It's a poorer region without cheap access to drives. I was thinking of splitting these up into 2tb drives.

Planning to pack the setup for this into a pelican case

Here are my questions:

1. Is this a good idea? (I mean the procedure itself, save me your concern trolling about transporting malware please. Old dead malware is really only good for studying/education, which is the intention here)
2. If so, what's the best value for 2tb hard drives. I was going to buy some stuff off of amazon but i got a suspicion i may be paying a premium here.
3. What are some options i should consider for copying the drives? I know that there are really beefy solutions out there but hey cost thousands, and you're limited by the write speed of the drive anyways.
4. Any good calculators that you know of to calculate my actual 'work capacity' here?

edit: or just downvote me to hell I guess

edit2: I'm far more interested in the answer to #3 than anything else

The original post: /r/datahoarder by /u/The_Asian_Hamster on 2024-08-18 16:16:13.

The original post: /r/datahoarder by /u/ddleather32 on 2024-08-18 16:06:34.

Original Title: Amazon Photos - Can I use Back up as Semi Sync option in PC? I don't see sync option. Can I put all my photos under year folder and use back up option to upload it? I don't expect any file changes to folder after I upload. I am curious to know your thoughts about Photos since they shut sync in PC

The original post: /r/datahoarder by /u/TheOnlyWonGames on 2024-08-18 16:02:50.

As the title states Vuescan is being such a pain in the ass. I have an Epson v600 which works great when Vuescan works. Every time now I plug in the scanner or turn it on Vuescan freezes and hangs, if I have the scanner on and turn Vuescan on it won’t open without crashing. I have no idea what to do anymore. I’ve already tried replacing the cable, replacing drivers, etc.

The original post: /r/datahoarder by /u/Flyingfishfusealt on 2024-08-18 13:00:55.

The way I have been hoarding data is buying a few SSDs at a time and slowly filling one up to around 60% (some more, some less. I buy new drives once or twice a year) while copying it for backups, the problem with this method obviously is that none of the drives are dedicated to any one type or topic of data and now that I have reached a decent level of storage I am attempting to rectify this sorry state of affairs and I am finding it somewhat tedious.

I can bang one out in a few days but I was hoping there was a software or script already made that would perform the following tasks:

1. Scan attached drives for directory and file hierarchy
2. add to database with something similar to [ drive UUID: "" , "filename": filename , "duplicate_exists": boolean, "duplicate_drive_UUID": [drive_UUID,drive_UUID, ...etc...]
3. save to yaml with direct code serialization and/or json and/or some sort of sql database (preferably sqlite3)

Thank you for any information you can provide to help with this. I know I should have done it different but I focused on duplicates more than fast acquisition of storage space (cost prohibitive).

The original post: /r/datahoarder by /u/cuteprints on 2024-08-18 07:53:07.

I can buy cheap 1TB laptop drives for ~$10 each which I think is pretty good for an offline backup, but the thing is that my main ZFS pool is 20TB with various dataset size and snapshots, and I don't want to buy 20 SATA dock/card to use all of the available drives at once.

Is there any way to achieve something like... I plug each drive in which will hold a fragment of the total data to backup/restore? Like back in the old days with multiple DVDs that you need to split into .001 .002... And then merging them together? This is for backup so I don't mind spending time pluging in/out the drives but I do like the idea of having cheap cold storage.

The original post: /r/datahoarder by /u/war4peace79 on 2024-08-18 07:00:17.

I inherited a couple hundreds of 35mm film rolls which I want to scan and add to my digital archive photo collection.

Looking for a good film scanner which allows me to put a film in and batch scan it to a bunch of digital photos. I know there are services who do this, but I don't know what most of those film rolls contain, and I am not comfortable with handing them over to a third party.

My budget is 500 €, but I can throw in 10-20% extra if a more expensive product is really worth it.

Those who had experience with such a scanner, what did you like about it, and what did you dislike?