It's A Digital Disease!

The original post: /r/datahoarder by /u/winston_w_wolf on 2024-07-25 08:46:21.

From time to time, I transfer files from my laptop's SSD to an external HDD (I have several external drives).

I notice that if the files have been in the SSD for a long time then the writing speed often gets excruciatingly slow (like under 2Mb/s). It happens with all the external HDDs that I have so I think the issue (if any) is with the SSD.

Now, if the files have only been on SSD recently then the SSD->HDD transfer speed almost always goes to 60-80MB/s (as expected with a USB 3).

Is it a known feature (problem) with SSD drives by design (meaning because of how SSD drives work)?

Thanks.

The original post: /r/datahoarder by /u/SlinkyOne on 2024-07-25 08:44:58.

The original post: /r/datahoarder by /u/codebreaker101 on 2024-07-25 06:52:56.

I'm looking for a ADF scanner to digitize pictures, books and magazines.

Both have:

• native optical resolution of 600 dpi
• ultrasonic jam sensor
• CIS image sensor
• adequate ADF capacity (>= 80)
• same scan speed

Why Avision:

• larger paper width (242mm vs 215.9)

Why Brother:

• Network connectivity

Any other parameters I should be looking at?

Any other ADF scanner I should be looking at for the same price range as those two scanners?

The original post: /r/datahoarder by /u/thrwaway070879 on 2024-07-25 03:58:28.

Picked up an enclosure which most likely is e-waste and 8 drives 3 4tb ironwolf SATA and 5 3tb SAS Seagate drives.

I don't currently have a SAS setup at home, but this isn't a skill question. I don't plan on buying SAS drives in the future, but if it opens doors for cheaper storage I might invest in hooking these up.

I'm only in for Benjamin a Jackson and a Lincoln at this point.

The original post: /r/datahoarder by /u/helix400 on 2024-07-25 03:24:19.

Original Title: I'm desiring a friendly daily offsite backup solution for terabytes of data that retains all file versions and prevents overwrites or deletions. Seems the only self-hosted way to get there is pull backups, append-only push, or push to ZFS?

Lately I've had a few bad experiences accidentally overwriting files and backing up those overwrites. I've also seen others hit by ransomeware attacks. Now I just want off-site backups where I can feel at ease.

The main goals I'm trying to achieve:

• 1. Always retain old file versions. Suppose someone (attacker, family member, me) ovewrites key files, and I don't discover it until two years later. I want three year old backups of it.
• 2. Keep attackers from being able to delete backup files. Suppose an attacker compromises a computer, I don't want that attacker to have access to username/password credentials to simply delete the remote backup files.
• 3. A GUI that allows for friendly restoration. I'm perfectly capable of consoles, but sometimes GUIs are just easier.
• 4. Provides emails of backup jobs. Life gets busy and email digests are the only thing that has consistently worked for me.
• 5. Nightly backups of family computers. (So they can copy phone photos over and not worry about backups.)

I've also got two Linux servers with enough storage, one at my house and one at a family member's house.

I've been in the weeds on all this. Both #1 and #2 are tricky. Many backup solutions simply ignore #2 because their model is push backups (the computer logs into some remote, then copies files to the backup). In order to push you need credentials to read/write/delete, so a compromised machine has easy access to delete backups. I've seen push model fixes for this: A) make the remote only capable of read or append, and not overwrite or delete. Or B) use ZFS on the remote and have some kind of post-backup script or daily job that simply creates ZFS snapshots. Now backups can't get deleted or overwritten. Problem here is I've personally found append-only filesystems to just not play well with push backups, and ZFS is one more complexity that I could goof up.

I've got experience with rsync, Bacula, CrashPlan, and Duplicati. Also been looking at Borgbackup, Urbackup, Kopia, Bareos, Restic, and Duplicity. But the details get so overwhelming. I think I've got it down to two options for me:

Option #1 - Bacula

Pros:

• Pull based
• Tons of options
• Proven for 24 years. Active community.
• Various tools to assist, such as bacula-web to read reports, baculum to manage, or other tools like Bacularis as a slick GUI (I'm surprised how good this tool looks with so little discussion about it.)
• Emails for individual backups. Digest email scripts and tools out there like bacula backup report

Cons:

• Bacula scales to enterprise, so interface isn't simple for home backups
• Significant learning curve (but once learned for your needs you're set)
• Seems to require installing multiple tools to get it working as I desire
• Feels old fashioned, especially with all the tape nomenclature

Option #2 - Duplicati

Pros:

• Friendly GUI. Very easy to select backups, configure, and restore
• Has digest emails through www.duplicati-monitoring.com

Cons:

• Dev team is feeling barebones. For example, their Linux build doesn't even work on the latest LTS Ubuntu.
• Securing remote backups requires something like ZFS and a remote script
• Duplicati is infamous for occasional problems with indexing, and I've experienced it too. Rebuilding database indexes can take days and the UI for it is weak.
• Requires saving the configuration, so if you lose the client you can't restore properly unless you know how you configured your options.

I'm just curious what others think about all this. Is there some easier tool out there specifically designed for secure remote backups that makes this process easier?

The original post: /r/datahoarder by /u/SaltKick2 on 2024-07-25 03:03:24.

I understand why a hot-swappable chassis would be preferable in a professional environment where time is money and you might have to swap fairly frequently due to volume. But for a personal scenario, is there much of a benefit? I'm new to all this and just getting started, do people really hot swap drives that frequently? I was thinking I'd just setup something in a RAID configuration and swap only as I upgraded space or one of the drives failed.

The original post: /r/datahoarder by /u/-SiriusWhite- on 2024-07-25 01:19:51.

In the case of when it's still under 50 GB? I reponed my MEGA account 6 years later, the folders are empty but I still had few GBs files from 7/8 years ago despite not logging in within 6 months since that time.

I bought a hard drive, put some important data in it. Now I want to keep important data in cloud as well. But I'm wondering if I'll have to keep reminders of logging in to MEGA even if my account has 50 GB total free space. I'm confused because I got those "warning" emails in 2022, but I still have the GB files from 6/7 years ago there

The original post: /r/datahoarder by /u/Erick2142 on 2024-07-24 23:36:49.

I'm really pissed right now. I had a setup with a couple of drive in a zfs stripe array and one of my drives failed and as a result, I have lost part of my data. It's not that bad as I had the important data backed up to another drive. So I decided to redo my setup with redundency in mind. Made a raid array and decided to give Crashplan a try while I was at it. Boy was that a mistake.

I signed up for the 14 days trial of Crashplan Professionnal. I immediately asked tham about the limitations of how much data you could back up as I saw on here that some people got emails from them after that backed up 15 TB, and that's pretty much how much I got. After 7 days, they confirmed that there would be no issues with the size. After I got this answer I decided to look for how to cancel my subscription in case I wouldn't want to renew.

Imagine my shock when I saw that there is no option on the website to cancel your subscription and that you have to go through customer service... So I did... I made 2 separate support cases to cancel my subscription and I received no answer. That got me a little pissed so I changed my subscription to go through paypal and cancelled the autorenew on paypal's website. That ought to do it, right? Wrong.

Turns out Crashplan uses Paddle.com to manage payments and even though I had cancelled the autorenew, they still were able to charge me after a week of trying (not really sure what happened there). Seeing that I opened a claim on Paypal's website and was only met with more nonsense. Here is the official answer from Paddle:

On the transaction date, the buyer was correctly charged for a recurring transaction of a digital product by Paddle.com. On this date, the buyer was immediately provided with proof of purchase via their nominated email and we have attached continued proof of product usage to confirm the buyer has utilised the service. Additionally, on the original transaction date, the cardholder was presented with and accepted the merchant 14 day refund policy and chargeback policy which has been attached. As the merchant received late communication from the buyer to cancel the subscription prior to the chargeback being issued; we contest the buyer's dispute as invalid.

Refund and chargeback policy:

Exception to the Right to Cancel

Your right as a Consumer to cancel your order does not apply to digital products after you have started to download or stream these and to Products which you have had the benefit of

Paypal then was asking me for proof of purchase of the product. I have sent them everything I thought they might need: emails requesting cancellation, order confirmation showing free trial period, etc. Not 30 minutes later, they ruled in favor of the merchant without specifying any reason. And the icing on the cake: they have closed out my account. So now, not only did I pay for the services, but I cannot use it WTF

TL;DR Tried to cancel my Crashplan service before trial ended. Got no answer + still got charged. Contacted Paypal to contest the charges. And they all told me to go F myself together.

The original post: /r/datahoarder by /u/Remy4409 on 2024-07-24 23:19:04.

It's not exactly data hoarding, but considering I want to use it in part for archiving, I figured it might fit here.

We have an Online Edit computer at work, mainly for color grading. We are currently running an older hardware card, an Atto R680 with 16 drives on it. We will soon replace all drives and I want to update the storage system at the same time, and I need one for another machine too for archiving.

For the archiving machine, I'll probably run TrueNas or something, that makes sense.

For the Windows editing station, I've been looking at software RAID for a long time but I have no idea how that would work in our situation. The drives already are in their own enclosure with their own psu and an SFF-8088 port. I was thinking of getting an LSI HBA instead of a RAID card and just connect the drives with a cable. What kind of solution can I run on Windows that would be somewhat reliable and fast? Running RAID 6 at the moment.

Is there a better solution? Something I'm not thinking about? I keep hearing software RAID is much better now, I wonder what's true?

Thank you!

The original post: /r/datahoarder by /u/atearth on 2024-07-24 21:52:07.

SanDisk Ultra Dual Drive Lux getting hot, is this normal?

Do other USB sticks get this much heat?

The original post: /r/datahoarder by /u/BowtieChickenAlfredo on 2024-07-24 21:08:56.

The original post: /r/datahoarder by /u/SGTARM87398 on 2024-07-24 20:37:38.

I have switch to Linux recently [since windows is now a f**ing spyware] so I need some software that run LTO tape of any kind preferably lto 5 (since it is the tape that I use)

so any of you have an Idea ??

thanks in advance 87398

The original post: /r/datahoarder by /u/pasquyno on 2024-07-24 20:24:17.

As the title states, I am trying to recover youtube videos from around 2015 that have been since deleted, videoara is down and archive.org links return nothing of value

Here are some links for you guys to experiment with

https://web.archive.org/web/20150318135724/https://www.youtube.com/watch?v=maDjxt6Jq_k&feature=applinks

https://web.archive.org/web/20150318135729/https://www.youtube.com/watch?v=OoxzOHvFDD4

If anyone is able to help, it would be greatly appreciated.

The original post: /r/datahoarder by /u/MrPeach4tlanta on 2024-07-24 19:32:57.

I have a handful of playlists in XSPF format, and I switched back to Windows from Linux a little while ago. Is there software I can use that automatically rearranges these playlist files? Or do I have to manually rearrange them? Here's a screenshot of what I'm talking about:

https://preview.redd.it/u0nvlr9aqied1.png?width=914&format=png&auto=webp&s=4676a06e1087d4560bba3f50c8db55f0dcebd09b

I want to be able to change these to the Windows file structure (and change the location of these files,) and I was wondering if there was a tool/player that can do that. I've done some internet searches already and wasn't able to find anything, which is why I'm posting this.

The original post: /r/datahoarder by /u/geoffbaker344 on 2024-07-24 17:48:04.

Asking here to confirm my understanding. Trying to determine if an external hard drive is a possible solution if:

1. I am trying to avoid monthly cloud subscription fees
2. I don’t access the data often, just need somewhere for it to go since it exceeds local storage capacity, but would like to access it freely if needed

Also, after searching I noticed a lot of emphasis on backing up external hard drives. Can this be done without a monthly subscription?

The original post: /r/datahoarder by /u/nukem2k5 on 2024-07-24 17:39:44.

My main computer is a 12 year old Intel 4790k with 16GB while my server has a Ryzen 2700X and 96GB.

I'm likely going to switch my server from unRAID to Win11 + snapRAID + Drivepool (even though unRAID has some great features). I don't really use my server for anything besides network file sharing (for myself) these days (with a few VMs for running cloud backups and stuff).

It makes no sense to have my vastly superior hardware dedicated only as a file server and some mild virtualization. I also don't really want to just swap the machines (use my old Intel rig as my server) because of fewer SATA ports, fewer drive bays, limited virtualization capabilities.

If I consolidate to using solely my Ryzen machine for everything (gaming, general use, file sharing, backups, etc.), is that foolish? Obviously, with only one machine, reboots and random issues will affect the uptime of my server functions/applications. Anything else I should consider?

The original post: /r/datahoarder by /u/Splitsurround on 2024-07-24 17:37:12.

I paid for a year of softraid, and so far it’s working fine (and I understand that this is the most important thing). But I thought part of what I’m paying for (the difference from the free version) is more “monitoring”.

I am new to this software so this could totally be on me, but I just don’t see where there’s any useful information. It shows in tiny text how much free space I have…and that’s it.

I was kinda hoping for some more info, a disk health meter or something. It just feels totally absent of information to me. Am I not looking in the right place (there’s nowhere to really look lol), or am I just expecting too much?

The original post: /r/datahoarder by /u/nukem2k5 on 2024-07-24 17:26:39.

My server has some WD Reds and I recently bought some Ironwolf to replace them. The server is always on but I only seldom have any disk traffic (mainly syncing my phone to the machine as backups).

Is this infrequent use (long-term data storage) negatively affecting the performance or lifespan of the NAS drives?

The original post: /r/datahoarder by /u/Kqyxzoj on 2024-07-24 16:44:56.

Is it possible to view archive.org DMCA takedown requests for a given url? The typical scenario would be this:

• You want to check the older version of a webpage.
• So you check the archive.org list of snapshots for that specific url.
• "Hey wait a minute, last time I checked the list of snapshots for this url there were way older snapshots."

So how can you check if older snapshots have been deleted. Either for that specific url, or for the entire domain. Github for example has this repository with text of DMCA takedown notices. Is there something similar for archive.org?

The original post: /r/datahoarder by /u/dstarr3 on 2024-07-24 16:01:15.

I've been curious regarding what fellow data hoarders think about these for a while. They also make an Industrial line of MicroSD cards that have error checking, but those are obscenely expensive per GB for use as bulk storage. But there are 512GB high endurance cards the size of your pinky nail for around $50 that aren't awful in terms of cost if you really need the density. For instance, in a safe deposit box or any number of creative hiding places around the house.

Thoughts? How much trust would you put into high- or max-endurance SD cards?

The original post: /r/datahoarder by /u/BestestBeekeeper on 2024-07-24 15:57:14.

TLDR; What if I mounted my entire server rack on rails to make it easier to slide out and access the back & sides?

Hey everyone,

So I have a bit of a weird one here. I recently setup a server rack in my garage, graduating from the office due to heat and noise (and wife-approval-factor).

It took me a long time to find this rack (used) as I needed something a decent size, but not a full rack. Height was never an issue, but the depth was a big factor. I have SuperMicro chassis' that are 27" deep, so I wanted something that would house them, without sticking out twice the depth of the 24" cabinets that this would be installed around.

The rack I was lucky enough to get is a 32U H x 24" W x 30" D. Perfect for all my equipment and not too deep to be cumbersome. I would up installing it on a 45 degree angle in the corner of my garage with 24" deep cabinets flanking it on either side. It fits perfectly and lines up exactly with the cabinets as though it is meant to be there.

Now the issue....

With a 2000VA Eaton UPS, 2 EBM's, 2 SuperMicro JBOD's, 2 4U Servers, + Misc Networking Equipment, KVM, etc, I have no doubt the entire rack is pushing 500-600lbs if not more. It is very difficult to roll the rack out of its location (the floor is rubberized gym flooring) when I need access to the back or sides (Which isn't often but often enough it seems to be annoying).

So I'm considering these or something similar:

Heavy Duty Drawer Slides 500lbs Ball Bearing Drawer Slides 32inch Long With Lock | VEVOR CA

Would this be a potential solution to my issue? Does anyone have a better solution they would recommend? Has anyone else found a good layout for a server rack when incorporating into standard depth cabinets and such?

The original post: /r/datahoarder by /u/ValouMazMaz on 2024-07-24 15:16:24.

I would like to get your opinion on what the practice is regarding expanding storage. My storage is almost full and I ordered a few HDDs at the same time (got a good deal on them).

Now the question is : is it in my best interest to put all of the new disks in my storage bay, or should I better keep the disks I don't immediately need offline and only install them when the time comes ?

I am running snapraid + mergerfs (mfs), which means that every disk would be written to if installed.

The original post: /r/datahoarder by /u/-Bionicman- on 2024-07-24 13:26:46.

I have about 20-30 folders of old documents from university (mostly typewritten with illustrations and handwritten notes).

I recently bought a duplex scanner (Epson DS-410) and am still pondering the choice of file format, DPI setting, and OCR. I would like to archive the documents in the best possible quality and then dispose of the folders.

Question 1: I am currently thinking about PDF/A and wondering if it is the best choice or if .tiff or .png would have an advantage? Unlike PDF, I noticed that I can't choose a compression level with PDF/A. Is PDF/A lossless?

Question 2: Can multiple .tiff or .png documents also be easily converted into a PDF, or is this not a good idea?

Question 3: For the PDF or PDF/A file format, my scanner has the option to create a searchable document (OCR) directly. Is this recommended or would it be better to add OCR afterwards either by using a specialized tool or to import it into paperless-ngx (which I don't have yet, probably will use it in near future) ?

Not sure if there is difference in OCR, if there is something as "bad" or "good" OCR.

My goal is: To create the scan in the best possible file format. If .tiff or .png is preferable as an archive file, the option to easily convert it to PDF should be available; otherwise those formats would not be an option for me.

I would appreciate your advice.

The original post: /r/datahoarder by /u/Dick_Sucker___69 on 2024-07-24 12:12:08.

i found a slightly used toshiba 18tb hdd for 190€ 10.55€/tb here: https://www.ss.com/msg/en/electronics/computers/completing-pc/hdd/mxcpf.html

is that a good deal? (i dont have much experience with storage drives)

The original post: /r/datahoarder by /u/maycauseanalleakage on 2024-07-24 10:04:29.

Hi, I'm after software (ideally on a Mac) to find duplicate images but also automatically prioritise which ones to keep based on rules (e.g. if I have a copy in the 'Camera' folder and in 'Cats' then prefer the 'Cats' one). Before I go reinventing the wheel and writing something myself, is there something like this already out there?