It's A Digital Disease!

The original post: /r/datahoarder by /u/stoopidoMan on 2024-08-07 17:45:06.

I am on WinRAR 7.01 64bit

I compressed Folder [Test A] and [Test B] (Both folders are 22.2 GB) I compress them with these settings:

1. Archive format: RAR
2. Compression method: best
3. Dictionary size: 1 GB
4. Archiving options:
   1. Create solid Archive
   2. Test archived files
5. Archive features:
   1. Use BLAKE2 file checksum
   2. Save identical files as references

the compressed result.rar file was : 4.74 GB

I changed folder name [Test A] and [Test B] to [Test C] and [Test D]

And then added [Test C] and [Test D] which is the same folder but with different name to result.rar with the same settings and the result was: 9.49 GB

I did the same test, but without [Save identical files as references] and with larger dictionaries and it gives me the same result. it basically double the file size.

Is this a bug in WinRAR, because I think sometimes it works and sometimes it doesn't depending on the folder size.

or this is how WinRAR works, so when new files are added the compression algorithm applies only on the new files.

Is there anyway for it to compress the whole thing when you add a new files, basically considering the whole content of files archive.

These folders are online account backups so it contains photos and data that are mostly identical and I don't want to extract the archive and then compress it again every time I rather add it, but if i do that the archive grow exponentially.

The original post: /r/datahoarder by /u/OhItsuMe on 2024-08-07 16:37:41.

For images, video, and audio? Those are the primary formats I'll be working with. I looked around but then I just got more confused.

Most I will archive losslessly compressed, but some image files I don't really care if they are lossy (the quality is already not that great), and I'm just trying to get the file size down.

As for audio, I would like to have both options (lossy and lossless).

And finally which containers do you usually like for these formats? I guess technically that doesn't matter very much, but still interested.

The original post: /r/datahoarder by /u/FriendsCanKnowThis1 on 2024-08-07 16:00:24.

Hello. I purchased a product from Home Depot, and then the product page went down (this seems common for Home Depot when they discontinue products). I am trying to sell the product because it doesn't fit my needs, but I can't get any photos so I can list them. The best I can find are low-quality thumbnails from Wayback Machine. Thank you!

I am looking for:

• Full resolution photos of this exact product
• Product dimensions (I have no idea how long, wide, and deep this product is whatsoever)

Product information:

• Husky 4-Piece Extra Wide Heavy Duty Welded Steel Garage Storage System
• Store SKU #1006420193
• Internet #317133079
• Home Depot product page (taken down): https://www.homedepot.com/p/Husky-4-Piece-Extra-Wide-Heavy-Duty-Welded-Steel-Garage-Storage-System-HTC411020-EX/317133079
• Wayback Machine link: http://web.archive.org/web/20230801000000*/https://www.homedepot.com/p/Husky-4-Piece-Extra-Wide-Heavy-Duty-Welded-Steel-Garage-Storage-System-HTC411020-EX/317133079

The original post: /r/datahoarder by /u/vahking on 2024-08-07 13:15:01.

Hey all, i was recording a podcast with a lumix gh5, the battery was about to die and I pulled it out to replace it forgetting to end the recording, and all I have is an mdt file, and the last 4 minutes which I recorded with a fresh battery. I have a 20gb .mdt file which wont open, ANY help is appreciated.

The original post: /r/datahoarder by /u/Umbranoturna on 2024-08-07 12:46:14.

Im looking to buy an 8TB HDD to store clutter, some game recordings and some movies.

I dont plan to move files around a lot, just plan on adding stuff to the pile.

The drive will go in my daily use Desktop.

Im not sure how much SMR with the Barracuda will impact performance if i for some reason do have to move some stuff around.

The Barrracuda is 125e, the Iron wolf is 190e.

I cant tell if the upcharge is worth it for me, when i dont want to put the drive in a NAS.

The original post: /r/datahoarder by /u/digilink on 2024-08-07 12:26:56.

I have spent the last two days (today will be attempt number three) trying to get warranty replacement for a failed Exos X18 18tb drive. I bought the drive last March, so still well within warranty and thought this should just be a simple warranty replacement, so I start down the path to initiate it online and this is where the frustrations begin.

I tried to login to https://www.seagate.com/support/warranty-and-replacements/start-claim/, provided my credentials, saw a spinning animation... then nothing but a blank page. Hmm that's weird.. let me try a different PC/internet connection, disable all my ad blockers etc... nope... same results.. well shit.... ok let me contact support so I pull a chat up... disaster now ensues.

The first rep on Monday opened a case for me to get this drive replaced and agreed it was covered under warranty (and I even checked with their tool) and I was to "receive an email" with further instructions. Well... no email ever came, nothing in spam folder etc. I reach out again yesterday with said case number... after going back and forth with this rep for 1.5 hours, he proceeds to tell me the drive is no longer in warranty and to provide proof of purchase... hmm ok, well that's frustrating, it was in warranty yesterday, but fine whatever, here it is and I provide a copy of the Amazon invoice where I got the drives from. I then get told it will take 24-48 hours for the warranty to be updated then I can request a replacement. I am furious at this point and demand an escalation, which he did. I checked this morning and the drive is "magically" back in warranty again.

So my question is... has anyone else had a frustrating experience like this? I wonder if they are dealing with any of the crowdstrike fallout and their systems might have been impacted or something. Is it possible to call them and get support to get this going, or chat only? All I get on this page is spinning animations instead of a link to talk to someone:

https://www.seagate.com/contacts/

I'd just like to get my failed drive replaced, and having to jump through these hoops makes me hesitant to ever buy another Seagate drive ever again.

The original post: /r/datahoarder by /u/TheColorlessWitch on 2024-08-07 12:20:48.

Hi,

I have done quite a lot of research on still image (and video for that matter) and pure text compression.

I have find that JPEG XL (JXL 0.10.3) seems to be the best in term of ratio time/performance and ability to perform well in lossless context on major already compress image like .jpg and .png. The ability to not reencode but just transcode for jpeg input file is also nice.

Through test I have found that in lossless mode (-q 100), using the effort 7 is the best. Above just take too much time (especially the effort 10, more compress for lossless the the time needed increase exponentially).

I have test HEIC, TIFF, AVIF, WEBP, and I have also test a lot the still quite experimental WEBP2. WEBP2 is impressive but compress less and take more time (in lossless mode at less). It seems that this codec still need time to attain maturity. (But I find that on very small data size image it performs surprising very well, image less than 25KB for example).

For JXL, one can hope 20% lossless compression for already compress standard JPEG and 40 to 50% lossless compression for PNG. I need to precise that I find these percentage on manga image in grayscale and in color, in both case these are image that most likely tend to compress well due to a lot area with same color.

For text compression is more complex. I have test the well know format and algorithms like: ZIP, 7ZIP, Brotli, Zstd, BZip2, XZ, Gzip, ARC, and *PAQ. I didn't do a lot of test for the moment but I have found for me (for diverse type of text base file) that ARC > *PAQ > BZip2 > Brotli > Zstd

It is quite surprising that the now quite old ARC performs so well. I want to precise that it didn't compress a big file of tens or hundreds of MB but instead a multitude of ≃1MB TAR file. The ratio difference between all the compression algo isn't that much but still.

For better compression I read that Large Text Compression Benchmark. It truly show jaw dropping compression ratio but most often than not it's at the expense of the compression time needed and therefore the usability. And a lot of these algo seems or are abandonware (I had to go to archive.org to find the file for nanozip!) and were not means to be use in "production".

But there are some nice algo that seems reasonable in time, seems alive and could be use in "production". The like of: zpaq 1.09 BCM 2.03 BSC 3.25 (Maybe bsc-m03 0.4.0) pcompress 3.1

One can hope compression ratio from 40 to 80% on pure text-based file. That is not negligable, one can compress 1TB into maybe just 200GB. The saving in HDD space and then in money can be interesting.

What are your thought on the need to losslessly reencode/compress VS just leave the original as it is?

The original post: /r/datahoarder by /u/helloimjah on 2024-08-07 12:20:34.

I have done some research of this with help of GPT, but still would like to ask feedback from real people and people who are already doing this. My main concern is that, on paper things looks great, but when you get there in real life, some problems/issues happen that were never accounted theoretically/on paper.

I started hoarding around 4 years ago, and if 4 years ago someone told me I was about to store everything that I hoarded in DVD's I would have probably peed myself from laughing.

As of now I don't have anything huge, just couple of terabytes of recorded content that just sits on my external drives. Perfect way I imagine backup is when it is done at least 2x times so what I want is I have 1 set of everything (main backup) just sitting on my desk or somewhere safe, never touched unless, my 2nd set of backup (stored on hdd/ssd) fails or get corrupted, it would be only time, the main backup would be used to restore crashed/corrupted 2nd backup.

Because money is somewhat I issue for me, I can't afford having/buying new hdds for content I am hoarding then parallel buy more hdds for 2nd backup and then more for 1st backup. Drives also break and needs changing every X years, so I am fine doing it with hoarded content and 2nd backup, but I simply can't afford do same for 1st main/long term backup.

So solution I came up with is pretty simple, I currently have 2 external drives as of now I have 0 backup of them, so I will soon try to get 1 drive to fit them both, and it will be used as 2nd backup, and when I buy more drives, I can already use 2 external drives that I have for 2nd backup and they wont be main place I store hoarded stuff (Or it will exactly be place to store hoarded content and instead use new drive as 2nd backup still thinking on that). Only problem is, if I keep doing this, it is guaranteed that sooner or later one of my 2nd backups will fail, I would need to check HDDs constantly and they also would need to be powered at least a little bit so they wouldn't lose charge and that would be both cost expensive to keep gear of 2 set of backups alive for 10-20 years and of course would take some time so this is where Blu-ray dvd's come in.

GPT:

For long-term data storage with minimal risk of data loss, especially methods that do not rely on electric charge retention, the following options are recommended:

1. Optical Discs ================

Blu-ray Discs (BD)

• Longevity: Archival-grade Blu-ray discs (M-Disc) are designed to retain data for over 100 years.
• Advantages: Not dependent on electric charge, less susceptible to magnetic interference and mechanical failure.
• Disadvantages: Limited capacity (up to 100 GB per disc), slower data access.

My main concern for 1st backup was data lose because of idle/not being connected to power, blu-ray discs pretty much solves it. Only disadvantages I see are ones GPT already mentioned, assuming writing speed is like 20mb/s would need about 80-90 minutes to write 1 full disk and these speeds doesn't bother me much, cost wise I might pay for 1TB of blu-ray dvds as much or even more than 1TB hdd, but if it guarantees me 10-20 years of data storage when in idle and perfect environment and minimal maintenance why not?

The original post: /r/datahoarder by /u/Jastibute on 2024-08-07 09:59:44.

Hi everyone, I asked the question yesterday but I probably didn't provide enough information for a reasonable response. So here it goes.

I have a NetApp 4246 and I'm looking to get an HBA and some cables to connect everything up.

I don't understand how the maths works with these disk shelves. Each drive can do a max of 2Gbps. The disk shelf can house 24 drives, so that's 48Gbps. Each SAS cable supports 4 SAS lanes @ 750MB/s, that's 3GB/s and 24Gbps. So my maths suggests you need one cable per 12 drives. So a standard 2 port HBA card should use both ports going to the disk shelf.

Everywhere it says that you just need one cable to connect the HBA to a single IOM unit. The documentation on page 46 shows a single-path, single-controller scenario where they have multiple disk shelves daisy chained from ONE cable.

How does that work!? It seems like you're getting half the performance with just a single disk shelf, let alone daisy chaining.

The original post: /r/datahoarder by /u/steviefaux on 2024-08-07 09:58:53.

So, at work with have the big Ricoh MFDs. Despite being the admin I'm not an expert with scanners. Anyone know how to use them to scan the slides when you have no slide adapter. I stupidly thought just putting them on the bed would work, but of course it doesn't so I now understand why the adapter is used. The adapter forces the scanner to scan the image not the hold slide.

Its one of these IM C4500

http://support.ricoh.com/bb/html/dr_ut_e/apc/model/imc4500/imc4500.htm

The original post: /r/datahoarder by /u/ChuckleBerry5 on 2024-08-07 09:01:36.

I was shocked to find that Xtreme Download Manager and Internet Download Manager can't find and download multiple files from a single link—essential for any download manager. Tried Jdownloader 2, but it's slow, the interface feels ancient, and it requires premium for higher speeds. Looking for recommendations to download files efficiently in 2024.

The original post: /r/datahoarder by /u/ht3k on 2024-08-07 07:14:11.

I need external JBOD DAS options between 8-16 drives sized 3.5" for hard drives. I currently have a 8x16 TB zfs setup and I'm looking to expand my rack. Here is my list of requirements from most important to least:

1. Quiet as possible and or customizable fan curves without a ton of tinkering
2. Hot swappable compatible
3. Power efficient
4. 2U or 3U. I may be willing to shell out a bit more for future storage

I'm in the U.S., budget is around $500-800. If I need to pony up a bit more for all these features I will. If it's too much more than that then I'll settle for one close enough around that starting budget

The original post: /r/datahoarder by /u/True-Entrepreneur851 on 2024-08-07 03:29:03.

Hello Everyone. I am new to data, I bought a Synology NAS DS423+ with 2 HDD 8GB each, I am planning to do the following for budget constraints and setup my nas with SHR config.

I have a very first question as I see in the raid calculator that I can compare SHR or SHR-2 and SHR-2 provides much much less space than the first one. What is the explanation behind ?

Second point : I have two drives that are 4GB each that I am planning to use for the remaining two bays. Should I do something and add them or wipe data before ? Then in the future I might be willing to upgrade to 2x8GB and 2x12GB but 12GB drives being a bit expensive I would do this way : Next month : replace a 4GB drive by a 12GB. In 3 months: replace the remaining 4GB drive by a 12GB.

Does that make sense to you or should I process differently ?

Thanks.

The original post: /r/datahoarder by /u/blueman541 on 2024-08-07 01:19:03.

For a long time I've been using Noisemagic NMT-3 with my NAS. The controller board is tiny, fits in the corner of a 120mm fan, has a wire thermal probe which I tape to my hottest drive.

At 30C it runs fans at 5v then ramps up to 12v at 42c max. This is perfect for my use case since that is the ideal range for hard drive temps. Allows my fans to dynamically run dead silent in cooler days and high rpm during hot summer or data scrub task.

However, the company is out of business (noisemagic.de). This is the NMT-2 model listed on a reseller page. I can't find any other links for NMT-3. Bought mine from frozenpc many many years ago. Only difference is the NMT-3 has a external wire probe.

What options are the for fan controlling? Most of the ones I see now are either very bulky large, manual knob/dials or require installed software to control.

The original post: /r/datahoarder by /u/AnnieAnoles on 2024-08-07 08:44:16.

Hey folks.

I'm thinking of building a small home server for torrenting / jellyfin. Currently i'm just using my regular desktop pc, but this draws a lot of power (lowest i've ever gotten it was 90w, usually hangs around 300w) so it's only on for maybe 14 hours / day instead of 24. Here's any relevant information i can think of;

Usage: As stated before, it will be used for torrenting & a jellyfin server me & my friends use. I dont download that much but i am always seeding, so my drive is always active.

Budget: The buidget excl. drives is about €450. I may be able to stretch to 470 but no higer unfortunatley.

Requirements: Not many. Min. 4 drives, low power during use. No restrictions on size, aesthetics, sound, ect.

Component List: PCPartPicker Part List. For the cpu i've picked the i3 12100. It's a good balance between cost and performance that I can still buy new. I haven't picked a motherboard yet, from my understanding these have a significant impact on power draw as they could limit the c-state the system achieves. Here's a collection of boards that meet my miimum specs. Are there any stand out good/bad ones? Is there a board that suit my needs well but have overlooked?

Memory is whatever was cheap but didnt look like it would explode. Ssd is purley based on price, maybe go 512gb and get better quality one? PSu might be swapped with pure power 12m 550w, rm550x is hard to come by atm.

Other questions:

-Is there a good way to tell if a component will negativley impact c-state achievability?

-Do low c-states even matter since my drives will be seeding all the time?

-Are there any specific qualities i should look for in components for a low power build?

-Have I overlooked anything with my current parts list?

If you have any further questions i'd gladly answer them :)

The original post: /r/datahoarder by /u/agrajag9 on 2024-08-07 07:14:24.

SAS has a lot of features that make it objectively more attractive than SATA, for example dual pathing.

But are there other characteristics that are unseen/invisible? Everybody likes to talk about "reliability" but I'm curious how much of that is truth vs bias and advertising? "I paid more so I expect more"?

Do manufacturers actually use a different line for some spinning rust than other spinning rust when the mechanical bits are all the same?

The original post: /r/datahoarder by /u/Ghostconn on 2024-08-07 04:13:42.

I've been using truenas for a few years now and love it. I'd like to utilize the hardware as my main computer and install an operating system. I'll be importing my existing pool. I really enjoy the simplicity of truenas as it's built around a gui. Is there a similar software/application I can use to manage and monitor my pool and shares? I really don't feel comfortable using cli for this. I'll more then likely use Debian however I'm comfortable using all apt based distros and fedora.

The original post: /r/datahoarder by /u/Lazy_Essay_4348 on 2024-08-07 03:35:31.

So I am looking for some sort of drive where I can store whatever files I'd like, but it should be accessible from both my Windows PC (which stays plugged in at my parents house) and my M1 Macbook. Specifically I would like to be able to access the files from anywhere while on my Macbook.

The use case that I am mainly thinking of is whenever I need to share files from my mac -> windows or vice versa. I am a university student so I have a lot of files that I would like to be able to access on both easily, rather than having to download two sets of the same slide set just so i can have them on each device. There are a other use cases but accomplishing this would ensure the remaining use cases are fulfilled.

I spoke with a friend of mine who goes by the name of ChatGPT. Chad recommended that I either use a NAS or a personal cloud storage solution such as NextCloud or OwnCloud. Another one of my friends also recommended me to use a NAS. I'm leaning towards getting a NAS but I am going to be open minded.

I was looking for some advice or recommendations on what I should do to accomplish the task at hand. I am open to any new ideas so please feel free to write whatever you think might aid me along my journey.

Edit: I forgot to state how much storage space I am looking for. Currently I have slightly less than a TB of files on a portable hard disk. I would like to transfer these files to whichever system I end up choosing. Hopefully start of with about 1-2TB and scale if needed.

The original post: /r/datahoarder by /u/bustergundam4 on 2024-08-07 03:15:34.

Bought a 600GB SAS drive instead of a SATA drive. Are there any cheap connectors I could get that would let me use it? ( Please provide links)

If not,does anyone here want it?I'd ship it you.

The original post: /r/datahoarder by /u/drew4824 on 2024-08-07 02:37:55.

I'm looking for a way to mass grab account upload details/metadata for entire accounts in an easy way. Essentially, all the stuff on the left side when viewing the actual content of the page from titles, tags, descriptions, etc. This isn't the link I'm trying to do that for but it would an account such as this.

https://archive.org/details/@vhs_collectors

The original post: /r/datahoarder by /u/Ralonset on 2024-08-07 01:31:32.

I've done some researching in this sub but I've been getting mixed answers about whether to use ext4 (because I use Linux) vs NTFS (I occasionally boot windows) vs one of the FAT systems (many people say not to use this though). I bought a synology NAS which I also plan to back up photos from my phone to, but I'm relatively new so I'm not even sure if that's how it works lmao. Any advice?

edit: yeah sorry for the confusion, I'm not trying to boot off of this thing, I'm just giving examples of the different OS I would be using to interact with it

The original post: /r/datahoarder by /u/Matt_Bigmonster on 2024-08-06 20:52:26.

Bit locker is very convenient, veracrypt is open source and have extra functionality.

I know it's mostly speculations but what encryption is more resistant to brute force attacks or have less chance of a backdoor?

The original post: /r/datahoarder by /u/Glass-Fix-4624 on 2024-08-06 14:17:48.

I didnt' even know that there was a distinction between SMR and other types of hdds before two days ago. Now that I'm building my first DAS, I heard people keep reccommending against buying and using SMR drives. The thing is, it looks like all my current HDDs and refurbished HDDs that I can get at reasonable price are all SMR. For now I've two 2tb drives, one 4tb drive, and one 1tb drive, and I've ordered one 12tb HDD as my main backup HDD.

But now considering the amount of bad opinions about SMRs, I'm starting to doubt if I'm doing something wrong by keeping data only on SMR drives. For now, I've experienced two HDD failures in the past 4-5 years

Also it looks like basically most of the HDDs are SMR? So I'll have to buy SDD to get CMR?

The original post: /r/datahoarder by /u/xxKEYEDxx on 2024-08-06 22:42:28.

I was recently gifted a Synology DS1817+ (8-bays) (Atom CPU).

I am currently running a Synology DS1019+ (16 TBx5) (celeron CPU) (SHR1) (Plex server) w/ DX517 (18 TBx5) (SHR1). I also have an old QNAP (10 TBx4) (Atom CPU) (RAID-5). Data is currently backed up from DS1019 to the DX517 and QNAP. All are on a UPS.

I plan on using the DS1019 & DX517 as the primary, and the DS1817 & QNAP for backup. I'm ordering 18 TB drives for the DS1019, and using the 5x16 TBs plus 3x14 TBs for the DS1817.

If the 8-bay was my primary, I'd do 2 without hesitation. But it's going to be the backup because it's older and slower.

So what's the verdict, 1 or 2?

The original post: /r/datahoarder by /u/Zealousideal_Rub5826 on 2024-08-06 21:55:29.

Hello, I am interested in replacing my Drobo with an UnRaid server. Does anyone have any suggestions for a small form factor case to store the drives? Something like the Drobo or the Synology 4 bay enclosure. I would like 6 bays if possible, without getting a full-size tower. The hot-swappable enclosures like Drobo and Synology are very appealing to me (I don't know if UnRaid supports hot swapping but something easy to get to would be nice). What type of case have you used for UnRaid?