It's A Digital Disease!

The original post: /r/datahoarder by /u/Hieuliberty on 2024-10-12 15:52:58.

I know WD Purple is for CCTV storing but it match my preferences:

• Only store video file (1080p movie, TV Shows. Mostly H.264)
• Ocassionally access.
• Watch alone, not share this drive to LAN or Internet.
• I just need this drive to work for 2 years. Will buy new media server later.

I'm considering between:

• WD Blue (WD20EZBX) | 2TB | SMR | 2 year warranty | $62
• WD Purple (WD23PURZ) | 2TB | CMR | 3 year warranty | $68

There's WD Blue model WD20EARZ but I can't find any shop sell it in my country.

Basically the most consideration is about CMR vs SMR. And I don't know if any bad things will happend if a "CCTV" drive attached to personal computer, which turn on/off 5 times a day. Is WD Purple expected to be running 24/7?

Thanks for any recommendation!

The original post: /r/datahoarder by /u/Relevant-Technology on 2024-10-12 15:33:31.

Can anyone tell me if these are good drives? Someone local is selling two 16TB Dell EMC Exos X16 drives. These show up as ST16000NM005G-2KH133. They were manufactured in Aug 2021. They have a total of 24 hours on them, and 12 power cycles. It says 16TB 512e, SATA 7.2k rpm on them.

Are these reliable? I currently have 2x WD red 8tb in my Synology nas and I'll most likely replace those or build a new proxmox/unraid server. Thanks.

The original post: /r/datahoarder by /u/BroodingSage on 2024-10-12 15:25:04.

I'm working on a creative project, and for that I extracted some game audio resource files to get some character dialogues in audio format. I've successfully obtained thousands of audio files, and the ones I need are definitely within them.

As you might have guessed, this has become a roadblock, I can't manually look through each audio file myself, so I'm looking for a software which can automate this process. What I have in mind is a program, to which I can give an audio sample in which only the required character is talking, and then it can filter these audio files using that sample.

Is there any program which can accomplish this?

TLDR: Need a program to segregate audio files with a particular voice from thousands of audio files, possibly using an audio sample as reference

The original post: /r/datahoarder by /u/RedTermSession on 2024-10-12 15:01:12.

The original post: /r/datahoarder by /u/dreamyrhodes on 2024-10-12 14:58:50.

I have a Icy Box like this:

https://www.amazon.de/ICY-BOX-IB-AC603a-U3-Adapter-Schutzbox/dp/B008S8PP6K

But it sometimes doesn't work, on SATA HDD but also on some SSD however I have at least one SSD where I know that it works. The drives that don't work on the USB adapter work in a computer and a NAS without issues (daily use).

The connector has been used in all cases on an USB 3.0 port.

Is the thing just too cheap or what might be the problem?

Edit: I already tried lsblk and also the disk management under Windows and the drive won't show up. Also the HDD doesn't spin up when I connect (although the blue LED on the adapter lights but doesn't blink indicating access).

The original post: /r/datahoarder by /u/locvez on 2024-10-12 12:38:19.

I've purchased two X20, 18Tb recertified drives from Server Part Deals and curious to the hive minds' thoughts on the performance. I have 4 X18, 18Tb drives which all have the following speed profile. Like, all 4 have this exact same line curve. Same speeds, same curve, everything, identical. Two drives were bought together and two other drives from different vendors at different times.

https://preview.redd.it/zu2rgaz8hbud1.png?width=793&format=png&auto=webp&s=6f6422aa9ed9087422d63a59cd014441bb1c1439

But my 2 X20 18Tb drives have the following speed profiles

https://preview.redd.it/9rn1ghxykbud1.png?width=794&format=png&auto=webp&s=1f9bd8310c89e0563ea018e69135c2269271f08f

https://preview.redd.it/bm0e86f0lbud1.png?width=783&format=png&auto=webp&s=6dbeedabaa469ff4c04d993ca47b306495ba3356

Is this anything to be concerned about? I started to preclear them in Unraid and one's currently sitting at 58% and the other is at 37% - Both started at the same time. I ran the tests a couple of times to make sure it wasn't a one-off. I made the mistake of running the X18's whilst dockers containers were running and noticed weird peaks and troughs until I turned them off.

The original post: /r/datahoarder by /u/NaoPb on 2024-10-12 09:44:04.

What would you recommend for a network card for my build? I have a 16x PCI Express port available and I'm looking for an affordable but good quality (enterprise grade?) network card.

[edit] Welp, I meant PCI Express card, not PCI-X. I'll see if I can have the title changed.

The original post: /r/datahoarder by /u/CONSOLE_LOAD_LETTER on 2024-10-12 09:21:30.

The original post: /r/datahoarder by /u/jdrch on 2024-10-12 06:01:08.

Spacedrive has been posted about before, but at the time [had no downloads](https://www.reddit.com/r/DataHoarder/comments/ueoaz4/comment/i6oe70i/.%20Now%20it%20does,%20and%20is%20available%20for%20Windows,%20macOS,%20and%20Linux%20(.deb%20file).

It has 2 superpowers:

1. The ability to generate media previews recursively for a folder hierarchy. This allows you to view your entire Linux ISO collection in a single gallery, regardless of which subfolders the media are in or whether or not they're folderized
2. The ability to tag files and display media previews of all files with a particular tag at once. This has the same effect as the 1st point above but allows you to view files from multiple folder hierarchies, e.g. D:\Videos and X:\Videos 😉, in the same window

Support for cloud storage will be coming soon, which means 1 and 2 above will be extended to them also. Also incoming are Android and iOS apps.

The original post: /r/datahoarder by /u/AdrianDoodalus on 2024-10-12 05:41:42.

Recently got some Netapp DS2246 shelves populated with 3.8tb SSDs. Everything works fine but it currently takes up 8U of rack space and a shitload of power.

Would like suggestions on a topload enclosure with at least 90 drive capacity. I've seen a few that might work but im not really sure what all's out there since i've never went shopping for something like this.

Ideally something with SFF 8644 plugs since I have a ton of those cables lying around.

The original post: /r/datahoarder by /u/SocietyTomorrow on 2024-10-12 05:13:42.

What is the dumbest thing you've ever done when you had more free space than ideas to use it on? Perhaps it was the first dumb thing you did, or did you upgrade to industrial grade stupid?

I'll start with mine.

I was still really new to datahoarding (circa 2011), and set up my first proper NAS ("ooh, ahh, I have 72TB now!") after having a small mountain of external USB drives. Before I copied everything over, I decided I wanted to try making an archive of all the news I could find in case someone ever wanted to go and start censoring the web someday (foreshadowing?)

I found out about Usenet, which I'd never heard of or used before, because of a site called GigaNews, and said to myself "this sounds like it would be a massive database of all the news in the world!" so of course without doing any research, paid for an account, downloaded the client, set up a directory on my NAS, and immediately clicked "Download All Headers" and happily let it sit for half a day seeing it was grabbing almost 1TB of data (surely that's all the news, after all, it's just news sites)

Half a day obviously was not enough time to download that much with ISPs of the time, of course. I only waited half a day, because that was exactly how long it took for me to have my GigaNews account banned, my internet turned off, and phone calls and later letters sent to me about a few dozen THOUSAND DMCA violations, prompting me to consider torching my house, passport, and hopes for the future, in order to flee to some non extraditing country. It all ended up OK in the end, though I never was able to get back on GigaNews, my ISP changed and my response to the DMCA letter was accepted and no penalties or any other trouble came of it, as I was able to prove I literally just downloaded headers and no actual content so thus didn't actually "steal" any pirated media that may have been there.

The original post: /r/datahoarder by /u/Pretend_Compliant on 2024-10-12 03:36:38.

I'm in a critical situation with a Google Takeout download and need advice:

• Takeout creation took months due to repeated delays (it kept saying it would start 4 days from today)
• Final archive is 5.3TB (Google Photos only) was much larger than expected since the whole account is only 2.2 TB and thus the upload to Dropbox failed
• Importantly, over 1TB of photos were deleted between archive creation and now, so I can't recreate it
• Archive consists of 2530 files, mostly 2GB each
• Download seems to be throttled at ~15MBps, regardless of how many files I start
• Only 3 days left to download before expiration

Current challenges:

1. Dropbox sync failed due to size
2. Impossible to download everything at current speed
3. Clicking each link manually isn't feasible

I recall reading about someone rapidly syncing their Takeout to Azure. Has anyone successfully used a cloud-to-cloud transfer method recently? I'm very open to paid solutions and paid help (but will be wary and careful so don't get excited if you are a scammer).

Any suggestions for downloading this massive archive quickly and reliably would be greatly appreciated. Speed is key here.

The original post: /r/datahoarder by /u/Axe_zilla on 2024-10-12 02:19:55.

Greetings,

I am experiencing issues with my data and am trying to find a way to fix my shit.

To get things out of the way here's my admission : I am a bad boy. I'm dumb and I was lazy and now I'm paying the price.

I have been backing up my archive using a method that I know isn't approved, I get it. I was backing up my artwork and music/media simply by copying files to other drives. No checksums, no RAID, not ZFS, just dragging files over on MacOS and hoping for the best.

OK, that's all for the excuse as to why I'm here. Now I'm looking for a remedy before I properly set up my archives with checksums or ZFS/whatnot with healthy files that are backed up properly.

I am trying to correct for my sins as I have files that appear to be there but won't open. Some folders are there on the hard drive but when I load them into, say Adobe Lightroom, the folder doesn't even show up and I can't open them from the finder even though it says they are there and this is not happening on a dying hard drive.

I believe the remedy is to compare folders from old backups to current ones to find which files are corrupted. At the moment, I'm running Meld on Ubuntu to compare, as well as using CCC on Mac to see which one works best with the minimal amount of terminal use but I don't think I'm doing it right.

I have probably half a dozen to a dozen copies of each directory on various old drives spanning a solid decade, and I'm looking for a proven route to compare these against each other to hopefully find drives/instances that are the healthiest.

Without having anything in place like checksums or whatnot, how would you recommend I go about doing this? I don't need to check every file against each other, I believe I just need to find the healthiest drives. I think if I find the archive where the damage began, I can use the drives before that to rescue my work.

I'm running MacOS, Windows and Linux, so I'm open for any workable solution and I come to you in search of wisdom. Feel free to trash me for my errors, but hopefully I'll get some guidance as to how I should go about this.

At least let this be a lesson to anyone out there who is considering making the same mistake I made. Do it right, take the time now because to do it later will cost you much more than what it takes to plan properly.

TIA

The original post: /r/datahoarder by /u/koboldtime on 2024-10-12 00:13:45.

I'm looking for a simple hard drive enclosure for a single internal hard drive to be used externally. My current external hard drive that holds my plex library is reaching the end of its life (clicking), so I'm prepping to buy a new one, but would like to plan ahead a little. I'm not currently able to, buy extra drives for raid or a nas, but would like to have both in the near future. To this end, I feel I'm better off buying an internal hard drive, my plan is to go with a seagate ironwolf with 8T, same size and brand as my external hdd which I've been happy with, but I need something to put it in. Ideally I wouldn't spend more then 50$, but might be able to go up to 75$ at the most. I don't need fancy features just need to plug it in and maybe keep it cool. I saw someone suggest the Sabrent ones, are they good? Thanks in advance!

The original post: /r/datahoarder by /u/imsodumb321 on 2024-10-11 23:52:36.

To make a long story short—I've been dealing with some pretty frustrating neurological issues since June that make it difficult for me to look at a computer screen without splitting headaches and vertigo. For whatever reason, this isn't an issue when it comes to mobile devices, like my phone and iPad Pro.

Media archiving is my hobby, so I'm trying to find ways I can keep at it while I'm stuck in brain jail for god knows how long. I've tried a few different apps, but I really miss the simplicity of WF, especially with how you can just input a url and it finds and downloads all the photos for you without having to scroll or deal with a million pop up ad. Anyone have any tips?

The original post: /r/datahoarder by /u/Unusual-Doubt on 2024-10-11 23:19:33.

I have installed this card on my 2nd PCIe slot. MB: ASUS B450 PRIME PLUS

Card name: Fujitsu 9300-8I LSI SAS3008 12G HBA IT Mode ZFS FreeNAS unRAID 2*SFF-8643 US

I bought this off eBay and the seller mentioned that this is in IT Mode already. I have managed to install the MegRAID Management Software and am able to see the controller in that sw. Here is the screenshot.

However, I attached 2 SAS disks to P1 and P2 wire along with the SATA power cable. I dont see the disks spinning when I touch them. So definitely they are not receiving any power or the controller is not seeing them not sure which one.

The "GoTo" menu only had Controller enabled. Rest of it was disabled.

Storage Manager Screen

I then downloaded sas3ircu and when I run that command I get this error :

H:\SAS3IRCU_P16>sas3ircu list

Avago Technologies SAS3 IR Configuration Utility.

Version 17.00.00.00 (2018.04.02)

Copyright (c) 2009-2018 Avago Technologies. All rights reserved.

SAS3IRCU: MPTLib2 Error 1

Any help is highly appreciated....

The original post: /r/datahoarder by /u/PandFThrowaway on 2024-10-11 22:18:56.

Pretty straight forwaard set up. Server connected to DS4246.Has been running for years no problem. I was pulling on a tangle of cables and the SFP+ got pulled out from the DS. How do I get it back? Is it ruined? Also seeing a couple drives not lighting up.

The original post: /r/datahoarder by /u/IveLovedYouForSoLong on 2024-10-11 21:56:39.

I’m developing a lossy document format that compresses PDFs ~7x-20x smaller or ~5%-14% of their size (assuming already max-compressed PDF, e.g. pdfsizeopt. Even more savings if regular unoptimized PDF!):

• Concept: Every unique glyph or vector graphic piece is compressed to monochromatic triangles at ultra-low-res (13-21 tall), trying 62 parameters to find the most accurate representation. After compression, the average glyph takes less than a hundred bytes(!!!)
• **Every glyph will be assigned a UTF8-esq code point indexing to its rendered char or vector graphic. Spaces between words or glyphs on the same line will be represented as null zeros and separate lines as code 10 or \n, which will correspond to a separate specially-compressed stream of line xy offsets and widths.
• Decompression to PDF will involve a semantically similar yet completely different positioning using harfbuzz to guess optimal text shaping, then spacing/scaling the word sizes to match the desired width. The triangles will be rendered into a high res bitmap font put into the PDF. For sure!, it’ll look different compared side-to-side with the original but it’ll pass aesthetic-wise and thus be quite acceptable.
• A new plain-text compression algorithm 30-45% better than lzma2 max and 2x faster, and 1-3% better than zpaq and 6x faster will be employed to compress the resulting plain text to the smallest size possible
• Non-vector data or colored images will be compressed with mozjpeg EXCEPT that Huffman is replaced with the special ultra-compression in the last step. (This is very similar to jpegxl except jpegxl uses brotli, which gives 30-45% worse compression)
• GPL-licensed FOSS and written in C++ for easy integration into Python, NodeJS, PHP, etc
• OCR integration: PDFs with full-page-size background images will be OCRed with Tesseract OCR to find text-looking glyphs with certain probability. Tesseract is really good and the majority of text it confidently identifies will be stored and re-rendered as Roboto; the remaining less-than-certain stuff will be triangulated or JPEGed as images.
• Performance goal: 1mb/s single-thread STREAMING compression and decompression, which is just-enough for dynamic file serving where it’s converted back to pdf on-the-fly as the user downloads (EXCEPT when OCR compressing, which will be much slower)

Questions: * Any particular pdf extra features that would make/break your decision to use this tool? E.x. currently I’m considering discarding hyperlinks and other rich-text features as they only work correctly in half of the PDF viewers anyway and don’t add much to any document I’ve seen * What options/knobs do you want the most? I don’t think a performance/speed option would be useful as it will depend on so many factors like the input pdf and whether an OpenGL context can be acquired that there’s no sensible way to tune things consistently faster/slower * How many of y’all actually use Windows? Is it worth my time to port the code to Windows? The Linux, MacOS/*BSD, Haiku, and OpenIndiana ports will be super easy but windows will be a big pain

The original post: /r/datahoarder by /u/Fit-Pumpkin-5727 on 2024-10-11 21:27:37.

the micro SD cards do offer a 10-year warranty period at least in the USA

I have noticed that Sandisk and Samsung have 1TB or larger micro SD cards for under $100

https://www.pcworld.com/article/2487395/this-speedy-1tb-samsung-microsd-card-is-back-to-its-best-ever-price.html

is it safe to backup important data in a SD card like this or do you still prefer a traditional SDD/HDD more ?

The original post: /r/datahoarder by /u/RadicalRingtail on 2024-10-11 19:57:44.

some context: i bought 2 new 16tb seagate ironwolf pro drives about a year ago for my NAS when i was building it to use for its main storage pool, had one fail on me 5 months into using it, got it RMA'd from seagate since it was under warranty, and just last night i had that exact replacement drive start to fail on me, only 10 months into using it. originally went with seagate cause i had good experiences with their drives in the past + theyre usually a bit cheaper than brands like WD, so this was a bit of a surprise to me, considering this is probably the first time ive had this bad of luck with HDDs failing on me

tl;dr: high failure rates/bad luck when buying new seagate ironwolf pro drives, not looking to buy seagate drives again

that being said: are there any brands & models that would be worth checking out instead of stuff from seagate? ive seen a lot of recommendations on here + other related subreddits saying WD Red drives are the way to go, also seen some recommendations for WD Ultrastar and Toshiba drives too, though im curious to see what others might recommend going with as replacements and why

The original post: /r/datahoarder by /u/sxl168 on 2024-10-11 19:55:52.

I'm checking to see if maybe someone might have a solution to my IBM Full Height SCSI drive issue. I don't think there is any kind of hardware problem as using the IBM ITDT software, the drive will load and eject tapes fine when commanded to and the eject button works fine. What happens is that when pushing a tape into the drive, it will not automatically pull the tape in and spool. Same with eject, it will unspool but the tape will sit there until the eject button is pressed or the eject command given in ITDT. The drive was probably in a library and some kind of mode was set on the drive and I don't know how to get it out of that mode.

The original post: /r/datahoarder by /u/SkyBotyt on 2024-10-11 19:49:38.

Hey everyone, Trying to find a more long term solution to my data hoarding problems. I am a creative and a video editor/videographer. I deal with huge files constantly, my weekly data usage could be measured in the TBs. Problem is, I am dirt broke, I do not have the budget for any full blown RAID or NAS solution. Currently I just have a bunch of 10-20tb WD elements drives. The problems I have with the current system is that there is a bunch of them, and I dont have any copies, which means that if one fails, its gone. (i do have the most important stuff copied, but losing the rest would still be pretty bad.)

The solution I've come up with is to get a 2 bay docking station for Internal HDs, then get two sata drives and put them both in, then have them mirrored. Then when I fill them up, I swap them out for a new set of drives, label the sata drives and put them in storage. What I like about this solution is that It gives me the option to expand into a RAID/NAS using those drives in the future, and when I need to use two drives at once, I can take out the mirrored drive, and plug in an old drive, without needing to swap out the current drive. And the sata drives will be cheaper (maybe I could even purchase Refurb drives since they'll be mirrored?), and store more compactly then the elements will. Are there problems with this idea? if not, would something like this docking station work? If so, why? and what would be a better solution?

Edit: Thanks for all your responses! what ive realized with all this is that I am actually ok with my current data management, while its not ideal, It doesnt make financial sense for me to invest in this just for it to be more convient, what I actually want is backups and redundency. So now my real questions is: What is the best solution to ensure the safety of my data for as little money as possible?

The original post: /r/datahoarder by /u/jedix123 on 2024-10-11 19:47:24.

The original post: /r/datahoarder by /u/Fit-Pumpkin-5727 on 2024-10-11 18:57:08.

I want a 5400 rpm type of HDD (8/10 TB without helium)

CRM only...BLUE, RED PLUS or IRONWOLF.

( I don't care about TOSHIBA because of their crappy after sales support)

which one DOES NOT feature the annoying head parking function ? is it possible to disable it with WIDLE3 ?

I have already checked their specs sheet.

do load/unload cycles indicate the disk head gets parcked every 8s or so ? no way do I want to deal with this if I can't turn it off.

if it's not posible to disable the head parcking I will pick a 72000 rpm HDD maybe a Seagate EXOS or WD datacenter

The original post: /r/datahoarder by /u/little_somniferum on 2024-10-11 17:51:01.

Only this sub can give me the best answer.

My friend is a freelancer photographer for newspapers. He's growing old and inevitable becoming a dinosaur with technology, but does great photography. He has these old Lacie externals running 24/7 and they contain about 500TB. I think building a small local server with SSDs and maximum capacity is a better solution but I'm in the dark here. Our friend has a monthly input of 1TB raw photo material and needs to store everything. Another issue that he doesn't want to see is that he has to plug in every device which is max 4TB to find what he needs.

Anybody willing to do some input would be much appreciated. I'm in Europe. What does he need?