It's A Digital Disease!

The original post: /r/datahoarder by /u/ptoki on 2025-01-05 23:00:38.

Long story short: For some time now I know that in quite many use cases the ssds arent trimmed as people assume they should be. And that probably impacts their health, longevity and performance.

I had few conversations about this with a number of people and there is not much understanding to if this happens and how.

So below is a set of performance test graphs. They are from linux Disks tool. But you can test your disk in almost any tool which is showing the performance in different areas of the disk. Just read performance, you dont have to do writes to see this.

https://imgur.com/a/zzA0rSw

Unfortunately I have my drives healthy but the last graph is showing how bad performance would look like. Basically very rare spikes to high throughput figures or longer dips into slow performance areas for longer areas or frequently.

My question/request is:

Test your ssds and let us know if you see that steady even performance with occasional dip or your performance is slow or erratic.

I can expand this topic further if there are questions but here is the majority of findings about this topic:

https://old.reddit.com/r/DataHoarder/comments/1hqbbma/i_recently_got_my_hands_on_05pb_of_drives_50x/m4pyuij/

The original post: /r/datahoarder by /u/AnyaLikesPeanuts on 2025-01-05 22:40:25.

I have a 1TB SD card that I purchased directly from SanDisk, and its speed seems to have gone down recently. When it was new, the read speed was around 90MiB/s, which is closer to 40MiB/s now. It has had ~900GB written to it, and I noticed it getting slower after I deleted around 100GB of files off it.

The original post: /r/datahoarder by /u/soundsinmyears on 2025-01-05 22:24:17.

I have 8-10 WD Red HDD's ranging from 10TB-14TB

For some reason they've all began failing one by one... first it was just two then a week later a third, then a fourth... my computer will recognize them but says the disks aren't mountable so I can't access any data off them. I'm using them as external USB devices basically, archiving video footage that sometimes i need to access 1-2 months later. I access the drives by using a USB-C to Sata adapter chord, which i thought was the issue so I switched to a Sabrent dual HDD bay to USB-C. But the same issues persist.

But when I use Ease US Data Recovery it finds all the files... I just have to recover them and throw them onto an SSD.

I feel like there has to be something I'm doing physically thats causing all of these drives to fail one by one? Any advice? My only thought could be that when I transfer footage I use shotput pro to transfer the files then verify their integrity... and when I do this the drives get super hot. Maybe they're overheating then dying?

And even more importantly, how can i just pay money to never have to worry about this again? Should I just get a NAS setup? Any recommendations for directors storing large video files? Something with a USB-C connection, I'm not too worried about speed I can let it go overnight but is RELIABLE. And I don't need to worry about again. I can't afford the 3-2-1 method which so many TB's of video files

The original post: /r/datahoarder by /u/Phantom23001 on 2025-01-05 21:12:34.

Have a growing library of 4k media, and while I want to have the highest quality content across the board, I really only have access to one device that outputs to 4k. Having said that, I’ve been compressing my media and found that compressing a 4k Blu-ray takes up much more space on the same settings as a regular Blu-ray for 1080p video.

TLDR, How much am I losing compressing a regular Blu-ray vs a 4k one if I’m watching stuff in 1080p?

The original post: /r/datahoarder by /u/GroundbreakingOwl237 on 2025-01-05 21:08:12.

Tout d'abord, bonjour !

Un peut de contexte en premier temps !

j'ai récupérer il y a peut a mon travail des SSD format SAS de 3,2tb

Je souhaiterais les revendre pour me faire un peut d'argent, mais je voudrais d'abord vérifier qu'il n'y ai aucune données sur ces dernier avant de les revendre a n'importe qui !

j'ai donc naïvement commander un adaptateur SAS to SATA sur amazon et après les avoirs tester sur mon laptop, évidement rien ne fonctionne, j'ai donc fait des recherche sur reddit et j'ai trouver qu'il fallait avoir une carte pci-e particulière pour pouvoir les lires !

j'aimerais m'éviter des frais et je peut récupérer pas mal de matériels informatique divers et variés notamment des pc fixe et des station de travail ( serveur tour )

ma première question c'est, est-ce qu'il existe des carte mères qui peuvent lire les SSD SAS nativement ?

deuxième question et je précise que je n'y connais rien en serveur, puis-je récupérer un serveur format rack pour lire mes SSD ? puis-je par exemple me servir d'un Serveur comme d'un pc fixe, y installer windows et le connecter bêtement sur un écran ?

Et troisième question, même si je pense déjà connaître la réponse, existe-t-il un moyen pour lire ces fameux SSD SAS depuis mon laptop ? ce qui me faciliterais grandement la tâches !

je me permet de vous préciser que j'ai des connaissance en informatique surtout au niveaux hardware mais ca s'arrête au ordinateur domestique donc essayer d'être un peut vulgaire dans les explications ^^

Merci d'avance pour vos réponse,

PS : si je me suis trompé de reddit je serais ravis d'être envoyé ailleurs, je débarque un peut ici !

The original post: /r/datahoarder by /u/pheasantjune on 2025-01-05 20:52:24.

I can't figure out the actual speed differences between these two.

I'll be editing photos from the drives, Using RAID1 config over USB C. Not ssd, probably ironwood pro 5,400 drives.

Could anyone direct me on which would be faster to edit from? Thanks!

The original post: /r/datahoarder by /u/United-Virus6377 on 2025-01-05 19:36:58.

Hi everyone, Are you aware of a solution that would provide some smart auto-tagging on videos? I'm editing videos, and I have a lot of rush, and I would like to organize them, so when I need a rush where we can see "people working on computers," for instance, it would return me the videos.

Preferably self-hosted/open source.

The original post: /r/datahoarder by /u/colinlikesfood79 on 2025-01-05 19:05:40.

Is there some software to conveniently browse or compile into a webpage, the data scraped from twitter by gallery-dl? Reading tweet, comment, and re-post content in a json file is less than appealing, and matching images/videos with tweet text is not fun either.

thoughts?

The original post: /r/datahoarder by /u/SoupJaded8536 on 2025-01-05 18:49:06.

I have a 10 x 4TB pool that shows disks about 70% of capacity. It was set up looong ago as a 3 column parity pool, and I've just been adding drives as needed. Now I'm at the point where I can't physically add any more drives and plan to migrate to a new Storage Spaces 5 column parity pool.

Here's the question: If I swap out 3 of the 4TB drives for 8TB (or larger) drives will SS fill the 8TB drives to capacity (or near as matters)? If so, I plan to dismount 5 of the 4TB drives to start a new pool, then move data and drives one by one until I have everything on the new pool and 3 4TB drives to be used as on-the-shelf spares.

The original post: /r/datahoarder by /u/washo8 on 2025-01-05 14:31:37.

Greetings all!

Usually, after each Windows reinstall I used to move lotsa files that have somewhat importance to a folder "Last install", just in case i would need those files later. It has collected files from a lot of separate installs over the course of ~10 years like that. While I try to keep stuff sorted, some folders are still called "Desktop" that have even more folders recursively, just a huge mess but it saves my ass sometimes.

Now, I realize keeping such a huge messy folder with lotsa mixed type small files in one place is a mistake. Would love to back it up in external hdd. I would also love to isolate it by packing to one file container to not drag all of the little files that just slow the transfer down. This is where i get stuck.

What file container format should I choose that is not read-only? (I swear i will sort those files out someday!!)

ISO might be a good choice due to it able to mount so easily and built-in support everywhere. However, making an ISO truncates some filenames due to messy structure and makes the container read-only

Zip/7z might solve the read-only restriction but it would need to extract stuff to perform a deep file search or to interact with files properly (loading thumbnails for example)

Veracrypt is the solution I am using right now, it's like an isolated drive that's modify-able. However, it creates a slight inconvenience with backups as modifying the drive doesn't change the "modified" date of the container, making my file sync tools useless (i would have to compare by hashes instead and it would be MUCH slower). Also I don't really need the encryption so any other solution might work.

Perhaps there are any solutions that could even split the containers to smaller chunks, so changing small stuff in archive wouldn't need to copy the ENTIRE thing?

The original post: /r/datahoarder by /u/koomahnah on 2025-01-05 16:20:51.

Fellow hoarders, I'm new to the club and just set up my Optiplex 3080 Micro very recently with an Immich instance. I noticed something very weird:

$ for i in `seq 1 100`; do sleep 0.5 && sha1sum "Zdjecia-sync-bucket/Automatic upload/Mi 9T/Camera/VID_20241019_161631.mp4"; done
10402ffa2b16431b4794e9f57edbf5dfeaa704dc  Zdjecia-sync-bucket/Automatic upload/Mi 9T/Camera/VID_20241019_161631.mp4
...
8de18fc5be706b662742b126a083f7a8efed68c9  Zdjecia-sync-bucket/Automatic upload/Mi 9T/Camera/VID_20241019_161631.mp4
...
3fca013cab8606bae4952bdd1d4867170bf13f11  Zdjecia-sync-bucket/Automatic upload/Mi 9T/Camera/VID_20241019_161631.mp4

The hash of the file is changing even though there's no one changing that file, or at least lsof doesn't show anyone to be doing that. I'm the only user of that machine. I've noticed that change only by accident: some time ago I've written myself a simple indexing tool, basically go over all the files, sha1sum them, serialize the index as json to validate it later. Through this validation I noticed that this file got corrupted. Soon I've restored it from backup, but then another file (in the same directory) gets this corruption.

That freaks my out. The SSD drive i'm using is Kingston KC3000 Fury Renegade. Is that thrash? Or is ext4 filesystem to blame?

01:00.0 Non-Volatile memory controller: Kingston Technology Company, Inc. KC3000/FURY Renegade NVMe SSD E18 (rev 01)

I bought that used together with the Optiplex, but there were just ~20 power cycles on it. What would you suggest?

Side note: rclone didn't even notice the file went corrupted, presumably because it checks modification time rather than hash.

The original post: /r/datahoarder by /u/IllRememberThisUser on 2025-01-05 16:16:06.

I'm looking for a script or windows application to download a set of images every X minutes, saving them as the current time date.

The image changes at the same URL very 10 minutes. I have created a super basic script before but it had no error correction and would get stuck.

I found seqdownload but its old, ran for while and now can't fetch the images.

The original post: /r/datahoarder by /u/spudbynight on 2025-01-05 12:21:49.

Does anyone have a good source for refurbished drives in the UK or EU? I keep seeing people getting refurbished enterprise drives in the US from reliable sources who also provide some level of guarantee.

The original post: /r/datahoarder by /u/ActFew7218 on 2025-01-04 20:17:12.

Hi all,

New to this subreddit. I am wanting to archive lots of my own personal data and I have a few questions.

1. At the moment I have three types of data storage that I use - 1) on my mac 2) on my external harddrive 3) on my icloud. Is there anywhere else I should store my data? I have read something things about NAS but not sure what it is
2. I have a number of backups dating 15 years on several hard drives. I recently tested my first hard drive from 2010 and it works, but I have read that the life of a harddrive isn't too long. Are there any recommendations to preserve the oldest backups? Is it simply a case of transferring the files to a new harddrive? (for context it was made using time machine)
3. Are my email inboxes backed up in time machine? In the event that my email provider overwrites my oldest emails because of lack of space online, will my mac keep the old emails?

Thanks a lot

The original post: /r/datahoarder by /u/Working-Permit4789 on 2025-01-04 19:14:32.

Is there a good way to organize the main drive? my second drive is pretty straight foward because i know everything thats in it but my main drive where do i even start, program files, .android, cryptic? i have no clue what half of these are (lmk if im just dense) but where do i start

The original post: /r/datahoarder by /u/ewlung on 2025-01-04 18:52:45.

I have a main data folder D:\Photos_and_Videos.

And I have consolidated some of my backup photos/videos in E:\From_Backup.

Is there any tools to compare files from E:\From_Backup with files on D:\Photos_and_Videos?

If the file exists in D:\Photos_and_Videos, move it to F:\Duplicate.

The filename and folder structures are not the same between D: and E:, same file might exists in different folder.

The original post: /r/datahoarder by /u/Izekun64 on 2025-01-04 17:17:16.

I have never done this before and don't know where to start. I started to run into storage issues when editing on my PC (I have lots of external HDDs connected, plus slow data transfers). I want this to be very cheap and reliable.

I have a decade of experience building computers. I don't plan on building this for another 6 months once I have moved.

The parts I have so far:

CPU : i7-5820K (From Friend)

CPU Cooler : Cooler Master GeminII M4 (From Friend)

Motherboard: MSI X99A Raider (From Friend)

Case : Thermaltake Versa H21 (Old Gaming PC Case)

PSU : Thermaltake Toughpower GF1 750W 80+ Gold (FREE and brand new in the plastic wrap)

Question 1:

Will need to purchase a dedicated GPU (The CPU here doesn't have any integrated graphics)? Also, give me suggestion on very low powered GPU if it just a display output. Thank you!

Question 2:

How much RAM should I purchase (ECC?)?

Question 3:

Do I need to install operating system (Windows or Linux) or Need to install TrueNAS on SSD (I don't know much about TrueNAS or really what this process entails)?

Question 4:

Any good videos about setting up and navigating TrueNAS (Also, maybe stuff that is better than TrueNAS)?

Question 5:

Can I connect 2 PC's to this at the same time?

Question 6:

I have heard people using 2.5gb cards to connect there computer directly to there NAS. Does this increase speed?

Question 7:

How easy is it to add a new drive to the NAS?

Thank you for your help!

The original post: /r/datahoarder by /u/Soggy_Bottle_5941 on 2025-01-05 10:38:27.

I have been using computers since 1990. During this 35 years time period, the advances in hardware, operating systems and software keeps it hard to maintain future proof and accessible file formats. Imagine you have a digital diary you kept on a proprietary XXX application which works on Windows 3.1, on a floppy disc, some 30 years ago... And you try to open this file now...

My own experiences taught me the future proof files should be:

• Non-proprietary
• Unencrypted
• Uncompressed
• Open Standard
• Common usage

File formats like these have been surviving the last 35 years:

• .TXT
• .JPG
• .PDF
• .HTML
• .CSV
• .ZIP

Needs a little update, but still usable:

• .DOC
• .XLS
• .PPT
• .MPEG

The real problem is with the haardware side. Floppy disc -> CD -> DVD -> Blu-Ray -> USB -> HDD -> SDD -> Cloud ... With every widespread hardware storage you have to migrate all your data.

So, what file formats survived these years and will be Future, Long Term Accesible in your opinion?

The original post: /r/datahoarder by /u/trintium on 2025-01-05 10:31:24.

I'm familiar with Ultrastar and EXOS drives thunking every few seconds when idle.

I recently purchased my first Toshiba MG09 drive. It's been working well for a few weeks. I didn't notice any rhythmic idle sounds until just now, but there's a much softer and more frequent clicking when idle. Maybe it was always there. Not sure. It does break rhythm from time to time, including when the drive is busy reading.

Is this normal and the equivalent of the other drive heads repositioning every few seconds, or something else?

The original post: /r/datahoarder by /u/Artistic_Pear1834 on 2025-01-05 08:42:48.

Hi, I’m thinking of ponying up the cash and getting a macmini to use as a ‘family computer/server’.

Then syncing the macmini to iCloud, but have external drives hooked up to the MM - for running Time Machine backups AND CarbonCopyClone (CCC) copies of files.

?? What am I missing/ not thinking about - flaws in this setup? Is this a bad idea??

Steps:

• we already use a ‘family apple ID’ (set up about a decade ago, before family sharing existed) - so we already work off shared folders/ files in the cloud.

• I would migrate ALL of our family docs out of icloud onto this MMfamilycomputer,

• I would migrate ALL of our photos (downloading full resolution/ not optimising for space) onto this MMfamilycomputer.

• we’re both data hoarders I suspect - so our folder structure is already set up to allow us to save things for ourselves OR for the family to access.

• I would then turn on icloud syncing - so all docs/ files/ photos etc are syncing (and accessible) via icloud for everyone.

• I would run backups off the MM using Time Machine & make fulls Copies to external SDD’s (already using) using CCC (and auto-run a second copy of that copy to a second external drive (aready using).

?? Am I missing out??

• As this wouldn’t technically be a ”proper server” would I be missing out on the ability to use Plex or other more ‘server’ like products? (I’m a server newbie obvs).

• I know CCC suggests partitioning a disk for bundled mac apps: eg PHOTOS.. would I be successfully getting around this issue, if I had a computer setup that could handle the size/ volume of photos in their original form?

• Any alternative suggestions would be appreciated. I go cross-eyed at the research required to set up NAS servers or server raids etc.. so something beginner level and icloud for syncing oriented would be ideal.

Thanks all!

The original post: /r/datahoarder by /u/medullarymedulla on 2025-01-05 08:33:30.

I recently built a new PC and inherited an old Thecus N2500 NAS. My goal is to host a media server that myself and my family can use to stream movies and access photos remotely. However, the Thecus software is super outdated and hasn’t had support in years.

The Thecus N2500 has 5 Seagate Constellation ES 2TB HDDs (10 TB total), and I attempted to use it to host a Jellyfin server. Unfortunately, the outdated software makes it pretty frustrating to work with. I'm considering ditching the Thecus interface altogether and repurposing the hardware with a modern OS like Ubuntu Server or OpenMediaVault.

I’m open to other options too—like using a Raspberry Pi or buying additional components—if it will make the setup more user-friendly and future-proof. My main priorities are:

1. A modern and usable interface for managing the NAS. (I am a beginner with this tech for the most part)
2. Reliable streaming for media files.
3. Simple access for myself and family to view photos and videos.

What’s the best solution for my setup? Should I salvage the Thecus hardware or start fresh? Any suggestions for software or additional components I might need?

The original post: /r/datahoarder by /u/NoamThePro10 on 2025-01-05 08:10:12.

Sorry if it's a dumb question but i just want to be sure that this drive will be good for my plex server

https://www.ivory.co.il/catalog.php?id=72212

It's a WD Purple Surveillance 4tb drive that costs 419 nis And beacuse it's a surveillance drive i'm not sure it will work good as a media drive in my plex server

Thanks!

The original post: /r/datahoarder by /u/subdubreddit on 2025-01-05 06:24:49.

bro i loved my nas, made my workflow so great, but now im scared to ever use them again cuz i lost so much stuff, its my fault for not doing backups, but ive never had an ssd or any other external storage go bust like that from a storm, so im back to using them and it sucks having to transfer all my files across computers again but i do feel safer. still doing back ups tho lol

it was a buffalo, i just didnt know if maybe its a cheap brand and theres something better that wont fail like that, seemed wild to me, i get it happens but like dam i was salty, they have a specific data recovery they recommended and they were gonna charge me like 6 grand, oof

so like if i wanted to use a NAS again is there anything to do to prevent that from happening, like do i need to plug it into a surge protector or something or is there no total protection from that

The original post: /r/datahoarder by /u/r4nchy on 2025-01-05 06:03:36.

I have always found it very difficult to organize my data, every now and then I forget where to store the data. and often store it in "new folders" so that I can organize it later but I never do.

The original post: /r/datahoarder by /u/lgittens on 2025-01-05 05:30:24.

I currently own a DS418 that is currently full and will be buying a Dx517 shortly. I am running a hybrid raid 1 for plex and will want to fully utilize the 5th slot on the expansion.

1. My thought is to buy a SSD for the operating system. This means that one of the storage pools will be split between the two devices. I'm wondering if this is a bad idea or should I just use that 5th slot with no redundancy?
2. If I do go the route of the SSD is there anything else I should concider aside from backing up my DSM settings?