The original post: /r/datahoarder by /u/grathontolarsdatarod on 2024-07-15 08:50:41.
But thanks to the generous and educational words of all the hoarders here, I survived!!! (I think).
I just want to say THANKS EVERYONE!!
So... I've always been a hoarder at heart. I used to back-up my files by dragging and dropping windows user directories and dumping them into a 'back-up' directory on the same harddrive, a USB stick, or external harddrive. From about 1997 until last year, this had been my MO.
I knew data recovery was a thing, so between new computers I'd repeat my insanity, shuck the drive from an old computer, chuck it in a drawer... For years. Not just between computers though... Before major life events, OS updates, experimenting with risking processes, you name it. Drag-and-Drop. I started volumes because I started hitting naming limits.
Pictures, papers, online sources, memes, clips of hilarious nights out, legally important documents and recordings.
It was like a ukarian doll of fractal history.
Over the last year I bought a desk and a desktop, then three monitors, shucked the drives, consolidated the keys and the memory sticks. I had a mission. I even read about the duodecimal system try and help out. Which was basically the state of the art when I "started" my habit.
I found linux. I used the three monitors to maximize my efforts. The split view file manager (just plasma, lol) was LIFE CHANGING.
Then I did some reading here. I bought a 4-bay enclosure and consolidated even more. Used the hiccup program (czkwaka, I think) and over a few days took over million files down to about 50, 000.
Then I read the room and started hoarding. Everyone thing I could find. 4x10tb in raid6 was filling up FAST.
I had outgrown my back-up capacity.
Bought another one, a 5-bay this time. Worried the heat would kill my server before amazon showed up. But jellyfin MUST play on.
BUILD THE RAIDS...
I still don't know what happened... But if you read this far, maybe you'll find it useful.
Was running a Ryzen5 in a beelink, but put a 1tb 2.5 inches in there - cause - waste not want not.
Bought an N100 with the second enclosure for transcoding (works AWESOME).
In building the new raids, either there was too much heat on the main board, too my heat on the USB connector, a caching problem, or fussed up mdadm by attaching two "/Dev/md0s" into the same machine, even though they were created with UUIDs and have different names in --detail --DEVICENAME.
My primary raid went down.
Latest JBODs in a draw back-up was only 3 weeks ago. But I had just de-duped (very carefully) and dumped in a LOT of jellyfin material from another beelink whose only purpose is to handle such things.
Went I did the jellyfin dump I wasn't impressed with the internal temps (mid-40s) and had to use a spinner cash my SSD (in a caddy) wasn't working well and a shovel in the heat, so I though I'd wait out the heat wave.
I suck at diagnostics cause I'm still VERY fresh in all of this. But I was trying to figure out if it was the drives, the OS, the USB, the new enclosure.... I still don't really know.
I took the 2.5 inche out of the beelink for better ventilation and sweated through rebuilding the primary raid.
I'll save you the DAY of reading and waiting. --assemble --force worked on a fresh debian install (no actual data is kept on the "computers" anymore) Thank god. For good measure I have custom names for the raid devices.
Mdadm said one of the drives failed. But after re-assembly and a reboot (auto mounted) its seems to be ticking along with the rest of them again. SMART test never showed a problem. So I suspect heat on the cheap beelink hardware.
Not everything is running at high 30s low/mid-40s and the backup raid is building with a panic sync of a few TBs is running to a spare drive.
So..... Build your data storage backwards people. Don't over reach, keep cool, schedule your damn back-up processes, and RAID6 is not over kill, and an untested back-up doesn't do you much good at all!
Hope this entertains or helps someone. I still have about 15 of 20 hours to wait until build is done and no jellyfin....
Thanks to this community!!!!