It's A Digital Disease!

The original post: /r/datahoarder by /u/marsmotwastaken on 2024-10-02 16:54:10.

Hi all I've fallen in to the habit of data hoarding and my pc is running out of space for my HDDs and i would I would like to access my files online so I think a getting a NAS would be good for me I love DIY stuff and tinkering with tech so it does not need to be a pre-build. Im looking for some thing around 200 Euro and with less than around 100 watts of power usage I'm thinking about getting a hp microserver or an older dell optiplex are these going to be good options for my data hoarding setup or do I need something better ?

The original post: /r/datahoarder by /u/Clockwork385 on 2024-10-02 16:47:19.

I'm using 24X 4TB SAS drive, and when I try to do parity, it's only allowing me to use 66% of the capacity (it looks to me like for every group of 3 drives it's using 1 as parity).

And when I select no parity no mirror, it still not giving me the full capacity of the 24 drive. why is that?

The original post: /r/datahoarder by /u/Ph00k4 on 2024-10-02 16:23:48.

I'm facing a situation where I can't view my liked Instagram posts on my computer, only on the mobile app. To get around this, I used the "Download Your Information" tool, which lets you export your likes in either JSON or HTML format.

Now that I have the complete list of posts I've liked, I need a way to automate the download of all images and videos from each of these posts, which are from different profiles.

I've found some programs that allow you to download all posts from a specific profile, but that's not what I'm looking for.

Someone recommended Instaloader, but I'm not sure how to use it for this specific task.

Any help or guidance would be greatly appreciated!

The original post: /r/datahoarder by /u/Loud-Position-9654 on 2024-10-02 16:03:12.

Code: https://github.com/sahil-lalani/bookmark-export

Also made a how-to video for it that goes over the code: https://youtu.be/XPI1wjjQ-5U

The original post: /r/datahoarder by /u/GermanPCBHacker on 2024-10-02 15:53:02.

We all know, that the cells of NAND discharge over time if not plugged in, leading to a loss of data. (Do we?) So far so good. But is this really that simple? Will plugging in the SD card, USB flash drive, SATA/PCIe SSD prevent this data loss effectively?

We all know, that writing data to a used cell first requires deleting (nulling) the cell. Okay, so with the logic of "Leaving it unpowered will lead to dataloss" we would not be fools to assume, that plugging in regularly or permanently on the other hand will retain the data. But that misses the point of the logic on how cells are written if I am not a complete fool. They are basically nulled and than given a very specific charge by applying an exactly defined voltage that no one except the manufacturer knows.

I am not sure, if it is possible to just constantly re-apply the same voltage, because I also would assume, that the applied voltage likely does not match the cells voltage - after all it is not a battery that is charged, but a somewhat isolated cell, that charges have to migrate through the surrounding insulation layer of the silicon substrate. So just giving it the same pulse again... Who knows how parasytic effects will effect the actuall charge state of the cell.

Furthermore I also doubt, that most, if not all firmwares even have a specific feature to constantly "repair" the state of cells, as charges slowly migrate through the insulation over time.

So is there any *real* documentation on wether such a "keep-alive" is happening or even exists on a hardware level? Not just opinion or single experiences, but actual evidence? I would absolutely love to know. For what its worth - if there was a nondestructive keep-alive, it would be adequate to just give an SSD 5V every 6 month for whatever time it takes for the firmware to complete a keep-alive cycle and we would be able to store data "forever "on SSDs, wouldn't we? (except for the case, where the firmware actively rewrites the cells and therfore needs to delete them first, which implies a lot of wear).

GO!

The original post: /r/datahoarder by /u/Intelg on 2024-10-02 14:32:08.

Original Title: Anyone use any of these? Can I use a reverse breakout SFF-8643 to get 12 sata ports to connect to low power ASMEDIA to nmve or am I forced to use a power hungry SAS controller card? If so whats a low power option?

The original post: /r/datahoarder by /u/qElCuco on 2024-10-02 13:46:22.

Newbie here. I'm planning on purchasing my first NAS in the coming weeks. I'm planning on storing a very modest media collection (~4TB) and our photos, videos, and music (also very modest, maybe 1TB in total).

I'd like to give myself a fair amount of headroom on the storage since I plan to have this for year. I do intend to stream my media to my Apple TV 4k using Infuse, though I may want to try Plex later on.

My plan is to purchase a 923+.

The thing I'm having the most difficult time deciding is storage configuration. I've read enough times that RAID is not backup, so I'd like to backup my RAID array to another disk in the 923+, or to an external HDD (depending on the chosen config).

Here are the options that I'm considering. I'm also open to suggestions. Again, I'm new to all of this.

Option 1

12TB x 3

RAID 1 Mirror = 10.9 TB usable

3rd drive used as a backup of the RAID array

Option 2

12TB x 4

RAID 6 or RAID 10 (I have no idea here) = 21.8 TB usable

No idea what to do for backup here. This pushes my budget. I may purchase the cheapest external HDD I can that could fit 8-10TB since I would never fill up a 21.8 TB drive.

Any advice would be appreciated!

The original post: /r/datahoarder by /u/igmyeongui on 2024-10-02 13:36:22.

Is there a script or app that can achieve this? There are a few FB pages that has so many rare content unavailable anywhere else. More specifically a few rock progressive bands from the 70’s. They decided to use FB to share their whole video archive.

The original post: /r/datahoarder by /u/frankysan on 2024-10-02 09:17:19.

https://preview.redd.it/x0jlqsuv8bsd1.jpg?width=3024&format=pjpg&auto=webp&s=c6f829718d0219dcb29060903d667178a96af0d1

This server traces its lineage directly back to an ancient single-cpu single-core box my friend group had at our hangout around the turn of the millennium. It's gone from FreeBSD to FreeNAS, TrueNAS and now runs ProxMox. It's a Chinese "X79" mobo with dual E5-2630s (planning to upgrade to 2667 v2s soon), 128 GB RAM, a 10Gb NIC, and a Tesla P4 GPU. The 10 storage drives adds up to 86 TB raw.

The original post: /r/datahoarder by /u/Ilegator on 2024-10-02 08:38:33.

The original post: /r/datahoarder by /u/CockroachEasy6432 on 2024-10-02 04:43:49.

I've got a couple of external 4TB HDDs that are mirrors of each other for my backup, and they've always been in HFS+ as I've been a Mac user for a long time. However, I have no use for Mac computers anymore and have recently moved back to Windows, but the drives are keeping me from getting rid of the Mac.

I need to convert them from HFS+ to NTFS, but I'm very concerned about losing the data or parts of it being damaged in the process. I currently only have two drives that are 100% mirrors of each other, and no other medium which could temporarily hold the entirety of this data. I would have to wipe one to convert to another, which increases the chance of data loss immensely. I'm worried I will have to buy more drives to be safe, so that one of them is completely untouched by the process.

My main issue comes from the actual process of copying the data, I'm unsure what the best software or method to copy the data is. I see recommendations for Tuxera and Paragon, but especially with the latter I read a lot of sketchy stories of data going missing or outright corrupting. If I tried to do the conversion in Windows, I would have to be certain that the drive cannot be touched by antivirus, as I already know some of my files will be flagged as false-positives. I can't afford this, and for 4TB the drives are EXTREMELY data dense. Not sure how to verify that the NTFS drive will be a 1:1. Just looking for some pointers as to how to most safely go about this process as it's causing me a lot of stress at the moment and holding things up.

The original post: /r/datahoarder by /u/honeststock_ on 2024-10-02 04:38:20.

I heard telegram going to ban accounts randomly. I stored 4 generations of phone files there. I don't wanna lose it.

The original post: /r/datahoarder by /u/DrTallFuck on 2024-10-02 04:30:53.

I am currently looking to expand my storage and was hoping for some advice on the best option for my current situation.

My current setup is a Minisforum UN1245 mini PC running Windows 11 with a MediaSonic 4 bay ProBox DAS with 4 x 12TB HDD. I have been using Snapraid for parity and the 3 data disks are getting full. My end goal is to get a small rack server once I move into a house in about 6 months, but until then I am sticking with the mini PC.

I am still relatively new to the whole data header/self hosting space and I am learning as I go. This mini PC has been great but unfortunately does not have a spare PCI port so I'll be limited long term to only USB attached storage (or a NAS, but I won't be messing with that until I can set up my home network on a house). I am planning to switch to Proxmox soon and running a Windows VM and then containers for other services. When I get the rack running I will likely get a disk shelf to directly attach to the server but for now I need more room in my current setup.

My question is would it be better to get an 8 bay DAS enclosure so I can add more disks, or would tit be just as good (or better) to run a 2nd 4 bay DAS alongside the ProBox I already have? I will likely start with adding 2 more disks when I expand (1 more data and 1 for parity to get to dual parity) and then have the option for 2 more data disks as needed. I am leaning towards just adding another 4 bay since I already have one that has been good to me for the past few months but I don't have enough experience to know if having 2 enclosures would be detrimental compared to having all disks in a single enclosure.

I also figure that once I eventually move to a rack server, I can use the mini PC and the DAS enclosure(s) as either a NAS with trueNAS or as a Proxmox backup server. That is a bridge that I will cross when the time comes.

Any advice would be greatly appreciated!

The original post: /r/datahoarder by /u/errorcrucible on 2024-10-02 04:16:43.

has anyone here used filegarden?

sorry if this dosent belong in this sub, but i'm having a hard time finding anything to do with filegarden and the its super frustrating haha.

My filegarden has gotten to the point where every time i try to open my folders it gives me a "unknown error occurred". I've tried turning off all my adblockers, and tried on various different browsers, still gives the error. Hoping to find a solution for this, thanks!

The original post: /r/datahoarder by /u/True-Entrepreneur851 on 2024-10-02 03:50:41.

I would like to start back upping (seriously) my personal data in case of disk crash as I read here. I plan to use duplicati to copy D:/ drive to an external physical drive ok nice but then comes the problem of encryption of external drive and would like to know how you can do it. Suppose I create my first backup of C:/ to external drive (step 1) and then use cryptomator (step 2) to lock this data, I have a doubt now how is it going to work out for new versions when I will have to backup additional versions of my C:/ data. Target will be encrypted (step 2) but duplicati will check existing version (step 1) that doesn’t exist anymore. If anyone could give me best routine that would be highly appreciated. Thanks.

The original post: /r/datahoarder by /u/Kemicall on 2024-10-02 00:44:52.

For about the last 7-8 years I have been running the following setup for my Plex server and it has suited me very well. I'm thinking about finally upgrading the Motherboard/CPU/Controllers and would appreciate recommendations.

Current Setup

• Supermicro CSE-846 24 Bay
• H8DM2-2 Supermicro Motherboard
• BPN-SAS-846TQ Backplane
• Dual AMD Opteron 2431's
• 16GB Memory
• 3 x Sonnet Tempo SATA Gen 2 PCI-X Controller Cards
• PC Power and Cooling 750 watt PSU
• Windows Server 2016 + SnapRAID + DrivePool
• 24 various 3.5 SATA drives for 89TB usable

This server is housed in an basement media closet and noise was a concern. I ended up with the desktop PSU + speedfan software to get the noise level tolerable while maintaining safe temps. The only other complaint is that with this setup that a majority of the drives are not properly visible to the OS. They function fine and no issues within SnapRAID, but if I pull SMART data only about 5-6 unique drives show up and the rest are duplicates.

I'm fortunate enough to be within driving distance of a Microcenter and figured one of the Motherboard/CPU/Memory bundles they offer would be fine. If you have a budget of around $500 what would you suggest upgrading to?

The original post: /r/datahoarder by /u/DisclosedForeclosure on 2024-10-02 00:41:41.

Hi, I'm new to this. What would be the best way today to mirror an old classic message board?

I'm talking about a scenario where I don't have the access to board's DB or server. So it's purely html backup. Web archive makes a good job at mirroring single specific pages but doesn't seem to be a good fit for message boards with thousands of subpages.

I started using Cyotek to scan and download all subpages, but it's quite slow (8 files per minute) and I don't know how to make it skip certain subpages. I.e. it unnecessarily goes through every user's profile page. It can get stuck there for many hours. Other issue is that downloaded html files would still have absolute asset links (js/css) in their code referencing the old domain - relative links would make the backuped sites more portable, but I'm not sure if any site downloader would automate such substitution?

The original post: /r/datahoarder by /u/BloodyStupid_johnson on 2024-10-02 00:35:35.

Original Title: Got too many random HDD's of various TB sizes that I handle like floppy disks. What's the next step I need to educate myself on for moving toward and consolidating more efficiently to an array of SSD's.

Just looking for advice on what I need to learn for next steps, have some technical skill but don't recognize all the terms used in this sub.

The original post: /r/datahoarder by /u/Jhh94 on 2024-10-01 22:39:38.

The original post: /r/datahoarder by /u/BrickAndroid on 2024-10-01 22:13:31.

I have a bunch of folders (hundreds of them) with thousands of images each, downloaded from Twitter artists. The problem is that these artists also post things that aren't art, like real photos, or reaction images, or screenshots of apps/games.

Is there a free program that could classify these images and sort them into folders?

I could manually check the correctness later, but I just need something to do the initial bulk of the work.

The original post: /r/datahoarder by /u/ALT703 on 2024-10-01 21:39:30.

I have about 20TB in assorted hard drives, like a 6TB one, an 8TB one, and 3-4 2TB ones, etc

Two of the 2TB ones are still in their factory packaging, everything else is used, but not heavily I don't think

What's the best way to make use of these? Is there a good way I can make use of them that minimizes risk of data loss? Happy to build my own setup from scratch, just looking for ideas or advise

Or, is the mixed brands/storage sizes/usage levels (not all brand new) just a recipe for disaster and I shouldn't really make use of them in a setup?

What do you guys think? Thank you!

The original post: /r/datahoarder by /u/aptquark on 2024-10-01 20:55:30.

Hello all, I just tried to install 3 x 12TB HGST ENT HDD's into a PC that already had 3 x 3TB ENT HP HDDs running with no issues.......and the fOOking HGSTs WILL NOT POWER UP! Power supply perfectly fine and so are the SATA cables. All I did was replace the 3 x HPs with the 3 HGSTs. WTF!? Get this, when I use the USB dock I have sitting on top of my PC...the fOOkers power up and I can initialize and configure them on to the system. THATS THE ONLY WAY. What am I missing? Thanks

The original post: /r/datahoarder by /u/Meat_Thick on 2024-10-01 20:20:16.

Hey my 1TB WD Blue ssd has failed obscenely and become read only and unusable. However, the support portal for western digital has been down for a few days now.

I'm just wondering if anyone knows how long it's been unavailable and when we might be able to request rmas again?

The original post: /r/datahoarder by /u/chmedly020 on 2024-10-01 20:06:17.

I have a couple T330s and know that this line of Dell servers cannot use a video card as a video output device. The only "graphics" on these machines is the VGA output. But, I've noticed that the T340s were available from Dell with graphics processors. Hence I'm wondering if "graphics compute" is possible with these processors in these machines. Can I use quicksync to transcode with jellyfin with an E-2126G CPU in a T340?

The original post: /r/datahoarder by /u/Blackwater_7 on 2024-10-01 19:46:53.

Is such a converter exist? What I want is being able to transfer data from PC to console. Currently each time I do it I have to plug the HDD to PC, and when I'm done I'm plugging it to my console. This is too much work for me when I do it multiple times a day.

But imagine I have a converter so I plug the HDD both at console and PC at the same time. I know that an external HDD can be only connected to one device at a time, and for that I would like to have remote controller. So I can just switch the channel with that. Both will be plugged 24/7 but only one will work depending on my input with the controller.

This is the solution I need. Is such a device exist for that, or technology is not there yet?