It's A Digital Disease!

The original post: /r/datahoarder by /u/BobbythebreinHeenan on 2024-05-19 14:18:55.

I’ve used external hard drives (almost always western digital) most of my life. One thing I’ve always noticed is that even when a drive isn’t in use, it will be spinning.

Is a drive that’s spinning while not in use bad? Is it better to disconnect it from the computer when not in use? Would just unplugging it from the computer sufficient? Or do i need to also disconnect it from power?

I’ve never had a hard drive fail on me btw.

The original post: /r/datahoarder by /u/csyst on 2024-05-19 13:44:06.

I am looking forward a solution so i can combine multiple harddrives and access them as one drive/mount. Of course something like RAID 0 or LVM would solve this requirement, but I don’t want to loose all the data of all drives in case one drive fails, only the data of the failed drive. But since both variants use striping both ways don’t work, i guess? Also I want to be able to extend the „virtual“ drive later and I don’t want to loose space for redundancy, since I can get all the files again.

Perhaps someone has an idea? Is there anything like a filesystem or mechanism that merges the drives on the fly?

The original post: /r/datahoarder by /u/PopularFortune444 on 2024-05-19 13:00:53.

youtube.com/watch?v=cANLgNQtSRg youtube.com/watch?v=p4z5cBxVMIY youtube.com/watch?v=2Ni8SNnEJpw youtube.com/watch?v=qm2JIRppxZU

The original post: /r/datahoarder by /u/banisheduser on 2024-05-19 10:25:53.

I have some media files saved, but I am very unlikely to ever watch them.

However, I know they're not available from the official site any more.

I sort of feel a responsibility to keep them archived but at the same time, I do wonder why I am bothering.

Have you ever felt this way about some data you've hoarded and what was the outcome?

The original post: /r/datahoarder by /u/cod201 on 2024-05-19 09:03:58.

tldr: extracted more than 1.4 mils folder ( each inside have files) into single folder in NTFS, ridiculous bad IO performance. Divided into 500 folders and got better IO, but i know i'm, dumb. Ask for better approach.

Hi,

I am using NTFS on Windows to run an research but i'm facing performance decrease, maybe due to data and index fragmentation.

I receive multiple ZIP files everyday to extract, inside zip file is multiple folder, each folder is an unit of data for us to digest using Python, and upload them to Elastic (these in bright cyan).

a brief structure of my data

Because the server is headless, i just recognized the problem when i connected it again, when it reached 1,5 millions folders, approx. 1.5TB inside JUST A SINGLE folder. (about 7TBs waiting to extracting but i stopped it)

https://preview.redd.it/cu59lxq1kc1d1.png?width=505&format=png&auto=webp&s=61a0830a1a703e0365f449ae768b5101181bd1ce

So, when I move/ rename.. a folder, it is extremely lagging, moving a small file barely took more than hour. Just view the properties take more than 1GB of RAM.

https://preview.redd.it/ocwa17rujc1d1.png?width=1160&format=png&auto=webp&s=b69e43200168de0724e3c4518a0fd087c55a45d9

I've just moving these data into other disc and dividing into 500 folders, based on its name (from AA to ZZ), the performance just got better, but idk is there any better ways to storage and using these data?

I use Python to work with these file (maybe upgrading to C#/go.. for better multi threaded performance) and after digesting, i would storage it for about 6-12 months before delete it.

I know my strategy is somewhat inefficient, so i'm asking if i could make it better. Thanks

The original post: /r/datahoarder by /u/Zone_Purifier on 2024-05-19 05:20:20.

In an attempt to make it work properly with some newer drives, I've been hunting for the latest SAS firmware for a AXUS Y3-24S6SF8 RAID Subunit. AXUS is a former subsidiary of ASUS, one which hasn't been around since 2014 and has some connection with Areca. This search has taken me to internet archive, ASUS, Areca, and random russian server harware sites. I've found some interesting leads and dead end phone numbers and emails, but nothing that yields the .bin (probably) that I need to update the system. If anyone has access to these files or might have ideas of where I can get firmware for such a seemingly obscure device, your help would be instrumental. Here's what I've found so far:

AXUS Website, latest archive

YOTTA III Firmware Page Archive

AXUS Closure Notice

AXUS Final Contact Information Notice

YOTTA 3 User Manual

The original post: /r/datahoarder by /u/Neurrone on 2024-05-19 04:49:50.

I currently have 4 x 4TB Seagate Ironwolfs and am looking to expand.

1. Are there price trackers that I could use to get a sense of what a good deal would be, in terms of price per TB? Something that can send alerts when prices drop below a certain level would be an added bonus
2. How safe is it to use used drives? They are half the price of new ones, but might fail more quickly.

The original post: /r/datahoarder by /u/uwishto on 2024-05-19 03:19:06.

I have this deal at my local Best Buy both of them cost $100.

And they are the same storage.

But which one is more reliable for the long term, in terms of back up and storage for long time

The original post: /r/datahoarder by /u/NeroD_175 on 2024-05-19 02:41:05.

I just thought about how cool it would be if something like that existed. Just imagining being able to download whatever I'm thinking about instantly just by typing in a few keywords to find it, regardless if it was deleted or not.. Sure there would probably be some issues with that, like privacy concerns etc. but to me, data is memory and memories are by far the most valuable thing we have as human beings. As a datahoarder who has lost precious data/memories before, this is my ultimate fantasy.

The original post: /r/datahoarder by /u/foxhound13 on 2024-05-19 01:51:47.

Complete newbie here, looking to purchase a Nas purely for storing and streaming video content to my laptop, what I'm trying to understand is the following:

Lots of the cheaper options have 1gb ram, will that do for standard video play back from the device to a computer. (standard size files no 4k likely no VR) I'm not sure if ram is even a bottleneck here or not.

Might be silly but How viable is using a torrent program to download video content to the NAS and is there any considerations i might want to make especially around download speed (Im fine with a lan connection if recommended)

Do most NAS units come with password Protection software/abilities and a Lan port

I'm in the market for an 8tb nas with drives included (4tb actual storage 4 redundancy I think) and room to grow for the cheapest possible if anyone has any recommendations.

I don't think i require plex or any fancy ui stuff just straight up storage I can play video files from, any help is appriciated.

The original post: /r/datahoarder by /u/cuong3101 on 2024-05-18 19:56:13.

I’m in a bit of a pickle and could use your collective wisdom. My OneDrive Edu account, which was my digital haven with 5TB of space, has been abruptly downsized to a mere 100GB. This change came without warning, as I missed the memo about Microsoft scaling down storage for educational institutions.

Here’s my current setup:

• Three 3.5 inch 1TB HDDs (one’s a dedicated movie library)
• A 500GB Samsung 860 Evo SSD
• A “2.5 / 3.5 inch 2 Bay USB3.0 1 to 1 Clone Hard Drive Dock” (sans RAID, so it’s just two separate drives)

With my OneDrive now bursting at the seams with 1.3TB-1.5TB of data, I’m exploring storage alternatives that won’t break the bank. The NAS Synology DS220j or DS223j are within my financial reach, but they offer only two hard drive slots.

There’s also an old family computer gathering dust back home, powered by an i5 (6th or 8th gen) and a 300W power supply of dubious origin. It’s been out of action for a while, and I’m not even sure it’s still operational. Reviving it would mean having it shipped over, which is a 4-5 day wait.

So, dear Redditors, what’s my best move here? Should I invest in a budget NAS, attempt to resurrect the old PC, or is there another ingenious solution I haven’t considered?

Thanks in advance for your help!

The original post: /r/datahoarder by /u/ReddSaiyan on 2024-05-18 17:43:48.

I bought a cheap used hard drive. Can someone please tell me if these stats and raw values are good for a used hard drive?

https://preview.redd.it/jh3pew34281d1.png?width=662&format=png&auto=webp&s=52f8d5eb9d229ccb962dbd258fd09fc5bd89b5ab

The original post: /r/datahoarder by /u/laggyservice on 2024-05-18 17:27:18.

The original post: /r/datahoarder by /u/Vegan_Salad69 on 2024-05-18 17:25:57.

Hi everyone,

I’ve been homelabbing for about three years now and I’m looking for some advice on optimizing my storage setup. Here’s a quick overview of my current configuration:

Server: Dell R750

• Front: Twelve 3.5" SATA/SAS bays
• Rear: Four 2.5" SATA/SAS bays
• RAID Card: PERC 350H (supports RAID 1, 0, 10) (don’t worry, I use it in passthrough mode and it works so far, even on ZFS. Yes, I know that’s also a problem, and I plan to buy an HBA or use the RAID card for its intended purpose)
• 4x 10Gig SFP+ Networking via DAC SFP+ Moduels (For more info on that i have a post of my entire homelab on my Profile)

Drives:

• Eight 4TB HDDs
• Two 3TB HDDs
• Two 2TB HDDs
• Two SAS 600GB HDDs
• Two 2TB SATA SSDs

Current Setup:

• Proxmox as the hypervisor
• TrueNAS Scale as a VM with passthrough for ZFS

I’m facing potential performance issues because TrueNAS Scale is virtualized. I’m considering a few options and would love your input:

1. Installing Proxmox on SSDs: Utilizing the SSDs for the OS to leverage their speed for the hypervisor.
2. Using SAS drives as cache: Taking advantage of the 12Gbps speed for caching to enhance overall performance.
3. Direct ZFS on Proxmox: Managing the drives directly in Proxmox with ZFS, but I’m concerned about losing functionalities like SMB sharing, permissions management, and user administration.

I also have an old Synology RS2818RP+ that I could use:

• Synology NAS with 16 bays and 10GbE connectivity.
• Equipped with the E10M20-T1 (a 10Gig Network card with 2x 2TB NVMe SSDs for network cache).
• Considering moving all large HDDs to the Synology and using it as the main storage, and only hosting the VMs' main system on the Dell's SSDs (so each VM gets 32GB; the main storage is on the Synology).
• Direct 10GbE connection to the Dell R750 for high-speed access. (Block Access)

Potential Solutions:

1. Bare Metal TrueNAS Scale: Install TrueNAS directly on the server for maximum performance, but lose the flexibility of Proxmox.
2. Proxmox with ZFS and Containers/VMs: Use Proxmox with ZFS for storage and handle additional services (SMB, permissions, Nextcloud) via LXC containers or VMs.
3. Hybrid Approach: Keep Proxmox with ZFS pools, and run a VM for TrueNAS Scale to manage SMB and NFS, with other services running in containers/VMs on Proxmox.
4. Leveraging Synology NAS:
   • Use the Synology as the main storage system with 10GbE connectivity to the Dell R750.
   • Utilize the NVMe SSD cache on the Synology for improved performance.
   • Keep the SSDs and SAS drives in the Dell R750 for caching and high-speed storage needs.

I’d love to hear your thoughts on these approaches or any other suggestions you might have. What would be the best way to balance performance, flexibility, and power efficiency in this setup?

Thanks in advance for your help! I hope you all have a good weekend. Best wishes, keep hoarding.

• a VeganSalat

The original post: /r/datahoarder by /u/useless_shoehorn on 2024-05-18 16:42:30.

Hi. I have a bunch of movies on a 12TB drive. Some of them didn't finish torrenting in windows/NTFS. I wanted to extend this drive in windows, but my block size was too small (4096). I have since repented and spun up a TrueNas server with the other 12TB drive (1M block size). However, as I'm copying to it I'm running out of space. Even if every file needed another 1M for the block size change, that's only 70GB right? What am I missing? How is the size on disk 17.1TB? What does that even mean?

https://preview.redd.it/ufx8hwhwp71d1.png?width=744&format=png&auto=webp&s=39f3db1e1495f396c4bf5b72d961521bdcde4c4d

https://preview.redd.it/p070vuhwp71d1.png?width=1248&format=png&auto=webp&s=501506ac4d6c18b41f04a61eefdd34d927f22c88

The original post: /r/datahoarder by /u/Acrobatic_Owl_4101 on 2024-05-18 16:18:38.

Hello, in the past I've seen various posts and websites on how to get fullsize images from behind the times museums that post VGA quality images that are mere thumbnails in today's modern screens. I have not seen tips for getting images from Past Perfect Online sites though. Does anyone know how I'd get a bigger image than the small "zoom in" I get clicking the image?

The original post: /r/datahoarder by /u/ReyArthurLXIX on 2024-05-18 14:49:37.

Extreme Picture Finder is saving videos in multiple files, an index followed by several videos named video0, video1 and so on, how can i save it all as 1 video?

The original post: /r/datahoarder by /u/willdab34st on 2024-05-18 10:22:22.

Hi, I was going to purchase a second controller for my R720XD, front disk container and backplane is a perc mini mono setup. However I remembered I pulled a few SAS HBA's out of it when I got it. They are Dell g164p 6GB/S HBA cards, apparently they are the same board as the more common H200E just with no internal headers, just external.

I'm thinking I can use these instead of purchasing another card, but unsure how I would hook this up, as far as I can deduce I'll need to plug in a backplane somewhere in the rear for the SATA board and I'll have to route the cables from outside the chassis. The g164p cards have a type of sfp connector, assuming this is standard for connecting backplanes etc and buying a backplane off of eBay for R720XD will work with this?

Related question, I would assume it's not possible to split the front drive bays so some are served with different controllers? If not then I'd have to mount new drives for the HBA controller in the rear somewhere, there does look established slots there already, 2.5".

The original post: /r/datahoarder by /u/Physical_Taste_4487 on 2024-05-18 10:19:13.

Hi. I currently have a WD My Cloud EX2 Ultra and want to move on from it. It has 2x16TB setup as JBOD. (Full of data) How do I go about using drives in a new enclosure? Is there any way of doing this whilst preserving the data? Or will I have to transfer the data to something else first? I don’t mind doing this if I have to as long as I can use the drives in some other NAS/Enclosure once it’s all done.

Any advice much appreciated.

The original post: /r/datahoarder by /u/Subversive_Lovelorn on 2024-05-18 09:24:47.

The Premise:

I have a vast collection of movies, EPUBs, and FLAC files that I have been accumulating for over twenty years. I would like to securely store these files (and access them regularly) on an external drive that can offer a high degree of durability and reliability.

The Question:

Would it be more prudent to purchase an external HD or an SSD for this purpose?

The original post: /r/datahoarder by /u/FredericoSquashi on 2024-05-18 08:59:23.

Hi everyone, I have a Mac mini that runs 24/7 as a sort of home server, currently just running Jellyfin with some media on an external SSD attached to the Mac Mini. I also use a MacBook as a day-to-day work drive.

I have a few pain points:

• I don't like the SSD attached to the Mac mini, it seems wrong somehow and I'm also worried about the lifespan.
• I don't have a real backup solution for all my data.
• I have a 200GB iCloud plan, which is convenient (Apple ecosystem) but annoying (sync issues and quite expensive). Here most of my private and work related data is stored.

My idea is to reduce the iCloud plan to about 50GB (just for photos and the iPhone/iPad backups). Switch to Hetzner or Mega and get a 1TB plan. Host my data encrypted there and mount the storage on my Mac Mini and MacBook and use that as my "cloud" and use the media folder with Jellyfin. Then I would look for another storage provider to create some sort of backup job for my data on Hetzner/Mega.

So my question is, will this work? Or is there a better way? :D

The original post: /r/datahoarder by /u/Hostile_18 on 2024-05-18 07:24:54.

Hi all. New to all this. Got my media NAS and am trying to decide between good x254 encodes at around 1.2-2.2gb or full 1080p remux episodes at around 7-12gb. My question is will the quality difference be worth it, and also will it significantly reduce the hard drives lifespan? I've got 6x 20tb Exos drives that have a work load of 550tb a year for 5 years, I guess half that if you run parity once a month. All this and that's not even counting huge 4k remux files. The X-files 1080p remux from the Blu-Ray discs is like 2tb alone.

What's anyone elses experience hosting large media files? Have the hard drives held up over time? I have about 6 clients outside the local network, if that makes a significant difference.

The original post: /r/datahoarder by /u/Lanky-Antelope7006 on 2024-05-18 00:47:35.

The original post: /r/datahoarder by /u/Yurt_lady on 2024-05-17 14:32:41.

I have a problem. I had a Kasa in my husband’s room at a memory care facility. I have Kasa care and an SD card.

My husband passed away on Sunday. The Kasa camera is great but the software is crap. I need to download 30 days of videos and it seems that I have to do it one at a time. I can’t do that.

Is there a way to write a batch file or something to download them? Can I hire someone from this group to download the videos? This is a nightmare.

The original post: /r/datahoarder by /u/EargasmicGiant on 2024-05-18 03:01:50.

How to hack my LS210d and change OS to Linux to install media server like jellyfin?