It's A Digital Disease!

23 readers
1 users here now

This is a sub that aims at bringing data hoarders together to share their passion with like minded people.

founded 2 years ago
MODERATORS
5401
 
 
The original post: /r/datahoarder by /u/ChocodiIe on 2024-12-17 19:30:57.
5402
 
 
The original post: /r/datahoarder by /u/Mysterious_Jelly_943 on 2024-12-17 19:29:58.

So for the last year i been collecting thigs off the internet mostly because i do t have internet or electricity for large portions of the day. Im off grid pretty much living off a generator running a couple times a day. But i have 1 gbit internet connection when the electricity is on.

Basically i need to figure out the most cost efficient way to for the biggest basically external hdd i can get. I dont need a nas i dont need something thets going to be on all the time and i dont need it to connect by wifi to anything.

I need something that connects thru usb 3 and basically works like an external. The data isnt so important that i cant get it again for the most part so im not super worried about a bunch of backup failsafes i just need a good way to store data.

So im thinking is mh next step to just buy a 20tb wd my book. Or is there a better way to house internal terabytes and what would be more cost effective in the long run. Or is there some other option im missing.

And i dont want to put internal hdds in to my computer because i want to access it on my laptop thru usb.

Also should i worry if it spends most of its time off should hdds be on all the time?

5403
 
 
The original post: /r/datahoarder by /u/Thin-Try5917 on 2024-12-17 18:56:05.

Hey there, I'm currently planning on building a new NAS. I already have a case (Intertech 30240, got it for free from work), but I'm not sure what motherboard I should use for this.

I need something that has at least 2 full size pcie slots, preferably at least 8 ram slots and ECC slots

while being ATX format. Any recommendations?

5404
 
 
The original post: /r/datahoarder by /u/Inmyprime- on 2024-12-17 18:15:35.

I have two Terramaster d5-300C enclosures I bought years ago and everything seemed fine on my old iMac 2015.

Recently, I upgraded to Mac Mini M4 pro (and also to Sequoia) and I have been having issues with the external hard drives ever since.

Often they unmount, randomly, or won’t unmount at all (it says some process is using them, even if I can’t see anything using them), they also randomly grey out sometimes and stop working (first aid won’t work, I have to reformat to get them to work again, something about invalid Node structure after I run first aid…).

I can’t work out if it is the drives failing or if it’s the enclosures (they are old) or something else. Any idea how to narrow it down?

I also run Carbon Copy Cloner to backup.

5405
 
 
The original post: /r/datahoarder by /u/Beensmokin777 on 2024-12-17 18:15:23.

I have a job where I handle alot of data in a lot of forms and was wondering if anyone knew of a dock/hub that I could mount to my desk instead of my current solution of 100 dangles and adapters.

5406
 
 
The original post: /r/datahoarder by /u/Extreme-Reporter-480 on 2024-12-17 17:51:10.
5407
 
 
The original post: /r/datahoarder by /u/DiskBytes on 2024-12-17 15:29:12.

Hi, does anyone know if there is a way that I can get the Dell H200 to work with Windows 7 64bit? I have a tower PC to put this card into, but it runs legacy stuff, which is why it's still on W7. Thanks.

5408
 
 
The original post: /r/datahoarder by /u/hasanismail_ on 2024-12-17 13:07:44.

I really need 4 18tb drives because my jellyfin media server is running out of space cause of all the crap my family requests via jellyseer so I really need to migrate to a better setup than a single disc I really want 4 18tb drives because I think that's future proof for me at least it's better than the single 14tb disc I'm using

5409
 
 
The original post: /r/datahoarder by /u/PONT05 on 2024-12-12 00:27:31.

I’ve exported some photos from my iPad ages ago and i’ve noticed the metadata of dates is wrong in many pics, though some pics are taken in exactly new year so i’m thinking if there’s a program to automatically correct the metadata of the other pics based of that info?

5410
 
 
The original post: /r/datahoarder by /u/Lysander_Au_Lune on 2024-12-12 00:10:38.
5411
 
 
The original post: /r/datahoarder by /u/lukeprice1990 on 2024-12-12 00:00:42.
5412
 
 
The original post: /r/datahoarder by /u/birdcatx7 on 2024-12-11 23:56:21.

The time has come, and I need one. I just want to make sure I get it right. From what I've skimmed from here and elsewhere, if I search on ebay for am LSI 16i it mode card, with sata breakout cables, and pay around $50 or less, I'm good right? Then I just plug everything up and it just works in it's own, correct?

5413
 
 
The original post: /r/datahoarder by /u/kittykat-309 on 2024-12-11 22:52:55.

I pay for 2TB of iCloud storage, yet my iPhone is always full, and it’s driving me insane. I’m the family keeper of all the photos—we’ve talking over 100,000 since 1940. I’ve tried to clean them up, but deleting is a nightmare sometimes, and uploading takes forever.

My current system: • Photos are stored on my iPhone and synced to iCloud Photos (optimize iPhone storage checked) • I also manually back up some photos in iCloud Drive to folders but since there aren’t thumbnails it takes forever to find the photo I’m looking for among other logistical nightmares.

I’ve tried saving them to a hard drive, but it kept crashing, so I gave up on that. I’m open to switching to another platform, but I need something that:

  1. Doesn’t take up space on my iPhone or computer (but lets me access the photos easily).
  2. Handles massive libraries without constant uploading/freezing issues.
  3. Is reliable and won’t randomly lose photos.

I’m at my wit’s end trying to keep this organized. What solutions do you recommend? Anyone else with a similar massive photo library figured this out?

Please help me before I lose my mind.

5414
 
 
The original post: /r/datahoarder by /u/ZOODUDE100 on 2024-12-11 22:29:44.

My usual downloaders for watching videos offline are a bust. Have you had success with any other free sites?

Tried Catch.tube, tubeninja,keepvid.

https://www.pbs.org/video/attenboroughs-life-journey-z2tyb3/

5415
 
 
The original post: /r/datahoarder by /u/BowzasaurusRex on 2024-12-11 22:27:26.
5416
 
 
The original post: /r/datahoarder by /u/lern2swim on 2024-12-11 21:41:10.

I currently have the storage in my pc (for games and such) plus a synology ds418. I'd have to figure out if there's somewhere synology lists which raid format I used, because I don't remember, but I've currently only got 2.26tb left free out of 26.1tb total on it. The nas is primarily movies & TV that I use through kodi and plex with a little bit of other stuff stowed on it. I'm trying to decide how to handle my next round of storage expansion. If I bought bigger drives to put in the current enclosure I'd still need to make use of the current drives. And if I moved the current drives into a new bigger enclosure and added more drives I'd have the 418 to use still. So, I'm thinking that maybe I should just buy a 2nd 418 or similar enclosure and just split the two of them. But I'd love to hear any other suggestions for what some of y'all would do in this situation.

5417
 
 
The original post: /r/datahoarder by /u/nicox11 on 2024-12-11 20:38:15.

I have a synology NAS with 6 8Tb WD Red plus drive (WD80EFZZ).

I am trying to buy 2 new 8Tb WD Red plus drives but cannot find my exact model.

What I find now is WD80EFPX, WD80EFAX or WD80EFBX.

What should I pick ?

I would want a slower drive (= than 7200rpm) as I'm not crazy on performance, but I would want something close to what I have already.

Thank you for your help.

5418
 
 
The original post: /r/datahoarder by /u/Mermaid_Natalia on 2024-12-11 19:42:40.

I've begun my digitization journey, and have hit a bunch of bumps. The pci capture card I initially bought wasn't compatible, so I downgraded to a basic usb capture. After struggling between different programs (vdub, OBS, wintv, amarectv), before settling on vdub. Now that I have a (fairly low quality) digitization setup, I'd like to try getting through some of my vhs.

I'd like to start with vhs recorded from tv. I've done only two experiments so far. The first output a 10g raw file, which was increased to a 14g file after post-processing in vdub for sharpening/deinterlacing. The second tape is a similar size. I have a TB of storage on my pc, but I foresee that filling up quickly-- plus I use it for gaming, so I don't want to overload it.

Here are my questions:

  1. How do you handle your large digital footprint? Do you transfer the large digital files to an external hard drive? USB? DVDs? Cloud? Have you experienced any data/quality loss from those methods?
  2. I am very eager to share my digitized tapes. I expect uploading directly to YouTube, or any other video sharing platform, would cause issues with copyright takedowns. Have you ever uploaded publicly? If so, where? What are your experiences?
5419
 
 
The original post: /r/datahoarder by /u/Daavid1 on 2024-12-11 19:28:02.
5420
 
 
The original post: /r/datahoarder by /u/LeviAEthan512 on 2024-12-11 18:49:33.

I've always been a data hoarder at heart, but I'm only now starting to do it properly, with an organised set up.

I just ordered a couple of Ironwolf Pro drives at a massive discount, but it's in too good to be true territory. Like $150 for 12TB. And it's definitely Pro, I even checked the SKU.

I know of those drives (also batteries) that advertise a capacity more than what they actually have. I was and am wary of this, but some research alleviated my fears enough to order, but not enough to sit quietly without complaining lol.

The store I bought from seems to be reputable. According to this post at least. It's Greeno. I'm Singaporean, not Filipino, but we have Lazada here, too. There are differences, but this shop in SG has good reviews. Literally the only suspicious thing is the price. It passes all other checks.

When the drives arrive, I'll check the serial number with Seagate and the SMART data of course. They (the store) promises a refund if the products are found to not be genuine, which is nice. And I trust the platform's QA/refund system.

But about total capacity, I haven't been able to find any way to verify besides actually filling it with 12TB of whatever data. Thing is, I don't have that much data. I guess I could paste multiple copies of my Steam library for example, but that sounds tedious and my layman ass doesn't know if it's impossible for the drive to recognise that it's duplicate data and link them together. Also, if like 10% of files are faked/corrupted, it could potentially take a really long time to click on stuff until something fails, not to mention it's basically proving a negative.

I know of using Window's own disk management or diskpart to make volumes, and as a second test, I've heard GSmartControl is good. Are these 100% reliable, or should I really scrounging together data to go the manual route?

If you're curious, this is the exact one I bought. Only 3 ratings, but the store has a ton, including this, which is basically the same. The only difference I can see is if the warranty comes from the local distro or... seemingly Seagate's own international office. I went with the local one because it sounds like less of a hassle. Different companies do international warranty differently, but I've had some that require you ship it to their global HQ.

5421
 
 
The original post: /r/datahoarder by /u/BreakfastAntelope on 2024-12-11 17:53:23.

What is the most precious (subjectively) data you have stored and backed up over the years?

5422
 
 
The original post: /r/datahoarder by /u/DiabloIV on 2024-12-11 16:49:15.

A radio DJ I work with has bought basically every jazz CD that has been released since the early 90's. He has no desire to digitize his library, but I want a plan for when he retires. I think the collection is impressive, and significant enough to preserve. I also fear that if he's gone management will break up, donate, sell, and otherwise dispose of the collection.

If I could do it for less than $5k I'd be happy. I wouldn't mind it taking months. as long as it doesn't require constant monitoring and input.

5423
 
 
The original post: /r/datahoarder by /u/asblade_ on 2024-12-11 15:41:29.

Hi All

So, I want to store a large quantity of data (photos, PDFs, video/ audio files, etc) in the most permanent way possible. Since it will be nothing that needs access all the time, I was aiming for something cheaper than one of those NAS systems. My initial idea was multiple high-capacity HDs, perhaps in a RAID. I considered media like blu-rays but I think it would be needed a huge amount of disks. Not be space efficient...

I am running out of ideas....I considered media like Blu-rays, but I think it would need a huge amount of disks. What does the hivemind here suggest?

5424
 
 
The original post: /r/datahoarder by /u/vghgvbh on 2024-12-11 14:09:34.

First of all: Thank You for all your help in advance!

I work voluntarily for a non-profit youth-club in my neighborhood.

At the beginning of next year, I will be taking on the role of chairman for our organization and wanted to address the procurement of new PCs for our association, funded through donations we have received.

One of the items on my to-do list for next year was to purchase 20 lifetime licenses of Macrium Reflect Workstation. The plan was to implement a scheduled weekly incremental backup of each PC locally and then automatically sync it to the local NAS via Syncthing.

However, my plan has now been destroyed as Macrium switched to subscription base only and is demanding over €1,000 annually for our setup year after year. Which we cannot afford.

So. What can we use now, for our use-case?

I always favored Macrium until now, because I made good personal experiences with it at home, and it always worked flawlessly for me.

Please note that we do not have a network administrator or similar expertise available, nor can we afford one. Our team consists of volunteers with some technical background, and our hardware setup is relatively simple: a basic NAS, twenty PCs and laptops, and some peripheral devices.

This context makes it even more important for us to find a straightforward and user-friendly solution that can be managed with the resources and skills we have at hand.

5425
 
 
The original post: /r/datahoarder by /u/Visible_Bake_5792 on 2024-12-11 12:16:48.

SSD are much more expensive than last year, at least in Europe. I found a 4 TB SSD on Ediloca web site. The price was low, I tried it. The SSD works and I tested its capacity with f3write/f3read, all good.

But it appears to have a quick 1TiB Flash zone followed by the remaining 2.7 TiB / 2.9 TB in slow flash.

(See https://en.wikipedia.org/wiki/Byte#Multiple-byte_units for difference between TiB and TB)

Speed varies with the file system and mount options, anyway I managed to reach ~ 350-400 MiB/s on the first TiB and then 50-60 MiB/s on the remaining space.

I asked the support if they had some clue: magical mkfs or mount options, firmware upgrade... They just told me that all SSDs are like this, and I should wait ~ 15 minutes until the quick zone recovers. I suspected that this was untrue, anyway I tried writing 250 GB files and wait half an hour between each file. Same result!

Has anybody seen a similar behaviour? Is there something I could do to make the performance more constant? I tried mounting a BTRFS filesystem with ssd_spread, that was worse (just slower everywhere).

My last test, with redundant or verbose lines suppressed. It was running on a N5105 mini PC with 16 GiB RAM. I previously tried on another machine with an utterly different SATA controller, and got the same results.

# mkfs.btrfs -f -d single -m dup --csum xxhash64 -O extref,no-holes,block-group-tree,free-space-tree,squota /dev/sda1

# mount -o ssd,lazytime,nobarrier,nodiratime /dev/sda1 /m/

# for I in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 ; do

sync; date "+%s $I" >> /tmp/ssd.log;

dd if=/dev/zero of=/m/$I bs=128k count=1907349 status=progress

date "+%s $I" >> /tmp/ssd.log ; sync; sleep 1800

done

250000048128 bytes (250 GB, 233 GiB) copied, 711.98 s, 351 MB/s

# There was some activity on the machine, dd probably fight for the buffer cache with other processes on the first file

250000048128 bytes (250 GB, 233 GiB) copied, 550.064 s, 454 MB/s

250000048128 bytes (250 GB, 233 GiB) copied, 629.865 s, 397 MB/s

250000048128 bytes (250 GB, 233 GiB) copied, 541.539 s, 462 MB/s

# All this was written at full speed, 1 TB = 932 GiB

# It seems that the quick zone is 1 TiB, so there is still # 1024-932 = 92 GiB in the quick zone

250000048128 bytes (250 GB, 233 GiB) copied, 2097.48 s, 119 MB/s

# This speed is consistent with 92 GiB at full speed and 233 - 92 = 141 GiB at slow speed

# All the remaining files were written at slow speed.

250000048128 bytes (250 GB, 233 GiB) copied, 4228.37 s, 59.1 MB/s

250000048128 bytes (250 GB, 233 GiB) copied, 3756.49 s, 66.6 MB/s

# At some point I run "fstrim -v -a" just in case. BTRFS should TRIM the SSD if needed in the background (discard=async)

# fstrim did not help

250000048128 bytes (250 GB, 233 GiB) copied, 3731.19 s, 67.0 MB/s

250000048128 bytes (250 GB, 233 GiB) copied, 3823.29 s, 65.4 MB/s

250000048128 bytes (250 GB, 233 GiB) copied, 3852.8 s, 64.9 MB/s

250000048128 bytes (250 GB, 233 GiB) copied, 3789.42 s, 66.0 MB/s

250000048128 bytes (250 GB, 233 GiB) copied, 3870.74 s, 64.6 MB/s

250000048128 bytes (250 GB, 233 GiB) copied, 3847.66 s, 65.0 MB/s

250000048128 bytes (250 GB, 233 GiB) copied, 3997.83 s, 62.5 MB/s

250000048128 bytes (250 GB, 233 GiB) copied, 4095.57 s, 61.0 MB/s

250000048128 bytes (250 GB, 233 GiB) copied, 4169.98 s, 60.0 MB/s

In the end:

# df -h /m

Filesystem Size Used Avail Use% Mounted on

/dev/sda1 3.8T 3.7T 75G 99% /m

# ls -sh /m

total 3.7T

233G 1 233G 11 233G 13 233G 15 233G 2 233G 4 233G 6 233G 8

233G 10 233G 12 233G 14 233G 16 233G 3 233G 5 233G 7 233G 9

#

view more: ‹ prev next ›