The original post: /r/datahoarder by /u/Hungry-Editor6066 on 2024-08-17 08:59:48.

The original post: /r/datahoarder by /u/Normal-Fall9455 on 2024-08-17 00:57:48.

Any advice, suggestion, caution

• RAID 5 setup on above,
• How much RAM will be sufficient (Supports DDR4 2400 / 2133 non-ECC, un-buffered memory)
• How to increase SATA ports? Storage- 4 x SATA3 6.0 Gb/s Connectors, support NCQ, AHCI and Hot Plug* 1 x M.2 Socket, supports M Key type 2230/2242/2260/2280 M.2 SATA3 6.0 Gb/s module *If M2_1 is occupied by a SATA-type M.2 device, SATA3_0 will be disabled.
• -It is mining MOBO so - 1 x PCI Express 3.0 x16 Slot (PCIE2: x16 mode) and 12 x PCI Express 2.0 x1 Slots
• -Gigabit LAN 10/100/1000 Mb/s (is it sufficient or network card needed?)

The original post: /r/datahoarder by /u/Practical-Plan-2560 on 2024-08-17 00:53:19.

I currently have an Asustor NAS (first NAS I've owned). And I'm getting close to maxing it out. I read there is an expansion unit I can buy which will get me a few more drive slots, but after that, I'm kinda out of luck.

My question is how do you scale your data storage solution? To me it's really awesome to just have one single connection and see all my data. I could of course add another NAS and just keep it completely separate and decide which files go on which NAS (maybe each has it's own purpose). But that is a lot of overhead and management required.

I'm not even sure it's possible to combine NAS devices infinitely like I'm wanting to do. Or maybe it's possible with another solution but not Asustor?

(I know I'm a bit premature in asking this, and planning for unlimited scability isn't totally necessary, but also don't wanna continue investing in Asustor devices if there is a better alternative out there.)

The original post: /r/datahoarder by /u/Revolutionary-Sky442 on 2024-08-16 23:14:28.

Hey everyone,

I plan to build a server with around 40-60TB (or more) of SSD storage. I want to keep this build cost-effective but also avoid a lot of hassle. I've built PCs before, and I remember using NVMe/M.2 drives sometimes disable certain SATA ports because they share the same bandwidth. This got me thinking about how to set up a bunch of SSDs without running into problems.

So, I’ve got a few questions:

1. Avoiding Bottlenecks: How can I avoid bottlenecks if I want to use 10+ SSDs in one system? I’ve heard that using adapters or extenders can make multiple drives share the same I/O, which might slow things down. What’s the best way to set things up so that each drive runs at full speed?
2. Best Setup: What’s the ideal mix of NVMe/M.2 and SATA SSDs to get a lot of storage without losing performance? Should I be looking at specific motherboards, RAID controllers, or PCIe expansion cards that can handle this?
3. Cost vs. Simplicity: I want to keep costs down but don’t want to complicate things. Are there trade-offs I should be aware of between going for higher-end stuff with more lanes and bandwidth versus cheaper options that might be harder to set up?
4. Future-Proofing: As storage needs grow, how should I plan for future expansion??

I’d appreciate any advice on how to approach this build, including specific hardware recommendations or tips on avoiding headaches.

If you're going to ask, "Why are you using SSD?" I am using it for random access hot storage.

P.S I've searched this and many other reddits/online resources but I did not find an answer I was looking for. Thank you!

The original post: /r/datahoarder by /u/jeffdagod on 2024-08-16 22:27:04.

People help me out, I'm dying lol!!!

Quick Rundown:

So I started storage spaces with the 3 drives (3X 4TB) knowing that 1 will be used for parity. So this leaves me at around 7.26Tb for storage. I added another drive to add memory but it seemed like it just optimized all the drives, it stated that my memory is at 10.89TB but I know I'm not using that yet. Why is it stating that I need to add 3 more drives because I am running low on space. I'm running a mini PC with a DAS if you guys needed to know the setup. Also if I add another drive (4TB) will I get the full 3.63TB which should put me at 14.52TB? Any help is greatly appreciated. I just need to know what I should do.

*Also I was messing around with the Storage Space size and I put 14Tb, it was originally at 10.89TB.

The original post: /r/datahoarder by /u/merlinthe_wizard on 2024-08-16 21:53:39.

Hello! I have a pile of external hdds and I don’t know how to organize and store them. They kind of just sit there in a pile. Wondering what type of solutions people have used on this sub?

The original post: /r/datahoarder by /u/accountmaster9191 on 2024-08-16 21:29:02.

my hdd is currently using btrfs. for months now, i haven't organized or moved any files/folders because the creation dates aren't transferred when i move it. i have tried cp -p, rsync -avN and rsync -av but none save the creation date. i read online that btrfs and most other filesystems don't allow the creation date to be modified. is there a filesystem that does allow this, and if there isn't what is an alternative to saving the creation dates.

The original post: /r/datahoarder by /u/dreameroftheblue on 2024-08-17 07:04:11.

I've used CDx to rip my album hoard for years, and it only recently suddenly started giving me this issue where it won't save any of my encoder settings? I'll try to pick a bitrate min, quality, or output samplerate, close the settings box, reopen it, and all the settings revert to default.

it still does lossless .wavs just fine, but I like to have both those (for my PC) and 320mbps versions (for my phone), and this is driving me up the wall. even redownloading the app or its portable version do the same thing.

am I doing something wrong?

The original post: /r/datahoarder by /u/rojo_salas on 2024-08-17 06:52:05.

Hey Guys!

If you’re using Backblaze B2 for your storage needs and want to make your URLs look more professional or customized, you can easily set up a custom subdomain to serve multiple B2 buckets. Here's a step-by-step guide on how to do this using Cloudflare.

Note: This is only based on my working knowledge and research. There may be another or better way to do this but this is my take for this. I may have said/done something wrong, feel free to correct me so I can correct this post.)

Materials Used:

• Custom Subdomain: b2.rojosalas.com (Replace with your own)
• CNAME for Backblaze: f002.backblazeb2.com (Based on your B2 Friendly URL)

Step 1: Set Up Your CNAME Record

1. Log in to Cloudflare:
   • Go to Cloudflare Dashboard and select your domain.
2. Navigate to DNS Settings:
   • Click on the DNS tab.
3. Add a CNAME Record: This CNAME setup directs requests from b2.rojosalas.com to your Backblaze B2 bucket endpoint.
   • Type: CNAME
   • Name: b2
   • Target: f002.backblazeb2.com
   • Proxy Status: Proxied (Cloudflare will handle the requests and apply your rules).

CNAME Record

Step 2: Create a Cloudflare Transform Rule

1. Navigate to Transform Rules:
   • Go to Rules > Transform Rules in the Cloudflare dashboard.
2. Create a New Transform Rule:
   • Click Create Transform Rule.
3. Configure the Rule: This rule will ensure that requests to b2.rojosalas.com will be properly mapped to the Backblaze B2 URL with the /file prefix.
   • Name: Provide a name for your rule (e.g., "Rewrite Backblaze B2 Paths").
   • When incoming requests match:
     • Expression: (http.host eq "b2.rojosalas.com") or (http.host eq "https://b2.rojosalas.com")
     • Path Matching: No separate field, include in the rule directly.
   • Rewrite URL:
     • Rewrite to: Dynamic
     • Value: concat("/file", http.request.uri.path)

Transform Rule

Example Usage

With the transform rule set up, you can now use your subdomain to access multiple buckets through the subdirectory. Here’s how it works:

• BucketA:
  • URL: https://b2.rojosalas.com/bucketA/images/pic1.png
  • Mapped to: https://f002.backblazeb2.com/file/bucketA/images/pic1.png
• BucketB:
  • URL: https://b2.rojosalas.com/bucketB/images/pic2.png
  • Mapped to: https://f002.backblazeb2.com/file/bucketB/images/pic2.png

With this setup, you can easily reference and serve files from multiple Backblaze B2 buckets under one custom subdomain.

Note: This is only based on my working knowledge and research. There may be another or better way to do this but this is my take for this. I may have said/done something wrong, feel free to correct me so I can correct this post.

The original post: /r/datahoarder by /u/DangerousDraper on 2024-08-17 04:47:52.

I'm on the hunt for a general purpose (win OS) DAM that's both free and self-hosted (local drive already has multi layer backup in place).

After years of managing gamedev assets in seperate tools (images + textures, sound, 3d models) I'd like to consolidate these into a single interface without going down the path of industrial software (windchill, team centre, etc.).

I'd even be happy if the recommendations were nothing more than a cataloguing interface which I could tag/link similar themes across the various media types.

The original post: /r/datahoarder by /u/hououinkyomaaaa on 2024-08-17 01:18:20.

The original post: /r/datahoarder by /u/justfindaway1 on 2024-08-17 00:50:21.

I copy the url of youtube channels so that jdownloader2 can grab all the videos and, by keeping everything as disabled completed downloads, I can use the "don't add already downloaded links" function to only download the new stuff.

However, unfortunately, from time to time some old stuff will somehow become a "new" download and be at risk of being downloaded again. (this is not just a problem with youtube, many other platforms too).

What method do you recommend to prevent downloading and storing duplicates? Unfortunately the files generated will also not bear identical names because of changes in quality / codec and possibly also channel owners changing the titles.

The original post: /r/datahoarder by /u/jammontreal on 2024-08-16 23:45:43.

I received a refurbished hard drive 12TB today and upon checking the SMART status, I found 5 historical entries in the Error Log:

https://preview.redd.it/dkwdwe2v34jd1.png?width=684&format=png&auto=webp&s=a0ed27f11c358ebb3721d5c2a2f14f07f0240a24

The vendor also did some tests after 38106 hours and apparently were successful:

https://preview.redd.it/vu1tbl2e54jd1.png?width=924&format=png&auto=webp&s=c8ca50c0728abea05b4b3961ca4c662859e5c08b

Is this disk broken? Should I return it? The SMART result is also weird. Was 0 power on time but 1.8PB data written. I think the vendor reset all the status.

https://preview.redd.it/aflhrxzk44jd1.png?width=1352&format=png&auto=webp&s=767c648cfb4e1e378afcf0c28e405d4444708a3c

The original post: /r/datahoarder by /u/LaundryMan2008 on 2024-08-16 21:27:37.

The original post: /r/datahoarder by /u/Killerclan117 on 2024-08-16 20:58:02.

I have gotten to the point that i've been hording HDD that are not even in use. Got a large mix of drives and some SSD's even.

Right now I have them on FB marketplace. Thinking Ebay might be an option but I know they take a decent cut of the profits when you use them now.

Anyone have any advice on where to post the drives? By this point I just want them gone.

The original post: /r/datahoarder by /u/thinvanilla on 2024-08-16 19:10:20.

The original post: /r/datahoarder by /u/Kriznick on 2024-08-16 18:20:13.

Hey, so I think I may be bottlenecking myself in my storage setup.

Preface: Storage setup is in my daily driver. No, there is no option to separate, due to price constraints for buying separate parts, nor do I want to turn on 2 computers when I want to access my data, nor do i want to leave 1 pc on 24/7.

I upgraded last christmas, and am finally making use out of the THOR v1 case I got a decade ago and making it the storage and gaming giant I always wanted. So upgraded everything, notably the following:

MB: Asrock b650e riptide wifi

CPU: Ryzen 7 7700x

M.2's: One WD Gen4x4 and two Gen3x4's

With that, I ALSO got a IBM M1115 LSI 9223-8i HBA and an Intel RES2SV240 expansion card to hook up the unreasonable amount of HDD/SSD's I've been hoarding (both from ArtOfServer of course, the patron saint of the sub).

So, for those who know how to count and know that PCIe lanes are a thing, unlike me who never had hit the cap before, that is more lanes than the Ryzen 7 can handle I THINK.

Ryzen 7 has 24 PCIe lanes to divy up.

16 go to graphics, 4 go to m.2 slot #1, and 4 divy up to other things.

I THINK I am using, with everything populated, AT LEAST 28, maybe more.

Is that correct? More importantly, does it matter? And if it does matter, will upgrading to a Ryzen 9 (if I can save up) correct it since it supports 28 lanes?

Thanks y'all

The original post: /r/datahoarder by /u/Rough_Community_1439 on 2024-08-16 18:14:09.

I am looking for a software that you can boot from a USB stick and then go and do a very very thorough check of a blank hard drive. I would like something pretty easy to use and that can run on a 686 processor.

The original post: /r/datahoarder by /u/Ninja-Trix on 2024-08-16 17:37:17.

https://preview.redd.it/3cfeqt2l82jd1.png?width=363&format=png&auto=webp&s=800e50a79f970e770fd130e7ca5da61eb21488b7

Before my drive failed, I had an archive of 1.26TB. Since then, I'm beyond where I started, and I still have to pull some more lost data. The old drive itself is completely dead, which is a shame because it had some lost media I spent an eternity digging for, and it's not worth it to me to send it to a clean room to pull what wasn't backed up elsewhere. It sucks, but I'm making due.

The music needs to get cleaned up as there are a bunch of duplicates and erronious metadata, but that will soon be rectified. Just happy to get this up and running. Contains all lossless FLAC in CD or higher, except where MP3, OGG, or M4A was the highest quality available ANYWHERE. I got music for DAYS!

The original post: /r/datahoarder by /u/WindowlessBasement on 2024-08-16 16:44:21.

The original post: /r/datahoarder by /u/Tbonesteakumz on 2024-08-16 16:27:42.

The original post: /r/datahoarder by /u/Emad_341 on 2024-08-16 16:21:47.

Recently I tried to download YouTube video but there are none good online downloader available and I don't have a PC that I can use IDM .Can anyone tell me any good online YouTube video downloader app /online downloader ? Tia

The original post: /r/datahoarder by /u/Apprehensive_Ask887 on 2024-08-16 15:57:14.

I just ordered a 4k Blu-ray drive / I play on building a pretty large media library with either plex or jellyfin - though I’m open to others. I want to backup photos/ documents from about 5 computers/ 12 phones & tablets. Probably create a few virtual machines. My main concern is being able to stream those 4k files remotely. Can someone just give me a list of like 3-4 synology products that would fit my needs? Good better best? Need some help . Price limit $1200

The original post: /r/datahoarder by /u/RiftTrips on 2024-08-16 15:20:53.

So I recently got some new HDDs for backing up my important files. I have already setup basic sync. Now when I make a change on my working drive do I need to run the sync again to update my backup drives files? Will it only update the changes or is it going to do the entire drive again?

Thanks for any help. I'm new to all of this.

The original post: /r/datahoarder by /u/Menaxerius_ on 2024-08-16 15:15:11.

https://preview.redd.it/havs33pheuid1.png?width=940&format=png&auto=webp&s=caf5dc8b4f0a8d3aa723ab7e0a5a7e4b06eb2345

Hi,

I'm currently on Truenas Scale, with one pool with two arrays 10x 10 tb and 10x 8 tb each in an z2 configuration. Mainly used as Plex & Arr Server. Storage is about to be 80% full. So I'm planning to do a full replacement of the System. I'm located in Switzerland, and power consumption is a bit of a concern, so thats something I want to factor in aswell, but isn't the main focus. 

Currently I'm on a Threadripper 1950X, with 64 GB Ram. 

Due to Pricing, I'm currently looking at Drives from Serverpartdeals, as their refurbished drives seem to be a great value, but I'm open for pretty much everything. 

My Plex Library is huge, so thats my main focus, as usability of Plex with metadata, previews and so on (see image):

My Budget isn't that limited, but I appreciate a good value. 

What would you get? And what configuration would you use? 

I have a bunch of 24 Bay 4 U Cases around, and will like use one of them again. 

Open for discussion. But I'm new to Unraid, but happy to learn. 

Thanks in advance

Mena