The original post: /r/datahoarder by /u/justinwypij on 2024-08-01 19:22:35.

Hey guys,

I've searched around and cant seem to find a clear answer. Is there any discernable difference in the life duration of an Ultra micro SD vs Extreme micro SD? Also is there any documentation of this?

Here is the use case.

1. I manage a traffic data surveying company in NY.
2. The technicians go into the field and install short term cameras
3. The cameras write to a micro SD card
4. the average amount of data written on a card a week is 32 gigs.
5. Each team has 65 cameras and and double the amount ( 130 micro sd cards) so if a transfer wasnt complete I have time to catch it before it goes into the field.
6. The camera writes to the card. The tech transfers the SD card to an SSD. Therefore every week the card is put into and out of a camera and into and out of a micro sd card reader.
7. The card is reformatted every time it is used via the digital video recorder
8. The cards are read by either a Samsung micro sd card reader OR a Sandisk micro sd card reader

At Everything But Stromboli the price difference between the cards is around $3 a card and im looking to but 350 cards. (If you guys know a different place to buy bulk cards PLEASE SHARE :P

Current Price difference

Sandisk Extreme 128GB - Quantity 350 - Price $3,955

Sandisk Ultra 128GB - Quantity 350 - Price $3167.50

The original post: /r/datahoarder by /u/Vorupini on 2024-08-01 17:14:04.

Hi there guys, I'll be honest, I'm a total noob and idk where to start lol.

I wanted to know what's the best way to get all the media files (music, images and videos) from games (both iso and apk). I've heard of Skraper, but I'm not sure if it gets the job done with APKs as well.

I'm just interested in the media inside the game (including unlockable content), not much in other type of written metadata.

Where do I start? Would you please lend me a hand? 😅

The original post: /r/datahoarder by /u/Zealousideal_Rate420 on 2024-08-01 08:09:58.

I'm trying to decide between Hetzner Storage Box or BackBlaze B2 for storing large but not critical data.

B2 uses S3, and even though I keep hearing how great S3 is, I have not seen the explanation outside of using it on AWS (where it fits perfectly with the ecosystem).

So, what does S3 offer for a casual hoarder that Hetzner Storage Boxes can't offer? Either for dumping non critical data and for backups

The original post: /r/datahoarder by /u/ECrispy on 2024-08-01 08:02:45.

use case - I have lots of local files, mostly in mhtml (saved from web using Chrome), pdf, txt, doc etc. I have these stored in folders with my own folder heirarchy.

issues - its getting too big to manage/search etc. size is 100GB+. There are lots of duplicates - i.e. same web page saved multiple times, or files that are very similar, or same content in multiple formats. its also hard to search/view

what I'd like: import all these into a selfhosted app. use some kind of tagging - the tags would initially use the local folder hierarchy, but hopefully some kind of AI and advanced document inspection (e.g mhtml files have a source url, which can be used to classify), identify duplicates (not just binary dups) etc. Have full text search over all docs.

is nextcloud/paperless-ng the recommended solution for documents, and can it do the above? Any other advice or tools I haven't heard of is welcome

The original post: /r/datahoarder by /u/ROMANREIGNS599 on 2024-08-01 05:38:08.

I heard that Jdownloader 2 automatically chooses the best quality. But I have tried downloading a 1080p 60 fps video twice. First, with no changes and second time, with selecting only the boxes of 1080p and 60 fps in plugin. This seems like something one can easily find online, but I couldn't :/.

Help me guys, I'm losing my sanity 🥲. I'm trying to download an age restricted video btw

The original post: /r/datahoarder by /u/RWKyuRem on 2024-08-01 04:15:15.

Hello everyone,

Recently, I bought a MacBook Pro for a good price, but the downside was that it came with only 512GB of space, which I'm on track of filling up.

So, I'm in the market for a centralized backup drive and also a centralized editing drive. I originally was hoping to buy Sabrent's 2-bay DAS, which has hardware RAID and JBOD, but it's not available for purchase in my country, nor is it shipped.

That's when I found the TerraMaster D4-320. It doesn't have onboard RAID or JBOD functionality, but from what I've read, you can create a RAID/JBOD array easily on macOS.

I have two 6TB WD NAS HDDs, which I wanted to put in the TerraMaster and create a RAID 1 array, and in the other two slots, add two Crucial MX500s as a JBOD or RAID 0 so I can edit off them.

Is this possible?

Thanks in advance to anyone who takes the time to help me :)

The original post: /r/datahoarder by /u/vriska1 on 2024-08-01 03:33:49.

The original post: /r/datahoarder by /u/ignoble93 on 2024-08-01 02:59:16.

New to streamlink. I noticed when I save log files there's no datestamp (e.g. [07:31:24 04:04:18.803008]) in each log line. Is there a right configuration to do it? Here's my command line:

    streamlink_cmd = [
            'streamlink',
            m3u8_url,
            'best',
            '-o', output_file,
            '--logfile', log_file,
        ]

Also a sample log extracted from streamlink:

[cli][info] Found matching plugin hls for URL https://.../playlist.m3u8
[cli][info] Available streams: 240p (worst), 480p, 720p, 1080p (best)
[cli][info] Opening stream: 1080p (hls)
cli][info] Writing output to
C:\Users\My-PC\Downloads\output_23_2024-07-31_22-33-08_362.ts
[cli][info] Stream ended
Interrupted! Exiting...
[cli][info] Closing currently open stream...

The original post: /r/datahoarder by /u/swansang on 2024-07-31 21:42:48.

Found my camera (Nikon Coolpix L23) from 2014 , it’s been lying under my bed for a decade. Switched its batteries and turns out it’s still working.

I wanted to check the old photos / videos and I couldn’t scroll past the 5th - 6th photo because the camera would freeze , and I had to remove the batteries and insert them back in because it was the only way to restart the camera , power button wouldn’t work.

So I took the SDHC card (SanDisk SDHC 4 GB) from the camera and inserted into my MacBook Pro M1. It took it a long time to read it.

When I clicked the folder that contains photos / videos it took it a long time too to open it , it also froze the SDHC card in the process. I couldn’t eject it by right clicking it.

When I opened the folder that contained the photos , I could preview only one or two of them , just like in the camera. Anytime I tried to preview or open any other photos it froze , the cursor spun for a while but in the end I couldn’t open them.

I ejected the SDHC card and now wonder , is there a chance that my Macbook got a virus from that decade old SDHC card? Or is the card is just corrupted?

Cursor spinning for a long time , SDHC card freezing , MB taking a while to read the card etc. worried me.

I scanned with Malwarebytes a couple of times and it says there is no threat. It’s been about a day and I’m having no problem with my pc. Just want to be sure 🤷

Thanks in advance!

The original post: /r/datahoarder by /u/nutrigrain on 2024-08-01 02:40:39.

I have recently started to look into doing backup of my photos and am able to do that with:

1. Use of immich iOS app to upload to my Ubuntu server to save all Photos and Videos
2. Use RSync to back that up to my NAS
3. Use BlackBlaze for remote backup.

I am left with files and documents I have on iCloud Drive. How do I back that up? It seems like there's no way for me to sync that to my NAS (QNAP) and then BlackBlaze to complete everything.

That's my only missing info that I'm wondering if you guys can help.

The original post: /r/datahoarder by /u/SillySeaCaptain on 2024-08-01 01:22:51.

A lot of the methods I've found online seem to be outdated, is there a current way to download playlists or songs?

The original post: /r/datahoarder by /u/leadbread on 2024-08-01 00:30:01.

Starting to get cold sweats. I have had a Sans Digital external RAID enclosure running three WD Red 3tb drives in RAID 5 for about ten years. I have not yet moved over to the two 16tb drives I have ready as an upgrade for reasons. A drive failed last night, showed up as a 3mb disk when I plugged it in separately, and the RAID stopped working/couldn't be accessed - after some fucking with the broken drive I installed one of the 16tb drives to rebuild the RAID, and while that was going, I noticed in Disk Management it showed up as an unallocated partition... I (probably very stupidly) allocated it to see what would happen but didn't format anything. Now with the broken drive installed, I see a 6tb raw partition in Disk Manager and the RAID manager shows a degraded RAID, and when I replace it with a working drive it automatically begins restoring the RAID, but at this point I don't even know what it's restoring so I stopped it. Everything is also Bitlocker encrypted so that adds another dumb layer to potential recovery efforts. I have no idea what to do now and I don't have any backups of pretty much my entire life since 2004. Any guidance for me?

The original post: /r/datahoarder by /u/Sauce_B0ss_ on 2024-08-01 00:29:34.

I am just looking for storage. It was used as a xbox one drive. Is 60 too expensive?

The original post: /r/datahoarder by /u/kbrongus3 on 2024-08-01 00:23:22.

Recently, a website I like to archive stuff from has switched to mobile app only. It used to have scripts blocking web browser devtools, but they decided that still isn't enough and went full lockdown mode. App only, no screencaps. Are we really limited to just ADP for this situation? The app doesn't write anything to android/data at all, so before I try to find a way through connecting to PC and using ADP, I'm wondering if there's something obvious I'm missing.

The original post: /r/datahoarder by /u/baconmaster24 on 2024-08-01 00:17:06.

Hey fellow data friends. The people over at r/golang suggested I share this lil utility I created in this thread. I made a utility called MyWhoop for people to download their Whoop data from the Whoop API. I mainly designed it for the purpose of retaining my data in case something happens one day, such as MySpace losing 12 yrs of data. Anyway, a friend of mine saw it and suggested I share it with the community. So, I spent some time polishing and documenting it. Anyway, feel free to use it to download your own Whoop data.

You can use it to download all your data in one single dump or configure it to download your data every day (last 24 hours). Anyway, thanks for reading, gang.

MyWhoop GitHub Repository

Getting Started Tutorial

PS: If there is interest, I can add more extensions to the exporter and notification services. Contributions are also welcome.

The original post: /r/datahoarder by /u/scndthe2nd on 2024-07-31 23:09:36.

Howdy!

The Problem

I'm going through and archiving my physical CDs and DVDs into ISOs, and I want to eventually burn them to BluRay disks so I can get them off my nas. I'm coming across the same problem I've always had with physical media, which is that I dont know What File is on Which disk?

So I'm looking into a database or library software for this.

What I Want

I want a database organization software where I can record the contents of a disk, then search that database for specific files after without having to plug the disc into the computer.

Nice to haves

I want the contents of any compressed files, including Checksums, File Sizes, Dates (created and modified). Ideally, this would also show if the file is available on more than one disk, and check my nas for files too.

Question

Is there any software that does this already? Should I just learn SQL and make my own database software?

The original post: /r/datahoarder by /u/HouseOnnaHill on 2024-07-31 22:05:58.

https://www.reddit.com/r/DataHoarder/comments/zkn59c/how_to_store_a_large_amount_of_data/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

The link for context.

The original post: /r/datahoarder by /u/ShyGuyGaming76 on 2024-07-31 20:16:58.

There's a lot of videos that I mean to download, but most of them just crap out and stop downloading halfway through. Does anyone know any good ones?

The original post: /r/datahoarder by /u/mastakebob on 2024-07-31 19:44:03.

Hoping for either affirmation or universal condemnation of my plan to resolve my storage capacity for the next ~decade.

The plan: Expand my NAS to meet my next ~decade of storage needs by buying a $250 syba or mediasonic 8 bay jbod enclosure and plugging it into my existing OMV NAS box via USB3. Will buy new drives to slap in as my data grows. At my current rate, 8 more bays will provide enough storage for a decade. This is instead of buying a Rosewill 24bay 4u chassis and learning all about server-grade technology (wtf is a backplane or a HBA?).

Background: I have a NAS that simply stores my Plex media. All Plex/Arr processing/management is done on the Plex minipc, the NAS simply provides the large media drive via an NFS share. The NAS is OpenMediaVault running mergeFS and SnapRAID, resides in a Fractal Design R5 case basically-max'd out with six mergerfs data drives and three snapraid parity drives providing 60TB of usable space (mixture of 8tb and 12tb SATA drives). I prioritize quietness and power efficiency, I have no real need for data redundancy or resiliency. If my Plex is down for 2 weeks while I wait for a new drive and a slow weekend to swap it in, no worries. I do run monthly full data backups onto spare jbod enclosures I store in my basement closet.

The Problem and Alternatives: At my current rate of data growth, I'll run out of storage in a bit over a year. I got basically no more R5 drive bays left. I need more cheap and quiet storage that will give me runway for the next 5-10 years. Swapping my existing 8TB for newer 12/16TB drives just feels like I'm kicking the can down the road. Buying a 24bay server chassis is the ideal, but I'm worried it'll be loud, I'll need to learn new hardware technologies, and maybe I'd need to invest in a lot of 'server-grade' new components like HBAs and backplanes?

My Plan: Just buy a $250 8bay USB jbod enclosure offa amazon, plug it into my current NAS via USB 3, and anytime I get close to running out of space, slap a new HDD into it. Not too concerned about visual looks as its in the basement, but am concerned about noise levels as that's where my telework desk is. Hoping the 'consumer-grade' jbod enclosures are somewhat quiet. As I'm just streaming media (and loading new media onto it), no need for superfast speeds or resilient/redundant architectures.

Thoughts? Affirmations? Condemnations? What super critical issue am I missing? Thanks for sharing your expertise!

The original post: /r/datahoarder by /u/fat_cock_freddy on 2024-07-31 19:25:42.

Hello! I'm looking for a service that I can mail a physical book to, and have them produce for me an epub or similar format ebook.

So far, I've found:

• https://boundbookscanning.com/
• ~~https://www.blueleaf-book-scanning.com/~~

Edit: I crossed off blueleaf because I reviewed their sample order and the quality of the ebook was complete trash.

I also saw 1dollarscan.com which is not an option because they refuse to return the original after scanning.

Has anyone used these services and can comment on the quality of their work? The price comes out roughly the same for either.

If there are other options available, I ask that they meet these requirements:

• I am aware there are in-home options for scanning / OCRing a book. I don't want to bother with this. I'm fine with paying.
• I do NOT want PDF. It has to be native ebook format, preferably epub. Converting a PDF to epub is NOT an option, I have played with this with various tools in the past and the result is vastly sub-par. I have queries out to the above companies as to how their conversion is done.
• They must NOT unbind the book and must mail it back to me afterwards.
• Homegamers welcome but I'm going to ask for past work & references.

Thanks in advance!

The original post: /r/datahoarder by /u/nsoifer on 2024-07-31 17:24:10.

Thinking about giving away my DS1522+ to a friend and upgrade to a rack. Any recommendations on what rack I could go for?

I would like to have something that can have 20-25 bays (after expanding) and has SHR. If not Synology, then something similar to SHR where I can combine different sizes of drives. Using it strictly for Plex.

I will put an SSD in one of the bays for the containers, so I don't need NVMe support. not sure about cache. I currently have cache set up in my DS, but no idea if it actually improves things or not. Moving the containers to an SSD though helped a lot so I will definitely use that on the rack.

I really like my DS1522+, it is just not big enough, so ideally I get something that is similar but bigger.

my top choice at the moment is the Synology RS2423RP+, but will gladly listen to any other suggestions.

The original post: /r/datahoarder by /u/poopin on 2024-07-31 17:22:12.

I would like to back up to the cloud. But I don’t know of anything that’s affordable also being reliable. This is going to be my personal movie/TV/audio collection if that matters.

Any ideas?

The original post: /r/datahoarder by /u/osrick97 on 2024-07-31 17:21:41.

I got some from a storage auction and have not had any luck with listing them online on ebay or facebook and was not getting any views. I tried googling more about them and found this subreddit and wanted to ask if anyone knows of companies that buy them. Here's what they look like

https://preview.redd.it/8ij03a2h2wfd1.jpg?width=500&format=pjpg&auto=webp&s=9298c93d3779a2bf00dfaa7f64ca34285ff3d8c3

https://preview.redd.it/4bb93cnh2wfd1.jpg?width=500&format=pjpg&auto=webp&s=9ccca066e0957669ea4285bed767cda5b0a3b422

The original post: /r/datahoarder by /u/sheepdot on 2024-07-31 14:44:52.

Hi,

My HPE LTO-8 drive "may be having problems finding servo code after loading a cartridge" according to the support ticket the RMI generates. It cannot successfully read from any tape I put in.

Has anyone had experience with a good shop for fixing tape drives?

Thanks.

The original post: /r/datahoarder by /u/blackbird2150 on 2024-07-31 14:22:58.

I've come to the point in my data acquisition journey where it's time to be serious about full backups. I have an 8-bay - 8x16TB (7D+1P) on Unraid (xfs).

I have a full set of drives already to act as the backup (again 8x16tb). The backups will be kept offsite, effectively cold storage, and I'll look to bring them home and run the backup process about every 2 months. For my personal data this will be a 4th copy, and my Media this will be the 2nd copy.

I have a Windows 11 machine with CS-382 case to allow for moving the backup hotswaps quickly, but I am figuring out how to establish the right process between windows and unraid to make it all work.... I'm reading through how to use rclone effectively so I suspect that will be the tool to operate the backups.

My 3 primary areas of risk are:

1. My dumbass makes a mistakes and deletes stuff
2. Drive failure
3. Weather (flood/fire) or other catastrophic loss event

My questions are:

1. For backups does one just move the data from the primary to the backup independent of it's source drive or do you want to focus on mirroring drive to drive?
   1. The latter makes more sense as if I concurrent drive failure (i.e. 2 drive failure with only 1 Parity) then i know exactly what I've lost.
2. Do you encrypt your backup drives (if you don't encrypt your primaries)? Thinking through what happens if someone takes my backup drives from the offsite location - what could they do with it? Minor concern all things considered.
3. How do you test your backups?
4. Any other tips you might have for someone starting out and needing to do an initial backup of 50tb?

Thanks!