The original post: /r/datahoarder by /u/BosnianSlavemaster on 2024-07-14 17:00:14.
I'm currently trying to download mapgenie.io (which also hosts rdr2map.com ) for offline use (or in case it ever goes down). I've tried just right clicking in the browser and clicking "Save as..." but it only saves the current page, and I have no idea how to structure them in folders if I ever have to download them. Not only that, but the pages look all messed (it seems because of javascript or something).
I've tried Httrack but, not only is it excruciatingly slow (because I imagine it is going over the same links again and again), but the display is also messed up. That, and when I tried changing a few settings, it just resulted in showing me html files in plain text instead of displayed. It is also unable to access the images used in those maps (because mapgenie uses tiles to display the map for zooming and stuff). For example, here's a tile from the RDR2 map, which I found by using the record function in the network tab, dev tools (F12). I've let it run for about 20 hours combined (on multiple tries, my longest being 14h, which amounted to over 60 GB!!!) and I haven't seen a single .jpg like this apart from the default markers the website uses (location icons).
Now, I'm looking for anything that could "allow me to download a website" (excuse me, I don't know the terminology, not sure what I'm doing), and I've seen curl and wget proposed, but these things elude me. To add to that, apparently it's hard to "download a website" because of javascript. Such is why it is displayed all messed up.
So, please, help me, because I know I'll regret not archiving this website. Also, I've got multiple terabytes of storage, I don't think this website is any bigger than 20 GB.