The original post: /r/datahoarder by /u/wobblydee on 2025-07-29 23:52:55+00:00.
Windows 11 mini pc
Ran wget with this entered
wget --mirror --convert-links --adjust-extension --page-requisites --no-parent http://example.com/
Thats what i found online somewhere to use
The website i saved is speedhunters.com an EA owned car magazine site thats going away
It seems to completely work but only a handful of images are present on the webpages with >95% articles missing the photos.
Due to the way wget did its files theyre all firefox html files for each page so i cant look to see if i have a folder of the images somewhere that i can find yet.
Did i mess up the command prompt or is it based on website construction?
I initially tried with httack on my gaming computer but after 8 hours i decided to get a mini pc locally for 20 bucks instead to run it and save power and thats when i went to wget. But i noticed httrack was saving photos but i couldnt click website links to other pages though i may just need to let it run its course.
Is there something to fix in wget while i let httrack run its course too
edit comment reply on potential fix in case it gets deleted
You need to span hosts, just had this recently.
/u/wobblydee check the image domain and put it in the allowed domains list along with the main domain.
Edit to add, now that i'm back at computer - the command should be something like this, -H is span hosts, and then the domain list keeps it from grabbing the entire internet - img.example.com should be whatever domain the images are from:
wget --mirror --convert-links --adjust-extension --page-requisites --no-parent -H --domains=img.example.com,example.com,www.example.com http://example.com/
yes you want example.com and www.example.com both probably.
oh edit 2 - didn't see you gave the real site - so the full command is:
wget --mirror --convert-links --adjust-extension --page-requisites --no-parent -H --domains=s3.amazonaws.com,speedhunters.com,www.speedhunters.com www.speedhunters.com