It's A Digital Disease!

23 readers

1 users here now

This is a sub that aims at bringing data hoarders together to share their passion with like minded people.

founded 2 years ago

MODERATORS

bOt@zerobytes.monster

Made a little tool to download all of Wikipedia on a weekly basis (zerobytes.monster)

submitted 3 months ago by bOt@zerobytes.monster to c/datahoarder@zerobytes.monster

0 comments fedilink hide all child comments

The original post: /r/datahoarder by /u/ternera on 2025-05-01 16:44:45.

Hi everyone. This tool exists as a way to quickly and easily download all of Wikipedia (as a .bz2 archive) from the Wikimedia data dumps, but it also prompts you to automate the process by downloading an updated version and replacing the old download every week. I plan to throw this on a Linux server and thought it may come in useful for others!

Inspiration came from the this comment on Reddit, which asked about automating the process.

Here is a link to the open-source script: https://github.com/ternera/auto-wikipedia-download

no comments (yet)

sorted by: hot top controversial new old

there doesn't seem to be anything here