this post was submitted on 14 Dec 2025
7 points (100.0% liked)
Linux Prepper podcast
59 readers
1 users here now
See rules. Enjoy this experimental place to share content related to Linux Prepper podcast, which is also selfhosted to follow on the fediverse through Castopod as @linuxprepper@podcast.james.network
You may also use our discussion forum directly or join our chat on Matrix for Linux Prepper & Living Cartoon Company.
founded 1 month ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
A robots txt with a bunch of scraper user agents is a start, but there's no guarantee they honor it. A firewall with some dynamic lists of known scanners and scrapers is a bit more forceful method.