Private project, not really security related: Crawling robots.txts to gather some statistics on which bots people are most often excluding - weirdly I couldn't find any recent/regularly updated stats on this.
this post was submitted on 10 Apr 2024
6 points (87.5% liked)
cybersecurity
5299 readers
54 users here now
An umbrella community for all things cybersecurity / infosec. News, research, questions, are all welcome!
Community Rules
- Be kind
- Limit promotional activities
- Non-cybersecurity posts should be redirected to other communities within infosec.pub.
Enjoy!
founded 2 years ago
MODERATORS
Not strictly teh ciberrz, but upgrading to OpenBSD 7.5. Might rebuild a mail server this weekend.
New PSIRT drop from Fortinet last night. Making sure none of our customers are running patched versions of the ask.