this post was submitted on 10 Apr 2024
6 points (87.5% liked)

cybersecurity

5303 readers
51 users here now

An umbrella community for all things cybersecurity / infosec. News, research, questions, are all welcome!

Community Rules

Enjoy!

founded 2 years ago
MODERATORS
 

Weekly thread to discuss whatever you’re working on, big or small, at work or in your free time.

you are viewing a single comment's thread
view the rest of the comments
[–] dubbel@discuss.tchncs.de 4 points 2 years ago (1 children)

Private project, not really security related: Crawling robots.txts to gather some statistics on which bots people are most often excluding - weirdly I couldn't find any recent/regularly updated stats on this.