The robots.txt
file on this instance (https://kbin.social/robots.txt) disallows crawling.
By contrast, it appears that lemmy.ml and lemmy.world allow crawling (https://lemmy.ml/robots.txt).
"Random" is the place where all the content from the Fediverse that couldn't be classified into any other magazine ends up.
The robots.txt
file on this instance (https://kbin.social/robots.txt) disallows crawling.
By contrast, it appears that lemmy.ml and lemmy.world allow crawling (https://lemmy.ml/robots.txt).