A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
If that sounds familiar, it’s because it’s similar to how bitcoin mining works. Anubis is not literally mining cryptocurrency, but it is similar in concept to other projects that do exactly that
Did the author only now discover cryptography? It's like a cryptocurrency, just without currency, what a concept!
It's a perfectly valid way to explain it, though
If you try to show up with "cryptography" as an explanation, people will think of encrypting messages, not proof of work
"Cryptocurrency with the currency" really is the perfect single sentence explanation
It's quite similar.
Anubis is no challenge like a captcha. Anubis is a ressource waster, forcing crawler to resolve a crypto challenge (basically like mining bitcoin) before being allowed in. That how it defends so well against bots, as they do not want to waste their resources on needless computing, they just cancel the page loading before it even happen, and go crawl elsewhere.
No, it works because the scraper bots don't have it implemented yet. Of course the companies would rather not spend additional compute resources, but their pockets are deep and some already adapted and solve the challenges.
The point was never that Anubis challenges are something scrapers can’t get past. The point is it’s expensive to do so.
Some bots don’t use JavaScript and can’t solve the challenges and so they’d be blocked, but there was never any point in time where no scrapes could solve them.
To solve it or not do not change that they have to use more resources for crawling, which is the objective here. And by contrast, the website sees a lot less load compared to before the use of Anubis. In any case, I see it as a win.
But despite that, it has its detractors, like any solution that becomes popular.
But let's be honest, what are the arguments against it?
It takes a bit longer to access for the first time? Sure, but that's not like you have to click anything or write anything.
It executes foreign code on your machine? Literally 90% of the web does these days. Just disable JavaScript to see how many website is still functional. I'd be surprised if even a handful does.
The only people having any advantages at not having Anubis are web crawler, be it ai bots, indexing bots, or script kiddies trying to find a vulnerable target.
I use uMatrix, which blocks js by default, so it is a bit inconvenient to have to enable js for some sites. websites which didn't need it before, which is often the reason I use them, now require javascript.
Sure, I'm not arguing against Anubis! I just don't think the added compute cost is sufficient to keep them out once they adjust.
Conceptually, you could just really twist the knobs up. A human can wait to read a page for 15 seconds. But you're trying to scrape 100,000 pages and they each take 15 seconds... You can make it expensive in both power and time that's a win.
I'm against it for several reasons. Running unauthorized heavy duty code on your end. It's not JS in order to make your site functional, it's heavy calculations unprompted. If they would add simple button "click to run challenge" would at least be more polite and less "malware-like".
For some old devices the challenge last over 30 seconds, I can type a captcha in less time than that.
It blocks behind the necessity to use a browser several webs that people (like the article author) tend to browse directly from a terminal.
It's a delusion. As shown by the article author solving the PoW challenge is not that much of an added cost. Span reduction would be the same with any other novel method, crawlers are just not prepared for it. Any prepared crawler would have no issues whatsoever. People are seeing results just because it's obscurity, not because it really works as advertised. And in fact I believe some sites are starting to get crawled aggressively despite anubis as some crawlers are already catching up with this new Anubis trend.
Take into account that the challenge needs to be light enough so a good user can enter the website in a few seconds running the challenge on a browser engine (very inefficient). A crawler interested in your site could easily put up a solution to mine the PoW using CUDA in a GPU which would be hundreds if not thousands of times more efficient. So the balance of difficulty (still browsable for users but costly to crawl) is not feasible.
It's not universally applicable. Imagine if all internet were behind PoW challenges. It would be like constant Bitcoin mining, a total waste of resources.
The company behind Anubis seems more shady to me each day. They feed on anti-AI paranoia, they didn't even answer the article author valid critics when he email them, they use clearly PR language aimed to convince and please certain demographics to place their product. They are full of slogans but lack substance. I just don't trust them.
Fair point. I do agree with the "clic to execute challenge" approach.
For the terminal browser, it has more to do with it not respecting web standard than Anubis not working on it.
As for old hardware, I do agree that a temporization could be good idea, if it wasn't so easy to circumvent. In such case bots would just wait in the background and resume once the timer is fullified, which would vastly decrease Anubis effectiveness as they don't uses much power to do so. There isn't really much that can be done here.
As for the CUDA solution, that will depend on the implemented hash algorithm. Some of them (like the one used by Monero) are made to vastly more inefficient on GPU than it is on the CPU. Moreover, GPU servers are far more expensive to run than CPU ones, so the result would be the same : crawling would be more expensive.
In any case, the best solution would be by far to make it a legal requirement to respect robot.txt, but for now the legislators prefer to look the other way.
because anime catgirls are the best
The current version of Anubis was made as a quick "good enough" solution to an emergency. The article is very enthusiastic about explaining why it shouldn't work, but completely glosses over the fact that it has worked, at least to an extent where deploying it and maybe inconveniencing some users is preferable to having the entire web server choked out by a flood of indiscriminate scraper requests.
The purpose is to reduce the flood to a manageable level, not to block every single scraper request.
And it was/is for sure the lesser evil compared to what most others did: put the site behind Cloudflare.
I feel people that complain about Anubis have never had their server overheat and shut down on an almost daily basis because of AI scrapers 🤦
Out of curiosity, what’s the issue with Cloudflair? Aside from the constant worry they may strong arm you into their enterprise pricing if you’re site is too popular lol. I understand support open source, but why not let companies handle the expensive bits as long as they’re willing?
I guess I can answer my own question. If the point of the Fediverse is to remove a single point of failure, then I suppose Cloidflare could become a single point to take down the network. Still, we could always pivot away from those types of services later, right?
Cloudflare has IP banned me before for no reason (no proxy, no VPN, residential ISP with no bot traffic). They've switched their captcha system a few times, and some years it's easy, some years it's impossible.
The problem is that the purpose of Anubis was to make crawling more computationally expensive and that crawlers are apparently increasingly prepared to accept that additional cost. One option would be to pile some required cycles on top of what's currently asked, but it's a balancing act before it starts to really be an annoyance for the meat popsicle users.
That's why the developer is working on a better detection mechanism. https://xeiaso.net/blog/2025/avoiding-becoming-peg-dependency/
The article is very enthusiastic about explaining why it shouldn't work, but completely glosses over the fact that it has worked
This post was originally written for ycombinator "Hacker" News which is vehemently against people hacking things together for greater good, and more importantly for free.
It's more of a corporate PR release site and if you aren't known by the "community", calling out solutions they can't profit off of brings all the tech-bros to the yard for engagement.
This… makes no sense to me. Almost by definition, an AI vendor will have a datacenter full of compute capacity.
Well it doesnt fucking matter what "makes sense to you" because it is working...
Its being deployed by people who had their sites DDoS'd to shit by crawlers and they are very happy with the results so what even is the point of trying to argue here?
New developments: just a few hours before I post this comment, The Register posted an article about AI crawler traffic. https://www.theregister.com/2025/08/21/ai_crawler_traffic/
Anubis' developer was interviewed and they posted the responses on their website: https://xeiaso.net/notes/2025/el-reg-responses/
In particular:
Fastly's claims that 80% of bot traffic is now AI crawlers
In some cases for open source projects, we've seen upwards of 95% of traffic being AI crawlers. For one, deploying Anubis almost instantly caused server load to crater by so much that it made them think they accidentally took their site offline. One of my customers had their power bills drop by a significant fraction after deploying Anubis. It's nuts.
So, yeah. If we believe Xe, OOP's article is complete hogwash.