11–20 of 84 posts
> The exact adversary Anubis targets defeats it trivially. Wrong, anubis stops mass crawling of web pages, by requiring a proof of work, which makes accessing these websites more expensive. Anubis was not built to stop individual users with llms.
Hmm, I don’t think I agree. The author is claiming that Anubis is meant to stop individuals using LLMs, but I don’t think that’s its purpose. I believe its purpose is to reduce mass scraping of data that puts excessive load on systems. It does that by increasing the cost of scraping, not by preventing it entirely. Specifically, bad actors were ignoring robots.txt and rotating IPs to make blocking difficult. Anubis se…
How big of a deterrent is a millionth of a cent per page going to be? A non-existent one. Just the bandwidth from a residential proxy will cost the scraper orders of magnitude more.
Proof of work just isn't a viable counter-abuse challenge, even for something as low-yield as scraping. (It might be economically viable as a counter against some types of DDoS attacks, since they're even lower yield. But in practice it just moves the attack surface to the proof of work validation service.)
> Anubis is a Web AI Firewall Utility that weighs the soul of your connection[1] using one or more challenges in order to protect upstream resources from scraper bots.
* https://anubis.techaro.lol/docs/
> The Weighing of the Heart would take place in Duat (the Underworld), in which the dead were judged by Anubis, using a feather, representing Ma'at, the goddess of truth and justice responsible for maintaining order in the universe. The heart was the seat of the life-spirit (ka). Hearts heavier than the feather of Ma'at were rejected and eaten by Ammit, the Devourer of Souls.
* https://en.wikipedia.org/wiki/Weighing_of_souls#Ancient_Egyp...
The Anubis cookies are bound to particular IP address. The scrapers are often using a large set of IP addresses, so they'll be paying a far higher cost than this suggests.
> The exact adversary Anubis targets defeats it trivially. Wrong, anubis stops mass crawling of web pages, by requiring a proof of work, which makes accessing these websites more expensive. Anubis was not built to stop individual users with llms.
Claude Opus can make an optimized version of the proof of work solver that’s more than 10000x faster than the javascript one, in about 5 minutes time. Who is this stopping exactly? It’s the wrong tool in the wrong place.
> The exact adversary Anubis targets defeats it trivially. Wrong, anubis stops mass crawling of web pages, by requiring a proof of work, which makes accessing these websites more expensive. Anubis was not built to stop individual users with llms.
Claude Opus can make an optimized version of the proof of work solver that’s more than 10000x faster than the javascript one, in about 5 minutes time. Who is this stopping exactly? It’s the wrong tool in the wrong place.
Exactly, it's the same as Cloudflare captchas where only certain blessed devices and browsers can seem to actually pass it. Ironically, adding Anubis accelerates the death of the open web.
Not a fan of either solution, but the sites simply going offline does not promote an "open web" either.
I was curious about the name: > Anubis is a Web AI Firewall Utility that weighs the soul of your connection[1] using one or more challenges in order to protect upstream resources from scraper bots. * https://anubis.techaro.lol/docs/ > The Weighing of the Heart would take place in Duat (the Underworld), in which the dead were judged by Anubis, using a feather, representing Ma'at, the goddess of truth and justice respo…
That's knowledge usually learnt in primary school.
I was curious about the name: > Anubis is a Web AI Firewall Utility that weighs the soul of your connection[1] using one or more challenges in order to protect upstream resources from scraper bots. * https://anubis.techaro.lol/docs/ > The Weighing of the Heart would take place in Duat (the Underworld), in which the dead were judged by Anubis, using a feather, representing Ma'at, the goddess of truth and justice respo…
>>I was curious about the name: That's knowledge usually learnt in primary school.
Earlier quoted context omitted.
Claude Opus can make an optimized version of the proof of work solver that’s more than 10000x faster than the javascript one, in about 5 minutes time. Who is this stopping exactly? It’s the wrong tool in the wrong place.
Practically, the people indiscriminately scraping don't bother to do the work to bypass it or implement the POW test, which results in reduced CPU load for all the properties that were having trouble with scrapers before. Until scrapers start implementing it en masse then, it still serves its purpose.
If it becomes widespread (as it has been doing), they will. Anubis' strategy only works while it remains a niche approach only adopted by a small number of sites.