Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why are anime catgirls blocking my access to the Linux kernel?

lock.cmpxchg8b.com

Post reply on HN
Print view View original

291–300 of 968 posts

Re: Why are anime catgirls blocking my access to the Linux kernel?

#291

Post by tptacek »

xena wrote: post #289
tptacek wrote: post #279

Earlier quoted context omitted.

Respectfully, I think it's you missing the point here. None of this is to say you shouldn't use Anubis, but Tavis Ormandy is offering a computer science critique of how it purports to function. You don't have to care about computer science in this instance! But you can't dismiss it because it's computer science. Consider: An adaptive password hash like bcrypt or Argon2 uses a work function to apply asymmetric costs t…

For what it's worth, kernel.org seems to be running an old version of Anubis that predates the current challenge generation method. Previously it took information about the user request, hashed it, and then relied on that being idempotent to avoid having to store state. This didn't scale and was prone to issues like in the OP. The modern version of Anubis as of PR https://github.com/TecharoHQ/anubis/pull/749 uses a d…

Right, I get that. I'm just saying that over the long term, you're going to have to find asymmetric costs to apply to scrapers, or it's not going to work. I'm not criticizing any specific implementation detail of your current system. It's good to have a place to take it!

I think that's the valuable observation in this post. Tavis can tell me I'm wrong. :)

Re: Why are anime catgirls blocking my access to the Linux kernel?

#292

Post by murderfs »

sugarpimpdorsey wrote: post #212
Lammy wrote: post #175

Earlier quoted context omitted.

[flagged]

> Anubis is a clone of Kiwiflare, not an original work, so you're actually sort of half-right: Interesting. That itself appears to be a clone of haproxy-protection. I know there has also been an nginx module that does the same for some time. Either way, proof-of-work is by this point not novel. Everyone seems to have overlooked the more substantive point of my comment which is that it appears kernel.org cheaped out a…

> My point is, cough up a few bucks for a license you chiselers.

You mean this one? https://github.com/TecharoHQ/anubis/blob/main/LICENSE

Re: Why are anime catgirls blocking my access to the Linux kernel?

#293

Post by mixologic »

ok123456 wrote: post #288

Why is kernel.org doing this for essentially static content? Cache control headers and ETAGS should solve this. Also, the Linux kernel has solved the C10K problem.

Because its static content that is almost never cached because its infrequently accessed. Thus, almost every hit goes to the origin.

Re: Why are anime catgirls blocking my access to the Linux kernel?

#294

Post by seabrookmx »

unclad5968 wrote: post #223
nemomarx wrote: post #134

Earlier quoted context omitted.

Mozilla is used in the user agent string of all major browsers for historical reasons, but not necessarily headless ones or so on.

Oh that's interesting, I had no idea.

There's some sites[1] that can print your user agent for you. Try it in a few different browsers and you will be surprised. They're honestly unhinged.. I have no idea why we still use this header in 2025!

[1]: https://dnschecker.org/user-agent-info.php

Re: Why are anime catgirls blocking my access to the Linux kernel?

#295

Post by Aurornis »

johnklos wrote: post #244

This is a usually technical crowd, so I can't help but wonder if many people genuinely don't get it, or if they are just feigning a lack of understanding to be dismissive of Anubis. Sure, the people who make the AI scraper bots are going to figure out how to actually do the work. The point is that they hadn't, and this worked for quite a while. As the botmakers circumvent, new methods of proof-of-notbot will be made…

> This is a usually technical crowd, so I can't help but wonder if many people genuinely don't get it, or if they are just feigning a lack of understanding to be dismissive of Anubis.

This is a confusing comment because it appears you don’t understand the well-written critique in the linked blog post.

> This is like those simple things on submission forms that ask you what 7 + 2 is. Of course everyone knows that a crawler can calculate that! But it takes a human some time and work to tell the crawler HOW.

The key point in the blog post is that it’s the inverse of a CAPTCHA: The proof of work requirement is solved by the computer automatically.

You don’t have to teach a computer how to solve this proof of work because it’s designed for the computer to solve the proof of work.

It makes the crawling process more expensive because it has to actually run scripts on the page (or hardcode a workaround for specific versions) but from a computational perspective that’s actually easier and far more deterministic than trying to have AI solve visual CAPTCHA challenges.

Re: Why are anime catgirls blocking my access to the Linux kernel?

#296

Post by seba_dos1 »

throwawayffffas wrote: post #196
debugnik wrote: post #109

Earlier quoted context omitted.

> as AI scrapers bother implementing the PoW That's what it's for, isn't it? Make crawling slower and more expensive. Shitty crawlers not being able to run the PoW efficiently or at all is just a plus. Although: > which is trivial for them, as the post explains Sadly the site's being hugged to death right now so I can't really tell if I'm missing part of your argument here. > figure out that they can simply remove "M…

> That's what it's for, isn't it? Make crawling slower and more expensive. The default settings produce a computational cost of milliseconds for a week of access. For this to be relevant it would have to be significantly more expensive to the point it would interfere with human access.

...unless you're sus, then the difficulty increases. And if you unleash a single scrapping bot, you're not a problem anyway. It's for botnets of thousands, mimicking browsers on residual connections to make them hard to filter out or rate limit, effectively DDoSing the server.

Perhaps you just don't realize how much did the scraping load increase in the last 2 years or so. If your server can stay up after deploying Anubis, you've already won.

Re: Why are anime catgirls blocking my access to the Linux kernel?

#297

Post by [deleted] »

johnklos wrote: post #244

This is a usually technical crowd, so I can't help but wonder if many people genuinely don't get it, or if they are just feigning a lack of understanding to be dismissive of Anubis. Sure, the people who make the AI scraper bots are going to figure out how to actually do the work. The point is that they hadn't, and this worked for quite a while. As the botmakers circumvent, new methods of proof-of-notbot will be made…

[deleted]

Re: Why are anime catgirls blocking my access to the Linux kernel?

#298

Post by overfeed »

cakealert wrote: post #276
johnklos wrote: post #244

This is a usually technical crowd, so I can't help but wonder if many people genuinely don't get it, or if they are just feigning a lack of understanding to be dismissive of Anubis. Sure, the people who make the AI scraper bots are going to figure out how to actually do the work. The point is that they hadn't, and this worked for quite a while. As the botmakers circumvent, new methods of proof-of-notbot will be made…

This arms race will have a terminus. The bots will eventually be indistinguishable from humans. Some already are.

> The bots will eventually be indistinguishable from humans

Not until they get issued government IDs they won't!

Extrapolating from current trends, some form of online ID attestation (likely based on government-issued ID[1]) will become normal in the next decade, and naturally, this will be included in the anti-bot arsenal. It will be up to the site operator to trust identities signed by the Russian government.

1. Despite what Sam Altman's eyeball company will try to sell you, government registers will always be the anchor of trust for proof-of-identity, they've been doing it for centuries and have become good at it and have earned the goodwill.

Re: Why are anime catgirls blocking my access to the Linux kernel?

#300

Post by herf »

We deployed hashcash for a while back in 2004 to implement Picasa's email relay - at the time it was a pretty good solution because all our clients were kind of similar in capability. Now I think the fastest/slowest device is a broader range (just like Tavis says), so it is harder to tune the difficulty for that.
Post reply on HN