Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why are anime catgirls blocking my access to the Linux kernel?

lock.cmpxchg8b.com

Post reply on HN
Print view View original

441–450 of 968 posts

Re: Why are anime catgirls blocking my access to the Linux kernel?

#441

Post by debugnik »

NoGravitas wrote: post #243
debugnik wrote: post #121

Earlier quoted context omitted.

That's not bypassing it, that's them finally engaging with the PoW challenge as intended, making crawling slower and more expensive, instead failing to crawl at all, which is more of a plus. This however forces servers to increase the challenge difficulty, which increases the waiting time for the first-time access.

The point is that it will always be cheaper for bot farms to pass the challenge than for regular users.

Why does that matter? The challenge needs to stay expensive enough to slow down bots, but legitimate users won't be solving anywhere near the same amount of challenges and the alternative is the site getting crawled to death, so they can wait once in a while.

Re: Why are anime catgirls blocking my access to the Linux kernel?

#443

Post by Almondsetat »

sugarpimpdorsey wrote: post #302
tptacek wrote: post #279

Earlier quoted context omitted.

Respectfully, I think it's you missing the point here. None of this is to say you shouldn't use Anubis, but Tavis Ormandy is offering a computer science critique of how it purports to function. You don't have to care about computer science in this instance! But you can't dismiss it because it's computer science. Consider: An adaptive password hash like bcrypt or Argon2 uses a work function to apply asymmetric costs t…

A lot of these passive types of anti-abuse systems rely on the rather bold assumption that making a bot perform a computation is expensive, but isn't for me as an ordinary user. According to whom or what data exactly? AI operators are clearly well-funded operations and the amount of electricity and CPU power is negligible. Software like Anubis and nearly all its identical predecessors grant you access after a single…

Scrapers are orders of magnitude faster than humans at browsing websites. If the challenge takes 1 second but a human stays on the page for 3 minutes, then it's negligible. But if the challenge takes 1 second and the scraper does ita job in 5 seconds, you already have a 20% slowdown

Re: Why are anime catgirls blocking my access to the Linux kernel?

#444

Post by jcelerier »

technion wrote: post #362
johnklos wrote: post #244

This is a usually technical crowd, so I can't help but wonder if many people genuinely don't get it, or if they are just feigning a lack of understanding to be dismissive of Anubis. Sure, the people who make the AI scraper bots are going to figure out how to actually do the work. The point is that they hadn't, and this worked for quite a while. As the botmakers circumvent, new methods of proof-of-notbot will be made…

It really should be recognised just how many people are watching Cloudflare interstitials on nearly every site these days (and I totally get why this happens) yet making a huge amount of noise about Anubis on a very small amount of sites.

Both are equally terrible - one doesn't require explanations to my boss though

Re: Why are anime catgirls blocking my access to the Linux kernel?

#445

Post by rnhmjoj »

nemothekid wrote: post #115
rnhmjoj wrote: post #29

Earlier quoted context omitted.

Which companies are we talking about here? In my case the traffic was similar to what was reported here[1]: these are crawlers from Google, OpenAI, Amazon, etc. they are really idiotic in behaviour, but at least report themselves correctly. [1]: https://pod.geraspora.de/posts/17342163

OpenAI/Anthropic/Perplexity aren't the bad actors here. If they are, they are relatively simply to block - why would you implement an Anubis PoW MITM Proxy, when you could just simply block on UA? I get the sense many of the bad actors are simply poor copycats that are poorly building LLMs and are scraping the entire web without a care in the world

> why would you implement an Anubis PoW MITM Proxy, when you could just simply block on UA?

That's in fact what I was asking: I've only seen traffic from these kind of companies and I've easily blocked them without an annoying PoW scheme.

I have yet to see any of these bad actors and I'm interested in knowing who they actually are.

Re: Why are anime catgirls blocking my access to the Linux kernel?

#446

Post by vova_hn »

ChocolateGod wrote: post #430

I have a S24 (flagship of 2024) and Anubis often takes 10-20 seconds to complete, that time is going to add up if more and more sites adopt it, leaning to a worse browsing experience and wasted battery life. Meanwhile AI farms will just run their own nuclear reactors eventually and be unaffected. I really don't understand why someone thought this was a good idea, even if well intentioned.

I have Pixel 7 (released in 2022) and it usually takes less than a second...

Re: Why are anime catgirls blocking my access to the Linux kernel?

#447

Post by [deleted] »

Almondsetat wrote: post #443
sugarpimpdorsey wrote: post #302

Earlier quoted context omitted.

A lot of these passive types of anti-abuse systems rely on the rather bold assumption that making a bot perform a computation is expensive, but isn't for me as an ordinary user. According to whom or what data exactly? AI operators are clearly well-funded operations and the amount of electricity and CPU power is negligible. Software like Anubis and nearly all its identical predecessors grant you access after a single…

Scrapers are orders of magnitude faster than humans at browsing websites. If the challenge takes 1 second but a human stays on the page for 3 minutes, then it's negligible. But if the challenge takes 1 second and the scraper does ita job in 5 seconds, you already have a 20% slowdown

[deleted]

Re: Why are anime catgirls blocking my access to the Linux kernel?

#448

Post by chrismorgan »

agwa wrote: post #315
johnklos wrote: post #244

This is a usually technical crowd, so I can't help but wonder if many people genuinely don't get it, or if they are just feigning a lack of understanding to be dismissive of Anubis. Sure, the people who make the AI scraper bots are going to figure out how to actually do the work. The point is that they hadn't, and this worked for quite a while. As the botmakers circumvent, new methods of proof-of-notbot will be made…

It sounds like you're saying that it's not the proof-of-work that's stopping AI scrapers, but the fact that Anubis imposes an unusual flow to load the site. If that's true Anubis should just remove the proof-of-work part, so legitimate human visitors don't have to stare at a loading screen for several seconds while their device wastes electricity.

> If that's true Anubis should just remove the proof-of-work part

This is my very strong belief. To make it even clearer how absurd the present situation is, every single one of the proof-of-work systems I’ve looked at has been using SHA-256, which is basically the worst choice possible.

Proof-of-work is bad rate limiting which depends on a level playing field between real users and attackers. This is already a doomed endeavour. Using SHA-256 just makes it more obvious: there’s an asymmetry factor in the order of tens of thousands between common real-user hardware and software, and pretty easy attacker hardware and software. You cannot bridge such a divide. If you allow the attacker to augment it with a Bitcoin mining rig, the efficiency disparity factor can go up to tens of millions.

These proof-of-work systems are only working because attackers haven’t tried yet. And as long as attackers aren’t trying, you can settle for something much simpler and more transparent.

If they were serious about the proof-of-work being the defence, they’d at least have started with something like Argon2d.

Re: Why are anime catgirls blocking my access to the Linux kernel?

#450

Post by debugnik »

heavyset_go wrote: post #381
chmod775 wrote: post #355

Earlier quoted context omitted.

It doesn't need to. Thanks to asymmetric cryptography governments can in theory provide you with a way to prove you are a human (or of a certain age) without: 1. the government knowing who you are authenticating yourself to 2. or the recipient learning anything but the fact that you are a human 3. or the recipient being able to link you to a previous session if you authenticate yourself again later The EU is trying t…

There isn't a technical solution to this: governments and providers not only want proof of identity matching IDs, they want proof of life, too. This will always end with live video of the person requesting to log in to provide proof of life at the very least, and if they're lazy/want more data, they'll tie in their ID verification process to their video pipeline.

You already provided proof of a living legal identity when you got the ID, and it already expires to make you provide proof again every few years.
Post reply on HN