Live data from Hacker News

Why are anime catgirls blocking my access to the Linux kernel?

lock.cmpxchg8b.com

531–540 of 968 posts

Re: Why are anime catgirls blocking my access to the Linux kernel?

#531
If people are truly concerned about the crawlers hammering their 128mb raspberry pi website then a better solution would be to provide an alternative way for scrapers to access the data (e.g., voluntarily contribute a copy of their public site to something like common crawl).

If Anubis blocked crawler requests but helpfully redirected to a giant tar ball of every site using their service (with deltas or something to reduce bandwidth) I bet nobody would bother actually spending the time to automate cracking it since it’s basically negative value. You could even make it a torrent so most of the be costs are paid by random large labs/universities.

I think the real reason most are so obsessed with blocking crawlers is they want “their cut”… an imagined huge check from OpenAI for their fan fiction/technical reports/whatever.

Re: Why are anime catgirls blocking my access to the Linux kernel?

#532
post #315

Earlier quoted context omitted.

It sounds like you're saying that it's not the proof-of-work that's stopping AI scrapers, but the fact that Anubis imposes an unusual flow to load the site. If that's true Anubis should just remove the proof-of-work part, so legitimate human visitors don't have to stare at a loading screen for several seconds while their device wastes electricity.

> If that's true Anubis should just remove the proof-of-work part This is my very strong belief. To make it even clearer how absurd the present situation is, every single one of the proof-of-work systems I’ve looked at has been using SHA-256, which is basically the worst choice possible. Proof-of-work is bad rate limiting which depends on a level playing field between real users and attackers. This is already a doome…

The proof of work isn't really the crux. They've been pretty clear about this from the beginning.

I'll just quote from their blog post from January.

https://xeiaso.net/blog/2025/anubis/

Anubis also relies on modern web browser features:

- ES6 modules to load the client-side code and the proof-of-work challenge code.

- Web Workers to run the proof-of-work challenge in a separate thread to avoid blocking the UI thread.

- Fetch API to communicate with the Anubis server.

- Web Cryptography API to generate the proof-of-work challenge.

This ensures that browsers are decently modern in order to combat most known scrapers. It's not perfect, but it's a good start.

This will also lock out users who have JavaScript disabled, prevent your server from being indexed in search engines, require users to have HTTP cookies enabled, and require users to spend time solving the proof-of-work challenge.

This does mean that users using text-only browsers or older machines where they are unable to update their browser will be locked out of services protected by Anubis. This is a tradeoff that I am not happy about, but it is the world we live in now.

Re: Why are anime catgirls blocking my access to the Linux kernel?

#534

TFA — and most comments here — seem to completely miss what I thought was the main point of Anubis: it counters the crawler's "identity scattering"/sybil'ing/parallel crawling. Any access will fall into either of the following categories: - client with JS and cookies. In this case the server now has an identity to apply rate limiting to, from the cookie. Humans should never hit it, but crawlers will be slowed down im…

Yes, I think you're right. The commentary about its (presumed, imagined) effectiveness is very much making the assumption that it's designed to be an impenetrable wall[0] -- i.e. prevent bots from accessing the content entirely.

I think TFA is generally quite good and has something of a good point about the economics of the situation, but finding the math shake out that way should, perhaps, lead one to question their starting point / assumptions[1].

In other words, who said the websites in question wanted to entirely prevent crawlers from accessing them? The answer is: no one. Web crawlers are and have been fundamental to accessing the web for decades. So why are we talking about trying to do that?

[0] Mentioning 'impenetrable wall' is probably setting off alarm bells, because of course that would be a bad design.

[1] (Edited to add:) I should say 'to question their assumptions more' -- like I said, the article is quite good and it does present this as confusing, at least.

Re: Why are anime catgirls blocking my access to the Linux kernel?

#535

This is a usually technical crowd, so I can't help but wonder if many people genuinely don't get it, or if they are just feigning a lack of understanding to be dismissive of Anubis. Sure, the people who make the AI scraper bots are going to figure out how to actually do the work. The point is that they hadn't, and this worked for quite a while. As the botmakers circumvent, new methods of proof-of-notbot will be made…

It's been going on for decades now too. It's a cat and mouse game that will be with us for as long as people try to exploit online resources with bots. Which will be until the internet is divided into nation nets, suffocated by commercial interests, and we all decide to go play outside instead.

Re: Why are anime catgirls blocking my access to the Linux kernel?

#536

Every time I see one of these I think it's a malicious redirect to some pervert-dwelling imageboard. On that note, is kernel.org really using this for free and not the paid version without the anime? Linux Foundation really that desperate for cash after they gas up all the BMW's?

For me it's the flipside: It makes me think "Ahh, my people!"

Re: Why are anime catgirls blocking my access to the Linux kernel?

#537
post #471
post #379

Earlier quoted context omitted.

I don't trip over CloudFlare except when in a weird VPN, and then it always gets out of my way after the challenge. Anubis screws with me a lot, and often doesn't work.

The annoying thing about cloudflare is that most of the time once you’re blocked: you’re blocked. There’s literally no way for you to bypass the block if you’re affected. Its incredibly scary, I once had a bad useragent (without knowing it) and half the internet went offline, I couldn’t even access documentation or my email providers site, and there was no contact information or debugging information to help me resol…

Now imagine your government provided internet agent gets blacklisted because your linked social media post was interpreted by an LLM to be anti-establishment, and we are painting a picture of our current trajectory.

Re: Why are anime catgirls blocking my access to the Linux kernel?

#538

Earlier quoted context omitted.

I am not again AI art completely since I think of it as an editing instead of art itself. My thoughts on AI art are nuanced and worth discussing some other day, lets talk about the author of anubis/story of anubis So, I hope you know the entire story behind Anubis, firstly they were hosting their own git server (I think?) and amazon's ai related department was basically ddosing their server in some sense by trying to…

> My thoughts on AI art are nuanced and worth discussing some other day If the argument is all this AI support for an anti-AI crawler, I don't think it's a good argument to ignore it. and as long as the 3C's aren't followed, there's no subtly here. You don't get to use "edited" art in a commercial product except in fair use cases. >Stop whining about free projects/labour man. Yes, I'm sure this sort of argument is co…

Okay I understand your point and I agree with you. Maybe, my wording was a little harsh and I understand your point.

lets see, the thing that I am trying to say is that, in my opinion even with AI art, that is not the point.

The point remains whether anubis is actually useful at what it does and is it worth it.

Also I understand free ruined the internet very quickly, but this is free and open source. Open source has made the internet better a 100 times imo.

I genuinely don't think that the author had any wrong intentions with using AI art.

and since we have opened the box, might as well, talk about the AI thoughts.

Basically, I think that there are two schools of thoughts towards AI/Ai art. Its efficacy at doing what it is advertised to say, its effect on markets, its effect on climate.

The efficacy at doing what it can do : I absolutely agree that there is a lot of hype around it but AI art even open source models have become good enough for some very "basic tasks" imo, Like I had a discord server and I actually used AI art as the logo because people wanted a logo and I am not an artist and I can't comission anyone just for something so small. I immediately told everyone this and nobody seemed to care and afterwards and after some more people joined,literally noone asked if it was ai generated or not. I was expecting a single person to notice but nope.

Another point is about climate, I think that AI itself is pretty efficient, there were posts about how it compares to a lightbulb but the problem is how massively it is used at scale and how much the demand can change and how it can affect the power demand of that datacentre and if power demand changes, then that requires the power generators to tweak their speed and tldr, in that process, it becomes really inefficient but still I am actually surprised that we are calling out on AI when there is bitcoin which literally all it does with its pow is waste energy and there are other cryptocoins which can literally have close to 0 fees and instant but nope the crypto market is in a bubble. Stablecoins are the only good thing to come out of it.

Now that being said, its effect on job markets. I am not an artist but I imagine it is frustating to see AI replicate art. but the point is, nobody wants AI art!! There is no economic incentive to make AI art unless genuinely can't comission someone (which is what I did in my server) or just are using it as a backup art untill you can comission one (which is what anubis did) but in my opinion, if someone can't already pay like I couldn't in my discord server, I would have just build one myself or used some CC:0 art with full credits .

Which is why I don't consider AI art "art". Its a backup, more like editing. The people will throw wrenches at you if you get caught using it. And that's a good thing. but we do need to realize maybe using ai art as a backup and people should definitely push against ai art so that if possible, real artists gets the job done.

I don't consider ai art to be anymore of a gimmick but even then, I mean I can understand if people are using ai art as backups untill they can comission real art or build one themselves.

Re: Why are anime catgirls blocking my access to the Linux kernel?

#539
post #118

Earlier quoted context omitted.

I am not again AI art completely since I think of it as an editing instead of art itself. My thoughts on AI art are nuanced and worth discussing some other day, lets talk about the author of anubis/story of anubis So, I hope you know the entire story behind Anubis, firstly they were hosting their own git server (I think?) and amazon's ai related department was basically ddosing their server in some sense by trying to…

> Stop whining about free projects/labour man. The same people comment oh well these AI scrapers are scraping so many websites and taking livelihood of website makers and now you have someone who just gave it to ya for free and you are nitpicking the wrong things. That isn't the issue. The issue is that this tool is not fit for purpose and is inappropriate to be used by the projects that have adopted it. The proof of…

Hm yea this is a fair critisicm actually as I also said in some other comment just now that we need to discuss more about if anubis is actually being useful or not at what its saying.

You raise a good point man, what do you suggest should be done instead of what anubis is doing right now for the same outcome(getting not effectively ddosed by AI scrapers) ?

Re: Why are anime catgirls blocking my access to the Linux kernel?

#540
post #471

Earlier quoted context omitted.

The annoying thing about cloudflare is that most of the time once you’re blocked: you’re blocked. There’s literally no way for you to bypass the block if you’re affected. Its incredibly scary, I once had a bad useragent (without knowing it) and half the internet went offline, I couldn’t even access documentation or my email providers site, and there was no contact information or debugging information to help me resol…

A "digital no-fly-list" is hella cyberpunk, though.

The question might become, what side of the black wall are you going to be on?

Seriously though I do think we are going to see increasing interest in alternative nets, especially as governments tighten their control over the internet or even break away into isolated nation nets.

Post reply on HN