Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why are anime catgirls blocking my access to the Linux kernel?

lock.cmpxchg8b.com

Post reply on HN
Print view View original

551–560 of 968 posts

Re: Why are anime catgirls blocking my access to the Linux kernel?

#551

Post by _flux »

throwawayffffas wrote: post #199
hiccuphippo wrote: post #183

Earlier quoted context omitted.

Wasn't sha256 designed to be very fast to generate? They should be using bcrypt or something similar.

Unless they require a new token for each new request or every x minutes or something it won't matter. And as the poster mentioned if you are running an AI model you probably have GPUs to spare. Unlike the dev working from a 5 year old Thinkpad or their phone.

Apparently bcrypt has design that makes it difficult to accelerate effectively on a GPU.

Indeed a new token should be requested per request; the tokens could also be pre-calculated, so that while the user is browsing a page, the browser could calculate tickets suitable to access the next likely browsing targets (e.g. the "next" button).

The biggest downside I see is that mobile devices would likely suffer. Possible the difficulty of the challange is/should be varied by other metrics, such as the number of requests arriving per time unit from a C-class network etc.

Re: Why are anime catgirls blocking my access to the Linux kernel?

#552

Post by msgodel »

zaptrem wrote: post #531

If people are truly concerned about the crawlers hammering their 128mb raspberry pi website then a better solution would be to provide an alternative way for scrapers to access the data (e.g., voluntarily contribute a copy of their public site to something like common crawl). If Anubis blocked crawler requests but helpfully redirected to a giant tar ball of every site using their service (with deltas or something to…

I'm generally very pro-robot (every web UA is a robot really IMO) but these scrapers are exceptionally poorly written and abusive.

Plenty of organizations managed to crawl the web for decades without knocking things over. There's no reason to behave this way.

It's not clear to me why they've continued to run them like this. It seems so childish and ignorant.

Re: Why are anime catgirls blocking my access to the Linux kernel?

#553

Post by zaptrem »

lmm wrote: post #547
zaptrem wrote: post #531

If people are truly concerned about the crawlers hammering their 128mb raspberry pi website then a better solution would be to provide an alternative way for scrapers to access the data (e.g., voluntarily contribute a copy of their public site to something like common crawl). If Anubis blocked crawler requests but helpfully redirected to a giant tar ball of every site using their service (with deltas or something to…

The kind of crawlers/scrapers who DDoS a site like this aren't going to bother checking common crawl or tarballs. You vastly overestimate the intelligence and prosociality of what bursty crawler requests tend to look like. (Anyone who is smart or prosocial will set up their crawler to not overwhelm a site with requests in the first place - yet any site with any kind of popularity gets flooded with these requests soon…

If they don’t have the intelligence to go after the more efficient data collection method then they likely won’t have the intelligence or willpower to work around the second part I mentioned (keeping something like Anubis). The only problem is when you put Anubis in the way of determined, intelligent crawlers without giving them a choice that doesn’t involve breaking Anubis.

Re: Why are anime catgirls blocking my access to the Linux kernel?

#554

Post by shiomiru »

zaptrem wrote: post #531

If people are truly concerned about the crawlers hammering their 128mb raspberry pi website then a better solution would be to provide an alternative way for scrapers to access the data (e.g., voluntarily contribute a copy of their public site to something like common crawl). If Anubis blocked crawler requests but helpfully redirected to a giant tar ball of every site using their service (with deltas or something to…

> I think the real reason most are so obsessed with blocking crawlers is they want “their cut”…

I find that an unfair view of the situation. Sure, there are examples such as StackOverflow (which is ridiculous enough as they didn't make the content) but the typical use case I've seen on the small scale is "I want to self-host my git repos because M$ has ruined GitHub, but some VC-funded assholes are drowning the server in requests".

They could just clone the git repo, and then pull every n hours, but it requires specialized code so they won't. Why would they? There's no money in maintaining that. And that's true for any positive measure you may imagine until these companies are fined for destroying the commons.

Re: Why are anime catgirls blocking my access to the Linux kernel?

#555

Post by zaptrem »

msgodel wrote: post #552
zaptrem wrote: post #531

If people are truly concerned about the crawlers hammering their 128mb raspberry pi website then a better solution would be to provide an alternative way for scrapers to access the data (e.g., voluntarily contribute a copy of their public site to something like common crawl). If Anubis blocked crawler requests but helpfully redirected to a giant tar ball of every site using their service (with deltas or something to…

I'm generally very pro-robot (every web UA is a robot really IMO) but these scrapers are exceptionally poorly written and abusive. Plenty of organizations managed to crawl the web for decades without knocking things over. There's no reason to behave this way. It's not clear to me why they've continued to run them like this. It seems so childish and ignorant.

The bad scrapers would get blocked by the wall I mentioned. The ones intelligent enough to break the wall would simply take the easier way out and download the alternative data source.

Re: Why are anime catgirls blocking my access to the Linux kernel?

#556

Post by Symbiote »

NobodyNada wrote: post #286
Symbiote wrote: post #249

Earlier quoted context omitted.

Earlier today I found we'd served over a million requests to over 500,000 different IPs. All had the same user agent (current Safari), they seem to be from hacked computers as the ISPs are all over the world. The structure of the requests almost certainly means we've been specifically targeted. But it's also a valid query, reasonably for normal users to make. From this article, it looks like Proof of Work isn't going…

The math in the article assumes scrapers only need one Anubis token per site, whereas a scraper using 500,000 IPs would require 500,000 tokens. Scaling up the math in the article, which states it would take 6 CPU-minutes to generate enough tokens to scrape 11,508 Anubis-using websites, we're now looking at 4.3 CPU-hours to obtain enough tokens to scrape your website (and 50,000 CPU-hours to scrape the Internet). This…

If the scraper scrapes from a small number of IPs they're easy to block or rate-limit. Rate-limits against this behaviour are fairly easy to implement, as are limits against non-human user agents, hence the botnet with browser user agents.

The Duke University Library analysis posted elsewhere in the discussion is promising.

I'm certain the botnets are using hacked/malwared computers, as the huge majority of requests come from ISPs and small hosting providers. It's probably more common for this to be malware, e.g. a program that streams pirate TV, or a 'free' VPN app, which joins the user's device to a botnet.

Re: Why are anime catgirls blocking my access to the Linux kernel?

#557

Post by guappa »

zaptrem wrote: post #549
elsjaako wrote: post #542

Earlier quoted context omitted.

There's a lot of people that really don't like AI, and simply don't want their data used for it.

While that’s a reasonable opinion to have, it’s a fight they can’t really win. It’s like putting up a poster in a public square then running up to random people and shouting “no, this poster isn’t for you because I don’t like you, no looking!” Except the person they’re blocking is an unstoppable mega corporation that’s not even morally in the wrong imo (except for when they overburden people’s sites, that’s bad ofc)

The looking is fine, the photographing and selling the photo less so… and fyi in denmark monuments have copyright so if you photograph and sell the photos you owe fees :)

Re: Why are anime catgirls blocking my access to the Linux kernel?

#558

Post by guappa »

ok123456 wrote: post #282
bawolff wrote: post #247

Earlier quoted context omitted.

Well the problem is that computers got good at basically everything. Early 2000s captchas really were like that.

The original reCAPTCHA was doing distributed book OCR. It was sold as an altruistic project to help transcribe old books.

And now they're using us to train car driving AI :(

Re: Why are anime catgirls blocking my access to the Linux kernel?

#559

Post by ChrisRR »

ksymph wrote: post #15

This is neither here nor there but the character isn't a cat. It's in the name, Anubis, who is an Egyptian deity typically depicted as a jackal or generic canine, and the gatekeeper of the afterlife who weighs the souls of the dead (hence the tagline). So more of a dog-girl, or jackal-girl if you want to be technical.

I'm assuming the aversion is more about why young anime girls are popping up, not about what animal it is

Re: Why are anime catgirls blocking my access to the Linux kernel?

#560

Post by qwery »

esperent wrote: post #469
ksymph wrote: post #15

This is neither here nor there but the character isn't a cat. It's in the name, Anubis, who is an Egyptian deity typically depicted as a jackal or generic canine, and the gatekeeper of the afterlife who weighs the souls of the dead (hence the tagline). So more of a dog-girl, or jackal-girl if you want to be technical.

Every representation I've ever seen of Anubis - including remarkably well preserved statues from antiquity - are either a male human body with a canine head, or fully canine. This anime girl is not Anubis. It's a modern cartoon characters that simply borrows the name because it sounds cool, without caring anything about the history or meaning behind it. Anime culture does this all the time, drawing on inspiration fro…

I think you're taking it a bit too seriously. In turn, I am, of course, also taking it too seriously.

> I do have an issue with claiming that the newly inspired creation is equivalent in any way to the original source

Nobody is claiming that the drawing is Anubis or even a depiction of Anubis, like the statues etc. you are interested in. It's a mascot. "Mascot design by CELPHASE" -- it says, in the screenshot.

Generally speaking -- I can't say that this is what happened with this project -- you would commission someone to draw or otherwise create a mascot character for something after the primary ideation phase of the something. This Anubis-inspired mascot is, presumably, Anubis-inspired because the project is called Anubis, which is a name with fairly obvious connections to and an understanding of "the original source".

> Anime culture does this all the time, ...

I don't know what bone you're picking here. This seems like a weird thing to say. I mean, what anime culture? It's a drawing on a website. Yes, I can see the manga/anime influence -- it's a very popular, mainstream artform around the world.

Post reply on HN