Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

It took a year to ship WebAssembly in Anubis

anubis.techaro.lol

Post reply on HN
Print view View original

181–190 of 223 posts

Re: It took a year to ship WebAssembly in Anubis

#181

Post by account42 »

gruez wrote: post #41
solid_fuel wrote: post #34

Earlier quoted context omitted.

The assumption is that requiring 1 second of compute before allowing a page load will have more impact on the people requesting 1000 pages/sec than it will on consumers requesting 1 page every minute. Personally I don’t understand where this confusion comes from - it’s a simple economic tactic. Yes the large scale scrapers won’t run out of compute but putting anubis in front of a website demonstrably reduces the amou…

>The assumption is that requiring 1 second of compute before allowing a page load will have more impact on the people requesting 1000 pages/sec than it will on consumers requesting 1 page every minute. Which doesn't hold. A 1s delay, despite being inperceptible is still a cost to human users. After all, 3600 x 1s delay adds up to an hour. Worse yet, the typical human visitor will always have to solve a challenge, bec…

Yes, at the end of the day this is DRM and as always it affects legitimate users more than it does those it is meant to block.

Re: It took a year to ship WebAssembly in Anubis

#182

Post by thunderfork »

YoshiRulz wrote: post #107
xena wrote: post #20

Earlier quoted context omitted.

I want to do this eventually, but it's hard to split things into the micro-tasks that would be required to make this work on Anubis. One of the ideas I'm throwing around is a world where Anubis helps fuzz old games to find timesaves in tool-assisted speedruns. It's harder than you think.

The tools that exist for fuzzing speedruns (TASers call it "botting") are fairly primitive, excluding a few game-specific ones. Because of that, there really aren't that many TASes where a distributed randomised search could make improvements which don't get immediately overshadowed within a day of human attention. Improving botting tools and increasing their adoption would be more effective IMO. (Incidentally, TASVi…

BOINC isn't really built for "small tasks", afaik. Once you get small enough, the work the server is doing to manage the tasks gets to be around the same cost as the server just doing the tasks itself.

Re: It took a year to ship WebAssembly in Anubis

#183

Post by account42 »

JodieBenitez wrote: post #123
andai wrote: post #115

Tangential but all the pages I see protected by Anubis are just static HTML. But bot traffic is only harmful on dynamic pages right?

I'm not a fan of Anubis for various reasons but the idea that bot traffic in only harmful with dynamic pages must die. CPU (yes, even to serve static pages) is not free, bandwidth is definitely not free. There's an idea that serving a static page to a bot has a marginal cost tending to zero, but it's never really zero and serving them by millions definitely has a cost. Also, while some pages may look like static HTML…

> Also, while some pages may look like static HTML pages, they may be generated on the fly by an expensive and/or slow backend

Yes and that should be fixed before you subject real users to resource-wasting scripts.

Re: It took a year to ship WebAssembly in Anubis

#184

Post by account42 »

jeroenhd wrote: post #127
andai wrote: post #115

Tangential but all the pages I see protected by Anubis are just static HTML. But bot traffic is only harmful on dynamic pages right?

Shitty scrapers hitting images over and over again is also a problem. Not just /pic/1.webp - /pic/2.webp, but in practice I've seen three AI scrapers that would request things like /pic/1.webp?version=webp&foo=bar and other random query parameters that none of my domains even use. It's just stuck brute-forcing files or something, sucking up loads of bandwidth downloading the same image for no reason. It also depends…

> Shitty scrapers hitting images over and over again is also a problem. Not just /pic/1.webp - /pic/2.webp, but in practice I've seen three AI scrapers that would request things like /pic/1.webp?version=webp&foo=bar and other random query parameters that none of my domains even use

That's a trivially solved problem - have the web server refuse (or redirect if you're feeling nice) requests with unexpected query parameters. No reason to make things worse for real users before optimizing your site.

And even if there are actual parametric resources you want to serve, apply limits only to them and let users visit the rest without unnecessary bot checks.

Re: It took a year to ship WebAssembly in Anubis

#186

Post by account42 »

andai wrote: post #176
Nnnes wrote: post #149

Earlier quoted context omitted.

Others have answered your question. A lot of the sites listed here https://anubis.techaro.lol/docs/user/known-instances/ have dynamic content; Git web interfaces in particular (Codeberg, the Linux kernel, FFMPEG, and more are on the list) are vulnerable to poorly or maliciously configured scrapers.

Yeah it makes sense for dynamic content. But so far I have only seen it on blogs, which could have just been a html file.

Even for git hosts, there is no reason to add bot checks to e.g. the repository root or other common URLs that random real users land on.

Re: It took a year to ship WebAssembly in Anubis

#187

Post by account42 »

trickypr wrote: post #121
andai wrote: post #115

Tangential but all the pages I see protected by Anubis are just static HTML. But bot traffic is only harmful on dynamic pages right?

The anubis docs website is used as an example of how anubis works. Also, if you are not using a big CDN (e.g. Cloudflare), you still need to pay for the traffic (even if you don't pay as much for the CPU). This is doubly true for smaller CDNs like Bunny.

You also need to pay for the traffic it takes to server you bot check scripts. And outside of big cloud companies, traffic is actually dirt cheap.

Re: It took a year to ship WebAssembly in Anubis

#188

Post by account42 »

chews wrote: post #2

It's wild to me that PoW systems are how we sort the bot problem... the bummer is that all this "work" is just wasted cycles, at least in crypto there is a token you can sell.

It's wild how many people happily jump on this DRM train now that they are hoping to gain from it.
Post reply on HN