371–380 of 702 posts
Allow git clone for free/unrestricted still.
Require the user to sign in to view html views. Sign in require a valid email or phone where a validation link is sent.
Or: Users signed in won’t see the Anubis. Users not signed in can still see the html views but have to use a very high work level?
Or: Limit unauthenticated requests from an IP to 5/minute. Authenticated requests can do a lot more before hitting the limit.
Earlier quoted context omitted.
Haha here's a few snippets (will auto expire in a week, future readers): https://ibb.co/mCgXPZPg https://ibb.co/qFxVRQGW https://ibb.co/9m6xV1xg I only just added the token wasting measure so I'm really curious to see what happens if another bot loads over 400 pages.
(FYI, these are comically distorted on mobile.)
No hot toaster pics though :( on second thought perhaps better in our imaginations
How about Allow git clone for free/unrestricted still. Require the user to sign in to view html views. Sign in require a valid email or phone where a validation link is sent. Or: Users signed in won’t see the Anubis. Users not signed in can still see the html views but have to use a very high work level? Or: Limit unauthenticated requests from an IP to 5/minute. Authenticated requests can do a lot more before hitting…
Want to render than seven-year old commit via HTML? Sign in. (But I have no idea.)
Side note: why are shallow clones evil? I always thought they were cheaper, but I guess that’s really just for my disk space. (since the server has to compute what blobs to give you instead of just “everything”?)
Normal clones can reuse delta-compressed data the server stored on disk. Shallow clones impose a negative constraint: do not transfer data outside the requested commit depth. Pre-computed delta chains than contain unrequested data become unusable and the server must do delta compression on the fly to satsify the shallow clone.
[0]: https://github.com/Homebrew/brew/pull/9383
[1]: https://github.com/CocoaPods/CocoaPods/issues/4989#issuecomm...
This is not a hard problem. It just requires that someone build a certification wall. The IETF should have done this long ago, right?
> because apparently what we have to offer is worth spending a ton of cycles to calculate the Anubis challenge. This statement holds the core misapprehension behind Anubis. It’s not a ton of cycles. There is no difficulty setting that would be inconvenient for bots but usable for humans on mobile devices. I noticed the other day that lists.ffmpeg.org had moved to Anubis difficulty level 6, which takes ~180sec for my…
Isn't the issue here the three order of magnitude difference between the (presumably optimized) JS implementation and the optimized C kernel on your browser? If the two stay within an order of magnitude of each other Anubis is a perfectly viable and scalable solution. Bots want to open many orders of magnitude more pages than a normal user, so the resource investment for each single page matters a lot more to them Fo…
The GPU cost to pretrain on that page once will dwarf by 2-3 OOMs the CPU cost to compute Anubis, scrape and post process it. And you’re not going to just train on it once!
At best you’re creating a speed bump for wannabe players scraping with no real plan. The folks training models people use just do not care.
Earlier quoted context omitted.
Which part of computer science is about getting DDoSed by billionaires who are above the law?
Where on Earth do people get the belief that: - It's the SOTA companies doing it? - Scrapers are doing it for training data ? Those are two assumptions I see in posts and threads around Anubis, that are taken at faith, and never once substantiated.
There are more than just the American top dogs (OAI, Anthropic, SpaceX, Facebook)... especially the Chinese government with all its infinite cash resources and next to zero ethical constraints.
I don't trust the US top dogs at all, but I think the fear of discovery alone would lead them to not use "residential proxy" services. Non-US/EU entities however... who cares?
Earlier quoted context omitted.
Which part of computer science is about getting DDoSed by billionaires who are above the law?
Where on Earth do people get the belief that: - It's the SOTA companies doing it? - Scrapers are doing it for training data ? Those are two assumptions I see in posts and threads around Anubis, that are taken at faith, and never once substantiated.
[0] https://www.ft.com/content/07611b74-3d69-4579-9089-f2fc2af61...
from what I see there are 2 solutions: 1) ban TV-proxy-as-a-service - straight up go to every representative there is and start pushing and lobbying and everything to stop spammers from distributing over non-computer devices, especially legally 2) make old commits more expensive to access than new ones. Legit users are not going to access those much, so they can pay the time. I assume diverse (unpredictable?) difficu…
