Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Creepy Crawlies

people.kernel.org

Post reply on HN
Print view View original

571–580 of 694 posts

Re: Creepy Crawlies

#571

Post by voidUpdate »

inigyou wrote: post #255
ynniv wrote: post #74

this is an increasingly common situation. it goes something like: - i have a free, niche resource - it becomes too popular - i make it more efficient - now it's really popular, and people are "abusing" it - let's make them proof-of-work - ... and proof-of-work harder - but now "legitimate" users can't use it - ??? the core problem is that the average person uses a mobile device where work is expensive, and the "attac…

What's actually stopping them isn't the PoW, it's the customisation effort. If one site has Anubis nothing scrapes it. If many sites have Anubis they write counter scrapers. Today if you make a slight change to the Anubis algorithm on your site, they'll burn CPU endlessly computing hashes with the original algorithm and submitting wrong ones. The author of Anubis hates this fact and will ban you if you mention it, so…

*They insist

Re: Creepy Crawlies

#572

Post by znnajdla »

inigyou wrote: post #296
jwilk wrote: post #222

Earlier quoted context omitted.

From the HN guidelines https://news.ycombinator.com/newsguidelines.html >: > Please don't comment on whether someone read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that".

What if it's really obvious they didn't read the article?

I did read the article. It just didn't occur to me that their combinatorial explosion of diffs was scrape-able. To be honest that sounds like an scrapers tarpit / honeypot now, because there is no value in scraping trillions of diffs. Sounds like the issue could be fixed by putting the diffs in a frontend app, not scrapable by URL, only by clicking around the app.

Re: Creepy Crawlies

#574

Post by GoblinSlayer »

supriyo-biswas wrote: post #558
arc-in-space wrote: post #546

Earlier quoted context omitted.

Disappointed at the lack of replies here, I've had a similar thought before and I'd be curious to hear about the technical viability of this

This was done in the past[1] and was widely regarded as a "cryptojacking" scheme serving "malicious" content to users. [1] https://krebsonsecurity.com/2018/03/who-and-what-is-coinhive...

If it was for profit. Here it's in response to abuse by AI companies.

Re: Creepy Crawlies

#575

Post by trashb »

Yea these AI bots are getting out of hand.

The article mentions that some requested pages are less likely to be legit traffic (old commits) and more likely to have bot activity. Perhaps they could increase difficulty on those pages for the proof of work. Keeping the "current" at a lower difficulty allows most normal users to use the pages as normal, while penalizing the bots.

One other way I have been thinking of is just delay the delivery of the pages, either limit bandwidth or just wait for a bit until you deliver the page. For one user a (lets say max)3s delay on some pages is not a huge deal, however at scale that adds up and means the client can't gather other pages in the meantime.

Another option is to lock the out of date html renderings behind a login page while keeping the git openly available.

Re: Creepy Crawlies

#576

Post by ksimukka »

Remember SETI?

Maybe we should serve the bots a problem worth solving and benefit both parties. They spend some energy/tokens on a problem and we pay them with content.

If only I had a bot problem, this would be interesting to explore.

Re: Creepy Crawlies

#577

Post by crote »

tptacek wrote: post #145
harshreality wrote: post #142

Earlier quoted context omitted.

If you want to argue that there's excessive enthusiasm about anubis, which is neither a permanent solution nor a solution against all bots run for all purposes (some bot authors are cleverer than others, and you don't have to block everything, just enough to reduce resource use to tolerable levels)... fine. However, the refrain from naysayers isn't that there's too much enthusiasm. It's that anubis doesn't make sense…

I'm not moralizing. The sites running Anubis are owned by people who like Anubis and are fully within their rights to run Anubis. I don't care how enthusiastic they are about it. It's their call. I'm just saying, it's bad computer science.

The real world doesn't care about the soundness of your computer science. All that matters is whether it works.

It's like saying we should just completely give up on all NP-hard problems because we'll never discover a way to find the optimal solution in all cases. Meanwhile in the real world, people are more than happy with, say, a route planner which virtually all of the time gives them a solution which is within a rounding error of optimal. Nobody is going to care that it is technically bad science.

Same with Anubis: Sure, it's a dead end in an ongoing arms race. It won't last forever. But right now it works pretty well, so let's hope it lasts until someone can figure out a better approach.

Re: Creepy Crawlies

#579

Post by sgsjchs »

inigyou wrote: post #513
sgsjchs wrote: post #402

Earlier quoted context omitted.

it'll load the cookie, make one request, move to a different ip, load the cookie, make one request, move to a different ip, ...

Why are you serving new cookies to someone who already has a cookie?

It discards the cookie after one request and appears as a new user.

Re: Creepy Crawlies

#580

Post by infp_arborist »

DenisM wrote: post #289
semiquaver wrote: post #89

> because apparently what we have to offer is worth spending a ton of cycles to calculate the Anubis challenge. This statement holds the core misapprehension behind Anubis. It’s not a ton of cycles. There is no difficulty setting that would be inconvenient for bots but usable for humans on mobile devices. I noticed the other day that lists.ffmpeg.org had moved to Anubis difficulty level 6, which takes ~180sec for my…

Memory-hard hash functions maybe? Like, you must dedicate 4gb of ram to compute the function. Not a problem for a one-off, but is a problem when reading lots of pages at once. Or… the site will serve a random seed and the device must compute 4gb of pseudo-random data, then supply a value at a random server-demanded offset.

Yes, that seems to work for Altcha: https://altcha.org/docs/integration/proof-of-work-captcha/

Edit: this is already in discussion for Anubis: https://github.com/TecharoHQ/anubis/issues/94

Post reply on HN