Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The bot situation on the internet is worse than you could imagine

gladeart.com

Post reply on HN
Print view View original

131–140 of 179 posts

Re: The bot situation on the internet is worse than you could imagine

#131

Post by carlosjobim »

nba456_ wrote: post #50
carlosjobim wrote: post #18

Earlier quoted context omitted.

You may be missing that it's easy and free for website owners to fix the problem. But it's hacker news after all. If somebody is bothered by a leaf falling on them on their walk to the corner store, the suggested solution here will be to have a full communist revolution.

It is absolutely not free or easy to stop bots.

Millions upon millions of people use Cloudflare to stop bots for free, and there are other alternatives as well. It's incredibly easy. So no, there's just as much need for government intervention, as there is a need for the government to stop leaves from falling on you when you're walking to the corner store.

Re: The bot situation on the internet is worse than you could imagine

#132

Post by lifeisstillgood »

rekabis wrote: post #115
lifeisstillgood wrote: post #91

Earlier quoted context omitted.

>>> People have a right to complete anonymity Why? (Am not trolling. Genuinely interested) I walk out my front door in the UK and I am not anonymous. Every transaction I make either identifies me through bank, railway or other id, or quite simply by my face standing in front of the coffee seller. My walk down the road is observed by neighbours and postmen. Should my government arrest me without cause or trample on my…

> Every transaction I make either identifies me through bank, railway or other id, or quite simply by my face standing in front of the coffee seller. My walk down the road is observed by neighbours and postmen. Are these the government? Is the bank the government? Is the rail company the government? No? Then you have answered your own question. A silo of identification between you and a service provider that uses the…

The problem here is that pretty much every part of modern life has been government and capitalism riding in the same cart - from cities installing electric power stations 100 years ago, to roads and inventions like the transistor and internet itself was government and private capital working towards common goals.

The issue is we want “good” government and “good” corporate behaviour but not the bad. And knowing the difference especially ahead of time requires engaged citizenry, lots of feedback mechanisms that are not overwritten by corruption and noise in the mechanism (ie primaries materringnmore than elections is a feedback mechanism fail in my book)

Re: The bot situation on the internet is worse than you could imagine

#133

Post by tredre3 »

GeoAtreides wrote: post #103
Aurornis wrote: post #102

Earlier quoted context omitted.

The Anubis difficulty setting is (or was) so high that nobody could visit the site without leaving it open for minutes or hours.

>Anubis was enabled in the tar pit at difficulty 1 (lowest setting) when requests were pouring in 24/7 >difficulty 1 (lowest setting) literally in the comment you're responding to

> After 2 minutes at 150 kHashes on mobile, I finally see the first pixel of the progress bar filling up. Seems like it will take hours or a day to finish. Some estimate would have been nice.

Literally the grandparent of the comment chain you're responding to.

Re: The bot situation on the internet is worse than you could imagine

#134

Post by drum55 »

bawolff wrote: post #128
GeoAtreides wrote: post #85

Earlier quoted context omitted.

>It's less proof of work and just annoying to users, and feel good to whoever added it to their site, this is being disproved in the article posted: >And so Anubis was enabled in the tar pit at difficulty 1 (lowest setting) when requests were pouring in 24/7. Before it was enabled, it was getting several hundred-thousand requests each day. As soon as Anubis became active in there, it decreased to about 11 requests af…

That doesnt mean the PoW is doing anything, it might just mean bots have js disabled. I dont think the person was claiming annubis doesnt work, they were disputing PoW is the reason it actually works.

Yes, Anubis is just non standard and obscure, the proof of work bit is completely irrelevant (except for getting people on their phone to not visit your website).

Re: The bot situation on the internet is worse than you could imagine

#136

Post by drum55 »

bawolff wrote: post #130
drum55 wrote: post #59

Earlier quoted context omitted.

It doesn't matter if your hottest loop is using string comparisons, as another poster pointed out in C you aren't even doing the majority of the second hash because you know the result (or enough of it) before finishing it. The JavaScript version just does whole hashes and turns them into a Uint8Array, then iterates through it.

I dont understand what you mean. Are you saying that in C you only calculate the first few digits of the hash? That's not how sha256 works. Edit: oh i think you mean in c the string comparison short curcuits. I would expect the same to be true in javascript too. Its true in most languages. Maybe you are just worried about general language overhead, which is a fair point. Is the anubis check even using multiple thread…

The language matters, but your original guess was actually correct, you can do tricks with sha256 where you only end up calculating a fraction of the total double hash in order to get a pass or fail.

Modern bitcoin miners do a double sha256 hash and increment in just a little bit more than a single hash of work. The input is 80 bytes, which is two compression rounds of 64 bytes in sha256, only the data in the second round has changed (the appended nonce), so you don’t bother doing the first compression round again. With other quirks you can end up doing multiple hashes at once “asicboost” due to partial collisions within the input too.

Re: The bot situation on the internet is worse than you could imagine

#137

Post by ctoth »

lm411 wrote: post #106

AI companies and notably AI scrapers are a cancer that is destroying what's left of the WWW. I was hit with a pretty substantial botnet "distributed scraping" attack yesterday. - About 400,000 different IP addresses over about 3 hours - Mostly residential IP addresses - Valid and unique user agents and referrers - Each IP address would make only a few requests with a long delay in between requests It would hit the se…

If you don't mind me asking, what sort of data are you licensing? I noticed that you explicitly don't mention it.

Re: The bot situation on the internet is worse than you could imagine

#138

Post by dheera »

throw10920 wrote: post #45
dheera wrote: post #37

Earlier quoted context omitted.

I don't get this kHash thing. Do we have captchas mining bitcoin in a distributed fashion for free now?

The page says > Anubis uses a Proof-of-Work scheme in the vein of Hashcash And if you look up Hashcash on Wikipedia you get https://en.wikipedia.org/wiki/Hashcash which explains how Hashcash works in a fairly straightforward manner (unlike most math pages).

Oh fun so now we're effectively draining users' phone and laptop batteries now just to prove that they have batteries and somehow that's a proxy for them being human

Re: The bot situation on the internet is worse than you could imagine

#139

Post by VladVladikoff »

Rasbora wrote: post #96
VladVladikoff wrote: post #41

>How can you protect your sites from these bots? JA4 fingerprinting works decently for the residential proxies.

TLS fingerprinting is not sufficient to stop residential proxies, the proxy acts as a transparent pass-through at the TLS layer making it trivial to use something like curl_cffi to mimic a real browser TLS fingerprint. However residential proxies do have a weakness, since they need to maintain 2 separate TCP conenctions you can exploit RTT differences between layers 3 and 7 to detect if the connection to your server…

Cool link thanks for sharing. Too bad it is so expensive. Maybe I’ll find a way to DIY.

Re: The bot situation on the internet is worse than you could imagine

#140

Post by dspillett »

oasisbob wrote: post #109
dspillett wrote: post #107

Earlier quoted context omitted.

> it's sending us traffic with fbclids in the mix. No idea why. The click IDs are likely to make the traffic look more like a human who has clicked a link rather than a bot? That way it gets past simple filters that explicitly let such requests in before bothering to check that the source address of the request seems to be a DC rather than a residential IP. > citing the competitive environment All the companies are c…

> The click IDs are likely to make the traffic look more like a human who has clicked a link rather than a bot? It's certainly possible. However, the traffic is still coming from Facebook's network with a FB proxy PTR record in DNS. Seems much more likely to fool your typical site owner than a bad actor.

> Seems much more likely to fool your typical site owner than a bad actor.

Sorry, I wasn't clear. That is exactly what I think they would be doing. While it won't fool blocks that check source first it will fool those who allow requests with those IDs through because they can't afford to block their content appearing on the distraction media sites.

Post reply on HN