Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The bot situation on the internet is worse than you could imagine

gladeart.com

Post reply on HN
Print view View original

161–170 of 179 posts

Re: The bot situation on the internet is worse than you could imagine

#161

Post by TheScaryOne »

lotsofpulp wrote: post #159
lm411 wrote: post #129

Earlier quoted context omitted.

I would love for the ad-supported model to die. I hate ads, and I hate having to serve ads. We get some subscription users but nowhere near enough to cover costs. Unfortunately, what I think will happen - and indeed already is - is that the AI companies themselves will replace much of the WWW. Sites like the one I am talking about will cease to exist. AI companies, once they can no longer scrape (steal) the data will…

Evidently, users and customers like not having to sift through hundreds or thousands of independent websites.

I much prefer having my thoughts distilled down into easily digestable and agreeable idioms that I can push around with absolute faith that they weren't just lies written by some PERSON on the internet.

Re: The bot situation on the internet is worse than you could imagine

#162

Post by righthand »

collingreen wrote: post #156
righthand wrote: post #119

Earlier quoted context omitted.

You’ll enjoy painting or some other art even if you aren’t interested in the arts. That’s what I’ve seen written about it.

Or they'll let you starve to death which is way way easier and way faster for "them"

Well yeah, art will be valueless in a flooded market. Starving is implied, “starving artist”.

Re: The bot situation on the internet is worse than you could imagine

#163

Post by ranger_danger »

shimman wrote: post #114
lm411 wrote: post #106

AI companies and notably AI scrapers are a cancer that is destroying what's left of the WWW. I was hit with a pretty substantial botnet "distributed scraping" attack yesterday. - About 400,000 different IP addresses over about 3 hours - Mostly residential IP addresses - Valid and unique user agents and referrers - Each IP address would make only a few requests with a long delay in between requests It would hit the se…

Do you not run Anubis or have strict fail2ban rules? I just straight up ban IPs forever if they lookup files that will never exist on my servers. That plus Anubis with the strictest settings. https://anubis.techaro.lol/

The more sophisticated bots run real headless browsers that anubis can't touch, and they only follow links that are actually visible on the page, so they wouldn't hit fail2ban.

They even sell access to proxy servers that successfully evade cloudflare captchas automatically.

Re: The bot situation on the internet is worse than you could imagine

#164

Post by ranger_danger »

afinlayson wrote: post #116
lm411 wrote: post #106

AI companies and notably AI scrapers are a cancer that is destroying what's left of the WWW. I was hit with a pretty substantial botnet "distributed scraping" attack yesterday. - About 400,000 different IP addresses over about 3 hours - Mostly residential IP addresses - Valid and unique user agents and referrers - Each IP address would make only a few requests with a long delay in between requests It would hit the se…

At some point there needs to be a check if it's a real human... But it's a cat and mouse game - any way we create to keep bots off gets a work around by clever engineers.

What makes a real human?

Re: The bot situation on the internet is worse than you could imagine

#165

Post by julianlam »

Trufa wrote: post #64
siva7 wrote: post #57

So the elephant in the room: How much of HN is bot generated? Those who know have every incentive not to share and those who don't have no way to figure it out. At this point i have to assume that every new account is a bot

I felt a vibe change, some are obvious and some not, but it does feel different, the main change i've seen is in downvotes, I don't say very controversial things and have had many things very quickly downvoted, and then slowly upvoted, I think hn was very slow to downvote in the past (except obvious trolls/spam). So for me the main worry is not even the comments, but the invisible bias generated by voting.

I've noticed that too. I thought I was just posting particularly bad takes.

Re: The bot situation on the internet is worse than you could imagine

#166

Post by throw10920 »

dheera wrote: post #138
throw10920 wrote: post #45

Earlier quoted context omitted.

The page says > Anubis uses a Proof-of-Work scheme in the vein of Hashcash And if you look up Hashcash on Wikipedia you get https://en.wikipedia.org/wiki/Hashcash which explains how Hashcash works in a fairly straightforward manner (unlike most math pages).

Oh fun so now we're effectively draining users' phone and laptop batteries now just to prove that they have batteries and somehow that's a proxy for them being human

Given that you're clearly completely ignorant of everything in the anti-spam space, you should probably do some research before making uninformed comments like this.

Re: The bot situation on the internet is worse than you could imagine

#167

Post by lm411 »

lotsofpulp wrote: post #159
lm411 wrote: post #129

Earlier quoted context omitted.

I would love for the ad-supported model to die. I hate ads, and I hate having to serve ads. We get some subscription users but nowhere near enough to cover costs. Unfortunately, what I think will happen - and indeed already is - is that the AI companies themselves will replace much of the WWW. Sites like the one I am talking about will cease to exist. AI companies, once they can no longer scrape (steal) the data will…

Evidently, users and customers like not having to sift through hundreds or thousands of independent websites.

Absolutely.

It's so much easier to know the truth when someone else tells me what it is and what to think about it.

How refreshing.

Re: The bot situation on the internet is worse than you could imagine

#168

Post by x______________ »

ctoth wrote: post #137
lm411 wrote: post #106

AI companies and notably AI scrapers are a cancer that is destroying what's left of the WWW. I was hit with a pretty substantial botnet "distributed scraping" attack yesterday. - About 400,000 different IP addresses over about 3 hours - Mostly residential IP addresses - Valid and unique user agents and referrers - Each IP address would make only a few requests with a long delay in between requests It would hit the se…

If you don't mind me asking, what sort of data are you licensing? I noticed that you explicitly don't mention it.

And self-ddos via HN advertising (a la slashdotted?:)

Re: The bot situation on the internet is worse than you could imagine

#169

Post by tripzilch »

lm411 wrote: post #129
everdrive wrote: post #112

Earlier quoted context omitted.

Thanks for sharing the perspective here. I think a lot of folks on HN have rightly said that a lot of the problems with the modern internet are due to the ad-supported business model. I don't think you were ever going to move away from it voluntarily -- too many people support it, even if they grumble about it. But maybe (and likely for worse) LLMs will finally kill this model.

I would love for the ad-supported model to die. I hate ads, and I hate having to serve ads. We get some subscription users but nowhere near enough to cover costs. Unfortunately, what I think will happen - and indeed already is - is that the AI companies themselves will replace much of the WWW. Sites like the one I am talking about will cease to exist. AI companies, once they can no longer scrape (steal) the data will…

> I hate ads, and I hate having to serve ads. We get some subscription users but nowhere near enough to cover costs.

I hate ads and I hate having to use an ad blocker to be able to not go crazy in order to use the Internet.

You merely hate "having" to serve ads because it denies you profit from the people you're exploiting with those ads.

Why is your business more deserving to exist on the Internet than my usage??

Re: The bot situation on the internet is worse than you could imagine

#170

Post by AugSun »

ranger_danger wrote: post #164
afinlayson wrote: post #116

Earlier quoted context omitted.

At some point there needs to be a check if it's a real human... But it's a cat and mouse game - any way we create to keep bots off gets a work around by clever engineers.

What makes a real human?

CC payment. This is the ultimate test.
Post reply on HN