Live data from Hacker News

Miasma: A tool to trap AI web scrapers in an endless poison pit

github.com

191–200 of 276 posts

Re: Miasma: A tool to trap AI web scrapers in an endless poison pit

#191

Earlier quoted context omitted.

[flagged]

> If I can poison them and their families, I will. Don't post anything online that you don't want to be brought up in court later.

Like the OP's solution it was about scrapers and the models they share their data with.

Re: Miasma: A tool to trap AI web scrapers in an endless poison pit

#192
post #138

Earlier quoted context omitted.

The problem I have, is they hammer my site so hard they take it down. The content is for everyone. They can have it. Just don't also take it away from everybody else.

Been there recently. Rate limit on nginx and anti-syn flood on pf solved it.

I'm being hit with 300 req/s 24/7 from hundreds of thousands of unique IP's from residential proxies. I can't rate limit any further without hurting the real users.

Re: Miasma: A tool to trap AI web scrapers in an endless poison pit

#193

Earlier quoted context omitted.

Odd thing about cookies… they disappear after one serving. Websites are an endless stream of cookies. The analogy doesn’t hold.

Bandwidth and compute constraints make websites all but an endless stream though.

That's exactly it. It costs me real time and money to serve the 97% of fake traffic that just takes without giving me anything in return.

Re: Miasma: A tool to trap AI web scrapers in an endless poison pit

#194

Way back in the day I had a software product, with a basic system to prevent unauthorised sharing, since there was a small charge for it. Every time I released an update, and new crack would appear. For the next six months I worked on improving the anti-copying code until I stumbled across an article by a coder in the same boat as me. He realised he was now playing a game with some other coders where he make the copy…

So many problems could be solved by letting go. Unfortunately social media and snowballing copyright maximalism has inflated egos to the point where more and more people think they need to control everything.

If only I could go back in time 26 years and let myself know I was right to focus on my customers.

Re: Miasma: A tool to trap AI web scrapers in an endless poison pit

#195

Earlier quoted context omitted.

The problem I have, is they hammer my site so hard they take it down. The content is for everyone. They can have it. Just don't also take it away from everybody else.

Unintentional denial-of-service attacks from AI scrapers are definitely a problem, I just don't know if "theft" is the right way to classify them. They shouldn't get lumped in with intellectual property concerns, which are a different matter. AI scrapers are a tragedy of the commons problem kind of like Kessler syndrome: a few bad actors can ruin low Earth orbit for everyone via space pollution, which is definitely a…

Theft isn't far off, it seems closer to me than using the word for IP violations.

When a crawler aggressively crawls your site, they're permanently depriving you the use of those resources for their intended purpose. Arguably, it looks a lot like conversion.

Re: Miasma: A tool to trap AI web scrapers in an endless poison pit

#196
post #138

Earlier quoted context omitted.

Been there recently. Rate limit on nginx and anti-syn flood on pf solved it.

I'm being hit with 300 req/s 24/7 from hundreds of thousands of unique IP's from residential proxies. I can't rate limit any further without hurting the real users.

Yeah, IP-based rate limits are nearly ineffective these days.

Re: Miasma: A tool to trap AI web scrapers in an endless poison pit

#197

Earlier quoted context omitted.

you're totally right about not being theft, but we have a term. you used it yourself, "distributed denial of service". that's all it is. these crawlers should be kicked off the internet for abuse. people should contact the isp of origin.

Firstly, since this argument is about semantic pedantry anyways, it's just denial-of-service, not distributed denial-of-service. AI scraper requests come from centralized servers, not a botnet. Secondly, denial-of-service implies intentionality and malice that I don't think is present from AI scrapers. They cause huge problems, but only as a negligent byproduct of other goals. I think that the tragedy of the commons…

ah, no fun, I was going to continue the semantic deconstruction with a whole bunch of technicalities about how you're not quite precisely accurate and you gotta go do the right thing and retract your statements.

boo. took all the fun out of it ;)

Re: Miasma: A tool to trap AI web scrapers in an endless poison pit

#199

Earlier quoted context omitted.

My own view is, I thought we were all agreed that the idea that Microsoft can restrict Wine from even using ideas from Windows, such that people who have read the leaked Windows source cannot contribute to Wine, was a horrible abuse of the legal system that we only went along with under duress? Now when it's our data being used, or more cynically when there's money to be made, suddenly everyone is a copyright maximal…

If you want people to read and learn from each other, you should incentivize people to make content worth reading and learning from. Making LLM training a viable loophole for copyright law means there won’t be incentives to produce such work.

I don't think that's the case.

People getting better at writing is only going to increase the quality of the output.

Increasing both competition and tooling (by providing every writer with the world's greatest encylcopedia/thesaurus/line-editor/brainstormer/planner/etc) is only going to make writers better.

Will there be lots of people who misuse the system? Are there lots of people who use thesaurus words without knowing what they're talking about? Can't you tell the difference?

I see in LLMs a lowering of the ground floor making it easier for people to get in. This will increase the total availability of content.

I also see in LLMs a raising of the top bar making it harder to be the best. If more people are writing and more people are trying to be the best, the best is going to get better.

Consider chess. Have we suddenly stopped playing chess now that a phone can beat 95+% of people? No. The market is stronger than ever and still growing. The greatest player in the world use the chess algorithms to refine their play and the play keeps expanding in new and interesting ways.

In both writing and chess, yes, there is an explosion of low and middling play. But since when have we not always had people producing content and playing chess that when compared to the masters of the field is generally viewed as substandard?

But here's the kicker. Some people's favorite genre is badly editted fanfic. Some people genuinely derive actual pleasure from things that you or I might call garbage. And what's wrong with that? Who am I to say that you can't love clutzy firecop loves suburban housewife paperbacks? Or Zelda/Harry Potter crossfics or whatever.

Re: Miasma: A tool to trap AI web scrapers in an endless poison pit

#200
post #155

Earlier quoted context omitted.

>What it the model then creates a virtual actor that is very close to the real actor? "Likeness" is a separate concept from copyrights https://en.wikipedia.org/wiki/Personality_rights

I wish I lived in the alternative timeline where open source folks didn't look a gift horse in the mouth and actually used these tools to copy left the shit out of software to the point where proprietary closed source software has no advantage. But instead we've got people posting "honey pots" that an LLM will immediately detect and route around.

I bet we'd cure all cancers in a month if everyone whining about slop actually went and did something about it.
Post reply on HN