Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FOSS infrastructure is under attack by AI companies

thelibre.news

Post reply on HN
Print view View original

201–210 of 631 posts

Re: FOSS infrastructure is under attack by AI companies

#201

Post by regularfry »

3029aGfqgaF wrote: post #178

Open source licenses are no longer strong enough. The copyright laundering use case was not been anticipated when the licenses were written. All licenses need a clause like the following: This software is for humans. AI training is prohibited and carries a default penalty of $1 trillion.

They are "no longer strong enough" to the extent that GitHub, backed by Microsoft, has had to indemnify their GitHub Copilot customers against copyright claims, and to provide a feature to explicitly prevent open source code from being regurgitated into your codebase.

That's not "no longer strong enough". That's a very strong system applying leverage to a powerful actor.

Re: FOSS infrastructure is under attack by AI companies

#202

Post by diggan »

TonyTrapp wrote: post #196
diggan wrote: post #56

> According to Drew, LLM crawlers don't respect robots.txt requirements and include expensive endpoints like git blame, every page of every git log, and every commit in your repository. They do so using random User-Agents from tens of thousands of IP addresses, each one making no more than one HTTP request, trying to blend in with user traffic. How do they know that these are LLM crawlers and not anything else?

As someone that is also affected by this: We see a manifold increase in requests since this LLM crap is going on. Many of these IPs come from companies that obviously work with LLM technology, but the problem is that it's 100s of IPs doing 1 request, not 1 IP doing 100s of requests. It's just extremely unlikely that anyone else is responsible for this.

> IPs come from companies that obviously work with LLM technology

Like from their own ASNs you're saying? Or how are you connecting the IPs with the company?

> is that it's 100s of IPs doing 1 request

Are all of those IPs within the same ranges or scattered?

Thanks a lot for taking the time to talk about your experience btw, as someone who hasn't been hit by this it's interesting to have more details about it before it eventually happens.

Re: FOSS infrastructure is under attack by AI companies

#203

Post by datadrivenangel »

Coffeewine wrote: post #189
Sharlin wrote: post #147

Earlier quoted context omitted.

Maxim 1: "Pillage, then burn."

Another Schlock Mercenary fan? Or does this adage have many adherents?

The adage predates the longest continuous webcomic, but not as a maxim.

Re: FOSS infrastructure is under attack by AI companies

#204

Post by voidUpdate »

nzeid wrote: post #108
voidUpdate wrote: post #82

Earlier quoted context omitted.

This sounds backwards to me, if you maintain a list of IPs but they are constantly cycling them, it'll get out of date quickly, but a captcha-like system will (hopefully) always stop bot traffic

Only if your CAPTCHA-like is hurled at every client indiscriminately. Otherwise you'll end up right back where Spamhaus started: maintaining your own list of good and bad actors. The advantage of a third party service is that you're sharing intel of bad actors.

I can't confirm but I believe it is applied to every client

Re: FOSS infrastructure is under attack by AI companies

#206

Post by spenczar5 »

ericholscher wrote: post #3

Yep -- our story here: https://about.readthedocs.com/blog/2024/07/ai-crawlers-abuse... (quoted in the OP) -- everyone I know has a similar story who is running large internet infrastructure -- this post does a great job of rounding a bunch of them up in 1 place. I called it when I wrote it, they are just burning their goodwill to the ground. I will note that one of the main startups in the space worked with us direct…

Thanks for writing about this. Is it clear that this is from crawlers, as opposed to dynamic requests triggered by LLM tools, like Claude Code fetching docs on the fly?

Re: FOSS infrastructure is under attack by AI companies

#207

Post by yubblegum »

pjc50 wrote: post #10
ericholscher wrote: post #3

Yep -- our story here: https://about.readthedocs.com/blog/2024/07/ai-crawlers-abuse... (quoted in the OP) -- everyone I know has a similar story who is running large internet infrastructure -- this post does a great job of rounding a bunch of them up in 1 place. I called it when I wrote it, they are just burning their goodwill to the ground. I will note that one of the main startups in the space worked with us direct…

> just burning their goodwill to the ground AI firms seem to be leading from a position that goodwill is irrelevant: a $100bn pile of capital, like an 800lb gorilla, does what it wants. AI will be incorporated into all products whether you like it or not; it will absorb all data whether you like it or not.

They are also gutting the profession of software engineering. It's a clever scam actually: to develop software a company will need to pay utility fees to A"I" companies and since their products are error prone voila use more A"I" tools to correct the errors of the other tools. Meanwhile software knowledge will atrophy and soon ala WALE we'll have software "developers" with 'soft bones' floating around on conveyed seats slurping 'sugar water' and getting fat and not knowing even how to tie their software shoelaces.

Re: FOSS infrastructure is under attack by AI companies

#208

Post by nemomarx »

chii wrote: post #85
UncleMeat wrote: post #30

Earlier quoted context omitted.

Yep. And it is much more far reaching than that. Look at the primary economic claim offered by AI companies: to end the need for a substantial portion of all jobs on the planet. The entire vision is to remake the entire world into one where the owners of these companies own everything and are completely unconstrained. All intellectual property belongs to them. All labor belongs to them. Why would they need good will…

> remake the entire world into one where the owners of these companies own everything and are completely unconstrained how has this been any different from the past 10,000 years of human conquest and domination?

in the past, you had to give some of your spoils to those who did the conquering for you, and laborers after that. if you can automate and replace all work, including maintening the robots that do that and training them, you no longer need to share anything.

Re: FOSS infrastructure is under attack by AI companies

#209

Post by sambull »

neutronicus wrote: post #185
maaaaattttt wrote: post #89

Earlier quoted context omitted.

I have this line of thought as well but then I wonder, if we are all out of jobs and out of substantial capital to spend, how do these owners make money ultimately? It's a genuine question and I'm probably missing something obvious. I can see a benevolant/post-scarcity spin to this but the non-benevolant one seems self defeating.

"Making money" is only a relevant goal when you need money to persuade humans to do things for you . Once you have an army of robot slaves ... you've rendered the whole concept of money irrelevant . Your skynet just barters rare earth metals with other skynets and your robot slaves furnish your desired lifestyle as best they can given the amount of rare earth metals your skynet can get its hands on. Or maybe a better…

Good thing AI for now needs power, water and a place to exchange heat. Our version of womp rats if it goes to far I guess.
Post reply on HN