Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FOSS infrastructure is under attack by AI companies

thelibre.news

Post reply on HN
Print view View original

441–450 of 631 posts

Re: FOSS infrastructure is under attack by AI companies

#441

Post by diggan »

Ndymium wrote: post #429
diggan wrote: post #425

Earlier quoted context omitted.

> They're deleted by default every 24 hours Hm, so it's a cache then? Requesting the same tarball 100 times shouldn't create 100 zip files if they're cached, and if they aren't cached they shouldn't fill up the disk.

They are a cache, but you can generate them for each branch, tag, and commit in at least three different formats... Now imagine you have a repo with several thousand commits.

Yeah, fair point. Although I think the only uniqueness is commits here, tarballs generated from branches and tags are ultimately the same as the ones generated by the commit that those reference. But I still agree with your overall point.

Re: FOSS infrastructure is under attack by AI companies

#442

Post by numpad0 »

ozornin wrote: post #345
floren wrote: post #288

Earlier quoted context omitted.

Just change the pictures in cmd/anubis/static/img/ to whatever you prefer, I think.

Or, alternatively, you know, pay the author for the work they've done

Or, alternatively, just embrace anime-porn content spectrum. I mean, just compare platforms that are free of it and ones that are chock full, and see which ones die and which grows.

Re: FOSS infrastructure is under attack by AI companies

#443

Post by namaria »

diggan wrote: post #390
reginald78 wrote: post #377

Earlier quoted context omitted.

Maybe I'm missing something, but doesn't this mean the work has to be done by the client AND the server every time a challenge is issued? I think ideally you'd want work that was easy for the server and difficult for the server . And what is to stop being DDoS'd by clients that are challenged but neglect to perform the challenge? Regardless, I think something like this is the way forward if one doesn't want to throw…

> I think ideally you'd want work that was easy for the server and difficult for the server. That's exactly how it works (easy for server, hard for client ). Once the client completed the Proof-of-Work challenge, the server doesn't need to complete the same challenge, it only needs to validate that the results checks out. Similar to how in Proof-of-Work blockchains where coming up with the block hashes is difficult,…

I think going full circle would be something like bitcoin being created on top of DoS prevention software and then eventually DoS prevention starting to use bitcoin. A tool being used for something than something else than the first something again is just... nothing? Happens all the time?

Re: FOSS infrastructure is under attack by AI companies

#445

Post by Ray20 »

lgeek wrote: post #258
ericholscher wrote: post #3

Yep -- our story here: https://about.readthedocs.com/blog/2024/07/ai-crawlers-abuse... (quoted in the OP) -- everyone I know has a similar story who is running large internet infrastructure -- this post does a great job of rounding a bunch of them up in 1 place. I called it when I wrote it, they are just burning their goodwill to the ground. I will note that one of the main startups in the space worked with us direct…

> One crawler downloaded 73 TB of zipped HTML files in May 2024 [...] This cost us over $5,000 in bandwidth charges I had to do a double take here. I run (mostly using dedicated servers) infrastructure that handles a few hundred TB of traffic per month, and my traffic costs are on the order of $0.50 to $3 per TB (mostly depending on the geographical location). AWS egress costs are just nuts.

I think uncontrolled price of cloud traffic - is a real fraud and way bigger problem then some AI companies that ignore robot.txt. One time we went over limit on Netlify or something, and they charged over thousand for a couple TB.

Re: FOSS infrastructure is under attack by AI companies

#446

Post by neutronicus »

lithocarpus wrote: post #293
chii wrote: post #85

Earlier quoted context omitted.

> remake the entire world into one where the owners of these companies own everything and are completely unconstrained how has this been any different from the past 10,000 years of human conquest and domination?

In my view it's the same thing, same trajectory -- with more power in the hands of fewer people further along the trajectory. It can be better or worse depending on what those with power choose to do. Probably worse. There has been conquest and domination for a long time, but ordinary people have also lived in relative peace gathering and growing food in large parts of the world in the past, some for entire generatio…

On the bright side, armies of robot slaves give us an off-ramp from the unsustainable pyramid scheme of population growth.

Be fruitful, and multiply, so that you may enjoy a comfortable middle age and senescence exploiting the shit out of numerous naive 25-year-olds! If it's robots, we can ramp down the population of both humans and robots until the planet can once again easily provide abundance.

Re: FOSS infrastructure is under attack by AI companies

#447

Post by tadfisher »

dbingham wrote: post #296
WhyOhWhyQ wrote: post #277

Earlier quoted context omitted.

I've pointed this out to a few people in this space. They tend to suggest that the value in AI is so great this means we should get rid of copyright law entirely.

That value is only great if it's shared equitably with the rest of the planet. If it's owned by a few, as it is right now, it's an existential threat to the life, liberty, and pursuit of a happiness of everyone else on the planet. We should be seriously considering what we're going to do in response to that threat if something doesn't change soon.

AI is going to implode within 2 years. Once it starts ingesting its own output as training data it is going to be at best capped at its current capability and at worst even more hallucinatory and worthless.

Re: FOSS infrastructure is under attack by AI companies

#450

Post by yyhhsj0521 »

mary-ext wrote: post #350

I do wonder if it's a customer on Alibaba Cloud that's doing all this or if it's entirely Alibaba's own doing that's been wrecking havoc on everyone's sites (mine included) I don't really like blocking an entire ASN, especially since I don't mind (responsible) crawling to begin with, but I was left with no choice

I doubt Alibaba uses its own cloud. It doesn't work like that at most cloud provider companies.
Post reply on HN