Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FOSS infrastructure is under attack by AI companies

thelibre.news

Post reply on HN
Print view View original

421–430 of 631 posts

Re: FOSS infrastructure is under attack by AI companies

#421

Post by nonchalantsui »

noosphr wrote: post #174
brushfoot wrote: post #39

At this rate, it's more than FOSS infrastructure -- although that's a canary in the coalmine I especially sympathize with -- it's anonymous Internet access altogether. Because you can put your site behind an auth wall, but these new bots can solve the captchas and imitate real users like never before. Particularly if they're hitting you from residential IPs and with fake user agents like the ones in the article -- or…

You don't need an authorization wall to have your stuff behind, you can just as easily use an anonymous micropayment service for each request. That we live in an internet where getting too many visitors is an existential crisis for websites should tell you that our internet is not one that can survive long.

Are there any reputable anonymous micropayment services?

Re: FOSS infrastructure is under attack by AI companies

#422

Post by diggan »

PaulDavisThe1st wrote: post #417
diggan wrote: post #398

Earlier quoted context omitted.

Offering read-only mirrors via git+ http:// might be a solution then, at least to shed the load if anything. It does remind me a bit about companies complaining about being scraped and trying to prevent it, instead of offering a API so no one would have to scrape them.

We do precisely this ... and we're still dealing with the load issues. Currently I have fail2ban doing a 10 day block on any IP addr that hits our read only http-git endpoint twice in 30 mins. The problem with this is that the default implementation of iptables doesn't scale well to 100k blocked addresses.

Fair enough, thanks for explaining to an outsider. I hope you manage to work out a good solution in the end!

Re: FOSS infrastructure is under attack by AI companies

#423

Post by aendruk »

Suppafly wrote: post #394
madeofpalk wrote: post #388

Earlier quoted context omitted.

I don't have a Facebook or Instagram account, but I definitely eat tacos and I was put off when I couldn't see a new taco place's opening hours without an instagram accoutn. I'm not sure why you think why people who don't have a Facebook account wouldn't eat at restaurants

You're in a tiny enough minority that it doesn't matter to them. It's like Amish complaining that they can't use a drive-thru window or something. Except it'd take you 30 seconds, one time, to solve your problem forever.

I’m a little optimistic here. Step by small step lately the more normal people in my life have been joining this demographic.

Re: FOSS infrastructure is under attack by AI companies

#424

Post by sofixa »

dreamcompiler wrote: post #413
Suppafly wrote: post #331

Earlier quoted context omitted.

>Most normies just think everyone has a Facebook/Instagram account and can't even imagine a world where that is not the case. The people without a basic internet presence aren't likely to be customers anyway so it's not a huge loss. It's trivial to setup a basic account for any site that doesn't contain any personal data you want to keep hidden, if you aren't willing to do that, you're in a tiny minority.

> It's trivial to setup a basic account for any site that doesn't contain any personal data you want to keep hidden It's equally trivial for a restaurant to set up a custom domain with their own 2-page website (overview and menu) on any of a hundred platforms that provide this service. Most of these services are not free like FB, but any business that can afford a landline phone can afford a real website.

> Most of these services are not free like FB

There are free ones as well, just as a subdomain (something.wordpress.com or something.wix.com), not a full top level custom domain.

Re: FOSS infrastructure is under attack by AI companies

#425

Post by diggan »

Ndymium wrote: post #383
zoobab wrote: post #262

Earlier quoted context omitted.

"disk was filled with generated zip archives" That's bad software design to generate ZIP files on the fly.

They're deleted by default every 24 hours and that time is configurable. Not useful when you get 60 requests per second though.

> They're deleted by default every 24 hours

Hm, so it's a cache then? Requesting the same tarball 100 times shouldn't create 100 zip files if they're cached, and if they aren't cached they shouldn't fill up the disk.

Re: FOSS infrastructure is under attack by AI companies

#426

Post by orthecreedence »

adamc wrote: post #113
phkahler wrote: post #74

So I'll just float an idea again that always gets rejected here. This is yet another problem that could be solved completely by... Eliminating anonymity by default on the internet. To be clear, you could still have anonymous spaces like Reddit where arbitrary user IDs are used and real identities are discarded. People could opt-in to those spaces. But for most people most of the time, things get better when you can v…

In the USA, where an increasingly authoritarian government has come into power... I think many would stop using the internet if that happened.

I mean, this has already happened, it just happened in a more sinister way than "FreedomNet now requires logins from all users!" Ad companies and social media track everything you do and can tie it together with various forms of packaged/bought/sold identity that follow you wherever you go. Even with agressive ad blocking, I get ads on Instagram for things I looked up in my browser that I have never used to log into Insta with. We're constantly deanonymized, it just happens below the surface. And all of this is hoovered up by the US dragnet surveillance programs.

So do I support a fully authenticated internet? Fuck no. If we can get good at bot detection, zip bomb the fuckers. In the meantime, work as hard as we can to dismantle the hellscape that the internet has become. I'm all for decentralized, sovereign identity systems that aren't owned by some profiteering corpo cretins or some proto-fascist state, but I don't want it to be a requirement to look at photos of dogs or plan my next trip.

Re: FOSS infrastructure is under attack by AI companies

#427

Post by diggan »

chneu wrote: post #304
Cthulhu_ wrote: post #249

Back when search engines caused this, the industry made an agreement and designed the robots.txt spec in order to avoid legal frameworks being made to stop them. Because of that, legal frameworks weren't being made. Now there's a new generation of hungry hungry hippo indexers that didn't agree to that and who feel intense pressure from competition to scoop up as much data as they can, who just ignore it. Legislation…

I'd be shocked if a single member of the US House or Congress know what robots.txt is.

I'm pleased to declare you shocked: https://www.google.com/search?q=site%3Acongress.gov+%22robot...

Re: FOSS infrastructure is under attack by AI companies

#428

Post by photonthug »

brushfoot wrote: post #39

At this rate, it's more than FOSS infrastructure -- although that's a canary in the coalmine I especially sympathize with -- it's anonymous Internet access altogether. Because you can put your site behind an auth wall, but these new bots can solve the captchas and imitate real users like never before. Particularly if they're hitting you from residential IPs and with fake user agents like the ones in the article -- or…

> What's left except for sites to start requiring credit cards, Worldcoin, or some equally depressing equivalent.

Just to say the quiet part out loud here.. one of the biggest reasons this is depressing is that it's not only vandalism but actually vandalism with huge compounding benefits for the assholes involved and grabbing the content is just the beginning. If they take down the site forever due to increasing costs? Great, because people have to use AI for all documentation. If we retreat from captcha and force people to put in credit cards or telephone numbers? Great, because the internet is that much less anonymous. Data exfiltration leads to extra fraud? Great, you're gonna need AI to combat that. It's all pretty win-win for the bad actors.

People have discussed things like the balkanization of the internet for a long time now. One might think that the threat of that and/or the fact that it's such an unmitigated dumpster fire already might lead to some caution about making it worse! But pushing the bounds of harassment and friction that people are willing to deal with is moot anyway, because of course they have no real choice in the matter.

Re: FOSS infrastructure is under attack by AI companies

#429

Post by Ndymium »

diggan wrote: post #425
Ndymium wrote: post #383

Earlier quoted context omitted.

They're deleted by default every 24 hours and that time is configurable. Not useful when you get 60 requests per second though.

> They're deleted by default every 24 hours Hm, so it's a cache then? Requesting the same tarball 100 times shouldn't create 100 zip files if they're cached, and if they aren't cached they shouldn't fill up the disk.

They are a cache, but you can generate them for each branch, tag, and commit in at least three different formats... Now imagine you have a repo with several thousand commits.

Re: FOSS infrastructure is under attack by AI companies

#430

Post by kmeisthax »

forgetfreeman wrote: post #274
billy99k wrote: post #197

Earlier quoted context omitted.

I already started incorporating AI into my workflow. It's definitely helped with productivity. At some point in the future, if you aren't using AI, you won't be able to compete in the job market.

I started a borderline smug response here pointing out how bullshit white collar and service jobs* where in deep shit but folks who actually work for a living would be fine. I scrapped it halfway through when it occurred to me that if everyone's broke then by definition nobody's spending money on stuff like contractors, mechanics, and other hardcore blue collar trades. Toss in AI's force multiplication of power deman…

Neo-Luddism is less stupid when you remember that the Luddites weren't angry that looms existed. Smashing looms was their tactic, not their goal.

Parliament had made a law phasing in the introduction of automated looms; specifically so that existing weavers were first on the list to get one. Britain's oligarchy completely ignored this and bought or built looms anyway; and because Parliament is part of that oligarchy, the law effectively turned into "weavers get looms last". That's why they were smashing looms - to bring the oligarchy back to the negotiating table.

The oligarchy responded the way all violent thugs do: killing their detractors and lying about their motives.

Post reply on HN