Does GitHub have this problem?
afaik github is microsoft and openai is a partner of microsoft so probably not
FOSS infrastructure is under attack by AI companies
81–90 of 631 posts
Re: FOSS infrastructure is under attack by AI companies
#82We need a project in the spirit of Spamhaus to actively maintain a list of perpetrating IPs. If they're cycling through IPs and IP blocks I don't know how sustainable a CAPTCHA-like solution is.
This sounds backwards to me, if you maintain a list of IPs but they are constantly cycling them, it'll get out of date quickly, but a captcha-like system will (hopefully) always stop bot traffic
Re: FOSS infrastructure is under attack by AI companies
#83“… he complained that LLM companies were crawling data without respecting robosts.txt“
Maybe that is the problem? They misspelled robots.txt
Re: FOSS infrastructure is under attack by AI companies
#84Earlier quoted context omitted.
That's what I've been doing! It works shockingly well. https://github.com/TecharoHQ/anubis
Unless I am missing something, the result of that generated work has no monetary value though.
I was inspired by https://en.wikipedia.org/wiki/Hashcash, which was proof of work for email to disincentivize spam. To my horror, it worked sufficiently for my git server so I released it as open source. It's now its own project and protects big sites like GNOME's GitLab.
Re: FOSS infrastructure is under attack by AI companies
#85Earlier quoted context omitted.
> just burning their goodwill to the ground AI firms seem to be leading from a position that goodwill is irrelevant: a $100bn pile of capital, like an 800lb gorilla, does what it wants. AI will be incorporated into all products whether you like it or not; it will absorb all data whether you like it or not.
Yep. And it is much more far reaching than that. Look at the primary economic claim offered by AI companies: to end the need for a substantial portion of all jobs on the planet. The entire vision is to remake the entire world into one where the owners of these companies own everything and are completely unconstrained. All intellectual property belongs to them. All labor belongs to them. Why would they need good will…
> remake the entire world into one where the owners of these companies own everything and are completely unconstrained
how has this been any different from the past 10,000 years of human conquest and domination?
Re: FOSS infrastructure is under attack by AI companies
#86So I'll just float an idea again that always gets rejected here. This is yet another problem that could be solved completely by... Eliminating anonymity by default on the internet. To be clear, you could still have anonymous spaces like Reddit where arbitrary user IDs are used and real identities are discarded. People could opt-in to those spaces. But for most people most of the time, things get better when you can v…
Ok, let's have a very concrete discussion then: how would you implement this on top of the current protocol stack?
Re: FOSS infrastructure is under attack by AI companies
#87As I already noted and got downvoted for it - the incentives to go or support open source (unless bros start realising troves of training data already scraped) are really diminishing.
Since apparently they (scrapers) have no intent In doing (releasing) but so, expect the commercial open source to achieve a de facto protocol status very soon. And the rest may not exist in a centralised and such free manner anymore.
Re: FOSS infrastructure is under attack by AI companies
#88The big takeaway here is that Google's (and advertisement in general) dominance over the web is going away. This is because the only way to stop the bots is with a captcha, and this also stops search indexers from indexing your site. This will result in search engines not indexing sites, and hence providing no value anymore. There's probably going to be a small lag as the current knowledge in the current LLMs dry up…
AI is good at solving captchas. But even if everyone added a captcha search engines will continue indexing. Because it is easy to add authentication for search engines to escape captcha, Google will just need to publish a public key.
Re: FOSS infrastructure is under attack by AI companies
#89Earlier quoted context omitted.
> just burning their goodwill to the ground AI firms seem to be leading from a position that goodwill is irrelevant: a $100bn pile of capital, like an 800lb gorilla, does what it wants. AI will be incorporated into all products whether you like it or not; it will absorb all data whether you like it or not.
Yep. And it is much more far reaching than that. Look at the primary economic claim offered by AI companies: to end the need for a substantial portion of all jobs on the planet. The entire vision is to remake the entire world into one where the owners of these companies own everything and are completely unconstrained. All intellectual property belongs to them. All labor belongs to them. Why would they need good will…
I have this line of thought as well but then I wonder, if we are all out of jobs and out of substantial capital to spend, how do these owners make money ultimately? It's a genuine question and I'm probably missing something obvious. I can see a benevolant/post-scarcity spin to this but the non-benevolant one seems self defeating.
Re: FOSS infrastructure is under attack by AI companies
#90To be clear, this is not an attack in the deliberate sense, and has nothing to do with AI except in that AI companies want to crawl the internet. This is more "FOSS sites damaged by extreme incompetence and unaccountability." The crawlers could just as well be search engine startups.
> has nothing to do with AI
Surely has very little to do with "intelligence".