481–490 of 631 posts
Can IPFS or torrent and large local databases decentralised by people be a solution to this? I personally have the resources to share and host TBs of data but didn't find a good use to it.
This way crawlers might contribute back by providing extra storage and bandwidth.
Though something like ZeroNet seems a better approach to allow dynamic content.
Earlier quoted context omitted.
Yep. The "wouldn't it be great if we had robots do all the labor you are currently doing" argument only works if there is some plan to make sure that my rent gets paid other than me performing labor.
And that's the ultimate fail of capitalist ethics - the notion that we must all work just so we can survive. Look at how many shitty and utterly useless jobs exist just so people can be employed on them to survive. This has to change somehow. "Machines will do everything and we'll just reap the profits" is a vision that techno-millenialists are repeating since the beginnings of the Industrial Revolution, but we haven…
This isn't a consequence of capitalism. The notion of having to work to survive - assuming you aren't a fan of slavery - is baked into things at a much more fundamental level. And lots of people don't work, and are paid by a welfare state funded by capitalism-generated taxes.
> "Machines will do everything and we'll just reap the profits" is a vision that techno-millenialists are repeating since the beginnings of the Industrial Revolution, but we haven't seen that happening anywhere.
They were wrong, but the work is still there to do. You haven't come up with the utopian plan you're comparing this to.
> For some strange reason, technological progress seem to be always accompanied with an increase on human labor.
No it doesn't. What happens is not enough people are needed to do a job any more, so they go find another job. No one's opening barista-staffed coffee shops on every corner in the time when 30% of the world was doing agricultural labour.
Earlier quoted context omitted.
I’m supremely confident that attestation will arrive in one form or another in the near future. Anonymous browsing and potentially-malicious bots look identical . This was sort of OK up until now.
Agreed, it seems inevitable. Unfortunately I think it will also result in further centralization & consolidation into a handful of "trusted" megacorps. If you thought browser fingerprinting for ad tracking was creepy, just wait until they're using your actual fingerprint.
Earlier quoted context omitted.
I've pointed this out to a few people in this space. They tend to suggest that the value in AI is so great this means we should get rid of copyright law entirely.
…meaning that whatever model results would have no protection, and would be free for anyone to use?
Earlier quoted context omitted.
>It's equally trivial for a restaurant to set up a custom domain with their own 2-page website (overview and menu) on any of a hundred platforms that provide this service. Sure but they don't want to. If you want to see the menu they have online you need to follow their rules, not your own.
And if they want me as a customer, they have to follow _my_ rules. Obviously the restaurant has enough other customers and I have enough other restaurants to go to, so we both will be fine.
Sure, but putting their menu behind a trivial to access account shows they don't want you as a customer. You're the one complaining, not them.
Like some sort of legal honeypot trap.
> According to Drew, LLM crawlers don't respect robots.txt requirements and include expensive endpoints like git blame, every page of every git log, and every commit in your repository. They do so using random User-Agents from tens of thousands of IP addresses, each one making no more than one HTTP request, trying to blend in with user traffic. How do they know that these are LLM crawlers and not anything else?
I can tell you what it looks like in case of a git web interface like cgit: you get a burst of one or two isolated requests from a large number of IPs each for very obscure (but different) URLs, like a file contents at a specific commit id. And the user agent suggesting it's coming from IPhone or Android.
Earlier quoted context omitted.
AI expert here. It's probably for collecting training data and the crawlers are probably very unsupervised. I'd guess that they're literally the most simplistic crawler code you can imagine combined with parallelism across machines. The good news is that it's easy to disrupt these crawlers with some easy hacks. Tactical reverse slowloris is probably gonna make a comeback.
Zipbombing rogue AI's will be the new hotness.
To be clear, this is not an attack in the deliberate sense, and has nothing to do with AI except in that AI companies want to crawl the internet. This is more "FOSS sites damaged by extreme incompetence and unaccountability." The crawlers could just as well be search engine startups.
The big takeaway here is that Google's (and advertisement in general) dominance over the web is going away. This is because the only way to stop the bots is with a captcha, and this also stops search indexers from indexing your site. This will result in search engines not indexing sites, and hence providing no value anymore. There's probably going to be a small lag as the current knowledge in the current LLMs dry up…
I actually envision Liapunov stability, like wolf and rabbit populations. In this scenario, we're the rabbits. Human content will increase when AI populations decrease, this providing more food for AI, which will then increase. This drowns out human expression, and the humans will grow quieter. This provides less fodder for the AI, and they decrease. This means less noise and the humans grow louder. The cycle repeats…