AI bots are destroying Open Access
go-to-hellman.blogspot.com
AI bots are destroying Open Access
1–10 of 52 posts
Re: AI bots are destroying Open Access
#2The only way--the only way--to solve these issues is with web servers requiring that all clients authenticate with a credential that is provably tied to a real-world entity--person or corporate entity--so that legal recourse is available to the server owner when abuse occurs. The internet is no longer high-trust; we're running web servers the same way we'd run an honor-system store where people just come in and steal, anonymously, and with no recourse.
Re: AI bots are destroying Open Access
#3The only way--the only way--to solve these issues is with web servers requiring that all clients authenticate with a credential that is provably tied to a real-world entity--person or corporate entity--so that legal recourse is available to the server owner when abuse occurs. The internet is no longer high-trust; we're running web servers the same way we'd run an honor-system store where people just come in and steal…
I guess I'm done using the Internet then.
Re: AI bots are destroying Open Access
#4Why are "AI" bots generating so much fuss. Is it because there are so many of them? Is it because AI companies are each writing their own (bad) crawlers instead of using existing ones?
Re: AI bots are destroying Open Access
#5Why are "AI" bots generating so much fuss. Is it because there are so many of them? Is it because AI companies are each writing their own (bad) crawlers instead of using existing ones?
AI bots operators are financially incentivized to not be good citizens, they want as much data as possible as fast as possible and don't care who they piss off in the process. Plus for now at least they have effectively unlimited money to throw at bandwidth, storage, IP addresses, crawling with full-blown headless browsers, etc.
Re: AI bots are destroying Open Access
#6Why are "AI" bots generating so much fuss. Is it because there are so many of them? Is it because AI companies are each writing their own (bad) crawlers instead of using existing ones?
This is explained in the article. Tl;dr for whichever reason these AI bots behave nothing like the web crawlers of old. To quote TFA:
> The current generation of bots is mindless. They use as many connections as you have room for. If you add capacity, they just ramp up their requests. They use randomly generated user-agent strings. They come from large blocks of IP addresses. They get trapped in endless hallways. I observed one bot asking for 200,000 nofollow redirect links pointing at Onedrive, Google Drive and Dropbox. (which of course didn't work, but Onedrive decided to stop serving our Canadian human users). They use up server resources - one speaker at Code4lib described a bug where software they were running was using 32 bit integers for session identifiers, and it ran out!
Re: AI bots are destroying Open Access
#7The only way--the only way--to solve these issues is with web servers requiring that all clients authenticate with a credential that is provably tied to a real-world entity--person or corporate entity--so that legal recourse is available to the server owner when abuse occurs. The internet is no longer high-trust; we're running web servers the same way we'd run an honor-system store where people just come in and steal…
We're just ruining the last good part of internet
Re: AI bots are destroying Open Access
#8The only way--the only way--to solve these issues is with web servers requiring that all clients authenticate with a credential that is provably tied to a real-world entity--person or corporate entity--so that legal recourse is available to the server owner when abuse occurs. The internet is no longer high-trust; we're running web servers the same way we'd run an honor-system store where people just come in and steal…
$125 and I can start an LLC.
That's a real-world corporate entity. Recourse ends at the "limited liability" in LLC.
Make that LLC owned by another? Offshore ownership? Might take a few thousand bucks.
Re: AI bots are destroying Open Access
#9Sheesh, just use BitTorrent. That's what open access licensing is for! BitTorrent's tit-for-tat approach limits the harm selfish actors can do, only greatly rewarding those whose behavior benefits others, and has been shown to be very robust against active disruption attempts for decades now. Moreover, it also confers some resistance to falsification of the published record, to linkrot, and to publishing companies going bankrupt.
Sooner or later we need to take back the legitimate internet from surveillance capitalism. Capitalism is great (it shares many of BitTorrent's virtues, not coincidentally) but surveillance capitalism is not.
Re: AI bots are destroying Open Access
#10The only way--the only way--to solve these issues is with web servers requiring that all clients authenticate with a credential that is provably tied to a real-world entity--person or corporate entity--so that legal recourse is available to the server owner when abuse occurs. The internet is no longer high-trust; we're running web servers the same way we'd run an honor-system store where people just come in and steal…
$125 and I can start an LLC. That's a real-world corporate entity. Recourse ends at the "limited liability" in LLC. Make that LLC owned by another? Offshore ownership? Might take a few thousand bucks.
Cheaper than that, and the Secretary of State doesn't actually verify anything.