LWN is currently under the heaviest scraper attack seen yet
social.kernel.org
LWN is currently under the heaviest scraper attack seen yet
1–10 of 134 posts
Re: LWN is currently under the heaviest scraper attack seen yet
#2Is it still ongoing? The thread appears to be over 24 hours old and as a quick test I had no issue loading the main page (which is as snappy and responsive as expected from a low-bandwidth site like LWN).
Re: LWN is currently under the heaviest scraper attack seen yet
#3the perverse incentive is if you ddos the website such that it shuts down, no other "AI" parasites can get the valuable data
big tech incentivised to ddos... what a world they've built
Re: LWN is currently under the heaviest scraper attack seen yet
#4the perverse incentive is if you ddos the website such that it shuts down, no other "AI" parasites can get the valuable data big tech incentivised to ddos... what a world they've built
This sounds like a conspiracy theory.
Re: LWN is currently under the heaviest scraper attack seen yet
#5the perverse incentive is if you ddos the website such that it shuts down, no other "AI" parasites can get the valuable data big tech incentivised to ddos... what a world they've built
Its called pulling up the ladder behind you, or building a moat!
Re: LWN is currently under the heaviest scraper attack seen yet
#6I've had luck blocking scrapers by overwriting JavaScript methods
" a.getElementsByTagName = function (...args) {//Clear page content}"
One can also hide components inside Shadow DOM to make it harder to scrape.
However, these methods will interfere with automated testing tools such as Playwright and Selenium. Also, search engine indexing is likely to be affected.
Re: LWN is currently under the heaviest scraper attack seen yet
#7So which is it? DDOS attack or "AI" scrapers?
Re: LWN is currently under the heaviest scraper attack seen yet
#8the perverse incentive is if you ddos the website such that it shuts down, no other "AI" parasites can get the valuable data big tech incentivised to ddos... what a world they've built
Umm... what data? That's a very old newsletter-like site. Everything that's public on it has been long scraped and parsed by whoever needed it. There's 0 valuable data there for "parasites" to parasite off of.
I also don't get the comments on the linked social site. IIUC the users posting there are somehow involved with kernel work, right? So they should know a thing or two about technical stuff? How / why are they so convinced that the big bad AI baddies are scraping them, and not some miss-configured thing that someone or another built? Is this their first time? Again, there's nothing there that hasn't been indexed dozens of times already. And... sorry to say it, but neither newsletters nor the 1-3 comments on each article are exactly "prime data" for any kind of training.
These people have gone full tinfoil hat and spewing hate isn't doing them any favours.
Re: LWN is currently under the heaviest scraper attack seen yet
#9the perverse incentive is if you ddos the website such that it shuts down, no other "AI" parasites can get the valuable data big tech incentivised to ddos... what a world they've built
Umm... what data? That's a very old newsletter-like site. Everything that's public on it has been long scraped and parsed by whoever needed it. There's 0 valuable data there for "parasites" to parasite off of. I also don't get the comments on the linked social site. IIUC the users posting there are somehow involved with kernel work, right? So they should know a thing or two about technical stuff? How / why are they s…
I don’t think they were talking about LWN specifically but just in general.
Re: LWN is currently under the heaviest scraper attack seen yet
#10the perverse incentive is if you ddos the website such that it shuts down, no other "AI" parasites can get the valuable data big tech incentivised to ddos... what a world they've built
This sounds like a conspiracy theory.
I don’t think they’re saying that’s actually happening here, just that it could happen and is accidentally incentivized.