Live data from Hacker News

Gentoo bugzilla closed due AI bot scraper overload

social.treehouse.systems

31–40 of 114 posts

Re: Gentoo bugzilla closed due AI bot scraper overload

#31
post #21

Earlier quoted context omitted.

How many pages does the average software developer visit everyday? 1.000? Price at 0.001 per load and it’ll be completely impractical for crawlers but super cheap for the average connected human on earth.

That would gate the internet to whole lower income countries

Aren't many of those countries basically being de-facto blocked by Cloudflare filtering out spam already anyways?

Genuine question, I'm not up to date on how Cloudflare operates right now

Re: Gentoo bugzilla closed due AI bot scraper overload

#32

It is time for micropayments integrated in the browser. Pay 5 cents to access each bug report. Not fun, it shouldn't be like this, but better than not having a bugtracker at all.

and do you happen to know a zero friction payment system that works internationally like the Internet itself does? because I promise you that unless you already have a captive audience, even a minuscule amount of friction to access your service will cost you an overwhelming percentage of organic human visitors.

Re: Gentoo bugzilla closed due AI bot scraper overload

#34
post #32

It is time for micropayments integrated in the browser. Pay 5 cents to access each bug report. Not fun, it shouldn't be like this, but better than not having a bugtracker at all.

and do you happen to know a zero friction payment system that works internationally like the Internet itself does? because I promise you that unless you already have a captive audience, even a minuscule amount of friction to access your service will cost you an overwhelming percentage of organic human visitors.

Something like Chaum's blind-signature based ecash, or GNU Taler would be ideal in terms of efficiency, but it's still centralized in distribution. Freenet currently uses something like this.

Another option I was thinking of would be a pretty inflationary (or demurrage) cryptocurrency in which you have some sort of RandomX or other CPU-bound PoW. A web server could act as a mining pool and use mining shares interchangibly with micropayments.

You could do this mining-share method with Monero right now, it's just that you have higher transaction size in Monero and no real analogue to Bitcoin's LN-based microtransactions. Also you would want the cryptocurrency to be more inflationary (or demurrage-based) to promote usage.

Monero's FCMP++ lays some groundwork for payment channels, but it still lacks the nessisary timelocks. Also there was DLSAG which could have enabled payment channels I think, but it's no longer relevant. I also insist that you would need to change the tokenomics to favor greater inflation (maybe you could make coinbase scale linearly with hashrate?), otherwise the miner reward would be economicially insufficient.

Re: Gentoo bugzilla closed due AI bot scraper overload

#35
post #30

What are they scraping the gentoo bugzilla for? I'm confused. Unless you're actively using Gentoo why would this be a resource? Very confusing. Also you'd think we'd have LLM BitTorrent by now, where if they want to scrape something we get a DHT hash for the content and share it with one another, rather than melt servers with the millionth request of the day.

Lots of build failure detailed investigations and gcc/kernel expertise in debugging misbehaving or outright ICEs.

That's it, I guess?

Re: Gentoo bugzilla closed due AI bot scraper overload

#36
post #28

Earlier quoted context omitted.

0.05 cents even. The beautiful thing is, minuscule amounts of micropayments are already enough to fix the incentives

Not sure it works, isn't it already like that if you have the cursed javascript-only pages? It's basically proof of work to access their content (and if you are scrapping en masse you likely also need some LLM getting involved)

I suppose the cost of hardware, network usage, and electricity is still under 0.05? I mean, we know the unit economics for LLMs don't make a ton of sense, except if enterprise customers really are what keeps the lights on.

Re: Gentoo bugzilla closed due AI bot scraper overload

#37
post #32

It is time for micropayments integrated in the browser. Pay 5 cents to access each bug report. Not fun, it shouldn't be like this, but better than not having a bugtracker at all.

and do you happen to know a zero friction payment system that works internationally like the Internet itself does? because I promise you that unless you already have a captive audience, even a minuscule amount of friction to access your service will cost you an overwhelming percentage of organic human visitors.

Friction-free is a big ask. Brave tried something a while back with their Brave Payments, but of course no one trusts them and it’s not going anywhere.

Re: Gentoo bugzilla closed due AI bot scraper overload

#39
post #28

Earlier quoted context omitted.

0.05 cents even. The beautiful thing is, minuscule amounts of micropayments are already enough to fix the incentives

Not sure it works, isn't it already like that if you have the cursed javascript-only pages? It's basically proof of work to access their content (and if you are scrapping en masse you likely also need some LLM getting involved)

With micropayments, the server owner makes 1 million requests times 0.05 cents = ~$500

The status quo with those js PoW pages doesn't really benefit the server owner at all, it's wasted energy.

I mean proof of work is always wasted energy, but I figure it's better to kill two birds with one stone.

CoinHive was one example of this. (I think this is a correct link? https://github.com/cazala/coin-hive). Although I think ideally you would want to have some sort of browser plugin or app that runs on bare metal instead of a proof-of-work in the browser, because RandomX is designed such that it's slow when implemented in JS (https://github.com/tevador/RandomX/blob/master/doc/design.md)

Re: Gentoo bugzilla closed due AI bot scraper overload

#40
post #28

Earlier quoted context omitted.

0.05 cents even. The beautiful thing is, minuscule amounts of micropayments are already enough to fix the incentives

Not sure it works, isn't it already like that if you have the cursed javascript-only pages? It's basically proof of work to access their content (and if you are scrapping en masse you likely also need some LLM getting involved)

Problem is I'd rather have the 0.05cents go to however runs the website (the bugtracker in this case) than go to the electricity company.
Post reply on HN