71–80 of 218 posts
Basically bots need to be (somehow) paying for the traffic they create, or prevented from creating it, or told to go away and then fined if they violate the request. No idea how to do these or even at what level in the stack they should happen, but they need to happen eventually somehow.
A couple years ago, I was running a website which allowed the public to view all the US government handouts to small businesses during the COVID-19 pandemic. It also tracked all the fraudulent loans being prosecuted by the DOJ, and allowed anyone to run structured queries over the public dataset. There was a "donate" button which took in ~$2k in donations over the lifetime of the site, and you could download the enti…
Building on AWS is a financial time bomb.
Earlier quoted context omitted.
I open source as much software as I can because I want the models to train on it and get better at it!
I feel the same about my old blog articles. I used to rank pretty well trashing crappy credit cards and encouraging people to switch to better options, then Google decided that 10 results for the card issuers website was better. Same shit when I manually wrote proto-gethuman posts on calling telecoms/banks/etc (and also pushed visitors to try an Indy ISP or credit unions), then Google felt it was better to drive user…
(I never publish my recommendations because they seem to complicated for people. Like using 30 GB plan/$10/monthly from T-Mobile for data and then using Tello for $10 plan for voice/text. This would require a 2 esim/sim card phone. I am currently using a Moto G Power 2024 phone from eBay $90/new, which was better than the $200 slightly used Pixel 6a from Swappa. Most people would just save the hassle, get a Galaxy phone with a phone contract.)
It feels like there is fundamentally missing infrastructure here that is needed to make these problems go away. Basically bots need to be (somehow) paying for the traffic they create, or prevented from creating it, or told to go away and then fined if they violate the request. No idea how to do these or even at what level in the stack they should happen, but they need to happen eventually somehow.
Disclosure: I am a shareholder and would love for them to solve the AI bot problem and the ad problem like this.
Earlier quoted context omitted.
The crawlers would have still just hammered their site though, right?
I think the idea is that they could store the data in BigQuery, and point users of the site there.
"One Reddit theory even suggested it was a deliberate rug pull designed to cripple the free site to push people towards paid products." I know this isn't really the point of the article, but I've been thinking about this a lot. I wonder if we're going to see more resources go this way. I used to publish little doo-dads as opensource software. Not because it was something that was legitimately ground breaking or anyth…
> But now I'm really reluctant to give more stuff to the free web. I feel the same way too. But guess what, all the code you did not publish gets into the training corpus anyway (when you gave Codex or whatever full read access to your filesystem).
This article, and possibly the owners of the site, seem to muddle together several problems: 1. bot traffic causing infrastructure cost 2. scraping circumventing paying for licenses 3. the risk of hacking
Webscraping is among the more benign things that gambling-on-anything can drive. And even that has a negative impact, as seen here.
I have no idea why those sites are legal.
*Edit - CO2 creation
It feels like there is fundamentally missing infrastructure here that is needed to make these problems go away. Basically bots need to be (somehow) paying for the traffic they create, or prevented from creating it, or told to go away and then fined if they violate the request. No idea how to do these or even at what level in the stack they should happen, but they need to happen eventually somehow.