291–300 of 422 posts
Earlier quoted context omitted.
I kind of wish the recent Google monopoly court ruling had forced Google to open up their index to anyone, not just Perplexity/other big players.
That's really a huge issue right now (to some extent even before the AI hype) that almost everywhere google is effectively the only entity explicitly allowed to scrape.
Earlier quoted context omitted.
What really confuses me is ... people always say, it's because companies are gathering data for AI training. Then why would they need to scrape the same page thousands of times per day? Edit: the article says millions of times per hour? (!?) The article is also astonished by this, and speculates it might be some kind of underground AI labs but... millions of them? Or does it only take one with too much money and a ba…
750k items in their content management sysem. N independent labs crawling wanting to check that every day could easily give bursts of millions per hour Millions per hour is tens per second though; perhaps the fix is performance improvements
That'll be great until.. they rewrite the scrapers in Rust! Then we're really hosed!
Earlier quoted context omitted.
What really confuses me is ... people always say, it's because companies are gathering data for AI training. Then why would they need to scrape the same page thousands of times per day? Edit: the article says millions of times per hour? (!?) The article is also astonished by this, and speculates it might be some kind of underground AI labs but... millions of them? Or does it only take one with too much money and a ba…
It isn't. AI scraping has nothing to do with it, for the reasons you said. Someone wants the web to go offline, they are DDoSing the entire web, and it's working. For some reason we are tackling the symptom instead of finding out who that person is. Come on, it can't be that hard to subpoena Bright Data. The law enforcement system knows how to track down someone who's trying to be anonymous on the internet.
Which powerful entities have historically hated a free and open internet?
...all of them??
Earlier quoted context omitted.
Anubis is by far the least annoying throttler I encounter. Entirely agreed, just crank it up when you get a flood, I much prefer waiting a couple seconds to interacting with custom UI for tens of seconds. I'm so glad to see that (essentially) HashCash is coming back. Now we just need it for email, like it was originally designed for...
I looked this up and realised it’s the page I’d seen briefly on a range of websites lately. It’s not annoyed me at all. Not nearly as much as having to complete captchas with slow refreshing tiles.
Earlier quoted context omitted.
A fun fact about Google captchas: they've often decided whether you will succeed or fail the captcha before you do the captcha.
This seems nonsensical. Care to elaborate?
reason why is 1. Google and others really needed the training data, and 2. it probably helped justify the cost of providing the captcha service for free worldwide (old free tier was 1M/mo)
Earlier quoted context omitted.
Most CF / Recaptcha problems are users going "off the golden path", and not realizing that their config changes are at fault. If you're on a consumer router, using a mainstream stock browser with stock settings (maybe plus uBlock Origin), with your Google account logged in, it's very, very likely to just work. If you're part of the .01% of users with opinions about that sort of thing... you're not worth optimizing fo…
At least for me, CF is fine; recaptcha is the only one I really have problems with. I dont care what recaptcha wants to optimize for. I dont think that using a vpn is that a rare thing anyway. If others have figured out how to do it without requiring spending 30 seconds to solve a captcha, I dont see why websites still use recaptcha/captchas for that. And that it is "my fault" not being logged into google I was least…
Parent was just starting a fact of how our digital overlords determine the probability of your browser being a bot. Why take it personal?
Earlier quoted context omitted.
https://www.businessinsider.com/lowercase-typing-altman-dors... most shibboleths are subtle like that
I'm...not old enough to read a Business Insider article about being annoyed by lowercase letters.
I wrote about this recently as well.
Earlier quoted context omitted.
Ok that's good - if you get it again can you let us know at hn@ycombinator.com? We definitely don't want to exclude legit users.
LI've emailed there in May and June about different topics and gotten no reply to a request to confirm receipt. Is there a backup method for when your algorithm throws people's email away without even informing the sender with a delivery failure notification?
There are other reasons why we might not respond, but overload is the main one.
Edit: I only see one email in the archive related to your account. It was from Sept 2024 and we responded to it. Are you talking about a different account?
Earlier quoted context omitted.
This seems nonsensical. Care to elaborate?
The captcha itself (matching pictures to text) is mostly for ML training data. I think pass/fail is mostly based on heuristics like how you moved your mouse which could get analyzed before you complete the captcha. https://www.techradar.com/news/captcha-if-you-can-how-youve-... reason why is 1. Google and others really needed the training data, and 2. it probably helped justify the cost of providing the captcha servi…