271–280 of 422 posts
Earlier quoted context omitted.
> Unfortunately whatever HN is using routinely blocks my login with "Sorry." I believe that's the HN application itself, not a WAF in front of it.
HN is surprisingly very very guilty of a whole lot of anti-user patterns and behaviour that other companies get regularly lamented. Poor accessibility, bad mobile support, no options to delete content beyond a narrow window.
bad mobile support
I feel like the solution is a better common crawl. As nice as it would be to block the frontier AI labs from getting access to information, we should reset the baseline of information accessibility so there's less marginal advantage on these labs. I worry a lot of the anti scraping rhetoric will just injure the open web and put somebody like cloudflare in charge.
What really confuses me is ... people always say, it's because companies are gathering data for AI training. Then why would they need to scrape the same page thousands of times per day? Edit: the article says millions of times per hour? (!?) The article is also astonished by this, and speculates it might be some kind of underground AI labs but... millions of them? Or does it only take one with too much money and a ba…
Grok actually shows a number of sources used for an answer. Once I asked it something simple and it apparently scanned 200 different websites. And it was just a short prompt. Now imagine millions of users asking for something multiple times a day.
Has no one noticed their miniflux instance failing to fetch feeds because of this?
Earlier quoted context omitted.
As someone dealing with similar on a large site, I'd love to see a private community to discuss some of these issues.
Disclaimer: this works for my very small number of personal services that I run. I have no idea how this would (or probably wouldn't) scale at all. Also, the methodology I describe below is based on what I'm able to do technically, which is pretty much limited to bash scripting. On my external-most device I have a firewall that logs addresses that attempt to connect to ports behind which there are no services, and th…
Earlier quoted context omitted.
Anubis is by far the least annoying throttler I encounter. Entirely agreed, just crank it up when you get a flood, I much prefer waiting a couple seconds to interacting with custom UI for tens of seconds. I'm so glad to see that (essentially) HashCash is coming back. Now we just need it for email, like it was originally designed for...
From my understanding this is also how cloudflare bot protection has worked for a long time, and then they look for entropy in user input to confirm the user is human. Also how recaptcha without images works.
10 years ago, apps had to explicitly state if they needed network access. And then the powers that be decided that really all apps need network access no matter what. And both ios and android make it hard to deny apps network access.
But really, this finally explains the hordes of really basic boring games that just advertise other boring games. Idle games and the like that really just want you to keep your phone unlocked and open. Millions of downloads on the app stores for entirely offline content (and ads) and no way to block the network access.
Earlier quoted context omitted.
On what grounds would it be illegal though? Things don't become illegal just because you don't like them. They may become illegal just because the president doesn't like them, but I don't think you're him, and in the absence of that, there has to be a majority of Congress and most of them want a reason.
Because they don't have the informed consent* of the owner of the device wich ends up running the code? * no, small print in a click-through agreement doesn't count.
Earlier quoted context omitted.
12,000 times less bad? https://news.ycombinator.com/item?id=48868910 >
Between me and you, I don't think Google would register that amongst the noise of just running the service anyway on their monitoring systems. You're right to point out that if you're trying to get the contents of dead objects the API is of no use though.
On dead/flagged items, there's some value. Whilst the title/URL context aren't available, just knowing what fraction of submissions and comments are moderated is interesting data, and it is possible to construct patterns against specific accounts.
I'm frequently encountering what appear to be banned accounts. Being able to trace those through the API to see where and when they were banned, or now much moderated activity they're generating, can be useful. I'm relying heavily on the "/replies?id=&by=" search endpoint (generally dang, tomhow, sctb, or pg as mod) currently to find out if there was a specific ban admonishment from a moderator. That's often but not always the case.
But it's not possible, say, to tell through the API what sites are banned. Looking at site history with "showdead" enabled can tell you that though, e.g.:
https://news.ycombinator.com/from?site=synthetica.cloud>
(From the New queue, one of several "dead" submissions not flagged, suggesting a site ban.)
Hypothesizing an undocumented "site" API endpoint ... doesn't seem to check out:
https://hacker-news.firebaseio.com/v0/site/synthetica.cloud?print=pretty
not found
And there's the "day" endpoint doesn't seem to work either, though it conspicuously does not report "not found", e.g.:
https://hacker-news.firebaseio.com/v0/day/2025-07-10?print=pretty
Earlier quoted context omitted.
If Google determines you're an undesirable user, doing the captcha is just an exercise to waste your time.
I certainly experienced this (the vicious try-again cycle) but curious if you have any sources for this?
I don't think you'll find an article by Google saying "yes, we sometimes completely block users while making it look like they're not blocked and wasting their time".