161–169 of 169 posts
Earlier quoted context omitted.
Are you sure? Have you tried, or did Cloudflare just tell you that?
I wouldn't need a cache if my $6 server could handle 1M hits a day.
Earlier quoted context omitted.
A CDN doesn't necessarily have to perform a MitM. We really need more nuanced terminology to distinguish the various approaches.
Ideally yes, the TLS termination does not need to happen for caching purposes. Challenge is that in practice every business wants to be sticky and try to provide more functionalities which do require TLS termination. Most people either trust CDN's or they do not understand MitM so it does not concerns them. Plus they are getting certificate management and DDOS prevention capabilities.
Earlier quoted context omitted.
I wouldn't need a cache if my $6 server could handle 1M hits a day.
1M hits a day is 11 and a bit hits per second. Your $6 server should be able to handle that. If it can't, then de-pessimize your code until it can.
Earlier quoted context omitted.
Please don't use Anubis, it makes visiting websites very difficult (often multiple minutes wait times) on old hardware and low-end smartphones.
As things stand right now, the current web may not be possible to maintain for old and low-end phones given the costs imposed by AI training crawlers. Please propose an alternative to both Cloudflare and Anubis, that shields websites against inhuman traffic without frequent operator intervention (or otherwise negates the capacity costs they pay for AI crawling) and is compatible with low-end smartphones. Certainly, I…
Earlier quoted context omitted.
Please don't use Anubis, it makes visiting websites very difficult (often multiple minutes wait times) on old hardware and low-end smartphones.
what kind of hardware would take minutes to complete the checks?
Earlier quoted context omitted.
The irony here is that the people blocking all other crawlers are the ones shoring up their monopoly. If you can't block Googlebot because you need the search traffic but you block everybody else so that nobody other than Google can index your site, how do you expect to ever get any search traffic that isn't from Google?
Search traffic is nose diving due to LLM use. The fundamental calculus with google is you install GA and it helps your SEO has changed and google is riding out what will eventually wither as people start to reevaluate the trade off.
Earlier quoted context omitted.
If a site cannot handle traffic from the real Googlebot that is a serious issue with the site itself since it's actually pretty conservative Also I should note there are lots of fake Googlebots...
It is mostly, but it doesn't take a lot of googling to find sites getting ridiculous amounts of traffic from googlebot on google IPs. Its one of the most common complaints about google's search indexing
Earlier quoted context omitted.
> not take down systems "Block on pages with ads" is probably about preventing the AI crawlers from clicking on the ads which maybe considered cheating by the ad company. If you want to prevent "bot attacks", maybe the "Block" option will do the trick. But of course, to do all that you need to put some trust on Cloudflare, because they're the one identifying the bots from normal users. For me, as someone who's hostin…
> If you want to prevent "bot attacks", maybe the "Block" option will do the trick. This morning, for the first time, I clicked on a link posted on HN and was told in no uncertain terms that I am a bot and will not be allowed to view this page. By Cloudflare. Your strategy is fine and similar to one of the checks in go-away. The point is that the unknown global DDoS adversary is using a very simple scraper, which doe…
If you want something more powerful, there's also Cloudflare workers, which you can program to do whatever you want. But that one may cost you to run.
Also in my case, the scraper IPs are all over the globe, and almost each access is from a different one, so simple IP blocking don't work unless I block Vietnam, Brazil, Bangladesh, Argentina and the US etc (https://i.imgur.com/eWI5meM.png). But then, the scrapers might just go to another country next month.
Since I deployed those rules, the access from scrapers has almost stopped, while normal search engine crawls are almost unaffected. So it do still work for me, even if it's very rudimentary. But hey, security though obscurity it alone is not enough, but it sometimes do delay the attacks.
