I ran into this, or something similar recently when our main connection went down (solar powered) and we switched to Starlink. Due to Starlink NAT issues I had tunneled our traffic to to a box colocated in a data center. This broke a number of web sites in weird ways. Became so annoying that I ended up bringing up a tunnel to our office in town to get back to the regular IP we used. Weird problems went away.
Tell HN: Impassable Cloudflare challenges are ruining my browsing experience
21–30 of 312 posts
Re: Tell HN: Impassable Cloudflare challenges are ruining my browsing experience
#22Earlier quoted context omitted.
I honestly don't see what's so hard about a bot simulating "the norm" within the margin of error. This cat-and-mouse game is just like a GAN, the end result is indistinguishable even by a bot.
Bot authors are lazy and won't until they have to.. once you do, you can then pretend they aren't bots and include them in the engagement numbers you feed prospective shareholders.
Agreed. From my past experiences though, a very good chunk of them will give up once there is a resistance. Basically, you want your bot protection to just be a little better than your competitor. Then the bot author will target them instead, because of the path of least resistance.
Re: Tell HN: Impassable Cloudflare challenges are ruining my browsing experience
#23> Cloudflare challenges have made large portions of the web unusable for me.
I guess the best web experience is when one filters Cloudfare, Google and Microsoft at the firewall.
Re: Tell HN: Impassable Cloudflare challenges are ruining my browsing experience
#24I deal with this fairly commonly, presumably because I use linux, and we all know only botnets use linux. Occasionally with cloudflare I'll just get summary rejection and supposed blocking of my IP, but either it's summary rejection or a pass without challenge.
Recently I had to deal with this for alibaba just to look at something, which I usually just use torbrowser with, and finally gave up as I couldn't pass the challenge. I suppose I shouldn't be surprised at that though, they trust me as much as I trust them.
The worst is usually adobe and cookielaw with all their related tracking crap, where I can't even get the captcha to render as it's so many layers buried in scripting I can't enable enough sites between ublock, noscript, privacy badger, and firefox strict modes. I treat adobe like malware, but unfortunately things like albertsons.com for groceries and other mega companies love to use it, and their sites literally do not work without allowing their heavy scripting/tracking.
There are other usually smaller captcha players that I haven't been human enough to pass with, I forget the names of the stupid to shame, but a few when I see them I recognize to just close the window and forget about whatever it was I was looking for there (like twitter/x).
Hooray commerce!
Re: Tell HN: Impassable Cloudflare challenges are ruining my browsing experience
#25Earlier quoted context omitted.
While you hit the nail on the head, I am still surprised that so many tools targeted at people like me (web hosting, developer tools, etc.) are protected that way.
Its not only about protection, most web developers would use Cloudflare since its a free CDN and would increase the app load time considerably.
increase -> decrease
Re: Tell HN: Impassable Cloudflare challenges are ruining my browsing experience
#26Earlier quoted context omitted.
While you hit the nail on the head, I am still surprised that so many tools targeted at people like me (web hosting, developer tools, etc.) are protected that way.
They are not targeting people like you. Bots are the target. If you look like a bot, how are they going to distinguish?
> If you look like a bot, how are they going to distinguish?
Some non-existant system of attesting that I'm person X (possibly through an e-ID card) who has issued a client certificate Y (cert chain, using my e-ID cert to sign) to be used with my device Z (presumably with a device fingerprint or IP range attached to the cert). Of course, this would mean no privacy, but that's not that different from being signed in through Google as an identity provider, we'd just shift the mechanism to be universal (like client certs already are). One of the options that would take more coordination than will probably happen (though very similar to some e-signature solutions in EU, which we already use) but I could see using something like that for a variety of professional/service sites, since signing in with the e-ID card directly is already a thing on some sites here (government sites, banking sites, utilities sites).
Re: Tell HN: Impassable Cloudflare challenges are ruining my browsing experience
#27Earlier quoted context omitted.
While you hit the nail on the head, I am still surprised that so many tools targeted at people like me (web hosting, developer tools, etc.) are protected that way.
Its not only about protection, most web developers would use Cloudflare since its a free CDN and would increase the app load time considerably.
You can separately configure (to a large degree) the caching vs protection features, though.
Re: Tell HN: Impassable Cloudflare challenges are ruining my browsing experience
#28You're collateral damage in the web's war against bots :( Unfortunately, I think the Cloudflare challenges are designed to filter out users similar to your profile... once you stray far enough from the norm, it just looks like a bot / suspicious traffic to them. Statistically there's not enough users like you (privacy-conscious Linux users on nonstandard browsers) for them to really care enough to do anything about i…
The sad part is that it's trivial to get around CF's bot protection if you're writing a bot (just use curl-impersonate and buy residential IPs), but it's pretty much impossible to bypass as a human if their magical black box doesn't like your browser and/or IP address.
How does it get around captchas?
Re: Tell HN: Impassable Cloudflare challenges are ruining my browsing experience
#29Earlier quoted context omitted.
The sad part is that it's trivial to get around CF's bot protection if you're writing a bot (just use curl-impersonate and buy residential IPs), but it's pretty much impossible to bypass as a human if their magical black box doesn't like your browser and/or IP address.
How does it get around captchas?
You pay contract workers in a third world country a tiny amount of money per day, to spend all day clicking boxes.
Re: Tell HN: Impassable Cloudflare challenges are ruining my browsing experience
#30You're collateral damage in the web's war against bots :( Unfortunately, I think the Cloudflare challenges are designed to filter out users similar to your profile... once you stray far enough from the norm, it just looks like a bot / suspicious traffic to them. Statistically there's not enough users like you (privacy-conscious Linux users on nonstandard browsers) for them to really care enough to do anything about i…
I honestly don't see what's so hard about a bot simulating "the norm" within the margin of error. This cat-and-mouse game is just like a GAN, the end result is indistinguishable even by a bot.
It depends on the defences. It starts trivial - just make a http request. Then there's http version, user agent header, other headers, header ordering, cookies, TLS ciphers, session resolution, timing, behaviour for page resources, ... and so many other things. It takes time, even if you order headless chrome.