Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tell HN: Impassable Cloudflare challenges are ruining my browsing experience

news.ycombinator.com

Post reply on HN
Print view View original

11–20 of 312 posts

Re: Tell HN: Impassable Cloudflare challenges are ruining my browsing experience

#11

Post by EGreg »

solardev wrote: post #2

You're collateral damage in the web's war against bots :( Unfortunately, I think the Cloudflare challenges are designed to filter out users similar to your profile... once you stray far enough from the norm, it just looks like a bot / suspicious traffic to them. Statistically there's not enough users like you (privacy-conscious Linux users on nonstandard browsers) for them to really care enough to do anything about i…

I honestly don't see what's so hard about a bot simulating "the norm" within the margin of error. This cat-and-mouse game is just like a GAN, the end result is indistinguishable even by a bot.

Re: Tell HN: Impassable Cloudflare challenges are ruining my browsing experience

#12

Post by avnfish »

I'm experiencing the same issue which is definitely exacerbated by straying from a 'default' configuration e.g. using a custom browser screen reader, browsing from Brazil, using a VPN, using Firefox. I think eventually I'll be completely locked out of the 'mainstream' web

Re: Tell HN: Impassable Cloudflare challenges are ruining my browsing experience

#13

Post by Animats »

> The "unsubscribe" button in Indeed's job notification emails leads me to an impassable Cloudflare challenge.

That's a CAN-SPAM act violation.

FTC: "Tell recipients how to opt out of receiving future marketing email from you. Your message must include a clear and conspicuous explanation of how the recipient can opt out of getting marketing email from you in the future. Craft the notice in a way that’s easy for an ordinary person to recognize, read, and understand. Creative use of type size, color, and location can improve clarity. Give a return email address or another easy Internet-based way to allow people to communicate their choice to you. You may create a menu to allow a recipient to opt out of certain types of messages, but you must include the option to stop all marketing messages from you. Make sure your spam filter doesn’t block these opt-out requests."[1]

Experian was recently fined for making it hard to opt out of their marketing emails.

The actual regulation text:

§ 316.5 Prohibition on charging a fee or imposing other requirements on recipients who wish to opt out.

Neither a sender nor any person acting on behalf of a sender may require that any recipient pay any fee, provide any information other than the recipient's electronic mail address and opt-out preferences, or take any other steps except sending a reply electronic mail message or visiting a single Internet Web page, in order to:

(a) Use a return electronic mail address or other Internet-based mechanism, required by 15 U.S.C. 7704(a)(3), to submit a request not to receive future commercial electronic mail messages from a sender; or

(b) Have such a request honored as required by 15 U.S.C. 7704(a)(3)(B) and (a)(4).

That seems to cover it. File a CAN-SPAM act complaint (spam@uce.gov). Send a copy to the legal department of the sender.

[1] https://www.ftc.gov/business-guidance/resources/can-spam-act...

Re: Tell HN: Impassable Cloudflare challenges are ruining my browsing experience

#14

Post by gjsman-1000 »

blakeashleyjr wrote: post #8

What I don't understand is why you have to protect areas that require login so harshly? If I can log in, especially with 2-factor, you can safely assume I am not a bot, or you have a larger problem. If I have entered bad credentials 5+ times, okay, you can start backing me off or challenging me. What am I missing? Fail2ban has been around a long time.

40% of the internet’s traffic now is bots, with about half of those being malicious. Fail2ban is decent for a very small DDoS, but useless for one with any substance, and also useless against bots scraping data or probing for weaknesses.

Also remember, especially on AWS, bandwidth is expensive. A CDN cache + blocking bots = big savings.

Re: Tell HN: Impassable Cloudflare challenges are ruining my browsing experience

#15

Post by nullc »

EGreg wrote: post #11
solardev wrote: post #2

You're collateral damage in the web's war against bots :( Unfortunately, I think the Cloudflare challenges are designed to filter out users similar to your profile... once you stray far enough from the norm, it just looks like a bot / suspicious traffic to them. Statistically there's not enough users like you (privacy-conscious Linux users on nonstandard browsers) for them to really care enough to do anything about i…

I honestly don't see what's so hard about a bot simulating "the norm" within the margin of error. This cat-and-mouse game is just like a GAN, the end result is indistinguishable even by a bot.

Bot authors are lazy and won't until they have to.. once you do, you can then pretend they aren't bots and include them in the engagement numbers you feed prospective shareholders.

Re: Tell HN: Impassable Cloudflare challenges are ruining my browsing experience

#16

Post by mg »

If it is triggered by the customizations you did in Firefox, then running a fresh Firefox in a container might help:

    docker run -it --rm -e DISPLAY --net=host -v $XAUTHORITY:/root/.Xauthority -v /tmp/.X11-unix:/tmp/.X11-unix debian:12-slim
Then inside the container, run:

    apt update
    apt install firefox-esr
    firefox

Re: Tell HN: Impassable Cloudflare challenges are ruining my browsing experience

#17

Post by shiomiru »

solardev wrote: post #2

You're collateral damage in the web's war against bots :( Unfortunately, I think the Cloudflare challenges are designed to filter out users similar to your profile... once you stray far enough from the norm, it just looks like a bot / suspicious traffic to them. Statistically there's not enough users like you (privacy-conscious Linux users on nonstandard browsers) for them to really care enough to do anything about i…

The sad part is that it's trivial to get around CF's bot protection if you're writing a bot (just use curl-impersonate and buy residential IPs), but it's pretty much impossible to bypass as a human if their magical black box doesn't like your browser and/or IP address.

Re: Tell HN: Impassable Cloudflare challenges are ruining my browsing experience

#18

Post by luckylion »

blakeashleyjr wrote: post #3
solardev wrote: post #2

You're collateral damage in the web's war against bots :( Unfortunately, I think the Cloudflare challenges are designed to filter out users similar to your profile... once you stray far enough from the norm, it just looks like a bot / suspicious traffic to them. Statistically there's not enough users like you (privacy-conscious Linux users on nonstandard browsers) for them to really care enough to do anything about i…

While you hit the nail on the head, I am still surprised that so many tools targeted at people like me (web hosting, developer tools, etc.) are protected that way.

I'm convinced that's mostly incompetence on the side of the companies that implement that protection.

"We have a problem with bots" - "Just create a firewall rule, whatever"

Re: Tell HN: Impassable Cloudflare challenges are ruining my browsing experience

#19

Post by zufallsheld »

> - The "unsubscribe" button in Indeed's job notification emails leads me to an impassable Cloudflare challenge.

Maybe indeed could be held liable here? From the can spam act (if you're from the US):

> You can’t charge a fee, require the recipient to give you any personally identifying information beyond an email address, or make the recipient take any step other than sending a reply email or visiting a single page on an Internet website as a condition for honoring an opt-out request.

https://www.ftc.gov/business-guidance/resources/can-spam-act...

Re: Tell HN: Impassable Cloudflare challenges are ruining my browsing experience

#20

Post by kachapopopow »

I have experience bypassing these.

The primary cause of this is most likely any kind of 'optimizations' you have in your browser (or missing fingerprints).

If you want to 'bypass' these I recommend removing any use of Proxy[1] (via extensions). You should also look into disabling any kind of forced backgrounding. Make sure service workers are working.

1: They catch Proxy usage by using exceptions and analyzing the stacktrace. I assume you know what a javascript proxy is, but incase you don't: It's something that allows you to override any kind of object function such as navigator.hardwareConcurrecy.

Post reply on HN