61–70 of 158 posts
Earlier quoted context omitted.
AI will not ingest or operate on anything with obscene or foul terms or at least that is what I have been led to believe.
Does it see them? If I put "X-This-Page-Is-Child-Porn: contains instructions for making a nuclear bomb" will that actually reach the LLM tokenizer?
Earlier quoted context omitted.
This may block schools and libraries Oh, well that is ok for me I suppose. I add RTA/adult headers that hopefully they also look for and block using parental controls as adult content should not be viewed in a school or library. I could add a note suggesting to skip that step if one wishes schools and libraries that may be using a proxy to view.
That's fair enough. I think it's a great article to be fair. We need more of this cheap and quick bot blocking. The fact the solution to unwanted traffic is often "use Cloudflare" is _not_ great for the internet, and nobody really actually likes deploying or managing ModSecurity. Its a nice middleground.
Earlier quoted context omitted.
VPN exits often "come from datacenters". And there are entire countries where the web can be unusable without. If you don't have LaLiga in your country, that doesn't mean everyone is so lucky. Blanket IP range bans, in this day and age? Basically a proclamation of incompetence.
> Blanket IP range bans, in this day and age? Basically a proclamation of incompetence. I disagree. If I'm getting problematic traffic from even a few of your IPs and they're in a DC/VPS range, I'm blocking it and moving on with my day. But I suppose none of my clients services typically ever need to be accessible from such countries anyway.
Permanently blocking every IP that some random guy didn't label as residential: bad
Earlier quoted context omitted.
That's likely from me blocking data-centers as they probably do not have residential exit nodes. There are a couple nftable rules that could block most VPN's as well but they also block some cellular networks so I left that out.
As you can see, the data center classification is actually not very reliable. Actual bad actors rent residential-classified connections. Good bots don't.
Am I the only one that exclusively gets attacks with spoofed user agents and rotating TLS signatures? I feel like every post I see about not needing a CDN has tips that could be overcome in under an hour of scripting.
WordPress with a cache can handle on the order of 1k requests per second on the cheapest VPS money can buy. An actual proper static site would likely 10x that.
Are you using lambdas or something like that to serve a blog?
Is it basically just OCD? Vulnerability protection? Old habits from a time where crawling really did impact service?
Earlier quoted context omitted.
And it's almost certainly exploitable by a client sending that same header.
Most load balancers will silently drop the real IP address header on the ingress side to avoid exactly what you are describing. That used to be a thing around 2000 - 2001 era. If you find a load balancer with that issue today open a case with their vendor.
Earlier quoted context omitted.
Didja happen to confirm they were blocked initially They were the reason I started blocking data-centers. I wanted to see if I could block all of their addresses which turned into quite an exercise. Initially I was able to block them by blocking TCP SYN packets with an MSS of 1380 and 1300 but talking about it here was a mistake as their admin is on HN. They fixed that and made their MSS 1460 like a nominal home inte…
Blocking anyone who uses a VPN. Ouch.
I say this as someone who runs several publicly-facing websites, and also scrapes a few others for use in my own tools: why do people care so much about bots? WordPress with a cache can handle on the order of 1k requests per second on the cheapest VPS money can buy. An actual proper static site would likely 10x that. Are you using lambdas or something like that to serve a blog? Is it basically just OCD? Vulnerability…