Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ban me at the IP level if you don't like me

boston.conman.org

Post reply on HN
Print view View original

411–420 of 516 posts

Re: Ban me at the IP level if you don't like me

#411

Post by Dylan16807 »

throwawayffffas wrote: post #87
lxgr wrote: post #69

Earlier quoted context omitted.

This is based on personal experience. At least two did not let me unsubscribe from abroad in the past.

Not letting you unsubscribe and blocking your IP are very different things. There are some that do not provide services in most countries but Netflix, Disney, paramount are pretty much global operations. HBO and peacock might not be available in Europe but I am guessing they are in Canada.

> Not letting you unsubscribe and blocking your IP are very different things.

When you posted this, what did you envision in your head for how they were prevented from unsubscribing, based on location, but not via IP blocking? I'm really curious.

Re: Ban me at the IP level if you don't like me

#412

Post by inetknght »

If a US IP is abusing the internet, you can go through the courts. If a foreign country is... good luck.

So, are hackers and internet shittery coming from China? Block China's ASNs. Too bad ISPs won't do that, so you have to do it yourself. Keep it blocked until China enforces computer fraud and abuse.

Re: Ban me at the IP level if you don't like me

#413

Post by BoredPositron »

zelphirkalt wrote: post #404
BoredPositron wrote: post #211

Earlier quoted context omitted.

The regulatory burden of conducting business with countries like Russia or China is a critical factor that offhand comments like yours consistently overlook.

It is funny how people immediately jump to conclusions, while I was merely pointing out an circular argument. People immediately think I am jumping to "aid one side". Shows much more about them than me, actually.

Nobody is jumping to any conclusions except you with this very comment. ;)

Re: Ban me at the IP level if you don't like me

#414

Post by socalgal2 »

xp84 wrote: post #398
0x457 wrote: post #382

Earlier quoted context omitted.

So weird to scrape wikipedia when you can just download db dumb from them.

Really makes you think about the calibre of minds being applied to buzzy problem spaces these days, doesn't it?

do we know they didn't download the DB? Maybe the new traffic is the LLM reading the site? (not the training)

I don't know that LLMs read sites. I only know when I use one it tells me it's checking site X, Y, Z, thinking about the results, checking sites A, B, C etc.... I assumed it was actually reading the site on my behalf and not just referring to its internal training knowledge.

Like how people are training LLMs, and how often does each one scrap? From the outside, it feels like the big ones (ChatGPT, Gemini, Claude, etc..) scrape only a few times a year at most.

Re: Ban me at the IP level if you don't like me

#417

Post by TZubiri »

socalgal2 wrote: post #415

Aren't many apartment buildings all coming from just a few IP addresses? https://en.wikipedia.org/wiki/Carrier-grade_NAT

Yes, and this makes ip banning have false positives.

But ultimately it's worth it, you are responsible for your neighbours.

Re: Ban me at the IP level if you don't like me

#418

Post by TZubiri »

threeducks wrote: post #203
bob1029 wrote: post #132

I think a lot of really smart people are letting themselves get taken for a ride by the web scraping thing. Unless the bot activity is legitimately hammering your site and causing issues (not saying this isn't happening in some cases), then this mostly amounts to an ideological game of capture the flag. The difference being that you'll never find their flag. The only thing you win by playing is lost time. The best wa…

> The best way to mitigate the load from diffuse, unidentifiable, grey area participants is to have a fast and well engineered web product. I wonder what all those people are doing that their server can't handle the traffic. Wouldn't a simple IP-based rate limit be sufficient? I only pay $1 per month for my VPS, and even that piece of trash can handle 1000s of requests per second.

I'd wager a bet that there's a package.json lying somewhere that holds a lot of dependencies

Re: Ban me at the IP level if you don't like me

#419

Post by Bender »

ikiris wrote: post #403
Bender wrote: post #269

Earlier quoted context omitted.

(malicious) Bot writers have exactly zero concern for robots.txt. Most bots are malicious. Most bots don't set most of the TCP/IP flags. Their only concern is speed. I block about 99% of port scanning bots by simply dropping any TCP SYN packet that is missing MSS or uses a strange value. The most popular port scanning tool is masscan which does not set MSS and some of the malicious user-agents also set some odd MSS v…

You do realize vpns and older connectivity exists that needs values lower than 1280 right?

You do realize vpns and older connectivity exists that needs values lower than 1280 right?

Of course. Nifty thing about open source means I can configure a system to allow or disallow anything. Each server operator can monitor their legit users traffic and find what they need to allow and dump the rest. Corporate VPN's will be using known values. "Free" VPN's can vary wildly but one need not support them if they choose not to. On some systems I only allow and MSS of 1460 and I also block TCP SYN packets with a TTL greater than 64 but that matches my user-base.

Re: Ban me at the IP level if you don't like me

#420

Post by jandrese »

8organicbits wrote: post #235

I've been working on a web crawler and have been trying to make it as friendly as possible. Strictly checking robots.txt, crawling slowly, clear identification in the User Agent string, single IP source address. But I've noticed some anti-bot tricks getting applied to the robot.txt file itself. The latest was a slow loris approach where it takes forever for robots.txt to download. I accidentally treated this as a 404…

> The latest was a slow loris approach where it takes forever for robots.txt to download

Applying penalties that exclusively hurt people who are trying to be respectful seems counterproductive.

Post reply on HN