Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An update on residential proxies and the scraper situation

lwn.net

Post reply on HN
Print view View original

71–80 of 422 posts

Re: An update on residential proxies and the scraper situation

#71

Post by dang »

nekusar wrote: post #21
dang wrote: post #17

Earlier quoted context omitted.

I've banned this account because we don't allow single-purpose accounts on HN, and your account has been doing that for quite some time now. We ban such accounts regardless of what the single purpose happens to be. Pre-existing agendas are not what HN is for and destroy the curious conversation that it is supposed to be for. https://news.ycombinator.com/newsguidelines.html Edit: If you don't want to be banned, you're…

This is a strong positive sign that poison fountain works. I wasn't aware of this project. Thanks for the heads up.

It's not a sign about that project in any way. I had never heard of it and have no opinion about it one way or the other.

It's just a sign that single-agenda accounts aren't allowed here—no more, no less. That's why I said "We ban such accounts regardless of what the single purpose happens to be".

Re: An update on residential proxies and the scraper situation

#73

Post by dang »

user- wrote: post #39
dang wrote: post #17

Earlier quoted context omitted.

I've banned this account because we don't allow single-purpose accounts on HN, and your account has been doing that for quite some time now. We ban such accounts regardless of what the single purpose happens to be. Pre-existing agendas are not what HN is for and destroy the curious conversation that it is supposed to be for. https://news.ycombinator.com/newsguidelines.html Edit: If you don't want to be banned, you're…

Just curious dang, did you warn them before banning? Im not against the ban perse (single purpose accounts are bad), just curious if they had a chance to change their contribution style.

No, but if they have a change of heart and genuinely want to use HN as intended, they're welcome to let us know. I've added that in an edit now.

Re: An update on residential proxies and the scraper situation

#74

Post by rao-v »

I’m skeptical that the problem they are trying to solve is truly unreasonable bandwidth demands.

Sometimes it feels like what people want is to only serve websites and content to good normal users but not evil bad “scrapers” (because maybe maybe your content will be monetized in some nebulous way) but … you put your content up publicly on the web! That should be part of reasonable use!

EDIT: Lwn.net is perhaps not a fair target of my ire.

“There is also a desire to not impede the operation of legitimate search engines, the Internet Archive, and other such groups. Some sites may add explicit allowlists to, for example, give the dominant search engine access to the site. Such measures have the effect of further entrenching a monopoly that already serves us poorly and should be avoided. We have, thus far, succeeded in that.”

Is reasonable! Many others are not

Re: An update on residential proxies and the scraper situation

#75

Post by zuzululu »

BoorishBears wrote: post #55
zuzululu wrote: post #28

Earlier quoted context omitted.

i know a few very large startups that used it to fake their way into an exit unethical yes but really raises the question as to what we see is real or not

"Raises the question of what we see is real" No they really don't, dishonest founders do that. You're one with the lower case shibboleth so I have no doubt you surround yourself with dishonest founders, but faking users is pretty damn low on the usecases for residential proxies. I said they're unethical because they tend to be hidden in innocuous seeming apps or sprung on unwitting individuals via clickwraps on their…

ive seen unscrupulous founders fake traction during diligence, which is my day job

but ive never seen one raise $4.5m for an ai agent startup built around pulling fresh web data, then openly cheer the unethical proxy infrastructure used to evade consent and blocks

then inventing a fantasy about who i associate with instead of answering that conflict is an unusually loud form of projection

Re: An update on residential proxies and the scraper situation

#76

Post by dataflow »

dredmorbius wrote: post #32
dang wrote: post #17

Earlier quoted context omitted.

I've banned this account because we don't allow single-purpose accounts on HN, and your account has been doing that for quite some time now. We ban such accounts regardless of what the single purpose happens to be. Pre-existing agendas are not what HN is for and destroy the curious conversation that it is supposed to be for. https://news.ycombinator.com/newsguidelines.html Edit: If you don't want to be banned, you're…

Seriously, dang? 10 comments (excluding subsequent in-thread replies) over four months, always in contexts in which either the topic of LLM scraping or Poison Fountain itself has already been mentioned. This strikes me as contextually informational, and is no different from other project representatives appearing in threads discussing their own subjects or posts. Such as, say, Jon Corbet (@corbet), of LWN, whose acti…

> Such as, say, Jon Corbet (@corbet), of LWN, whose activity on HN shows a similar pattern and roughly equivalent frequency.

I took a look at the most recent comments from both accounts and they don't look similar to me in this respect.

I think there are two questions here though:

1. Was the violation egregious?

2. Did it deserve an immediate ban, or did they deserve a warning etc.?

Seems to me the answer to (1) is yes, but the answer to (2) I'm less sure about.

Re: An update on residential proxies and the scraper situation

#77

Post by TZubiri »

>We have not gone with tools like Anubis, partly because it causes annoying delays for those trying to get to the site, but also partly because it seems inevitable that the scrapers will eventually find their way around it. Indeed, there are some indications that is already happening. A proof-of-work requirement is not a huge obstacle when you have millions of other people's machines to do the work on.

The first argument that it introduces delays to users is solid, but I would advise reconsidering on the second one that a PoW workaround will be found. The moment it does you'll be able to tell because Bitcoin will crash to 0.

Will bots use infected computers to do compute to work around it? Maybe, but it requires a CPU in addition to a network reputation, 2 mechanisms are stronger than one.

Re: An update on residential proxies and the scraper situation

#78

Post by TZubiri »

rao-v wrote: post #74

I’m skeptical that the problem they are trying to solve is truly unreasonable bandwidth demands. Sometimes it feels like what people want is to only serve websites and content to good normal users but not evil bad “scrapers” (because maybe maybe your content will be monetized in some nebulous way) but … you put your content up publicly on the web! That should be part of reasonable use! EDIT: Lwn.net is perhaps not a…

Why the air quotes? Evading a ban and using (potentially ill gotten) residential ips to circumvent that refusal of service, is a bad actor.

Re: An update on residential proxies and the scraper situation

#79

Post by Groxx »

harshreality wrote: post #61

> ...we have tried to minimize the impact on real readers as much as possible. We have not gone with tools like Anubis, partly because it causes annoying delays for those trying to get to the site, but also partly because it seems inevitable that the scrapers will eventually find their way around it. Indeed, there are some indications that is already happening. A proof-of-work requirement is not a huge obstacle when…

Anubis is by far the least annoying throttler I encounter. Entirely agreed, just crank it up when you get a flood, I much prefer waiting a couple seconds to interacting with custom UI for tens of seconds.

I'm so glad to see that (essentially) HashCash is coming back. Now we just need it for email, like it was originally designed for...

Re: An update on residential proxies and the scraper situation

#80

Post by Catloafdev »

rao-v wrote: post #74

I’m skeptical that the problem they are trying to solve is truly unreasonable bandwidth demands. Sometimes it feels like what people want is to only serve websites and content to good normal users but not evil bad “scrapers” (because maybe maybe your content will be monetized in some nebulous way) but … you put your content up publicly on the web! That should be part of reasonable use! EDIT: Lwn.net is perhaps not a…

If it weren't a real problem, these types of articles and services wouldn't exist.
Post reply on HN