Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Disrupting the largest residential proxy network

cloud.google.com

Post reply on HN
Print view View original

211–220 of 230 posts

Re: Disrupting the largest residential proxy network

#211

Post by ErroneousBosh »

Ronsenshi wrote: post #202
ErroneousBosh wrote: post #172

Earlier quoted context omitted.

Why are you scraping sites in the first place? What legitimate reason is there for you doing that?

Just today I wanted to get a list of locations of various art events around the city which are all located on the same website, but which does not provide a page with all events happening this month on a map. I need a single map to figure out what I want to visit based on distance I have to travel, unfortunately that's not an option - only option is to go through hundreds of items and hope whatever I picked is near m…

If you want the data, why not contact the organisation with the website?

Why is hammering the everloving fuck out of their website okay?

Re: Disrupting the largest residential proxy network

#212

Post by tonymet »

mrweasel wrote: post #207
tonymet wrote: post #204

Earlier quoted context omitted.

what industry is that? Every industry is on the cloud.

No, no they really aren't, but I was thinking the "scraping industry" in the sense that that's a thing. Getting hosting in smaller datacenters is simple enough, but you may need to manage your own hardware, or VMs. Many will help you get your own IP ranges and ASN, that's going to go a long way, if you don't want to get bundled in with the bad bots. This differs obviously, but having an ASN in our case means that we…

Scraping isn’t an industry. There are legitimate and illegitimate scraping pursuits.

There are lots of healthy / productive businesses in the cloud and lots of scumbags, just like any enterprise.

I still have no idea about your point, by the way.

Re: Disrupting the largest residential proxy network

#213

Post by kstrauser »

plagiarist wrote: post #206
kstrauser wrote: post #187

Earlier quoted context omitted.

I’ve been enduring that exact same traffic pattern. I used Anubis and a cookie redirect to cut the load on my Forgejo server by around 3 orders of magnitude: https://honeypot.net/2025/12/22/i-read-yann-espositos-blog.h...

Aha, that's where the anime girl is from. What sort of traffic was getting past that but still thwarted by the cookie tactic? I guess the bots are all spoofing consumer browser UAs and just the slightest friction outside of well-known tooling will deter them completely.

Yep, that’s why that’s all over the place now. The cookie thing is more of a first line of defense. It turns away a lot of shoddy scrapers with nearly no resources on my side. Anubis knocks out almost all of the remainder.

Re: Disrupting the largest residential proxy network

#214

Post by bettystaplesmd »

progbits wrote: post #43

I'm surprised by the negative takes... Yes, proxies are good. Ones which you pay for and which are running legitimately, with the knowledge (and compensation) of those who run them. Malware in random apps running on your device without your knowledge is bad.

[dead]

Re: Disrupting the largest residential proxy network

#215

Post by piskov »

what wrote: post #104
piskov wrote: post #23

Earlier quoted context omitted.

Yes you do. Private VPS for personal VPN in Netherlands (digital ocean), then Hungary (some small local DC) — both are blocked from day one. > You've been blocked by network security. To continue, log in to your Reddit account or use your developer token. If you think you've been blocked by mistake, file a ticket below and we'll look into it.

Sounds like you just need to sign in or use the api?

I don’t have a reddit account; I open it few times a month from search results

Re: Disrupting the largest residential proxy network

#216

Post by Saris »

ErroneousBosh wrote: post #211
Ronsenshi wrote: post #202

Earlier quoted context omitted.

Just today I wanted to get a list of locations of various art events around the city which are all located on the same website, but which does not provide a page with all events happening this month on a map. I need a single map to figure out what I want to visit based on distance I have to travel, unfortunately that's not an option - only option is to go through hundreds of items and hope whatever I picked is near m…

If you want the data, why not contact the organisation with the website? Why is hammering the everloving fuck out of their website okay?

1 request per second is nowhere even close to hammering a website.

They made the data available on the website already, there's no reason to contact them when you can just load it from their website.

Re: Disrupting the largest residential proxy network

#217

Post by Saris »

ErroneousBosh wrote: post #172
Ronsenshi wrote: post #129

Earlier quoted context omitted.

One thing about Google is that many anti-scraping services explicitly allow access to Google and maybe couple of other search engines. Everybody else gets to enjoy CloudFlare captcha, even when doing crawling at reasonable speeds. Rules For Thee but Not for Me

Why are you scraping sites in the first place? What legitimate reason is there for you doing that?

I've used change detection for in-stock alerts before, or event updates. Plenty of legitimate uses.

Re: Disrupting the largest residential proxy network

#218

Post by tgsovlerkhgsel »

direwolf20 wrote: post #81
tgsovlerkhgsel wrote: post #66

Earlier quoted context omitted.

Play Protect blocks malicious apps, not network traffic, so no, it obviously doesn't interfere with Google's apps. AFAIK it also left SmartTube (an alternative YouTube client) alone until the developer got pwned and the app trojanized with this kind of SDK, and the clean versions are AFAIK again being left alone. No guarantee that it won't change in the future, of course, but so far they seem to not be abusing it.

Does malicious mean interfering with Google's business model, or does it include intrusive advertising?

Malicious here means "most people who aren't trying to argue semantics or otherwise be smartasses about it would consider it malware". That's why the example I gave is a semi-popular software the allows watching YouTube without ads without a premium subscription, i.e. at least in the case I observed, I don't believe this was weaponized against apps that interfere with their business model.

As for "intrusive advertising is malicious", see the second part of the first sentence.

Re: Disrupting the largest residential proxy network

#220

Post by chii »

sitzkrieg wrote: post #188
Spooky23 wrote: post #170

Earlier quoted context omitted.

Running a tor node is pretty stupid from a liability perspective, but at least you have more deniability and you are making an informed choice. These residential proxies are pretty much universally shady. I doubt most of the users understand what they are consenting to.

tor nodes are zero risk as long as they're not an exit been running nodes since 2017 on two providers and zero issues

yes, but i was likening being part of the proxy network to being a tor exit node. I should've made my comment clearer.
Post reply on HN