Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Disrupting the largest residential proxy network

cloud.google.com

Post reply on HN
Print view View original

131–140 of 230 posts

Re: Disrupting the largest residential proxy network

#131

Post by chii »

wmf wrote: post #122
vlovich123 wrote: post #120

Earlier quoted context omitted.

> Some users may knowingly install this software on their devices, lured by the promise of “monetizing” their spare bandwidth. Sounds like they’re targeting networks even if the users are ok participating in, precisely what you’re saying is ok. As for malware enrolling people into the network, it depends if the operator is doing it or if the malware is 3rd parties trying to get a portion of the cash flow. In the latt…

Users are OK with acting as proxies because they don't understand all the shady stuff their proxy is being used for. Also consumer ISPs generally ban this.

But then would you make the same arguments for running a tor node (presumably, you don't know what shady stuff is there, but you know there's shady stuff)?

Re: Disrupting the largest residential proxy network

#132

Post by chii »

Ronsenshi wrote: post #129
edg5000 wrote: post #118

Residential proxies are the only way to crawl and scrape. It's ironic for this article to come from the biggest scraping company that ever existed! If you crawl at 1Hz per crawled IP, no reasonable server would suffer from this. It's the few bad apples (impatient people who don't rate limit) who ruin the internet for both users and hosters alike. And then there's Google.

One thing about Google is that many anti-scraping services explicitly allow access to Google and maybe couple of other search engines. Everybody else gets to enjoy CloudFlare captcha, even when doing crawling at reasonable speeds. Rules For Thee but Not for Me

> many anti-scraping services explicitly allow access to Google and maybe couple of other search engines.

because google (and the couple of other search engines) provide enough value that offset the crawler's resource consumption.

Re: Disrupting the largest residential proxy network

#133

Post by ghxst »

walletdrainer wrote: post #113
7thpower wrote: post #91

Earlier quoted context omitted.

They have a robust KYC that appears to serve, at least in large part, as a way to stay off the shit list of companies with the resources to pursue recourse. Source: went through that process, ended up going a different route. The rep was refreshingly transparent about where they get the data, why the have the kyc process (aside from regulatory compliance). Ended up going with a different provider who has been cheaper…

I’ve certainly never been asked to do KYC with Luminati after using them for hundreds of terabytes over the years. It’s not like I’m using some bigco email address or given them any other reason to skip KYC either.

They do KYC when you want to unblock certain domains.

Re: Disrupting the largest residential proxy network

#134

Post by chii »

Imustaskforhelp wrote: post #33
londons_explore wrote: post #20

Earlier quoted context omitted.

> that's how you get legal threats from media companies or the police called to your house. Or residential proxies get so widespread that almost every house has a proxy in, and it becomes the new way the internet works - "for privacy, your data has been routed through someone else's connection at random".

> Or residential proxies get so widespread that almost every house has a proxy in, and it becomes the new way the internet works - "for privacy, your data has been routed through someone else's connection at random". Is this a re-invention of tor, maybe I2P?

> Is this a re-invention of tor

in a way, yes - the weakness of tor is realistically the lack of widespreadness. Tor traffic is identifiable and blockable due to the relatively rare number of exit nodes (which also makes it dangerous to run exit nodes, as you become "liable").

Engraining the ideas of tor into regular users' internet usage is what would prevent the internet from being controlled and blockable by any actor (except perhaps draconian gov't over reach, which while can happen, is harder in the west).

Re: Disrupting the largest residential proxy network

#135

Post by ghxst »

tokyobreakfast wrote: post #37
londons_explore wrote: post #9

We need more residential proxies, not less. I've had enough of companies saying "you're connecting from an AWS IP address, therefore you aren't allowed in, or must buy enterprise licensing". Reddit is an example which totally blocks all data to non-residential IP's. I want exactly the same content visible no matter who you are or where you are connecting from, and a robust network of residential proxies is a stepping…

> I've had enough of companies saying "you're connecting from an AWS IP address I run a honeypot and the amount of bot traffic coming from AWS is insane. It's like 80% before filtering, and it's 100% illegitimate.

Most of them abuse the ip pool attached to lambda from my experience.

Re: Disrupting the largest residential proxy network

#136

Post by ThePowerOfFuet »

direwolf20 wrote: post #81
tgsovlerkhgsel wrote: post #66

Earlier quoted context omitted.

Play Protect blocks malicious apps, not network traffic, so no, it obviously doesn't interfere with Google's apps. AFAIK it also left SmartTube (an alternative YouTube client) alone until the developer got pwned and the app trojanized with this kind of SDK, and the clean versions are AFAIK again being left alone. No guarantee that it won't change in the future, of course, but so far they seem to not be abusing it.

Does malicious mean interfering with Google's business model, or does it include intrusive advertising?

malicious ≠ intrusive.

Re: Disrupting the largest residential proxy network

#137

Post by walletdrainer »

ghxst wrote: post #133
walletdrainer wrote: post #113

Earlier quoted context omitted.

I’ve certainly never been asked to do KYC with Luminati after using them for hundreds of terabytes over the years. It’s not like I’m using some bigco email address or given them any other reason to skip KYC either.

They do KYC when you want to unblock certain domains.

Also not my experience, even though I’ve had to email them for whitelisting.

It might just be because my account is very old?

Re: Disrupting the largest residential proxy network

#139

Post by ryanjshaw »

miki123211 wrote: post #127
trollbridge wrote: post #26

Earlier quoted context omitted.

Yeah, it serves the purpose of blocking this kind of proxy traffic that isn't in Google's personal best interests. Only Google is allowed to scrape the web.

Google does not use residential proxies. This does nothing against your ability to scrape the web the Google way, AKA from your own assigned IP range, obeying robots.txt, and with an user agent that explicitly says what you're doing and gives website owners a way to opt out. What Google doesn't want (and I don't think that's a bad thing) is competitors scraping the web in bad faith, without disclosing what they're do…

> This does nothing against your ability to scrape the web the Google way

I thought that Google has access to significant portions of the internet that non-Google bots won’t have access to?

Re: Disrupting the largest residential proxy network

#140

Post by iammrpayments »

wmf wrote: post #122
vlovich123 wrote: post #120

Earlier quoted context omitted.

> Some users may knowingly install this software on their devices, lured by the promise of “monetizing” their spare bandwidth. Sounds like they’re targeting networks even if the users are ok participating in, precisely what you’re saying is ok. As for malware enrolling people into the network, it depends if the operator is doing it or if the malware is 3rd parties trying to get a portion of the cash flow. In the latt…

Users are OK with acting as proxies because they don't understand all the shady stuff their proxy is being used for. Also consumer ISPs generally ban this.

You could say the same about google’s terms of service.
Post reply on HN