Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Disrupting the largest residential proxy network

cloud.google.com

Post reply on HN
Print view View original

121–130 of 230 posts

Re: Disrupting the largest residential proxy network

#121

Post by bigiain »

throwoutway wrote: post #89
progbits wrote: post #43

I'm surprised by the negative takes... Yes, proxies are good. Ones which you pay for and which are running legitimately, with the knowledge (and compensation) of those who run them. Malware in random apps running on your device without your knowledge is bad.

> Malware in random apps running on your device without your knowledge is bad. And ones that have all the indicators of compromise of Russia, Iran, DPRK, PRC, etc

Am I the only one cynically thinking that "Russia, Iran, DPRK, PRC, etc" is the "But think of the chiiildren!!!" excuse for doing this?

And when Google say

"IPIDEA’s proxy infrastructure is a little-known component of the digital ecosystem leveraged by a wide array of bad actors."

What they really mean is " ... leveraged by actors indiscriminately scraping the web and ignoring copyright - that are not us."

I can't help but feel this is just Google trying to pull the ladder up behind then and make it more difficult for other companies to collect training data.

Re: Disrupting the largest residential proxy network

#122

Post by wmf »

vlovich123 wrote: post #120
progbits wrote: post #43

I'm surprised by the negative takes... Yes, proxies are good. Ones which you pay for and which are running legitimately, with the knowledge (and compensation) of those who run them. Malware in random apps running on your device without your knowledge is bad.

> Some users may knowingly install this software on their devices, lured by the promise of “monetizing” their spare bandwidth. Sounds like they’re targeting networks even if the users are ok participating in, precisely what you’re saying is ok. As for malware enrolling people into the network, it depends if the operator is doing it or if the malware is 3rd parties trying to get a portion of the cash flow. In the latt…

Users are OK with acting as proxies because they don't understand all the shady stuff their proxy is being used for. Also consumer ISPs generally ban this.

Re: Disrupting the largest residential proxy network

#123

Post by walletdrainer »

VladVladikoff wrote: post #114
walletdrainer wrote: post #111

It’s interesting that when Luminati, an Israeli company, does this, it’s fine. When the Chinese do this? Very bad.

They are both bad. You are showing your own bias.

Personally, I don’t think either of them are actually meaningfully bad. A bit naughty, maybe?

I do think the disparity in attention is fascinating. These new Chinese players have been getting nonstop press while everyone ignores the established giant.

Re: Disrupting the largest residential proxy network

#124

Post by xhcuvuvyc »

vlovich123 wrote: post #120
progbits wrote: post #43

I'm surprised by the negative takes... Yes, proxies are good. Ones which you pay for and which are running legitimately, with the knowledge (and compensation) of those who run them. Malware in random apps running on your device without your knowledge is bad.

> Some users may knowingly install this software on their devices, lured by the promise of “monetizing” their spare bandwidth. Sounds like they’re targeting networks even if the users are ok participating in, precisely what you’re saying is ok. As for malware enrolling people into the network, it depends if the operator is doing it or if the malware is 3rd parties trying to get a portion of the cash flow. In the latt…

> These SDKs, which are offered to developers across multiple mobile and desktop platforms, surreptitiously enroll user devices into the IPIDEA network.

?

Re: Disrupting the largest residential proxy network

#125

Post by BatteryMountain »

edg5000 wrote: post #118

Residential proxies are the only way to crawl and scrape. It's ironic for this article to come from the biggest scraping company that ever existed! If you crawl at 1Hz per crawled IP, no reasonable server would suffer from this. It's the few bad apples (impatient people who don't rate limit) who ruin the internet for both users and hosters alike. And then there's Google.

Saying the quiet part out loud...Shhhs

Re: Disrupting the largest residential proxy network

#126

Post by miki123211 »

Nextgrid wrote: post #105
vachina wrote: post #96

Earlier quoted context omitted.

The proxies is the reason why you get spam in your Google search result, spam in your Play store (by means of fake good reviews), basically spam in anything user generated. It directly affects Google and you, I don’t see why they should not do this.

Spam in Google search results is due to Google happily taking money from the spammers in exchange for promoting their spam, or that the spam sites benefit Google indirectly by embedding Google Ads/Analytics. I don't see any spam in Kagi, so clearly there is a way to detect and filter it out. Google is simply not doing so because it would cut into their profits.

The reason you don't see spam in Kagi is because nobody is targeting Kagi specifically.

They can probably get away with a lot of stupid rules that would backfire if anybody tried to cater to them specifically.

Re: Disrupting the largest residential proxy network

#127

Post by miki123211 »

trollbridge wrote: post #26
xyzzy_plugh wrote: post #4

> These efforts to help keep the broader digital ecosystem safe supplement the protections we have to safeguard Android users on certified devices. We ensured Google Play Protect, Android’s built-in security protection, automatically warns users and removes applications known to incorporate IPIDEA SDKs, and blocks any future install attempts. Nice to see Google Play Protect actually serving a purpose for once.

Yeah, it serves the purpose of blocking this kind of proxy traffic that isn't in Google's personal best interests. Only Google is allowed to scrape the web.

Google does not use residential proxies.

This does nothing against your ability to scrape the web the Google way, AKA from your own assigned IP range, obeying robots.txt, and with an user agent that explicitly says what you're doing and gives website owners a way to opt out.

What Google doesn't want (and I don't think that's a bad thing) is competitors scraping the web in bad faith, without disclosing what they're doing to site owners and without giving them the ability to opt out.

If Google doesn't stop these proxies, unscrupulous parties will have a competitive advantage over Google, it's that simple. Then Google will have to decide between just giving up (unlikely) or becoming unscrupulous themselves.

Re: Disrupting the largest residential proxy network

#128

Post by Nextgrid »

miki123211 wrote: post #126
Nextgrid wrote: post #105

Earlier quoted context omitted.

Spam in Google search results is due to Google happily taking money from the spammers in exchange for promoting their spam, or that the spam sites benefit Google indirectly by embedding Google Ads/Analytics. I don't see any spam in Kagi, so clearly there is a way to detect and filter it out. Google is simply not doing so because it would cut into their profits.

The reason you don't see spam in Kagi is because nobody is targeting Kagi specifically. They can probably get away with a lot of stupid rules that would backfire if anybody tried to cater to them specifically.

"SEO spammers being more advanced than multi-billion-dollar search conglomerate" is a myth. Spam sites have an obvious objective: display ads, shill affiliate links or sell products. All these have to be visible, since an ad or product you can't see/buy is worthless. It is trivial to train a classifier to detect these.

But let's play devil's advocate and say you are right and spammers are successfully outsmarting Google - well, Kagi does use Google results via SerpAPI by their own admission, meaning they too should have those spam results. Yet they somehow manage to filter them out with a fraction of the resources available to Google itself with no negative impact on search quality.

Re: Disrupting the largest residential proxy network

#129

Post by Ronsenshi »

edg5000 wrote: post #118

Residential proxies are the only way to crawl and scrape. It's ironic for this article to come from the biggest scraping company that ever existed! If you crawl at 1Hz per crawled IP, no reasonable server would suffer from this. It's the few bad apples (impatient people who don't rate limit) who ruin the internet for both users and hosters alike. And then there's Google.

One thing about Google is that many anti-scraping services explicitly allow access to Google and maybe couple of other search engines. Everybody else gets to enjoy CloudFlare captcha, even when doing crawling at reasonable speeds.

Rules For Thee but Not for Me

Post reply on HN