41–50 of 100 posts
Oh please. An IP address is an IP address. The whole idea that some IPs are more special than others flies in the face of net neutrality.
Just because the idea of net neutrality exists doesn't mean it's true. I'm sure there's a specific context for it, and 'security' is not that context. It was about data/packet prioritisation wasn't it? Unrelated to security.
They have their problems but how else am I supposed to scrape data from companies that want to hide it?
Do you allow the data you've scraped to be scraped? Do you share it as freely as you desire the 'companies that want to hide it' would? Or do you consider the scraped data is 'hard earned reward for effort' and therefore has value that others should subscribe to your service for?
It's an anti-consumer evil practice, but if it is a national threat that's on the nation's security infra.
The nation's security infra is a reflection of the nation's security legislation and regulation.
Aside: TheSInIoT is my DMZ's wifi password.
Specific Domains:
proxyjs.brdtnet.com
proxyjs.luminatinet.com
proxyjs.bright-sdk.com
clientsdk.bright-sdk.com
clientsdk.brdtnet.com
*.brdtnet.com
*.luminatinet.com
*.luminati.io
They have their problems but how else am I supposed to scrape data from companies that want to hide it?
I'm a bit on the fence about this, but leaning towards the "bad luck". I'm sure there's a large swathe of nuance that I'm missing, but my simplistic view is: If they don't want to be scraped then they don't get included in "the thing" which, at minimum, is a data point for consumers to make consumer decisions about. It strongly depends on how scraped data is being used. If your 'cat' hasn't been able to catch their '…
Let's try this for example. Suppose you want to create a price comparison site.
A lot of the major retailers don't want these, because they want the customer going to their site when they want to buy something, not to the price comparison site that tells the customer which retailer has the best price and then might not always be them.
If the comparison site doesn't have the prices from retailers who sometimes have the best price then it can't serve its function -- customers still have to check sites manually. And comparison sites are pro-consumer whereas blocking them is anti-competitive, so the ones doing the scraping are the good guys.
No, your paranoia-outage is the true "national security threat" - a threat to freedom. Stop fanning the flames and calling for more government intervention.
They are basically 'won't someone please think of the children?', but with higher stakes.
They have their problems but how else am I supposed to scrape data from companies that want to hide it?
Apologies for the double post, but I've got an alternate perspective: Do you allow the data you've scraped to be scraped? Do you share it as freely as you desire the 'companies that want to hide it' would? Or do you consider the scraped data is 'hard earned reward for effort' and therefore has value that others should subscribe to your service for?
Earlier quoted context omitted.
I'm a bit on the fence about this, but leaning towards the "bad luck". I'm sure there's a large swathe of nuance that I'm missing, but my simplistic view is: If they don't want to be scraped then they don't get included in "the thing" which, at minimum, is a data point for consumers to make consumer decisions about. It strongly depends on how scraped data is being used. If your 'cat' hasn't been able to catch their '…
> I generally come from a position of suspicion as to why someone wants to scrape data that the owner goes to certain lengths to protect Let's try this for example. Suppose you want to create a price comparison site. A lot of the major retailers don't want these, because they want the customer going to their site when they want to buy something, not to the price comparison site that tells the customer which retailer…
The I guess the site is either useless, or can be used in combination with customer's own research that includes sites not included in the comparison. The site is still useful, it's just not exhaustive. There's nothing much in the world that's exhaustive, it's all a range of percentages.
Also, maybe the source that isn't scrape-able becomes less popular as a result of not being included in the price comparison list. If they're always more expensive, then neither you nor they have any advantage in listing on your site. If they're always cheaper, then your site may have no purpose to serve.
Everyone seems to want _everything_ despite _everything_ not being necessary to provide a service. The lack of a certain set of data may itself be a data point, and should be used as marketing material for or against the company resisting the scraping.
Just because residential proxies are somewhat of an 'easy answer' doesn't mean they're right. Use more creativity and imagination! Or follow a new idea. It's not like price comparison sites are a revolutionary idea.
(I'm a consumer who happily does significant research before a buying decision, going to various sites and checking prices, warranties, model numbers, reviews, availability, delivery times, and all the shit. And prefer doing the research myself than trusting a price comparison site, so I'm not the target market, thus my bias in my commentary and opinion on that topic).
> so the ones doing the scraping are the good guys.
Everyone thinks they're the good guys. If there's profit involved then there's always an element of delusion.
When it comes to involunary proxies, those are really just one of many possible symptoms stemming from a real problem: Shitty security. (Edit: And shitty contract/privacy laws.)
Shitty security is tolerated by our markets, is is protected from fixes due to copyright law, and it is even encouraged by parts of our government that want to exploit the flaws. We will gain far more from fundamental quality-improvements than we will from whack-a-mole-ing on this symptom.