121–130 of 196 posts
Earlier quoted context omitted.
It’s legal if it’s in the contract. Standard for contracts to allow for mistakes and confirmations of prices
It's not mistake if you do it deliberately!
Earlier quoted context omitted.
If data is your competitive advantage or product, then what? Accept that your market no longer exists and that there's no way to stop theft?
You're going to need to explain how scraping publicly available information on a website is theft. If information is your competitive advantage maybe you shouldn't have it on a publicly accessible website, and should instead stick it behind an API with pay tiers and a very clear license regarding what you may do with it as an end user. Note, a simple sign up being required to view a website makes it not publicly avai…
Earlier quoted context omitted.
I know a guy at Nike that had to deal with a similar problem. As I recall, they basically gave in -- instead of trying to fight the scrapers, they built them an API so they'd quit trashing the performance of the retail site with all the scraping.
The real Jedi move
Earlier quoted context omitted.
You're going to need to explain how scraping publicly available information on a website is theft. If information is your competitive advantage maybe you shouldn't have it on a publicly accessible website, and should instead stick it behind an API with pay tiers and a very clear license regarding what you may do with it as an end user. Note, a simple sign up being required to view a website makes it not publicly avai…
It is copyright information, no? So technically it is intellectual property theft if the scraping use is for commercial purposes.
Earlier quoted context omitted.
If data is your competitive advantage or product, then what? Accept that your market no longer exists and that there's no way to stop theft?
You're going to need to explain how scraping publicly available information on a website is theft. If information is your competitive advantage maybe you shouldn't have it on a publicly accessible website, and should instead stick it behind an API with pay tiers and a very clear license regarding what you may do with it as an end user. Note, a simple sign up being required to view a website makes it not publicly avai…
Seriously? Do I need to explain why a song doesn’t enter the public domain when it is played on the radio?
Where I was working we stopped caring about ips browser etc because it was just a race. What we did was analyzing behaviour of clicks and acted on that. When we recognized it we went on serving a fake page. It cuts down a little bit of costs because it was static pages. In general it took a lot of time for them to discover the pattern and it was way more manageable for us.
Earlier quoted context omitted.
You're going to need to explain how scraping publicly available information on a website is theft. If information is your competitive advantage maybe you shouldn't have it on a publicly accessible website, and should instead stick it behind an API with pay tiers and a very clear license regarding what you may do with it as an end user. Note, a simple sign up being required to view a website makes it not publicly avai…
> You're going to need to explain how scraping publicly available information on a website is theft. Seriously? Do I need to explain why a song doesn’t enter the public domain when it is played on the radio?
At some point people are gonna have to accept this.
I scrap government sites a lot as they don't provide apis. For mobile proxies, I use the proxidize dongles and mobinet.io (free, with Android devices). As stated in the article, with cgNAT it's basically impossible to block them as in my case, half the country couldn't access the sites anymore (if you place them in several locations and use one carrier each there).
I feel like I'm getting a glimpse into the dark underbelly of the web.
Earlier quoted context omitted.
I know a guy at Nike that had to deal with a similar problem. As I recall, they basically gave in -- instead of trying to fight the scrapers, they built them an API so they'd quit trashing the performance of the retail site with all the scraping.
I think there's an opportunity for a new JS framework to have something like randomly generated dom that will always display the page and elements the same to a human but constantly break paths for computers. Like displaying a table with semantic elements, then divs, then using an iframe with css grid and floating values over the top. This almost seems like a problem for AI to solve.
Still, maybe AI comes into it. Maybe poisoning the data is the right way to do it conditioned on ML-juiced anomaly detection.