Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Scraping Problem and Ethics

blog.osvdb.org

Post reply on HN
Print view View original

1–10 of 130 posts

Re: The Scraping Problem and Ethics

#3

Post by duey »

The OSVDB website contains no signup page for commercial access. No pricing either, purely sign up via contacting someone. From my experience whenever I see this, I just refuse to use the service and look elsewhere. Contacting someone is annoying and opens you up to repeat sales calls. Perhaps they should make commercial API access easier to access rather than complain about scrapers.

Re: The Scraping Problem and Ethics

#4

Post by lugg »

Pretty sad. McAfee I dont really expect much less from. They've always been a bunch of scam artists / con's in my mind.

I dontbhave time to look but couple of things striked me as odd. Is there a reason you dont lock down your request limits? Also why dont you secure access? Allow free accounts to be made and issue apis keys for them, at least that way you can much more easily rate limit access to the apis and heavily rate limit front end web requests down to reasonable numbers.

Re: The Scraping Problem and Ethics

#5

Post by awor »

duey wrote: post #3

The OSVDB website contains no signup page for commercial access. No pricing either, purely sign up via contacting someone. From my experience whenever I see this, I just refuse to use the service and look elsewhere. Contacting someone is annoying and opens you up to repeat sales calls. Perhaps they should make commercial API access easier to access rather than complain about scrapers.

Did you read the article? In both cases the offending parties were told to contact RBS to obtain a proper license/account.

Re: The Scraping Problem and Ethics

#7

Post by kijin »

awor wrote: post #5
duey wrote: post #3

The OSVDB website contains no signup page for commercial access. No pricing either, purely sign up via contacting someone. From my experience whenever I see this, I just refuse to use the service and look elsewhere. Contacting someone is annoying and opens you up to repeat sales calls. Perhaps they should make commercial API access easier to access rather than complain about scrapers.

Did you read the article? In both cases the offending parties were told to contact RBS to obtain a proper license/account.

That's exactly the problem that GP is talking about.

McAfee were just being the usual assholes, but the first guy mentioned in the blog post could have been converted into a paying customer if the pricing scheme were clearly outlined on the web. Since "Contact us" account tiers are usually reserved for the very high end, he probably assumed that it would cost him an arm and a leg.

It shouldn't be too difficult to come up with a handful of moderately priced account tiers, each targeting a different type of customer, as well as an open-ended tier at the end ("Contact us") for those with special needs and deep pockets.

Re: The Scraping Problem and Ethics

#8

Post by DigitalSea »

If I were in OSVDB's shoes, I would call these people out in an email and ask them to pay a licencing fee. McAfee have always had a shady past, even when they shook themselves clean of John, they have a history of scam-like behaviour to make a quick-buck.

You should be rate-limiting how many requests free API users can make, like; Twitter, Facebook and every other Internet provider does via their API. Make it harder for people to obtain the information (to the best of your abilities) and paying will become more of an option because they won't go to the trouble of scraping it if it's impossible and will take a lot of time to do so.

Think of your offering as a car. You currently have no car alarm or immobiliser, if you install a immobiliser and car alarm, you will make it very hard for a thief to steal your car.

Post reply on HN