Congrats on the launch! Look forward to playing around with it. I remember a similar startup making quite a splash on HN a few years ago — http://www.kimonolabs.com/ . Do you know why they shut it down and what you guys are doing differently?
Thank you! Kimono Labs built an amazing product and has been finally acquired by Palantir. The main difference between us comes from our machine learning model that gives stability to extraction over time and parse webpages in a generic way. We are also working on automating navigation, which is something they didn't do =)
Sorry to be a pain here, but I very much doubt your ML thing is working at all. Opening a website and finding a dom element is trivial, so the only thing I'd get when buying from you is the promise that this will be resilient to website updates. But at the same time, for $500/month, you can definitely have people updating the selectors manually...
Sorry to be a pain here, but I very much doubt your ML thing is working at all. Opening a website and finding a dom element is trivial, so the only thing I'd get when buying from you is the promise that this will be resilient to website updates. But at the same time, for $500/month, you can definitely have people updating the selectors manually...
Save your money. Create robust regression scripts and hire a freelancer to fix it when the schema changes. anyone serious crawling/scraping data from websites won’t leave it up to some automated ml magic to extract the right data for them.
The Freelancer approach does not work, if the data you extract is time critical. But then again, why would anyone not try to find an api for such data and rely on dom parsing.
So, OPs product is worth it, if you validated and trust their ML model to work correctly, believe they can guarantee a certain uptime and the data you extract is time sensitive or mission critical. Also the data cannot be resourced from an api.
People with such needs may be willing to pay good bucks, but good luck finding early adopters as well as the data-niche where there is demand for something like this.
how do you avoid getting banned by the companies you scrape? Most ToSs have a clause like: > We prohibit crawling, scraping, caching or otherwise accessing any content on the Service via automated means... [etc]
Congrats on the launch! Seems a little similar to Diffbot.com but they do not require a client download.
Correct! Also, Diffbot extracts automatically generic entities (e.g. product name and price, comments, etc) while we let our users choose exactly the data they want on any webpage =)