Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Launch HN: Dashblock (YC S19) – Turn Any Website into an API

news.ycombinator.com

Post reply on HN
Print view View original

31–40 of 102 posts

Re: Launch HN: Dashblock (YC S19) – Turn Any Website into an API

#31

Post by turtlebits »

HPouillot wrote: post #23
turtlebits wrote: post #19

Very cool, but super slow, especially for an API point, which I would expect you could use directly from a front end. Tested on a site I regularly visit dashblock (3 selectors, ~20 items): 16.911 seconds curl (no scraping): 60 ms chrome: 987 ms edit: added chrome

Indeed, we are rendering the whole page with the javascript, that's why it takes longer than a curl. For now, it's especially useful for dynamic pages but we also plan on supporting pages that don't require rendering.

Sounds good. It does make sense to check to see if your selectors work with raw HTML on publish to verify if you require JS or not.

Re: Launch HN: Dashblock (YC S19) – Turn Any Website into an API

#32

Post by _xnmw »

A productivized scraping service - useful! Entire companies are built around scraping certain popular sites - this is a disruptive idea indeed. A growing catalog of up-to-date scrapers for popular websites would put of lot of freelancers out of work. I would invest in this.

However, the ML claim is highly suspect. There is no way that a machine could reliably understand the semantic content of a website - that would require Artificial General Intelligence. If anyone could do that, it would've been Google. But even Google relies on human-edited structured metadata to define the content of sites (i.e. Rich Snippets and the like).

Re: Launch HN: Dashblock (YC S19) – Turn Any Website into an API

#35

Post by MCorbani »

css wrote: post #15

how do you avoid getting banned by the companies you scrape? Most ToSs have a clause like: > We prohibit crawling, scraping, caching or otherwise accessing any content on the Service via automated means... [etc]

Agreed, some websites are really reticent about scraping. But let's think about Google, they are scraping the whole web regardless of the ToS of the websites, so it all boils down to one question : do you create value for the website owner ? That's why we want to focus on use-cases where we create value for both, our users and the website owner. If you think about Yodlee/Plaid in the banking sector, they built partnerships with the Banks but continued scraping them because most of them didn't provide an API.

Re: Launch HN: Dashblock (YC S19) – Turn Any Website into an API

#36

Post by MCorbani »

the_watcher wrote: post #28

This is cool, looking forward to trying it out. Manual scraping is doable, but there have been plenty of times that I've just decided not to do something because I'd have to spend an hour getting/scraping the data. Hopefully this will take that time down to 10 minutes or less.

Definitely =) Let us know know if we can be of any help!

Re: Launch HN: Dashblock (YC S19) – Turn Any Website into an API

#37

Post by omarhaneef »

I really want to try it because I think I need something like this.

However, how do I know it is a legitimate product and not some virus/scam software?

I know YC is a vote of confidence, if it is a YC company, and all the copy sounds legit, and you sound like a pair of honest, hard working entrepreneurs.

But is there some way to check before I run the software?

Edit: Note that I would not have this concern if it were a web platform like Kimono used to be.

Re: Launch HN: Dashblock (YC S19) – Turn Any Website into an API

#38

Post by w457uiw4gftyi »

Are you supporting the use case where web site providers consider scraping to be hostile? Ie, spinning up new cloud instances until one isn't blacklisted by the site, all behind the scenes so the consumer of your API doesn't have to worry about such things?
Post reply on HN