Show HN: Kimono – Never write a web scraper again

Post by tinco » Thu, Jan 16, 2014, 2:58 PM UTC

theg2 wrote: post #199

tinco wrote: post #187

Earlier quoted context omitted.

No. This site isn't about UX or innovation, it's about tech start ups. It's a constant reminder that something can be successful even if it was written in a LISP dialect and has a bunch of UX misses as long as the core idea is valid and the product usable enough. I've used a bunch of HN skins that were supposedly better designed, but none of them stuck. Apparently it's just plain unnecessary for HN to be better.

So because it's written in LISP it gets a free pass in other areas? That seems like a dubious claim.

Post by romaniv » Thu, Jan 16, 2014, 3:23 PM UTC

LordIllidan wrote: post #202

precisioncoder wrote: post #192

Earlier quoted context omitted.

I like HN because it gives me exactly what I want. A list of interesting links, ranked by what people that I truest more than news sites think of them. Oh and offers comment sections for each of them. HN is wonderful link discovery during compile time or if you need a break.

I think we all like HN, otherwise we wouldn't be here. That doesn't mean it can't improve.

Post by thatthatis » Thu, Jan 16, 2014, 3:43 PM UTC

loceng wrote: post #166

pranade wrote: post #144

Earlier quoted context omitted.

At the moment, we rely on users to be responsible. We spell it out in the terms and FAQ. We've been in private beta, keeping usage very limited until today. We fully understand the seriousness of the issue as we scale. We're committed to becoming a responsible bot that respects robots.txt

I would say how you're scraping differs from say how Google, a search engine, scrapes. I'm not sure there is a way in robots.txt to define for each use? Knowing the data in a structured way, but then allowing it to be displayed in full off-site is quite different than using the scraped data for linking into a website.

Post by Flimm » Thu, Jan 16, 2014, 4:13 PM UTC

theg2 wrote: post #199

tinco wrote: post #187

Earlier quoted context omitted.

No. This site isn't about UX or innovation, it's about tech start ups. It's a constant reminder that something can be successful even if it was written in a LISP dialect and has a bunch of UX misses as long as the core idea is valid and the product usable enough. I've used a bunch of HN skins that were supposedly better designed, but none of them stuck. Apparently it's just plain unnecessary for HN to be better.

So because it's written in LISP it gets a free pass in other areas? That seems like a dubious claim.

Post by rpedela » Thu, Jan 16, 2014, 4:22 PM UTC

bambax wrote: post #172

> Web scraping. It's something we all love to hate. You wish the data you needed to power your app, model or visualization was available via API. But, most of the time it's not. So, you decide to build a web scraper. You write a ton of code, employ a laundry list of libraries and techniques, all for something that's by definition unstable, has to be hosted somewhere, and needs to be maintained over time. I disagree.…