Live data from Hacker News

Show HN: Kimono – Never write a web scraper again

kimonify.kimonolabs.com

71–80 of 234 posts

Re: Show HN: Kimono – Never write a web scraper again

#71
post #68

I like the concept and it looks similar at Import.io

Yes, in concept quite similar. We wanted to make something that you can use from within your browser as part of your natural workflow, without installing any other software.We also really wanted to figure out the right data association intelligently based on user selections vs. asking users to think through a data model up front

Re: Show HN: Kimono – Never write a web scraper again

#73
Constructive Tone: I figured that it might be nifty to scrape cedar pollen count information from a calendar and then shoot myself an email when it was higher than 100 gr/m3.

This would be a pretty difficult thing to grab when scraping normally, but the app errors before loading the content:

https://www.keepandshare.com/calendar/show_month.php?i=19409...

JS error: An error occurred while accessing the server, please try againError Reference: 6864046a

Re: Show HN: Kimono – Never write a web scraper again

#74
post #48

Earlier quoted context omitted.

Great, it should work well on webkit browsers, what version of Safari are you using?

7.0.1, the latest. I also don't have Flash installed, but it doesn't look like you're using Flash. The entire top bar doesn't show for me. Feel free to email me and I can send you screenshots.

Thanks - we're not using flash, so it must be something else. Will follow up over email

Re: Show HN: Kimono – Never write a web scraper again

#75

Earlier quoted context omitted.

Pages buried in here: https://fannin4.wcjc.edu/ The course catalog is public, so no login is needed. I want to scrape various data related to courses, to populate forms automatically and such.

Yeah, of course it won't work with HTTPS sites. They'd have to proxy those HTTPS sites and perform a MITM just to do it.

As opposed to HTTP where they proxy it and MITM it? I don't understand the objection.

Re: Show HN: Kimono – Never write a web scraper again

#76
post #73

Constructive Tone: I figured that it might be nifty to scrape cedar pollen count information from a calendar and then shoot myself an email when it was higher than 100 gr/m3. This would be a pretty difficult thing to grab when scraping normally, but the app errors before loading the content: https://www.keepandshare.com/calendar/show_month.php?i=19409... JS error: An error occurred while accessing the server, please…

Thanks for letting us know. Just tried and am getting the same error. The page is loading content dynamically from another source... We'll look into this and see if we can get it working on this page

Re: Show HN: Kimono – Never write a web scraper again

#77
really excited to see this. i've had the idea (and nearly this execution) in mind for years but no use or ambition to get it done.

given the pricing though i'm almost motivated to make my own. as a hosted service the fees make sense with the offerings. but not only would i rather host my own- it would be cheaper all around. would you consider adding a free or cheap self hosted option?

aside, i think there is a mislabel on the pricing page. i'm guessing the free plan should not have 3 times the "apis" than the lite plan.

Re: Show HN: Kimono – Never write a web scraper again

#79

Earlier quoted context omitted.

Pages buried in here: https://fannin4.wcjc.edu/ The course catalog is public, so no login is needed. I want to scrape various data related to courses, to populate forms automatically and such.

Yeah, of course it won't work with HTTPS sites. They'd have to proxy those HTTPS sites and perform a MITM just to do it.

I kind of just assumed that's what they were doing.

Re: Show HN: Kimono – Never write a web scraper again

#80

really excited to see this. i've had the idea (and nearly this execution) in mind for years but no use or ambition to get it done. given the pricing though i'm almost motivated to make my own. as a hosted service the fees make sense with the offerings. but not only would i rather host my own- it would be cheaper all around. would you consider adding a free or cheap self hosted option? aside, i think there is a mislab…

Yes, we're in beta right now, while we're still working out the bugs. For beta, it's free for 30 APIs.
Post reply on HN