101–110 of 234 posts
This is excellent. Even it if doesn't work for scraping all sites, it simplifies the average use case so much that it's not even funny. Feature proposal: deal with pagination.
Another feature, simple one: Allow to add some filters to the data stream. For example: only posts that contain word "bitcoin" in the name or only those with 50 upvotes or more.
Earlier quoted context omitted.
Yes, it's a great point. We're working on updating the query param passthrough to handle params within the URL structure.
Apart from that here are other items. You may have these on your list, but can count my vote to prioritize. 1. Pagination 2. Image URLs 3. Focus on page types such as product pages, posts etc. That way its easy to go from content to content. Will help crawling too 4. Link back to original page included in JSON Finally common sites/pages used by multiple users of your systems should not count against the API count req…
Earlier quoted context omitted.
The people who scrape data to avoid paying for APIs are the same people who will not pay for a service to make scraping easier ;)
The people who scrape data at Scraperwiki -- which was made by the same people who opened up parliamentary transcripts in the UK for the first time, and the UN's proceedings, and data about how MPs in London vote -- generally don't have an option to buy anything because the data's hidden by governments from the people who paid for it, on purpose. But by all means take this opportunity to dismiss all of us as freeload…
It would have cost me $348/year to move those to new scraperwiki.
This is really slick! Btw. Who made your intro video?
I would seriously consider rethinking that Favicon.
Do you have something against vegetables?
The presentation is beautiful and the website is great, but the tech broke so I have no idea how or if this even works. This is a wonderful concept and one I've talked about doing with others. I was really excited to try this. I watched the demo video and it seemed straightforward. I went to try and use it on the demo page it provides, going through and adding things, but when I went to save it, I just received an er…
Sorry you had a bad first experience. We've tested this on a lot of sites, and it works stably across a lot of different cases, but haven't solved it everywhere. Thanks for letting us know about the discussion page. We'll look into the bugs right now
...i will comment that the user experience for first adding elements is not what i expected. once u understand it's great overall, but i'd have some popup tooltip type guide thing going on for the first time a user uses it explaining that you should choose one element first, and then check a matching element. ...Then explain the number bubbles next to the property field. You don't get what those numbers are at first. ..Then for the property field, somehow make it more obvious that you should fill it out.
Basically you have the interface setup so any action can be done at any time, but it should be presented as a forced sequential set of steps to the end user, at least at first.
Earlier quoted context omitted.
Sorry you had a bad first experience. We've tested this on a lot of sites, and it works stably across a lot of different cases, but haven't solved it everywhere. Thanks for letting us know about the discussion page. We'll look into the bugs right now
hacker news has probably been scraped too many times. that's why it's not working. it didn't work for me, but other sites worked easily. ...i will comment that the user experience for first adding elements is not what i expected. once u understand it's great overall, but i'd have some popup tooltip type guide thing going on for the first time a user uses it explaining that you should choose one element first, and the…