171–180 of 234 posts
Wow, this is looking good, I wish I had it available to me 6 months ago! Nice job :D I don't know if it's just me or not, but it's not working for me in Firefox (OSX Mavericks 10.9.1 and Firefox v26). The X's and checkmarks aren't showing up next to the highlighted selections. Works fine in Safari.
I disagree. Web scraping is mostly fun. You don't need "a ton of code" and "a laundry list of libraries", just something like Beautiful Soup and maybe XSLT.
The end of the statement is truer: it's not really a problem that your web scraper will have to be hosted somewhere, since the thing you're using it for also has to be hosted somewhere, but yes, it needs to be maintained and it will break if the source changes.
But I don't see how this solution could ever be able to automatically evolve with the source, without the original developer doing anything?
Looks very nice. There seems to be an issue with international characters though (æ/ø/å).
Earlier quoted context omitted.
Would there any way to fake the beginning of an OAuth session with Facebook, Google or any other OAuth authenticated site? Kind of like replaying cookies to hijack sessions?
The route of proxying the web page presents much difficulty in doing actual authentication on Facebook or Google's website via the proxied webpage without first rewriting most of the javascript and hijacking their Ajax calls on the fly. The approach I took was to hijack the Cookies from the browser once the user has signed in after on e.g. Facebook via the browser extension. The route of proxying the website does in…
It would be nice to have a view also on the raw html code, e.g., to create a field containing the url of an image in the page.
> Web scraping. It's something we all love to hate. You wish the data you needed to power your app, model or visualization was available via API. But, most of the time it's not. So, you decide to build a web scraper. You write a ton of code, employ a laundry list of libraries and techniques, all for something that's by definition unstable, has to be hosted somewhere, and needs to be maintained over time. I disagree.…