Phantom Js (Amazing library for web scraping)

#1

Post by swamiyeswanth » Mon, Sep 28, 2015, 6:15 AM UTC

Phantom Js (Amazing library for web scraping)
advencode.wordpress.com

Re: Phantom Js (Amazing library for web scraping)

#2

Post by samsk » Mon, Sep 28, 2015, 7:04 AM UTC

Well, one can use headless browser or simply fetch json file(s) directly (the one that is used as source for dynamic menu). The second way, uses a 'bit' less resources that can be used for scrapping itself and not for page rendering ;) But yes, in few cases it might be maybe worth it...

Re: Phantom Js (Amazing library for web scraping)

#3

Post by xlm1717 » Mon, Sep 28, 2015, 1:54 PM UTC

samsk wrote: post #2

Well, one can use headless browser or simply fetch json file(s) directly (the one that is used as source for dynamic menu). The second way, uses a 'bit' less resources that can be used for scrapping itself and not for page rendering ;) But yes, in few cases it might be maybe worth it...

PhantomJS is a headless browser. It's still running webkit, but it loads pages just like a regular browser and allows you to run custom JS scripts on pages, making it useful for QA and TDD.

The only snag I've hit with PhantomJS is it starts to get buggy if you run it using a long-running script and intend to open several tabs often.