Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PhantomJS: Archiving the project, suspending development

github.com

Post reply on HN
Print view View original

31–40 of 138 posts

Re: PhantomJS: Archiving the project, suspending development

#31

Post by apocalyptic0n3 »

nkozyra wrote: post #22
tnolet wrote: post #21

Earlier quoted context omitted.

Big misunderstanding in browser land. The HTML delivered to you over the wire, the stuff Nokogiri sees, is not the stuff you see on your screen or even when doing a “view source”

OK, obviously the stuff you see on your screen not matching the HTML delivered makes sense, but explain the HTML source not matching what's sent via the HTTP response. DOM can be modified, of course, JS can introduce more dynamic HTML, but view-source should always represent any non-redirected HTTP response. What is Nokogiri getting that the browser isn't (or vice versa)?

> JS can introduce more dynamic HTML, but view-source should always represent any non-redirected HTTP response

That is both true and false. Because the JS can introduce dynamic content, the source returned by the HTTP response often doesn't match the source that is rendered by the browser itself. In many cases, a site will return a skeleton (just HTML) and then make an Ajax request to populate it. In my case, it was just the skeleton HTML with a few hundred lines of JS plus a long string of JSON

Re: PhantomJS: Archiving the project, suspending development

#32

Post by tonto »

TheAceOfHearts wrote: post #17

Some people are mentioning headless Chromium, so I wanna mention another tool I've used to replace some of phantomjs' functionality: jsdom [0]. It's much more lightweight than a real browser, and it doesn't require large extra binaries. I don't do any complex scrapping, but occasionally I want to pull down and aggregate a site's data. For most pages, it's as simple as making a request and passing the response into a…

Right before release is a bad time to realize there are problems with your build

Re: PhantomJS: Archiving the project, suspending development

#33

Post by Analemma_ »

There is one thing about this that saddens me: PhantomJS still starts up much faster than headless Firefox or Chrome, at least for me, which makes some of our integration tests take a long longer than they should.

Has anyone here figured out any tricks to get headless Chrome booted fast?

Re: PhantomJS: Archiving the project, suspending development

#34

Post by chucksmash »

nkozyra wrote: post #23
epx wrote: post #9

Earlier quoted context omitted.

Not sure whether it is as easy to use as PhantomJS.

I'd say Puppeteer is on-par with Phantom for ease of basic use. It has a richer, deeper API, of course, but at its core it's modern Javascript.

+1 on Puppeteer. Using it for something now. For small projects, the ability to have the JS you want to run within the context of the page itself live side by side with your browser instrumentation code feels magical. Head and shoulders nicer experience than in cases where half of your logic is second class code-as-a-string (e.g. trying to work directly with Gremlin Server from a non-JVM language by POSTing Groovy-as-a-string)

Re: PhantomJS: Archiving the project, suspending development

#35

Post by vaviloff »

Analemma_ wrote: post #33

There is one thing about this that saddens me: PhantomJS still starts up much faster than headless Firefox or Chrome, at least for me, which makes some of our integration tests take a long longer than they should. Has anyone here figured out any tricks to get headless Chrome booted fast?

Also PhantomJS was a single statically linked binary with no dependencies that you could literally drop into a server and run scripts at once.

Re: PhantomJS: Archiving the project, suspending development

#36

Post by mrskitch »

petercooper wrote: post #4

Two alternatives: Headless Chrome with Puppeteer: https://github.com/GoogleChrome/puppeteer Firefox-based Slimer.js: https://github.com/laurentj/slimerjs (same API as Phantom which is useful if using a higher level library like http://casperjs.org/ )

I maintain a puppeteer-as-a-service repo here: https://github.com/joelgriffith/browserless. It’s pretty feature rich at this point, allowing you to specify concurrency, sessions timeouts, and comes with a robust IDE (which you can play with here: https://chrome.browserless.io).

I’m working on building out a serverless model, which is the holy grail of headless workflows, but it’s a bit more challenging to operationalize than one would think.

I’m hoping that these efforts will lower the bar for folks wanting to get started with puppeteer and headless Chrome!

Re: PhantomJS: Archiving the project, suspending development

#37

Post by mrskitch »

tnolet wrote: post #24

I’m super biased in this, having spend considerable time programming against PhantomJs, Selenium and now Headless Chrome / Puppeteer for my startup https://checklyhq.com . This whole area of automating browser interactions is an extremely hard thing to get stable. In my experience, the recent Puppeteer library takes the cake but PhantomJs is the spiritual father here. I will not talk about Selenium for blood pressure…

Have you seen _my_ startup (https://browserless.io/). The stability part is something I’m trying to solve once and for all with this project.

Re: PhantomJS: Archiving the project, suspending development

#39

Post by chrisweekly »

tonto wrote: post #32
TheAceOfHearts wrote: post #17

Some people are mentioning headless Chromium, so I wanna mention another tool I've used to replace some of phantomjs' functionality: jsdom [0]. It's much more lightweight than a real browser, and it doesn't require large extra binaries. I don't do any complex scrapping, but occasionally I want to pull down and aggregate a site's data. For most pages, it's as simple as making a request and passing the response into a…

Right before release is a bad time to realize there are problems with your build

Better than right after release.

Re: PhantomJS: Archiving the project, suspending development

#40

Post by pbiggar »

Analemma_ wrote: post #33

There is one thing about this that saddens me: PhantomJS still starts up much faster than headless Firefox or Chrome, at least for me, which makes some of our integration tests take a long longer than they should. Has anyone here figured out any tricks to get headless Chrome booted fast?

I have the same problem, so it's not just you.
Post reply on HN