Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PhantomJS: Stepping down as maintainer

groups.google.com

Post reply on HN
Print view View original

81–90 of 236 posts

Re: PhantomJS: Stepping down as maintainer

#81

Post by pavlov »

mrfusion wrote: post #61

Naive question here. What makes headless mode so difficult?

It's a very good question. One might imagine that the browser renders everything into a buffer at some point, and you could simply ask the engine to give you a pointer to that data.

The reality is very different. WebKit/Blink rendering is intimately tied with the graphics system of each platform, in particular through the use of native widgets and native window system compositors.

For example, on the Mac, a lot of compositing within the browser window is done using Core Animation layers. This is a really good idea for performance, because it leverages the work done by Apple to improve their GUI performance.

The downside is that capturing the output becomes very tricky when the browser doesn't do the final compositing. Previously this didn't really matter because 99.99% of browser rendering is for end users and they don't need to capture the output (or if they do, they would just use platform GUI functionality like screen capture).

An increasing demand for headless rendering has effectively forced browser engine teams to rethink some of the internal APIs so that a pipeline can be built to capture the final rendering.

Re: PhantomJS: Stepping down as maintainer

#82

Post by robmcm »

Doctor_Fegg wrote: post #75

> Chrome is faster and more stable than PhantomJS. And it doesn't eat memory like crazy. Wait, can someone tell me where to download this doesn't-eat-memory-like-crazy version of Chrome? Activity Monitor is showing me 2GB of Chrome processes right now and that's even with The Great Suspender having paused almost all my tabs.

Is it Chrome eating memory of the websites you have visited? Suspended tabs suggests issues, which may not free up till you close them.

Re: PhantomJS: Stepping down as maintainer

#83

Post by RandomBookmarks »

wfunction wrote: post #28
redsummer wrote: post #4

I'm curious - What is the utility of headless browsers? Are there people who earn money by getting it to automatically fill out forms, enter competitions etc?

While we're on the topic, does anyone know where one might find scripts to scrape bank statements so you don't have to download them manually every month? (This is one thing I would find headless browsers useful for...)

>Scrape bank statements...

There is Kantu: https://www.a9t9.com/kantu/web-automation

It uses screenshots and OCR to automate web browsing and scraping, to you do not even have to "touch" the DOM. What you do is you simply draw a frame around the areas that you need to have extracted and OCR'ed. It also works with PDFs.

Re: PhantomJS: Stepping down as maintainer

#84

Post by jffry »

ohitsdom wrote: post #69

PhantomJS is a great tool, I implemented it as a PDF report generating system. But will Chromium be able to replace it in this regard? Will Chromium have paging features? Will it be able to repeat table headers when a table body content extends to the next page?

I can't answer all of your questions, but this thread may interest you: https://news.ycombinator.com/item?id=14102248

Re: PhantomJS: Stepping down as maintainer

#87

Post by gildas »

Thanks Vitaly for your work! SEO4Ajax would certainly not exist without PhantomJS. It helped us to deliver the service efficiently at the time.

Unfortunately, we had quite a few compatibility issues with it leading us to migrate to Chrome (with xvfb) one year ago. Since then, we must confess that we are very happy of this choice. Chrome is indeed very stable, fast and more importantly for us, always up-to-date.

Re: PhantomJS: Stepping down as maintainer

#88

Post by lightbyte »

pricechild wrote: post #33
wfunction wrote: post #28

Earlier quoted context omitted.

While we're on the topic, does anyone know where one might find scripts to scrape bank statements so you don't have to download them manually every month? (This is one thing I would find headless browsers useful for...)

Every UK bank I've used allows some format of csv/qif/ofx export. Are you in the US?

My bank offers that, but it costs I believe ~$15 a month.

Re: PhantomJS: Stepping down as maintainer

#89

Post by asdfgadsfgasfdg »

demonshalo wrote: post #20
redsummer wrote: post #4

I'm curious - What is the utility of headless browsers? Are there people who earn money by getting it to automatically fill out forms, enter competitions etc?

A great example would be PDF generation for things like invoices. Rather than generating a pdf with something like PHP or Java. Render a regular html page with all the css you want (super easy compared to drawing a PDF using PHP) and then proceed to use a pdf printer on that page. You could run such a thing as a microservice using a headless browser or PhantomJS. There are probably better ways to do this but that's o…

This sounds horrible.. In general all these html->pdf ways of generating pdfs sounds horrible -- why don't people use latex for this?

Re: PhantomJS: Stepping down as maintainer

#90

Post by slig »

tomp wrote: post #46
redsummer wrote: post #4

I'm curious - What is the utility of headless browsers? Are there people who earn money by getting it to automatically fill out forms, enter competitions etc?

I wrote a PhantomJS script to download data from my bank accounts. They offer no API and disfunctional text-based exports, their websites are ridden with "good" (=terrible) web & security practices, like 3-characters-of-the-whole-password authentication, single-tab sessions, frames, etc. that makes it pretty much impossible to scrape with Python but relatively easy with a fully-fledged browser (although that still re…

If your bank has a mobile app, it might be easier to MITM and figure out their API and use it directly.
Post reply on HN