Live data from Hacker News

Ghost.py is a webkit web client written in python

jeanphix.me

11–17 of 17 posts

Re: Ghost.py is a webkit web client written in python

#11
post #4

Could someone describe the key differences to Phantomjs (indirectly referred to in credits via Casper.js) Phantomjs has just recently stopped their Python support.

Phantom js has to run as it's own process, so no support for node or anything similar. This looks like it can run within something like django.

Re: Ghost.py is a webkit web client written in python

#12

I have done this somewhat more directly with PyQt4. I wonder wether it's useful to have this layer in between. Whereby it seem to have some useful tools. This is the main code: * https://github.com/jeanphix/Ghost.py/blob/master/ghost/ghost... * https://github.com/jeanphix/Ghost.py/blob/master/ghost/utils... If you are interested, this is some own code where I just use PyQt4 directly: * https://github.com/albertz/goog…

I once used a similar technique to mass download some elevation data files from the USGS website.

I forget the exact details, but fetching the URL just kicked off a job on their server and returned some Javascript to execute. The Javascript did "something" while the data was being fetched/processed on the server, and eventually decided when it could start the real download.

I spent a while trying to figure out the Javascript, but finally came up with the PyQt/WebKit approach.

It's the ugliest download code in the world, but it's up on GitHub: https://github.com/jl2/GIS-Stuff/blob/master/map_download/ne...

I'm not sure how useful something like Ghost would have been. I was basically using it as a glorified urllib.request, though, and it doesn't look like that's the main use case for Ghost.

Re: Ghost.py is a webkit web client written in python

#15
post #4

Could someone describe the key differences to Phantomjs (indirectly referred to in credits via Casper.js) Phantomjs has just recently stopped their Python support.

Phantom js has to run as it's own process, so no support for node or anything similar. This looks like it can run within something like django.

Is it possible to use Phantom JS or Ghost.py to access a web site that is using Facebook Connect? Just wonder if anyone has tried it yet.

Re: Ghost.py is a webkit web client written in python

#16
post #4

Could someone describe the key differences to Phantomjs (indirectly referred to in credits via Casper.js) Phantomjs has just recently stopped their Python support.

Phantom js has to run as it's own process, so no support for node or anything similar. This looks like it can run within something like django.

You can use PhantomJS with Node e.g. with child_process and messages via stdout. Works pretty well. Running this in its own process context might actually be a benefit e.g. when you spawn multiple phantomjs browsers

There is a project on github where this was taken a step further via dnode ( https://github.com/sgentle/phantomjs-node ) - you get access to the phantomjs objects, get/set properties and access the phantomjs api methods.

Will certainly have a deeper look into ghost.py

Re: Ghost.py is a webkit web client written in python

#17

Is it possible to run jQuery inside Ghost.py to parse HTML with it?

Why not use pyquery (https://bitbucket.org/olauzanne/pyquery/)? Plays nice with WebTest (http://webtest.pythonpaste.org/) and django-webtest (https://bitbucket.org/kmike/django-webtest/).
Post reply on HN