Live data from Hacker News

From Python to Go and Back Again

docs.google.com

151–160 of 167 posts

Re: From Python to Go and Back Again

#151
post #125
post #116

Earlier quoted context omitted.

Javascript is like Flash, it's great for the author of a page to show off, or force an ad on you, but how does it actually benefit the end user? Not at all, never has.

Really now. JS can potentially improve UI and UX, providing users with a smoother experience.

I notice you are careful to say "potentially" because it never actually happens.

Re: From Python to Go and Back Again

#152

Earlier quoted context omitted.

The way I deploy Go apps at $EMPLOYER2: - go get - go test - go build - copy to target It's possible with Python, it's easier with Go. It's a place where we could use a lot of progress.

It seems weird how you can't easily package python into an executable without Docker.

http://nuitka.net/

Re: From Python to Go and Back Again

#153
post #80

what is strange is that everyone seems to be using Python 2.7 in some form. There is all this new work being done in Python 3 asyncio but there is nothing that is being used... and this is from a guy who pretty is a core developer in several python projects. I look at Ruby or Go... or even Java and every new language feature has a much more rapider adoption curve. is this a pretty solid statement that the entire Pyth…

We used Python 2.7 for pypy compatibility. I really dig the new async/await primitives added to Python 3.5 and think the asyncio path is the best path forward. I hope the work to get all the latest asyncio stuff working on PyPy picks up some steam.

The main issue with asyncio is it is incompatible with some of the best packages in the ecosystem atm, e.g sqlalchemy

Re: From Python to Go and Back Again

#154

There are people who have been working in Go for years, successfully, but who don't post comments with the same frequency and dogged determination as the middlebrow dismissers. Both Python and Go are fine. They both have their strengths and weaknesses. I personally wouldn't write a web app in Go (at least, anything beyond the most basic admin interface). I also personally wouldn't write a very large and complex Pytho…

You can write Fortran in any language.

Re: From Python to Go and Back Again

#155

I am surprised the GIL hasn't been mentioned once in the presentation or any of the comments.

node.js has a GIL and nobody talks about that either. The GIL is not relevant because Python is so slow that running it in two threads is hardly an improvement.

This is the weirdest and most nonsensical description of the GIL I've ever seen, and assuredly one coming from a Python/JS non-programmer. In both languages the serialization of runtime is a feature explicitly designed to simplify the programming model and its implementation.

There is only one case where the serialized runtime presents a problem, and that is in CPU-bound 90s-style shared memory parallel computation that the industry as a whole has been trying to escape for the past 20 years, because thinking about individual threads and the lifetimes of shared memory allocations turned out to be an incredibly shitty abstraction.

Even if you want to shoot yourself in the foot, both Python and Node.js provide facilities to allow e.g. concurrent array access (in Python via the multiprocessing package). The reason those approaches aren't more popular in those languages is exactly because the model itself is defective. Anyone worth their salt working in a computation-heavy domain stopped writing explicit threading code a long time ago.

Re: From Python to Go and Back Again

#156
post #27

Earlier quoted context omitted.

i enjoy using ocaml a lot, but the lack of a good orm and the bad windows support are indeed very painful. the tooling used to be bad, but at least under linux i'm pretty happy with it these days for my small hobby projects. it's one of those languages that i'd love to be able to use at work; i miss the type system when i'm doing c++.

Having worked with and without ORMs, I'm not sure what a "good ORM" is. On the other hand, a somewhat type-safe generic SQL query builder would be very nice to have.

the other end of the process is pretty useful too - unmarshalling raw sql results into ocaml objects or records, converting foreign keys into references, etc.

Re: From Python to Go and Back Again

#157
post #100

Earlier quoted context omitted.

Did you try requests_futures lib? It's all about async and network speed with crawling. Not so much cpu.

I thought it would be IO bound (that's why I started with Python at first place), but since I was extracting links as well and working a bit on graph it turned out to be more CPU intensive. But well, maybe I could have written better code, better libraries, maybe multiprocessing (would have been painful though with multiprocessing). I do admit, I didn't look much into how I could improve it within Python. I just went…

well.. extracting links etc is super fast with lxml's xpath. It is written in C, and I don't think it would be faster if you write your own parser.

For example, to extract links from hacker news homepage, you would just do

    xpath('//tr/td[@class="title"]/a/@href')
This will be really fast. You can do it even faster with a more specific xpath. I extracted about 10k links a second from documents this way and was still network bound. Usually you are primarily limited by websites throttling you.

Re: From Python to Go and Back Again

#158
post #157

Earlier quoted context omitted.

I thought it would be IO bound (that's why I started with Python at first place), but since I was extracting links as well and working a bit on graph it turned out to be more CPU intensive. But well, maybe I could have written better code, better libraries, maybe multiprocessing (would have been painful though with multiprocessing). I do admit, I didn't look much into how I could improve it within Python. I just went…

well.. extracting links etc is super fast with lxml's xpath. It is written in C, and I don't think it would be faster if you write your own parser. For example, to extract links from hacker news homepage, you would just do xpath('//tr/td[@class="title"]/a/@href') This will be really fast. You can do it even faster with a more specific xpath. I extracted about 10k links a second from documents this way and was still n…

I was using beautifulsoup with lxml backend I believe. I should have mentioned earlier. There were some other graph manipulation stuff too, like favoring links with more inlinks, keeping web crawler polite but still busy by looking at other domains. This is more expensive that extracting links I guess. I had a submission deadline, but whatever I tried in that time with Python didn't work. It was just easier to write faster code in Go (except maybe where regex are involved, now I remember I used some Go markup parser instead that is now in their library).

Re: From Python to Go and Back Again

#159

Go is one of my working languages. Like with every other language (Python, OCAML, C#, some Java, Swift) i have a love-hate relationship with go. What i agree on: What i agree on with the presentation: Concurrency using goroutines and channels is f... hard besides very primitive scenarios. Even fork-join isn't that easy. There also the lack of expressiveness hurts: It's nearly impossible to build higher level abstract…

I agree that error handling in Go is a headache. But since we are forced to handle every single error where it occurs, we can at least make the best of it and add context information before returning it. I'm using a small utility library to wrap the original error and add function name, line, file name and optionally a descriptive message that explains what failed.

You are not forced to handle errors. Or do you check for the errors returned by fmt.Print? I guess not.

This was my point: In many areas go forces the developer to the right thing (no unnecessary imports, gofmt...) and does not rely on developers discipline. But when i comes to error handling it does.

What i would wish for is some extended error handling supported by the compiler. I don't want a stack trace, but the compiler easily could produce for example a line number where the error was returned.

Re: From Python to Go and Back Again

#160
> Go is generally 50-100x faster than CPython

(slide 20)

I find that a weird sentence in an otherwise carefully written article. The author is talking about writing software which does a lot of socket IO. So I would expect the performance discussion to make some reference to this; I assume what he's talking about in the quote is the behavior of pure CPU-bound code but he doesn't discuss to what extent this is really relevant to his project.

Post reply on HN