Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

From Python to Go and Back Again

docs.google.com

Post reply on HN
Print view View original

121–130 of 167 posts

Re: From Python to Go and Back Again

#121

Post by twic »

falcolas wrote: post #57
thristian wrote: post #33

Earlier quoted context omitted.

The way I deploy Python apps at $EMPLOYER: - CI system detects a commit and checks out the latest code - CI system makes a virtualenv and sets up the project and its dependencies into it with "pip install --editable path/to/checkout" - CI system runs tests, computes coverage, etc. - CI system makes a output directory and populates it with "pip wheel --wheel-dir path/to/output path/to/checkout" - Deployment system dow…

The way I deploy Go apps at $EMPLOYER2: - go get - go test - go build - copy to target It's possible with Python, it's easier with Go. It's a place where we could use a lot of progress.

Presumably, though, what both of you actually do is:

build.sh

Once you'd done the up-front work of figuring out how to do deployment sanely, it became equally easy for both of you.

Re: From Python to Go and Back Again

#122

Post by trentnelson »

windlep wrote: post #42
sethammons wrote: post #41

Earlier quoted context omitted.

I should reach out to our team that took python/twisted dealing with sockets and lots of concurrency and ported to Go and see if they would put together a similar presentation. Our case is a bit different, but we saw over 130x improvement in throughput going to Go. While they were in there, they increased monitoring, stability, and maintainability. More case studies to help others make informed choices. Sending that…

I should note, we don't care about throughput for the most part. Our constraint is purely the memory use of holding open the connections. The aim is to hold as many connections as possible within 10-20% of the machines RAM, and not exceed it. As such, we need to be careful about resource usage and spikes. Goroutines feel cheap, but if you're holding 140k connections, and just 20k of them do something that spins up a…

Why 10-20% of RAM? How much RAM does each machine have? What else are they doing? Are they virtualized?

Re: From Python to Go and Back Again

#123

Post by trentnelson »

alextgordon wrote: post #64
gtrubetskoy wrote: post #61

I am surprised the GIL hasn't been mentioned once in the presentation or any of the comments.

node.js has a GIL and nobody talks about that either. The GIL is not relevant because Python is so slow that running it in two threads is hardly an improvement.

I get linear scaling with PyParallel: http://pyparallel.org/

Re: From Python to Go and Back Again

#124

Post by joepvd »

eliben wrote: post #44
latch wrote: post #9

Most people who rewrote their apps from X to Go and saw improved performance and readability benefited much more from the rewrite than they did from Go. At best, the fact that Go has a relatively weak ecosystem, means that they had to write from scratch a lot of things they were getting for free in X. But, because in X is was a library, they only used 5% of the features, but paid a high performance cost and had a com…

> Most people who rewrote their apps from X to Go and saw improved performance and readability benefited much more from the rewrite than they did from Go Precisely, and isn't this presentation the perfect example of this phenomenon? * Initially we had an implementation in language X * We then rewrote it in language Y - the lessons learned by making the system anew (this time knowing the exact problematic spots, what…

Possibly, the better understanding of the problem domain would make a rewrite from scratch better than the existing stuff. That the improvements have little to do with the language in which the newly conceived solution was implemented.

Re: From Python to Go and Back Again

#125

Post by Cyph0n »

gaius wrote: post #116
hyperbovine wrote: post #113

This page displays exactly nothing with JavaScript disabled. I realize that no js makes me something of a Luddite, but there are solid reasons for turning it off, particularly on mobile. Is it really too much to ask of google that the text of a presentation in some way live inside HTML? Novel concept, I know.

Javascript is like Flash, it's great for the author of a page to show off, or force an ad on you, but how does it actually benefit the end user? Not at all, never has.

Really now. JS can potentially improve UI and UX, providing users with a smoother experience.

Re: From Python to Go and Back Again

#126

Post by peteretep »

thwd wrote: post #109
peteretep wrote: post #88

Earlier quoted context omitted.

> the middlebrow dismissers The totally misplaced condescension is one of many reasons the Go community appears to be a net negative.

Come visit, we're quite friendly! You know what really grinds our gears, though? People who don't read documentation. If you don't read documentation, you'll get a negative vibe. Because your question is literally sitting at the top of the FAQ. It's been asked 2^1024 times before. WHY WON'T YOU READ THE DOCS?

This is just bizarre.

Re: From Python to Go and Back Again

#127

Post by tome »

latch wrote: post #117
laumars wrote: post #114

Earlier quoted context omitted.

Weird, I could have sworn I read that HN was written in Haskell. That error aside, my points were still valid. Most of latch's post were an exaggerated and largely inaccurate generalisation. Personal opinions of Go aside, it's daft to argue "most static languages [are] cumbersome, error prone, slow, inflexible and difficult to test." I know I've been heavily down voted in my previous comment, but I've developed in we…

I wasn't going to reply, but your misquote is dishonest. I specifically said that it was those thing with respect to dealing with web requests within the further scope of talking to the database and dealing with user input. I also pointed out cases where Go's either "great" or "good", with one of those being some types of web services (thus further scoping my "generalisation".)

> I specifically said that it was those thing with respect to dealing with web requests within the further scope of talking to the database and dealing with user input.

Although I corrected your interlocutor on a factual inaccuracy, I actually fully agree with him/her on the principle, and as the author of a strongly-typed database access library (Opaleye for Haskell) I'm in a very good position to!

Re: From Python to Go and Back Again

#128

Post by fauigerzigerk »

0x434D53 wrote: post #55

Go is one of my working languages. Like with every other language (Python, OCAML, C#, some Java, Swift) i have a love-hate relationship with go. What i agree on: What i agree on with the presentation: Concurrency using goroutines and channels is f... hard besides very primitive scenarios. Even fork-join isn't that easy. There also the lack of expressiveness hurts: It's nearly impossible to build higher level abstract…

I agree that error handling in Go is a headache. But since we are forced to handle every single error where it occurs, we can at least make the best of it and add context information before returning it.

I'm using a small utility library to wrap the original error and add function name, line, file name and optionally a descriptive message that explains what failed.

Re: From Python to Go and Back Again

#130

Post by benji-york »

trentnelson wrote: post #122
windlep wrote: post #42

Earlier quoted context omitted.

I should note, we don't care about throughput for the most part. Our constraint is purely the memory use of holding open the connections. The aim is to hold as many connections as possible within 10-20% of the machines RAM, and not exceed it. As such, we need to be careful about resource usage and spikes. Goroutines feel cheap, but if you're holding 140k connections, and just 20k of them do something that spins up a…

Why 10-20% of RAM? How much RAM does each machine have? What else are they doing? Are they virtualized?

He said "within 10-20% of the machines RAM", i.e., utilizing 80-90% of the machine's RAM, without exceeding it.
Post reply on HN