Live data from Hacker News

Why we switched from Python to Go

getstream.io

71–80 of 406 posts

Re: Why we switched from Python to Go

#71

Why does anyone write web apps in Python? PHP? Ruby? Modern Java and Go are so much faster than the alternatives that it's stupid to consider anything else if performance is important. Golang and Java can manage over a half million HTTP responses a second. Node is pretty fast but why bother when Java is many times more mature in features, tooling, and and supports concurrency... And uses less ram and is usually faste…

Why does anyone write web apps in Python? PHP? Ruby?

First: Because squeezing every last nanosecond's worth of performance out of your web app is actually an extremely rare problem to have. And if you truly cared about performance over programmer convenience, you'd practice what you preach and build your web apps in hand-rolled assembly, but I'd bet a lot of money that you don't do that.

The typical web application -- I'd be willing to bet over 99.999% of all deployed production web applications serving requests today -- has a bottleneck at the database and the network that dwarfs any overhead from language performance.

I remember a bit over ten years ago when there was debate in the Python web world about which templating engine to use to generate HTML. And people argued endlessly over microbenchmarks of them, to figure out which was fastest, but I remember one blog post which showed a pie chart of time spent in the average request/response cycle. Nearly all of it was accessing the database, and template rendering was a tiny, tiny sliver, so the author humorously labeled it "obviously this is the part we need to focus all our optimization work on". Language choice is similar.

Second: Because what else your company does matters. Where I work, web applications are how we expose data and interfaces to that data. But there's a gigantic stack behind that, of data intake, data parsing, data processing, analytics, the whole nine yards. It's all in Python, because Python has hands-down the strongest ecosystem of any popular programming language for that stuff. So the web applications which serve as the interfaces to the data are also written in Python; it means we have one language to worry about, one language to work in, one language every software engineer knows. I've been pulled onto projects doing things that didn't involve web applications at all, and I've been able to be productive because those projects were still in Python, and I could read code and get up to speed on what was happening, and take care of mundane things for a more domain-experienced person whose domain expertise was then free to apply to things I couldn't do.

Third: Because programmer convenience really and truly does matter. When I first started doing this nearly twenty years ago, people posted comments like yours, incredulous at the idea that someone would use PHP or Perl given their performance characteristics compared to Java (or C -- plenty of web apps used to be written in C!). But even then we knew: servers are cheaper than people. The average salary of a quality software engineer (or "web developer" as we were known then) would buy you a lot of compute time, either on your own (in-house or colo'd) metal, or nowadays on someone else's cloud.

So you choose based on convenience to humans. PHP, for all its faults, was an incredibly convenient language to write web apps in, and compared to the usual CGI model that preceded it, was a breath of fresh air when it took off. Today, frameworks written in Python, Ruby, PHP, etc. are similarly in a good position compared to more heavyweight things like the Java world (which for better or worse is still suffering the lingering effects of its mid-2000s "enterprise" reputation), or even Go (which is still young and still seems to come up short, both language- and ecosystem-wise, on some of the things actual working web programmers want. In particular, programmer-friendly ORMs in statically-typed language really really really want generics or a good equivalent, and Go's historic attitude toward that has not been great.

Re: Why we switched from Python to Go

#72
post #69
post #68

Earlier quoted context omitted.

Yeah, plus even if performance is important, the app layer isn't necessarily the best place to optimize. It doesn't really matter how fast you sprint between database calls if the database and its IO dominate your site's performance profile, which they often do...

Everyone seems to say this and also to write really slow websites.

The vast majority of slow websites I've seen written with RoR were slow because the DB later want optimised. 1+n query problems, pulling way more data than needed and then processing it in Ruby, missing indices etc.

Re: Why we switched from Python to Go

#74
post #21

Earlier quoted context omitted.

> Go also doesn't have the problem of conflicting system level packages. Sure, as long as you're not using CGO and dynamic linking. Otherwise you'll get the exact same problems as the others. > Go doesn't have to deal with interpreter dependencies As long as Go is forward compatible this will old true, but I don't think this was the point of the comment. 'go get' is a half-baked package manager and yes it fetch packa…

>There is a reason why some gophers are working on an actual package manager... Finally! I had to deal with a golang project once, the dependency management made me want to pull my hair out after using Bundler and Cargo

Yes! Once you've used Bundler and Cargo anything else seems just silly and unacceptable.

Re: Why we switched from Python to Go

#75
post #55

Oh, terrific, another one. The only solid reason this article gave was the second one and, really, is the only one that would ever make sense: If the language one's using becomes the bottleneck then, by all means, change it[1]. Everything else is fluff and opinions. Every single other reason is stuff that is entirely subjective and/or python already has and even the article itself acknowledges, at least for one "reas…

I don't think that performance is the key reason to switch from Python to Go (although it is nice): as someone who switched from Python to Go, what really made it worthwhile for me was knowing that my code is significantly more likely to be correct when it runs. Go's typing is much nicer than Python's, for this use case.

Performance is just gravy.

Re: Why we switched from Python to Go

#76
post #56

I think there's a great value in code being straight forward and simple and that is very much not appreciated among many programmers. A good language enables you to "compress" your code without making the code flow impossible to follow. A bad language encourages you to obfuscate the flow of the program using wacky abstraction techniques. Go doesn't exactly hit the sweet spot for semantic compression due to lack of co…

[deleted]

Re: Why we switched from Python to Go

#77

Author here. Super cool to see this post on HNews. For those of you working with Go. One of the guys on our team wrote a little tool called VirtualGo. It's pretty handy when you're working on multiple go based projects ( https://github.com/getstream/vg )

Thank god something like this exists. I gagged at the workspace structure when I first started Go - it gave me enough pain starting out one weekend that my interest faded away. I've been looking for a reason to try it out again and a bunch of comments here got me really excited again.

Re: Why we switched from Python to Go

#78

How do these shallow articles get upvoted so much ? they don't have much specific information except very generic "developer productivity". Let me give a specific example where moving to Go really helped our tooling: Go has some great interfaces, specifically their net & ssh client. In order to perform operations against some machines, we have to tunnel through bastions, however we'd also like the tool to work when a…

Actually moving to Go was a very thorough and long process. We did small example projects and tried out many of the libraries we needed. The blogpost is more of a high level summary of why Go is such a great language.

I thought it was very thoughtful, detailed, well written and touched on all the points I would need to make a similar decision if it ever came down to it. Thanks for writing it.

Re: Why we switched from Python to Go

#79
post #37

Earlier quoted context omitted.

> conflicting system level packages In Python, if you have control over the target system, this is solved with virtualenv. You can install whatever versions of libraries you want in there with pip without causing conflicts with system level packages. If you intend to ship to end-users, yeah, you are kinda screwed. You are stuck using one of the various "freeze" methods, which, in my experience, kinda blow.

Solved as long as there aren't native dependencies not managed by pip, which there often are.

Wheels solved this problem in 2013.

For context, you can install opencv, tensorflow, ROS, matplotlib, and the entire scipy stack in a virtualenv, with no external dependencies, using wheels.

This means that you can generate images, train a machine learning algorithm on them, compare the results to conventional CV algorithms, and display them in an ipython notebook all from a venv. There's a huge amount of C++ and even qt integration in that pipeline, all isolated.

It won't be maximally performant (ie for massive training), but for that you'd want distributed docker deploys or similar anyway.

Re: Why we switched from Python to Go

#80
post #47

If you can switch from Python to Go, you weren't using Python anyways, you were use Gothon or Javthon. If you want static binaries, great tooling and an excellent imperative and functional language, try F# with mkbundle. The compelling reasons to use go are shrinking.

one problem: windows

F# runs everywhere. I dev on OSX using Rider and VS Code. Deploy to Linux.

http://fsharp.org/use/linux/

http://fsharp.org/

> F# runs on Linux, Mac OS X, Android, iOS, Windows, GPUs, and browsers. It is free to use and is open source under an OSI-approved license.

Post reply on HN