Live data from Hacker News

Why we switched from Python to Go

getstream.io

281–290 of 406 posts

Re: Why we switched from Python to Go

#281

Earlier quoted context omitted.

Wait! Is this a go 1.9 thing? I've been using 1.83 all this time.

I don't think it's a 1.9 thing - I've got code from 2017-02 which uses it (the quickest example I could find in my repos)

WTF? You are absolutely right. I could've sworn I had issues with this before. 25% of my irritation with Go just evaporated (the other 75% is how hard it is to run Delve, but I'll figure that out.)

Re: Why we switched from Python to Go

#282
post #140

Earlier quoted context omitted.

That applies to all languages with exception of C, because they didn't want to standardize it as part of the language, hence we got ANSI C + POSIX instead.

Go provides built in functionality to develop web applications. Things like template rendering, routing, a generic SQL interface, etc. Which are libraries not commonly found on most languages. Python, OTOH, requires libraries like Flask, Jinja2, and SQLAlchemy to provide the same functionality Go provides as defaults. Thats my whole point.

And they're somewhat awkward versions of all the above such that most projects pull in deps like gorilla/* for those things anyway.

PHP has template rendering, routing, and database conns built right in too ;)

Re: Why we switched from Python to Go

#283
post #273
post #198

Earlier quoted context omitted.

Would you kindly share you experience on using Nim for that project?

For that project, we had very specific requirements: easily handle SSL/TLS with contexts and control over self-signed vs certificate checking, JSON processing, speed, nice syntax, and one of the most challenging requirements: statically compiled, linkable against musl and libressl, while still supporting mingw_64 for windows. Only a few languages have flexible compilers that can do this; for example, rust can't (afai…

You can also check out this: [redacted] But there's almost no features in it as I'm lazy :) I think that the best feature in this lib is a python-like range type: [redacted]

Re: Why we switched from Python to Go

#284

Python was my entry into the programming world, and I've been an evangelist ever since... Or I was until I ran into distribution and parallelism. Since then, Nim has been my go-to language of choice. It is all that Python was, plus unbelievable speed, compiling to shippable binaries, and some other cool language features that admittedly, are still beyond my scope of abilities. Still quite lacking in libraries compare…

I use both Go and Python regularly, and both have their uses and issues. Go for me is a rather simple tool that doesn't get in my way, and just lets you do stuff - on it's own it's relatively boring. While it's probably not everyone's cup of tea, I like it's simplicity. It does have some downsides the article here mentions like package management and versioning. I understand why vendoring is being pushed as 'the official solution', but it still feels backwards to me.

The thing that eventually makes me grab Go time and time again is how damn easy it is to drop into code of some library - even (and maybe especially) the standard library, something I rarely did or do in any other language. With Go there's little friction to do this, for multiple reasons. The source of truth for pretty much any project is almost exclusively Github, godoc.org documentation has links directly into the source-code for every single function and struct, and it forces you to set up a dev env that let's you understand it's structuring, where your libraries and it's code will end up. About the last part I had some mixed feelings at first, but after a while you appreciate it that the libraries you're using aren't tucked away somewhere in some system directory. Add to that that Go is a simple, pretty readable language, and you end up with a very transparent library system. Would I ever even think about jumping into the paramiko library (which I used on multiple occasions)? Not at all, so I didn't. The first time I used the crypto/ssh lib, I also didn't think about jumping into the code, but somehow, I did because it was the natural thing to do, and made me understand the internals and SSH a lot better (paramiko was slightly easier to get started with though).

Another thing I really like is that the language encourages not writing 'applications' but rather libraries that can be reused by virtually anyone, and your final application will be a relatively small front-end shim for it. While this is not hard in Python, creating a separate library for smaller stuff somehow always felt as overhead, while in Go it feels like the natural thing to do. In python, there's yet another ton of overhead if you want to add something to PyPI - while in Go it's a `git push` away. This also creates serious issues and pitfalls, but it makes contributing a library a lot easier.

Re: Why we switched from Python to Go

#285

Why does anyone write web apps in Python? PHP? Ruby? Modern Java and Go are so much faster than the alternatives that it's stupid to consider anything else if performance is important. Golang and Java can manage over a half million HTTP responses a second. Node is pretty fast but why bother when Java is many times more mature in features, tooling, and and supports concurrency... And uses less ram and is usually faste…

The answer is that for a huge variety of software, performance is not important, or perhaps is only important for a subset of the application. My personal experience is that the dynamic languages you've laid out generally have frameworks that are extremely conducive to rapid prototyping (Django is my favorite). I've seen and done the dance many times -- start with a Django/Rails/Laravel app, get a free admin and buil…

And then you end up with slow applications or websites. Fast response times under high load can pay off and easily recoup longer development times. And it's much harder to change the system once you're successful.

Cost difference is another topic. While servers can be cheaper than developers, saving 90% of server cost can definitely clear some budget for enhancements.

Re: Why we switched from Python to Go

#286
post #137
post #82

Earlier quoted context omitted.

> All the big companies are using Java and Go almost exclusively for high volume endpoints Please cite “all the big companies.” Also exclusively? No seriously, that is a bold claim. Google isn’t one of them because most of those hotepots are still written in C++. I don’t know how you can claim this. Based on some occaion company blog posts, changing just one or two endpoints out of say 100? What about Rust? I also kn…

Pick the Fortune 500 list, take out the SV darlings, the majority of their backend stacks will be a mix of Java and .NET deployments. Easy to find out just by looking at their open job positions.

I'd say that you find a fair share of C++ as well. Especially for high volume endpoints.

Re: Why we switched from Python to Go

#287
post #207

Earlier quoted context omitted.

Well, we've seen for example, Twitter do that. I saw some slides[0] that other day from sometime in 2007[1] where they said they were using Ruby and spawning 180 rails instances to handle 600 requests per second. Like, that's insane. A compiled language can handle that load with just one instance. It's nothing. [0]: https://www.slideshare.net/Blaine/scaling-twitter/3-First_So... Also, if your audience is small enough…

Twitter's issue was more with concurrent IO than anything. This problem was solved about 10 years ago with the release of Ruby 1.9 and since then only one Ruby process per core is required, just like NodeJS. In that time most CPU intensive parts of the web stack have also been re-written as native extensions. Recent benchmarks of a properly setup Ruby environment vs Go/Gin are showing Ruby/Sinatra as having 50% of th…

Contrived benchmarks are not useful. Specially when you have tiny datasets.

If you want to do anything interesting, Python/Ruby are slow as hell, which is why you cannot do anything interesting in them.

For example, while in Go, you can load say 1000 rows from the db and perform some data manipulation on it in the code to get a desired result, you cannot do this in Python because it will very very slow.

So what you do is you write complicated sql queries and essentially offload all your work from the application server(s) on to the database server.

Now imagine that these rows on the database don't actually change very often. You could just load them once, keep them in memory (in a global object), and only update them once in a while (when needed). You can always do whatever search/manipulation operation directly on the data that is readily available and always respond very quickly.

This would be _unthinkable_ if you are using Ruby or Python, so instead you keep hammering your database with the same query, over and over and over again.

Re: Why we switched from Python to Go

#288

I'm surprised to see that people choose anything other than C++ if they care about performance. Are you really trying to profile and optimise python and go? It will never be worth it! Just write the same thing in good modern C++ and you get an automatic 100x speed up for most cases. Then optimise to reach the absolute limits of the hardware. Python and go it seems!

Some people choose Go rather than C++ for the same reason you went with C++ rather than an assembly language.

Most of the time, things only need to be fast enough and trading speed for ease of development, deployment, and maintenance is an easy decision to make.

Re: Why we switched from Python to Go

#289
post #216

Earlier quoted context omitted.

If that's really the case, why do people spawn multiple instances of their app? A python application can be anywhere from 10x to 50x slower than a native application. It also probably consumes at least 5x more memory. Writing the same app in a compiled language is not even an optimization. It's just baseline work to ensure the code is not super slow. Like, if you know you will sort a list of 10 items, choosing quicks…

> why do people spawn multiple instances of their app For concurrency (number of requests handled at once) instead of speed (end-to-end time of of a single request)

If that was the only reason (which it is not), it would still be a very good reason to stop writing code in these languages.

Why waste 10x more memory?

Re: Why we switched from Python to Go

#290
post #165

Why does anyone write web apps in Python? PHP? Ruby? Modern Java and Go are so much faster than the alternatives that it's stupid to consider anything else if performance is important. Golang and Java can manage over a half million HTTP responses a second. Node is pretty fast but why bother when Java is many times more mature in features, tooling, and and supports concurrency... And uses less ram and is usually faste…

I was wondering the same thing, and it frustrates me. I think people obviously enjoy using Python/Ruby or whatever for small scripts, and by inductive reasoning they think they can enjoy programming even larger projects in these languages. They also rationalize the slowness by pretending that performance doesn't matter or that the bottle neck is the database. They abuse the adage about "premature optimization" being…

It's about flexibility.

You will have to close that IT-only part of your brain for a moment, and keep in mind that people create software for a reason. And every nascent project lives or dies by how promptly it adapts against incorrect assumptions or changes in the environment.

Difference in hardware costs do not even enter into the radar. They happen in a different universe that good decision making completely ignores at this point.

After you have a proper solution to a problem, and enough scale so that it's worthwhile to collect those gains, you move your software to another language. It's not a big deal.

Post reply on HN