Live data from Hacker News

Why we switched from Python to Go

getstream.io

271–280 of 406 posts

Re: Why we switched from Python to Go

#271

> Go forces you to stick to the basics. This makes it very easy to read anyone’s code and immediately understand what’s going on. I think all new language design must incorporate this goal going forward. It’s been simply too valuable. When I see something introduced that does x, y incrementally better but makes no effort for the above goal I get sad and move on. It’s easy to get caught up in the technical challenges…

Some people insist on being obtuse (coincidentally, I'm looking in the general direction of a template heavy C++ user), and for worse or better, personal preference is a large factor in language adoption as opposed to what works best for the industry as an engineering practice.

Re: Why we switched from Python to Go

#272
post #216
post #68

Earlier quoted context omitted.

Yeah, plus even if performance is important, the app layer isn't necessarily the best place to optimize. It doesn't really matter how fast you sprint between database calls if the database and its IO dominate your site's performance profile, which they often do...

If that's really the case, why do people spawn multiple instances of their app? A python application can be anywhere from 10x to 50x slower than a native application. It also probably consumes at least 5x more memory. Writing the same app in a compiled language is not even an optimization. It's just baseline work to ensure the code is not super slow. Like, if you know you will sort a list of 10 items, choosing quicks…

> why do people spawn multiple instances of their app

For concurrency (number of requests handled at once) instead of speed (end-to-end time of of a single request)

Re: Why we switched from Python to Go

#273
post #198

Earlier quoted context omitted.

Agreed completely. We're in the process of porting our Python 'shim' (aka agent, at https://Userify.com - plug SSH/sudo key management) to Nim right now, so that we can provide a fully static shim for CoreOS and other minimal distros, and eventually Windows; there are a few languages that can do this cleanly, such as Go, Ocaml, and Lua, but Nim is just blindingly fast and actually pretty fun to code in. Great stuff.

Would you kindly share you experience on using Nim for that project?

For that project, we had very specific requirements: easily handle SSL/TLS with contexts and control over self-signed vs certificate checking, JSON processing, speed, nice syntax, and one of the most challenging requirements: statically compiled, linkable against musl and libressl, while still supporting mingw_64 for windows. Only a few languages have flexible compilers that can do this; for example, rust can't (afaik).

The experience so far has been outstanding. Nim has functioned flawlessly with a minimum of magic. It seems to work very cleanly and the compiler is cleanly integrated, but still swappable (ie between gcc, clang, ming..) Nicely color-coded, too.

Exceptions are caught with full tracebacks, and pre-compile checks quickly point out exact location of syntax errors. (Good, clear error messages are surprisingly missing from many languages.)

Here's an awesome example; in my first day of coding, I was able to replicate python's "+" string concatenator ("hello" + "world" versus "hello" & "world" in Nim) with a one-liner:

    proc `+` (x, y: string): string = x & y
This is pretty amazing; not only is it readable and concise (and more than a passing similarity to python's lambda, of course) but nim comes with the ability to define new operators right in the language, and the compiler raises an error if operators, procs, types, etc would introduce ambiguity.

Nim compiles quickly and its type inference (where it guesses what type of variable you're working with) makes strong typing mostly painless, and you still get all of the advantages (type safety, speed) of static typing.

There are some trade-offs that are made (obviously), but the language designers seem to make trade-offs in favor of speed and robustness over language features -- but this still leaves a lot of room for features.

I also like how the syntax has a lot of similarities to Python's. The only thing I've missed so far is a nim interpreter, so that I can get up to speed faster on the syntax or try things out quickly. The tutorial on the Nim website is definitely not for beginning coders (who would probably be quickly scared off by words like lexical), but it quickly covers the language syntax for experienced coders and seems to borrow a lot of the best ideas from other languages.

Nim is basically awesome. The few downsides are that the standard library is still pretty light (but that gives you an opportunity to build something great and have it be widely adopted), that there's no interpreter, and that the tooling is still a bit lighter than older languages. All of these will be improved with time.

And, it's fast. Really fast. Compare nim in these benchmarks[1] to any other mid-level (or even low-level) language and it really shines. It's generally much faster than Go, for instance.

1. https://github.com/kostya/benchmarks

Re: Why we switched from Python to Go

#274

Python was my entry into the programming world, and I've been an evangelist ever since... Or I was until I ran into distribution and parallelism. Since then, Nim has been my go-to language of choice. It is all that Python was, plus unbelievable speed, compiling to shippable binaries, and some other cool language features that admittedly, are still beyond my scope of abilities. Still quite lacking in libraries compare…

I tried Nim in non commercial capacity and liked it. Wrote an Aho-corasick string matching algorithm in it to see how it fared with Lua/Luajit and it was fairly close in speed. The code was also quite pleasant to write.

I remember that debugging it was a bit of a pain if just not a real option then. Maybe things have changed.

Re: Why we switched from Python to Go

#275
post #250
post #213

Earlier quoted context omitted.

> Most recently the cryptography package just stopped installing on deployment. Had to add a pip upgrade on deploy, [...] Erm... Why the heck are you running pip when deploying software? O_o It should be a build step, not a deployment step.

Most deploy scripts I've seen tend to do this. Which is insane in my opinion, but the python toolset encourage this kind of approach by making it the default easy thing to do. That's one of the things I prefer about Go: the compiler can cross-compile, so I can compile the entire codebase on my osx machine and produce a single statically linked linux binary that I can just copy over scp to the server. Deployment becom…

> Most deploy scripts I've seen tend to do this. Which is insane in my opinion [...]

Also, most programmers don't know a thing about administering servers. Ubiquity doesn't mean that it's a proper way, as you clearly see yourself.

> [...] the python toolset encourage this kind of approach by making it the default easy thing to do.

It's not quite the fault of Python tooling in particular. It's really the easiest way in general. Similarly, the easiest way to deploy a unix daemon is to run it as root user, because the daemon will have automatically access to all files and directories it needs (data storage, temporary files, sockets, pidfiles, log files). Anything else that is easier to manage requires to put a non-zero effort.

> That's one of the things I prefer about Go: [static linking]

Static linking has its own share of administrative problems. It's not all nice and dandy.

Re: Why we switched from Python to Go

#276

Why does anyone write web apps in Python? PHP? Ruby? Modern Java and Go are so much faster than the alternatives that it's stupid to consider anything else if performance is important. Golang and Java can manage over a half million HTTP responses a second. Node is pretty fast but why bother when Java is many times more mature in features, tooling, and and supports concurrency... And uses less ram and is usually faste…

> Why does anyone write web apps in Python? The new generation of Python web frameworks is pretty fast (check out Sanic[0][1] for example) , Python has a huge growing ecosystem, you get rapid time to marked and if you find some part of your application to be the bottleneck, you can replace it with C/Rust, which you probably don't need, because your company won't ever scale as far. [0] https://github.com/channelcat/sa…

The problem with asyncio is that you can't take advantage of both it and the bazillion synchronous-io libraries already in the ecosystem.

For large web apps it's still pretty much a nonstarter unless you want to not be able to take full advantage of libraries like sqlalchemy.

Nodejs had a great advantage by starting with asyncio from the beginning.

M:N threading would have been a much better fit, though that's out of question given the GIL.

Re: Why we switched from Python to Go

#277

Python was my entry into the programming world, and I've been an evangelist ever since... Or I was until I ran into distribution and parallelism. Since then, Nim has been my go-to language of choice. It is all that Python was, plus unbelievable speed, compiling to shippable binaries, and some other cool language features that admittedly, are still beyond my scope of abilities. Still quite lacking in libraries compare…

There is a huge need for python to be compiled or made faster.

I wonder if python 4 could make this happen.

Re: Why we switched from Python to Go

#278
post #158

Earlier quoted context omitted.

For me, the ecosystem matters as much as the language itself these days (actually more). There are languages that I've used at home just as a learning experience, but ideally, I want a language that I can put into production at work. That means quite a list of criteria: only a small number of languages have a well-supported AWS SDK, for example. I won't say that Go is mainstream yet, but it feels very "production-rea…

I'd say that simplest "production-ready" criterion is this: does it have an officially supported and stable release of a IntelliJ IDE (there is a slight Java-ecosystem bias, but I think it's only slight). So the mainstream languages are...: C/C++, C#, F#, Go, Groovy, Java, JavaScript, TypeScript, Kotlin, Objective-C, PHP, Python, Ruby, Scala, SQL, Swift, VB.NET (source: https://www.jetbrains.com/products.html ). Soun…

I'm the opposite; I don't stick much time into learning a language if it doesn't have a high quality vim plug-in and command line tooling. It's fairly shallow, but I want to learn how things work behind the scenes, and I don't like fighting the editor over my preferences. It's a lot of small things, but why not enjoy your hobby time? This is just my personal preference and I understand and respect that there are lots of folks who enjoy their IDEs like I enjoy vim/Unix.

Re: Why we switched from Python to Go

#279
post #273
post #198

Earlier quoted context omitted.

Would you kindly share you experience on using Nim for that project?

For that project, we had very specific requirements: easily handle SSL/TLS with contexts and control over self-signed vs certificate checking, JSON processing, speed, nice syntax, and one of the most challenging requirements: statically compiled, linkable against musl and libressl, while still supporting mingw_64 for windows. Only a few languages have flexible compilers that can do this; for example, rust can't (afai…

If you do concatenation a lot, it's better (for performance, of course) if you make it a template (so Nim will just replace a + b with a & b):

  template `+`(a, b: string): string = a & b
But probably C compiler is smart enough to inline your proc :)

Re: Why we switched from Python to Go

#280
post #42

>Go forces you to stick to the basics. This makes it very easy to read anyone’s code and immediately understand what’s going on. I’m not criticizing Go since I have no LoC in it, but in other restricted-and-flat languages and areas our company’s expertise quickly (read: in half a decade) went to the limit with no chance to turn the lang partly into dsl and level up. It is like playing rpg where you stuck to level 5 a…

I understand what you mean by "inability to create something that only your team can use and understand", but the article doesn't say anything about that and isn't really addressing developing new code in Go. The article is about Python not matching their needs after a few tries at optimization and Go having a similar development time (although if the code was already implemented in Python then I assume the Go code w…

At a high enough level, Python does not live up to that zen. It does not restrict how you will organize your code, it just restricts instruction level options.

Keep in mind that the Zen of Python is mostly about how Python differs from its ancestors (basically, Perl). It's not a work on software engineering.

Post reply on HN