Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why we switched from Python to Go

getstream.io

Post reply on HN
Print view View original

181–190 of 406 posts

Re: Why we switched from Python to Go

#181

Post by cdoxsey »

takeda wrote: post #59
btym wrote: post #28

Earlier quoted context omitted.

Docker takes about 10m to build: https://jenkins.dockerproject.org/job/docker/job/docker.gith... Kubernetes takes about 16m: https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/...

Doesn't sound too fast, that's about how long it takes for FreeBSD kernel to be compiled[1] and I would imagine a kernel is more complex. The whole system takes about 50 minutes[2] [1] https://ci.freebsd.org/job/FreeBSD-head-amd64-LINT/buildTime... [2] https://ci.freebsd.org/job/FreeBSD-head-amd64-build/buildTim...

Go's main compilation performance gains come from incremental compilation. Since C++ doesn't have real module support merely parsing the entire dependency graph from headers can take a very long time. In go if you have .a pkgs for all your dependencies your program will compile very fast.

If you use docker as part of your ci build the dependencies as a separate step so they can be cached between builds. This is especially important for libraries like sqlite, which can take a lot longer to compile.

Re: Why we switched from Python to Go

#182

Post by cdoxsey »

catnaroek wrote: post #34
squiguy7 wrote: post #27

Earlier quoted context omitted.

It can be a bear to try and track down some race conditions but at least it will not report false positives. However, it will not catch all race conditions or data races you may have.

Is Go even memory-safe in the presence of data races? Don't get me wrong, “memory safety” isn't a hard requirement in my book, but “memory safety or a formal semantics” is.

If you're curious, the behavior is well defined: golang.org/ref/mem

Re: Why we switched from Python to Go

#183

Post by Dowwie »

Wait just one minute, Thierry...

Are you telling me that Python helped you create a viable tech-oriented newsfeed and activity stream business, serving 500 companies and more than 200Million "end users"? That sounds like a great incentive for any entrepreneur to get started with Python.

You've gotten this far by using the language you've turned away from! Further, I'm confident you didn't solve every challenge you've had on your own but rather did so with the help of the worldwide community of Pythonistas who answer questions and help to solve problems.

It takes a village to raise a startup. Don't try to burn the village down after you've grown up and can afford to explore other villages.

Re: Why we switched from Python to Go

#184

Post by viraptor »

wiz21c wrote: post #155
setheron wrote: post #30

How do these shallow articles get upvoted so much ? they don't have much specific information except very generic "developer productivity". Let me give a specific example where moving to Go really helped our tooling: Go has some great interfaces, specifically their net & ssh client. In order to perform operations against some machines, we have to tunnel through bastions, however we'd also like the tool to work when a…

I think the article is not that bad. I specially like the comparison of : well python was a bit faster to develop with, but much longer to optimize. Note they actually used Python long enough to actually spend quite some time to consider the move to go; this means that performance of Python were ranging from good enough to barely enough for a (hopefully) long period of time. Also note the scale of operations of that…

I'm a bit sceptical about the comparison of optimisation. Specifically when the AST was mentioned (did they parse the expression as Python code?) and Python came out that much slower even after fixes. As long as they were interpreting that, rather than compiling the expression into native code, I don't see a good reason for Go to be faster. Interpreting expressions like that in Go would be almost as dynamic / lookup heavy as in Python.

I'd like to see both apps for comparison / more context.

Re: Why we switched from Python to Go

#185

Post by JelteF »

nilved wrote: post #160
djsumdog wrote: post #146

Earlier quoted context omitted.

Virtualenv is nice, but lately I've grown to use docker containers instead. You can use the official containers for go, python or ruby, feed it the gemfile or requirements.txt or whether, tag it, push it and you have a permanent snapshoted image.

Virtualenv is not nice; it's a pragmatic yet laughable hack.

It's definitely a hack, but for development environments it's quite an effective one. Which is why we actually created a (more user friendly) clone for Go development. It's called Virtualgo, and it's mentioned near the end of the article as well. https://github.com/GetStream/vg

Re: Why we switched from Python to Go

#186

Post by dualogy »

Scramblejams wrote: post #120
sitkack wrote: post #47

If you can switch from Python to Go, you weren't using Python anyways, you were use Gothon or Javthon. If you want static binaries, great tooling and an excellent imperative and functional language, try F# with mkbundle. The compelling reasons to use go are shrinking.

Would love to, but what I would miss from Go is that low latency GC. If ya need it, ya really really need it. If only somebody wrote a nice ML that compiled to Go! I love a lot of things about Go, but the language certainly isn’t one of them.

I'm hacking on a PureScript-to-Go transpiler (ie. alt backend for existing PureScript compiler once PR #3117 gets accepted). Watch http://github.com/metaleap/gonad/ --- hoping to "get there" within 2 weeks. Taking unreasonably long already, given that purs already does its jobs on its own like parsing, type-checking, transforming-ML-to-imperative. One reason I'm taking a bit longer than the quickest-dirtiest approach would, I want to preserve the type information as much as possible rather than have all functions accepting and returning and passing `interface{}`s in a JS-like manner. Ie it's meant to generate sane readable human-like idiomatic Go code. Type-classes mapped to interface types, etc. Will also have to attack the need to have any Go package be represented readily on the PureScript side as an existing module that'll show up in auto-complete and pass compilation, so to quickly auto-generate some kind of dummy FFI bindings whenever a Go-land dep is imported and used. Fun ride!

There's clearly value in combining Go's compilation speeds, stdlib functionality, rich ecosystem, lean fast binaries, GC etc with a rich and cutting-edge Haskell-ish/ML-ish type system (and the leaner syntax and compressed idioms). Will be great for clear thinking and expressive high-level type-driven dev and DSLs (and naturally, implicitly generics/code-gen haha), without having to wrestle with GHC/cabal/stack build annoyances/times and Haskell's whacky academic overly-PhD-ish "wrappings" around raw straightforward real-world needs such as http-server and db-client, where again the Go ecosystem shines.

Re: Why we switched from Python to Go

#187

Post by stefano »

sethammons wrote: post #104
albinofrenchy wrote: post #67

Earlier quoted context omitted.

I tried using Go this weekend and basically just abandoned it when I learned that it only had 'generic's for three built-in types and other than that you are forced to essentially dynamic cast everywhere. I just don't get the appeal. Go seems a lot like what you'd get if you just removed every language feature that anyone has ever complained about; for good reason or not.

I venture to say that if you are casting/converting everywhere, you are likely doing it wrong. An interface{} says nothing. It should generally be avoided. However, I do find this to be a pain when working with numbers. Floats and ints mixed up is not fun. Python is so vastly easier to work with in that arena.

In Go, if you need a container which is not a built-in map or array, you end up casting to interface{}, because that's the only reasonable thing your container can accept. This is clearly not "doing it wrong", and it's an extremely common use case.

Re: Why we switched from Python to Go

#188

Post by lmm »

skeletal88 wrote: post #179
slackingoff2017 wrote: post #46

Why does anyone write web apps in Python? PHP? Ruby? Modern Java and Go are so much faster than the alternatives that it's stupid to consider anything else if performance is important. Golang and Java can manage over a half million HTTP responses a second. Node is pretty fast but why bother when Java is many times more mature in features, tooling, and and supports concurrency... And uses less ram and is usually faste…

Can you point a clueless (about Java web dev) person to a nice, lightweight framework that's easy to learn and to set up? Please no XML configuration files and other such nonsense. For me as a Python/Rails/C++ dev Java has a reputation of being too large, too complex and otherwise.. unwieldy. Hearing things like "To test a bug I had to start 6 services on my computer and then I ran out of memory (computer had 16GB)"…

Not the most lightweight, but I'd highly recommend wicket running on embedded jetty, a la the second code block on https://cwiki.apache.org/confluence/pages/viewpage.action?pa... . You do still have to use XML for Maven I'm afraid (there are alternatives but I wouldn't recommend them) but it's a relatively good use of XML and you can use eclipse's GUI to add dependencies rather than adding them directly if you like.

Wicket is a true OO approach to GUI which is quite different from the page-template style of Rails/Django/..., but I find it makes for much more compositional style, with lots of small reusable components that are just compositions of a few smaller components. And while not being able to monkeypatch everything can chafe initially, when you come to upgrade to a newer version of the framework you'll really appreciate the safety a compiled/typechecked language can offer.

Re: Why we switched from Python to Go

#189

Post by lmm »

mlevental wrote: post #58
sitkack wrote: post #47

If you can switch from Python to Go, you weren't using Python anyways, you were use Gothon or Javthon. If you want static binaries, great tooling and an excellent imperative and functional language, try F# with mkbundle. The compelling reasons to use go are shrinking.

one problem: windows

Scala or OCaml are similar options if you don't want to use the CLR.

Re: Why we switched from Python to Go

#190

Post by JelteF »

Dowwie wrote: post #183

Wait just one minute, Thierry... Are you telling me that Python helped you create a viable tech-oriented newsfeed and activity stream business, serving 500 companies and more than 200Million "end users"? That sounds like a great incentive for any entrepreneur to get started with Python. You've gotten this far by using the language you've turned away from! Further, I'm confident you didn't solve every challenge you've…

Hi, Jelte from Stream here. The post was definitely not meant to indicate that people should stop using Python. We still use it happily for the website and I'm still a big fan of it myself.

However, for the API we've outgrown it in performance requirements. That's what the article is about, together with the things we found during the switch that we liked and disliked about Go.

Post reply on HN