Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why we’re writing machine learning infrastructure in Go, not Python

towardsdatascience.com

Post reply on HN
Print view View original

71–76 of 76 posts

Re: Why we’re writing machine learning infrastructure in Go, not Python

#71

Post by RandyRanderson »

dewey wrote: post #63
RandyRanderson wrote: post #61

Earlier quoted context omitted.

"I have written a lot of Go (hundreds of thousands of LOC)..." Go only appeared about 10 years ago. The industry standard is 50 lines of production code a day. 50/day * 250 working days a year = 12.5k loc/year. So it would take you how many continuous years to write 200k LOC?

There's probably a very big difference between "industry standard" and working with a new language at a startup where a lot of things are written from scratch in a short amount of time. 50 lines of code on average for a year won't get you very far if you work on greenfield project. The industry standard is probably very heavily biased by big corporate code bases where refactoring / architecture takes a lot more time…

"There's probably a very big difference between" - that is precisely my point.

I have a lot of time playing ms flight sim and so obviously I know how to design airplanes is his argument.

Writing "greenfield" applications that no one (comparatively) is using is relatively easy. Creating systems that have four+ 9's availability, integrate with various disparate APIs, don't lose data, and,and etc is where, IMO, the learning is.

Once one has this experience, watching this micro services "movement" is, in my mind, related to global warming - like watching someone jump out of a plane with no parachute, smiling.

Re: Why we’re writing machine learning infrastructure in Go, not Python

#72

Post by luord »

Great, another one of these articles, but this time I feel more confident in my usual reply, having been working in Go exclusively for a while.

> Implementing all of this functionality in Python may be doable with recent tools like asyncio, but the fact that Go is designed with this use case in mind makes our lives much easier.

This just makes me think about Armin Ronacher's article on back pressure but, sure, whatever.

> Building a cross-platform CLI is easier in Go

No, it isn't.

> The performance benefits of a compiled Go binary versus an interpreted language are also significant

Ah, yes, because performance is such a key feature of command line interfaces, as evidenced by bash and its outstanding performance in every benchmark.

> The Go ecosystem is great for infrastructure projects

And the reality discussed in this point would be different if docker wasn't written in Go. Had the docker developers chose anything else, this point would apply to that hypothetical language, so it isn't an inherent advantage of Go as a language.

> Go is just a pleasure to work with

No, it really, really isn't, but that's not the point.

This is ultimately the real reason they chose go: whoever made the original decision liked it and everything else is post-hoc rationalization.

Which is fine, most of this tends to be subjective.

Re: Why we’re writing machine learning infrastructure in Go, not Python

#73

Post by sagichmal »

apta wrote: post #67
loicd wrote: post #48

Earlier quoted context omitted.

Out of curiosity, would you share those flaws you have found?

This is by no means a comprehensive list: * no generics * null pointers * no compile time checked enums or sum types * The golang time package is garbage * golang interfaces are garbage, can't tag types with an interface without implementing a dummy method and hope that no other type implements it. Can't find what interfaces a type implements without an IDE * profiling and debugging tools are nothing in front of JVM…

> no incremental compilation

Not true, unless you're using an esoteric definition of "incremental".

> imports don't support cyclic imports

Why on earth would you consider this bad?

Re: Why we’re writing machine learning infrastructure in Go, not Python

#74

Post by qtplatypus »

sagichmal wrote: post #73
apta wrote: post #67

Earlier quoted context omitted.

This is by no means a comprehensive list: * no generics * null pointers * no compile time checked enums or sum types * The golang time package is garbage * golang interfaces are garbage, can't tag types with an interface without implementing a dummy method and hope that no other type implements it. Can't find what interfaces a type implements without an IDE * profiling and debugging tools are nothing in front of JVM…

> no incremental compilation Not true, unless you're using an esoteric definition of "incremental". > imports don't support cyclic imports Why on earth would you consider this bad?

The problem that I've ran into with the lack of cyclic imports is that I like to arrange the contents of packages logically. However sometimes in order to do this it would result in a circular dependency between two packages. So rather then getting a logical arrangement we have to refactor it into an artificial package to resolve this depenancy.

Re: Why we’re writing machine learning infrastructure in Go, not Python

#75

Post by sagichmal »

qtplatypus wrote: post #74
sagichmal wrote: post #73

Earlier quoted context omitted.

> no incremental compilation Not true, unless you're using an esoteric definition of "incremental". > imports don't support cyclic imports Why on earth would you consider this bad?

The problem that I've ran into with the lack of cyclic imports is that I like to arrange the contents of packages logically. However sometimes in order to do this it would result in a circular dependency between two packages. So rather then getting a logical arrangement we have to refactor it into an artificial package to resolve this depenancy.

Circular imports are always a design error, no "logical" to it.

Re: Why we’re writing machine learning infrastructure in Go, not Python

#76

Post by cutler »

apta wrote: post #29
cutler wrote: post #7

I'm a bit green on infrastructure & deployment but I don't quite get this. If your ML algorithm code is still Python how does deployment with Go make that much difference? It sounds like you're not replacing the Python ML code so why is this such a big deal?

> It sounds like you're not replacing the Python ML code so why is this such a big deal? How would you write an article about moving to a hipster language then? :-)

I've re-read your reply a few times but I honestly don't get your point.
Post reply on HN