Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why we’re writing machine learning infrastructure in Go, not Python

towardsdatascience.com

Post reply on HN
Print view View original

61–70 of 76 posts

Re: Why we’re writing machine learning infrastructure in Go, not Python

#61

Post by RandyRanderson »

cle wrote: post #28
jrimbault wrote: post #19

Earlier quoted context omitted.

>These two issues are usually brought up by people who haven't written a lot of C. >In day to day work it's really not an issue. Polymorphism and error handling, these are issues, address them. Go's a fine language, but don't dismiss actual real issues that have been addressed in other languages for 40+? years.

I have written a lot of Go (hundreds of thousands of LOC), and appreciate Go's approach to both of these. I disagree that these are major issues. A bigger issue, IMO, is that people coming to Go from other languages expect it to have similar features to what they're used to. Go is small and conceptually simple, but its ideas and philosophy are pretty different from most mainstream languages.

"I have written a lot of Go (hundreds of thousands of LOC)..."

Go only appeared about 10 years ago. The industry standard is 50 lines of production code a day. 50/day * 250 working days a year = 12.5k loc/year. So it would take you how many continuous years to write 200k LOC?

Re: Why we’re writing machine learning infrastructure in Go, not Python

#62

Post by [deleted] »

whateveracct wrote: post #54
cle wrote: post #28

Earlier quoted context omitted.

I have written a lot of Go (hundreds of thousands of LOC), and appreciate Go's approach to both of these. I disagree that these are major issues. A bigger issue, IMO, is that people coming to Go from other languages expect it to have similar features to what they're used to. Go is small and conceptually simple, but its ideas and philosophy are pretty different from most mainstream languages.

Go necessitates so much handrolling. Loops, concurrency patterns, resource management. You can't abstract over much, so you end up doing these things by hand constantly. Therefore you rely on brainpower, ad hoc linters (if one exists), and tests to not blow things up. Go has its benefits and reasons to use it (especially the network effects around it). But when it comes to the three things I mention, Go is truly path…

[deleted]

Re: Why we’re writing machine learning infrastructure in Go, not Python

#63

Post by dewey »

RandyRanderson wrote: post #61
cle wrote: post #28

Earlier quoted context omitted.

I have written a lot of Go (hundreds of thousands of LOC), and appreciate Go's approach to both of these. I disagree that these are major issues. A bigger issue, IMO, is that people coming to Go from other languages expect it to have similar features to what they're used to. Go is small and conceptually simple, but its ideas and philosophy are pretty different from most mainstream languages.

"I have written a lot of Go (hundreds of thousands of LOC)..." Go only appeared about 10 years ago. The industry standard is 50 lines of production code a day. 50/day * 250 working days a year = 12.5k loc/year. So it would take you how many continuous years to write 200k LOC?

There's probably a very big difference between "industry standard" and working with a new language at a startup where a lot of things are written from scratch in a short amount of time. 50 lines of code on average for a year won't get you very far if you work on greenfield project.

The industry standard is probably very heavily biased by big corporate code bases where refactoring / architecture takes a lot more time than implementing the actual changes afterwards. Doesn't seem like a very useful calculation you are trying to do there to proof the previous poster wrong.

Re: Why we’re writing machine learning infrastructure in Go, not Python

#64

Post by flavio81 »

icandoit wrote: post #35
jrimbault wrote: post #19

Earlier quoted context omitted.

>These two issues are usually brought up by people who haven't written a lot of C. >In day to day work it's really not an issue. Polymorphism and error handling, these are issues, address them. Go's a fine language, but don't dismiss actual real issues that have been addressed in other languages for 40+? years.

In some circles C has a better reputation than C++. Surely you see that other people value other things than you do? If you value simplicity check out the programming language zig. https://ziglang.org/

>If you value simplicity check out the programming language zig.

I value simplicity, that's what i use Brainfuck. Far simpler than Go and Zig.

https://fatiherikli.github.io/brainfuck-visualizer/

Re: Why we’re writing machine learning infrastructure in Go, not Python

#65

Post by flavio81 »

cle wrote: post #28
jrimbault wrote: post #19

Earlier quoted context omitted.

>These two issues are usually brought up by people who haven't written a lot of C. >In day to day work it's really not an issue. Polymorphism and error handling, these are issues, address them. Go's a fine language, but don't dismiss actual real issues that have been addressed in other languages for 40+? years.

I have written a lot of Go (hundreds of thousands of LOC), and appreciate Go's approach to both of these. I disagree that these are major issues. A bigger issue, IMO, is that people coming to Go from other languages expect it to have similar features to what they're used to. Go is small and conceptually simple, but its ideas and philosophy are pretty different from most mainstream languages.

>Go is small and conceptually simple, but its ideas and philosophy are pretty different from most mainstream languages.

No, they aren't. They are the same as languages from 40 years ago: We have had channels, coroutines, and good garbage collectors from long, long ago. Plus the syntax is directly borrowed from Algol 68 (from 1968).

Go looks nice because it fills a niche (a high performance garbage collected language that is easier to learn than Java and provides faster compilation times than C++), but there's absolutely nothing different here.

For "pretty different from mainstream" see Forth, Prolog, Erlang, etc.

Re: Why we’re writing machine learning infrastructure in Go, not Python

#67

Post by apta »

loicd wrote: post #48
apta wrote: post #30

Earlier quoted context omitted.

I write a lot of code in golang, and if anything, my views before using the language were even further cemented by even more flaws I found while using it.

Out of curiosity, would you share those flaws you have found?

This is by no means a comprehensive list:

* no generics

* null pointers

* no compile time checked enums or sum types

* The golang time package is garbage

* golang interfaces are garbage, can't tag types with an interface without implementing a dummy method and hope that no other type implements it. Can't find what interfaces a type implements without an IDE

* profiling and debugging tools are nothing in front of JVM and .NET tools

* no const/immutablility

* no ternary operator, need to write 6 lines instead of a single line if it existed

* error handling is error prone and verbose

* no default method implementation in interfaces

* unit testing frameworks are sucky and rely on code gen, can't mock arbitrary types, but only by defining an interface

* that unnused variables and imports are compile time errors, makes it very annoying and time consuming to prototype or when debugging

* no private/public/etc modifier. if you want to change the visibility of a type, you need to rename it in all files it's listed in

* no incremental compilation (yes Java compiles faster in large projects where I only modify a handful of files, or even more)

* pervasive use of single letter identifiers in golang code bases

* pervasive use of int, which has a platform dependent size

* pervasive instantiation of structs by calling them structFoo { Field1: field1, Field2: field2, ... FieldN: fieldN }, which makes it easy to miss when a new field gets added. Rust and Zig solve this by making it required to specify all fields and values.

* defer works on the function scope, not the current scope

* defer doesn't handle functions that return errors, meaning many errors are missed

* no struct or file private, only package private

* the global scope is polluted with special functions like len, copy, delete, make, new, append, for no good reason

* golang imports don't support cyclic imports

* golang doesn't allow you to import a a specific function or struct from a package. You have you always qualify anything with the package name, which makes things verbose and awkward.

Re: Why we’re writing machine learning infrastructure in Go, not Python

#68

Post by 7thaccount »

threeseed wrote: post #23
cigaaa wrote: post #21

Earlier quoted context omitted.

I don’t think people who work in infrastructure currently will be surprised that Go is a better choice than Python for infra, but for those who are newer to the field of ML or only work on model development (vs deployment), it is likely surprising that a major part of production ML is best done in a language other than Python.

I don't buy this comment at all. I've worked with hundreds of Data Scientists, many new to the industry and they all know that R and Scala are important and popular languages for ML. Majority of Data Engineering today is using Spark, which is written in Scala and even when you write Python code using it you can't escape Java/Scala internals being exposed.

Is Scala really that prevalent? R & Python for sure.

Re: Why we’re writing machine learning infrastructure in Go, not Python

#69

Post by loicd »

apta wrote: post #67
loicd wrote: post #48

Earlier quoted context omitted.

Out of curiosity, would you share those flaws you have found?

This is by no means a comprehensive list: * no generics * null pointers * no compile time checked enums or sum types * The golang time package is garbage * golang interfaces are garbage, can't tag types with an interface without implementing a dummy method and hope that no other type implements it. Can't find what interfaces a type implements without an IDE * profiling and debugging tools are nothing in front of JVM…

Thanks for the writing. I had noticed some of the things you wrote but not all (I am not a Go programmer though I keep track of what is happening there). Your post made me look a bit more into 'defer'. I did not expect it to be a source of errors.

Re: Why we’re writing machine learning infrastructure in Go, not Python

#70

Post by apta »

loicd wrote: post #69
apta wrote: post #67

Earlier quoted context omitted.

This is by no means a comprehensive list: * no generics * null pointers * no compile time checked enums or sum types * The golang time package is garbage * golang interfaces are garbage, can't tag types with an interface without implementing a dummy method and hope that no other type implements it. Can't find what interfaces a type implements without an IDE * profiling and debugging tools are nothing in front of JVM…

Thanks for the writing. I had noticed some of the things you wrote but not all (I am not a Go programmer though I keep track of what is happening there). Your post made me look a bit more into 'defer'. I did not expect it to be a source of errors.

No problem. As a side note, the Zig language does a much better job here, especially with defer. It works at the scope level (how it's supposed to be), and they have an `errdefer` construct used to call functions that return errors.
Post reply on HN