Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why we’re writing machine learning infrastructure in Go, not Python

towardsdatascience.com

Post reply on HN
Print view View original

51–60 of 76 posts

Re: Why we’re writing machine learning infrastructure in Go, not Python

#51

Post by _y5hn »

shrimpx wrote: post #46
vardump wrote: post #36

Earlier quoted context omitted.

Well... there's also other side of the coin. There's value in visibility and lack of magic. > lack of decent error handling (if err !=nil { return nil,err} ) Errors are in your face, instead of having exceptions performing invisible gotos to somewhere far up in call tree. Implicit error handling is more code, but your error handling is going to be much more robust. > or lack of decent polymorphism... Lack of polymorp…

Try/catch vs explicitly checking error values is like implementing sandboxing with os-level isolation versus with programming discipline. Why one would prefer offering guarantees like "will never crash/panic" using the latter technique instead of a guaranteed catch-all is incomprehensible to me.

Performance and making logic explicit.

Re: Why we’re writing machine learning infrastructure in Go, not Python

#52

Post by _y5hn »

allovernow wrote: post #39
vardump wrote: post #36

Earlier quoted context omitted.

Well... there's also other side of the coin. There's value in visibility and lack of magic. > lack of decent error handling (if err !=nil { return nil,err} ) Errors are in your face, instead of having exceptions performing invisible gotos to somewhere far up in call tree. Implicit error handling is more code, but your error handling is going to be much more robust. > or lack of decent polymorphism... Lack of polymorp…

>Composition makes code inherently more maintainable and easier to refactor. My understanding of composition comes from interfaces in C#. In my experience composition is actually more difficult to code and refactor, because it leads to bloat from repeated code as you cannot simply inherit. Refactoring then can become tedious as you may need to manually edit every repeated code block. Whereas with inheritance I define…

Composition should lead to clean components for reuse. You might need some extra references, but it is more explicit, static and flexible. Inheritance sounds great in theory, but is a hierarchical blind-alley, though usable for static designs that never change much.

Re: Why we’re writing machine learning infrastructure in Go, not Python

#53

Post by whateveracct »

vardump wrote: post #36
toolslive wrote: post #10

"in the land of the blind, the one-eyed man is king" Golang is probably a step up from Python, but it's just that. There are a lot of issues with Golang. From the top of my head, lack of decent error handling (if err !=nil { return nil,err} ) or lack of decent polymorphism are the most annoying. There's a github repo dedicated to what's bugging people: https://github.com/ksimka/go-is-not-good

Well... there's also other side of the coin. There's value in visibility and lack of magic. > lack of decent error handling (if err !=nil { return nil,err} ) Errors are in your face, instead of having exceptions performing invisible gotos to somewhere far up in call tree. Implicit error handling is more code, but your error handling is going to be much more robust. > or lack of decent polymorphism... Lack of polymorp…

> Lack of polymorphism also means you don't have to guess about concrete types when reading code. When troubleshooting, you can see what's going on without going through whole inheritance tree.

Parametric polymorphism would mean you never have to even consider the type of the code. It's a freeing abstraction with plenty of very simple reference implementations. And it lends itself to composition over inheritance.

And sadly I doubt Go will ever manage to get it.

Re: Why we’re writing machine learning infrastructure in Go, not Python

#54

Post by whateveracct »

cle wrote: post #28
jrimbault wrote: post #19

Earlier quoted context omitted.

>These two issues are usually brought up by people who haven't written a lot of C. >In day to day work it's really not an issue. Polymorphism and error handling, these are issues, address them. Go's a fine language, but don't dismiss actual real issues that have been addressed in other languages for 40+? years.

I have written a lot of Go (hundreds of thousands of LOC), and appreciate Go's approach to both of these. I disagree that these are major issues. A bigger issue, IMO, is that people coming to Go from other languages expect it to have similar features to what they're used to. Go is small and conceptually simple, but its ideas and philosophy are pretty different from most mainstream languages.

Go necessitates so much handrolling. Loops, concurrency patterns, resource management. You can't abstract over much, so you end up doing these things by hand constantly. Therefore you rely on brainpower, ad hoc linters (if one exists), and tests to not blow things up.

Go has its benefits and reasons to use it (especially the network effects around it). But when it comes to the three things I mention, Go is truly pathetic compared to a language like Haskell. It's amazing how much effort is wasted on those activities. I can see why a corporation would not care, but as an engineer I can do better with my time and effort for my personally-owned endeavors.

Re: Why we’re writing machine learning infrastructure in Go, not Python

#56

Post by cle »

whateveracct wrote: post #54
cle wrote: post #28

Earlier quoted context omitted.

I have written a lot of Go (hundreds of thousands of LOC), and appreciate Go's approach to both of these. I disagree that these are major issues. A bigger issue, IMO, is that people coming to Go from other languages expect it to have similar features to what they're used to. Go is small and conceptually simple, but its ideas and philosophy are pretty different from most mainstream languages.

Go necessitates so much handrolling. Loops, concurrency patterns, resource management. You can't abstract over much, so you end up doing these things by hand constantly. Therefore you rely on brainpower, ad hoc linters (if one exists), and tests to not blow things up. Go has its benefits and reasons to use it (especially the network effects around it). But when it comes to the three things I mention, Go is truly path…

As an individual, many of Go's benefits aren't as useful. It's optimized for team settings, emphasizing consistency, readability, explictness, and low cognitive overhead, which are all major weaknesses of Haskell, and major reasons it's not widely used in industry.

Re: Why we’re writing machine learning infrastructure in Go, not Python

#57

Post by whateveracct »

cle wrote: post #56
whateveracct wrote: post #54

Earlier quoted context omitted.

Go necessitates so much handrolling. Loops, concurrency patterns, resource management. You can't abstract over much, so you end up doing these things by hand constantly. Therefore you rely on brainpower, ad hoc linters (if one exists), and tests to not blow things up. Go has its benefits and reasons to use it (especially the network effects around it). But when it comes to the three things I mention, Go is truly path…

As an individual, many of Go's benefits aren't as useful. It's optimized for team settings, emphasizing consistency, readability, explictness, and low cognitive overhead, which are all major weaknesses of Haskell, and major reasons it's not widely used in industry.

For many applications, Go has higher cognitive overhead than Haskell. Modulo learning Haskell of course, but still.

Go is made to treat people as Resources. Full stop. That explains everything about it's benefits.

Re: Why we’re writing machine learning infrastructure in Go, not Python

#58

Post by nhumrich »

calebkaiser wrote: post #15
nhumrich wrote: post #6

> Making all of these overlapping API calls in a performative, reliable way is a challenge. Pythons asyncio is pretty hard to beat. For non-cpu intensive tasks, I find it a pleasure to work with. Goroutines can still have race conditions. > Originally, we wrote the CLI in Python, but trying to distribute it across platforms proved to be too difficult Sure, I get that go can cross-compile. But what makes python hard?…

We thought similarly to you about the relative ease of "pip install"—which is why we originally wrote the CLI in Python. Pretty immediately, however, we heard back from users who experienced friction. With the Go binary, we're able to share a one line bash command that users on Mac/Linux (Windows coming soon) can run to install the Cortex CLI, which removed the need for us to instruct users on how to configure their…

If you are having them bash a shell script, that shell script could install a python package as well. If your users like the bash install, I dont think rewriting in go was required for that. I guess I dont understand why this is any easier. For "cross platform" windows users dont even have bash.

Re: Why we’re writing machine learning infrastructure in Go, not Python

#59

Post by sseth »

allovernow wrote: post #39
vardump wrote: post #36

Earlier quoted context omitted.

Well... there's also other side of the coin. There's value in visibility and lack of magic. > lack of decent error handling (if err !=nil { return nil,err} ) Errors are in your face, instead of having exceptions performing invisible gotos to somewhere far up in call tree. Implicit error handling is more code, but your error handling is going to be much more robust. > or lack of decent polymorphism... Lack of polymorp…

>Composition makes code inherently more maintainable and easier to refactor. My understanding of composition comes from interfaces in C#. In my experience composition is actually more difficult to code and refactor, because it leads to bloat from repeated code as you cannot simply inherit. Refactoring then can become tedious as you may need to manually edit every repeated code block. Whereas with inheritance I define…

In go, composition has a special feature where if you include another type anonymously in a struct, the methods of that type are automatically available for the struct as well. It is also possible to "override" these methods at the struct level and call the anonymous member directly.

So you can get some of the conveniences of inheritance. The behaviour is of course different, because the composed member has no way of knowing the identity of the including struct.

Re: Why we’re writing machine learning infrastructure in Go, not Python

#60

Post by RandyRanderson »

tus88 wrote: post #26

Sounds like fairly generic deployment infrastructure that has nothing to do with machine learning. But why pass up the opportunity to use a buzzword to get on the front page of HN?

I count at least 3: ML, Go, Python. Challenge: could they fit more in there under the title len limit?
Post reply on HN