Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rewriting a large production system in Go

matt-welsh.blogspot.com

Post reply on HN
Print view View original

31–40 of 196 posts

Re: Rewriting a large production system in Go

#31

Post by rdtsc »

derefr wrote: post #6

You know, every time I see some Googler shocked at the effectiveness and various advantages of coding in Go, I wonder why Google never adopted Erlang. They could have been getting all these same advantages (and then some) a decade ago :)

One silly yet simple reason could be the Not Invented Here Syndrome. For a company that size relying on Ericsson or Erlang Solutions for support could have been seen as something they didn't want to deal with. So they just wrote their own.

It is too bad though. Erlang, I think, has some features I like better such as hot code reloading, better supervision strategies, separate heap per lightweight process (hence no stop-the-world GC), and is battle tested for longer. Also, at least to me, actors with pattern matched receive, as central building blocks, makes more sense than channels. So I'll keep Erlang as my concurrent server side go-to language for now.

Re: Rewriting a large production system in Go

#32

Post by npalli »

bane wrote: post #30

> So programming in Go is making me soft. Something similar happened to me years ago when I started picking up Perl. All the BS C++ and Java made me go through to get anything done seemed like such a huge waste of time I ended just writing lots of stuff in Perl until I had pretty much forgotten how to C++ and how to Java. Go might be a new Perl in that sense. I'd also add that as Google moves more and more production…

"For each system that Google brings on-line with Go, even if it took a team of 10 6 months, they've likely saved themselves millions of dollars in new capex infrastructure acquisition"

I thought the old systems were in c++. So, how did the infrastructure savings accrue? In fact, I don't recall anyone doing analysis on that front.

Re: Rewriting a large production system in Go

#33

Post by mdwelsh »

bane wrote: post #30

> So programming in Go is making me soft. Something similar happened to me years ago when I started picking up Perl. All the BS C++ and Java made me go through to get anything done seemed like such a huge waste of time I ended just writing lots of stuff in Perl until I had pretty much forgotten how to C++ and how to Java. Go might be a new Perl in that sense. I'd also add that as Google moves more and more production…

David Andersen at CMU teaches the undergraduate distributed systems course in Go: http://www.cs.cmu.edu/~dga/15-440/F12/

Re: Rewriting a large production system in Go

#34

Post by mdwelsh »

zzzeek wrote: post #2

> If I could get out of the 1970s and use an editor other than vi, maybe I would get some help from an IDE in this regard, but I staunchly refuse to edit code with any tool that requires using a mouse. So you've shown that Go is appealing to rigid curmudgeons. Personally I'm still hung up on the "every function (edit: that does anything which might itself return an error code, which in large scale code is quite a lot…

The joke goes, "If every other line of code you write starts with 'if err != nil', you might be a Go programmer."

Re: Rewriting a large production system in Go

#35

Post by jussij »

> Go's type inference makes for lean code, but requires you to dig a little to figure out what the type of a given variable is if it's not explicit. So given code like:

   foo, bar := someFunc(baz)
> You'd really like to know what foo and bar actually are, in case you want to add some new code to operate on them.

The gocode utility (which works with several editors) does exactly this.

Re: Rewriting a large production system in Go

#36

Post by mdwelsh »

kailuowang wrote: post #15

It disheartens me a bit when people compare Go with C++. It gave me the impression that Go was created to be better C++ (or Java) mostly to make life easier for the thousands of developers at Google who are stuck with those two languages.

OP here. What do you think we should be using apart from C++ or Java?

Re: Rewriting a large production system in Go

#37

Post by dgacmu »

npalli wrote: post #32
bane wrote: post #30

> So programming in Go is making me soft. Something similar happened to me years ago when I started picking up Perl. All the BS C++ and Java made me go through to get anything done seemed like such a huge waste of time I ended just writing lots of stuff in Perl until I had pretty much forgotten how to C++ and how to Java. Go might be a new Perl in that sense. I'd also add that as Google moves more and more production…

"For each system that Google brings on-line with Go, even if it took a team of 10 6 months, they've likely saved themselves millions of dollars in new capex infrastructure acquisition" I thought the old systems were in c++. So, how did the infrastructure savings accrue? In fact, I don't recall anyone doing analysis on that front.

Matt linked to my course, but also: I did a tiny analysis of this with my pi searcher (http://da-data.blogspot.com/2013/05/improving-pi-searchers-s... ) -- it's a toy compared to the kind of system that Matt described, but my experience was extremely positive. Rewriting it in Go made it easier to architect the system right to take advantage of persistence. It's hugely faster.

We also have a paper accepted to SOSP this year (the academic weenies will understand that) where the system - a Paxos variant - is implemented in Go. We also implemented about four other Paxos variants in Go to compare against them, a task that would have been absolutely grueling in C++. In Go it wasn't too bad. Our performance isn't super-blazing, but it turns out to be one of the faster publicly available Paxos implementations anyway. I'll have to write up a bit about our experience with it -- I'm with Matt 100% on this one.

Re: Rewriting a large production system in Go

#38

Post by mseepgood »

seiji wrote: post #29
derefr wrote: post #6

You know, every time I see some Googler shocked at the effectiveness and various advantages of coding in Go, I wonder why Google never adopted Erlang. They could have been getting all these same advantages (and then some) a decade ago :)

I wonder why Google never adopted Erlang. Collectively, google believes they are right and the world is wrong. Anything pre-existing is dirty and unworthy of their genius if they didn't invent it themselves. So, even though we have 20 years of Erlang and production concurrency experience out there in one solid language, it's just ignored (except for parts they want to get "inspired by"). All of that is fine in isolat…

> except for parts they want to get "inspired by"

Go's concurrency was not "inspired by" Erlang. It was inspired by Hoare's CSP. Pike and Luca Cardelli created a CSP based language called Squeak in 1985: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.26.8...

Re: Rewriting a large production system in Go

#39

Post by dodyg »

lbarrow wrote: post #11

It's interesting to see that more and more internal projects at Google are getting redone in Go. However, this sentence stands out to me in the article: "Before doing the rewrite, we realized we needed only a small subset of the functionality of the original system -- perhaps 20% (or less) of what the other projects were doing with it." I'm guessing that a lot of the benefit of the rewrite came simply from the simpli…

This is why a "go" rewrite story is not that appealing to me. You can rewrite a subset of a system and gain many improvements simply by benefits of hindsight and clearer requirements regardless of languages.

Re: Rewriting a large production system in Go

#40

Post by seiji »

mseepgood wrote: post #38
seiji wrote: post #29

Earlier quoted context omitted.

I wonder why Google never adopted Erlang. Collectively, google believes they are right and the world is wrong. Anything pre-existing is dirty and unworthy of their genius if they didn't invent it themselves. So, even though we have 20 years of Erlang and production concurrency experience out there in one solid language, it's just ignored (except for parts they want to get "inspired by"). All of that is fine in isolat…

> except for parts they want to get "inspired by" Go's concurrency was not "inspired by" Erlang. It was inspired by Hoare's CSP. Pike and Luca Cardelli created a CSP based language called Squeak in 1985: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.26.8...

I was just ranting from a base of https://news.ycombinator.com/item?id=6235030. I don't actually know anything.
Post reply on HN