Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rewriting a large production system in Go

matt-welsh.blogspot.com

Post reply on HN
Print view View original

161–170 of 196 posts

Re: Rewriting a large production system in Go

#161

Post by mdwelsh »

pvdm wrote: post #82
mdwelsh wrote: post #59

Earlier quoted context omitted.

I would love to open source this system, but even if we did I'm not sure we'd be able to convince you that using Go was better than some other language in terms of developer productivity. How would having access to the source help?

You mentioned in the article that you read a section of code by one of the lead engineers and immediately you became convinced that using Go is the "way-to-go". Well, I would like to experience that eureka moment as well.

Good point. Maybe you could check out one of the many other open source Go projects out there as an alternative. I'm not sure reading our code would give you that eureka moment, since it only made sense to me since I was so familiar with the old code :-)

Re: Rewriting a large production system in Go

#162

Post by dragonwriter »

zzzeek wrote: post #155
dragonwriter wrote: post #74

Earlier quoted context omitted.

> If I'm going to use Go, or any language, I definitely want to use it idiomatically. Idioms evolve, and they evolve because they are challenged.

fine, but I've learned enough languages to know that the absolute worst thing you can do when you start out with language Y is make it act just like your previous language X. It's a very natural instinct for almost everyone (just read "Python is not Java" for an example), but for at least the first year or two of using a new language I think you have to do it as idiomatically as possible, before you have any insight…

> fine, but I've learned enough languages to know that the absolute worst thing you can do when you start out with language Y is make it act just like your previous language X.

Sure, if you want to learn the idiomatic Go way of doing things, you do things the idiomatic way. Once you've reached the point where you have familiarity with the idiomatic way and have a reasoned analysis of why you believe the idiomatic way is wrong (at least for you doing the things you want to do with the language), that's no longer the case.

If you reached the point where you feel comfortable arguing that the idiom is wrong, you've should also have reached the point where you can use the language constrained by features, not by conventional idiom.

Re: Rewriting a large production system in Go

#163

Post by gnuvince »

Tichy wrote: post #115
sytelus wrote: post #86

The number of strongly typed garbage collected compiled languages is surely shrinking. I can understand someone's disgust towards Java considering dwindling velocity in adding next-gen features and now sleeping in same bed as lawyer run Oracle. That pretty much had left C# in the arena until Go came on the scene. I was honestly hoping Go would give us head-to-head battle with C# but from initial looks Go pretty much…

Sometimes less is more. Not implementing every fancy feature under the sun might be a bonus and keeps the language clean. I know I never managed to like generics in Java. Don't know C#, though.

Languages like SML and OCaml are pretty clean, pretty simple and have an awesome parametric polymorphism system.

Re: Rewriting a large production system in Go

#164

Post by nijk »

pvdm wrote: post #82
mdwelsh wrote: post #59

Earlier quoted context omitted.

I would love to open source this system, but even if we did I'm not sure we'd be able to convince you that using Go was better than some other language in terms of developer productivity. How would having access to the source help?

You mentioned in the article that you read a section of code by one of the lead engineers and immediately you became convinced that using Go is the "way-to-go". Well, I would like to experience that eureka moment as well.

Read some code at golang.org

Re: Rewriting a large production system in Go

#165

Post by drivebyacct2 »

dubbledidu wrote: post #156
jawr wrote: post #150

Earlier quoted context omitted.

Would you care to elaborate on this?

* Their implementation sucks so much that most decently built concurrency implementation will beat Go-routines by a wide margin. Just have a look at all those people on the Go mailing list whining about the scheduler. Not even talking about the fact that building concurrency mechanisms into the language is plain stupid. We all saw how well that worked out in plenty of other languages before. * You know why pretty muc…

This is a pretty fucking funny and ironic comment.

Re: Rewriting a large production system in Go

#166

Post by drivebyacct2 »

pjmlp wrote: post #93
sytelus wrote: post #86

The number of strongly typed garbage collected compiled languages is surely shrinking. I can understand someone's disgust towards Java considering dwindling velocity in adding next-gen features and now sleeping in same bed as lawyer run Oracle. That pretty much had left C# in the arena until Go came on the scene. I was honestly hoping Go would give us head-to-head battle with C# but from initial looks Go pretty much…

Yep, Go would be hardly mentioned here if it wasn't being developed at Google. Having said this, Go can be a nice replacement for many use cases one would use C for.

How is this same fucking absurd statement written in EVERY SINGLE THREAD HERE about Go? And disproven everytime?

Re: Rewriting a large production system in Go

#167

Post by pmahoney »

derefr wrote: post #87
laureny wrote: post #80

Earlier quoted context omitted.

> type safety is in practice incompatible with VM-supported hot code upgrade. That's not true. First, it's very easy to hot reload changes that have been made to the code that are backward compatible. The JVM spec describes in very specific details what that means (adding or removing a method is not backward compatible, modifying a body is, etc...). This is how Hotswap works, the JVM has been using it for years. As f…

> As for changes that are backward incompatible, you can still manage them with application level techniques, such as rolling out servers or simply allow two different versions of the class to exist at the same time (JRebel does that, as do other a few other products in the JVM ecosystem). Neither of these allow for the whole reason Erlang has hot code upgrade in the first place: allowing to upgrade the code on one s…

Tomcat (and other app servers) has support for doing hot reloads of Java web apps while not reloading the HTTP layer (and not dropping TCP connections).

http://www.javacodegeeks.com/2011/06/zero-downtime-deploymen...

I have implemented a similar system for JRuby apps running inside a Servlet container. There are many caveats. I don't actually recommend it because for a while you're using nearly twice the memory (and JRuby is particularly memory hungry). Also there are many ways to leak the old class definitions such that they are not GC'd (e.g. thread locals). But it's certainly possible.

I suspect that Erlang, Java, and all languages are in the same boat: some parts can be upgraded live in the VM while other parts require a full restart (maybe coordinating with multiple nodes and a load balancer to achieve zero-downtime).

Re: Rewriting a large production system in Go

#168

Post by Maro »

dgacmu wrote: post #37
npalli wrote: post #32

Earlier quoted context omitted.

"For each system that Google brings on-line with Go, even if it took a team of 10 6 months, they've likely saved themselves millions of dollars in new capex infrastructure acquisition" I thought the old systems were in c++. So, how did the infrastructure savings accrue? In fact, I don't recall anyone doing analysis on that front.

Matt linked to my course, but also: I did a tiny analysis of this with my pi searcher ( http://da-data.blogspot.com/2013/05/improving-pi-searchers-s... ) -- it's a toy compared to the kind of system that Matt described, but my experience was extremely positive. Rewriting it in Go made it easier to architect the system right to take advantage of persistence. It's hugely faster. We also have a paper accepted to SOSP th…

What you say sounds cool. I wrote a bunch of Paxos implementations for distributed databases in C++:

https://github.com/scalien/scaliendb/tree/master/src/Framewo...

Can you share your Go code?

Re: Rewriting a large production system in Go

#169

Post by kemist »

dubbledidu wrote: post #156
jawr wrote: post #150

Earlier quoted context omitted.

Would you care to elaborate on this?

* Their implementation sucks so much that most decently built concurrency implementation will beat Go-routines by a wide margin. Just have a look at all those people on the Go mailing list whining about the scheduler. Not even talking about the fact that building concurrency mechanisms into the language is plain stupid. We all saw how well that worked out in plenty of other languages before. * You know why pretty muc…

> Their implementation sucks so much that most decently built concurrency implementation will beat Go-routines by a wide margin.

It's worth noting that they're aware of the schedulers problems, and Dmitry is working on a significant revamp. https://docs.google.com/document/d/1TTj4T2JO42uD5ID9e89oa0sL...

Re: Rewriting a large production system in Go

#170

Post by icedchai »

dubbledidu wrote: post #156
jawr wrote: post #150

Earlier quoted context omitted.

Would you care to elaborate on this?

* Their implementation sucks so much that most decently built concurrency implementation will beat Go-routines by a wide margin. Just have a look at all those people on the Go mailing list whining about the scheduler. Not even talking about the fact that building concurrency mechanisms into the language is plain stupid. We all saw how well that worked out in plenty of other languages before. * You know why pretty muc…

I'm interested in more details on the packaging system issues. (The biggest problem I see is lack of versioning, but there must be more.)
Post reply on HN