Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rewriting a large production system in Go

matt-welsh.blogspot.com

Post reply on HN
Print view View original

81–90 of 196 posts

Re: Rewriting a large production system in Go

#81

Post by packetslave »

zzzeek wrote: post #19
MetaCosm wrote: post #17

Earlier quoted context omitted.

Don't confuse not understanding something with understanding it and thinking it is a poor idea. The Go authors understand exceptions perfectly -- they just thought that it was a bad idea: http://golang.org/doc/faq#exceptions

yes I've read that and their reason is "it results in convoluted code" - which is not at all my experience, programming in Java and Python for many years, it's worse in Java for sure due to the heavy emphasis on checked exceptions, but in Python they are a dream. "It also tends to encourage programmers to label too many ordinary errors, such as failing to open a file, as exceptional." also not true in my experience.…

One might also note that Google doesn't use exceptions in C++, either.

http://google-styleguide.googlecode.com/svn/trunk/cppguide.x...

Re: Rewriting a large production system in Go

#82

Post by pvdm »

mdwelsh wrote: post #59
pvdm wrote: post #4

Unless it is open sourced, I can't validate the claims.

I would love to open source this system, but even if we did I'm not sure we'd be able to convince you that using Go was better than some other language in terms of developer productivity. How would having access to the source help?

You mentioned in the article that you read a section of code by one of the lead engineers and immediately you became convinced that using Go is the "way-to-go". Well, I would like to experience that eureka moment as well.

Re: Rewriting a large production system in Go

#83

Post by dalek_cannes »

packetslave wrote: post #61
seiji wrote: post #29

Earlier quoted context omitted.

I wonder why Google never adopted Erlang. Collectively, google believes they are right and the world is wrong. Anything pre-existing is dirty and unworthy of their genius if they didn't invent it themselves. So, even though we have 20 years of Erlang and production concurrency experience out there in one solid language, it's just ignored (except for parts they want to get "inspired by"). All of that is fine in isolat…

Collectively, google believes they are right and the world is wrong. Anything pre-existing is dirty and unworthy of their genius if they didn't invent it themselves. Someone a bit more objective might say "Google believes that it's quite hard to integrate third-party software into their huge existing infrastructure and make it work at their enormous scale." They might also say "Google believes the infrastructure they…

I used to work at a place that tried to reinvent everything from the wheel up, too. Over time I came to realize that avoiding third party libraries and technology stacks was more about long term support and hackability than anything else.

Unfortunately, newer recruits never realize this and accept the internal stack as religion. They never bother to learn third party alternatives.

We had a joke that went: "If you were ever fired and had to find a job elsewhere, you'll have to start by implementing your own X", X being a heavily used internal library that allowed an entire generation of developers to do certain things without ever knowing the underlying system calls.

Re: Rewriting a large production system in Go

#84

Post by pcwalton »

packetslave wrote: post #81
zzzeek wrote: post #19

Earlier quoted context omitted.

yes I've read that and their reason is "it results in convoluted code" - which is not at all my experience, programming in Java and Python for many years, it's worse in Java for sure due to the heavy emphasis on checked exceptions, but in Python they are a dream. "It also tends to encourage programmers to label too many ordinary errors, such as failing to open a file, as exceptional." also not true in my experience.…

One might also note that Google doesn't use exceptions in C++, either. http://google-styleguide.googlecode.com/svn/trunk/cppguide.x...

Most of the arguments against exceptions presented in that document apply specifically to C++, particularly in a codebase with a large amount of exception-unsafe code (which is a C++-specific problem).

"On their face, the benefits of using exceptions outweigh the costs, especially in new projects...Things would probably be different if we had to do it all over again from scratch."

Re: Rewriting a large production system in Go

#85

Post by pvdm »

packetslave wrote: post #54
pvdm wrote: post #4

Unless it is open sourced, I can't validate the claims.

I'm not sure anyone is asking you to "validate the claims". If you'd like some reassurance that OP knows what he's talking about, you might consider that Harvard tends not to hire stupid professors and then give them tenure.

I am well aware of Matt Welsh's reputation and what institution he was from. However, even Einstein or Feynman's scientific papers are subjected to peer review.

Re: Rewriting a large production system in Go

#86

Post by sytelus »

The number of strongly typed garbage collected compiled languages is surely shrinking. I can understand someone's disgust towards Java considering dwindling velocity in adding next-gen features and now sleeping in same bed as lawyer run Oracle. That pretty much had left C# in the arena until Go came on the scene. I was honestly hoping Go would give us head-to-head battle with C# but from initial looks Go pretty much feels like C# 1.0. No generics, no partial classes, no linq... C# 5.0 even has async and dynamic types. I like minimalism but these features are something we take for granted in languages that were created post-2000 era. So it all boils down to people choosing Go for singular reason: It's not from Microsoft! If licensing for C# wasn't a barrier and let's stretch our imagination by thinking C# was created by some other group in Google, would Go have a chance?

Re: Rewriting a large production system in Go

#87

Post by derefr »

laureny wrote: post #80
derefr wrote: post #63

Earlier quoted context omitted.

Interesting, thanks for that; it's pretty much what I guessed (especially the bit about the supervision tree and hot-code-upgrade advantages being mooted by your infrastructure.) On a tangent, though: > What it does not have is type safety I've tried to work this out before (I'm designing a new language for Erlang's VM), but as far as I can tell, type safety is in practice incompatible with VM-supported hot code upgr…

> type safety is in practice incompatible with VM-supported hot code upgrade. That's not true. First, it's very easy to hot reload changes that have been made to the code that are backward compatible. The JVM spec describes in very specific details what that means (adding or removing a method is not backward compatible, modifying a body is, etc...). This is how Hotswap works, the JVM has been using it for years. As f…

> As for changes that are backward incompatible, you can still manage them with application level techniques, such as rolling out servers or simply allow two different versions of the class to exist at the same time (JRebel does that, as do other a few other products in the JVM ecosystem).

Neither of these allow for the whole reason Erlang has hot code upgrade in the first place: allowing to upgrade the code on one side of a TCP connection without dropping the connection to the other side. Tell me how to do that with a static type system :)

Re: Rewriting a large production system in Go

#88

Post by derefr »

cageface wrote: post #72
derefr wrote: post #52

Earlier quoted context omitted.

"Exotic tech" is the most amusing ad-hominem insult I've heard in a while. Erlang has more users than Go, at least, and more companies you could name off the top of your head have Erlang deployed somewhere (Github and Heroku, for just two.) Also, Google doesn't have "unique requirements." They have a unique set of overlapping, pretty common requirements. Some requirements in that set (e.g. serving data on dl.google.c…

Which hominem am I ad-ing exactly here? Plenty of sophisticated users have taken a long hard look at Erlang and chosen other tech, for a variety of good reasons. I guess Twitter is a bunch of NIH bumblers for passing on it as well? Go clearly has a pretty different set of design priorities, not the least of which are static typing and native code compilation. I'm inclined to give the people running Google's infrastru…

Hey, don't group me in together with the downvoted guy here. I agree that Google have their own reasons to pick their own tech.

I'm just saying that Erlang was probably a better solution than C++ for some of the things they were doing, and they could have switched to it years and years ago. They might then have created Go, and switch from Erlang to Go for those same projects. There'd be nothing wrong with that. I'm just surprised they were using C++ of all things to begin with, before rewriting in Go.

Re: Rewriting a large production system in Go

#89

Post by Daishiman »

derefr wrote: post #66
zaphar wrote: post #53

Earlier quoted context omitted.

You're assuming IO-bound highly-concurrent C++ server don't have other requirements besides those two. Maybe it's IO-bound highly concurrent text processing. Erlang will suck at this despite the two pieces it's excellent at. Go is pretty fast at processing text and google does a lot of text processing.

What exactly does "text processing" mean, by the way? Erlang is very good at processing streams of bytes --you can pattern match on binaries to get new sub-binaries (which are basically equivalent to Go's array slices) to pass around, etc. It just gets awkward when you have to convert those streams into codepoints to do case-insensitive comparisons and such. But to reply more directly, "IO-bound" means something spec…

Google has both CPU-bound and IO-bound problems at massive scales. A language that is not highly performant in either area would be insufficient.

Re: Rewriting a large production system in Go

#90

Post by f2f »

onan_barbarian wrote: post #77
lbarrow wrote: post #11

It's interesting to see that more and more internal projects at Google are getting redone in Go. However, this sentence stands out to me in the article: "Before doing the rewrite, we realized we needed only a small subset of the functionality of the original system -- perhaps 20% (or less) of what the other projects were doing with it." I'm guessing that a lot of the benefit of the rewrite came simply from the simpli…

Hmmn. I think there's a fundamental advantage in any rewrite that's allowed to drop functionality - even a C++ to C++ rewrite. It's not just the language. It's the fact that (a) you already have an instantiation of the idea to look at as a reference and (b) you're happily cutting bits off the original. The latter suggests that you're not only not supporting all of the original use cases, you're probably in a nice sta…

ironically, that's not the common experience; the term 'second system syndrome' was coined for a reason.

"allowed to drop functionality", as used by you, is a strawman -- the team _added_ functionality to the part they were rewriting without incurring the wrath of the second system. sure, the line counts are off, but your argument does not stand in the general.

Post reply on HN