Making the move from Scala to Go

The former VP of Platform Eng at Twitter said in 2015, "What I would have done differently four years ago is use Java and not used Scala as part of this rewrite. [...] it would take an engineer two months before they're fully productive and writing Scala code." The VP of Platform Eng at Twitter expressed regret for the choice of Scala.

LinkedIn's SVP of Eng Kevin Scott said, "We are not getting rid of Scala at LinkedIn. We've recently made the decision to minimize our dependence on Scala in our next generation front end infrastructure which is rolling out this year. We've also made a decision to focus our development infrastructure efforts on Java 8, Javascript, Objective-C, Swift, C++, and Python given the nature of things that we are building right now and will be building in the foreseeable future. That said, we have a ton of Scala code running in production, and will continue to provide basic support for Scala and those Scala systems for the foreseeable future."

And Coda's Yammer letter is around for those to Google.

Companies don't ditch languages that they have a huge investment in. It's just not worth it when you reach a certain level. It's not just the code to run your services, but the huge investment in performance, tooling, and monitoring - not to mention the bugs that come up from code that's just a little different. Twitter is past the scale where it can ditch a language. So is LinkedIn. But they can regret choices or decide to concentrate future development in a different direction.

I think that Java 8 has dulled some people's enthusiasm for Scala. Java 8 comes with a lot of nice features that people really missed in Java and might choose Scala because it did have them.

But I think more generally, there's been a movement away from being enamored with features, cleverness, and terse syntax in a language. Scala is a language of features, cleverness, and terse syntax. I was at a presentation by Rob Pike (one of the Go folk) and he argued that there are many things you can add to a language that feel clever and satisfying to write, but obscure what's actually going on and ultimately the things that are annoying, that are time sinks, and that cause problems aren't that you had to use a loop rather than something more clever. It was a long time ago so this is potentially more what I took away from the presentation than Pike's sentiment.

If we look at Go, Go is boring as hell. I mean, it has one cool thing in it - the goroutines (with cool being defined as something generally not found in all programming languages). It doesn't even have a lot of the cool things Java has. Java has annotations, inheritance, generics, a cool lambda syntax, "final" immutable references, advanced codegen. . . Go doesn't even have a way for me to declare an immutable variable. But it's so easy to pick up because there's basically nothing unfamiliar to someone who has experience with a mainstream, imperative language. Really, the syntax might be a bit different and it will always take a bit to adjust to a new language, but goroutines is really the only feature of the language that should be "new" to users (maybe multiple return vars, but that's stretching it). It generally emphasizes the ease of understanding and debugging over terseness of code. People seem to be enjoying that (at least some people). Part of it might just be that when a new philosophy comes onto the block, a lot more is written about it than gets written about old philosophies. For example, DHH just argued that Rails' all-in-one package is still a huge value add today, even though it doesn't get written about as much as it did a decade ago (https://www.quora.com/What-makes-Rails-a-framework-worth-lea...). It's possible that Scala's value add is as good as it once was, but the people who found that value add useful have long since stopped blogging about it.

I haven't used Scala that much. I like that it has a repl, I like immutability (which should be nice with Shenandoah GC), I like the fact that I can make a basic POJO (case class) without writing a novel of private fields, getters, setters, hash code, and equals. Scala definitely has some great parts that I wish Java had. Scala also has some hell that can make it harder to reason about (gratuitous operator overloading, implicits http://yz.mit.edu/wp/true-scala-complexity/).

Scala also came onto the scene at a time when Java didn't look so hot. Java 5/6 era Java (2004 & 2006) just didn't support the cool FP stuff that Scala did. Java 7 didn't come out until 2011 and still didn't have lambdas. It was 2014 before Java got a lambda. So Scala was this new language that ran on the JVM that had features! But maybe people didn't want all the features, but really only wanted a few of them like lambdas, streams, and the ability to make simple POJOs/case classes (which is still missing in Java, though things like http://immutables.github.io/ can help). I'm sure that there are many other reasons to use Scala. I'm not trying to argue anyone away from a language they enjoy using. The point I'm trying to make is that for many people, they might have chosen Scala for some of these simpler features that now mostly exist in Java. Other people might have been enamored with how productive that Scala made them at first and later decided that they were creating things that were harder to debug or for others to read and understand. Still others might not have realized that everything is terrible and there's no amazing new technology that's going to make them 100x happier than they used to be when they actually write real programs. Don't underestimate that last bit. Scala's been around and has plenty of people who have used it and not created anything in a shorter amount of time or more reliably than they had in Java, Python, or whatnot. . .but don't yet realize that everything is terrible. I'm not saying that languages don't matter. I think they do. Still, it's hard to find something that doesn't have huge terrible portions that will drive you nuts on occasion.

Scala isn't as shiny and new as it once was. A new language is always awesome before you have to debug what you write in it. Java 8 has implemented a lot of what people thought was missing from Java. People have dulled on "magic" (really just harder to follow code) that might be hard to discover how it works rather than easy-to-follow code. That doesn't mean Scala is terrible or anything, but it's advantage over Java is probably smaller for people who want some basic FP and the zeitgeist seems to be moving against languages that are more implicit or dynamic (regardless of whether that's happening in terms of LOC).

Personally, I like to measure things like that by how fast a team/developer can get up to speed on maintaining an already existing non-trivial codebase in the language. Shipping greenfield projects is often easier than adding features to an existing one, particularly in languages like Scala.