The Road to Scala 3

When I look around, I see more jobs, conferences, talks, meetups, libraries, open source communities ...

Sure, some companies are moving to other languages, that might be growing quicker than Scala ever was. But this isn't zero-sum.

Having led engineering at a startup that developed ~95% of our code in Scala and then moving to a FANG team that developed most of its services in Java (still using Spark Scala to a large degree and beginning to introduce Kotlin), these are the reasons I can use to justify using Java over either: 1) it is already there, 2) most engineers know Java well [enough], 3) other teams predominantly use Java and it makes certain types of code sharing more streamlined. I'm assuming most Java shops have existing tooling that will lock them into the JVM no matter what. That's fine, the JVM is incredible.

Kotlin can be introduced very easily into most Java codebases as interoperability is a tenet. Scala not so much. Holistic interop between Scala and Java can fill an entire intermediate-level book.

At the startup, we had a difficult time hiring for Scala. A number of candidates we moved forward with who were well-versed in it wanted to explore the language and embrace the FP aspects, which led to incomprehensible code for the engineers used to procedural OOO. In full retrospect, this was an issue with our hiring standard; Scala should not have been a specific part of our bar. It was a startup, we had crushing customer demands, there was no time to teach every engineer what an IO Monad was. It was a constant battle and perhaps the most frustrating aspect to me leading all the way up to acquisition. None of these issues would have existed if we chose e.g. Go; we would have quickly found a team of engineers who wanted to build applications instead of tinkering with the language. To me, that's the best part about Go -- it's boring.

Java is another boring language, but it's also a very verbose one, even when fully utilizing Lombok/G-libraries/etc.. Kotlin fixes the verbosity issue to a large enough degree for my needs without introducing enough interesting features to attract those darn academic types. To me, Kotlin is about as good as a language can get while remaining on the current iteration of the JVM. We have been slowly introducing Kotlin and plan to use it for most new non-Spark-related projects. I would only consider Scala for the same purpose if 1) interop was trivial to the degree that Kotlin's is, and 2) it was reduced to a reasonable subset of its full complexity. I believe the right steps are being taken with the language, but it still has a ways to go in those regards.

From a few years ago, with Scala 2, but the fundamentals may not have changed much.

Scala had, and has, a large bespoke standard library, which is big enough that if you tried to ship it with even a trivial app, it exceeded limits not just on memory, but on numbers of classes, methods, and so forth. (And even if those limits have been raised on newer phones, older ones are still a nontrivial market segment.)

So the way people got around that was by using "tree shakers", which would grovel through the app and its libraries, and chuck out portions of the library which weren't actually used. This works, but significantly complicates the development toolchain.

Between that and the official support for Kotlin as the "better Java" dialect of choice, you have to really like Scala to make it worth the effort...

> Java is another boring language, but it's also a very verbose one, even when fully utilizing Lombok/G-libraries/etc..

Incidentally, I find golang to be more verbose than Java. The amount of one-off functions you have to write to get around the lack of generics and/or map/filter calls on collections makes the code longer and harder to follow, and harder to debug. Not to mention error handling.

I generally agree with the gist of your post though.

What ended up happening with the startup you worked at out of curiosity?

I agree with almost everything you're saying. I'll just add that while Scala missed it's opportunity with Android, the big problem has always been a borderline dogmatic approach to pure FP by the community at the cost of simplicity. Scala libraries are generally complex and difficult to use (sometimes even for Scala engineers with several years experience).

Also, core FP principles are not that hard (immutability, expressions, HOF, functional error handling, pattern matching), but when you go full / pure FP, that is a steep cliff and an end goal in itself that has serious time /monetary/ complexity costs. The real goals should always be a healthy balance of reasonably fast business functionality delivery and a good code base.