Very on brand for Germany to invest tech 15 years past its prime.
As a fellow german, this was the very first thought that popped into my head.
The Sovereign Tech Fund invests in Scala
91–99 of 99 posts
Re: The Sovereign Tech Fund invests in Scala
#92Scala isn't as hot as it used to be. I think the rough Scala 2->3 transition, coupled with improvements in the base Java language, emergence of Kotlin + Android support, and popularity of Python in data science and data pipelines (lets just do everything in one language became popular) kind of made Scala not quite as popular as it could have been. Plus the long compile times are a pain. However it seems to have a rea…
Scala is too complicated. Most scala code bases I have worked on have no enforced structure, the language allows for all sorts of unconventional programming paradigms
I think its because we dont have too many established paradigms for functional programming. Having said that I think Scala is just marvellous. I had to work with a Scala codebase written by a set of very mature devs and it was an absolute joy. It influenced how I write Java.
Re: The Sovereign Tech Fund invests in Scala
#93Why would the governments invest money on such a niche language? "Scala is widely used to build and operate essential systems across multiple industries." - very bold statement.
I don't know what it looks like on the ground now, but Scala was the defacto language of data infrastructure across the post-Twitter world of SV late stage/growth startups. In large part, this was because these companies were populated by former members of the Twitter data team so it was familiar, but also because there was so much open source tooling at that point. ML teams largely wrote/write Python, product teams…
Did the Scala 3 changeover blunt its momentum, you think? Or just Python won out in a sort of worse-is-better way?
Re: The Sovereign Tech Fund invests in Scala
#94I've always liked Scala as a language, but it's challenging to write high-performing and memory-efficient code on the JVM in general. Whenever you raise this issue, you'll encounter a horde of JVM fanboys who insist that it’s not true, giving you all kinds of nonsense excuses and accusing you of not measuring performance or memory consumption properly. If you genuinely want to produce well-performing JVM code, you're essentially writing C-style Java. As soon as you introduce abstraction, performance issues inevitably arise – largely due to the fact that features and modernizations from Project Valhalla haven’t yet been implemented/shipped. Scala proponents will suggest using macros and opaque types, but at scale this approach becomes incredibly cumbersome and even then you won't be able to completely prevent boxing that would actually be unnecessary; you could just as well be writing Rust.
Re: The Sovereign Tech Fund invests in Scala
#95Scala isn't as hot as it used to be. I think the rough Scala 2->3 transition, coupled with improvements in the base Java language, emergence of Kotlin + Android support, and popularity of Python in data science and data pipelines (lets just do everything in one language became popular) kind of made Scala not quite as popular as it could have been. Plus the long compile times are a pain. However it seems to have a rea…
I agree. I took Martin's fantastic online Scala class way back when and I was a fan. Now Clojure is a great JVM alternative, and as you say Java has kicked it up a notch.
Re: The Sovereign Tech Fund invests in Scala
#96Earlier quoted context omitted.
That's sorta the curse of Lisps too.
Yeah sorta I would say Clojure is a big exception to that - Clojure applications tend to be more uniform than even non lisps
Clojure is an exception to pretty much everything in the category. I really wish I had the opportunity to use it in my professional career.
Re: The Sovereign Tech Fund invests in Scala
#97Earlier quoted context omitted.
I don't know what it looks like on the ground now, but Scala was the defacto language of data infrastructure across the post-Twitter world of SV late stage/growth startups. In large part, this was because these companies were populated by former members of the Twitter data team so it was familiar, but also because there was so much open source tooling at that point. ML teams largely wrote/write Python, product teams…
Did the Scala 3 changeover blunt its momentum, you think? Or just Python won out in a sort of worse-is-better way?
Mostly the latter. Scala 3 is almost completely irrelevant to the big data space so far. Databricks took six years before upgrading their proprietary Spark runtime to Scala 2.13. Flink dropped the Scala API before even moving to 2.13. I don't know if Scio will seriously attempt the move to Scala 3. All of them suffer from Twitter libraries being abandoned, which isn't insurmountable, but an annoyance still.
And I don't think it matters anymore. I predict that the JVM will eventually be out of the equation. We're already seeing query engines being replaced by proprietary or open source equivalents in C++ or Rust. Large scale distribution is less of a selling point with modern cloud computing. Do you really need 100 executors when you can get a bare metal instance with 192, 256 or 384 cores?
People want a dataframe API in Python because that's what the the ML/DS/AI crowd knows. Queries and processing will be done in C++ or Rust, with little or even zero need for a distributed runtime. The JVM and Scala solve a problem that simply won't exist anymore.
Re: The Sovereign Tech Fund invests in Scala
#98Earlier quoted context omitted.
This is nonsense. And, not that there's anything wrong with classes, but compilers no longer need to 'insert fake classes' for anonymous functions since Java 8.
Pretty sure everything in the jvm has to be tied to a class structure by definition.
Yes, and? Classes are not inherently bad, hence saying that JVM should not be invested in because its bytecode requires classes is nonsense.
re: inline lambdas, they have to be _tied_ to a class, and defined as methods, indeed. But they no longer have to be standalone classes themselves since JDK8
Re: The Sovereign Tech Fund invests in Scala
#99Earlier quoted context omitted.
Did the Scala 3 changeover blunt its momentum, you think? Or just Python won out in a sort of worse-is-better way?
Mostly the latter. Scala 3 is almost completely irrelevant to the big data space so far. Databricks took six years before upgrading their proprietary Spark runtime to Scala 2.13. Flink dropped the Scala API before even moving to 2.13. I don't know if Scio will seriously attempt the move to Scala 3. All of them suffer from Twitter libraries being abandoned, which isn't insurmountable, but an annoyance still. And I don…
Yeah, this is certainly the correct take. There's an alternate timeline where the Scala community focused during the peak on making it a better language for numeric computing/ML rather than the Nth category theoretic framework, but here we are. At a job almost a decade ago, we made some progress on an open source dataframes (and unfortunately proprietary data visualization) library for Scala, but we didn't get far enough before the company closed and the project died [1].
Still my favorite language I had the privilege to work with professionally for over a decade. However, in this post-JVM world, I'm actually excited to see a lot more OCaml discussion on here lately. The Jane Street work on OxCaml is terrific after a long period of stagnation with the language. I'm using it for most of my projects these days.