Scala at scale at Databricks • Hacker News Board

Re: Scala at scale at Databricks

#171

Post by lmm » Mon, Dec 13, 2021, 1:50 AM UTC

StreamBright wrote: post #166

esarbe wrote: post #159

Earlier quoted context omitted.

> Scala is the worst of the category of "functional looking languages built on the top of an imperative ecosystem". Could you elaborate? Because that's not been my experience at all. I have found Scala to provide many tools to wrap around the idiosyncrasies of the underlying imperative that they are barely noticeable anymore.

I found F# really good at with this -> "wrap around the idiosyncrasies of the underlying imperative". In the video Paul Phillips elaborates on many subject much better than I could ever do. I also found the .net ecosystem libraries better, even though there are fewer libraries available. Some links that I found great on the subject: https://www.reddit.com/r/scala/comments/4246qc/heres_why_sca... https://techblog.bozh…

Paul Phillips misses the point, and the "more than one way to do it" complaints are superficial. There's no language that prevents you from writing two functions that do the same thing, and that's all that most of those examples are (e.g. a -> b is just calling a function that returns (a, b)). Most of the time there aren't multiple semantic ways to do the same thing, which is what matters.

SBT is awful, don't use it.

"Minor versions" is overstating things - 2.11 was released in 2014, 2.12 was released in 2016, 2.13 in 2019, and 3.0 onwards are backwards compatible with 2.11. So that's one compatibility break every 2-3 years, which is better than most languages.

Scala is the only vaguely mainstream language I've found where you can do a whole enterprise-scale project in the language itself - no reflection, no AOP, no decorators, no macros, no monkeypatching, no nothing. So until I find another language where I can do custom context-like types (e.g. a type to represent "this must happen in a database transaction", and libraries already offer me functions like "take this list of database actions and combine them into a single database action" that work with my custom type) and "walk the object graph" style tasks (e.g. JSON serialization) in plain old type-safe code, I'm sticking with Scala.

Re: Scala at scale at Databricks

#172

Post by jroper » Tue, Dec 14, 2021, 5:01 AM UTC

scalaIsKindaMeh wrote: post #142

AzzieElbab wrote: post #123

Earlier quoted context omitted.

Scala compilation + graalvm compilation = very long breaks :)

Interesting, in my experience long compile times in Scala equates with a bad code base. And invariably involves an uber object with a whole lot of implicit conversations and is imported on every other source file.

I think the pertinent part of the comment you're replying to was graalvm, not so much Scala. I was compiling a small Scala codebase that took 15 minutes to compile - most of that was not my code, it analyses the reachability of the entire classpath and compiles it to native machine code.

Re: Scala at scale at Databricks

#173

Post by esarbe » Tue, Dec 14, 2021, 8:33 PM UTC

StreamBright wrote: post #166

esarbe wrote: post #159

Earlier quoted context omitted.

> Scala is the worst of the category of "functional looking languages built on the top of an imperative ecosystem". Could you elaborate? Because that's not been my experience at all. I have found Scala to provide many tools to wrap around the idiosyncrasies of the underlying imperative that they are barely noticeable anymore.

I found F# really good at with this -> "wrap around the idiosyncrasies of the underlying imperative". In the video Paul Phillips elaborates on many subject much better than I could ever do. I also found the .net ecosystem libraries better, even though there are fewer libraries available. Some links that I found great on the subject: https://www.reddit.com/r/scala/comments/4246qc/heres_why_sca... https://techblog.bozh…

It's not experience with Scala at all, but we all have different journeys. Should you ever find yourself using Scala again, I hope you make the same pleasant discoveries as I have.

Thanks for elaborating your point of view.

Re: Scala at scale at Databricks

#174

Post by tlarkworthy » Wed, Dec 15, 2021, 11:20 PM UTC

SOLAR_FIELDS wrote: post #131

tlarkworthy wrote: post #31

Earlier quoted context omitted.

I considered it a near miss when a Scala shop rejected me because I used Maven to compile their coding challenge. Clearly the company was full of over-complicaters and actively recruiting for it.

As opposed to Gradle or SBT? If I'm being honest here, a quite significant part of day-to-day backend JVM software development positions is dealing with the build tool. While I wouldn't outright reject a candidate for not being familiar with the latest standard toolchain (Gradle), I would certainly consider it as one of many factors - Gradle is pretty complicated, yes, but also quite powerful in what it is capable of…

> If I'm being honest here, a quite significant part of day-to-day backend JVM software development positions is dealing with the build tool.

perhaps, because your work has over complicated it [1].

[1] If you are doing multi-distributions or multi-repo stuff perhaps it's warranted, but those complex build features better be providing serious value to the business to be worth losing 10% of the development workforce's time on build tooling.